Decoding defects for resilient environments

Corinne Mynatt‍

We don’t always think about the nitty gritty of how our cities function — unless you’re a civil engineer or city council worker perhaps — yet three pioneering startups are using cutting edge technologies to address the challenges of aging infrastructure and our changing natural environment. Vertify, Near Space Labs, and Pallon have harnessed and adapted the possibilities of computer vision, AI, ML and data along with proprietary developments in robotics and sensing technologies to mitigate erosion in our built environment and beyond. From our sewers, skyscrapers, and all the way up to the stratosphere, they are all changing the game in how we can fix pipes for water networks, repair aging buildings, and prepare for and respond to natural disasters.

Amongst the bright city lights, yellow taxis, steaming sewers and occasional car horns, pretty much every New Yorker’s daily life is peppered by walking under sidewalk scaffolding or ‘sheds’ as they are known. There are over two million linear feet of them in New York City — enough to cover a sidewalk all the way to Richmond, Virginia. As charming as they are, it’s an annoyance to most pedestrians who dodge their structural poles when navigating the sidewalk everyday, not to mention the costs involved for everyone. It is Local Law 11 in New York City, established in 1998, that requires buildings of six stories or more to submit a façade inspection report every five years. If it is deemed unsafe or requiring repairs, the sheds go up.

Founder of Vertify Analytics, Nile Berry, realized there was ‘such a clear opportunity to improve an archaic process’ in how these building inspections take place, and the repairs that follow. As an early adopter of drone technology and with experience creating architectural renderings and documentation for his first company Nova Concepts, Nile realized there was a latent opportunity to help engineers document buildings with these flying machines.

When we spoke, Nile showed me an example of the clunky and antiquated documentation architects and engineers have to deal with, typically acquired by workers repelling off the sides of buildings or by inspecting buildings with binoculars from below. With this loose collection of images and reference numbers, the architect pieces together the faults, pinpointing them to the location on the building with this piecemeal information — this is where Vertify comes in to streamline the process.

Vertify is centered around the idea that visualization is the easiest way to communicate with anyone, ‘creating clear images that are organized in a way that is logical and makes sense.’ Using drones to capture high res images of buildings, they create a full 3D map and digital twin, which can then be analyzed, annotated, and used for inspection reports. They can also undertake thermal mapping to help energy retrofits, making buildings and cities more efficient and sustainable. By ‘injecting new technology and a baseline of really good data’, every capture helps train the next, increasing accuracy and evolving the software. With 16,000 buildings in New York City alone that fall under Local Law 11, Vertify is streamlining the building reporting and repair process. And although Berry has been laser-focused on his city, its application will certainly expand beyond these ZIP codes.

Whilst Vertify is at city level, a bit higher up, Near Space Labs is ‘commercializing the stratosphere’ according to Co-founder Rema Matevosyan, with the aim of ‘solving challenges at the intersection of increasingly urbanized life and the increase in vulnerability because of climate change.’ Up 60,000 to 85,000 feet in the sky, the stratosphere is a sweet spot for remote sensing, capturing imagery with up to 10 cm (per pixel) of detail for post-catastrophe response, conservation, and even urban infrastructure planning. Using a fleet of wind-powered robotic balloons, they are able to scale their operations much more easily and efficiently than similar solutions using drones, satellites, or airplanes.

Unlike satellites, Near Space Labs nimble and robotic balloons can come back down to earth to be refitted or have software or hardware upgraded as and when needed, taking advantage of the latest sensor technologies for example. Their balloon-centered approach also allows clients to access imagery whenever they need it, at the rate they need it, whether that is daily, monthly, or quarterly — something not as easily achieved with satellite, plane, or drone-captured imagery.

Although the balloons and hardware are the core of how they operate, Rema notes that it is a full-stack approach that gives their work the competitive edge, along with over fourteen years of stratospheric wind navigation and modeling experience. It is their precise wind models that help drive the balloons (albeit autonomously), whilst their proprietary optical sensors provide high resolution imagery — not to mention their thermal sensors or those that can detect greenhouse gasses.

For Near Space Labs, the ultimate goal is to contribute what they can to mitigating climate catastrophes — both in the short and long term. According to Rema, ‘being the best imagery and data provider for climate adaptation is what we strive to become.’ Working with fire departments on live wildfire situations, post-flooding or post-tornado monitoring, the data they capture is certainly on the benevolent side of the surveillance spectrum. They can show rescue teams where debris is so they can get to people quickly, or show utility companies where power lines have gone down so they can repair them — all of which helps to inform future planning. Another key aspect is the insight they can provide insurers, which ultimately helps consumers in the long run, according to Matevosyan. But they also work with municipal clients such as New York City, helping them routinely track runoff or wastewater, to improve the effectiveness of permeable surfaces for example.

Back on the city streets, where charming steaming manhole covers are ever present throughout New York for instance, the constant maintenance of aging sewers is a core reality of this essential infrastructure. For example, this one city’s underground system transports at least one billion gallons of water per day through over 30,000 miles of sewers. We don’t want to think about the magnitude of what could happen if this network of pipes isn’t maintained properly — which is where Pallon comes in, helping both civil engineers and private companies utilize their software to process data on faults and flow in pipes and sewers.

For Pallon, their work started when one of the founders Dominik Boller had been doing research on detecting manhole covers in Google Street images. Tangentially, another engineering friend had expressed his dismay at watching hours and hours of sewer inspection footage, thinking that there must be a more streamlined way of doing things. It was this gap in knowledge with the application of computer vision and AI that gave them the impetus to dig into these possibilities underground and found their Swiss-based startup.

We’ve heard of fatbergs in the UK, but blockages are not the only problem that can occur in this subterranean world. Sewers are prone to physical defects or chemical corrosions; if not installed well or if the bedding beneath isn’t done properly, major problems can arise, not to mention the real effects of cars driving over our sewers in the roads above, putting pressure on these systems. Just as with building inspections that Vertify is tackling, Pallon is also ameliorating the process of defect detecting, combining footage with AI to work more efficiently than the human eye can.

Similar to the clunky processes still in place for building inspection, most sewer inspectors manually inspect sewers with cameras and note the defects as they navigate the pipes in real time. But Pallon’s software means the defects don’t need to be coded on site. After the data is captured, Pallon’s computer vision tracks millions of points in inspection footage that is then used to create 3D models and thereby localize the defects. This saves the time of coding on site, but also increases accuracy on what can be a subjective opinion on the quality of a pipe, or even the defect’s exact location. The 3D models Pallon’s software stitches together allows them to quantify the defects with ML, which when exported to standard exchange formats can be tracked into Geographic Information System mapping (GIS). This combination of data and insight helps prioritize what needs fixing first — a key challenge in the sewer maintenance industry.

According to co-founder Christian Koch, ‘there is a shortage of sewer inspectors and engineers, so there’s a high demand for becoming more efficient and effective in this industry. And like Vertify, Pallon also benefits from learning and evolving from a growing stock of data that ML and AI use to improve accuracy. But it goes beyond the AI as co-founder Koch told me, as ‘after the AI algorithm codes the defects, we calculate AI uncertainties for quality control, to ensure that we provide a high quality output.’

What’s impressive about the startups here is their adaptability to existing systems and hardware — seeing the potential for drones with data processing; using AI to analyze data captured with common hardware; or exploiting the reduced-carbon impact of stratospheric balloon flight to monitor natural disasters or large-scale environmental areas. All of these startups are gathering data that can be used to improve what they do in the future. More data means more accuracy, saving time and keeping people safer with effective disaster relief response and future planning, timely repairs on aging buildings for those walking below, and keeping what needs to be underground in our sewers functioning properly to keep municipalities churning on. As Christian Koch said, ‘infrastructure is the backbone of public health and environmental protection,’ and with these startups, we’re on the right path to resilience.

‍