Eversource uses image recognition and AI for inspections

Eversource Energy is New England’s largest energy provider with nearly four million electricity and natural gas customers in Connecticut, Massachusetts and New Hampshire. Eversource’s transmission and distribution network spans more than 4,000 miles and, with thousands of substations, forms the backbone of a reliable and resilient network in New England.

As part of its electrical operations, Eversource Engineering and Operations perform regular distribution system inspections. Inspections are usually time-consuming and labor-intensive, so digital tools can play a vital role in optimizing these inspections. SmartInspect, is a digital platform that uses advanced image recognition and artificial intelligence (AI) algorithms, developed in-house by Eversource to improve data speed and accuracy. The image-rich tool is used to support the company’s electrical transmission and distribution (T&D) inspection activities.

What is SmartInspect:

“We started with the goal of developing a common platform to facilitate inspections across multiple business lines,” says Umair Zia, Director of Distribution Engineering. “To cover all uses and all points of use, the following minimum technical requirements have been identified:

  • Platform independent (PC, iOS, macOS and Android)
  • Possibility of offline synchronization
  • Lightweight design for field use
  • Clean and simple UI/UX
  • Ease of development and maintainability
  • To facilitate adoption, the following key data element requirements have been identified:
  • Customizable inspection forms
  • Importing GIS records
  • Importing Trap Records and Active Events
  • Ability to attach inspections to assets and upload multiple images
  • Ability to run AI models on images


“After considering technical and functional requirements, we created SmartInspect, a flexible system that allows quick and easy reporting of inspection data,” added Jaydeep Deshpande, Program Manager, Advanced Analytics Group at Eversource. “Given the technical requirements, we decided to host the software solution on Microsoft Azure. To support cross-platform functionality, we have developed the software as a web application. Additionally, to facilitate a mobile-first approach, we have adopted the Progressive Web App philosophy. This allowed us to distribute the mobile app to iOS and Android devices without needing to go through their respective app stores.

SmartInspect makes extensive use of open-source frameworks. The frontend application is built using React. The backend is hosted in an Azure Cosmos DB database. Cosmos DB delivers millisecond response times, automatic and instantaneous scalability, and guarantees high availability.

SmartInspect is a platform ecosystem designed to support various inspection and investigation activities. It provides digital capabilities to record field equipment information through forms as well as images. Images help preserve the wealth of information that forms alone cannot capture. Additionally, visual information can now also be processed using AI to improve situational awareness and reduce manual effort. We found marked similarities in inspection activities across all business units. Therefore, the platform maintains a common design language for submitting inspections and filling out inspection forms.

SmartInspect currently supports the following inspection activities:

  • Damage Assessment (DA)
  • Circuit scans
  • Vegetation management
  • Transmission line inspections

1. Damage assessment

In SmartInspect DA mode, the app displays all active events and their respective circuit boards. Crews can use the in-app navigation feature to navigate to the desired location and begin damage assessment. Once at the damage location, users can locate the nearest pole on the circuit and attach the damage to the asset with appropriate field notes, a standardized checklist, and corresponding images. The checklist and images are automatically linked to the active event via SmartInspect.

2. Circuit scans (CS) including vegetation patrols

Routine circuit scans are performed to collect a variety of data, for example, the health of vegetation on the circuit, the health of distribution infrastructure, third-party attachments, and equipment inventory. company-owned distribution on the circuits. Much like DA mode, Circuit Scan uses similar functionality to patrol circuits, albeit on clear sky days. The examiner can view the issues discovered by issue types and geotag the results by physical location as well as the nearest circuit and conductor spans or pole. This information can then be turned into work plans for distribution repair and/or upgrade work, or cutting and removal work in the event of vegetation issues.

3. Transmission line inspections

Functionality and feature set are consistent with other use cases. Engineering personnel can review structural problems and make repair recommendations. SmartInspect makes inspection records and images easy to access and helps organize data efficiently, including inspections performed via drones.

How SmartInspect works:

Figure 1 illustrates the SmartInspect process flow. The software has separate modules to support the entire inspection process. In the Scheduling and Queuing module, users can create work items, assign them to field resources, and track the progress of assigned tasks. In the inspection module, field resources can create new inspection records and in the review module, users can review field data.

Figure 2 shows the job scheduling and queuing module. Users/supervisors can create new work queues for each business use. For example, in Figure 2, a circuit scan program has been created. The program allows you to select specific circuits and service territories. For convenience, SmartInspect connects to GIS and allows circuit maps to be viewed from the platform. A list of all authorized users is available as a drop-down list for assignment. Once created, this creates a unique to-do list for the user.

Figure 3 shows some Mobile UI screens. Field crews use the mobile interface to capture and report damage in the field. Users can view all assets of assigned circuits. Assets are also differentiated with icons corresponding to the equipment they carry. Users can also use the built-in navigation plugin to navigate to tour locations. Once there, the user can submit the inspection against the asset. The inspection is automatically linked to the GIS record, allowing for later cross-referencing. Simultaneously, the user interface also displays nearby active events. This allows users in the field to associate the damage to the active event and the operated device. As users continue to work on inspections, the job queue reflects corresponding updates with the number of inspections submitted as well as the percentage of the inspection completed.

Figure 4 shows the review screen where users can review all submitted inspections. The filtering capability allows users to quickly analyze incoming information. Inspection tiles display high-level details about damage type and crew name. Users can click on the inspection cards and view the images as well as the checklist.

Figure 5 shows the AI ​​assistance feature in the review panel. As Eversource develops more advanced models, reviewers have expanded their ability to use the AI ​​assist feature to quickly diagnose issues. This feature allows the user to extract third-party attachments from a damage assessment study or identify missing distribution assets from a vegetation study.

SmartInspect provides a versatile, mobile and lightweight platform to support various utility inspection tasks. It categorizes images and checklists efficiently, helping to streamline inspection processes.

Comments are closed.