Efficient visual monitoring of offshore windmill installations with online image annotation and deep learning computer vision

The number of offshore windmills keeps growing around the world since they are an essential part of any strategy to produce energy without using nuclear power plants or burning fossil resources. Due to the growing number of installations new monitoring and inspection strategies and routines need to be developed that are not only effective but efficient. In this work we will present first results from a joint project of industry partners and academia that aims at the development of new approaches making use of modern imaging technologies and new methods from artificial intelligence and computer vision research. We investigate three different strategies for collecting digital photos or video and posterior computational analysis using Convolutional Neural Networks (CNN). Our results indicate, that patterns of interest like rust or coating damage can be detected and classified with F 1 scores between 0.80 and 0.94 in photos collected by inspectors. In videos collected with unmanned aerial vehicles (UAV) rust patterns, oils spills and coating damage were detected with F 1 = 0.91. To support users in the time-consuming task of visual data exploration, we present a new visualization method referred to as “virtual twin”. Using hand - recorded image collections, a data-driven 3D model of a windmill is generated, that can be used to investigate damage patterns in the full context of the windmill and in full detail. Altogether our results indicate, that although each single approach has its limitations, a combination of different imaging methods, deep learning computer vision algorithms and sophisticated visual data exploration by experienced users appears to have potential to overcome the bottleneck in data analysis, interpretation and decision making in the context of the future inspections of offshore windmills, platforms or other constructions.