Algorithms in the Arctic: Removing bad weather from images to make Arctic shipping safer

Arctic shipping traffic is on the increase. One day, these ships will be autonomous. New technology that can remove rain, snow and fog from the images produced by the ship’s cameras and sensors will increase safety in extreme conditions.

Imagine an autonomous ship sailing through one of the world’s most extreme ocean areas. Sea ice is everywhere. Fog, snow or rain make visibility extremely poor. Just like ship captains see through their eyes, autonomous navigation algorithms perceive the world through sensors, and bad weather is just as impenetrable for sensors as it is for sea captains.

Getting rid of poor visibility

With the rise of Arctic shipping, something that can remove the bad weather from the images so the algorithms can see the surroundings as if it were a clear, sunny day could be extremely useful. Now, Ph.D. candidate Nabil Panchi at NTNU’s Department of Marine Technology has developed an algorithm that can do just that.

“We have put in place a new piece of the big puzzle for better modeling of sea ice,” Panchi said.

Current AI algorithms work well on clear images, but they struggle when images become blurry or degraded due to bad weather.

Panchi, who is also a naval architect, has used thousands of images from the Arctic to train the new algorithm so it filters out visual impediments such as rain, snow, and fog, as well as water droplets on the lenses of the cameras that many vessels are equipped with.

Panchi is affiliated with the DigitalSeaIce project, which is focused on multi-scale integration and digitalization of Arctic sea ice observations and prediction models. The main objective is to build a digital infrastructure that integrates regional sea ice forecasting models and local ice-related models with shipboard and satellite-based Arctic sea ice and environmental observations.

Understanding the environment via images

“Our work is about understanding the Arctic environment through the use of images. We are creating algorithms that work in all weather conditions” says Panchi.

His research is based on thousands of images taken on a voyage with the research ship Kronprins Haakon in the Arctic during the summer of 2023.

In collaboration with his academic supervisor, Associate Professor Ekaterina Kim, he published the article “Deep Learning Strategies for Analysis of Weather-Degraded Optical Sea Ice Images” in IEEE Sensors.

Panchi and Kim are introducing two ways of helping ships travel more safely in bad weather in the Arctic, by “removing” the weather from images. One uses artificial intelligence to clean up the images, so that existing algorithms work as they should. A slightly more efficient way is to develop new algorithms that work during bad weather.

“Both strategies allow us to understand the Arctic in all weather conditions,” Nabil says.

Cleaned images already in use in cities

Algorithms that can remove weather from images have been in use for a long time, but primarily in urban areas. They are used to develop autonomous cars, and in security and camera surveillance.

Current algorithms that analyze sea ice are largely based on images taken from ships in good weather conditions. The problem is that images from the Arctic are often unclear due to the fog, rain, and snow that are common weather conditions in these waters. These types of images are poor material for the existing algorithms that are designed to understand the Arctic environment.

The algorithms also need to be trained to analyze the type of ice surrounding the ship, so they can indicate where it is safe to break through the ice, and which areas the ship should avoid.

The first open-access dataset of sea ice images

In order to remove fog and raindrops, algorithms must be trained to clean up weather-affected sea ice images.

“This area of research had largely been ignored so far. The problem has been limited access to clear images from the Arctic—until now. We hope that our new open-access dataset helps in future development of weather resilient technology,” Panchi says.

Panchi’s supervisor Ekaterina Kim has worked extensively in the Arctic, and in recent years she has been exploring how AI can be adopted to solve some of the challenges that exist in polar regions.

The two NTNU researchers have now made the SeaIceWeather dataset publicly available online. It contains thousands of images and is the first open-access data set for sea ice.