Remote-sensing satellites play a crucial role in various sectors, including aerial mapping, weather forecasting, and monitoring deforestation. However, the existing systems are largely passive data collectors, unequipped to make decisions or detect changes. Traditionally, collected data must be transmitted to Earth for processing, a task that could take hours or even days, hindering prompt response to emergent situations such as natural disasters.

To circumvent this issue, a research team led by DPhil student Vit Ruzicka, from the Department of Computer Science at the University of Oxford, embarked on a project to train a machine learning program in space. The group's proposal was selected in 2022 by the Dashing through the Stars mission, following an open call for projects to be executed aboard the ION SCV004 satellite, launched earlier the same year. The team subsequently uploaded the code for the program to the satellite already in orbit.

The innovative model, dubbed RaVAEN, was trained to identify changes in cloud cover from aerial images taken directly onboard the satellite. This approach deviates from the traditional method of training models on Earth. Based on few-shot learning, the model learns key features from a limited number of samples, compressing data into smaller representations for greater speed and efficiency.

Ruzicka elaborated on the model's working: "RaVAEN compresses large image files into vectors of 128 numbers. During the training phase, the model learns to retain only the informative values in this vector; those that correlate with the change it is trying to detect - in this case, cloud presence. As a result, the training is remarkably fast due to a very small classification model to train."

While the initial part of the model was trained on Earth, the subsequent part that discerns whether an image contains clouds was trained directly on the satellite. Contrary to the typical development of a machine learning model which involves multiple rounds of training with a cluster of computers, this compact model completed the training phase using over 1300 images in just about one and a half seconds.

Upon testing with new data, the model automatically detected the presence or absence of a cloud within a tenth of a second. The model was able to encode and analyze a scene approximating an area of around 4.8x4.8 km2, nearly 450 football fields.

Ruzicka pointed out that the model can be adapted for various tasks and data types. Plans are underway to develop advanced models capable of distinguishing between significant changes, such as flooding, fires, and deforestation, and natural alterations like seasonal leaf color changes. Furthermore, the researchers aim to create models for more complex data, including hyperspectral satellite images, which could facilitate the detection of methane leaks, a vital step in combating climate change.

The adoption of machine learning in space could also mitigate issues with on-board satellite sensors, which often require recalibration due to harsh environmental conditions. Ruzicka stated, "Our proposed system could be utilized in constellations of non-homogeneous satellites, where reliable information from one satellite can be applied to train the rest of the constellation."

Vit's DPhil research supervisor, Professor Andrew Markham, believes that machine learning has immense potential to enhance remote sensing by making space-based sensing increasingly autonomous and reducing delays between data acquisition and action.

The project, an impressive proof-of-principle, was a joint venture between the University of Oxford, the European Space Agency (ESA) F-lab via the Cognitive Cloud Computing in Space (3CS) campaign, the Trillium Technologies initiative Networked Intelligence in Space (NIO.space), and partners at D-Orbit and Unibap.

Research Report:Fast model inference and training on-board of Satellites

Research Report:RaVAEN: unsupervised change detection of extreme events using ML on-board satellites

Related Links
University of Oxford
All about the robots on Earth and beyond!