GPSBots: An Introduction to Sensor Fusion

An Introduction to Sensor Fusion

An Introduction to Sensor Fusion

Sensor fusion is the practice of taking the readings from many different sensors and making a single map that the AI can use to sense its environment.

The sensors fill out the map of the current surroundings of the robot. Each sensed square has a few data points about that square, such as passibility, ground type, temperature, and any other info from the sensors you have. In this case, we only have passibility.

For example, if the ultrasonic sensor detects that at 45 degrees there is an object 3 feet away, that point on the sensor grid is marked as being impassible (red). The AI can then merely look at the map instead of interrogating each sensor individually. You can add more sensors (such as a second ultrasonic, or IR) and have them add to the value in the square, firming up the data. Another advantage is that the sensor subsystem can be entirely separate from the rest of the system which makes programming and debugging much easier.

Sensors can also be fused in time, a single sensor can take several measurements in a row and average them out or add them or perform other functions on them.

Fusing sensors together can achieve several goals:

Improved confidence in the sensor readings: IR may be fooled by the sun or shadows but ultrasonic will not.

Improved performance: Multiple sensors can be read at once, feeding into one data set. Different sensors tuned for different environments (ie smoke, night, day, etc).

Improved reliability: If one sensor fails, the entire set doesn't go down.

Types of Sensor Fusion

Data Fusion (Low Level)

In low level fusion, raw data is combined to produce a new "sense" that is the combination of all the sensors. For example, visible light, IR, and thermal cameras can be tied together into a single image which is sent to the rest of the system. The AI doesn't have to look at each signal separately, just the combined stream.

Feature Level Fusion (Intermediate Level)

Feature level fusion combines processed data. For example, a single data set may be created from several edge detection modules. The AI only has to look at the processed image to find all the edges.

Decision Level Fusion (High Level)

Voting methods, statistical methods, neural nets, etc are all at the decision level. In effect, it's like having multiple people on a commitie voting on what to do next. The space shuttle uses three independant computers to make sure that if one has bad signals or bad data, the other two can vote to override it.