Camera as a Sensor

Andy Slote - Director of Customer Success for ObjectSpectrum

May 1, 2022

When thinking of devices for the Internet of Things, cameras may not immediately come to mind. However, a camera may be the most effective sensing device for some applications because it’s more appropriate for the environment. For example, cameras are often more practical when measurements of multiple physical quantities are needed.  

We are, of course, talking about a digital camera. What the camera “sees” and captures is interpreted by applications to extract valuable information. Some of the same data may be obvious to a human viewing the image, but algorithms can discern more than we can while making more complex interpretations. Decisions also come more quickly, making inferences people cannot.

There are two primary types of cameras with different images of interest. The first is familiar to everyone – producing the digital “pictures” using visible light, like the ones we take with our smartphones. The second kind generates a “thermal” image formed using the infrared radiation emitted by elements of the subject in view. Both types need their pixels analyzed to provide valuable data from “still” images or videos.

An object within a digital image is typically identifiable by humans, and one of the goals is to use AI to do the same. Interpreting pixels as objects of interest using pattern matching capabilities is the job of Machine Learning Models. For example, in creating a vast library of images of people as potential matches, the goal of the ML is to identify a person or persons correctly in every image.

As data needs become more sophisticated, the algorithms follow suit. For example, instead of just identifying a vehicle in a video, you might need to estimate the approximate distance from the camera and to know if it’s moving or still. You can get more granular by flagging trucks or even recognizing particular delivery trucks.

Thermal images provide a view of the relative temperatures of the objects in view, making identification possible. For example, a vehicle’s temperature profile differs significantly from a human’s. Likewise, a machine in a factory shows a particular temperature profile when operating normally but a different one when there is an issue warranting action. Like the standard digital image, the data must be analyzed and interpreted, and typically uses Machine Learning, to provide helpful information.

When talking about a “Camera As A Sensor,” a camera that incorporates Machine Learning can provide a tailored data stream, data by exception, etc., just like any sensor. In the simplest case, temperature values can be relayed by a thermal camera continuously or only send a basic alert for values outside a predetermined range. There are advantages to a Camera As A Sensor implementation vs. standard sensors for some use cases. For example, a thermal camera enables a non-contact solution when monitoring temperature. It may also make it possible to assess the temperature of multiple objects in a single view, subverting the need to attach a sensor to every measurement location. On the other hand, the complexity may not justify the additional logic required to interpret an image. As with all sensors and their supporting application logic, a thorough analysis lets you select the best option.