© Fraunhofer IIS
CNNLok provides a camera-based, infrastructure-free location of mobile objects.
At CNNLok, mobile computers with cameras use a special deep learning architecture to localize themselves in a learned environment. This can be simple Android smartphones to embedded computers with multiple cameras are used. By continuously re-exploring the environment and using an application-specific motion model, reliable positions are calculated.
CNNLok can also be used when the environment changes, such as when the loading of high racks changes in the logistics sector. Localization in areas where infrastructure cannot be deployed is also something that can be learned. In contrast to standard radio-based or optical technologies, no sensor infrastructure is required for self-positioning; nor do any optical markers have to be installed in the environment, as other self-localizing systems often require. Static elements in the existing environment are all that is needed to calculate positions. As a result, CNNLok is self-sufficient and can be used very flexibly in a wide variety of application fields. As the technical feasibility of this idea has now been confirmed, a team from Fraunhofer IIS is working on a practical solution for CNNLok. Although there are some hurdles still to be overcome, development of the self-learning technology is progressing rapidly.
The mobile processing unit uses a so-called convolutional neural network to calculate the current position based on a camera image. First of all, data is automatically captured from thousands of camera images and the corresponding positions. Using this data, an existing network is further trained for the purpose of adapting it to the target environment. The finished network is used on the target platforms to determine the positions of new images. To prevent the system from degenerating over time, updated information is continuously collected on a central processing unit for further training of the network and distribution to the mobile processors.
The mobile processing unit is typically either a simple smartphone or an ARM- or Intel-based single-board computer with a standard camera. Thanks to the high flexibility of the platform, many application scenarios are possible that cannot be accomplished by standard, infrastructure-based solutions. Adapted motion models and special pre-processing of the collected data allow new data to populate an existing positioning system. Because continuously learning more about the environment is a very computing-intensive task, the system needs to be connected up to other hardware via a network or docking station or similar means. Depending on the dynamics of the area, a central processor with powerful standard deep learning hardware – such as graphics cards or special vector processors – may be additionally required. This takes over the duties of the mobile processing units at appropriate times – for example, when they need to be charged.
Fraunhofer Institute for Integrated Circuits IIS