Smart Embedded Network of Sensing Entities

The expected results of SENSE are to combine the aspects of:

• embedded intelligent middleware in smart devices,
• adaptive configuration,
• flexible cooperation (among devices),
• high-level perception and adaptation and
• dynamic networking

The SENSE system will encompass aspects including:

• construction of a modality-neutral embedded test platform;
• raw sensory processing;
• transformation of sensory data into semantic knowledge;
• communication between nodes to produce a consistent world view;
• sharing of knowledge between intelligent nodes;
• automatic recognition of unusual and alarm situations;
• communication between the intelligent network and an operator; and
• automatic discovery and configuration of new intelligent nodes.

Embedded systems in SENSE will develop their own semantic symbols based on an analysis of their environment. SENSE will incorporate research from machine learning to discover statistical regularities in its environment, and compress these regularities into informative semantic symbols. At the local level, SENSE will use algorithms such as "Expectation-Maximization" to optimize each node's set of semantic symbols. Sharing of knowledge between nodes is also a topic of research, both in distributed systems and artificial intelligence. The SENSE system will use a mature algorithm called "belief propagation". This algorithm specifies how to share probability distributions over semantic concepts between nodes, such that a self-consistent world view results.

The unique feature of SENSE is that it combines these various technologies from embedded systems, robotics, networking and machine learning research in a new way. The result is a framework for the development of smart networks of embedded components which are flexible, adaptive and device-independent. Networks that cover those challenges are called ad hoc networks or self-organizing networks. Their development is driven by the wireless community, but some of their principles are also of interest for wired networks. Advantages can be seen in lower effort for installation, initialization and maintenance as well as their inherent fault tolerance and their possibility to save energy within the network. This is true for both wired and wireless types but typically only relevant for the latter. In contrast to self-organizing networks, traditional networks have a very time demanding commissioning phase which also involves expert knowledge.

Dynamic addition or removal of nodes is a further challenge. In ad hoc networks, connections are constantly created and destroyed. This is called “plug and participate”.
SENSE will progress the state of the art through innovation at all levels varying from raw sensor data to high level abstraction, reasoning and interpretation:
1. Development of a distributed processing solution to large-scale systems design. SENSE tackles the problems of scalability and complexity through a complete decentralization of both processing and knowledge, relying upon the fusion of information at a high semantic level to allow computation.
2. Automatic learning of semantic symbols. Unsupervised learning is a key feature of SENSE. The sensory input used by the system will be decided by the system itself. Although some a priori knowledge must be given to the system, SENSE attempts to make its own decisions about what it senses, how it should describe what it senses, what are normal activities and how best to share this information with other sensing elements of the system
3. Transferability between application domains. By providing minimal ‘hardwired’ knowledge about either architecture or the specific goals of the system SENSE presents a generic approach to large-scale system development. In addition, traditional hardwired design requires a thorough understanding of the application and goals of the system. In the SENSE system, this is replaced by flexible adaptation to changes in the environment.
4. Semantic abstraction. SENSE decides what is important and what is normal, and automatically generalizes to provide its own view of the world.
5. Generic solutions to systems design. SENSE addresses two specific forms of sensor data (audio and visual). However, the low level extraction of features (on which the semantic levels operate) ensures that developments in the higher layer is applicable to any form of sensorial data.

6. Reliability in complex systems through distributed processing. Reliability in traditional applications suffers because of their own complexity. The failure of a single component may cause the break down of the entire system. In SENSE each sensor is an autonomous entity, and the overall system adapts to changes in the structure and to the loss of components.

Vision and audio sensors were selected to fit the application domain, and also because they are complex enough to allow for significant advancement in sensor processing technology. However, the framework designed will be generic enough to accommodate a wide range of sensors. The middleware and software framework will be designed to easily incorporate additional sensor types, which will allow for easier extension of project results, and easier adoption of the technology by third parties.
The SENSE system and research is designed to provide a general solution to information fusion from embedded sensor systems in general, and for security related applications in particular. Although the network will be developed and tested in an airport environment, the work is intended to be transferable to other security domains. With respect to requirements, the airport application is very similar to other applications in public buildings and public areas, such as shopping centers, railway and bus stations, office buildings, football stadiums, and so on. System tests will focus on general scenarios which are found in many public domains: persons in queues, persons in open areas, and persons at security checkpoints, or cars at unusual locations for example. These general scenarios, and the technology developed for the SENSE system, will then be applicable to other open or public environments. The SENSE technology will be modular, and portions thereof will therefore be appropriate for integration with existing security systems (existing video systems, for example). This will provide an upgrade path from current systems to a full distributed SENSE system, improving the chances for technology uptake.