Smart Embedded Network of Sensing Entities

In general, it can be stated that the project reached most of its objectives:

• With the described approach, it is possible to establish networks of stationary complex sensor nodes that can dynamically and automatically re-configure themselves (by adding or removing nodes).
• Several methods have been successfully applied to convert low-level sensor data into high-level semantic information, and to use them for detecting potential threats in a surveillance application - detection of unusual behaviour
• We understand better how to use a shared semantic vocabulary for dynamic node disco¬very and configuration, by means of detecting activation correlations between high-level symbols of neighbour nodes. In particular we know the constraints to be considered for successful correlation.
• A specific test platform consisting of multi-board and -processor hardware units has been developed from scratch for performing in situ tests.

Looking at the project objectives in detail, following achievements can be observed.
In the description of work for the SENSE project (annex I to the project contract), following objectives were defined:
Objective 1: To understand how to build networked systems of embedded components that can dynamically and automatically re-configure themselves
Objective 2: To understand how to convert low-level local information to semantic knowledge
Objective 3: To understand how to use semantic-level knowledge for network-centric compu¬tation
Objective 4: To understand how a shared semantic vocabulary influences dynamic node discovery and configuration
Objective 5: To understand how perception and information processing can be combined using low-level and high-level feature fusion
Objective 6: To understand how to facilitate networks of heterogeneous devices using a high-level semantic layer
Since it was necessary to build and test a prototypic SENSE system during the project in order to achieve the intended knowledge gain, a new objective has been added:
Objective 7: To build a test-bed platform to enable the intended knowledge gain, and in consideration of the application context require¬ments. The application context will be the security environment of the John Paul II International Airport Kraków-Balice.

During the course of the project, objective 6 was identified of little relevance to the project, since in SENSE only homogeneous nodes are used. And objective 5 is similarly considered of lower priority than the other objectives, because due to the nature of detected audio events their fusion with video objects is of smaller relevance to objective 7 than initially assumed.
In the following, metrics for final evaluation of the achievements of the relevant objectives are given, together with the observed degree of achievement and accompanying explanation:

Objective 1

• The addition of a node shall be detected by the network, and neighbourhoods automatically be updated
  • Degree of achievement: largely
  • Explanation: the functionality has been implemented and tested. In 80% of the tests, new nodes have been detected correctly, and communication to neighbour nodes correctly established.
• The permanent breakdown or removal of a node shall be detected by the network, and neighbourhoods automatically be updated
  • Degree of achievement: little
  • Explanation: the functionality has been implemented in principle, but could not be tested at a reportable depth.

Objective 2

• Events and objects reported from the sensor modalities to the reasoning modules shall be converted into knowledge of normal behaviour
  • Degree of achievement: largely
  • Explanation: local learning, based on the GNG algorithm, has been implemented and tested, since it serves as base for detection of unknown unusual behaviour and neighbour node detection. In addition, these events and objects are used for training the models used for detecting predefined unusual behaviour. The achievement is considered 'largely' rather than 'fully', because audio events are not used for training normality in the reasoning modules and not all predefined unusual behaviour models could be implemented.
• Events and objects reported from the sensor modalities to the reasoning modules shall be used to detect predefined alarm sceneries and unusual behaviour.
  • Degree of achievement: partially
  • Explanation: the unusual event detection functionality was implemented for three types of "unusualness". It was tested to some degree in laboratory, also using airport data, but due to lack of time no tests at the airport could have been carried out. For predefined alarm recognition, all planned functionality were implemented (in Matlab), but only parts of the functionality could be ported to the nodes platform, i.e. detection of running persons and unattended luggage. Both were tested in laboratory as well as at the airport, but for the latter no true positives could be observed.

Objective 3

• Computation shall happen only de-centralized on sensor nodes
  • Degree of achievement: fully
  • Explanation: the user interface only displays reported events, all other computations are done within the node.
• Semantic knowledge shall be usable for detecting unusual behaviour or scenarios extending over single sensor boundaries
  • Degree of achievement: little
  • Explanation: there was not enough time to realize events such as crowd detection and move or persons lurking in large areas covered by several nodes. But it was observed that correlations of distributed events can easily be detected at the GUI based on reported unusual events.

Objective 4

• High-level symbols represent the semantic vocabulary of SENSE nodes. The detection of common symbols between existing and new nodes (by means of correlated activation) shall be used to enable discovery of new neighbourhoods
  • Degree of achievement: largely
  • Explanation: In principle, the functionality has been implemented as expected, and also tested in laboratory, and at the airport. However, it does not work sufficiently robust, mainly due to the fact that it strongly depends on the results of the preceding steps in the processing pipeline.

Objective 5

• Correlation of high-level information shall be used to improve their local views (about their environments)
  • Degree of achievement: partial
  • Explanation: In principle, the use of beliefs for adapting the local activation probabilities was implemented, but it was not implemented on the node platform due to lack of time.

Objective 6

• The gain of knowledge necessary for objectives 1 to 5 is possible with the realised prototypic platform
  • Degree of achievement: fully
  • Explanation: though late in the project, the prototypic platform was used.
• The fulfilment of this objective can be evaluated by checking the coverage of the "must have requirements"
  • Degree of achievement: partially
  • Explanation: as described in the previous chapters, only a fraction of the planned experiments could be carried out.

Permanent Testing Stand in Environment of Krakow Airport
As an additional result of the SENSE project a permanent testing environment (see description above) remains at the airport.
The permanent SENSE nodes network with use of four SENSE nodes along with the entire infrastructure for remote access and data recording allows for further data collection from the real environment of the Krakow airport. The installation is mounted, started up and ready for experiments. Recordings can be made simultaneously from four nodes placed in four out of thirteen possible localizations depending of the needs of the user.