Tutorial HD-4: Mathematical Approaches for Monitoring Human Activities from Air- and Spaceborne Sensors

Presented by: Beril Sirmacek
German Aerospace Center (DLR)

Recently automatic detection of people and understanding their behaviors from images became a very important research field, since it can provide crucial information especially for police departments and crisis management teams. Tracking people, understanding their moving directions and speeds can be used for detecting abnormal situations. Detecting moving directions of people can be used to estimate locations where a crowd can congregate. Understanding behavioral dynamics of large people groups can also help to estimate future states of underground passages, shopping center like public entrances, or streets which can also affect the traffic. Due to the importance of the topic, in order to be able to monitor human activities, researchers tried to develop algorithms which can work on outdoor camera images or video streams. Some researchers developed local interest point extraction based crowd detection methods to classify terrestrial images as crowd and non-crowd regions. They observed that dense crowds produce a high number of interest points. Therefore, they used density of interest features for classification. After finding local features and their density descriptors, they used classifiers to classify crowds in images. Unfortunately, street cameras have also limited coverage area to monitor large outdoor events. In addition to that, in most of the cases, it is not possible to obtain close range street images or video streams in the place where an event occurs. Therefore, to analyze behaviors of people especially in very big outdoor events, the best way is to use remotely sensed (air- or spaceborne) images which began to give more information to researchers with the development of sensor technology. Since most of the previous approaches in this field needed clear detection of face or body features, curves, or boundaries to identify people which is not possible in airborne images, new approaches are needed to extract information from these images.

With this tutorial, we would like to give information about air- and spaceborne sensors and related software technologies for monitorung human activities. We provide detailed explanations on mathematical approaches with applications on real-scenarios.

Detailed outline of the talk is as provided below:

  1. Introduction
  2. Local Features
    1. General Introduction
    2. SIFT Features
    3. Harris Features
    4. FAST Features
  3. Feature Selection Methods
    1. Support Vector Machines for Feature Selection
    2. Mean Shift Segmentation Based Feature Selection
  4. Dense Crowd Detection
  5. Person Detection
  6. Person Tracking
    1. Kalman Filtering Based Tracking
    2. Particle Filtering Based Tracking
  7. Behavior Understanding
    1. Graph Theory
    2. Graph-Cut and Sub-Graph Matching for Behavior Understanding
  8. Conclusions

Beril Sirmacek received the BSc and MSc degrees from the Department of Electronics and Communication Engineering in Yildiz Technical University, Istanbul, in 2006 and 2007 respectively, and the PhD degree from the Department of Electrical and Electronics Engineering, Yeditepe University, Istanbul in 2009. During her PhD study, she was a research and teaching assistant and a member of the computer vision research laboratory at Yeditepe University. In this period, she has also made collaborations with many different universities and worked as a visiting researcher in remote sensing laboratory in Department of Information Engineering and Computer Science in University of Trento, Italy. She is currently working as a research fellow in the Department of Photogrammetry and Image Analysis in Remote Sensing Technology Institute (IMF) of German Aerospace Center (DLR), as a guest lecturer in Institute of Computer Science at University of Augsburg, and a teaching assistant for image processing course at Technical University of Munich (TUM), Germany. Besides, she is pursuing habilitation degree at University of Osnabrueck. Dr. Sirmacek has books and many important publications in reputable journal and conferences in remote sensing and optical image processing field.