Subproject A01

Subproject A01

Multimodal 3D Body Imaging Camera

The subproject A01 focuses on the innovation of a multimodal sensor concept for precise and contactless detection of the body surface and the velocity vectors of all points on this surface. In the first phase of the project, the focus is on the tracking of hand movements, which in the second step is to be transferred to whole-body applications. The primary research question deals with the integration of an RGB-D camera with an imaging radar, overcoming challenges such as spatial calibration, temporal synchronization, analysis of reflectance behavior based on sensor characteristics, and the development of a coherent strategy for data fusion of both sensors. On the radar front, the project includes the development of new signal shapes and signal processing algorithms. These innovations should facilitate fast data acquisition allowing for velocity measurements and simultaneous high-resolution radar imaging. On the other side, the reconstruction of hand shapes and the estimation of hand posture using RGB-D data are researched. In addition, it is researched how velocity information from the imaging radar can improve RGB-D-based hand tracking.

 

                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                 

Contacts

Prof. Dr.-Ing. Marc Stamminger

Principal Investigator

Prof. Dr.-Ing. Martin Vossiek

Principal Investigator

Tobias Schwab, M. Sc.

Doctoral Candidate

Vanessa Wirth, M. Sc.

Doctoral Candidate

Johanna Bräunig, M. Sc.

Associated Doctoral Candidate

Lukas Engel, M. Sc.

Associated Doctoral Candidate

Dr.-Ing. Ingrid Ullmann, Akad. Rat

Postdoc

 

 

Additional Information

  • Wirth V. et al.:
    Automatic Spatial Calibration of Near-Field MIMO Radar With Respect to Optical Depth Sensors
    (2025). DOI: 10.48550/arXiv.2403.10981
  • Eskofier B., Vossiek M.:
    Invisible Sensing: Radar-based Biomonitoring. IEEE Open Journal of Engineering in Medicine and Biology (2024), S. 1-2. ISSN: 2644-1276. DOI: 10.1109/OJEMB.2024.3409086
  • Fröhlich AC. et al.:
    A Millimeter-Wave MIMO Radar Network for Human Activity Recognition and Fall Detection. 2024 IEEE Radar Conference (RadarConf24) (Denver, CO, 6 Mai 2024 – 10 Mai 2024). DOI: 10.1109/RadarConf2458775.2024.10548702
  • Gambietz M. et al.:
    Unobtrusive Gait Reconstructions using Radar-based Optimal Control Simulations. Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA 2024
  • Krauß D. et al.:
    A Review and Tutorial on Machine Learning- Enabled Radar-Based Biomedical Monitoring. IEEE Open Journal of Engineering in Medicine and Biology (2024), S. 1-22. ISSN: 2644-1276. DOI: 10.1109/OJEMB.2024.3397208
  • Müller J. et al.:
    End-to-End Learning for Human Pose Estimation from Raw Millimeter Wave Radar Data. Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, USA 2024
  • Schüßler C. et al.:
    Radar-Based Recognition of Static Hand Gestures in American Sign Language. 2024 IEEE Radar Conference (Denver, Colorado, 6 Mai 2024 – 10 April 2024)
  • Bräunig J. et al.:
    A Radar-Based Concept for Simultaneous High-Resolution Imaging and Pixel-Wise Velocity Analysis for Tracking Human Motion. IEEE Journal of Microwaves 4 (2024), S. 639-652. ISSN: 2692-8388. DOI: 10.1109/JMW.2024.3453570
  • Krauß D. et al.:
    A Review and Tutorial on Machine Learning- Enabled Radar-Based Biomedical Monitoring. IEEE Open Journal of Engineering in Medicine and Biology (2024), S. 1-22. ISSN: 2644-1276. DOI: 10.1109/OJEMB.2024.3397208
  • Rückert R. et al.:
    Data Compression for Close-Range Radar Imaging. IEEE Transactions on Radar Systems 1 (2024), S. 421-433. ISSN: 2832-7357. DOI: 10.1109/TRS.2024.3387288
  • Bräunig J. et al.:
    Radar-based Recognition of Activities of Daily Living in the Palliative Care Context Using Deep Learning. IEEE EMBS International Conference on Biomedical and Health Informatics (BHI), New York City: 2023. DOI: 10.1109/BHI58575.2023.10313506
  • Bräunig J. et al.:
    A Realistic Radar Ray Tracing Simulator for Hand Pose Imaging. EuRAD 2023 (Berlin)
  • Bräunig J. et al.:
    An Ultra-Efficient Approach for High-Resolution MIMO Radar Imaging of Human Hand Poses. IEEE Transactions on Radar Systems (2023), S. 1-13. ISSN: 2832-7357. DOI: 10.1109/TRS.2023.3309574
  • Guendel RG., Ullmann I., Fioranelli F., Yarovoy A.:
    Continuous People Crowd Monitoring defined as a Regression Problem using Radar Networks. European Radar Conference (Berlin, 20 September 2023 – 22 September 2023)
  • Schüßler C. et al.:
    Achieving Efficient and Realistic Full-Radar Simulations and Automatic Data Annotation by Exploiting Ray Meta Data from a Radar Ray Tracing Simulator. 2023 IEEE Radar Conference (RadarConf23) (2023 IEEE Radar Conference (RadarConf23). San Antonio, TX, US, 21 Juni 2023 – 26 Juni 2023). Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) (Grant Number: 44241933) (Hrsg.): IEEE Xplore 2023. DOI: 10.1109/RadarConf2351548.2023.10149641
  • Wirth V. et al.:
    Markerless RGB-D Hand Pose Estimation for Activity Monitoring of Musculoskeletal Diseases. IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI’23) (Pittsburgh, USA, 15 Oktober 2023 – 18 Oktober 2023)
  • Wirth V. et al.:
    ShaRPy: Shape Reconstruction and Hand Pose Estimation from RGB-D with Uncertainty. International Conference on Computer Vision (ICCV) Workshops (Paris, 2 Oktober 2023 – 6 Oktober 2023). Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops 2023.
  • Ullmann I. et al.:
    A Survey on Radar-Based Continuous Human Activity Recognition. IEEE Journal of Microwaves (2023), S. 1-13. ISSN: 2692-8388. DOI: 10.1109/JMW.2023.3264494
  • Ullmann I. et al.:
    Radar-Based Continuous Human Activity Recognition with Multi-Label Classification. IEEE SENSORS (Wien, 29 Oktober 2023 – 1 November 2023)
  • Vossiek M. et al.:
    A Tutorial on the Sequential Sampling Impulse Radar Concept and Selected Applications. IEEE Journal of Microwaves 3 (2023), S. 523-539. ISSN: 2692-8388. DOI: 10.1109/JMW.2022.3228724
  • Coppers B. et al.:
    Feasibility of using optoelectronic measurement of hand movement for characterizing hand function in rheumatoid arthritis. 75th EULAR (European Alliance of Associations for Rheumatology), European Congress of Rheumatology 2022 (Copenhagen, 1 Juni 2022 – 4 Juni 2022). DOI: 10.1136/annrheumdis-2022-eular.1741
  • Coppers B. et al.:
    Evaluation of marker-based motion capturing to charactherize basic hand movements in rheumatic patients. 27th Congress of the European Society of Biomechanics (Porto, 26 Juni 2022 – 29 Juni 2022)
  • Coppers B. et al.:
    Establishment and evaluation of a sensor-based method to assess hand function in patients with rheumatoid arthritis. World Biomechanics Congress 2022 (Taipei (online), 10 Juli 2022 – 14 Juli 2022)
  • Richer R. et al.:
    Detection of Acute Psychosocial Stress from Body Movements using Machine Learning. 79th Annual Scientific Meeting of the American Psychosomatic Society (Long Beach, CA, 23 März 2022 – 26 März 2022). Lippincott Williams & Wilkins (Hrsg.): Psychosomatic Medicine 2022
  • Rohleder N. et al.:
    Effect of Acute Psychosocial Stress on Body Movements. 79th Annual Scientific Meeting of the American Psychosomatic Society (Long Beach, CA, 23 März 2022 – 26 März 2022). Lippincott Williams & Wilkins (Hrsg.): Psychosomatic Medicine 2022
  • Körner G. et al.:
    Applicability and Performance of Standard Compression Methods for Efficient Data Transmission and Storage in Radar Networks. IEEE Journal of Microwaves 2 (2021), S. 78-96. ISSN: 2692-8388
    DOI: 10.1109/JMW.2021.3119781
  • Phutane U. et al.:
    Evaluation of Optical and Radar Based Motion Capturing Technologies for Characterizing Hand Movement in Rheumatoid Arthritis — A Pilot Study. Sensors 21 (2021), Art.Nr.: 1208. ISSN: 1424-8220. DOI: 10.3390/s21041208
  • Root K., Ullmann I., Gmehling S., Vossiek M.:
    Close range ISAR imaging with target velocity determination for security screening applications. Emerging Imaging and Sensing Technologies for Security and Defence VI SPIE Security + Defence (Online, 13 September 2021 – 18 September 2021). DOI: 10.1117/12.2602072