Research Internship Project
Unsupervised learning of robotic multimodal data
Context
LEAT lab has been working for several years on the design of bio-inspired neural models. One of
them is inspired by the self-organization of the biological brain. This model named ReSOM has been
previously applied to the classification of multimodal data such as the representation of digits from
visual, auditory and motor data [1]. Thanks to a collaboration with a research lab on robotics, we
have jointly developed a dataset of data collected onto the iCuB humanoid robotic platform (arm,
head, eyes positions and images from the camera embedded onto the robot).
The goal of this internship will be to work on this dataset and to work on the unsupervised learning
of this data thanks to the software framework developed at LEAT and integrating the ReSOM neural
model.
Project mission
The project mission will be organized in several periods:
- Bibliographic study on multimodal processing with machine learning [3]
- Introduction to the existing Sw and Hw solutions [2] developed at LEAT for Edge AI applications
- Sw development for the preparation of data
- Machine learning based on the ReSOM neural architecture
- Analysis of the results on a computer
- Experimentations and validation of the system embedded onto the iCub robotic platform
- Publication in an international conference.
References
[1] Brain-inspired self-organization with cellular neuromorphic computing for multimodal unsupervised
learning, L Khacef, L Rodriguez, B Miramond, MDPI Electronics, 2020
[2] A unified software/hardware scalable architecture for brain-inspired computing based on self-organizing
neural models, AR Muliukov, L Rodriguez, B Miramond, L Khacef, J Schmidt, Q Berthet, , Frontiers in
neuroscience 16, 825879, 2022
[3] Modeling multisensory enhancement with self-organizing maps, JG Martin, MA Meredith, K Ahmad –
Frontiers in computational …, 2009 – frontiersin.org
Practical information
Location: LEAT Lab / SophiaTech Campus, Sophia Antipolis
Duration: 6 months from february 2025
Profile: embedded programming, micro-controller, sensors
Research keywords: Embedded systems, signal processing, Edge AI
Contact and supervision
Benoît Miramond, Laurent Rodriguez
LEAT Lab – University Cote d’Azur / CNRS
Polytech Nice Sophia
04.89.15.44.39. / benoit.miramond@univ-cotedazur.fr