publications | Stefan K Ehrlich

2025

Brain-computer interface improved symptoms of isolated focal laryngeal dystonia: A single-blind study.

Stefan K Ehrlich, Garett Tougas, Jacob Bernstein, and 3 more authors

Movement Disorders (in press), 2025

Abs Bib PDF

Background and objective: Laryngeal dystonia (LD) is a focal task-specific dystonia, affecting speaking but not whispering or emotional vocalizations. Therapeutic options for LD are limited. We developed and tested a non-invasive, closed-loop, neurofeedback, brain–computer interface (BCI) intervention for LD treatment. Methods: Ten patients with isolated focal LD participated in the study. The personalized BCI system included visual neurofeedback of individual real-time electroencephalographic (EEG) activity during symptomatic speaking compared to asymptomatic whispering, presented in the virtual reality (VR) environment of real-life scenarios. During five consecutive days of intervention, patients used the BCI to learn to modulate their abnormally increased brain activity during speaking and match it to near-normal activity of asymptomatic whispering. Changes in voice symptoms and EEG activity were quantified for the evaluation of BCI effects. Results: Compared to baseline, LD patients had a statistically significant reduction of their voice symptoms on Days 1–5 of BCI intervention. Thi was paralleled by improved controllability of the visual neurofeedback and a significant reduction of left frontal delta power, including superior and middle frontal gyri, on Day 1 and left central gamma power, including premotor, primary sensorimotor, and inferior parietal areas, on Days 3 and 5. The majority of patients (70%) reported sustained positive effects of the BCI intervention on their voice quality 1 week after the study participation. Conclusion: The closed-loop BCI neurofeedback intervention specifically targeting disorder pathophysiology shows significant potential as a novel treatment option for patients with LD and likely other forms of task-specific focal dystonia.
@article{ehrlich2025bci, title = {Brain-computer interface improved symptoms of isolated focal laryngeal dystonia: A single-blind study.}, author = {Ehrlich, Stefan K and Tougas, Garett and Bernstein, Jacob and Buie, Nicole and Rumbach, Anna F and Simonyan, Kristina}, journal = {Movement Disorders (in press)}, year = {2025}, }
Lightweight Structured Multimodal Reasoning for Clinical Scene Understanding in Robotics

Saurav Jha and Stefan K Ehrlich

arXiv preprint arXiv:2509.22014, 2025

Abs Bib PDF

Healthcare robotics requires robust multimodal perception and reasoning to ensure safety in dynamic clinical environments. Current Vision-Language Models (VLMs) demonstrate strong general-purpose capabilities but remain limited in temporal reasoning, uncertainty estimation, and structured outputs needed for robotic planning. We present a lightweight agentic multimodal framework for video-based scene understanding. Combining the Qwen2.5-VL-3B-Instruct model with a SmolAgent-based orchestration layer, it supports chain-of-thought reasoning, speech-vision fusion, and dynamic tool invocation. The framework generates structured scene graphs and leverages a hybrid retrieval module for interpretable and adaptive reasoning. Evaluations on the Video-MME benchmark and a custom clinical dataset show competitive accuracy and improved robustness compared to state-of-the-art VLMs, demonstrating its potential for applications in robot-assisted surgery, patient monitoring, and decision support.
@article{jha2025lightweight, title = {Lightweight Structured Multimodal Reasoning for Clinical Scene Understanding in Robotics}, author = {Jha, Saurav and Ehrlich, Stefan K}, journal = {arXiv preprint arXiv:2509.22014}, year = {2025}, primaryclass = {cs.CV}, url = {https://arxiv.org/abs/2509.22014}, }
AffectMachine-Pop: A controllable expert system for real-time pop music generation

Kat R Agres, Adyasha Dash, Phoebe Chua, and 1 more author

arXiv preprint arXiv:2506.08200, 2025

Abs Bib PDF

Music is a powerful medium for influencing listeners’ emotional states, and this capacity has driven a surge of research interest in AI-based affective music generation in recent years. Many existing systems, however, are a black box which are not directly controllable, thus making these systems less flexible and adaptive to users. We present AffectMachine-Pop, an expert system capable of generating retro-pop music according to arousal and valence values, which can either be pre-determined or based on a listener’s real-time emotion states. To validate the efficacy of the system, we conducted a listening study demonstrating that AffectMachine-Pop is capable of generating affective music at target levels of arousal and valence. The system is tailored for use either as a tool for generating interactive affective music based on user input, or for incorporation into biofeedback or neurofeedback systems to assist users with emotion self-regulation.
@article{agres2025affectmachine, title = {AffectMachine-Pop: A controllable expert system for real-time pop music generation}, author = {Agres, Kat R and Dash, Adyasha and Chua, Phoebe and Ehrlich, Stefan K}, journal = {arXiv preprint arXiv:2506.08200}, year = {2025}, primaryclass = {cs.HC}, url = {https://arxiv.org/abs/2506.08200}, }
Impact of Audio-Visual Complexity on Symptomatology of Laryngeal Dystonia: A Virtual Reality Study

Jimmy Petit, Stefan K Ehrlich, Garrett Tougas, and 3 more authors

The Laryngoscope, 2025

Abs Bib PDF

Background: Laryngeal dystonia (LD) is an isolated focal dystonia characterized by involuntary spasms in laryngeal muscles selectively impairing speech production. Anecdotal observations reported the worsening of LD symptoms in stressful or vocally demanding situations. Objectives: To examine the impact of surrounding audio-visual complexity on LD symptomatology for a better understanding of disorder phenomenology. Methods: We developed well-controlled virtual reality (VR) environments of real-life interpersonal communications to investigate how different levels of audio-visual complexity may impact LD symptoms. The VR experiments were conducted over five consecutive days, during which each patient experienced 10 h of 4100 experimental trials in VR with gradually increasing audio-visual complexity. Daily reports were collected about patients’ voice changes, as well as their comfort, engagement, concentration, and drowsiness from using VR technology. Results: After a weekly VR exposure, 82% of patients reported changes in their voice symptoms related to changes in background audio-visual complexity. Significant differences in voice symptoms were found between the first two levels of the audio-visual challenge complexity independent of study sessions or VR environments. Conclusion: This study demonstrated that LD symptoms are impacted by audio-visual background across various virtual realistic settings. These findings should be taken into consideration when planning behavioral experiments or evaluating the outcomes of clinical trials in these patients. Moreover, these data show that VR presents a reliable and useful technology for providing real-life assessments of the impact of various experimental settings, such as during the testing of novel therapeutic interventions in these patients.
@article{petit2025impact, title = {Impact of Audio-Visual Complexity on Symptomatology of Laryngeal Dystonia: A Virtual Reality Study}, author = {Petit, Jimmy and Ehrlich, Stefan K and Tougas, Garrett and Bernstein, Jacob M and Buie, Nicole E and Simonyan, Kristina}, journal = {The Laryngoscope}, volume = {135}, number = {2}, pages = {787--793}, year = {2025}, publisher = {Wiley Online Library}, }

2024

Predicting the Performance of Human-Agent Collaboration: Insights from Uncertainty Reduction Theory, Dynamic Capabilities Perspectives, and Human Brains

Sohvi Heaton, Jin Ho Yun, and Stefan K Ehrlich

SSRN Preprints, 2024

Abs Bib PDF

We investigate the factors influencing the performance of collaboration between humans and autonomous agents, focusing on how and why this performance varies. Our hypotheses are grounded in theories of uncertainty reduction and dynamic capabilities. Results from a laboratory experiment involving approximately 45 minutes of human-agent collaboration per subject indicate that a passive information-seeking strategy affects collaboration performance, particularly when humans observe correct actions performed by the agent. Additionally, people’s adaptability, assessed through a measure of dynamic capabilities, positively influences performance. This effect is particularly strong when individuals feel a high level of safety with the agent. Using a multidisciplinary approach, we highlight unique challenges in human-robot interaction, particularly increased uncertainty, to enhance our understanding of how these factors affect the effectiveness of existing theories in human-agent collaborative settings.
@article{heaton2024predicting, title = {Predicting the Performance of Human-Agent Collaboration: Insights from Uncertainty Reduction Theory, Dynamic Capabilities Perspectives, and Human Brains}, author = {Heaton, Sohvi and Yun, Jin Ho and Ehrlich, Stefan K}, journal = {SSRN Preprints}, year = {2024}, }
Framing image registration as a landmark detection problem for label-noise-aware task representation (HitR)

Diana Waldmannstetter, Ivan Ezhov, Benedikt Wiestler, and 8 more authors

arXiv preprint arXiv:2308.01318, 2024

Abs Bib PDF

Accurate image registration is pivotal in biomedical image analysis, where selecting suitable registration algorithms demands careful consideration. While numerous algorithms are available, the evaluation metrics to assess their performance have remained relatively static. This study addresses this challenge by introducing a novel evaluation metric termed Landmark Hit Rate (HitR), which focuses on the clinical relevance of image registration accuracy. Unlike traditional metrics such as Target Registration Error, which emphasize subresolution differences, HitR considers whether registration algorithms successfully position landmarks within defined confidence zones. This paradigm shift acknowledges the inherent annotation noise in medical images, allowing for more meaningful assessments. To equip HitR with label-noise-awareness, we propose defining these confidence zones based on an Inter-rater Variance analysis. Consequently, hit rate curves are computed for varying landmark zone sizes, enabling performance measurement for a task-specific level of accuracy. Our approach offers a more realistic and meaningful assessment of image registration algorithms, reflecting their suitability for clinical and biomedical applications.
@article{waldmannstetter2023framing, title = {Framing image registration as a landmark detection problem for label-noise-aware task representation (HitR)}, author = {Waldmannstetter, Diana and Ezhov, Ivan and Wiestler, Benedikt and Campi, Francesco and Kukuljan, Ivan and Ehrlich, Stefan K and Vinayahalingam, Shankeeth and Baheti, Bhakti and Chakrabarty, Satrajit and Baid, Ujjwal and others}, journal = {arXiv preprint arXiv:2308.01318}, year = {2024}, primaryclass = {eess.IV}, }
Somatosensory Mismatch Response in Patients with Cerebral Palsy

Stefan K Ehrlich, Susmita Roy, and Renée Lampe

Applied Sciences, 2024

Abs Bib PDF

Background: Mismatch negativity (MMN), an event-related potential (ERP) component occurring at specific recording sites and latency, is associated with an automatic change detection response, generally elicited using oddball paradigms wherein infrequent stimuli are embedded in repeated, frequent stimuli. To verify the presence of mismatch-related ERP responses to somatosensory stimulation in individuals with cerebral palsy (CP), we conducted a preliminary study involving healthy participants and patients with CP. Methods: Both groups underwent ’frequent’ and ’infrequent’ stimulation applied to the ring finger and thumb of their left hand, respectively. ERPs were recorded at frontal, central, and parietal scalp locations using electroencephalography. A healthy cohort tested the experimental protocol and showed evidence that mismatch-related ERP responses were observable. Subsequent analysis focused on the patient group. Results: Statistically significant differences between the two types of stimuli were observed on the frontocentral and parietal channels between 150 and 250 ms after the stimulus onset in the patient group. Furthermore, a late discriminative response was observed in the frontal and parietal channels. Conclusion: The results demonstrate the presence of mismatch-related ERP responses in individuals with CP.
@article{roy2024somatosensory, title = {Somatosensory Mismatch Response in Patients with Cerebral Palsy}, author = {Ehrlich, Stefan K and Roy, Susmita and Lampe, Ren{\'e}e}, journal = {Applied Sciences}, volume = {14}, number = {3}, pages = {1030}, year = {2024}, publisher = {MDPI}, }
The brain tumor segmentation (brats) challenge: Local synthesis of healthy brain tissue via inpainting

Florian Kofler, Felix Meissen, Felix Steinbauer, and 8 more authors

arXiv preprint arXiv:2305.08992, 2024

Abs Bib PDF

A myriad of algorithms for the automatic analysis of brain MR images is available to support clinicians in their decision-making. For brain tumor patients, the image acquisition time series typically starts with an already pathological scan. This poses problems, as many algorithms are designed to analyze healthy brains and provide no guarantee for images featuring lesions. Examples include, but are not limited to, algorithms for brain anatomy parcellation, tissue segmentation, and brain extraction. To solve this dilemma, we introduce the BraTS inpainting challenge. Here, the participants explore inpainting techniques to synthesize healthy brain scans from lesioned ones. The following manuscript contains the task formulation, dataset, and submission procedure. Later, it will be updated to summarize the findings of the challenge. The challenge is organized as part of the ASNR-BraTS MICCAI challenge.
@article{kofler2023brain, title = {The brain tumor segmentation (brats) challenge: Local synthesis of healthy brain tissue via inpainting}, author = {Kofler, Florian and Meissen, Felix and Steinbauer, Felix and Graf, Robert and Ehrlich, Stefan K and Reinke, Annika and Oswald, Eva and Waldmannstetter, Diana and Hoelzl, Florian and Horvath, Izabela and others}, journal = {arXiv preprint arXiv:2305.08992}, year = {2024}, }

2023

Temporal signature of task-specificity in isolated focal laryngeal Dystonia

Stefan K Ehrlich, Giovanni Battistella, and Kristina Simonyan

Movement Disorders, 2023

Abs Bib PDF

Background and Objective: Laryngeal dystonia (LD) is focal task-specific dystonia, predominantly affecting speech but not whispering or emotional vocalizations. Prior neuroimaging studies identified brain regions forming a dystonic neural network and contributing to LD pathophysiology. However, the underlying temporal dynamics of these alterations and their contribution to the taskspecificity of LD remain largely unknown. The objective of the study was to identify the temporal–spatial signature of altered cortical oscillations associated with LD pathophysiology. Methods: We used high-density 128-electrode electroencephalography (EEG) recordings during symptomatic speaking and two asymptomatic tasks, whispering and writing, in 24 LD patients and 22 healthy individuals to investigate the spectral dynamics, spatial localization, and interregional effective connectivity of aberrant cortical oscillations within the dystonic neural network, as well as their relationship with LD symptomatology. Results: Symptomatic speaking in LD patients was characterized by significantly increased gamma synchronization in the middle/superior frontal gyri, primary somatosensory cortex, and superior parietal lobule, establishing the altered prefrontal-parietal loop. Hyperfunctional connectivity from the left middle frontal gyrus to the right superior parietal lobule was significantly correlated with the age of onset and the duration of LD symptoms. Asymptomatic whisper in LD patients had not no statistically significant changes in any frequency band, whereas asymptomatic writing was characterized by significantly decreased synchronization of beta-band power localized in the right superior frontal gyrus. Conclusion: Task-specific oscillatory activity of prefrontal-parietal circuitry is likely one of the underlying mechanisms of aberrant heteromodal integration of information processing and transfer within the neural network leading to dystonic motor output.
@article{ehrlich2023temporal, title = {Temporal signature of task-specificity in isolated focal laryngeal Dystonia}, author = {Ehrlich, Stefan K and Battistella, Giovanni and Simonyan, Kristina}, journal = {Movement Disorders}, volume = {38}, number = {10}, pages = {1925--1935}, year = {2023}, publisher = {Wiley Online Library}, }
Human-robot collaborative task planning using anticipatory brain responses

Stefan K Ehrlich, Emmanuel Dean-Leon, Nicholas Tacca, and 3 more authors

Plos one, 2023

Abs Bib PDF

Human-robot interaction (HRI) describes scenarios in which both human and robot work as partners, sharing the same environment or complementing each other on a joint task. HRI is characterized by the need for high adaptability and flexibility of robotic systems toward their human interaction partners. One of the major challenges in HRI is task planning with dynamic subtask assignment, which is particularly challenging when subtask choices of the human are not readily accessible by the robot. In the present work, we explore the feasibility of using electroencephalogram (EEG) based neuro-cognitive measures for online robot learning of dynamic subtask assignment. To this end, we demonstrate in an experimental human subject study, featuring a joint HRI task with a UR10 robotic manipulator, the presence of EEG measures indicative of a human partner anticipating a takeover situation from human to robot or vice-versa. The present work further proposes a reinforcement learning based algorithm employing these measures as a neuronal feedback signal from the human to the robot for dynamic learning of subtask-assignment. The efficacy of this algorithm is validated in a simulation-based study. The simulation results reveal that even with relatively low decoding accuracies, successful robot learning of subtask-assignment is feasible, with around 80% choice accuracy among four subtasks within 17 minutes of collaboration. The simulation results further reveal that scalability to more subtasks is feasible and mainly accompanied with longer robot learning times. These findings demonstrate the usability of EEG-based neuro-cognitive measures to mediate the complex and largely unsolved problem of human-robot collaborative task planning.
@article{ehrlich2023human, title = {Human-robot collaborative task planning using anticipatory brain responses}, author = {Ehrlich, Stefan K and Dean-Leon, Emmanuel and Tacca, Nicholas and Armleder, Simon and Dimova-Edeleva, Viktorija and Cheng, Gordon}, journal = {Plos one}, volume = {18}, number = {7}, pages = {e0287958}, year = {2023}, publisher = {Public Library of Science San Francisco, CA USA}, }
Panoptica–instance-wise evaluation of 3D semantic and instance segmentation maps

Florian Kofler, Hendrik Möller, Josef A Buchner, and 8 more authors

arXiv preprint arXiv:2312.02608, 2023

Abs Bib PDF

This paper introduces panoptica, a versatile and performance-optimized package designed for computing instance-wise segmentation quality metrics from 2D and 3D segmentation maps. panoptica addresses the limitations of existing metrics and provides a modular framework that complements the original IOU-based Panoptic Quality with other metrics, such as the distance metric ASSD. The package is open-source, implemented in Python, and accompanied by comprehensive documentation and tutorials. panoptica employs a three-step metrics computation process to cover diverse use cases. We demonstrate the efficacy of panoptica on various real-world biomedical datasets, where an instance-wise evaluation is instrumental for an accurate representation of the underlying clinical task. Overall, we envision panoptica as a valuable tool facilitating in-depth evaluation of segmentation methods.
@article{kofler2023panoptica, title = {Panoptica--instance-wise evaluation of 3D semantic and instance segmentation maps}, author = {Kofler, Florian and M{\"o}ller, Hendrik and Buchner, Josef A and de la Rosa, Ezequiel and Ezhov, Ivan and Rosier, Marcel and Mekki, Isra and Shit, Suprosanna and Negwer, Moritz and Al-Maskari, Rami and others}, journal = {arXiv preprint arXiv:2312.02608}, year = {2023}, }
Listening to familiar music induces continuous inhibition of alpha and low-beta power

Alireza Malekmohammadi, Stefan K Ehrlich, Josef P Rauschecker, and 1 more author

Journal of Neurophysiology, 2023

Abs Bib PDF

How the brain responds temporally and spectrally when we listen 40 to familiar versus unfamiliar musical sequences remains unclear. This study uses EEG techniques to investigate the continuous electrophysiological changes in the human brain during passive listening to familiar and unfamiliar musical excerpts. EEG activity was recorded in twenty participants while passively listening to 10 seconds of classical music, and they were then asked to indicate their self assessment of familiarity. We analyzed the EEG data in two manners: familiarity based on the within-subject design, i.e., averaging trials for each condition and participant, and familiarity based on the same music excerpt, i.e., averaging trials for each condition and music excerpt. By comparing the familiar condition with the unfamiliar condition and local baseline, sustained low beta power (12-16 Hz) suppression was observed in both analyses in frontocentral and left frontal electrodes after 800 ms. However, sustained alpha power (8-12 Hz) decreased in frontocentral and posterior electrodes after 850 ms only in the first type of analysis. Our study indicates that listening to familiar music elicits a late sustained spectral response (inhibition of alpha/low-beta power from 800 ms to 10 s). Moreover, the results showed alpha suppression reflects increased attention or arousal/engagement due to listening to familiar music; nevertheless, low-beta suppression exhibits the effect of familiarity.
@article{malekmohammadi2023listening, title = {Listening to familiar music induces continuous inhibition of alpha and low-beta power}, author = {Malekmohammadi, Alireza and Ehrlich, Stefan K and Rauschecker, Josef P and Cheng, Gordon}, journal = {Journal of Neurophysiology}, volume = {129}, number = {6}, pages = {1344--1358}, year = {2023}, publisher = {American Physiological Society Rockville, MD}, }
Modulation of theta and gamma oscillations during familiarization with previously unknown music

Alireza Malekmohammadi, Stefan K Ehrlich, and Gordon Cheng

Brain research, 2023

Abs Bib PDF

Repeated listening to unknown music leads to gradual familiarization with musical sequences. Passively listening to musical sequences could involve an array of dynamic neural responses in reaching familiarization with the musical excerpts. This study elucidates the dynamic brain response and its variation over time by investigating the electrophysiological changes during the familiarization with initially unknown music. Twenty subjects were asked to familiarize themselves with previously unknown 10 s classical music excerpts over three repetitions while their electroencephalogram was recorded. Dynamic spectral changes in neural oscillations are monitored by time-frequency analyses for all frequency bands (theta: 5-9 Hz, alpha: 9-13 Hz, low-beta: 13-21 Hz, high beta: 21-32 Hz, and gamma: 32-50 Hz). Time-frequency analyses reveal sustained theta event-related desynchronization (ERD) in the frontal-midline and the left pre-frontal electrodes which decreased gradually from 1st to 3rd time repetition of the same excerpts (frontal-midline: 57.90 %, left-prefrontal: 75.93 %). Similarly, sustained gamma ERD decreased in the frontal-midline and bilaterally frontal/temporal areas (frontalmidline: 61.47 %, left-frontal: 90.88 %, right-frontal: 87.74 %). During familiarization, the decrease of theta ERD is superior in the first part (1-5 s) whereas the decrease of gamma ERD is superior in the second part (5-9 s) of music excerpts. The results suggest that decreased theta ERD is associated with successfully identifying familiar sequences, whereas decreased gamma ERD is related to forming unfamiliar sequences.
@article{malekmohammadi2023modulation, title = {Modulation of theta and gamma oscillations during familiarization with previously unknown music}, author = {Malekmohammadi, Alireza and Ehrlich, Stefan K and Cheng, Gordon}, journal = {Brain research}, volume = {1800}, pages = {148198}, year = {2023}, publisher = {Elsevier}, }

2022

Brain computer interface to distinguish between self and other related errors in human agent collaboration

Viktorija Dimova-Edeleva, Stefan K Ehrlich, and Gordon Cheng

Scientific Reports, 2022

Abs Bib PDF

When a human and machine collaborate on a shared task, ambiguous events might occur that could be perceived as an error by the human partner. In such events, spontaneous error-related potentials (ErrPs) are evoked in the human brain. Knowing whom the human perceived as responsible for the error would help a machine in co-adaptation and shared control paradigms to better adapt to human preferences. Therefore, we ask whether self- and agent-related errors evoke different ErrPs. Eleven subjects participated in an electroencephalography human-agent collaboration experiment with a collaborative trajectory-following task on two collaboration levels, where movement errors occurred as trajectory deviations. Independently of the collaboration level, we observed a higher amplitude of the responses on the midline central Cz electrode for self-related errors compared to observed errors made by the agent. On average, Support Vector Machines classified self- and agent-related errors with 72.64% accuracy using subject-specific features. These results demonstrate that ErrPs can tell if a person relates an error to themselves or an external autonomous agent during collaboration. Thus, the collaborative machine will receive more informed feedback for the error attribution that allows appropriate error identification, a possibility for correction, and avoidance in future actions.
@article{dimova2022brain, title = {Brain computer interface to distinguish between self and other related errors in human agent collaboration}, author = {Dimova-Edeleva, Viktorija and Ehrlich, Stefan K and Cheng, Gordon}, journal = {Scientific Reports}, volume = {12}, number = {1}, pages = {20764}, year = {2022}, publisher = {Nature Publishing Group UK London}, }
Neuro-cognitive assessment of intentional control methods for a soft elbow exosuit using error-related potentials

Nicholas Tacca, John Nassour, Stefan K Ehrlich, and 2 more authors

Journal of NeuroEngineering and Rehabilitation, 2022

Abs Bib PDF

Soft exosuits offer promise to support users in everyday workload tasks by providing assistance. However, acceptance of such systems remains low due to the difficulty of control compared with rigid mechatronic systems. Recently, there has been progress in developing control schemes for soft exosuits that move in line with user intentions. While initial results have demonstrated sufficient device performance, the assessment of user experience via the cognitive response has yet to be evaluated. To address this, we propose a soft pneumatic elbow exosuit designed based on our previous work to provide assistance in line with user expectations utilizing two existing state-of-the-art control methods consisting of a gravity compensation and myoprocessor based on muscle activation. A user experience study was conducted to assess whether the device moves naturally with user expectations and the potential for device acceptance by determining when the exosuit violated user expectations through the neuro-cognitive and motor response. Brain activity from electroencephalography (EEG) data revealed that subjects elicited error-related potentials (ErrPs) in response to unexpected exosuit actions, which were decodable across both control schemes with an average accuracy of 76.63 +/- 1.73% across subjects. Additionally, unexpected exosuit actions were further decoded via the motor response from electromyography (EMG) and kinematic data with a grand average accuracy of 68.73 +/- 6.83% and 77.52 +/- 3.79% respectively. This work demonstrates the validation of existing state-of-the-art control schemes for soft wearable exosuits through the proposed soft pneumatic elbow exosuit. We demonstrate the feasibility of assessing device performance with respect to the cognitive response through decoding when the device violates user expectations in order to help understand and promote device acceptance.
@article{tacca2022neuro, title = {Neuro-cognitive assessment of intentional control methods for a soft elbow exosuit using error-related potentials}, author = {Tacca, Nicholas and Nassour, John and Ehrlich, Stefan K and Berberich, Nicolas and Cheng, Gordon}, journal = {Journal of NeuroEngineering and Rehabilitation}, volume = {19}, number = {1}, pages = {124}, year = {2022}, publisher = {Springer}, }

Brain–computer interfaces for treatment of focal dystonia

Kristina Simonyan, Stefan K Ehrlich, Richard Andersen, and 8 more authors

Movement Disorders, 2022

Bib PDF

@article{simonyan2022brain,
  title = {Brain--computer interfaces for treatment of focal dystonia},
  author = {Simonyan, Kristina and Ehrlich, Stefan K and Andersen, Richard and Brumberg, Jonathan and Guenther, Frank and Hallett, Mark and Howard, Matthew A and del R Mill{\'a}n, Jos{\'e} and Reilly, Richard B and Schultz, Tanja and others},
  journal = {Movement Disorders},
  pages = {10--1002},
  year = {2022},
}

2021

A Comparative Pilot Study on ErrPs for Different Usage Conditions of an Exoskeleton with a Mobile EEG Device

Svea Marie Meyer, Ashish Rao Mangalore, Stefan K Ehrlich, and 3 more authors

In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021

Abs Bib PDF

Exoskeletons and prosthetic devices controlled using brain-computer interfaces (BCIs) can be prone to errors due to inconsistent decoding. In recent years, it has been demonstrated that error-related potentials (ErrPs) can be used as a feedback signal in electroencephalography (EEG) based BCIs. However, modern BCIs often take large setup times and are physically restrictive, making them impractical for everyday use. In this paper, we use a mobile and easy-to-setup EEG device to investigate whether an erroneously functioning 1-DOF exoskeleton in different conditions, namely, visually observing and wearing the exoskeleton, elicits a brain response that can be classified. We develop a pipeline that can be applied to these two conditions and observe from our experiments that there is evidence for neural responses from electrodes near regions associated with ErrPs in an environment that resembles the real world. We found that these error-related responses can be classified as ErrPs with accuracies ranging from 60% to 71%, depending on the condition and the subject. Our pipeline could be further extended to detect and correct erroneous exoskeleton behavior in real-world settings.
@inproceedings{meyer2021comparative, title = {A Comparative Pilot Study on ErrPs for Different Usage Conditions of an Exoskeleton with a Mobile EEG Device}, author = {Meyer, Svea Marie and Mangalore, Ashish Rao and Ehrlich, Stefan K and Berberich, Nicolas and Nassour, John and Cheng, Gordon}, booktitle = {2021 43rd Annual International Conference of the IEEE Engineering in Medicine \& Biology Society (EMBC)}, pages = {6203--6206}, year = {2021}, organization = {IEEE}, }
Visually-guided grip selection for soft-hand exoskeleton

Xingying Chen, Simone Löhlein, John Nassour, and 3 more authors

In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021

Abs Bib PDF

This paper presents a visually-guided grip selection based on the combination of object recognition and tactile feedback of a soft-hand exoskeleton intended for hand rehabilitation. A pre-trained neural network is used to recognize the object in front of the hand exoskeleton, which is then mapped to a suitable grip type. With the object cue, it actively assists users in performing different grip movements without calibration. In a pilot experiment, one healthy user completed four different grasp-and-move tasks repeatedly. All trials were completed within 25 seconds and only one out of 20 trials failed. This shows that automated movement training can be achieved by visual guidance even without biomedical sensors. In particular, in the private setting at home without clinical supervision, it is a powerful tool for repetitive training of dailyliving activities.
@inproceedings{chen2021visually, title = {Visually-guided grip selection for soft-hand exoskeleton}, author = {Chen, Xingying and L{\"o}hlein, Simone and Nassour, John and Ehrlich, Stefan K and Berberich, Nicolas and Cheng, Gordon}, booktitle = {2021 43rd Annual International Conference of the IEEE Engineering in Medicine \& Biology Society (EMBC)}, pages = {4713--4716}, year = {2021}, organization = {IEEE}, }
Demonstrating the viability of mapping deep learning based EEG decoders to spiking networks on low-powered neuromorphic chips

Matthijs Pals, Rafael J Pérez Belizón, Nicolas Berberich, and 3 more authors

In 2021 43rd Annual International Conference of the IEEE Engineering in Medicine & Biology Society (EMBC), 2021

Abs Bib PDF

Accurate and low-power decoding of brain signals such as electroencephalography (EEG) is key to constructing brain-computer interface (BCI) based wearable devices. While deep learning approaches have progressed substantially in terms of decoding accuracy, their power consumption is relatively high for mobile applications. Neuromorphic hardware arises as a promising solution to tackle this problem since it can run massive spiking neural networks with energy consumption orders of magnitude lower than traditional hardware. Herein, we show the viability of directly mapping a continuous-valued convolutional neural network for motor imagery EEG classification to a spiking neural network. The converted network, able to run on the SpiNNaker neuromorphic chip, only shows a 1.91% decrease in accuracy after conversion. Thus, we take full advantage of the benefits of both deep learning accuracies and low-power neuro-inspired hardware, properties that are key for the development of wearable BCI devices.
@inproceedings{pals2021demonstrating, title = {Demonstrating the viability of mapping deep learning based EEG decoders to spiking networks on low-powered neuromorphic chips}, author = {Pals, Matthijs and Beliz{\'o}n, Rafael J P{\'e}rez and Berberich, Nicolas and Ehrlich, Stefan K and Nassour, John and Cheng, Gordon}, booktitle = {2021 43rd Annual International Conference of the IEEE Engineering in Medicine \& Biology Society (EMBC)}, pages = {6102--6105}, year = {2021}, organization = {IEEE}, }

2020

Neuroengineering challenges of fusing robotics and neuroscience

Gordon Cheng, Stefan K Ehrlich, Mikhail Lebedev, and 1 more author

Science robotics, 2020

Abs Bib PDF

@article{cheng2020neuroengineering,
  title = {Neuroengineering challenges of fusing robotics and neuroscience},
  author = {Cheng, Gordon and Ehrlich, Stefan K and Lebedev, Mikhail and Nicolelis, Miguel AL},
  journal = {Science robotics},
  volume = {5},
  number = {49},
  pages = {eabd1911},
  year = {2020},
  publisher = {American Association for the Advancement of Science},
}

Calibration-free error-related potential decoding with adaptive subject-independent models: a comparative study

Florian M Schönleitner, Lukas Otter, Stefan K Ehrlich, and 1 more author

IEEE Transactions on Medical Robotics and Bionics, 2020

Abs Bib PDF

Error-related potentials (ErrPs) provide an elegant method to improve human-machine interaction by detecting incorrect system behavior from the electroencephalogram of a human operator in real time. In this paper, we focus on adaptive subject-independent decoding models particularly suitable for ErrP classification. As individualized decoding models require a time-consuming calibration phase, such models provide a promising alternative. Based on an investigation of the characteristics of inter-subject variations in the signal and feature space, we evaluate the performance of a decoding model solely trained on prior data and the effectiveness of adapting this model to a new subject in a comparative study. Our results show that such a generalized model can decode ErrPs with an acceptable average accuracy of 72.7+/-9.66% and that supervised adaptation can significantly improve the accuracy of the generalized model after adaptation with 85 trials by on average +3.8+/-5.1%. We show that adaptation of subject-independent decoding models is superior to the traditional calibration procedure. Unsupervised adaptation only proved effective for some subjects and requires further attention to be practical for a broader range of subjects. Consequently, our work contributes to the development of calibration-free ErrP decoding in the broader scope of enhancing usability of ErrPs for human-machine interaction.
@article{schonleitner2020calibration, title = {Calibration-free error-related potential decoding with adaptive subject-independent models: a comparative study}, author = {Sch{\"o}nleitner, Florian M and Otter, Lukas and Ehrlich, Stefan K and Cheng, Gordon}, journal = {IEEE Transactions on Medical Robotics and Bionics}, volume = {2}, number = {3}, pages = {399--409}, year = {2020}, publisher = {IEEE}, }

2019

A computational model of human decision making and learning for assessment of co-adaptation in neuro-adaptive human-robot interaction

Stefan K Ehrlich and Gordon Cheng

In 2019 IEEE International Conference on Systems, Man and Cybernetics (SMC), 2019

Abs Bib PDF

Studies have demonstrated the potential of using error-related potentials (ErrPs), online decoded from the electroencephalogram (EEG) of a human observer, for robot skill learning and mediation of co-adaptation in collaborative human-robot interaction (HRI). While these studies provided proof-of-concept of this approach as a highly promising avenue in the field of HRI, a systematic understanding of the dyadic interacting system (human and machine) remained unexplored. This research aims to address this gap by proposing a computational model of the human counterpart and simulating the integrated dyadic system. The model can be employed for the systematic study of both human behavioral and technical factors influencing co-adaptation as exemplarily demonstrated in this paper for hypothetical variations of ErrP-decoder performance. The obtained findings have practical implications for future steps along this line of research, for instance to what extent and how improvements of ErrP-decoder performance can benefit co-adaptation in ErrP-based HRI. The proposed computational model enables the prediction of human behavior in the context of ErrP-based HRI. As such it allows the simulation of future empirical studies prior to their conductance and thereby providing a means for accelerating progress along this line of research in a resource-saving manner.
@inproceedings{ehrlich2019computational, title = {A computational model of human decision making and learning for assessment of co-adaptation in neuro-adaptive human-robot interaction}, author = {Ehrlich, Stefan K and Cheng, Gordon}, booktitle = {2019 IEEE International Conference on Systems, Man and Cybernetics (SMC)}, pages = {264--271}, year = {2019}, organization = {IEEE}, }
A comparative study on adaptive subject-independent classification models for zero-calibration error-potential decoding

Florian M Schönleitner, Lukas Otter, Stefan K Ehrlich, and 1 more author

In 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS), 2019

Abs Bib PDF

Today, a substantial part of human interaction is the engagement with artificial technological and information systems. Error-related potentials (ErrPs) provide an elegant method to improve such human-machine interaction by detecting incorrect system behaviour from the electroencephalography (EEG) signal of a human operator or user in real time. In this paper, we focus on adaptive subject-independent classification models particularly suitable for the task of ErrP decoding. As such, they provide a promising method to overcome the need of individualized decoding models, which require a time consuming calibration phase. In a comparative study we evaluate the performance of a decoding model solely trained on prior data and the effectiveness of adapting this model to a new subject. Our results show that such a generalized model can decode ErrPs with an acceptable accuracy of (72.73 +/- 5.27)% and that supervised adaptation can significantly improve the accuracy of the generalized model. Unsupervised adaptation did only prove useful for some subjects with high initial model accuracy and requires more sophisticated methods to be practical for a broader range of subjects. Consequently, our work contributes to the development of calibration-free ErrP decoding, which can potentially be used to improve human-robot interaction.
@inproceedings{schonleitner2019comparative, title = {A comparative study on adaptive subject-independent classification models for zero-calibration error-potential decoding}, author = {Sch{\"o}nleitner, Florian M and Otter, Lukas and Ehrlich, Stefan K and Cheng, Gordon}, booktitle = {2019 IEEE International Conference on Cyborg and Bionic Systems (CBS)}, pages = {85--90}, year = {2019}, organization = {IEEE}, }
A prototype of a P300 based brain-robot interface to enable multi-modal interaction for patients with limited mobility

Jonas F Braun, Germán Dı́ez-Valencia, Stefan K Ehrlich, and 2 more authors

In 2019 IEEE International Conference on Cyborg and Bionic Systems (CBS), 2019

Abs Bib PDF

Patients who lost their ability to move and talk are often socially deprived. To assist them, we present a prototype of a telepresence humanoid robotic system that aims to extend the social sphere and autonomy of the patients via an EEG based brain-computer interface. The system enables a multi-modal and bidirectional communication. It empowers the patient to interact with the robot and command it using a high level P300 BCI that interprets the patient’s answers to questions asked by the robot. Additionally, the system allows interaction with other people. By forwarding some of the robot’s sensations to the patient, the patient’s senses and action space are extended and a telepresence of the patient is created. A use-case validation of the system shows success in achieving bidirectional communication between an able-bodied test subject and the robotic system as well as in interactions with other people.
@inproceedings{braun2019prototype, title = {A prototype of a P300 based brain-robot interface to enable multi-modal interaction for patients with limited mobility}, author = {Braun, Jonas F and D{\'\i}ez-Valencia, Germ{\'a}n and Ehrlich, Stefan K and Lanillos, Pablo and Cheng, Gordon}, booktitle = {2019 IEEE International Conference on Cyborg and Bionic Systems (CBS)}, pages = {78--84}, year = {2019}, organization = {IEEE}, }
A feasibility study for validating robot actions using EEG-based error-related potentials

Stefan K Ehrlich and Gordon Cheng

International Journal of Social Robotics, 2019

Abs Bib PDF

Validating human-robot interaction can be a challenging task, especially in cases in which the robot designer is interested in the assessment of individual robot actions within an ongoing interaction that should not be interrupted by intermittent surveys. In this paper, we propose a neuro-based method for real-time quantitative assessment of robot actions. The method encompasses the decoding of error-related potentials (ErrPs) from the electroencephalogram (EEG) of a human during interaction with a robot, which could be a useful and intuitive complement to existing methods for validating human-robot interaction in the future. To demonstrate usability, we conducted a study in which we examined EEG-based ErrPs in response to a humanoid robot displaying semantically incorrect actions in a simplistic HRI task. Furthermore, we conducted a procedurally identical control experiment with computer screen-based symbolic cursor action. The results of our study confirmed decodeability of ErrPs in response to incorrect robot actions with an average accuracy of 69.0+/-7.9% across 11 subjects. Cross-comparisons of ErrPs between experimental tasks revealed high temporal and topographical similarity, but more distinct signals in response to the cursor action and, as a result, better decodeability with a mean accuracy of 90.6+/-3.9%. This demonstrated that ErrPs can be sensitive to the stimulus eliciting them despite procedurally identical protocols. Re-using ErrP-decoders across experimental tasks without re-calibration is accompanied by significant performance losses and therefore not recommended. Overall, the outcomes of our study confirm feasibility of ErrP-decoding for human-robot validation, but also highlight challenges to overcome in order to enhance usability of the proposed method.
@article{ehrlich2019feasibility, title = {A feasibility study for validating robot actions using EEG-based error-related potentials}, author = {Ehrlich, Stefan K and Cheng, Gordon}, journal = {International Journal of Social Robotics}, volume = {11}, number = {2}, pages = {271--283}, year = {2019}, publisher = {Springer}, }
A closed-loop, music-based brain-computer interface for emotion mediation

Stefan K Ehrlich, Kat R Agres, Cuntai Guan, and 1 more author

PloS one, 2019

Abs Bib PDF

Emotions play a critical role in rational and intelligent behavior; a better fundamental knowledge of them is indispensable for understanding higher order brain function. We propose a non-invasive brain-computer interface (BCI) system to feedback a person’s affective state such that a closed-loop interaction between the participant’s brain responses and the musical stimuli is established. We realized this concept technically in a functional prototype of an algorithm that generates continuous and controllable patterns of synthesized affective music in real-time, which is embedded within a BCI architecture. We evaluated our concept in two separate studies. In the first study, we tested the efficacy of our music algorithm by measuring subjective affective responses from 11 participants. In a second pilot study, the algorithm was embedded in a real-time BCI architecture to investigate affective closed-loop interactions in 5 participants. Preliminary results suggested that participants were able to intentionally modulate the musical feedback by self-inducing emotions (e.g., by recalling memories), suggesting that the system was able not only to capture the listener’s current affective state in real-time, but also potentially provide a tool for listeners to mediate their own emotions by interacting with music. The proposed concept offers a tool to study emotions in the loop, promising to cast a complementary light on emotion-related brain research, particularly in terms of clarifying the interactive, spatio-temporal dynamics underlying affective processing in the brain.
@article{ehrlich2019closed, title = {A closed-loop, music-based brain-computer interface for emotion mediation}, author = {Ehrlich, Stefan K and Agres, Kat R and Guan, Cuntai and Cheng, Gordon}, journal = {PloS one}, volume = {14}, number = {3}, pages = {e0213516}, year = {2019}, publisher = {Public Library of Science San Francisco, CA USA}, }

2018

Human-agent co-adaptation using error-related potentials

Stefan K Ehrlich and Gordon Cheng

Journal of neural engineering, 2018

Abs Bib PDF

Objective. Error-related potentials (ErrP) have been proposed as an intuitive feedback signal decoded from the ongoing electroencephalogram (EEG) of a human observer for improving human-robot interaction (HRI). While recent demonstrations of this approach have successfully studied the use of ErrPs as a teaching signal for robot skill learning, so far, no efforts have been made towards HRI scenarios where mutual adaptations between human and robot are expected or required. These are collaborative or social interactive scenarios without predefined dominancy of the human partner and robots being perceived as intentional agents. Here we explore the usability of ErrPs as a feedback signal from the human for mediating co-adaptation in human-robot interaction. Approach. We experimentally demonstrate ErrPs-based mediation of co-adaptation in a human-robot interaction study where successful interaction depended on co-adaptive convergence to a consensus between them. While subjects adapted to the robot by reflecting upon its behavior, the robot adapted its behavior based on ErrPs decoded online from the human partner’s ongoing EEG. Main results. ErrPs were decoded online in single trial with an avg. accuracy of 81.8%+/-8.0% across 13 subjects, which was sufficient for effective adaptation of robot behavior. Successful co-adaptation was demonstrated by significant improvements in human-robot interaction efficacy and efficiency, and by the robot behavior that emerged during co-adaptation. These results indicate the potential of ErrPs as a useful feedback signal for mediating co-adaptation in human-robot interaction as demonstrated in a practical example. Significance. As robots become more widely embedded in society, methods for aligning them to human expectations and conventions will become increasingly important in the future. In this quest, ErrPs may constitute a promising complementary feedback signal for guiding adaptations towards human preferences. In this paper we extended previous research to less constrained HRI scenarios where mutual adaptations between human and robot are expected or required.
@article{ehrlich2018human, title = {Human-agent co-adaptation using error-related potentials}, author = {Ehrlich, Stefan K and Cheng, Gordon}, journal = {Journal of neural engineering}, volume = {15}, number = {6}, pages = {066014}, year = {2018}, publisher = {IOP Publishing}, }

2017

A closed-loop brain-computer music interface for continuous affective interaction

Stefan K Ehrlich, Cuntai Guan, and Gordon Cheng

In 2017 international conference on orange technologies (ICOT), 2017

Abs Bib PDF

Research on human emotions and underlying brain processes is mostly performed open-loop, e.g. by presenting emotional stimuli and measuring subject’s brain responses. Investigating human emotions in interaction with emotional stimuli (closed-loop) significantly complicates experimental setups and has so far rarely been proposed. We present concept and technical realization of an electroencephalography (EEG)-based affective Brain-Computer Interface (BCI) to study emotional brain processes in continuous closed-loop interaction. Our BCI consists of an algorithm generating continuous patterns of synthesized affective music, embedded in an online BCI architecture. An initial calibration is employed to obtain user-specific models associating EEG patterns with affective content in musical patterns. These models are then used in online application to translate the user’s affect into a continuous musical representation; playback to the user results in closed-loop affective brain-interactions. The proposed BCI provides a platform to stimulate the brain in a closed-loop fashion, offering novel approaches to study human sensorimotor integration and emotions.
@inproceedings{ehrlich2017closed, title = {A closed-loop brain-computer music interface for continuous affective interaction}, author = {Ehrlich, Stefan K and Guan, Cuntai and Cheng, Gordon}, booktitle = {2017 international conference on orange technologies (ICOT)}, pages = {176--179}, year = {2017}, organization = {IEEE}, }
Effects of short-term piano training on measures of finger tapping, somatosensory perception and motor-related brain activity in patients with cerebral palsy

Ana Alves-Pinto, Stefan K Ehrlich, Gordon Cheng, and 3 more authors

Neuropsychiatric disease and treatment, 2017

Abs Bib PDF

Playing a musical instrument demands the integration of sensory and perceptual information with motor processes in order to produce a harmonic musical piece. The diversity of brain mechanisms involved and the joyful character of playing an instrument make musical instrument training a potential vehicle for neurorehabilitation of motor skills in patients with cerebral palsy (CP). This clinical condition is characterized by motor impairments that can affect, among others, manual function, and limit severely the execution of basic daily activities. In this study, adolescents and adult patients with CP, as well as a group of typically developing children learned to play piano for 4 consecutive weeks, having completed a total of 8 hours of training. For ten of the participants, learning was supported by a special technical system aimed at helping people with sensorimotor deficits to better discriminate fingers and orient themselves along the piano keyboard. Potential effects of piano training were assessed with tests of finger tapping at the piano and tests of perception of vibratory stimulation of fingers, and by measuring neuronal correlates of motor learning in the absence of and after piano training. Results were highly variable especially among participants with CP. Nevertheless, a significant effect of training on the ability to perceive the localization of vibrations over fingers was found. No effects of training on the performance of simple finger tapping sequences at the piano or on motor-associated brain responses were registered. Longer periods of training are likely required to produce detectable changes.
@article{alves2017effects, title = {Effects of short-term piano training on measures of finger tapping, somatosensory perception and motor-related brain activity in patients with cerebral palsy}, author = {Alves-Pinto, Ana and Ehrlich, Stefan K and Cheng, Gordon and Turova, Varvara and Blumenstein, Tobias and Lampe, Ren{\'e}e}, journal = {Neuropsychiatric disease and treatment}, pages = {2705--2718}, year = {2017}, publisher = {Taylor \& Francis}, }
A simple and practical sensorimotor EEG device for recording in patients with special needs

Stefan K Ehrlich, Ana Alves-Pinto, Renée Lampe, and 1 more author

In Neurotechnix2017, CogNeuroEng 2017, Symposium on Cognitive Neural Engineering, 2017

Abs Bib PDF

In studies involving patients with special needs, the use of electroencephalography (EEG) recordings is among the most delicate measurement modalities. The quietness needed and the long preparation time can be challenging especially in young ages. Furthermore, the invasive appearance of the instrumentation involved is not appealing and can raise distrust in patients. We developed a customized EEG device which adresses these issues by merging commercially available EEG hardware with an unobtrusive headphones design. The resulting device has very short preparation times, non-clinical appearance, and delivers adequate data quality with respect to recording of sensorimotor rhythms. Our device was employed in a study investigating sensorimotor-related brain activity in adolescents and adults with cerebral palsy (CP) conducted at a day-care center. Experimenters reported convenient data collection and overall acceptance of the system among patients. The changes in sensorimotor rhythms over time during a hand motor task meet the observations described in the literature, supporting the functionality of our EEG device for the assessment of sensorimotor-related measures of brain activity in patients with sensorimotor disorders of neuronal origin.
@inproceedings{ehrlich2017simple, title = {A simple and practical sensorimotor EEG device for recording in patients with special needs}, author = {Ehrlich, Stefan K and Alves-Pinto, Ana and Lampe, Ren{\'e}e and Cheng, Gordon}, booktitle = {Neurotechnix2017, CogNeuroEng 2017, Symposium on Cognitive Neural Engineering}, year = {2017}, }

2016

A neuro-based method for detecting context-dependent erroneous robot action

Stefan K Ehrlich and Gordon Cheng

In 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), 2016

Abs Bib PDF

Validating appropriateness and naturalness of human-robot interaction (HRI) is commonly performed by taking subjective measures from human interaction partners, e.g. questionnaire ratings. Although these measures can be of high value for robot designers, they are very sensitive and can be inaccurate and/or biased. In this paper we propose and validate a neuro-based method for objectively validating robot behavior in HRI. We propose to detect from the electronencephalogram (EEG) of a human interaction partner, the perception of inappropriate / unexpected / erroneous robot behavior. To validate this method, we conducted an EEG experiment with a simplified HRI protocol in which a humanoid robot displayed context-dependent erroneous behavior from time to time. The EEG data taken from 13 participants revealed biologically plausible error-related potentials (ErrP) whose spatio-temporal distributions match well with related neuroscientific research. We further demonstrate that perceived erroneous robot action can reliably be modeled and detected from human EEG signals with classification accuracies on avg. 69.7+/-9.1%. These findings confirm principal feasibility of the proposed method and suggest that EEG-based ErrP detection can be used for quantitative evaluation and thus improvement of robot behavior.
@inproceedings{ehrlich2016neuro, title = {A neuro-based method for detecting context-dependent erroneous robot action}, author = {Ehrlich, Stefan K and Cheng, Gordon}, booktitle = {2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids)}, pages = {477--482}, year = {2016}, organization = {IEEE}, }

2015

Augmenting affect from speech with generative music

Gerhard Johann Hagerer, Michael Lux, Stefan K Ehrlich, and 1 more author

In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems, 2015

Abs Bib PDF

In this work we propose a prototype to improve interpersonal communication of emotions. Therefore music is generated with the same affect as when humans talk on the fly. Emotions in speech are detected and conveyed to music according to music psychological rules. Existing evaluated modules from affective generative music and speech emotion detection, use cases, emotional models and projected evaluations are discussed.
@inproceedings{hagerer2015augmenting, title = {Augmenting affect from speech with generative music}, author = {Hagerer, Gerhard Johann and Lux, Michael and Ehrlich, Stefan K and Cheng, Gordon}, booktitle = {Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems}, pages = {977--982}, year = {2015}, }

2014

When to engage in interaction-And how? EEG-based enhancement of robot’s ability to sense social signals in HRI

Stefan K Ehrlich, Agnieszka Wykowska, Karinne Ramirez-Amaro, and 1 more author

In 2014 IEEE-RAS International Conference on Humanoid Robots, 2014

Abs Bib PDF

Humanoids are to date still limited in reliable interpretation of social cues that humans convey which restricts fluency and naturalness in social human-robot interaction (HRI). We propose a method to read out two important aspects of social engagement directly from the brain of a human interaction partner: (1) the intention to initiate eye contact and (2) the distinction between the observer being initiator or responder of an established gaze contact between human and robot. We suggest that these measures would give humanoids an important means for deciding when (timing) and how (social role) to engage in interaction with a human. We propose an experimental setup using iCub to evoke and capture the respective electrophysiological patterns via electroencephalography (EEG). Data analysis revealed biologically plausible brain activity patterns for both processes of social engagement. By using Support Vector Machine (SVM) classifiers with RBF kernel we showed that these patterns can be modeled with high within-participant accuracies of avg. 80.4% for (1) and avg. 77.0% for (2).
@inproceedings{ehrlich2014engage, title = {When to engage in interaction-And how? EEG-based enhancement of robot's ability to sense social signals in HRI}, author = {Ehrlich, Stefan K and Wykowska, Agnieszka and Ramirez-Amaro, Karinne and Cheng, Gordon}, booktitle = {2014 IEEE-RAS International Conference on Humanoid Robots}, pages = {1104--1109}, year = {2014}, organization = {IEEE}, }