EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyography

NeurIPS 2024

Jehan Yang*, Maxwell Soh*, Vivianna Lieu, Douglas J. Weber, Zackory Erickson
Carnegie Mellon University
*Equal Contribution
arXiv Code FlexWear-HD Dataset

Abstract

This paper introduces the first generalization and adaptation benchmark using machine learning for evaluating out-of-distribution performance of electromyography (EMG) classification algorithms. The ability of an EMG classifier to handle inputs drawn from a different distribution than the training distribution is critical for real-world deployment as a control interface. By predicting the user’s intended gesture using EMG signals, we can create a wearable solution to control assistive technologies, such as computers, prosthetics, and mobile manipulator robots. This new out-of-distribution benchmark consists of two major tasks that have utility for building robust and adaptable control interfaces: 1) intersubject classification, and 2) adaptation using train-test splits for time-series. This benchmark spans nine datasets, the largest collection of EMG datasets in a benchmark. Among these, a new dataset is introduced, featuring a novel, easy-to-wear high-density EMG wearable for data collection. The lack of open-source benchmarks has made comparing accuracy results between papers challenging for the EMG research community. This new benchmark provides researchers with a valuable resource for analyzing practical measures of out-of-distribution performance for EMG datasets. Our code and data from our new dataset can be found at emgbench.github.io.

Out-of-Distribution Classification


This benchmark is used for practical out-of-distribution applications for learning-based EMG control interfaces. We present two types of out-of-distribution tasks for machine learning: A) intersubject classification, and B) adaptation using train-test splits for time-series.

Distribution shifts


Distribution shifts occur over multiple sessions and samples, and the model should generalize or adapt to these changes. This benchmark evaluates the model's ability to generalize or adapt to distribution shifts in the data.

BibTeX

@misc{yang2024emgbenchbenchmarkingoutofdistributiongeneralization,
      title={EMGBench: Benchmarking Out-of-Distribution Generalization and Adaptation for Electromyography}, 
      author={Jehan Yang and Maxwell Soh and Vivianna Lieu and Douglas J Weber and Zackory Erickson},
      year={2024},
      eprint={2410.23625},
      archivePrefix={arXiv},
      primaryClass={cs.LG},
      url={https://arxiv.org/abs/2410.23625}, 
}