Ultrasound-based Recognition of Finger Gestures using Spiking Neural Networks equipped with Spike-Timing-Dependent Plasticity

Abstract

In recent years, researchers constantly attempt to derive Ultrasound-based (US-based) hand gesture recognition (HGR) solutions suitable for edge applications. This process involves improving several design aspects of US-based HGR systems such as the transducers, the wearable US acquisition systems and the algorithms employed in terms of energy consumption, computational complexity and robustness. The subject of this paper is the latter. In this paper, we present a spiking framework for US-based HGR. The proposed approach leverages a single-layer Spiking Neural Network (SNN) equipped with Spike-Timing-Dependent Plasticity (STDP) as a feature descriptor for Rate-based (RB) coded A-line US signals coupled with a lightweight linear support vector machine (SVM) classifier. According to our findings, our proposed approach achieves performance comparable to that of the state-of-the-art in the ultrasound-based adaptive prosthetic control (Ultra-Pro) dataset. Furthermore, we demonstrate that our feature descriptor exhibits inter-session generalization capabilities, i.e. re-training is not required between within-day sessions and thus reduces the burden of periodic extensive data collection from the user.