Use of machine learning for the automated segmentation and detection of swallows via digital cervical auscultation in children

Abstract

Introduction Cervical auscultation (CA) is the most commonly used adjuvant to the clinical feeding evaluation worldwide. Yet, the translation of digital CA into clinical practice is limited by the time-consuming manual segmentation of swallow sounds by trained experts required for analyses. Automated segmentation of swallow sounds via machine learning has the potential to reduce the time required for manual segmentation of swallow sounds and increase acceptability and availability of digital CA in clinical practice. We aimed to evaluate the use of a machine learning technique to automatically extract and segment swallowing sounds from digital CA recordings of thin fluid swallows collected from infants and children with paediatric feeding disorders assessed during a videofluroscopic swallow study (training set, n = 19, 275 swallows) and clinical feeding evaluation (validation set, n = 15, 246 swallows).Material and Methods Annotation of swallowing segments from audio files of 1 min duration were completed by a CA trained speech pathologist. The digital audio was downsampled to 16 kHz and segmented into frames of 0.96 s with a 50% overlap using appropriate signal processing methods. For the machine learning stage, a transfer learning approach was adopted, where a pre-trained deep neural network was used as the initial model, and further training is performed on the labelled data of interest. Training was performed by applying each frame of audio data into the pre-trained model and computing the output of the top-most convolutional layer, known as the embedding. These embeddings were then fed into a feedforward neural network and training was performed using the backpropagation method. Class imbalance was addressed by randomly sampling the embeddings such that there were equal proportions of "swallow“ and "non-swallow“ classes.Results From our preliminary experiments, our transfer learning model was able to achieve an accuracy of 96% on the seen training data, with a precision and recall of 0.77 and 1, respectively for the "swallow“ class and 1 and 0.96 for the "non-swallow“ class. On the unseen test data, an accuracy of 78% was achieved, with a precision and recall of 0.42 and 0.52, respectively for the "swallow“ class, and 0.89 and 0.84 for the "non-swallow“ class. Discussion/Conclusion Based on our preliminary results, our transfer learning approach shows promise in automatically segmenting digital audio recordings to identify the location of swallowing sounds. It is expected that further investigation and tuning will result in further improvements in accuracy.