Electromagnetic articulography (EMA) is one of the instrumental phonetic research methods used for recording and assessing articulatory movements. Usually, articulographic data are analysed together with standard audio recordings. This paper, however, demonstrates how coupling the articulograph with devices providing other types of information may be used in more advanced speech research. A novel measurement system is presented that consists of the AG 500 electromagnetic articulograph, a 16-channel microphone array with a dedicated audio recorder and a video module consisting of 3 high-speed cameras. It is argued that synchronization of all these devices allows for comparative analyses of results obtained with the three components. To complement the description of the system, the article presents innovative data analysis techniques developed by the authors as well as preliminary results of the system’s accuracy.
Electromagnetic Articulography (EMA) is a technology created over two decades ago. EMA enables to acquire spatiotemporal data from sensors placed on the tongue in order to obtain information about the positioning of the tongue, its shape and dynamics during vocalizations of various sounds of human speech. The articulograph is often supported by an audio recorder and a vision system. In this paper, a novel system integrating EMA, audio and visual data recording is presented. The articulatory data was obtained with a Carsten's AG500 articulograph. The vision system was constructed from 3 high-speed cameras (Gazelle GZL-CL-22C5M-C) manufactured by Point Grey. The cameras registered movements of markers attached to the face of the speakers. The audio recorder consisted of a 16-channel microphone array and an electronic device that registered and processed signals from the microphones. The microphone array made it possible to map sources of sound propagation on the speaker’s face. The simultaneous recording of signals from EMA, the video system and the audio recorder is controlled from a computer with a host program and is supported by a synchronizer. The electromagnetic articulograph registers signals from EMA sensors which return their spatiotemporal positions with the sampling frequency of 200 Hz. The readouts of the spatial positioning of sensors attached to the tongue provide information about its shape and movements in time. There are three cameras registering the movements of external articulators and organs (e.g. lips, jaw and cheeks) from the front and side views. The cameras register movies with the frame rate of 200 FPS. The circular microphone array with 16 microphones records 16-channel audio with 96 kHz sampling rate and 16 bit resolution. During the recording sessions, the participants read aloud words that are displayed on the screen. An application on the host computer sends commands to AG500 which in turn generates synchronization signals in the TTL standard to external devices. These signals are used for activating the audio recorder and the synchronizer in the video system. Articulographic and simple acoustic analysis is performed with created in MATLAB software called phonEMAtool. This software is very useful and ergonomic for fast feature extraction of tongue movements during speech. The application allows to display simultaneously: speech waveform, EMA sensors position and orientation, phonetic segmentation. Before an analysis, the data from AG500 are pre-processed twice with a Savitzky-Golay filter so as to remove undesirable noise. In the paper an exemplary analysis performed by the phonEMAtool of particular articulatory gestures in the articulation of [m] in the Polish word Tamara is presented. Another analysis is beamforming of audio signals in order to obtain three-dimensional acoustic field distribution images. In the paper an example of this technique applied to the analysis of the nasal consonant in the word Tamara [tamara] has been shown. Analysis indicated that the highest intensity of the acoustic field during the pronunciation of the consonant [m] occurs in the nose region and for vowel [a], the highest intensity is observed in the mouth. Due to movement registration of facial markers the reconstruction of positions of external articulators can be obtained. With additional face triangulation using Delaunay algorithm some differences between positions of external articulators can be easily tracked. The measurement system described in this paper is effective and allows for an examination of the vocal tract in 3 ways: tongue movements, acoustic field intensity distribution and external articulator movements. A particularly useful tool is the dedicated acoustic camera based on multi-channel audio recorder and a microphone array. The results obtained with this equipment are unique and show great research and application potential.
Background: Children with difficulties in listening and understanding speech despite normal peripheral hearing, can be diagnosed with the diagnosis Auditory Processing Disorder (A). However, there are doubts about the validity of this diagnosis. The aim of this study was to examine the relation between the listening difficulties of children between 8 and 12 years with suspected A and the attention, working memory, nonverbal intelligence and communication abilities of these children.Material and methods: In this case-control study we examined 10 children who reported listening difficulties in spite of normal peripheral hearing (3 referred by speech-language pathologist in the Northern Netherlands, 6 by an audiological center in the Southern Netherlands and one by parental concern) and 21 typically developing children (recruitment through word of mouth and by the website Taalexpert.nl), ages 8;0 to 12;0 years. The parents of all children completed three questionnaires about history, behavioral symptoms of ADHD, and communication skills (Children’s Communication Checklist). Teachers of the children completed the Children’s Auditory Processing Performance Scale (CHAPPS). Children were assessed for auditory processing abilities (speech-in-noise, filtered speech, binaural fusion, dichotic listening), nonverbal intelligence (Raven’s Coloured Progressive Matrices), and working memory (Clinical Evaluation of Language Fundamentals). Auditory and visual attention was studied with four behavioral tests of the WAFF battery of the Vienna Test System (Schuhfried).Results: Preliminary analysis shows no differences between groups on the auditory processing tests and nonverbal intelligence quotient. Children in the experimental group have poorer communication performance (parent report), poorer listening skills (teacher report), and poorer working memory and attention skills (behavioral tests).Conclusions: The results of this study showed that there is a difference between children with listening complaints and typically developing children, but that the problems are not specific to the auditory modality. There seems to be no evidence for the validity of an auditory deficit.
