We conduct a tightly integrated computational and experimental research program across two sites (BU, MPIEA) to study spoken language recognition from the psychophysical, neurophysiological, and engineering perspectives.

The program proceeds in three fronts:

(1) Psychophysics (Ghitza, BU). We measure and model the results of human performance in tasks designed to gain a better understanding on the interplay between neuronal oscillators in different frequency bands, and between the oscillations and the speech syllabic structure.

(2) Human Neuroimaging (Poeppel, Max Planck Institute). We formulate the intra-relationship among delta, theta, beta and gamma oscillations, using MEG data recorded while subjects perform intelligibility tasks.

(3) Automatic Speech Recognition (Ghitza, BU). We explore a new perspective to the development of ASR systems that incorporates the insights from the behavioral and brain sciences, specifically rhythmic brain activity. We ascertain whether the proposed cortical computation principle could be used as an adjunct to conventional features used in ASR systems, e.g. in lattice re-scoring of n-best lists - and ultimately result in a decrease in word error rate.