Project Detail
Interagency Autism Coordinating Committee (IACC) logo
Office of Autism Research Coordination (OARC) logo

HCC: Medium: Automatic detection of atypical patterns in cross-modal affect  

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). The expression of affect in face-to-face situations requires the ability to generate a complex, coordinated, cross-modal affective signal, having gesture, facial expression, vocal prosody, and language content modalities. This ability is compromised in neurological disorders such as Parkinson's disease and autism spectrum disorder (ASD). The PI's long term goal is to build computer-based interactive, agent based systems for remediation of poor affect communication and diagnosis of the underlying neurological disorders based on analysis of affective signals. A requirement for such systems is technology to detect atypical patterns in affective signals. The objective of this project is to develop that technology. Toward that end the PI will develop a play situation for eliciting affect, will collect audio-visual data from approximately 60 children between the ages of 4-7 years old, half of them with ASD and the other half constituting a control group of typically developing children. The PI will label the data on relevant affective dimensions, will develop algorithms for the analysis of affective incongruity, and will then test the algorithms against the labeled data in order to determine their ability to differentiate between ASD and typical development. While automatic methods for cross-modal recognition of discrete affect classes already have yielded promising results, automatic detection and quantification of atypical patterns in affective signals, and the ability to do so in semi-natural interactive situations, is unexplored territory. The PI expects this research will lead to new methods for affect recognition based on facial affective features (with special emphasis on facial frontalization algorithms and on modeling of facial expressive dynamics), vocal affective features, and lexical affective features, as well as to new methods for automated measurement of cross-modal affective incongruity. Broader Impacts: The expression of affect in special populations is a largely neglected area in affective computing and robotics; yet, these populations may be among the most important beneficiaries of these technologies. Affective expression impairments afflict many individuals, including those with neuro-developmental disorders such as autism and those with neuro-degenerative disorders such as Parkinson's disease. Because these impairments concern a core aspect of human communication and, hence, may cause profound social isolation in these individuals, intervention is highly desirable. However, one-on-one intervention by therapists, if effective, would be available only to relatively few individuals, thereby making computer-based intervention critical for broader access to such treatment. Accurate processing of the affective signal will be of use as a research and diagnostic tool for a range of neurological disorders. The CSLU research team will continue its tradition of disseminating research findings and technology, including speech corpora and software, to the research community. Project Status


Funder National Science Foundation
Fiscal Year Funding $410,666.67
Current Award Period 2009-2012
Project Number 905095
Principal Investigator Van Santen, Jan
Received ARRA Funding? Yes
Strategic Plan Question Question 1: When Should I Be Concerned? (Diagnosis)
Subcategory Symptomology
Strategic Plan Objective Green dot: Objective has greater than or equal to the recommended funding. 1LB. Develop at least five measures of behavioral and/or biological heterogeneity in children or adults with ASD, beyond variation in intellectual disability, that clearly relate to etiology and risk, treatment response and/or outcome by 2015. IACC Recommended Budget: $71,100,000 over 5 years.
Federal or Private? Federal
Institution Oregon Health & Science University
State/Country Oregon
Web Link 1 HCC: Medium: Automatic detection of atypical patterns in cross-modal affect (External web link)
Web Link 2 HCC: Medium: Automatic detection of atypical patterns in cross-modal affect (External web link)
Web Link 3 HCC: Medium: Automatic detection of atypical patterns in cross-modal affect (External web link)
History/Related Projects HCC: Medium: Automatic detection of atypical patterns in cross-modal affect | $0.00 | 2011 | 0905095
HCC: Medium: Automatic detection of atypical patterns in cross-modal affect | $0.00 | 2012 | 0905095