This project is an interdisciplinary, integrated research and education program to develop novel technologies in manipulating mirror images, aimed at studying and enabling behavioral modeling of children with autism spectrum disorder (ASD). Central to the research is a "virtual-mirror" device that combines a network of calibrated depth and visual sensors to render a viewpoint-dependent dynamic view of an arbitrary-shaped virtual mirror on a room-size see-through display. Through multimodal and spatially-diverse sensors, the proposed system provides high-fidelity, non-intrusive capturing of eye gaze, facial expression, body pose, body movement, and other human behavioral patterns. New multimedia processing algorithms will be developed for transferring 2D and 3D physical appearances, as well as behaviors from a source individual to a target individual with limited target training data to be rendered on regular displays and the virtual mirror. Children with ASD typically lack interest in social interactions, but appear to be highly interested in their own image in mirrors and others imitating their actions. The software and hardware systems developed in this project can provide unprecedented capability in creating novel behaviors of self in both traditional visual medium and immersive devices. The system is expected to provide greater flexibility to therapists, teachers, and caretakers in creating material for video modeling and self-modeling therapy. Deployment over the web lowers the technology barrier, increases access to evidence-based treatments, and promotes consistent practice of behavioral modeling in home and beyond clinics. By combining visual feedback and real-time rendering of new behaviors, the virtual mirror is expected to deliver more effective behavioral modeling for children with ASD. Success will indicate that the self/other system in ASD can be modified, and suggest that a fundamental deficit in ASD is subject to environmental manipulation. The proposed research program is a collaborative effort of PIs from electrical engineering, psychology, medicine, and education. The educational objective is to develop a research program in which problem identification, data sharing, and problem solving occur collaboratively from the very beginning. Outreach programs including device demonstrations at pediatric clinics, television documentaries of research results, and visits to high schools in rural districts are planned to promote the use of technologies in solving important health and societal problems, as well as to broaden the participation of under-represented groups in STEM activities. Information about this project, including but not limited to the description, personnel, acknowledgements, latest results, and publications, will be posted at http://www.vis.uky.edu/mialab/NSF_autism.
Interagency Autism Coordinating Committee (IACC)
Autism Research Database (AFD)
