There is an urgent need for direct, continuous, and objective measures of social behavior in individuals with Autism Spectrum Disorder (ASD) that are sensitive to treatment-related change. Tracking progress by measuring changes in behavior is crucial to determine whether the treatment is working or needs to be modified. Direct observational measures, such as live or video-based scoring, are time and resource intensive, and all but impossible to implement in clinics and homes, where treatments are increasingly delivered. There are currently no standardized, objective, and reliable measures of key social behaviors that can be used in clinical settings to assess treatment outcomes in individuals with ASD. The goal of this project is to develop visual analytics tools to enable the automatic detection and quantification of social behaviors of interest from video captured by a wearable camera, a task this I-Corps team refers to as First Person Vision (FPV). In previous work, this team developed a method for automatically detecting moments when a child makes eye contact with an adult social partner by analyzing video recorded by a pair of commercially available glasses worn by the adult which had a camera embedded in the bridge over the nose. By identifying and localizing the child's head and face in the video and analyzing the image regions containing the eyes in conjunction with the head pose, the proposed algorithm can determine whether the child's gaze is directed towards the adult's eyes. The proposed approach can generate predictions for eye contact at the frame level, and can also be used to detect eye contact events from which measures of duration can be derived.
If properly developed, the proposed measurement approach may enable clinical providers to scale their intervention efforts, with the potential of improving the quality of life for thousands of children with autism and their families. More broadly, analytics tools for video captured from body-worn cameras (first person video) will provide new opportunities to model and analyze human behavior, create personalized records of visual experiences, and improve the treatment of a broad range of mental and physical health conditions. This team will conduct additional customer discovery in the areas of human resources training and product marketing, with the potential of expanding the focus of our analytics from gaze behavior to other social behaviors relevant to these contexts, such as facial expressions and gestures. The team also expects to develop a prototype of the hardware (wearable camera system with wide field of view) and incorporate analytic capabilities developed by our group (algorithms for automated detection of eye contact, attention to objects, gaze shifts) into the system. the proposed prototype will also include options for visualizing and reporting the results of the automated analysis back to the user.
Interagency Autism Coordinating Committee (IACC)
Autism Research Database (AFD)
