Federal Autism Activities: Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication, GAO-14-16, November 20, 2013

Comments from the Department of Defense
Comments from the Department of Education
Comments from the Department of Health and Human Services
Comments from the National Science Foundation
Full PDF (PDF – 3.7 MB)

Comments from the Department of Defense

The Assistant Secretary of Defense PDF version (PDF – 3.7 MB)
Health Affairs
1200 Defense Pentagon
Washington, DC 20301-1200

October 4, 2013:

Ms. Marcia Crosse
Director, Health Care
U.S. Government Accountability Office
441 G Street, NW
Washington, DC 20548

Dear Ms. Crosse:

This letter is the Department of Defense's (DoD) response to the Government Accountability Office (GAO) Draft Report, GAO-14-16, "Federal Autism Activities: Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication," dated September 11, 2013 (GAO Code 291093). Thank you for the opportunity to review and comment on the draft report. After careful review, we concur with the draft report findings and conclusion. The DoD will participate and cooperate with the Interagency Autism Coordinating Committee (IACC), the National Institutes of Health, and other federal agencies in efforts to prevent unnecessary duplication of federal autism research activities. Such efforts will include a determination of methods for identifying and monitoring the autism research conducted by other agencies, including taking full advantage of monitoring data the IACC develops and makes available.

My points of contact on this matter are Dr. Terry Rauch (Functional) and Mr. Gunther Zimmerman (Audit Liaison). Dr. Rauch may be reached at (703) 681-8390, or Terry.Rauch@ha.osd.mil. Mr. Zimmerman may be reached at (703) 681-4360, or Gunther.Zimmerman@dha.mil.

Signed by:

Jonathan Woodson, M.D.

GAO Draft Report Dated September 11, 2013:
GAO-14-16 (GAO Code 291093):

"Federal Autism Activities: Better Data And More Coordination Needed To Help Avoid The Potential For Unnecessary Duplication"

Department Of Defense Comments To The GAO Recommendation

Recommendation 1: To improve the usefulness of the Interagency Autism Coordinating Committee (IACC) data and enhance its efforts to coordinate Health and Human Services (HHS) autism activities and monitor all federally funded autism activities, we recommend that the Secretary of HHS direct the IACC and National Institutes of Health (NIH), in support of IACC, to take the following three actions:

Provide consistent guidance to federal agencies when collecting data for the portfolio analysis and web tool so that information can be more easily and accurately compared over multiple years;
Create a document or database that provides information on non-research autism-related activities funded by the federal government and make this publicly available; and;
Through its monitoring of federal autism activities--including The Office of Autism Research Coordination's annual collection of data for the portfolio analysis and the IACC's annual process to update the strategic plan-identify projects that may result in unnecessary duplication and thus may be candidates for consolidation or elimination, and identify potential coordination opportunities among agencies.

DoD Response: Overall, the Department concurs with the draft report findings and conclusion and has no additional comments at this time.

Recommendation 2: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, we recommend that the Secretary of HHS, the Secretary of Defense, the Secretary of Education, and the Director of National Science Foundation each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

DoD Response: The DoD will participate and cooperate with IACC, NIH, and other federal agencies in efforts to prevent unnecessary duplication of federal autism research activities. Such efforts will include a determination of methods for identifying and monitoring the autism research conducted by other agencies, including taking full advantage of monitoring data the IACC develops and makes available.

[End of section]

Note: (HTML text files accessed December 2, 2013)

Comments from the Department of Education

United States Department of Education PDF version (PDF – 3.7 MB)
Office of Special Education and Rehabilitative Services
400 Maryland Ave. S.W.
Washington, DC 20202-2500
[hyperlink, http://www.ed.gov]

October 21, 2013:

Ms. Marcia Crosse
Director
Health Care
U.S. Government Accountability Office
441 G Street, NW
Washington, DC 20548

Dear Ms. Crosse:

I am writing to provide the U.S. Government Accountability Office (GAO) with comments from the U.S. Department of Education (Department) on the draft report, "Federal Autism Activities: Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication" (GAO-14-16). The Department appreciates the work that went into this study. The draft report contains two recommendations.

Recommendation 1: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, we recommend that the Secretary of Health and Human Services direct the IACC and NIH, in support of the IACC, to take the following three actions:

provide consistent guidance to federal agencies when collecting data for the portfolio analysis and web tool so that information can be more easily and accurately compared over multiple years;
create a document or database that provides information on non-research autism-related activities funded by the federal government and make this publicly available; and;
through its monitoring of federal autism activities--including OARC's annual collection of data for the portfolio analysis and the IACC's annual process to update the strategic plan--identify projects that may result in unnecessary duplication and thus may be candidates for consolidation or elimination, and identify potential coordination opportunities among agencies.

Response: This three-part recommendation mentions the Interagency Autism Coordinating Committee (IACC), the Department of Health and Human Services (HHS), the National Institutes of Health (NIH), and the Office of Autism Research Coordination (OARC). While it does not mention Education directly, we intend to work with the OARC to ensure the most accurate reporting possible. Our technical comments provide examples of the ways in which we have worked with the OARC on coding issues in the past, and how we hope to continue to work in the future. The Department has recently shared information on non-research autism activities with the OARC for the HHS Report to Congress on Activities Related to Autism Spectrum Disorders and Other Developmental Disabilities Under the Combating Autism Act of 2006 (FY 2010-2012), and we will continue to provide relevant information about Department programs for similar efforts to monitor and report autism non-research activities.

Recommendation 2: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, we recommend that the Secretary of Health and Human Services, the Secretary of Defense, and the Secretary of Education, and the Director of NSF each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of the monitoring data the IACC develops and makes available.

Response: The Department would be pleased to participate in development of more refined methods for the identification and monitoring of multiagency autism research, as described in GAO's second recommendation. I am the Department's representative on the IACC and will continue to work with the OARC and the IACC to provide them with information for Congress and the public.

The Department is concerned about, and does not concur with, GAO's conclusion that "the majority of federally-funded autism research was potentially duplicative..." The IACC objectives are broad and projects classified under them cannot be fairly judged "potentially duplicative" without more substantial exposition. Research projects with similar descriptors or titles may have different subject populations, sample sizes, methodologies, and outcome measures, and may explore different mechanisms or hypotheses. We note also that growth of the scientific knowledge base for any subject, and certainly for a subject as complex as autism, is dependent upon multiple studies investigating similar research questions.

Thank you for the opportunity to comment on this draft report. We also are including technical comments with this response.

Sincerely,

Signed by:

Michael K. Yudin
Acting Assistant Secretary for Special Education and Rehabilitative Services

[End of section]

Note: (HTML text files accessed December 2, 2013)

Comments from the Department of Health and Human

Department of Health & Human Services PDF version (PDF – 3.7 MB)
Office of The Secretary
Assistant Secretary for Legislation
Washington, DC 20201

October 24, 2013:

Marcia Crosse
Director, Health Care
U.S. Government Accountability Office
441 G Street NW
Washington, DC 20548

Dear Ms. Crosse,

Attached are comments on the U.S. Government Accountability Office's (GAO) report entitled, "Federal Autism Activities: Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication" (GAO-14-16).

The Department appreciates the opportunity to review this report prior to publication.

Sincerely,

Signed by:

Jim R. Esquea
Assistant Secretary for Legislation

Attachment:

General Comments Of The Department Of Health And Human Services (HHS) On The Government Accountability Office's (GAO) Draft Report Entitled, "Federal Autism Activities: Better Data And More Coordination Needed To Help Avoid The Potential For Unnecessary Duplication" (GAO-14-16):

The Department appreciates the opportunity to review and comment on this draft report.

HHS recognizes and supports the essential function and critical importance of coordination among research investments funded by Federal agencies. However, we are concerned that this report may provide a risk of misinforming policy makers and the public. This report indicates that it did not assess the research projects for actual duplication, because such an effort would be outside of the scope of the report, according to the report.

Although this report acknowledges that duplication is necessary in science for the sake of replicating or corroborating results, it does not appreciate the full extent of the necessity of replication and the extensive policies in place at HHS and other federal agencies to prevent redundant projects. HHS recognizes that scientific endeavors and the path of research discovery are not linear undertakings and often require verification and validation efforts. In addition, the advancement of scientific knowledge builds upon the cumulative efforts of what may seemingly be similar research undertakings on the surface, but in effect take what has been learned from previous research findings to another discovery frontier via the application of that knowledge.

HHS is concerned about the report's implication that it is wasteful when more than one funding Agency addresses an objective or aim of the Strategic Plan for Autism Research. It must be recognized that the goals and objectives of the Strategic Plan represent complex scientific questions that require a multidisciplinary approach, with multiple scientific strategies. For example, to develop effective interventions for autism spectrum disorder (ASD) that will address the full range of symptoms and degrees of disability found in the ASD population, research studies on multiple intervention types, such as behavioral, pharmacological, educational, and occupational, may need to be undertaken simultaneously to facilitate rapid progress that benefits individuals with varying needs. Based on the urgent need to address rapidly the health and services issues that are the most pressing in the community, it is not only appropriate, it is critical that multiple agencies address the complex questions related to understanding the neurobiology of ASD and identifying efficacious intervention strategies for use across the lifespan.

HHS is supportive of and committed to the call for greater coordination among federal research funding agencies and actively engages in efforts to minimize risk of research duplication in all activities. HHS agrees that there should be continued vigilance and coordination to avoid unnecessary duplication across research projects, and the HHS will continue its procedures for avoiding duplication.

In the report, it is cited that "84 percent of the autism research projects funded by federal agencies were potentially duplicative." HHS believes that this statement is misleading or could be easily misconstrued. It is important to recognize the difference between appropriately addressing complex problems using multiple strategies and funding redundant or duplicative projects. We do not believe that research is necessarily duplicative if two agencies fund the same broad objective in the IACC Strategic Plan. GAO does not outline in the report the purpose and process for developing the strategic plan, yet for several areas that were determined to be gap areas in research and that were highlighted in the strategic plan, such as services to underserved populations and research regarding services to adults on the spectrum, it is a testament to agencies' responsiveness to needs expressed by the field that multiple research projects were funded to address these areas of need.

GAO made the following statement while addressing the shortcomings of the Interagency Autism Coordinating Committee (IACC)'s data: "We found that, when looking across multiple years, some agencies funded more autism research projects than were suggested in the associated strategic plan objective...." This sentence implies that this is an indicator of wasteful spending, but that could be misleading. IACC stated as it developed its Strategic Plan that it intended the objectives to describe the minimum effort that would be required to address the complex issues set in each objective. The recommended number of projects or amount of funding in the objectives was described by IACC as a "floor, not a ceiling," recognizing that this minimum effort may only begin to address the gap areas described in each objective and that, in most cases, further efforts were likely to be required to fully address these areas of need.

GAO Recommendation 1: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, we recommend that the Secretary of Health and Human Services direct IACC and NIH, in support of IACC, to provide consistent guidance to federal agencies when collecting data for the portfolio analysis and web tool so that information can be more easily and accurately compared over multiple years.

HHS Response: While HHS appreciates the value of consistency, we do not concur with the GAO characterization of the process or the quality of the Portfolio Analysis. Autism science has moved quickly in the past five years, with new fenders as well as new areas of funding. It is important to appreciate that in such a changing landscape of scientific funding, accuracy may need to trump consistency and that guidelines must evolve to be responsive to new opportunities.

The Office of Autism Research Coordination (OARC), which conducts the IACC Portfolio Analysis and publishes the data in a publicly accessible report and online database on behalf of IACC, has used a consistent methodology and provided consistent guidance to funders participating in IACC ASD Research Portfolio Analysis during the past five years, but has balanced the need for consistency with the need to be responsive to feedback from IACC and from participating funders. This feedback has included suggestions for improving the clarity of the instructions in the guidance, addressing problems encountered in the previous year, and addressing issues that are unique to a specific participating agency or organization. OARC made updates to each year's guidance to address those types of issues and feedback, with the goal of improving the guidance to make the process easier and clearer for participating funders and to ensure the most current, accurate, and precise reporting of data. The updates did not, however, change the overall strategy or methodology for conducting the analysis.

One example of such a clarification, mentioned in the GAO report, was the addition of text to the 2011 and 2012 IACC Portfolio Analysis guidance to clarify the types of projects that fit within two IACC Strategic Plan objectives related to evaluation of new and existing pre-service and in-service training programs. IACC requested that OARC make a clarification in the guidance to assist funders in determining which projects fit within these two objectives, to improve the accuracy and precision of reporting. The following clarification was added in response to IACC's request: "For these objectives, please include projects that are related to evaluating pre-service or in- service training for service providers. This may include projects that are launching, piloting or testing new or innovative services practitioner training approaches in community based settings, evaluating the effectiveness of these programs in order to improve service practitioner training efforts, and/or contributing to the evidence base supporting various practitioner training methods. This may also include efforts to test new or innovative services approaches in various settings, make improvements in services methodologies, test effectiveness, develop best practices and improve targeting and dissemination of services. Many of these projects may be focused on practical, field-based and community-based approaches used by services agencies and organizations in their efforts to improve services. When including these projects, please be sure to note the evaluative component in the project description to ensure that the projects that are included are designed to test and improve programs rather than simply to administer them (service provision)." This change in the guidance, which provided additional detail about coding to those two particular objectives, fulfilled IACC's request to OARC for the guidance to be clarified with regard to those objectives, and enabled agencies to report more accurately and precisely the projects that were responsive to these two objectives.

In 2011 and 2012, IACC Portfolio Analysis guidance also was updated in response to participant feedback requesting an option for the OARC to assist participating agencies by offering to conduct the initial coding of projects submitted by the agency, followed by verification by agency officials, to reduce the burden on contributing agencies. Feedback on the new option was overwhelmingly positive. Incorporating such suggestions from participating funders provides important opportunities for refining and optimizing the data collection process, which has been especially important because the Portfolio Analysis is a relatively new effort, having begun in 2008, and this scope and type of data have not been previously collected for most disease/disorder areas, so the process itself was relatively new. With any new process, refinement and adjustments are to be expected and are desirable for the purpose of improving processes. Changes in the guidance such as those described in these two examples did not change the overall methodology of the analysis, but they did provide an opportunity to refine and clarify the instructions, enabling more efficient, accurate, and precise data collection. In the future, OARC will continue to balance the need for maintaining consistency in the methodology of the Portfolio Analysis and accompanying guidance while also solving problems encountered and being responsive to IACC and participant feedback.

With regard to accuracy, GAO cited 16 projects that were missed by OARC's data collection efforts. OARC examined this list of projects and determined that two of the projects cited by GAO were included within OARC's data sets, and the remaining 14 projects either pre- dated the involvement of those particular funders in the Portfolio Analysis effort or were projects that were not submitted by the agency at the time of the Portfolio Analysis project but may have been identified as relevant to the autism portfolio at a later date. Since the IACC Portfolio Analysis effort is dependent on agency submissions and cross-verification between the agency and OARC, projects can only be reflected in the report if they are submitted by a participating agency at the time of the data collection for any given year's report. Nevertheless, with the over 5,500 autism-research-related projects spanning both federal agencies and private organizations that have been captured by OARC's Portfolio Analysis effort in the past five years, OARC has provided IACC with access to a level of data describing the public-private research landscape that is not available for most other diseases, disorders, and health conditions either through government or private efforts, enabling them to use this information to assess the status of the field year by year and to make informed recommendations to guide future efforts.

While a core group of funders have been involved in the IACC Portfolio Analysis effort since its inception in 2008, other funders have been invited or have volunteered to participate as the effort has continued. IACC in its discussions of the Portfolio Analysis effort has recognized that the addition of new funders over time may make comparisons of the overall funding from year to year challenging, but decided during its discussions that having the most current and comprehensive information regarding research-related efforts that are ongoing throughout the community is its highest priority. To keep the Portfolio Analysis effort up to date and as comprehensive as possible, IACC has expressed the intent to continue including new funders who may be supporting projects that are relevant to ASD research as the opportunity arises. It is expected that in the future, the pool of funders included in the analysis will largely stabilize as a large proportion of these relevant funders become participants in the effort, and this will enhance comparability of overall funding data over time.

In addition to changes in funders over time, dramatic changes in the number and scope of the objectives in the early years of the IACC Strategic Plan-due to the committee's effort to continually update the Strategic Plan to reflect the current needs of the ASD community-have presented a challenge in terms of comparing data across years in the first three years of the effort. However, in the most recent three years, the stabilization of the Strategic Plan itself, due to the lack of new objectives or major revisions to the objectives made by the committee, has provided improved opportunities for meaningful multi-year comparisons. As the Portfolio Analysis effort continues in future years and the group of participating funders also increasingly stabilizes, OARC anticipates additional opportunities to make meaningful comparisons of data across multiple years. The first of such comparative data, reflecting five years of funding, will be used by IACC to assist them in their 2013 Strategic Plan Update process in the fall of 2013 and will be published in the upcoming edition of the IACC Portfolio Analysis Report and Web Tool that are in preparation. As in past years, all documents used by the committee, the reports, and the database updates will be fully accessible to the public via the IACC website at [hyperlink, https://iacc.hhs.gov/].

GAO Recommendation 2: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, we recommend that the Secretary of Health and Human Services direct IACC and NIH, in support of IACC, to create a document or database that provides information on non-research autism-related activities funded by the federal government and make this publicly available.

HHS Response: HHS agrees that it is important for Congress and the public to have access to information about non-research autism-related activities. HHS does not concur with the recommendation that IACC and NIH be directed to create a new document or database to catalogue these efforts, as such a project may duplicate ongoing efforts. IACC and several federal agencies already provide publicly accessible information on non-research autism-related activities in several ways described below.

For example, in 2010, OARC, on behalf of the Office of the Secretary of HHS, coordinated the Report to Congress on Activities Related to Autism Spectrum Disorders and Other Developmental Disabilities Under the Combating Autism Act of 2006 (FY 2006-FY 2009) to provide information requested by Congress in Section 399DD of the Combating Autism Act of 2006. This report was provided to Congress by the Secretary of HHS to inform Congress of the broad scope of both research and non-research activities that were being supported across HHS and the Department of Education. The 76-page report describes research and services programs and projects across 9 agencies, including education, transition planning, employment, home- and community-based services, legal services, rehabilitative services, inclusion and a variety of other services, and other non-research- related programs, in addition to descriptions of government research programs. The report is available to the public on the IACC website at [hyperlink, https://iacc.hhs.gov/publications/report-to-congress/2009/]. The next Report to Congress on federal autism activities, which will encompass descriptions of a broad array of research, as well as services and other non-research programs and projects that were supported by 13 federal agencies and Departments from 2010-2012 is in preparation and is expected to be sent to Congress by the Secretary of HHS and then released to the public in the fall of 2013.

Given that OARC and several federal agencies are already making significant efforts to provide Congress and the public with detailed information about a wide array of federal non-research autism-related activities in publicly accessible formats, it will be important for IACC and OARC to continue to post links to these reports and resources on the [IACC web site. OARC will also continue to ensure that IACC sets aside time during its meetings to discuss these kinds of documents, projects, and resources to facilitate coordination and public awareness of these important federal activities.

GAO Recommendation 3: To improve the usefulness of IACC data and enhance its efforts to coordinate HHS autism activities and monitor all federally funded autism activities, we recommend that the Secretary of Health and Human Services direct IACC and NIH, in support of IACC, through its monitoring of federal autism activities - including OARC's annual collection of data for the portfolio analysis and IACC's annual process to update the strategic plan - identify projects that may result in unnecessary duplication and thus may be candidates for consolidation or elimination, and identify potential coordination opportunities among agencies.

HHS Response: HHS does not concur with the GAO recommendation if the intent is for the Secretary of HHS to direct the IACC to conduct analysis with the specific goal of identifying duplication of effort, as such an effort would not be likely to provide the type of information on "actual duplication" needed by agency officials in determining overlap prior to making funding decisions. However, HHS will continue to support IACC in the many different types of activities it conducts on an annual basis to coordinate HHS efforts and identify opportunities for cross-agency coordination. HHS does not concur that there is too much federal activity related to autism and that the IACC role should include identification of autism-related projects for elimination.

IACC conducts many different types of activities as a part of its process to monitor federal activities and coordinate federal agency efforts on ASD. These include hosting presentations from and discussions with agency officials regarding their programmatic activities during IACC meetings, examining and discussing published studies and government reports related to government-funded efforts, gathering information on federal activities during the annual Strategic Plan updating process, and conducting quantitative analyses of publications and grant funding. All of these activities help agencies gain various levels of information about other agencies' projects and programs and help agencies identify opportunities for enhanced coordination of efforts.

The IACC Portfolio Analysis and Strategic Plan update processes are designed to do the following: help 1ACC gather information that can be used to support their efforts to provide advice to the Secretary of MIS, develop an annual Summary of Advances in ASD research, develop and annually update a Strategic Plan for ASD research using the latest information about advances in the field and the status of research funding across the federal government and private funders, monitor federal activities, and coordinate HHS activities. While the Portfolio Analysis and Strategic Plan processes cover a broad range of information about research activities and funding, they are not designed to provide the types of information that would be needed to make individual funding decisions about federal grants. These processes examine the portfolio of Strategic Plan-related activities and funding at a high level, analyzing aggregate data from multiple agencies and organizations, and they examine the portfolio after funding decisions have already been made because the analysis relies on publicly available federal data.

To determine whether or not particular grants may overlap with other grants and thus whether certain grants would be candidates for budget reductions or elimination from consideration for funding would require the determination of "actual duplication" prior to award and not "potential duplication" post-award, which is what is provided by GAO in this report. For example, if IACC were to determine that having more than one grant assigned to each objective in the IACC Strategic Plan constituted "potential duplication" and that, therefore, almost all grants awarded are "potentially duplicative," this would not give program officials enough specificity to help them identify actual instances of duplication that they would not have otherwise identified through the extensive processes they already use to determine duplication/grant overlap during the grant award process. Even if IACC further refined its analysis to identify several grants on the same general topic within an objective as "potential duplication," this type of information still would lack adequate specificity to be useful to agency officials who are involved in the grant award process.

IACC's Strategic Plan questions and objectives encompass broad and complex areas of research for which multiple agencies and projects, often using different approaches, would be expected and needed in order to achieve the desired progress and advances in the ASD field. Thus, evidence of multiple projects addressing a question within a field of research would not be deemed to be duplicative. While any time that more than one project is funded there is a potential for duplication, agency funding decisions would not be changed based on potential duplication. They would only be changed in cases of actual duplication, which may be defined as the same researchers requesting funding from more than one source to conduct the same aspects of the same project. Co-funding of the same project by two agencies, the same researcher conducting follow-on projects to a previously funded project, or two researchers addressing a similar question with different approaches would be examples of projects that could look related to a non-expert but are not considered to be overlap or actual duplication.

Determination of "actual duplication" or grant overlap requires detailed subject matter expertise that would allow an individual to distinguish related activities from duplicative activities and access to pre-award data during the period prior to when an award is made. Agency program officials have the responsibility to check each grant for potential overlap with other funding sources prior to award, and these officials have both the expertise and access to grant application data and federal databases, as well as the knowledge of federal rules and policies, that would be required to determine if actual overlap exists. In addition to these issues (lack of adequate subject matter expertise and lack of access to pre-award grant data) that would prevent IACC from being able to add an effective layer of information to the federal grant award process, the committee would be limited by the amount of time the members have in their meetings during the year to carry out such an effort.

IACC typically meets four to six times per year as a full committee (in addition to subcommittee and working group meetings) and is responsible for providing advice to the HI-IS Secretary, completing an annual update of the Strategic Plan, developing an annual summary of research advances, coordinating HHS activities; and monitoring federal activities. Monitoring federal activities includes not only the Portfolio Analysis effort but also interactions with external invited experts and discussion of agency activities during IACC meetings. Even if all future IACC meeting time were to be used solely to examine the ASD research portfolio, it would be unlikely that the committee could complete a more detailed analysis than that resulting from GAO's intensive one and a half year analysis, which identified only potential duplication. Such information that is limited to potential duplication would not provide an added value for program officials in their already extensive process to determine actual grant overlap prior to funding.

Program officials already have access to the broad types of information, such as which projects are categorized under each Strategic Plan objective, through the IACC Portfolio Analysis Report and the IACC/OARC Portfolio Analysis Web tool, which they can use as a source of information when making determinations. The IACC/OARC Portfolio Analysis Web Tool is publicly available online at [hyperlink, https://iacc.hhs.gov/apps/portfolio-analysis-web-tool/projects].

GAO Recommendation 4: To promote better coordination among federal agencies that fund autism research and avoid the potential for unnecessary duplication before research projects are funded, we recommend that the Secretary of Health and Human Services, the Secretary of Defense, and the Secretary of Education, and the Director of NSF each determine methods for identifying and monitoring the autism research conducted by other agencies, including by taking full advantage of monitoring data the IACC develops and makes available.

HHS Response: HHS does not concur with the finding that "Federal Agencies' Coordination and Monitoring of Autism Activities was Limited." HHS agrees with GAO's conclusion that it is critically important to avoid unnecessary duplication in research related to Autism Spectrum Disorders, We have not encountered any actual examples of unnecessary duplication in interactions between agencies, and none were cited in the GAO's report.

HHS has robust procedures in place for avoiding duplication before grant and contract awards are made and to keep the funding decision-making process fair and equitable. In addition, the internal NIH Autism Coordinating Committee (NIH ACC) and IACC provide-opportunities for monitoring and collaboration within NIH and across federal Agencies. These policies and coordinating bodies have served HHS well in terms of identifying and preventing duplication prior to making funding decisions. We will continue to monitor the internal NIH ACC procedures, as well as participation on the IACC, to make full use of these opportunities.

IACC is able to use its Portfolio Analysis to broadly track progress of Federal agencies in achieving Strategic Plan objectives. IACC is not equipped with the scientific expertise or access to pre-award data that would be necessary to make an accurate determination regarding grant overlap issues. As the GAO report indicates, NIH funded 81 percent of the autism-related research from fiscal years 2008 through 2012. Therefore, coordination among the NIH Institutes and Centers (ICs) that fund autism research represents a large component of HHS' ongoing efforts to avoid unnecessary duplication in research. To achieve this coordination, NIH convenes meetings of its internal ACC at least monthly. Program, communications, and policy staff from the National Institute of Mental Health (NIMH), the Eunice Kennedy Shriver National Institute of Child Health and Human Development (NICHD), the National Institute of Neurological Disorders and Stroke (NINDS), the National Institute on Deafness and Other Communication Disorders (NIDCD), and the National Institute of Environmental Health Sciences (NIEHS) are members of the ACC. ACC members collaboratively plan and co-fund major autism-related research initiatives and scientific workshops, and they review the IACC Portfolio Analysis for gaps in research when planning such activities. Moreover, they share information related to autism research activities at their respective Institutes, including information about activities in which the NIH ICs participate or coordinate with other federal agencies. They also share information they learn about nongovernment-funded autism research through participation in scientific meetings and other activities sponsored by nongovemment organizations. Therefore, collaborations and information exchanges through the NIH ACC provide important opportunities for averting unnecessary duplication before it happens, both within and beyond the NIH.

IACC affords another opportunity for cross-Agency information-sharing. It is a forum for announcing major funding initiatives to all IACC members and to others who attend or listen to IACC meetings, including staff from federal agencies and non-government organizations and members of the patient, provider, and research communities, as well as the general public. For example, in the past two years, the following major research initiatives and scientific workshops have been presented to the IACC:

CDC-sponsored Study to Explore Early Development
CDC-sponsored Read the Signs Act Early campaign
NIH-supported Autism Centers of Excellence initiatives
ARRA-supported JobTIPS Web site
HRSA-sponsored Combating Autism Act initiatives
CDC-sponsored Autism Prevalence Workshop
Workshop on Epilepsy and Autism Co-sponsored by private funders and the NIH
CDC-NIH sponsored Minnesota Somali Project
NIMH-sponsored Study of Health Outcomes in Children with Autism and their Families
NICHD-sponsored National Children's Study
CDC/HRSA-sponsored study on Changes in Prevalence of Parent-reported Autism Spectrum Disorder in School-aged U.S. Children
NIH-sponsored workshop and funding initiative on Minimally Verbal ASD.

In addition to the NIH ACC and IACC, NIH as a whole has extensive policies in place concerning monitoring and managing potential overlap in funding. Avoidance of duplication before funding is the goal, but monitoring and addressing duplication also occurs during review of yearly progress reports from the funded investigators. Avoiding overlap is a formal and critical responsibility of NIH staff, including grants management, program, and review staff. Further, NM's comprehensive internal database, Information for Management, Planning, Analysis, and Coordination (IMPAC II), provides information systems to support the full life cycle of grants administration, including information about grant applications prior to award. IMPAC II is the database used extensively by NIH grants and program staff to obtain and manage detailed information used for funding decisions, among other things. IMPAC II currently includes grants and applications for NIH, the Food and Drug Administration, the Centers for Disease Control and Prevention, the Agency for Healthcare Research and Quality, the Substance Abuse and Mental Health Services Administration, and the Department of Veterans Affairs. The U.S. Department of Agriculture will be added to the database in the next year, and discussion is ongoing with the Department of Defense and the National Science Foundation to investigate the addition of their applications and grants to IMPAC II.

NIH addresses any type of overlap, whether it is scientific, budgetary, or commitment of effort, prior to the issuance of a Notice of Grant Award and while reviewing annual progress reports. Submission of identical applications to one or more components of the Public Health Service (PHS) is not allowed, and NIH will not accept similar grant applications with essentially the same research. If there is partial duplication, the pending application, other applications, or the active award will be modified prior to NIH's funding the pending application. To make these determinations, staff consult, as necessary, with the Principal Investigator, other funding components within NIH, other government agencies, or nongovemment organizations to resolve questions of overlap.

The application instructions require the applicant (1) to indicate whether his/her application has been sent to other agencies outside the PHS and (2) to include on the cover of the application to which agencies the application has been submitted. This information is part of every electronic application. It is also important to note that the application instructions, as well as the NIH Grants Policy Statement (10/2011), which is a term and condition of all NIH grant awards, contain the NIH's policy on similar, essentially identical, or identical applications. To prevent duplicate research from a single investigator or institution, it is the responsibility of program and grants management staff to review routinely Other Support documentation that is provided in the grant application and updated prior to award (and which includes all financial resources whether federal, nonfederal, commercial, or organizational, available in direct support of_the key personnel's research endeavors) to determine if there is budgetary, scientific, or time commitment overlap. Review of IMPAC II grants information and Other Support documentation is used to identify potentially overlapping or duplicative applications from different research teams.

Additionally, applications to NIH are grouped by scientific discipline for review by individual Scientific Review Groups. The reviewers can thus easily identify multiple grant applications for essentially the same project. In these cases, application processing may be delayed or the application(s) may not be reviewed. In addition, peer reviewers are asked to rate the significance and innovation of grant applications, including whether it will substantially advance the field and whether the research projects utilize innovative and novel approaches, or address an unmet need.

[End of section]

Note: (HTML text files accessed December 2, 2013)

Comments from the National Science Foundation

National Science Foundation PDF version (PDF – 3.7 MB)
4201 Wilson Boulevard
Arlington, VA 22230
[hyperlink, http://WWW.NSF.GOV]

October 23, 2013:

Dr. Marcia Crosse
Director, Health Care
U.S. Government Accountability Office
441 G Street, NW, Room 5A-14
Washington, DC 20548

Dear Dr. Crosse:

Thank you for providing the National Science Foundation (NSF) an opportunity to comment on the U.S. Government Accountability Office's (GAO) draft report entitled Federal Autism Activities: Better Data and More Coordination Needed to Help Avoid the Potential for Unnecessary Duplication (GAO-14-16).

NSF supports fundamental research across all areas of science and engineering from astronomy to geology to zoology. However, NSF does not generally support biomedical research. With an annual budget of nearly $7 billion (fiscal year 2013), NSF funds approximately 21 percent of all federally supported fundamental research conducted by America's colleges and universities. NSF support results in scientific and engineering advancements that are central to our nation's future well-being. NSF also supports the preparation of tomorrow's science, technology, engineering and mathematics (STEM) workforce.

As the draft report accurately indicates, NSF is not a member of the Interagency Autism Coordinating Committee (IACC). NSF does not have autism research initiatives or programs. As a result, NSF makes an extremely limited number of autism-related annual awards. Appendix Ill of the draft report identifies 28 NSF-supported, autism-focused research projects from 2008-2012; during that same period, NSF made on the order of 55,000 awards. Proposals that align with NSF programmatic areas and that might have relevance to autism are reviewed for scientific merit using two criteria: intellectual merit and broader impacts. Reviewers and NSF program officers, who are scientific experts, determine whether the proposed research activity is meritorious. Such research may be funded for reasons not identified in the IACC strategic plan.

Contrary to the draft report's statement, NSF does take steps to prevent duplication. NSF requires that proposals submitted to NSF provide complete information about current and pending support, including funding from other federal agencies, regarding the proposed research activity. As necessary, NSF consults with other agencies regarding potentially duplicative research proposals to ensure that NSF does not fund the same research activity supported by another agency. It should be noted that the cataloging by the IACC of NSF research awards does not suggest that grants in a category, such as "Biology," are necessarily duplicative of other agency awards about the biology of autism. Additionally, NSF provides publicly available information about our research and education awards on NSF's website. NSF also provides this information to the IACC for their portfolio reporting.

NSF welcomes the opportunity to collaborate and exchange information with other federal agencies about science related to autism. NSF will continue to identify NSF-supported research related to autism and provide this information to the IACC. We look forward to receiving your final report.

Sincerely,

Signed by:

Kathryn Sullivan
Senior Advisor
Office of the Director/Office of International and Integrative Activities

[End of section]

Note: (HTML text files accessed December 2, 2013)