Section IV. Program Evaluation
CDMRP Program Evaluation Efforts
Research Information Management and Dissemination—The CDMRP Website
——Institute of Medicine Report
In 1993, the Institute of Medicine (IOM) advised the U.S. Army Medical Research and Materiel Command (USAMRMC) on strategies for managing the FY93 Congressional appropriation for peer-reviewed breast cancer research. In late 1995, the USAMRMC commissioned the IOM to review the progress of the Breast Cancer Research Program (BCRP). The IOM was asked to include a review of the portfolio of funded research, assess program management and achievement, and recommend areas for funding that have not been funded or areas that need added emphasis. The result of this review was a report published in 19971 that concluded with 3 major recommendations and 13 secondary recommendations.
One of the major recommendations was that the office of the Congressionally Directed Medical Research Programs (CDMRP) should, "develop and implement a plan with benchmarks and appropriate tools to measure achievements and progress toward goals of the BCRP both annually and over time."2 The IOM recommended an evaluation of the funding mechanisms, particularly Innovative Developmental and Exploratory Awards (IDEA) and recruitment/training awards, to determine whether these mechanisms are effective. The IOM suggested looking at the records of publications and presentations and at the ability of BCRP-funded investigators to obtain funding from other sources as potential measures of success. Also considered relevant was the tracking of investigators who were recruited into breast cancer research.
The CDMRP is accountable for the expenditure of Congressional appropriations for targeted diseases that are managed by the USAMRMC. Congress is interested in making sure that appropriations are handled effectively and are having a positive effect on the public’s health. Scientists are eager to know the outcomes and progress of research funded by the Department of Defense (DOD). Finally, consumers who lobby for the Congressional appropriations want to know that the research supported by the CDMRP is having a positive impact on the lives of patients.
Since the inception of the CDMRP, the progress of every contract has been monitored for the duration of the award. Funded investigators must submit annual and final reports that summarize the research efforts of the project. Investigators are asked to provide a bulleted list of key research accomplishments and a list of reportable outcomes to include manuscripts, abstracts, patents, degrees obtained, and employment or funding received based on work supported by the award. These products are entered into a database, and work is currently under way to list all publications, patents, and licensures associated with an individual award on the CDMRP website, http://cdmrp.army.mil.
——Era of Hope
The BCRP sponsored a multidisciplinary meeting in November 1997 called the Era of Hope to disseminate information on funded BCRP research projects. This meeting provided scientists the means to share ideas leading to potential collaborative efforts and novel approaches to tackle difficult breast cancer research problems. FY92-95 grant recipients attended, presented their research advances, and shared the progress of fellow scientists in breast cancer prevention, detection, diagnosis, treatment, and quality of life. The interaction of BCRP grant recipients and other renowned scientists, physicians, and health care providers with breast cancer survivors and consumers contributed to the success of the meeting. Over 50 representatives from the media attended the Era of Hope, approximately 175 press releases covering the meeting and the DOD BCRP grant recipients’ research advances were printed, and 245 television and radio broadcasts disseminated information throughout the duration of the meeting. For more details about the Era of Hope, refer to Section V (Era of Hope). Due to the success of the first meeting, the Second Era of Hope will be held 8-12 June 2000.
The CDMRP assembled a group of scientists and administrators, known as the Portfolio Analysis Working Group, to systematically analyze the CDMRP research portfolio. As an initial starting point in analyzing the portfolio of funded research projects, the Portfolio Analysis Working Group proceeded to develop and define a unique research classification system specific to the CDMRP. Although this classification system has been utilized in the analysis of FY92-94 breast cancer awards, the system was designed to be flexible, allowing for modification over time as the field of cancer research evolves. Using the CDMRP classification system in conjunction with one developed by the National Cancer Institute and pilot tested by the CDMRP, the Portfolio Analysis Working Group is in the process of analyzing the entire CDMRP portfolio. The classification of CDMRP-funded research will provide the foundation to assess program outcomes.
Information about the CDMRP can be accessed by the public at the internet address http://cdmrp.army.mil. As the CDMRP continues to evolve, it is important to disseminate program information to the public. The CDMRP website is updated as new information becomes available. The information base that is currently available on the CDMRP website includes, but is not limited to, the following:
——Research Programs: Breast Cancer, Prostate Cancer, Ovarian Cancer, Neurofibromatosis, Special Populations, Defense Women’s Health, and Osteoporosis
Facts, figures, and documents related to the programs that are being managed by the office of the CDMRP are highlighted on the CDMRP website. Information such as the vision, mission, Congressional appropriation, and funding summary is provided for every program.
The CDMRP supports a diverse portfolio of research that will impact the health of all people. Outcomes of the research, such as peer-reviewed publications, patents and licensures, and infrastructure enhancement accomplishments, are disseminated to the public under each research program.
As noted in Section I, CDMRP Overview, consumers play an important role in all research programs administered by the CDMRP. Information about consumer representation on scientific peer review was compiled into the Consumer Reviewer Resource Book and can be downloaded from the website.
Important dates associated with each CDMRP are posted on the website to disseminate program information to the public in a prompt manner. For instance, dates associated with the release of Program Announcements, submission deadlines, and notification of funding status are made available. Additionally, special events such as the Second Era of Hope and the CDMRP display schedule are also highlighted on the web (see Second Era of Hope and Display Schedule below).
Information regarding the CDMRP funding history, such as a historical overview of the CDMRP, Congressional appropriations, and dollars available for research, is disseminated to the public. Additionally, detailed information about research that is funded by the CDMRP can be obtained by performing searches on the CDMRP database. This database can be searched by a variety of descriptors, such as research program, fiscal year, award category, research project title, principal investigator, and/or institution.
——Press Releases, Program Announcements, Publications
Publicly available documents, such as Program Announcements, press releases, information papers, reports, and related documents, are posted on the website and can be downloaded by viewers. This enables researchers to access information on the CDMRP as well as to obtain necessary proposal submission information in a timely manner.
——Second Era of Hope
As noted under Program Evaluation Efforts, the BCRP sponsored a successful multidisciplinary meeting called the Era of Hope in the fall of 1997 where BCRP grant recipients presented their research advances. The Second Era of Hope meeting for the BCRP will be held 8-12 June 2000 in Atlanta, Georgia. Information about the first and second Era of Hope meetings is available on the CDMRP website.
A CDMRP display is presented at national scientific meetings to disseminate program information. To foster program awareness, a list of the scientific meetings and dates where the CDMRP display will be presented is posted on the website. The CDMRP encourages all interested researchers to visit the display at professional meetings and talk with Program Managers.
Resources, web links, and documents that are relevant to the research programs managed by the CDMRP are provided.
1 A Review of the Department of Defense’s Program for Breast Cancer Research. Institute of Medicine, National Academy Press, Washington, DC, 1997.
2 A Review of the Department of Defense’s Program for Breast Cancer Research. Institute of Medicine, National Academy Press, Washington, DC, 1997, p. 12.