Gayle Vaday, PhD; Program Manager, Breast Cancer Research Program:
In this part of our presentation, I'm going to be discussing the award mechanisms and review processes of our cancer research programs. And, I'll be touching on some of the topics that Captain Kaime already described, but I hope that this information will be useful and helpful to those of you who are interested in applying to our cancer research programs. Each of our programs are funded as a result of the advocacy efforts for each of the specific diseases. And, they each have their own program goals and vision that are established each year by the Integration Panel. And, this ensures that the programs remain focused on the needs of that disease community as well as the research that will address those needs.
When each program offers a funding opportunity or an award mechanism, it releases what's called a program announcement. And, in that program announcement is where you will find the description of the program's goals and vision for that year. In addition to that, each program announcement describes the award mechanism that the program announcement is associated with. And, this is an important point that award mechanisms are designed specifically by the Integration Panel to address a need or to fill a gap in research so we do our best to explain what the intent of the award mechanism is. And, finally, the award mechanisms also describe the review criteria-how the applications will be reviewed by the peer reviewers and the programmatic reviewers. And, this really breaks down how each proposal will be evaluated. So, taking all of this information into consideration, the program announcements really are the communication tool from our programs to the research community. So, we really encourage you to review each one each year because they do change from year to year as Captain Kaime described. And, they also differ from program to program, while one program may offer the same title of award mechanism like an Idea Award, they can differ quite significantly. So, we encourage you to read those carefully.
So while each program has its own individual vision and goals, one common goal across all of our programs is to find scientifically meritorious proposals and to fund those that best fulfill our program goals. And, we feel that the two-tier review process really enables us to strive toward achieving this goal. The first tier accomplished by the peer review panels is done completely independent from the Integration Panels' role at programmatic review. And although their functions are independent, we really do avoid cross-over or duplication of efforts. There is a partnership between those two functions because they do influence one another as I'll show you in the next couple slides where I will describe each tier of review in more detail.
So the peer review process at CDMRP is something that is applied to every proposal that is submitted. Every proposal undergoes a scientific peer review. We receive, depending on the program, hundreds to thousands of proposals. It is a very competitive process. At the scientific peer review level, each proposal is evaluated for its own scientific merit - how it measures up independently. In fact, we ask the reviewers to evaluate each proposal against the gold standard and not to compare proposals - that is done later on. We use specific criteria that are defined in the program announcement. In fact, in the program announcement we order them in rank order of importance. Again, that is an important piece to note in the program announcement. But because we have these criteria, we feel it is a robust way to really identify the strengths and weaknesses of each proposal. Finally, some of our cancer programs have emphasis areas. You may also hear them as topic areas, areas of emphasis, areas of focus, overarching challenges - all of them are different across programs, but the peer reviewers are asked to evaluate each proposal and how those proposals fit into any of those categories.
So as Captain Kaime mentioned, our peer review process does not have standing panels. We recruit expert reviewers based on their expertise, on their basic or clinical research expertise, depending on the panel or the award mechanism. In addition, we do strive to have a diversity of reviewers from different academic ranks and demographics. We also recruit new reviewers into our process. We ensure that the majority of reviewers at our peer review sessions are experienced CDMRP reviewers but we really do welcome new reviewers into the process because we think it is important to bring new, fresh eyes, fresh ears, and fresh perspectives in order to keep a high quality review.
The other experts who serve on our peer review panels are the consumer reviewers. Consumers must be nominated by an advocacy organization or support group so they are actively involved in their disease community. Each provide a broad perspective and although they may have their own individual stories, they represent a voice for their communities. And we've had, over the years, hundreds of advocacy and support groups represented at the peer review level by consumers. The outcomes of peer review are the reviewer critiques, and scores. From the majority of our award mechanisms, they get captured into what is called the summary statement and that's passed on and provided to the Integration Panel. So, the Integration Panel conducts programmatic review. And, this is the point in our process where proposals are compared to one another. It's a difficult decision-making process. As I mentioned, it's very competitive. We have hundreds of proposals that the Integration Panel must decide which ones really are worthy of funding. In addition to having high scientific merit, the Integration Panel also carefully considers other criteria including how well a proposal adheres to the award mechanism's intents. As I mentioned in our program announcements, we described carefully what the programs are looking for, what gap are we trying to fill. So, the Integration Panel really stays true to that. On top of having a high merit proposal, you must also make sure your proposal adheres to that intent. In addition, the relevance of the program's mission is very important. It must have direct relevance to the cancer research for that program or the cancer topic. Broadly applicable proposals are very important but they must remain focused because our programs really are distinct.
Captain Kaime mentioned that our programmatic reviewers do not have access to the entire application. And, this is on purpose. We don't want a repeat of a scientific evaluation. There's no need to peer review it again. The programmatic reviewers are provided with a summary statement or the reviewer critiques from the scientific peer review as well as the abstracts and any required statements specific for that mechanism. So, it's very critical to write an excellent technical abstract, a lay abstract, or a public abstract that's written in nonscientific terms as well as very clearly written innovation impact statements, and any other statements that are required for that mechanism, because that's all the programmatic reviewers will have as they make those funding recommendations.
So, the final outcome again are the list of awards that are recommended for funding and then, from there on, we begin the award negotiations and the peer review period of performance. Now, that I described the review process, I want to shift gears and talk more specifically about our award mechanisms. As I've mentioned several times now, it's important to understand the intent of the award mechanism and the review criteria in order to submit a successful application. Our cancer research programs offer award mechanisms that really fall into two categories: career development awards and research awards. Our career development awards really span the pipeline along the career path of the scientist. Starting with the predoctoral and postdoctoral awards, physician scientist research awards, and the new investigator awards. I'm going to talk in more detail about a couple of these because I think it's important to better understand. I know that many of you are interested in applying to these award mechanisms so I'd like to provide a little more detail about each of those and what we look for.
For the predoctoral and postdoctoral award mechanisms, the PI is the trainee. It is the predoc or the postdoc. We recognize that the mentor is the PI of the lab, but for the application process, the mentor is not the PI, it is the trainee. We are looking for predocs and postdocs who are committed to a career in that specific cancer. We're really looking to train the next generation of researchers in that area. And, they should prepare the application with the appropriate guidance from the mentor. The mentor should have expertise in that specific cancer of interest to include current or recent publications or funding and that really should demonstrate that that mentor has an active research program in that cancer. A very important aspect of the predoctoral and postdoctoral applications is the individualized training plan, and this needs to be very specifically detailed. It's not enough to say that you'll attend journal clubs or you'll be surrounded by other researchers - that is not a training plan. What we are asking for are specific details about how you will be trained to be a successful researcher in that area, what kind of training sets you apart as an outstanding candidate for a predoc or postdoctoral award.
Our new investigator awards differ quite significantly across programs, but the common goal is to fund researchers early in their careers and to provide them with the funding to become established in that specific cancer research area. We are looking for accomplished investigators who show strong potential to be successful in their cancer areas and also potentially become leaders in the field. New investigator applications do not need to come from investigators within that field; we do welcome researchers from outside our cancer research areas who want to become established in that disease. For some of the programs, a designated collaborator may be required and that collaborator must have expertise in that cancer area, and they should provide the guidance especially for those who are coming from a different field.
So our research program, our research awards, also span the pipeline of the development of ideas beginning with initial concepts and early ideas, clinical and translational research, big team science awards, as well as advanced studies and clinical trials.
So again I'm going to talk a bit more about the award mechanisms that we receive the most applications for-beginning with initial concepts. So we offer what is called a concept award and an exploration hypothesis development award in several of our programs. These are small awards with short performance periods. Typically they're 1 to 2 pages, and these are designed in order to fund those very early ideas-just the ideas that you might have written on a post-it note or on a napkin-that you want to just test their feasibility, really see if it's worth pursuing. That is the gap that we try to fill through this award mechanism. Innovation is key, we are looking for out-of-the-box ideas, and no preliminary data is required-in fact, it's discouraged or not allowed for some of our programs, because of that reason we do not want a continuation of an ongoing research. We want to fund that early concept that needs to be tested. These award mechanisms are open to all investigators, junior to senior investigators; in fact, we have received applications for these mechanisms from postdocs and funded them. So that is something to keep in mind. The reviewers who review these applications are blinded to the identity of the PI and the institution, and we instruct the applicants to write their applications in order not to identify themselves. That's done for a reason because we want the reviewers to focus on the concept not on the reputation or the experience of the PI or the institution; we really are focused on the concept. Finally, human subjects research is limited in these award mechanisms. Because these are supposed to be early concepts, studies that are more advanced that would not qualify for an expedited or an exempt kind of study really don't fit the intent of this mechanism.
Our early idea award mechanisms like our idea award, synergistic idea award, idea development award are larger than the concept award, but they are also focused on those early ideas. We are funding innovative high-risk, high-gain research through these mechanisms-research that can leap frog advancements rather than funding the next incremental step. We recognize that it's important to fund ongoing research and to have that kind of continuous funding; however, that's not the gap that this research, these mechanisms, are trying to fill. We are trying to get those early ideas to see if they can get off the ground and get funding through other bigger award mechanisms.
Preliminary data is not required; in fact it's not encouraged for some of our programs. We really do want to provide the opportunity for investigators to generate preliminary data through these mechanisms.
The last set of category of awards I would like to describe are the team science awards. These are larger mechanisms in which we encourage investigators from different disciplines and different institutions to collaborate, to really reach beyond their comfort zone and expand their research to form big teams that can adequately address a problem in cancer research from different angles. In some of our programs, these award mechanisms also require integration or involvement of consumer advocates so we really do form these unique teams that can address a problem from unique perspectives. So it is our hope that with these mechanisms we can have greater outcomes than if we had these researchers work independently.
Just a recap of our cancer research programs-we have focus programs in breast cancer, ovarian cancer, prostate cancer, and lung cancer. We also have 2 programs, a Peer Reviewed Medical Research Program and the Peer Reviewed Cancer Research Program, for which several different cancer topics are funded. As Captain Kaime mentioned, each topic can change from year to year; in fact, each program can offer different award mechanisms depending on how much funding it receives that year through its congressional appropriations or through the different vision and goals that program establishes each year. Another important point I wanted to make is that the cancer programs in CDMRP may offer award mechanisms at different times of the year so it is important to stay up-to-date on what we are offering and what we are looking for. The best way to stay up-to-date is to be on our mailing list. As Carolyn described, we have a person in the back who at the end of session you can ask them to scan your card if you are not on our mailing list. You can also send an e-mail to help at cdmrp.org and ask to be added to our list or you can visit our booth here at AACR. It's booth number 434.
Our application process is a two-step process. The first is a required pre-application that's submitted through e-receipt. That's our online system and that pre-application may involve either a letter of intent or a pre-proposal. It's important not to miss that deadline. We cannot accept applications if you don't submit a pre-application. The applications are submitted through grants.gov, typically about 2 weeks or 3 weeks after the pre-application deadline. And to find all of our CDMRP funding opportunities, you can type in CFD numbers 12.420. You will find also all the funding opportunities of the U.S. Army Medical Research and Materiel Command there on that website. Finally, I want to end with our new website that we are going to be launching in the next few weeks. It's a new look for us, we hope it will be useful to all applicants and people who are interested in our programs. I do want to point out we have a search awards tool on there. So if you are interested in seeing what kind of research our programs have funded in the past, I invite you to go ahead and search those awards, and our contact information is there. If you have any further questions after AACR, please feel free to contact us. Thank you very much.
So now we are going to take a little time to open the floor for any questions that you may have for any of our panel members.
Q: Can you briefly comment on the mechanisms for recruiting scientific reviewers?
A - Vaday: So our review processes are really conducted through a couple of contractors and they recruit scientists based on the expertise that's needed for specific panels. For our larger programs, we have many, many panels because we receive so many different applications. For example, for the idea award mechanisms, we have a database of researchers who might have expertise that is applicable to the different panels such as cell biology, molecular biology, pathobiology. In addition for those larger award mechanisms where we receive fewer applications, we may recruit specifically for the type of expertise that's needed to review the applications that we received. So it depends on the award mechanism. It depends on the program. But we do recruit based on the expertise that's needed for panels or award mechanisms.
Q: So I have a simple question and a little bit more nuanced question. The simple question is, the conferences, are they open to the public?
A - E. Melissa Kaime, MD, FACP; Captain, Medical Corps, USN; Director, CDMRP: Unfortunately not. Just by sheer cost to be able to invite the entire American public-we couldn't. Who we do invite is certainly all of the investigators. They are judged by their peers-that can sometimes be very critical-but also the scientific peer reviewers and Integration Panel members. Consumers that were involved in the process are there as well. So unfortunately we can't open it widely to the public. I wish we could, but we can't just by sheer size and cost.
Q: Ok. The more nuanced question is I wonder if you could elaborate on the inclusion of preliminary data. If the award mechanism says it's not required-the implication was it's a disadvantage to do that. Can somebody elaborate on that?
A - Vaday: Yes. I'll try to answer that question. So when we say that preliminary data is not required, it's important to look at the award mechanism because if you are talking about an idea award mechanism, it can differ for each program. But what we mean by not requiring preliminary data is that we're trying to fund those early ideas where you want to generate the preliminary data to go on and get more funding. If you submit an RO1 type of proposal where you have generated very elegant preliminary data, it looks like it's going to work great, and you can continue to fly-that's not going to fit the award mechanisms where we say preliminary data is not required. So you really need to look at each program and the words and the way it is worded in the program announcement. But it is a disadvantage if the program is specifically stating we're looking for those early ideas. It should be based on a sound rationale, be scientifically sound, but that doesn't mean we are looking for continuation type of project, very, very incremental and that's important but it's not what we try to fund through some of our innovative type of award mechanisms.
Q: Actually my question is sort of answered. I have a similar question about the preliminary data, but I still want to say a couple of words. Because you are still looking for the early idea, however, given the current funding situation, only a few persons and proposals can be funded. So in this case, I'm sure, every PI is struggling to spend every penny to getting data so at this time of the application I think most people would have preliminary data so what should we do? So do we not put it in or should we put it in?
A - Vaday: We don't want to instruct you on how to submit your application because really we don't do that. If you call us in our program office and say, is this going to fit your award? Is this what you are looking for? Will it get funded? That's not up to us. We leave it to our process to do the work for us. But, we understand the scientific dilemma now that there is very limited funds for all research. But keep in mind that although this is a federal agency, we are indebted to the consumer advocates who lobby every year for these funds. So we have to try to stay true to that and although it may be submitting proposals you may think you need to submit preliminary data because you are accustomed to doing that for other agencies, you need to tailor your applications to what we are describing in our program announcement. You may have preliminary data in an idea award that may get funded but you really need to make it clear that it is an early idea and not something that has been successful in your research lab for some time-an ongoing project. It's really up to you to tailor your application to suit that award mechanism.
Q: The interinstitutional breast cancer research program award. I've talked some to Dr. Miller and plan to talk to her again, but I'm still confused a bit about what's considered to be an existing breast cancer research program in an institution. If you already have people doing breast cancer research, yet somehow you still qualify to submit for the interinstitutional award. I don't know if that was a question or not, you know what I'm saying.
A - Vaday: So this is specifically in the breast cancer research program. So I'm going to talk very generally about it, and I can speak with you more specifically in our breakout session. But this is an award mechanism that's designed to provide training for faculty members at institutions that don't currently have an established breast cancer research program so that they can get mentorship by an established investigator at another institution. When we say established, this initially was an HBCU/MI (Historically Black Colleges and Universities and Minorities Institutions) mechanism. We've expanded it because we recognize that there are some institutions where there isn't a research program in breast cancer and yet they have the strong infrastructure. They just need to have that training, that mentorship, in order to develop a new breast cancer program. So that is the intent of that mechanism. I can talk to you more specifically offline, but that is the intent of the mechanism for that program.
Q: I have two questions on two of the different funding mechanisms. For the team science award, is there an infrastructure that you look for to have in place to manage intellectual property since you are getting so many different scientists together at different institutes? Is there something you look for or something that's required?
A: Carolyn Best, PhD, Program Manager, Prostate Cancer Research Program: We try to spell that out pretty clearly in the program announcement. Typically we do at a minimum ask for an intellectual property plan where you discuss how you are going to manage those issues. You can even submit letters of support from the other institutions with their support for the intellectual material property plan. We typically, unless it's a mechanism that's very focused on product development type of thing, we typically don't ask for MTAs at the stage of application; there may be an exception or two. But again if you look to the specific language in the program announcement to which you are applying, we try to make it clear in what you need to submit at that stage.
Q: Ok, my second question for the postdoctoral awards. If you are awarded with that and then become an assistant professor, is that something you can take with you or transition when you leave?
A: Best: I think it depends upon the program typically. I don't know what it is for the breast program. Vaday: Well, it really depends first of all on your institution because the awards are made to the institution not to you as the PI. So if your institution is willing to relinquish it, we take that into consideration, and we also consider how far along. We realize that for a 3 year award, you don't know if you are going to get a faculty position in 2 years or 2 1/2 years. So because we're really trying to train the next generation and if you are so successful with that award mechanism, getting that award from us, that you get a faculty position, we will try to facilitate that transfer of the award. Best: I also like to note that in every program announcement, we have a section on award administration and we're pretty detailed there in terms of what can and cannot be done with the award-if the PI can be changed, if the institution can be changed-and so you need to look to that section probably before you apply to make sure that you are going to be allowed to do what you are hoping to do with the award. Vaday: I would also encourage you to talk to your grants manager. Every award is assigned to a grants manager and that person is there to help facilitate these kinds of issues. So as soon as you think you might be transitioning, that's the person you want to talk to.
Q: Breast idea collaborative award-it says there that you need to justify that it is collaborative but not individual award. What justification brings the realization that it is collaborative?
A Vaday: So this is actually applicable to the breast and the prostate or any other program that offers a synergistic idea award or some kind of collaborative option. What we are not looking for are the type of proposals where two projects are just kind of put together so that the applicants can submit together and apply for this larger amount of funding. What we are looking for are an integration of projects where each PI really has a distinct but very integrated role with the other PI so that they can answer a common question.
Q: Is this award blindly reviewed or with names?
A: Vaday: This award mechanism is reviewed with the identifications-not blind. Yes, in fact, the personnel and the environment are review criteria that the peer reviewers do evaluate. It's really only our early concept awards like the concept award or the exploration hypothesis development award that are blinded to the reviewers.