How Do Diversity Supplements Impact Careers in Biomedical and Behavioral Research?

1. Executive Summary

This document encapsulates the proceedings of “How Do Diversity Supplements Impact Careers in Biomedical and Behavioral Research?”—the second seminar of the 2022–2023 Scientific Workforce Diversity Seminar Series (SWDSS). The National Institutes of Health (NIH) Chief Officer for Scientific Workforce Diversity (COSWD) hosted the seminar on November 17, 2022. Marie A. Bernard, M.D., COSWD, moderated a panel discussion on how diversity supplements can influence the career pathways of individuals from diverse backgrounds in biomedical and behavioral science. A panel of six invited experts from academic institutions and NIH Institutes and Centers (ICs) shared perspectives and discussed current research on the evaluation, implementation, and effective communication of diversity supplements.

A question-and-answer session, moderated by Dr. Bernard, followed the panel presentations.

This document delineates the main points from the speakers’ presentations and the subsequent discussion on how diversity supplements impact careers in biomedical and behavioral research. The seminar recording and panelists’ presentation materials are available on the COSWD website.

2. Opening Remarks

Marie A. Bernard, M.D., NIH COSWD

Dr. Bernard introduced NIH diversity supplements, which are administrative supplements that support research experiences for individuals from diverse backgrounds by allowing program directors and principal investigators (PIs) with existing NIH research grants to apply for funds to recruit and support students, postdoctoral researchers, and investigators from diverse backgrounds, including those from groups shown to be underrepresented in the U.S. biomedical, clinical, behavioral, and social sciences research enterprise. As stated in the Notice of NIH’s Interest in Diversity, these groups include (but are not limited to) women, individuals from underrepresented racial and ethnic groups shown to be underrepresented in health-related sciences, individuals from disadvantaged groups, and individuals with disabilities. Supplements are also available to established PIs who are or have become disabled and need additional support to complete their NIH-funded research project. When applying for a diversity supplement, applicants must convey that the individual they wish to recruit will enhance scientific workforce diversity. Dr. Bernard emphasized NIH’s interest in better understanding the impact of diversity supplements; the seminar was an opportunity to learn about evaluations completed by various NIH ICs and NIH-wide, as well as hear external perspectives.

3. How Do Diversity Supplements Impact Careers in Biomedical and Behavioral Research?

Ericka Boone, Ph.D., Director, Division of Biomedical Research Workforce, Office of Extramural Research, NIH

Dr. Boone discussed how the NIH Office of Extramural Research (OER) stores and tracks diversity supplement information and aims to improve tracking methods to enhance the searchability and recognition of supplementees. Diversity supplements are permitted for more than 70 different NIH grant award codes. Supplements cover research costs such as salaries, travel support, and supplies up to $100,000 per year, depending on career level. Since 2017, NIH has invested approximately $90 million in diversity supplement awards, supporting about 1,000 individuals each year. Thus, monitoring, tracking, and reporting on diversity supplement applications and awards is a critical process for NIH. However, a significant challenge is storing diversity supplement data within NIH internal systems, searching and evaluating this data, and determining how to make it available outside NIH.

OER is implementing improved methods to enable better capture and storage of structured diversity supplement data elements within NIH systems. In previous years, diversity supplements only listed the name of the PI of the parent grant, which obscured name recognition for the supplement awardee and limited trackability of the awards. Today, diversity supplement awards are issued as separate Type 3 awards, which enhances trackability and connectedness of the awardee to future awards. This change enables greater visibility and recognition and allows us to view NIH-supported career trajectories using a wider lens.

Additionally, NIH uses various system flags and labels to track diversity supplement awards, which is currently the most reliable tracking method. However, flags and labels must be used consistently to enable internal and external data searches. OER is examining other ways to optimize searchability, develop customized reports, and update structured elements to enhance NIH’s ability to track, identify, and account for its diversity supplement data. In addition, NIH now requires all supplementees to have an electronic Research Administration (eRA) Commons ID and will potentially require an Open Researcher and Contributor ID (ORCID), for better internal and external award tracking over time. Dr. Boone concluded by encouraging NIH’s extramural colleagues to learn more about diversity supplements, share supplement information with their trainees, and contact their NIH program officer to learn about diversity supplement priorities across NIH Institutes, Centers, and Offices.

4. NIH Diversity Supplements by Year, Administering Institute, and Medical School

Dowin H. Boatright, M.D., M.B.A., M.H.S., Vice Chair of Research, Department of Emergency Medicine, NYU Grossman School of Medicine

Dr. Boatright presented his team’s research on the awarding and receipt of NIH diversity supplements and how NIH diversity supplement program use changed between 2005 and 2020. Dr. Boatright’s research team analyzed publicly available data from the NIH Research Portfolio Online Reporting Tools, Expenditure and Results (RePORTER) database, focusing on R01-associated diversity supplements due to the high prevalence of R01 grants and their accepted eligibility since the start of the diversity supplement program. From 2005 to 2020, NIH ICs awarded 93,285 R01 grants, of which 2,145 (2.3%) received at least one diversity supplement.¹ Very few diversity supplements were associated with R01 grants before 2011. Between 2011 and 2020, diversity supplements had an 83% compound annual growth rate. The initial rise in diversity supplements coincided with the 2011 publication of an influential paper reporting racial disparities in the receipt of R01 grants² and a 2012 study examining the racial and ethnic patterns of NIH R01 Type 1 award probabilities for physician investigators.³ NIH also established the position of Chief Officer for Scientific Workforce Diversity in 2014.

Dr. Boatright’s team found significant variations across NIH ICs in the percentage of R01 grants with a diversity supplement, even when accounting for each IC’s budget.¹ Trends in diversity supplements by medical schools closely reflected overall trends by fiscal year, with large increases in diversity supplement awards starting in 2012⁴. Two-thirds of diversity supplements were awarded to medical schools in the top 40 NIH-funded medical schools ranked by the Blue Ridge Institute for Medical Research; however, there was no difference in the likelihood of a medical school receiving a diversity supplement award based on NIH funding. The top 40 NIH-funded medical schools also showed considerable school-to-school variations in the percentage of R01 grants awarded diversity supplements, suggesting variation in school prioritization of diversity supplement promotion.

The findings show that the number of diversity supplements awarded between 2005 and 2020 increased exponentially. However, only a small percentage of R01 grants received a diversity supplement during this time, suggesting many opportunities to increase their usage. Dr. Boatright noted that study limitations included the unavailability of certain data from ICs, including the number of applications received and the budget reserved for diversity supplements.

5. NINDS Program Strategy and Outcomes of Funded Diversity Supplementees

Michelle D. Jones-London, Ph.D., Chief, Office of Programs to Enhance Neuroscience Workforce Diversity, National Institute of Neurological Disorders and Stroke, NIH

Dr. Jones-London discussed the National Institute of Neurological Disorders and Stroke (NINDS) diversity supplement program strategy⁵ and analyses of the impact of diversity supplements on the career pathways of supplementees. In 2021, NINDS funded new and continuing diversity supplement awards at a total expenditure of $8.5 million. Growth in the NINDS diversity supplements program can be attributed to increased investment by its leadership and outreach to extramural investigators. To inform researchers of the program, NINDS automatically sends diversity supplement application information to eligible awardees and holds periodic open houses to explain the program goals. In addition to the general NIH parent funding announcement, NINDS leverages other research areas with dedicated funding sources to prioritize diversity supplements in high-impact spaces, such as the Helping to End Addiction Long-term^® (HEAL) Initiative, NINDS mission relevant Alzheimer’s Disease and Related Dementias, and the Brain Research Through Advancing Innovative Neurotechnologies^® (BRAIN) Initiative.

NINDS views diversity supplements as a feeder program and bridge to the next stage in an individual’s career, enabling individuals to gain research experience, publish pilot data and papers, and apply for their own grants. For graduate students, postdoctoral fellows, and junior faculty supplementees, grant terms and conditions stipulate that subsequent year funding is contingent on the receipt of an application, such as an F or a K award. This timing allows funding security in that if an applicant needs to resubmit their application, they can do so while still being covered by their supplement. It also empowers trainees to submit research projects for which they have intellectual ownership. These expectations have led to strong F and K outcomes in the NINDS portfolios.

Between 2017 and 2019, diversity supplementees represented 35 of 81 F and K awardees in the NINDS-funded trainee community, translating into enhanced workforce diversity and further transition mechanisms for diversity supplement recipients, such as F99, K00, and K99/R00 awards. Another analysis comparing the general post-training biomedical and neuroscience workforce to supplementee outcomes indicated that the retention rate in academic research was the same for both groups. However, the diversity supplement program showed a higher rate for participants transitioning to science-related, non-research positions. This means that diversity supplements recipients are using their research and scientific skills in their work but applying them in nonacademic settings. Only 5% of supplementees transitioned to non-science-related positions, a lower percentage than the general workforce. These results demonstrate that the diversity supplement program is achieving its goal of enhancing diversity in biomedicine.

6. Diversity Supplement Awards Analysis, FY2010–2021

Jamelle E. Banks, M.P.H., Senior Health Science Policy Analyst, National Institute on Aging, NIH

Ms. Banks discussed an analysis done by the National Institute on Aging (NIA) to evaluate the impact of its diversity supplement awards, the third such analysis it has performed. Between FY10 and FY21, NIA awarded 416 diversity supplements to existing NIH grants, with 2021 having 85 awardees, the most of any year. Of the 416 awardees, the majority were female (69%), African American (42%) individuals, and Hispanic (40%) individuals, and most were at the pre- (34%) or postdoctoral (37%) career level at the time of their application.

To understand awardee indicators of productivity, NIA assessed data related to individuals who received a diversity supplement between FY10 and FY18, focusing on subsequent grants, publications, and career progression. After receiving a diversity supplement, 53% of supplementees subsequently applied for NIH grants, with 28% receiving funding. Compared to other NIH ICs, NIA received and funded most of these subsequent applications and awards. NIA also looked at subsequent grant funding rates per person by career level, comparing supplementees with NIA awardees at the same career stage. At the pre- and postdoctoral levels, the funding rate was lower for diversity supplement awardees. However, the funding rate for supplementees is comparable at the faculty level. In terms of total subsequent funding, supplementees received less funding than non-supplementees. Ms. Banks stated that NIA awardees might not be the best comparison group because diversity supplement awardees have more individuals who are underrepresented in biomedical sciences. Ideally, NIA would compare the supplement awardees to individuals that applied for but did not receive the supplement award; NIA plans to conduct such analyses in the future.

NIA also analyzed diversity supplement recipients’ publications, another indicator of productivity. From FY10 to FY18, diversity supplement recipients produced 2,493 publications and scored similar relative citation ratios (RCRs), a measure of a publication’s scientific influence, both before and after receiving the diversity supplement. However, several papers published by awardees after receiving their diversity supplement scored exceptionally high RCRs. Another analysis showed that supplementees were more likely to publish on an aging-related topic after receiving the diversity supplement award, which may indicate that supplementees are successfully establishing careers in aging research.

In addition, between FY10 and FY21, most supplementees stayed in research-related fields, and supplementees at all career levels transitioned into a STEM-related field, with 81% holding an academic position. Of those in academic positions, 31% were doctoral students, 18% held postdoctoral positions, and 20% were on a tenure track. Overall, NIA analyses indicate that diversity supplements help awardees achieve future grant success, publish papers after receiving an award, and establish and progress in careers in research-related fields.

7. NIDA Diversity Supplement Program Overview and Goals

Albert Avila, Ph.D., Scientific Diversity Officer, National Institute of Biomedical Imaging and Bioengineering, NIH; previously Director, Office of Diversity and Health Disparities, National Institute on Drug Abuse

Dr. Avila discussed the National Institute on Drug Abuse (NIDA) diversity supplement program, how NIDA tracks supplementees and evaluates the program, and the impact of the NIH Diversity Supplement Professional Development and Networking Workshop. The NIDA diversity supplement program is for individuals at the postbaccalaureate through early-career investigator level and emphasizes applying for subsequent NIH grants to prepare supplementees to pursue independent funding. As part of this emphasis, the NIDA Notice of Award encourages diversity supplement recipients above the postbaccalaureate level to submit an independent application for a fellowship or similar support before the end of the program’s first year. NIDA also follows up with awardees and their mentors throughout the program to encourage grant submissions. NIDA also helps awardees navigate the NIH grant application process, including providing a grant and networking workshop they may attend during their time as a NIDA Diversity Supplement scholar.

NIDA tracks its supplementees through an internal database, which uses a combination of data from NIDA and NIH’s internal grants database. NIDA tracks and analyzes diversity supplementee data, such as race and ethnicity, gender, career stage, current institution, and subsequent grant activity. The database now contains supplementee data going back about 10 years. Dr. Avila stressed that adding data is cumbersome. Therefore, it could be valuable to have a similar database across ICs to streamline the process. This type of database could also enable a near real-time view of how supplementees are progressing and help identify those needing additional support. Dr. Avila used the database to provide a snapshot of 2016–2020 diversity supplementee grant activity and outcomes. During this time, NIDA supported approximately 120 supplementees, who submitted about 130 grants. Approximately 30 grants were funded, so of those who submitted grants, 45% were successful.

NIDA’s program also emphasizes networking and communication. NIDA encourages supplementees to join the NIDA Diversity listserv and Office of Research Training, Diversity, and Disparities newsletter to receive regular communications about travel awards, job opportunities, and scientific society awards to help with their professional development activities. In addition, NIDA invites diversity supplement awardees to attend and present at the NIH Diversity Supplement Professional Development and Networking Workshop. This event aims to foster a diverse, supported, and informed scientific workforce equipped with the knowledge to succeed in the NIH grant process. NIDA expanded the workshop in 2022. Approximately 600 scholars from more than 20 NIH ICs and programs participated in the virtual two-day event, which included poster sessions, professional development opportunities, and breakout sessions organized by career stage. Dr. Avila shared workshop participant feedback, which indicates that the diversity supplements and the workshop empower participants, suggesting that NIDA’s program and outreach are making a difference.

8. Impact of Diversity Supplements

José A. Luchsinger-Stuart, M.D., M.P.H., Professor of Medicine and Epidemiology, Columbia University Irving Medical Center

Dr. Luchsinger-Stuart discussed his experience as a diversity supplement recipient and how the experience benefited his biomedical research career. Dr. Luchsinger-Stuart was awarded a P01 diversity supplement in Risk and Modifiers of Alzheimer’s disease in 1999. He applied for a diversity supplement after completing a fellowship and attending the NIA Summer Institute, now known as the Butler-Williams Scholars Program, which introduces young faculty to the field of aging research. At that time, Dr. Luchsinger-Stuart held a full-time clinical position without paid time for research. The diversity supplement award covered 50% of the research costs, enabling him to pursue a research career.

Dr. Luchsinger-Stuart’s diversity supplement enabled him to follow a “typical” career trajectory. He was subsequently awarded K08 grant funding, renewed his P01 for a new project, and now has independent R01, P, U, and K24 grants. In particular, the K24 grant enabled him to help other individuals from diverse backgrounds apply for independent grants. Dr. Luchsinger-Stuart emphasized that diversity supplements are an effective initial bridge to research career pathways. They are ideal for young faculty that have proper training for a potential research career and need initial protected and mentored time to begin their career.

9. Question-and-Answer Session

Q. What is the future for diversity supplements? What are NIH’s plans for evaluating diversity supplements?

Dr. Boone: As a data-driven organization, NIH will undertake an evaluation of our parent diversity supplement programs to improve the experience of supplementees, as well as strengthen the requirements for applicants. There has been an upward trend of support for these programs, and NIH leadership will continue to encourage ICs to expand their support and funding for supplements.

Q. Can we have more than one diversity supplement in a single R01? Could you share the grants management language?

Dr. Jones-London: I am happy to share the exact grants management language. On the first question, having more than one supplement on a single R01 depends on the NIH IC. At NINDS, we offer multiple supplements for one parent grant. The expectation is that the PI should be explicit about the outcomes from the first investment. If it is too soon to know those outcomes, PIs must demonstrate that their project can be leveraged for the next individual diversity supplement award. We have more specific instructions on our website.

Q. What approaches are used to track trainees during the award in the years after completion? For NIA, was the jump in awards associated with an increase in Alzheimer’s disease (AD) funding?

Ms. Banks: NIA experienced an increase in the budget, which accounts for the increase in funds for the diversity supplement program. However, about 44% of the parent grants for diversity supplements are directed toward AD research. We track individuals using QVR, an internal NIH database, as well as NIA grant history, if applicable. We also use a Google search to match supplementees with the information in our system for career progression.

Q. Do you collect data on why applications are accepted or not accepted to identify trends of potential remaining areas of bias? Do first-time applicants receive special considerations and reviews akin to early-stage investigators?

Dr. Avila: At NIDA, we give special consideration to all our applicants. We provide applicants who are not successful upon their first application a detailed summary of their review and have follow-up calls with them, so that they can succeed in subsequent submissions.

Q. Have you been able to look closely at the gap measures—looking at the difference between underrepresented scientists and well-represented groups in the general pool versus the diversity supplement groups?

Dr. Boatright: Currently, we cannot perform such an analysis. We are in discussions with the Association of American Medical Colleges to get access to their faculty roster. After merging those data with NIH data, we could compare underrepresented faculty who did not receive supplements with well-represented faculty. We would still be limited to medical school faculty; however, they account for two-thirds of NIH funding.

Q. What is being done to make the availability of diversity supplements more widely known?

Dr. Luchsinger-Stuart: Columbia has different diversity initiatives. First, this resource is available to potential applicants and PIs with R01s and other grants. The other audience is the recipients of mentoring awards and training grants. Leaders of research education cores should lead outreach efforts to the public and aspiring scientists from underrepresented groups. Outreach to K24s is also important because those stakeholders are accountable for helping grantees with these mechanisms and present opportunities to make diversity supplement information more widely available.

Dr. Boone: At NIH, we expend a lot of effort on outreach and communication for various funding opportunities. No matter how high our level of engagement is in these efforts, more remains to be done. We need repeat engagement, such as conference participation, webinars, and ambassador programs with people who receive awards, to use as voices for outreach. We have our own internal ambassadors at NIH who engage in a lot of outreach at conferences to share messages about supplements. I want to ask individuals in extramural communities to inform us of better ways for NIH to communicate these opportunities.

Dr. Boatright: Medical schools respond to data as well. Data transparency could help engage medical schools in competing to be the one with the most supplements.

Q. What resource would you recommend to people seeking more information on applying for diversity supplements?

Dr. Boone: I would look at parent funding opportunity announcements (FOAs). Your program officer is your best friend and is dedicated to bringing individuals into the workforce. They have experience in the development of successful applications.

Q. Do you think the diversity supplement program is achieving its goals?

Dr. Avila: Yes, but much more work remains before we achieve our final goals. We have seen a large increase in application numbers. As an agency, we could put more effort and resources into this space to be more effective.

Ms. Banks: I agree. When you consider the goals of the supplements, the NIH data show that progress: awardees remain in STEM-related fields, are publishing more, and are seeking continual grant funding. Continuing to track awardees is important. NIH can consider core metrics to implement across all ICs.

Dr. Boatright: From a medical school standpoint, supplements support diverse science and make money for institutions. Opportunities exist for everyone to win.

Dr. Jones-London: Yes. The program is achieving its goals, but it is one tool in the toolkit. No single program will change the whole ecosystem. We need to look systemically at what can be done. The supplements are an entry point for acquiring individual awards and for networking with the community for outreach about opportunities. A supplement is a great way for a trainee to develop a relationship with NIH and for NIH to hold institutions and PIs accountable for a standard of mentorship.

Dr. Luchsinger-Stuart: The supplements are accomplishing their goals, as evidenced by the data presented in this session. Now, efforts must be made to increase outreach to make more people aware of grant mechanisms.

Q. What are the criteria for reviewing diversity supplements?

Dr. Avila: Most ICs provide information about their specific criteria. For example, NIDA wants to see the next steps and ensure that we are helping to support the applicants.

Dr. Jones-London: Most of the criteria are outlined in the parent FOA. The main component is the mentorship plan, which includes some of the same criteria as mentored K awards. The work must be conducted within the scope of the parent grant. An applicant can contact the program officer for the parent grant. Each IC may provide more details in the instructions as well. My IC hosts a webinar during which we walk potential applicants through the application process.

Q. What message do you want the SWDSS audience to walk away with today?

Dr. Avila: For anyone considering a diversity supplement, do not hesitate to contact us. If you are not sure whom to contact, look at the program announcement and others at NIH. We are here to support you in your career, including individuals from diverse backgrounds. NIH is moving in the right direction. If your PI or mentor is not aware of this opportunity, do not hesitate to mention it to them and others. Spread the word yourself.

Ms. Banks: We should consider how we define success for the diversity supplements and what we want to answer about the program so that we can better track data over time.

Dr. Boatright: Diversity supplements are a powerful tool. Even though we have seen a lot of growth, some data show that supplements are underutilized. So, I encourage everyone to apply.

Dr. Jones-London: The diversity supplement program is a great program for improving diversity in the workforce. This is a great opportunity for PIs to increase diversity in their labs through mentoring individuals at multiple career stages (e.g., students, high school teachers, junior faculty).

Dr. Luchsinger-Stuart: Diversity supplements are a great opportunity. Applicants should provide clarity about the next steps. Supplements also provide practice for submitting other awards that have similar requirements. Four considerations are the science, the candidate, the mentor, and the institution. Going through the supplement is a good way to gauge where applicants are.

References

1.

Hill KA, Desai MM, Chaudhry SI, Nguyen M, Boatright D. NIH diversity supplement awards by year and administering institute. JAMA. 2021;326(23):2427–2429. doi:10.1001/jama.2021.19360 [PMC free article: PMC8693216] [PubMed: 34932089] [CrossRef]

2.

Ginther DK, Schaffer WT, Schnell J, et al. Race, ethnicity, and NIH research awards. Science. 2011;333(6045):1015–1019. doi:10.1126/science.1196783 [PMC free article: PMC3412416] [PubMed: 21852498] [CrossRef]

3.

National Institutes of Health. Draft report of the advisory committee to the director working group on diversity in the biomedical research workforce. Published June 13, 2012. Accessed May 4, 2023. https://acd​.od.nih.gov​/documents/reports​/DiversityBiomedicalResearchWorkforceReport.pdf

4.

Hill KA, Desai MM, Chaudhry SI, et al. National Institutes of Health diversity supplement awards by medical school. J Gen Intern Med. 2023;38(5):1175–1179. doi:10.1007/s11606-022-07849-y [PMC free article: PMC10110783] [PubMed: 36344641] [CrossRef]

5.

Jones-London M. NINDS strategies for enhancing the diversity of neuroscience researchers. Neuron. 2020;107(2):212–214. doi:10.1016/j.neuron.2020.06.033 [PMC free article: PMC7351051] [PubMed: 32652046] [CrossRef]