Portfolio Management at NIOSH

We are often asked how the National Institute for Occupational Safety and Health accomplishes its mission of generating new knowledge in the field of occupational safety and health and transferring that knowledge into practice globally. Furthermore, how do we know we are doing a good job? This blog summarizes how we carry out our mission and how we are “graded” for our performance. These are not simple answers. As such, this blog is longer than most. For your convenience we have broken it down into the following sections: setting priorities; the sector approach; generating new knowledge; transferring research to practice; and evaluating progress.

NIOSH accomplishes its mission through a portfolio of programs. Each of these programs selects research goals that are of the most relevance to real world problems, conducts research of the highest scientific quality that is directed by those goals, and measures the impact that its research has in the daily lives of workers. NIOSH research is carried out through partnership with labor, industry, academia, occupational safety and health practitioners, professional societies, insurance companies, and with government agencies.

Importantly, NIOSH chose to refer to its collection of programs as a portfolio in order to draw a comparison with financial portfolios. Like financial portfolios that have to be evaluated for short term and long term performance, the NIOSH portfolio of programs must be closely managed to ensure that the taxpayers are achieving a good return on their investment in occupational safety and health research. Transparent accountability is just as crucial for research performance as it is for financial performance.

Setting Research Priorities for the Nation

To identify basic and applied research needs that are the most relevant for the problems of today and the problems of the future, NIOSH established in 1996 a public-private framework known as the National Occupational Research Agenda (NORA). NORA is a multipartite partnership designed to develop an agenda for occupational safety and health research for the United States. As the sole occupational safety and health research agency in the United States government, NIOSH serves as the steward of NORA. In this role NIOSH encourages partner involvement, convenes discussions of research priorities and serves as the funding conduit for taxpayer-supported research. Since 1996, NORA has become the framework through which occupational safety and health research at NIOSH and in academia is carried out.

The Sector Approach

In 2006, NORA entered its second decade (2006–2016) organized around eight major industrial categories to better move research to practice within each major industry sector group. Sector definitions follow the North American Industry Classification System (NAICS), which has replaced the U.S. Standard Industrial Classification (SIC) system. NAICS provides definitions for 20 sectors which NIOSH aggregated into eight sector groups as follows:

• Agriculture, Forestry & Fishing
• Construction
• Healthcare & Social Assistance
• Manufacturing
• Mining
• Services
• Transportation, Warehousing & Utilities
• Wholesale and Retail Trade

A group of NIOSH scientists and partners constitutes each of eight NORA Sector Councils. Each Council sets priorities for research and drafts its own strategic goals (in terms of intermediate and ultimate outcomes), timelines, and performance measures. The priorities set by the Sector Councils provide guidance to the entire occupational safety and health community for moving research into practice in that particular industry sector. In addition, a Cross-sector Research Council composed of NIOSH and partner representatives ensures good coordination among the eight Sector Councils. If you would like to get involved in any of the Sector Research Councils, please contact the NORA coordinator.

In support of the sector programs, NIOSH directly manages 24 cross-sector programs in its portfolio. The cross-sector programs support the NORA Sector Council goals, as well as supporting their own goals to meet persistent and emerging issues in occupational safety and health.

Generating New Knowledge

Once priorities are set in each industry sector, NIOSH scientists (intramural) and university researchers (extramural) compete for research funding in each area. Research activities are measured for how many outputs they generate (e.g., papers, presentations, conferences, workplace interventions) as well as how well intermediate (e.g., adoption of an improved practice, development of a new workplace standard, reduction in a risk factor that leads to injury or illness) and ultimate (e.g., reduction in a particular type of injury or illness) outcomes are achieved. Since ultimate outcomes often occur through the operation of several different causes, it is difficult to isolate the contribution that a particular research output made to specific injury or illness reduction. Intermediate outcomes are easier to isolate from a causal perspective and are the preferred measure of performance or impact.

Transferring Research Knowledge into Practice

Research to Practice (r2p) focuses on the transfer and translation of research findings, technologies, and information into highly effective prevention practices and products for use in the workplace. The most important aspects of a successful r2p effort are (1) involving partners throughout all phases of the research processfrom conceiving, planning, conducting, translating, through evaluating research; and (2) building r2p principles into the evaluation system for each intramural and extramural research project and program. In NIOSH, the goal of r2p is to reduce illness and injury by increasing workplace use of NORA research findings. NIOSH scientists use many strategies for transferring research findings into practice including development of practical resource documents and training materials; media coverage; and promoting the commercialization of inventions through patenting, marketing, and licensing.

Evaluating Progress and Getting a Grade

NIOSH understands that independent, expert review is one of the most valid and accepted methods of evaluating scientific research programs. In 2005, NIOSH requested that the National Academies (NA) undertake an evaluation of eight NIOSH research programs. The NA were asked to evaluate not only how relevant NIOSH programs were for tackling the occupational safety and health problems of today and tomorrow, but also the impact NIOSH research has had in reducing the risk of worker injury, illness, and death and how well the programs are positioned to respond to emerging issues. NIOSH asked the NA to give each program a numeric “grade” for relevance and impact (on a scale from 1 to 5, where 5 is the highest) and to provide recommendations on how best to improve the program. More information on the the NA evaluation can be found on the NIOSH website.

To date, NA panels have evaluated the following programs in the NIOSH portfolio:

• Hearing Loss Prevention—Relevance: 3; Impact: 4
• Mining—Relevance: 4; Impact: 4
• Agriculture, Forestry, and Fishing—Relevance: 4; Impact: 3
• Respiratory Diseases—Relevance: 5; Impact: 4
• Health Hazard Evaluations—Score pending
• Personal Protective Technology—Score pending
• Construction—Score pending
• Traumatic Injury—Score pending

Conclusion

NIOSH believes that setting research goals in partnership with multiple stakeholders, quantitatively measuring how well NIOSH performs in achieving those goals, and independently evaluating all research programs in its portfolio for relevance and impact is the best formula for ensuring that NIOSH stakeholders can be confident that they are receiving a return on their investment in occupational safety and health research.

Relevance, scientific quality and impact are the three most important attributes of a good research program and NIOSH aims to have the best.

Thank you for your support of NIOSH!

—John Howard MD
NIOSH Director