What Measures Can Companies Use to Evaluate Safety Management Practices and Identify Opportunities for Improvement?

Posted on by Libby L. Moore, PhD, Steven J. Wurzelbacher, PhD, I-Chen Chen, PhD, Michael P. Lampl, MS, Steven J. Naber, PhD

 

What measures can companies use to evaluate safety efforts and identify opportunities for improvement?  The most commonly used measures of safety performance are lagging indicators such as injury counts and costs.1,2 While lagging indicators can be beneficial, using them as the only measure of safety can be a barrier to safety improvement. For example, companies with few injuries may not have enough information to identify trends3, as is often the case for smaller companies.  In addition, low injury rates occur for reasons other than an absence of hazards, including underreporting and chance.4 Further, these measures are not preventive because they rely on information from adverse outcomes that have already happened.1 Injuries can point to a need for safety improvement. However, because they provide little information on injury causes or guidance for improving work conditions, on their own they are not sufficient for directing safety efforts.5,6

Ideally, companies use several measures that together indicate how well they’re doing and why they are at their current level of safety performance.4 In recent years, employers and safety professionals have increased their use of leading indicators to assess safety management effectiveness. Leading indicators are factors that occur in advance of lagging indicators. They include measures of activities, practices, and programs for preventing injuries and minimizing severity when injuries do occur.

Because leading indicators are used to make decisions about safety efforts and resource allocation, selecting the right indicators is important.7 They should be valid, reliable, and help identify deficiencies in safety management programs. Improvement of deficiencies should prevent injuries and lead to improvement of other related lagging indicators3,8,9 and increase overall organizational performance.1 An improvement in outcomes following an improvement in leading indicators demonstrates an association of leading with lagging indicators. This association is a necessary feature of leading indicators.1,6,7

Numerous leading indicator questionnaires have been developed to evaluate organizational safety management and many have been found to be valid and reliable. However, many were evaluated using cross-sectional data collected for research purposes. Because cross-sectional data look at a fixed point in time, the influences between leading indicators and injury outcomes can be difficult to determine. Further, while data collected for research purposes provide information about leading indicators, evaluating leading indicator questionnaires in the practical context in which they will be used is also beneficial.

NIOSH researchers examined the reliability and validity of the Ohio Bureau of Workers’ Compensation (OHBWC) SH-26 Safety Management Self-Assessment Questionnaire as a leading indicator tool. Results were recently published in the Journal of Safety Research10. Employers who were enrolled in any of five OHBWC grant and incentive programs completed and submitted the SH-26 questionnaire to OHBWC annually as a requirement of the programs. OHBWC provided questionnaire data from 2012 – 2015 to NIOSH, which researchers used in the study. OHBWC developed the questionnaire to provide policyholders (with whom OHBWC safety consultants do not necessarily have direct contact) with a method to facilitate safety management self-improvement. In addition, questionnaire responses provide OHBWC consultants with information about employers and are a catalyst for engagement when consultants provide on-site assistance.

The questionnaire contains 30 statements and 32 hazards that employers use to identify safety program management elements and hazards for improvement. The person in each company most familiar with the safety and claims management process completed the questionnaire. They rated on a scale the extent to which program element statements described their organization’s programs and practices, and indicated which hazards were present in their work operations. OHBWC provided NIOSH with workers’ compensation claims data from these same employers, which researchers used as lagging outcomes.

Statistical analyses indicated that higher ratings on an employee health promotion/wellness scale were associated with lower WC claim rates as predicted. Ratings on two scales (i.e., safety and health training and education, and return to work [RTW] practices) were unexpectedly associated with higher WC claim rates. Analyses indicated that previous and current years’ Safety and Health Training and Education and RTW practices predicted current and future years’ WC claim outcomes. In addition, current year WC claim outcomes predicted Safety and Health Training and Education and RTW practices in the future. These findings suggest that WC outcomes may have motivated employers to develop or strengthen their training and RTW practices. These practices may have encouraged early reporting of discomfort or injuries as a means for reducing injury severity, which may have led to higher claim rates during the study period. An additional finding was that for all scale items, more than 15% of respondents selected the highest rating. This may have occurred because employers, knowing OHBWC could view their responses, wanted to create a favorable impression of their companies.

Twelve of 32 hazards assessed on the questionnaire were associated with higher claim rates or costs as predicted.  The hazards are:

  • Power/brake/forging press
  • Insect bites, stings, poisonous vegetation
  • Airborne contaminants
  • Confined spaces
  • Trenching and excavation
  • Exposure to electrical hazards
  • Powered tools
  • Elevated noise levels
  • Exposure to chemicals, hazardous substances
  • Slips/trips/falls
  • Welding, brazing, soldering, molten metal
  • Sustained awkward postures

 

This study contributed a new perspective to leading indicator research by analyzing several years of employer questionnaire and WC claims data. Analyses provided insight into associations between leading and lagging indicators, emphasizing the importance of both for safety improvement. Safety and health management program ratings were predictive of WC claim outcomes, although hazard assessments were a stronger leading indicator.

Overall, this study supported the use of employer-completed hazard assessment questionnaires for targeting and prioritizing improvement efforts. Employer-completed safety management scales may be useful for directing improvement efforts for employers themselves and for external safety consultants, although the conditions under which questionnaires are completed, including submission to insurers, require additional consideration.

We would be interested in your experience and opinions on the following. Please use the comment section below.

  • What leading or lagging indicators are employers using to evaluate safety management practices and identify improvement needs?
  • Are they effective for preventing injuries and creating a safer workplace?
  • What research could be done to improve indicators?

 

Libby L. Moore, PhD, is a Health Scientist in the NIOSH Center for Workers’ Compensation Studies (CWCS).

Steve Wurzelbacher, PhD, is the Manager of the NIOSH Center for Workers’ Compensation Studies (CWCS).

I-Chen Chen, PhD, is a Statistician in the NIOSH Division of Field Studies and Engineering (DFSE).

Michael P. Lampl, MS, is the Director of Research & Grants with the Ohio Bureau of Workers’ Compensation (OHBWC).

Steven J. Naber, PhD, is the Business Intelligence & Analytics Manager with the Ohio Bureau of Workers’ Compensation (OHBWC).

References

  1. Center for Safety & Health Sustainability & American Industrial Hygiene Association. (2020). Best practice guide for leading health metrics in occupational health and safety programs. https://aiha-assets.sfo2.digitaloceanspaces.com/AIHA/resources/Guidance-Documents/Best-Practice-Guide-for-Leading-Health-Metrics-in-Occupational-Health-and-Safety-Programs-Guidance-Document.pdf.
  2. Reiman, T., & Pietikainen, E. (2012). Leading indicators of system safety – Monitoring and driving the organizational safety potential. Safety Science, 50, 1993 – 2000.  DOI: 10.1016/j.ssci.2011.07.015
  3. Amick, B. C., III, & Saunders, R. (2013). Developing leading indicators of work injury and illness. Toronto: Institute for Work & Health.  https://www.iwh.on.ca/sites/iwh/files/iwh/reports/iwh_issue_briefing_leading_indicators_2013.pdf.
  1. Health & Safety Executive. (2001). A guide to measuring health & safety performance. https://www.hse.gov.uk/opsunit/perfmeas.pdf.
  1. Erikson, S. J. (2009). Letter to the Editor: Performance indicators. Safety Science, 47, 468.
  1. Wurzelbacher, S., & Jin, Y. (2011). A framework for evaluating OSH program effectiveness using leading and trailing metrics. Journal of Safety Research, 42, 199 – 207. doi.org/10.1016/j.jsr.2011.04.001.
  1. Robson, L. S., Ibrahim, S., Hogg-Johnson, S., Steenstra, I. A., Van Eerd, D., & Amick, B. C., III. (2017). Developing leading indicators from OHS management audit data: Determining the measurement properties of audit data from the field. Journal of Safety Research, 61, 93-103. doi: 10.1016/j.jsr.2017.02.008.
  1. Pawlowska, Z. (2015). Using lagging and leading indicators for the evaluation of occupational safety and health performance in industry. International Journal of Occupational Safety and Ergonomics, 21(3), 284 – 290.  doi=10.1080/10803548.2015.1081769.
  1. Sinelnikov, S., Inouye, J., & Kerper, S. (2015). Using leading indicators to measure occupational health and safety performance.  Safety Science, 72, 240 – 248.  doi.org/10.1016/j.ssci.2014.09.010.
  1. Moore, L. L., Wurzelbacher S. J., Chen I. C., Lampl M. P., Naber S. J. (2022). Reliability and validity of an employer-completed safety hazard and management assessment questionnaire.  Journal of Safety Research, 81, 283 – 296.  doi.org/10.1016/j.jsr.2022.03.005.

 

Posted on by Libby L. Moore, PhD, Steven J. Wurzelbacher, PhD, I-Chen Chen, PhD, Michael P. Lampl, MS, Steven J. Naber, PhD

2 comments on “What Measures Can Companies Use to Evaluate Safety Management Practices and Identify Opportunities for Improvement?”

Comments listed below are posted by individuals not associated with CDC, unless otherwise stated. These comments do not represent the official views of CDC, and CDC does not guarantee that any information posted by individuals on this site is correct, and disclaims any liability for any loss or damage resulting from reliance on any such information. Read more about our comment policy ».

    there is a new ANSI document, Z16.1 that presents the new measure of safety performance. while the self assessment questionnaire is a good means to measure, management systems audits could do the same thing. The z16 standard uses the term “balanced set of metrics”. the publication came out in April and should have been used as a reference for this article, as it updates the current understanding of how to measure safety performance.

    Thank you for your comments and for bringing attention to the new ANSI/ASSP Z16.1 Standard. The standard describes a comprehensive process for employers to use to develop leading, lagging, and impact metrics tailored to their individual organizations. The metrics can be used to identify deficiencies in their occupational safety and health management systems (OSHMS) and make and track improvements to those systems. Overall, the standard provides excellent information on performance metrics and ways to use them. The OHBWC SH-26 Management Self-Assessment Questionnaire, which covers many of the key components of OSHMS, was developed for employers to quickly evaluate their OSHMS. It has wide accessibility, and its demonstrated validity and reliability differentiate it from some similar tools that have not been evaluated for their measurement properties.

Post a Comment

Your email address will not be published.

All comments posted become a part of the public domain, and users are responsible for their comments. This is a moderated site and your comments will be reviewed before they are posted. Read more about our comment policy »

Page last reviewed: July 7, 2022
Page last updated: July 7, 2022