“So You Have an Under-Reporting Problem?” System Barriers to Incident Reporting

The reporting of safety occurrences and safety-relevant issues and conditions is an essential activity in a learning organisation. Unless people speak up, be it concerns about unreliable equipment, unworkable procedures, or any human performance issue, trouble will fester in the system. In my experience in safety investigation, safety culture and human factors across industries, one of the clearest signs of trouble in a safety-related organisation is a reluctance among staff to report safety issues.

Non-reporting can be hard to detect, especially when managers are disconnected from the work. There may be a built-in motivation not to be curious about a lack of reports: under-reporting gives the illusion that an organisation has few incidents or safety problems, and this may give a reassurance of safety (while a preoccupation with failure, or ‘chronic unease’, might be true for those who have worked in high reliability organisations). Where it is discovered that relevant events or issues are not being reported, too often this is seen as a sign of a person, or team, gone bad – ignorant, lazy or irresponsible. This might be the case, but only if ten or so other issues have been discounted. I have tried to distill these below, along with some relevant Safety Culture Discussion Cards (Second Edition). A selection is below, but others are also relevant.

1. Is the purpose of reporting understood, and is it consistent with the purpose of the work and organisation?

Yes / No / Don’t Know

This is the first and most fundamental thing. As Donella Meadows noted in her book Thinking in Systems, “The least obvious part of the system, its function or purpose, is often the most crucial determinant of the system’s behaviour“. The purpose of reporting may be completely unknown, vague, ambiguous, or (most likely) seemingly inconsistent with the work or with the purpose of at least part of organisation (e.g. a department or division), and its related goals. The purpose of reporting may relate primarily to monitoring and compliance with rules, regulations, standards or procedures. The purpose may relate to checking against organisational goals and numerical targets (e.g. relating to equipment failures, ‘errors’, safety outcomes, etc). In both cases, there is probably little perceived value to the reporter (and may be little value to the organisation) and incompatible purposes, and hence a disincentive to report. From a more useful viewpoint, the purpose of reporting may be viewed in terms of learning and improving how the work works. Things like demands, goal conflicts, performance variability and capability, flow and conditions become the things of interest. In other words, the purpose of reporting should be compatible with the purpose of the work itself.

2. Are people trated fairly (and not blamed or published) when reporting?

Yes / No / Don’t Know

An unjust culture is probably the most powerful reason not to report. If people are blamed or punished for their good-will performance, others will not want to report. The effect has a long shelf-life; these particular stories live on in the organisation, serving as disincentives long after the incident. Punishment may take several froms: inquisitorial investigation interview, ‘explain yourself’ meeting with the boss, formal warning, public admonishment and shaming, non-renewal of contract, loss of job, prosecution…even vigilante revenge attack. A culture of fear may be cultivated by designed organisational processes. The independent Rail Safety and Standards Board (RSSB) estimated that up to 600 RIDDOR incidents were not reported between 2005 and 2010 due to pressure from Network Rail. One key reason was that contractors were under pressure to meet accident targets – a clear disincentive to report, built into the system. The fears of staff were reasonable and took various forms similar to those listed above. In the case of possible punishment by external organisations, a question mark arises over internal support by the organisation. In these cases, if the organisation is not supportive (legally and emotionally), there is motivation not to report.

3. Are there no incentives to under-report?  

Yes / No / Don’t Know

Messages from organisations that accidents, incidents, hazards, etc., ‘must be reported’ are cancelled out immediately by institutional perverse incentives not to report. They are often linked to targets of various kinds (either clearly related to safety or not), league tables which count safety occurrences, bonuses, prizes, etc. Many incentives combine reward and punishment and so are devastatingly effective in preventing reporting. The US OSHA notes in a recent whistleblower memo regarding safety incentive and disincentive policies and practices that, “some employers establish programs that unintentionally or intentionally provide employees an incentive to not report injuries“. Perverse incentives were identified by RSSB in Network Rail’s (at least as it operated then) Safety 365 Challenge, in which “staff and departments were rewarded for having an accident-free year with gifts of certificates and branded fleeces and mugs. But failure to get a certificate could lead to staff and departments being downgraded…” (as reported here). Anson Jack, the RSSB’s director of risk, noted that the initiatives together with the culture at Network Rail led to unintended consequences of under-reporting.

4. Do people know how to report, have good access to a usable reporting method?  

Yes / No / Don’t Know

It is easy to assume that people have the relevant information and instruction on how to report, but especially with a complicated reporting systemor form, it’s worth asking whether people really understand how to report. Complicated forms and unusable systems are off-putting, as is asking for help from colleagues or a supervisors (especially in a stressed environment). Even when people know how to report, if the reporting system is hard to access or requires excessive or seemingly irrelevant input, then you have accessibility and usability barriers in the system.

5. Are people given sufficient time to make the report?  

Yes / No / Don’t Know

Reporting incidents and safety issues needs time to think and time to write. Ideally, the report allows the person to tell the story of what happened, not just tick some boxes, and allows then to reflect on  the system influences. The time provided for the activity will send a message to the person about how important it is. If people have to report in their breaks from operational duty or after hours, then under-reporting has to be expected.

6. Do people have appropriate privacy and confidentiality when reporting? 

Yes / No / Don’t Know

Reporting safety related issues and events can be sensitive in many ways. The issue may be serious, with possible further consequences or may cause some embarrassment, awkwardness or ill-feeling. If people have to go to their manager’s office or to a public PC in the café, expect them to be put off. Beyond privacy, confidentiality is crucial. Incident reporting programmes that have switched to confidential reporting programmes have seen significant increases in reports, and not just because of a reduction in fear of retribution. As Dekker & Laursen (2007) reported, with non-confidential, punitive reporting systems, people may actually be very willing to report – at a very superficial level, focusing on the first story (‘human error’) not the second story (systemic vulnerabilities). Even with confidential systems, the first question that comes to mind for some is not about what, how or why, but rather who. Even confidential reporting systems often require identifying details (such as a name, or else date, time, shift, location, etc), which might be used to try to identify the reporter. Confidentiality shouldn’t be an issue in a culture which is fair, open and values learning, but it seems to remain key to encouraging reporting.

7. Are the issues or occurrences investigated by independent, competent, respected individuals?  

Yes / No / Don’t Know

If the investigator is not independent(and is instead subject to interference), if he or she lacks training and competency in investigation, or is simply not respected, then under-reporting will occur. The location of the investigation function within the organisation will be relevant; independence must be in reality, not just on an organisation chart and in a safety management manual. Ideally, investigators would be carefully selected and would have chosen the role, rather than being forced to do it. Once they are in the role, training in human factors and organisational factors is useful, but even more useful is a systems thinking and humanistic approach, with values including empathy, respect and genuineness.

8. Are reporters actively involved and informed at every stage of the investigation?  

Yes / No / Don’t Know

The best investigations, in my experience, involve reportees (and others involved) properly in the investigation. The reporter, despite sharing the same memory and cognitive biases as all of us (including investigators), is essential to tell their story, make sense of the issues, and think about   possible fixes. How those involved understand the event (including different and seemingly incompatible versions of accounts, which will naturally arise from different perspectives) gives valuable information. Whether those involved are seen as co-investigators or subjects will affect the result and the likelihood of reporting. During and after the investigation, a lack of feedback is probably the most common system problem; often the result of flaws in the safety management system or an under-resourced investigation team. More generally, people need to see the results of investigations in order to trust them. Not providing access to reports may confirm fears about reporting. On the other end of the scale, overwhelming people with batches of reports and forcing them to read lengthy and sometimes irrelevant reports will not help.

9. Does anything improve as a result of investigations, and are the changes communicated properly?  

Yes / No / Don’t Know

The vast majority of organisational troubles and opportunities for improvement are due to the design of the system (94% if you accept Demming’s estimate, p. 315), not the individual performance of the workers. If occurrence reports lead to no systemic changes, it seems nearly pointless to report. Often, individuals have reported the same issue before, to no effect. This teaches them that reporting is pointless, going back to Question 1.Even if system changes are made following reports, not communicating to the wider population, via communication channels that they use, means that people may not know they changes have happened, or that they resulted from reporting.

10. Is there a local culture of reporting, where reporting is the norm and encouraged by colleagues and supervisors?  

Yes / No / Don’t Know

People naturally want to fit in. If colleagues and supervisors discourage reporting, as is sometimes the case, then individuals will be uncomfortable, and will have to balance feelings of responsibility against a need to get along with colleagues. The answer to this last question will nearly always be dependent on the answers to the previous questions, though.

If your organisation has a problem with under-reporting, the chances are there are a few No’s and Don’t Know’s in the answers to the above. In nearly all cases, under-reporting is a system problem. If you’re not sure, and want to find out, ask those who could report about the what gets in the way of reporting (for other people, of course). The Safety Culture Discussion Cards might help.

Author: stevenshorrock

This blog is written by Dr Steven Shorrock. I am interdisciplinary humanistic, systems and design practitioner interested in human work from multiple perspectives. My main interest is human and system behaviour, mostly in the context of safety-related organisations. I am a Chartered Ergonomist and Human Factors Specialist with the CIEHF and a Chartered Psychologist with the British Psychological Society. I currently work as a human factors and safety specialist in air traffic control in Europe. I am also Adjunct Associate Professor at University of the Sunshine Coast, Centre for Human Factors & Sociotechnical Systems. I blog in a personal capacity. Views expressed here are mine and not those of any affiliated organisation, unless stated otherwise. You can find me on twitter at @stevenshorrock or email contact[at]humanisticsystems[dot]com.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.