“So you have an under-reporting problem?” System barriers to incident reporting.

The reporting of safety occurrences and safety-relevant issues and conditions is an essential activity in a learning organisation. Unless people speak up, be it concerns about unreliable equipment, unworkable procedures, or any human performance issue, trouble will fester in the system. In my experience in safety investigation, safety culture and human factors across industries, one of the clearest signs of trouble in a safety-related organisation is a reluctance among staff to report safety issues.

Non-reporting can be hard to detect, especially when managers are disconnected from the work. There may be a built-in motivation not to be curious about a lack of reports: under-reporting gives the illusion that an organisation has few incidents or safety problems, and this may give a reassurance of safety (while a preoccupation with failure, or ‘chronic unease’, might be true for those who have worked in high reliability organisations). Where it is discovered that relevant events or issues are not being reported, too often this is seen as a sign of a person, or team, gone bad – ignorant, lazy or irresponsible. This might be the case, but only if ten or so other issues have been discounted. I have tried to distill these below, along with some relevant Safety Culture Discussion Cards.

“So you have an under-reporting problem?” Questions for the curious.

1. Is the purpose of reporting understood, and is it consistent with the purpose of the work and  organisation?  

Yes / No / Don’t Know

This is the first and most fundamental thing. As Donella Meadows noted in her book Thinking in Systems, “The least obvious part of the system, its function or purpose, is often the most crucial determinant of the system’s behaviour“. The purpose of reporting may be completely unknown, vague, ambiguous, or (most likely) seemingly inconsistent with the work or with the purpose of at least part of organisation (e.g. a department or division), and its related goals. The purpose of reporting may relate primarily to monitoring and compliance with rules, regulations, standards or procedures. The purpose may relate to checking against organisational goals and numerical targets (e.g. relating to equipment failures, ‘errors’, safety outcomes, etc). In both cases, there is probably little perceived value to the reporter (and may be little value to the organisation) and incompatible purposes, and hence a disincentive to report. From a more useful viewpoint, the purpose of reporting may be viewed in terms of learning and improving how the work works. Things like demands, goal conflicts, performance variability and capability, flow and conditions become the things of interest. In other words, the purpose of reporting should be compatible with the purpose of the work itself. See Card 1c, Card 1g and Card 4c.

2. Are people trated fairly (and not blamed or published) when reporting?  

Yes / No / Don’t Know

An unjust culture is probably the most powerful reason not to report. If people are blamed or punished for their good-will performance, others will not want to report. The effect has a long shelf-life; these particular stories live on in the organisation, serving as disincentives long after the incident. Punishment may take several froms: inquisitorial investigation interview, ‘explain yourself’ meeting with the boss, formal warning, public admonishment and shaming, non-renewal of contract, loss of job, prosecution…even vigilante revenge attack. A culture of fear may be cultivated by designed organisational processes. The independent Rail Safety and Standards Board (RSSB) estimated that up to 600 RIDDOR incidents were not reported between 2005 and 2010 due to pressure from Network Rail. One key reason was that contractors were under pressure to meet accident targets – a clear disincentive to report, built into the system. The fears of staff were reasonable and took various forms similar to those listed above. In the case of possible punishment by external organisations, a question mark arises over internal support by the organisation. In these cases, if the organisation is not supportive (legally and emotionally), there is motivation not to report. See Card 3h, Card 3d and Card 3g and Card 3f.

3. Are there no incentives to under-report?  

Yes / No / Don’t Know

Messages from organisations that accidents, incidents, hazards, etc., ‘must be reported’ are cancelled out immediately by institutional perverse incentives not to report. They are often linked to targets of various kinds (either clearly related to safety or not), league tables which count safety occurrences, bonuses, prizes, etc. Many incentives combine reward and punishment and so are devastatingly effective in preventing reporting. The US OSHA notes in a recent whistleblower memo regarding safety incentive and disincentive policies and practices that, “some employers establish programs that unintentionally or intentionally provide employees an incentive to not report injuries“. Perverse incentives were identified by RSSB in Network Rail’s (at least as it operated then) Safety 365 Challenge, in which staff and departments were rewarded for having an accident-free year with gifts of certificates and branded fleeces and mugs. But failure to get a certificate could lead to staff and departments being downgraded…(as reported here). Anson Jack, the RSSB’s director of risk, noted that the initiatives together with the culture at Network Rail led to unintended consequences of underreporting. See Card 1f and Card 4h.

4. Do people know how to report, have good access to a usable reporting method?  

Yes / No / Don’t Know

It is easy to assume that people have the relevant information and instruction on how to report, but especially with a complicated reporting system or form, it’s worth asking whether people really understand how to report. Complicated forms and unusable systems are off-putting, as is asking for help from colleagues or a supervisors (especially in a stressed environment). Even when people know how to report, if the reporting system is hard to access or requires excessive or seemingly irrelevant input, then you have accessibility and usability barriers in the system. See Card 3j.

5. Are people given sufficient time to make the report?  

Yes / No / Don’t Know

Reporting incidents and safety issues needs time to think and time to write. Ideally, the report allows the person to tell the story of what happened, not just tick some boxes, and allows then to reflect on  the system influences. The time provided for the activity will send a message to the person about how important it is. If people have to report in their breaks from operational duty or after hours, then under-reporting has to be expected. See Card 2h.

6. Do people have appropriate privacy and confidentiality when reporting? 

Yes / No / Don’t Know

Reporting safety related issues and events can be sensitive in many ways. The issue may be serious, with possible further consequences or may cause some embarrassment, awkwardness or ill-feeling. If people have to go to their manager’s office or to a public PC in the café, expect them to be put off. Beyond privacy, confidentiality is crucial. Incident reporting programmes that have switched to confidential reporting programmes have seen significant increases in reports, and not just because of a reduction in fear of retribution. As Dekker & Laursen (2007) reported, with non-confidential, punitive reporting systems, people may actually be very willing to report – at a very superficial level, focusing on the first story (‘human error’) not the second story (systemic vulnerabilities). Even with confidential systems, the first question that comes to mind for some is not about what, how or why, but rather who. Even confidential reporting systems often require identifying details (such as a name, or else date, time, shift, location, etc), which might be used to try to identify the reporter. Confidentiality shouldn’t be an issue in a culture which is fair, open and values learning, but it seems to remain key to encouraging reporting. See Card 3l and Card 3f. 

7. Are the issues or occurrences investigated by independent, competent, respected individuals?  

Yes / No / Don’t Know

If the investigator is not independent(and is instead subject to interference), if he or she lacks training and competency in investigation, or is simply not respected, then under-reporting will occur. The location of the investigation function within the organisation will be relevant; independence must be in reality, not just on an organisation chart and in a safety management manual. Ideally, investigators would be carefully selected and would have chosen the role, rather than being forced to do it. Once they are in the role, training in human factors and organisational factors is useful, but even more useful is a systems thinking and humanistic approach, with values including empathy, respect and genuineness. See Card 3n and Card 2a.

8. Are reporters actively involved and informed at every stage of the investigation?  

Yes / No / Don’t Know

The best investigations, in my experience, involve reportees (and others involved) properly in the investigation. The reporter, despite sharing the same memory and cognitive biases as all of us (including investigators), is essential to tell their story, make sense of the issues, and think about   possible fixes. How those involved understand the event (including different and seemingly incompatible versions of accounts, which will naturally arise from different perspectives) gives valuable information. Whether those involved are seen as co-investigators or subjects will affect the result and the likelihood of reporting. During and after the investigation, a lack of feedback is probably the most common system problem; often the result of flaws in the safety management system or an under-resourced investigation team. More generally, people need to see the results of investigations in order to trust them. Not providing access to reports may confirm fears about reporting. On the other end of the scale, overwhelming people with batches of reports and forcing them to read lengthy and sometimes irrelevant reports will not help. See Card 3k, Card 3m and Card 8a.

9. Does anything improve as a result of investigations, and are the changes communicated properly?  

Yes / No / Don’t Know 

The vast majority of organisational troubles and opportunities for improvement are due to the design of the system (94% if you accept Demming’s estimate, p. 315), not the individual performance of the workers. If occurrence reports lead to no systemic changes, it seems nearly pointless to report. Often, individuals have reported the same issue before, to no effect. This teaches them that reporting is pointless, going back to Question 1. Even if system changes are made following reports, not communicating to the wider population, via communication channels that they use, means that people may not know they changes have happened, or that they resulted from reporting. See Card 3i, Card 3c, Card 6d and Card 6h.

10. Is there a local culture of reporting, where reporting is the norm and encouraged by colleagues and supervisors?  

Yes / No / Don’t Know

People naturally want to fit in. If colleagues and supervisors discourage reporting, as is sometimes the case, then individuals will be uncomfortable, and will have to balance feelings of responsibility against a need to get along with colleagues. The answer to this last question will nearly always be dependent on the answers to the previous questions, though. See Card 3a and Card 3o.

If your organisation has a problem with under-reporting, the chances are there are a few No’s and Don’t Know’s in the answers to the above. In nearly all cases, under-reporting is a system problem. If you’re not sure, and want to find out, ask those who could report about the what gets in the way of reporting (for other people, of course). The Safety Culture Discussion Cards might help.

About stevenshorrock

I am a systems ergonomist/human factors specialist and work psychologist with a background in practice and research in safety-critical industries. My main interest is human and system behaviour in the context of safety-related organisations. I seek to enable improvement via a combination of systems thinking, design thinking and humanistic thinking. I am a Chartered Ergonomist and Human Factors Specialist with the CIEHF and a Chartered Psychologist with the British Psychological Society. I currently work as a human factors and safety specialist in air traffic control in Europe. I am also Adjunct Associate Professor at University of the Sunshine Coast, Centre for Human Factors & Sociotechnical Systems. I blog in a personal capacity. Views expressed here are mine and not those of any affiliated organisation, unless stated otherwise. You can find me on twitter at @stevenshorrock
This entry was posted in Culture, Safety, systems thinking. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s