Using the Safety Culture Discussion Cards to help understand textual data

What we call our data are really our own constructions of other people’s constructions of what they and their compatriots are up to’ (Geertz, 1973)

Probably the most common approach to trying to understand safety culture is via safety climate questionnaires, usually comprising a set of items with a Likert-scale to indicate the level of agreement with each item. Unfortunately, such questionnaires alone do little, if anything, to help understand the meanings that people ascribe to their values, beliefs and behaviour, and so do not explain why we do things, why we do things in the way that we do them, or the conflicts between what we say and what we do. To gain a deeper understanding, a qualitative, interpretive approach is more fruitful, not necessarily to supplant questionnaires, but at least to supplement them. Prior to interactive methods such as focus groups and interviews, one source of data from the questionnaire itself can be a useful starting point to an interpretive approach – the free-text comments written by the respondents.


I recently used the Safety Culture Discussion Cards to help analyse several hundred typed/written unstructured comments from a safety culture questionnaire – a fairly large amount of textual data. Many of the comments were several paragraphs long and referred to a variety of issues, and were mostly very interesting, well thought out and well-written. Making sense of rich textual data is never easy. But a common approach to understanding is via ‘content analysis’ (Krippendorff, 2004), or textual analysis. This often involves reading the text and applying a set of codes or categories to try to understand the data.

In this case, I decided to try to use the Safety Culture Discussion Cards to help code the data. The aim was to get a detailed understanding of the issues that questionnaire respondents were motivated to comment on – the specific issues, the way the writers related issues to each other, and the number of times that each issue was mentioned. An assumption was that issues mentioned more often by respondents reflect concerns that are important to them.

The cards cover most relevant aspects of safety culture but are (deliberately) not mutually exclusive, so this had to be kept in mind during the analysis. Prior to and during the coding, it was necessary to remove or combine cards as appropriate in order to achieve some satisfactory level of mutual exclusivity.


I started the analysis by reading all of the comments very carefully, and coding pieces of text within each comment using the eight elements of safety culture covered by the cards (Management Commitment; Resourcing; Just Culture, Reporting & Learning; Risk Awareness & Management; Teamwork; Communication; Responsibility; Involvement). Because a person’s comment could cover all sorts of issues, it is not possible to apply just one element code to each comment. Even a particular sentence within a comment could cover two or more issues, such as ‘Management Commitment’ and ‘Resourcing’. So at this stage, a sentence or paragraph could be coded using one or more elements.

The next stage was to re-read the comments and now apply more specific codes to the various pieces of text. The specific codes relate to the codes on the safety culture discussion cards, from 1a to 8e, noting also where the text was positive/favourable or negative/unfavourable in nature, or sometimes both. Since some of the cards overlap, where a piece of text could be coded using more than one card (and the cards could not reasonably be mutually exclusive) the codes were combined.

The final stage involved rechecking the use of the codes for each comment to ensure consistency and calculating the usage of each code. (An even more rigorous application of this method would involve having independent coders repeat the exercise with all or some of the text, as I and Amy Chung did when analysing comments relating to HF/Ergonomics practitioners’ views on barriers to research application; see Chung and Shorrock, 2010.) This allowed the relative frequency of each issue to be determined, and gave an impression of the perceived pertinence of the various issues.

The frequency of each element as well as the top 20 issues were calculated. The quantitative data, combined with discussion of the actual content of the comments, added substantially to the data received from the Likert-scale standard questionnaire items.

A final interesting output from this exercise is the ability to the the cards to visualise the narratives in the comments by mapping the relationships between issues and the possible meanings emerging. This will be the subject of a different blog entry. The exercise also revealed a few issues that are not covered by the existing cards, as well as the issues that are covered by the cards but that were not mentioned at all by the commenters. This is useful feedback for the further development of the cards.

References

Geertz, C. (1973). The interpretation of cultures: Selected essays. Basic Books.
Krippendorff, K. (2004). Content analysis: An introduction to its methodology. Thousand Oaks, CA: Sage.
Chung, A.Z.Q. and Shorrock, S.T. (2011). The research-practice relationship in ergonomics and human factors – surveying and bridging the gap. Ergonomics, 54(5), 413-429.

Advertisements

About stevenshorrock

I am a systems ergonomist/human factors specialist and work psychologist with a background in practice and research in safety-critical industries. My main interest is human and system behaviour in the context of safety-related organisations. I seek to enable improvement via a combination of systems thinking, design thinking and humanistic thinking. I am a Chartered Ergonomist and Human Factors Specialist with the CIEHF and a Chartered Psychologist with the British Psychological Society. I currently work as a human factors and safety specialist in air traffic control in Europe. I am also an Adjunct Senior Lecturer at the University of New South Wales, School of Aviation. I blog in a personal capacity. Views expressed here are mine and not those of any affiliated organisation, unless stated otherwise. You can find me on twitter at @stevenshorrock
This entry was posted in Culture, Safety. Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s