The Real Focus of Safety-II

Posted on 09/10/2018 by

Safety-II has become a talking point. It is discussed not only among safety professionals, but – perhaps more importantly – among front line practitioners, managers, board members and regulators in a wide array of industries. Its practical and inclusive focus on everyday work seems to strike a chord, acknowledging the reality of work for those who actually do the work.

There are, however, a few myths and misconceptions about Safety-II, some of which I highlighted in What Safety-II Isn’t. One is that Safety-II is about exceptional performance – excellence. This is perhaps associated with the use of the term ‘success’ and the phrase ‘go well’ in the literature on Safety-II (e.g., the EUROCONTROL [2013] White Paper). ‘Success’ is used here in a rather general sense, that work achieves its goals, in line with one definition of the term: “The success of something is the fact that it works in a satisfactory way or has the result that is intended.“ (Collins). The word is also commonly used to refer to exceptional attainment (i.e., that someone is. ‘successful’). This is not what is meant from the viewpoint of Safety-II, though the scope of Safety-II is inclusive of excellence, or especially desirable sociotechnical system performance.

Safety-II should be seen as focusing on all forms of work and all outcomes, routine and (perceived as) ‘unremarkable’ work, incidents and accidents, and exceptional performance. It is not about how things go well, so much as how thing go, but with the aim of course that things do go well. This is clearly depicted in the graph from the EUROCONTROL White Paper in Safety-I and Safety-II.

The focus of Safety-I and Safety-II. From EUROCONTROL (2013). From Safety-I to Safety-II: A White Paper. Brussels, p. 25. https://www.skybrary.aero/bookshelf/books/2437.pdf

What this shows is that the focus of Safety-II in terms of work and outcomes includes the focus of Safety-I. But Safety-II does not include Safety-I in terms of its precepts and concepts, which are quite different. (Importantly, both approaches can and should be practised – see Mind your Mindset: Safety-I and Safety-II – though some adjustments and compromises are naturally to be expected.) Both Safety- I and Safety-II include a focus on accidents, actual and potential. (In reality, accidents are a typically fraction of the 0.1%, in the graph above, though potential accident scenarios are a much greater, albeit unquantifiable, proportion.) The difference is that this is the whole focus of Safety-I, which reacts to events and risks primarily via an analytical approach, considering the human role in terms of contributions to accidents (causal or mitigating).

For Safety-II, the major focus is on less remarked-upon work and outcomes, as well as work and outcomes that are especially wanted (and might be seen as goals) or especially unwanted (anti-goals). But Safety-II does not focus specifically on ‘excellence’, and does not ignore accidents and other unwanted events (And ‘best practice’ really makes no sense, since what is best in one context – place or time – will not be best in another. Practice is always contextual.)

A key reason for this focus on everyday work is that work-as-done is the reason why sociotechnical systems are effective, including safe operations, and also the reason why they fail. By ignoring work-as-done, whether it is more or less congruent with work-as-prescribed or work-as-imagined, or whether it is quite different (see the messy reality), we don’t know how the system is functioning and whether it is drifting into an unwanted state, or shifting toward an especially wanted state (see Work and how to survive it: Lesson 2. Understand variation inside your organisation).

Focusing on normal work also makes sense from a Safety-I point of view, with its focus on accidents, actual or potential. This was highlighted in 1984 by sociologist Charles Perrow in his book Normal Accidents. Perrow was making the point that unusual events such as accidents are not fundamentally different to normal, everyday system functioning. They are, in some important senses, equivalent. Big accidents don’t have big causes. It’s just that ‘normal disorders’ combine in unexpected, often emergent, ways. ‘Normal disorders’ might be seen as degraded aspects of the system and context (e.g., technology used beyond design intent, degraded tools, excessive and overly complex procedures, stretched shift systems, competency gaps) along with differences between work-as-imagined and work-as-done. An important point is that it is normally the context of work that is disordered, while work-as-done tends to adapt, adjust and stretch to make things work, in locally rational ways. Work-as-done strives to create order in a system that is fundamentally disordered and not as-imagined from afar.

Adapted from EUROCONTROL (2013). From Safety-I to Safety-II: A White Paper. Brussels. https://www.skybrary.aero/bookshelf/books/2437.pdf

So while we want to ensure that work goes well, aiming for excellence, the focus of Safety-II is on the whole picture, but especially work that we might consider routine, everyday, and even unremarkable. This is the work that may end up in incident reports, or excellence reports, or simply keep the organisation running effectively. If we don’t look, we’ll never know.

Advertisements
Posted in Safety, systems thinking | Tagged , , , , , , , , , , , , , , | Leave a comment

The problem with professional appropriation: The case of ‘human factors’ and ‘ergonomics’

Posted on 08/10/2018 by

In a recent article in the Sydney Morning Herald newspaper by journalist Liam Mannix (A difficult position: Experts question whether ergonomics holds up), a Sydney University Professor calls out physical ‘ergonomics’ as bad science and practice:

Every year, companies around the world spend hundreds of millions of dollars on ergonomic chairs, keyboards and consultants, believing they are taking science-backed steps to care for their workers.

Ergonomists are regularly called as expert witnesses in court, where their findings can decide workplace injury claims worth hundreds of thousands of dollars. Ergonomics is promoted by work safety organisations around the country.

Yet “ergonomics does not have a firm basis in science”, says Sydney University professor Chris Maher, a leading authority on back pain.

But it seems that some who operate under the label of ‘ergonomics’ and ‘ergonomist’ are neither qualified nor experienced. The article notes that there are only 82 certified professional ergonomists in Australia, according to the Human Factors & Ergonomics Society of Australia, plus another 250 or so full members qualified to practice. (There would be many more, however, who are full members of other professional Human Factors and Ergonomics [HF/E] societies.)

“But there are thousands of people calling themselves ergonomists who aren’t,” says Associate Professor Jodi Oakman, head of the Centre for Ergonomics and Human Factors at La Trobe University.

“People will go out doing ergonomic work station assessments, they’ll call themselves an ergonomist – and they have no training whatsoever. It’s not a protected title,” she says.

Leon Straker, a Distinguished Professor at Curtin University added:

I don’t like a product being given the title ‘ergonomic’ – it’s not correct. If you don’t know who I am, what my job is, you cannot know my ergonomic requirements.”

Stephen Hehir, chair of the Human Factors & Ergonomics Society of Australia’s professional affairs board, remarked to Liam Mannix that many of the studies weren’t published in leading ergonomics journals, and most of the interventions they tested weren’t done by qualified ergonomists. 

“Imagine if they were reviewing surgical outcomes and including those operating without a medical licence rather than only qualified surgeons,” he said.

So it seems that the primary problem may not be with the evidence-based discipline and profession, so much as what I will call ‘professional appropriation’.

5116676245_3a94df7f7b_z.jpg

“There are thousands of people calling themselves ergonomists who aren’t,” says Associate Professor Jodi Oakman, head of the Centre for Ergonomics and Human Factors at La Trobe University. Photo: Jisc infoNet C BY-NC-ND 2.0 https://flic.kr/p/8N9izX

Professional appropriation

If we accept that HF/E is a profession, with registration schemes, codes of conduct, etc, then the next question is whether it is ethically acceptable to appropriate a professional title. Here, I define professional appropriation as taking as one’s own professional identity the label of a recognised profession, without undertaking the requirements to practise the profession, as accepted by professional bodies. The requirements to join a profession typically involve the following:

  • extended study, resulting in an appropriate qualification (for HF/E, these can include human factors /ergonomics or allied disciplines such as HCI, psychology, industrial engineering, biological sciences)
  • supervised experience
  • registration with a recognised regulator or professional body (professional society, association, or government body)
  • adherence to the Code of Professional Conduct of a professional society
  • other requirements, such as continuing professional development.

Professional appropriation seems to happen when individuals with limited exposure to a discipline appropriate an associated title based on this limited exposure. With limited explosive and experience, it may not be clear that professional appropriation is problematic.

The title ‘human factors specialist’ is sometimes appropriated, and this has happened historically with the title with ‘psychologist’, a term that is now legally protected in some countries. Despite being a discipline (with academic courses, journals, text books, professors, etc) and a profession (with certification, chartership, Codes of Professional Conduct, etc), HF/E professional titles are widely appropriated. Some describe themselves as ‘human factors experts’ without qualifications in human factors and without professional accreditation by a professional body. In most cases, this is probably done quite innocently, without understanding the unintended consequences.

Professional appropriation has occurred with a number of professions. The world of user experience/UX (an emerging profession) is apparently experiencing a growth in the use of terms such as ‘UX Psychologist’ by individuals who are not suitably qualified and experienced in psychology (e.g., Chartered or Registered Psychologists, in the UK). While some titles are legally protected (such as ‘Psychologist’ in Australia), other titles are only protected in their variant forms (e.g. ‘Psychologist’ is not legally protected in the UK, but ‘Counselling Psychologist’ and ‘Occupational Psychologist’ are legally protected). Other than legal protection of titles, we are left with legal protection of services, and associated laws (e.g., advertising laws, health and safety laws).

Equivalency

One could argue that professional titles are archaic, and that anyone should be able to choose whatever title one chooses. This argument seems to fall down quickly once one considers just a few professions, for instance physicians and surgeons, nurses and pharmacists, architects and structural engineers, accountants and solicitors, social workers and psychologists.

If one accepts that appropriate qualifications and experience are necessary to work as a professional (by definition), then the next question is whether Human Factors/Ergonomics should be included in this list of professions. Is HF/E a profession that requires suitably qualified and experienced people?

Whatever our view on this, HF/E is already a profession that requires appropriate qualifications and experience. This is evidenced by professional registration in many countries (including Chartership in the UK, as per Chartered Accountants, Chartered Psychologists and Chartered Architects). If one still rejects the idea that one needs to be suitably qualified and experienced, then one risks saying that professional standards in Human Factors/Ergonomics are unimportant and that the quality of Human Factors/Ergonomics professional services, including ethical considerations, is unimportant. This devalues HF/E to such an extent that to offer professional services becomes illogical. One cannot offer professional services (e.g., consultancy, training, expert witness) in something that one does not consider to be a profession. QED.

10082263794_f84c523928_z

NATS employs 25-30 Qualified Human Factors/Ergonomics Specialists. Photo: NATS.- UK Air Traffic Control CC BY-NC-ND 2.0 https://flic.kr/p/gmWeLo

Risk

From the client’s point of view, the above may not seen terribly relevant. What matters more to clients is risk management. What is the risk of professional appropriation? The ‘risk’ concerns problems or opportunities that may not be properly recognised or managed. The risks could be risks to process safety, occupational safety, health, wellbeing, productivity, efficiency, quality, morale, etc,  By hiring someone who is not suitably qualified and experienced, you are hiring someone who is lacks the required competency to help recognise, understand and manage problems and opportunities relating to system performance and human wellbeing. And someone who is not suitably qualified and experienced may be unaware of this. The Dunning-Kruger effect shields us from the limits of our knowledge and skills.

The risks of professional appropriation are quite obvious and immediate for some professions (e.g., surgery, dentistry, anaesthesia), while for others the risks are obvious to some but usually emerge after some time as a project develops (e.g., civil and structural engineering, safety engineering). For still others, the risks are less obvious and may take longer to come to light. HF/E tends to fall into the latter two categories.

One particular risk of hiring someone who is not suitably qualified and experienced is second order problems. With relatively little knowledge and skill in a profession, we tend to be more focused on first order problems – immediate issues. With more knowledge and skill, we are more focused also on second order problems – possible unintended consequences. This requires systems thinking, which happens to be the foundation of HF/E. For instance, focusing only on non-technical skills training and labelling this as ‘human factors training’, without addressing underlying system and design problems to an appropriate degree, can consume an organisation’s ‘Human Factors budget’ and leave people (usually a small and diminishing proportion of the total number of people) to cope with systemic and design problems using their non-technical skills: an ethical dilemma.

And there are very specific risks to professional appropriation. The SMH article recounts a case where a worker was awarded tribunal-ordered compensation – after she suffered an injury caused by a so-called ‘ergonomics intervention’. 

Cakir was working as a web publishing officer with the Department of Employment and Workplace Relations when she was given an “ergonomic assessment of [her] workstation” by an injury management consultant, according to tribunal papers.

The ‘ergonomics intervention’ was apparently not prescribed by a SQE ergonomist, but by an exercise physiologist (the article does not question the validity of exercise physiology).

The risks of professional appropriation are real but hard for clients to see. Clients can, however, ask if those who use the title ‘ergonomist’, ‘human factors expert’ are suitably qualified and experienced. (Note that ‘expert’ is a term that most bona fide experts seem to avoid. I’ve met a handful of people in HF/E who I would truly consider experts. I am not one of them. Though just to confuse matters, note that in some countries, especially in mainland Europe, the term ‘expert’ simply refers to a specialist or someone occupying a particular job role.)

Professional desertification

If anyone can simply adopt any professional title, then one particular system-wide risk is the illusion that the market for associated services is already well-served. For instance, if everyone with a few days of life coaching or NLP training (or even no training at all) adopts the title ‘psychotherapist’, and if employers and clients are none the wiser, then why the need for suitably qualified and experienced psychotherapists (e.g., meeting the standards laid down for full membership by BACP and UKCP, in the UK, requiring many years of formal study, and supervised [often unpaid] practice)? The same goes for any profession.

I wonder if this has become a hidden reality in some sectors when it comes to Human Factors and Ergonomics. As Associate Professor Jodi Oakman, head of the Centre for Ergonomics and Human Factors at La Trobe University, pointed out, “there are thousands of people calling themselves ergonomists who aren’t.” In the National Health Service (NHS) in England, there was, at the time of writing this post, just one Chartered Ergonomist and Human Factors Specialist formally practicing in the role of an HF/E specialist.

5893752031_c0c8133972_z

The National Health Service in the UK has a focus on Human Factors, but only a few qualified Human Factors and Ergonomics Specialists, out of 1.5 million staff. Photo: Lydia CC BY 2.0 https://flic.kr/p/9YP29k

And yet, ‘human factors’ is a huge buzzword in the NHS. There are many courses, and many external consultants (often from aviation) describe themselves as human factors specialists or ‘experts’. The training provided is typically in behavioural (non-technical skills). Non-technical skills are vitally important but NTS training is – I would estimate – somewhere between 1/100th or 1/1000th of the whole scope of discipline of HF/E, if one were to count the pages of text books or journal articles, or hours of teaching on HF/E degrees. In fact, NTS training is more properly aligned with Applied Psychology, because its principles are behavioural, not design-led. (HF/E is primarily about fitting the task to the person, not vice versa.)

This is not to de-emphasise the importance of this training. I have supported such training in healthcare and aviation, and strongly encourage it. But the effect of labeling this ‘Human Factors Training’ – something that has been inherited from airlines – seems to have had unintended consequences. The most obvious of these is the widespread lack of understanding (including at Board level) about

  • the true focus of HF/E (socio-technical systems)
  • its primary means of gaining insight (understanding system interactions, which we might call ‘work’ for our purposes), and
  • its primary means of intervention (design).

In Frank Hawkins’ 1987 book ‘Human Factors in Flight’, he remarked that “There seems to be little justification for any large organisation not employing, in house, one or more degree-qualified Human Factors specialists. In fact, without some level of in-house expertise, Human Factors problems are unlikely to be recognised adequately to generate a call for reference to an external consultant” (p. 328-329).

It may be the case that the professional appropriation of HF/E is somehow associated with the professional desertification of HF/E. The same would likely happen, to varying degrees, with dietetics, architecture, nursing. and psychology.

Involvement and inclusion

At this point, having described some of the problematic aspects of professional appropriation, I find myself dissatisfied and conflicted. On the one hand, professional services, including those done by people who identify themselves as ‘Human Factors Specialists’ and ‘Ergonomists’, should obviously abide by professional standards, including ethical standards. But there are a few problems (see also Human Factors and Ergonomics: Looking Back to Look Forward).

First, there are not sufficient numbers of SQE HF/E specialists (internal or external) to meet demand for HF/E, let alone get involved in solving problems that could benefit from a professional HF/E approach. (This is similar, however, to clinical and counselling psychology in the NHS, for which there are long waiting lists.)

Second, there are relatively few HF/E courses, and little funding, for those who wish to become suitably qualified in HF/E. This applies more, to degree-level courses, which are also a significant investment in time and money. Still, an increasing number of people, for instance front line professionals and other those coming from other allied professions, are signing up for diploma and degree level courses in order to apply HF/E theory and method to their work. (See here for a discussion of becoming an HF/E practitioner.)

Third, it is crucial that HF/E is not merely a discipline and profession, but a broader endeavour aimed at improving system performance and human wellbeing. This is similar to psychology and psychotherapy (regarding mind, behaviour and mental health) and dietetics (regarding diet). This seems to apply various disciplines and professions that centre on human needs. HF/E theory and methods can be applied by many professions with various qualifications and experience as part of their professional work, given appropriate competency. It is not necessary that everyone undertakes a degree in HF/E, but neither is it sufficient to undertake a one- or two-day training course alone to be considered a specialist of any aspect of HF/E. There are, however, training courses in aspects of HF/E that are recognised by professional bodies affiliated with the International Ergonomics Association. There are also specific membership grades such as CIEHF’s ‘Technical Membership’ that apply to specific aspects of HF/E, as relevant to one’s own professional work. Ultimately, I consider HF/E expertise as emergent, from interaction between those with expertise in theory, findings and methods, and those with expertise in work and the context of work.

We can take some practical steps. It is helpful, for instance, when offering HF/E-related training courses or services, to indicate the scope of HF/E covered, relative to the scope of the discipline as a whole. This can be made more obvious in the title of the course, For example, a course entitled ‘Human Factors in <Operating Theatres>’ might cover human factors issues in operating theatres, including the interactions between people, activities, context and tools, and methods for improving these by design (of artefacts, tools, work, etc). Alternatively, a course could be titled, ‘Human Factors for <Surgeons/Pilots/etc>’. Such a course would be more adapted to the needs of a particular stakeholder group. This night be a blend of NTS training and training related to the design of various aspects of work (routines, checklists, equipment, etc), with an aim to help improve work design or at least compensate or mitigate unwanted effects.

And of course, in providing consultancy and training we must be clear about our own qualifications and experience. I ultimately consider my practice cross-disciplinary, and dip into several other disciplines that I find especially helpful in helping to improve system performance and human wellbeing (e.g., philosophy, anthropology, practice theory, community organising, counselling and psychotherapy, graphic design). My approach is to integrate aspects of these into an eclectic, cross-discplinary practice, but of course I stop short of describing myself as a professional or specialist in any of them. I know that my interpretation and implementation of these disciplines is narrow, often shallow, and selective. So I simply indicate the cross-disciplinary influences on my practice. Even within a discipline, our competency soon reaches its limits, and understanding these is a critical aspect of ethical practice. Physical ergonomics, for instance (the topic of the SMH news report) is not an area of competency for me. My last experience was part of my ergonomics post-graduate degree and I have not practised this, outside of basic anthropometry, for 21 years. I am simply not competent to practise it.

Summing up

As with may human-centred professions, there is a balance between professional standards and inclusion. The way to address this balance is by total honesty and clarity, abiding by ethical standards of professional practice, collaborating between different areas of knowledge and practice, carefully drawing from useful theory and applicable methods, but avoiding appropriating professional titles, which can have significant unintended consequences for professional standards, system performance and human wellbeing.

Afterword

From my previous post on this topic (Suitably Qualified and Experienced? Five Questions to ask before buying Human Factors training or consultancy), here are the five criteria and questions that apply to paid-for human factors and ergonomics (HF/E) consultancy and training support and employment, that may help with reflection and discussion.

1. Qualification – Do they have a recognised qualification in HF/E?

2. Accreditation – Do they have an appropriate level of membership of an HF/E related professional organisation?

3. Code of Ethics – Do they abide by a code of ethical conduct from an HF/E related society or association?

4. Experience – Do they have experience in the HF/E work and the domain of interest?

5. Social recognition – Is the person recognised as an HF/E specialist by other qualified HF/E specialists?

The aim of these criteria and questions is to ensure that professional standards – including ethical standards – are met. The criteria and questions are framed above in the context of HF/E, but in fact they apply to any professions, such as psychology, dietetics, or physiotherapy. Proper consideration of the criteria and questions should help to protect organisations, individuals, and the integrity of the profession.

Related posts:

Posted in Human Factors/Ergonomics | Tagged , , , , , , , , | Leave a comment

Work and how to survive it: Lesson 2. Understand variation inside your organisation

Posted on 01/10/2018 by

Much of my practice is informed by counselling and psychotherapy as well as humanistic psychology more generally. One of my problems with these fields, however, is that insights and discussions are largely kept within the world of psychotherapy. What a waste! The vast majority of people are not engaged in psychotherapy and for the most part, psychotherapy pays little attention to applying itself to the mundane issues of everyday life, outside of counselling rooms. This is a second in a series reflecting [for now] on excerpts from Life and How To Survive It, by the psychotherapist Robin Skynner and the comedian John Cleese, with some reflections on work and organisations. 

Other posts in the series:

In Chapter 1, John Cleese and Robin Skynner are talking about people and families at different levels of mental health. Cleese asks about families that are unusually mentally healthy.

Robin …in trying to describe excellent mental health, and compare it with ill-health, and with the ‘average’ health in between that most of us enjoy most of the time … it’s difficult not to talk as if they are quite different from one another, and inhabited by different people. But, in fact, our level of health is changing all the time. We all feel more healthy in our better moments, when we are ‘in a good mood’, when things are going well, when we feel loved and valued, when we have done our best. And we can all feel less healthy under stress, when our usual sources of support are removed, when we have ‘let ourselves down’, when we ‘get out of bed on the wrong side’. Also, our level of health is not the same in all areas of our functioning. A person who is ‘average’ overall may be outstandingly healthy in some respects, even though functioning poorly in others.
John And obviously the overall level can change over time, too. Otherwise you’d be out of a job. I mean people can get more mentally healthy, can’t they?

In my last post, I wrote about the everyday experience of work, which is often ignored in safety, for several reasons, sometimes beyond the control of safety practitioners. Within this great area of day-to-day activity, many things are happening that we can easily miss unless we pay attention to them. One is that performance changes over time. One aspect of this is what is sometimes called ‘practical drift’. In Friendly Fire,  Scott Snook defines practical drift as “the slow uncoupling of practice from procedure” (p. 24). It is one way how we end up in the work archetype of The Messy Reality.

This is very hard to see from the inside, as it tends to happen slowly and tends to help achieve a range of goals that are more positively reinforced within the organisation (e.g., cost efficiency and production). But without paying attention to normal, everyday work, we don’t see what is going on. Importantly, we don’t see changes in the normal operating point, and associated behaviours, especially when these changes happen slowly and are only exposed to those who are closely associated with the work, whether front-line staff, middle managers or the Board

Drift
Figure 1: Drift toward failure. Adapted from EUROCONTROL (2013).

It often takes an outsider to see this practical drift. As Edward Hall (1959) wrote in his book The Silent Language, “culture hides much more than it reveals, and strangely enough, what it hides, it hides most effectively from its own participants” (p.39). We are victims of our cultures – professional, organisational, and national – and insights often require an outside perspective. By ‘outsider’, I simply mean someone who is seen as an outsider by those in a particular in-group, or at least someone who is on the edge of the group.

Outsiders not only see this drift more clearly, but have ‘permission’ to ask about it. This can be associated with their relative innocence. Outsiders may be able to ask the sorts of questions that a child asks: Why do you do that? What do you do it like that? What is that for? The outsider will often, however, need a basic knowledge of the work, especially for less observable forms of work and work that is very complex.

‘Permission to question’ can also be because the questioner has been accepted into a particular role. One of these has been termed ‘barbarian’, by Steele (1975) in Consulting for Organizational Change. Steele characterises this role as “violating comfortable but limiting norms and taboos that are preventing the system from being as effective as it might be. (A counter measure against tunnel vision.)”.  This relates to the archetype Taboo. In-group members will find it difficult to raise taboo issues and will often need exceptional interpersonal skill to do so in a way that helps others gain insight.

An outsider may be a cultural insider, e.g., an air traffic control supervisor or anaesthetist from elsewhere. In this case, the person is an outsider in terms of workgroup and location, but an insider in terms of profession. Supervisors observing the work of other workgroups is one way to help people ‘see’ (and improve) their performance. They may be able to see things and ask questions that true insiders can’t.

Another kind of shift or change is where performance moves towards exceptionally good performance, where work is sustainably productive, innovative, healthy, joyful, etc.  Again, if normal, routine, day-to-day performance is unknown and generally ignored (not subject to anything like the same kind of attention as incidents), then we may just gratefully accept the marginally reduced number of incidents (on the left hand side of Figure 2), but not see the way that work is changing for the better, including the ‘good practice’ that contributes to it. In our Ignorance and Fantasy of this day-to-day work, we may well implement changes (rules, limits, targets, league tables, incentives, punishments, etc) that pull the operating point back, halting progress.

Postive drift
Figure 2: Shift toward exceptional performance. Adapted from EUROCONTROL (2013).

As well as changes over time, a second thing is happening that we can easily miss unless we pay attention: there are differences between different parts of an organisation. As Skynner reminds us, “our level of health is not the same in all areas of our functioning”. In travelling to over 50 air traffic control units and centres of various kinds, I have seen and heard about large variations in many aspects of practice and performance. In most cases, where units and facilities are isolated geographically or culturally (e.g., by profession), these differences are unknown or not appreciated beyond the facility, and often beyond the department, work group, or room. Therefore, good practice in one area of an organisation is not known in another that is similar in context and could benefit. For example, one particular air traffic control tower had developed its own refresher training arrangements. These innovative practices could have been of great help to other towers but, lacking day-to-day contact with the tower in question, were unknown. (See Issues 25 and 26 of HindSight Magazine, on ‘Work-as-Imagined and Work-as-Done’ and ‘Safety at the Interfaces: Collaboration at Work’.)

These differences may also be papered over by the way that we measure performance. For instance, if we average measures across the whole organisation, or if we measure things that do not reflect differences between different areas of an organisation, then again we will be less likely to see and pay attention to them. This means we must pay careful attention to the way that differences may express themselves in terms of department, location, profession, gender, age, experience, and on. In many cases, the differences within organisations are greater than the differences between them, but if we don’t pay attention to what’s going on, we’ll never really know.

References

EUROCONTROL (2013). From Safety-I to Safety-II. A White Paper. Brussels:  

EUROCONTROL Network Manager, September 2013. Authors: Hollnagel, E., Leonhardt, J., Shorrock. S., Licu, T. [pdf] (Contributor)

EUROCONTROL (2017) HindSight Magazine. Safety at the Interfaces: Collaboration at Work. Issue 26, Winter. Brussels: EUROCONTROL. [webpage] [pdf]

EUROCONTROL (2017) HindSight Magazine. Work-as-Imagined and Work-as-Done. Issue 25, Summer. Brussels: EUROCONTROL. [pdf]

Skynner, R. and Cleese, J. (1994). Life and How To Survive It. Mandarin.

Snook, S.A. (2000). Friendly fire. Princeton, NJ: Princeton University Press.

Posted in Human Factors/Ergonomics, Safety, systems thinking | Tagged , , , , , , , , , , , | 2 Comments

Work and how to survive it: Lesson 1. Understand ‘how work goes’

Posted on 29/09/2018 by

I have recently been reading Life and How To Survive It, nearly 20 years after first reading it. It is a book on relationships and psychology, written in conversational question and answer style, by the psychotherapist Robin Skynner and the comedian John Cleese. 

Robin Skynner was a child psychiatrist and family therapist who practised psychotherapy with individuals, couples, families, groups, and institutions, where he employed group-analytic principles. Skynner, a former WWII bomber pilot, was a pioneering thinker and practitioner whose insights on families and groups are of value to those seeking to understand behaviour in organisations. In this sporadic series of posts, I will share a few of these, as they might apply to work and organisations. 

Posts in the series:

At the start of Life and How To Survive It (p. 2) John Cleese asks about families that are unusually mentally healthy.

John Well, I’d like to know more about them. Especially as I’ve never heard anyone talk about them.

Robin No, that’s right, the research is hardly mentioned.

John I wonder why. You’d think everyone would want to learn about exceptionally well adjusted people and find out what they know that we don’t. Yet even the other shrinks I know don’t seem familiar with this research. But then, the odd thing about psychiatry is that it’s based upon the study of people who aren’t doing very well – people who have more ‘problems’ than normal.

Robin Yes, that’s basically true.

John And the more you think about that, the stranger it seems. I mean, if you wanted to write a book about how to paint, or play chess, or be a good manager you’d start by studying people who are good at those things. And you wouldn’t expect heavy sales of a book called Play Championship Golf by Learning the Secrets of the Worst 20 Players in the World.

Robin True. Doctors do at least study normal physical functions – anatomy, physiology – before going on the wards to study disease. Psychiatrists seem interested almost entirely in people who are abnormal.

Using this analogy, safety scientists and practitioners might be considered a branch of organisational psychiatry, almost entirely focused on the ‘abnormal’ of work and organisations (albeit in terms of outcomes, not necessarily behaviour or processes). The trouble with this is that we fail to understand ordinary day-to-day work and organisational behaviour, let alone that which is especially effective. (The exceptions to this are branches of study and practice on High Reliability Organisations and Appreciative Inquiry, which are more interested in the latter. But these are rarely part of normal safety management and represent niche areas of safety research.)

For a few reasons, possibly chief among them regulatory requirements in highly regulated industries, the vast majority of effort of safety scientists and practitioners is on abnormal and unwanted outcomes, and the work and processes that precede these. My estimation, based on significant contact with safety practitioners and researchers in many countries, is that this tends to take up over 90% of work hours, and many safety practitioners I know place the estimate closer to 100%. Rarely among those working as safety scientists or practitioners is there, or has there ever been, any significant systematic study of normal work (e.g., via ethnography, systems thinking, systems ergonomics, work psychology, organisational behaviour).

The disconnect between our focus of attention (unwanted events) and what we desire (safe or, more generally, effective work and systems) is what I have previously characterised as déformation professionnelle, a play on words referring to job conditioning or acclimatisation, which affects most or all professions, in some way. As noted by literary theorist Kenneth Burke, “A way of seeing is also a way of not seeing — a focus upon object A involves a neglect of object B” (1935, 1984, p. 49). In the case of safety, object A is relatively tiny in number, and object B is huge in number. Because it is so ordinary, we tend not to ‘see’ it (see Figure 1 below, from the EUROCONTROL (2013) White Paper on Safety-I and Safety-II).

Figure 1: The focus of safety.

What this means in practice for safety is that analyses and conclusions about unwanted situations can be based on flawed assumptions about normal work, from the perspective of work-as-imagined (see the archetype, Ignorance and Fantasy). Following safety incidents, even or especially ‘first of a kind’ incidents, an investigation might recommend a new rule. In such cases, where normal work has not been studied and understood, the rule can bring unintended consequences. [Readers with front-line experience will now bring several examples to mind.] The reason is that the rule acts as an unreasonable constraint on normal work, perhaps requiring significantly more time or other resources, which are unavailable, or reduced demand, which is not possible. In the absence of additional resources or reduced demand, the rule may be bypassed or, if enforced, causes secondary problems and leaves the system in a more pressured or fragile state. Meanwhile, those recommending the rule remain unaware of its failure, and assume – through lack of feedback and no further related incidents – that the rule is successful. Examples of unintended consequences in interventions can be found under the Congruence archetype.

Seeing only how things go wrong means that we neglect how things go right (e.g., a desired situation), and – most importantly – how things go, in a more ordinary or general sense. Things can go wrong in countless ways, but in many forms of work, desired outcomes tend to come about in a relatively small number of ways, at a fundamental level. A golfer can hit the ball in any direction and at a wide range of angles. The number of ways to miss a hole is effectively infinite. In comparison, the number of ways to hole the ball is relatively small. The same goes when reverse parking a car, when landing an aircraft or piloting a ship to port. There are many variations in how this is done, and some ways are especially effective, but there are countless ways in which to get it wrong. Hence training is focused, in the main on how to get it right, and not on how not to get it wrong (though many trip hazards will be important to know about).

By studying ordinary, everyday functioning in organisations, and ‘exceptionally well adjusted’ functioning, we can better understand when a sociotechnical system really is healthy or unhealthy, in what ways, how and when this is expressing itself, who is affected, and why (considering sociotechnical system interaction).

This does mean that safety scientists and practitioners, and anyone else interested in the quality and improvement of human work and sociotechnical systems, must spend more time understanding (and in) the world of work-as-done, and the messy reality of work (remembering that is it for the most part the work context that is messy, not the work itself). This is no mean feat when one’s work is driven by regulatory requirements, but if we wish to understand work and systems, and not just sporadic symptoms of unwanted interactions, then we must somehow prioritise time and other resources. As I reflected in this post, if you want to understand work, you have to get out from behind your desk.

Posted in Human Factors/Ergonomics, Safety, systems thinking | Tagged , , , , , , , , , , , | Leave a comment

HindSight 27 on Competency and Expertise is out now!

Posted on 07/09/2018 by

HindSight Issue 27 is now available in print and online at SKYbrary and on the EUROCONTROL website. You can download the full issue, including an online supplement, and individual articles. HindSight magazine is free and published twice a year, reaching tens of thousands of readers in aviation and other sectors worldwide. You will find an introduction to this Issue below, along with links to the magazine and the individual articles.

HS27 cover_Page_01

Welcome

“Welcome to Issue 27 of HindSight magazine. The theme of this issue is ‘Competency and Expertise’. It is a topic that links to all previous Issues of HindSight.

Our ability to work effectively depends on the competency and expertise front-line practitioners and all involved in the operational, technical, support, and management functions. Safety isn’t something that is just ‘there’ in the aviation system. People actively create safety. But how do we create safety? And what do we need to do to help ensure that we can continue to do so? Competency and expertise is an important part of the answer.

In this issue, we have articles from operational, safety, human factors and psychology specialists. This is part of what makes HindSight unique – it brings together those who do the operational work, those who support operational work in a variety of ways, and those who study operational work to help better understand it. We are proud to give a voice to some of the world’s leading academic thinkers, and to operational and support specialists who have stories, experience and practical insights to convey. The key is that the articles are interesting and useful to the primary readers of HindSight: air traffic controllers and professional pilots, and hopefully to others who support operational work. Do we succeed? Let us know! In this Issue we explore the nature of competency and fundamental applications and implications for operational training, selection, and procedures, including non-technical skills and contingency. We then zoom out to regulatory and future issues. The regular feature on ‘Views from Elsewhere’ continues with articles from surgery and rail. These articles raise questions for us in aviation, and provide some practical ideas. And in this issue we have articles drawing from the world of sport. HindSight continues online over at SKYbrary with further articles in the online supplement, from aviation and other industries, on the theme of competency and expertise.

We also have ‘What we do’ good practice snippets. We’d particularly like to hear from more readers for this section. And this brings me to the next Issue, which will feature articles on ‘Change’. All readers have been affected by changes, in procedures, regulations, technology, people, incentives, organisation, etc. The pace of change will only increase. How do we change to adapt to the dynamic world of air traffic management? And how do we as individuals, teams, and organisations adapt to these changes? Let us know, in a few words or more, for your magazine on the safety or air traffic management – HindSight.”

HindSight 27 Articles

Foreword

Editorial

Op-ed

Fundamental Issues

Non-technical Skills

Contingency

View from the Air

Regulatory Issues

Future Issues

Views from elsewhere

What we do

Interview

HindSight 27 On-line Supplement

See all editions of HindSight magazine

Posted in Human Factors/Ergonomics, Safety | Tagged , , , , , , , , , , , , , | Leave a comment

Twelve Properties of Effective Classification Schemes

Posted on 31/08/2018 by

Most organisations seem to use a classification system (or taxonomy) of some sort, for instance for safety classification, and much time is spent developing and using such taxonomies. Importantly, decisions may be made on the basis of the taxonomy and associated database outputs (or it may be that much time is spent on development and use, but little happens as a result). There is therefore a risk of time and money spent unnecessarily, with associated opportunity costs. Still, taxonomies are a requirement in all sorts of areas, and several things should be kept in mind when designing and evaluating a taxonomy. This posts introduces twelve properties of effective classification systems.

Effective classification schemes are difficult to develop. The following properties
need to be considered to develop a valid classification scheme that is accepted and produces the desired results.

1. Reliability

A classification scheme must be used reliably by different users (inter-coder reliability or consensus) and by the same users over time (intra-coder reliability or consistency). Reliability will depend on many factors, including the degree of true category differentiation, the adequacy of definitions, the level of hierarchical taxonomic description being evaluated, the adequacy of the material being classified, the usability of the method, the adequacy of understanding of the scheme and method, and the suitability of reliability measurement. Adequate reliability can be very difficult to achieve (see Olsen and Shorrock, 2010 $$), and the heterogeneity of methodologies employed by researchers measuring reliability of incident coding techniques make it more difficult to to critically compare and evaluate different schemes (see Olsen, 2013 $$). However, if a classification scheme cannot be used reliably, then it is usually fair to say that it is not fit for purpose, especially for analysing large data sets (though it may be that reliability is achieved for certain users in certain contexts)

2. Mutual exclusivity

Categories should be mutually exclusive on the same horizontal level, so that it is only possible to place subject matter into one category. This relates to reliability. There are varying degrees of mutual exclusivity, since categories often have things in common, or overlap to some degree, depending on the criteria. Mutual exclusivity tends to be lower for abstract or unobservable concepts. This is especially true for psychological labels, and even more so those that are all-consuming (such as ‘situation awareness’, ‘mental model’, or ‘information processing’). For properly differentiated categories with clear definitions, appropriate guidance can reduce sources of confusion (see Olsen and Williamson, 2017 $$).

3. Comprehensiveness (or ‘content validity’)

It should be possible to place every sample or unit of subject matter somewhere. However, choices must be made about the granularity of categories. Highly detailed classification schemes and classification schemes that offer little granularity suffer from different problems concerning mutual exclusivity, usability, face validity, usefulness, etc.

4. Stability 

The codes within a classification system should be stable. If the codes change, prior classification may be unusable, making comparison difficult. On the other hand, it should be possible to update a classification scheme as developments occur that truly affect the scope and content (e.g., new technology). Ideally, changes should have minimal impact.

5. Face validity 

A classification system should ‘look valid’ to people who will use it or the results emanating from it. An industry classification scheme should incorporate contextual and domain-specific information (‘contextual validity’), but should also sit comfortably with pertinent theory and empirical data (‘theoretical validity’). The best approach here is to stick with what is well-understood and accepted.

6. Diagnosticity (or ‘construct validity’)

A classification scheme should help to identify the interrelations between categories and penetrate previously unforeseen trends. This may relate more to the database and method than the taxonomy itself.

7. Flexibility

A classification scheme should enable different levels of analysis according to the needs of a particular query and known information. This is often achieved by a modular and hierarchical approach. Shallow but wide taxonomies tend to suffer from low flexibility.

8. Usefulness

A classification scheme should provide useful insights into the nature of the system under consideration, and provide information for the consideration of practical measures (e.g., for improvement).

9. Resource efficiency

The time taken to become proficient in the use of a classification scheme, collect supporting information, etc., should be reasonable. Continued difficulties in using a classification scheme, after initial training and supervised practice, usually indicate a design problem and signal the need for (re-)testing.

10. Usability

A classification scheme should be easy to use in the applied setting. This means that the developers should be able to demonstrate a human-centred design process akin to ISO 9241-210. The most relevant aspects of usability should be determined. For instance, some users may have formal training in the use of the classification scheme, little time to make inputs, limited understanding of terms and acronyms, etc.

11. Trainability

It should be possible to train others how to use the classification scheme and achieve stated training objectives, including any required levels of reliability. In some cases, there may be valid reasons to go to only to the original developers for training (e.g., the taxonomy is sensitive or commercialised). In such cases, there is a need to consider why this is the case, and the possible related implications (e.g., lack of peer reviewed, public domain accounts of development; lack of independent testing).

12. Evaluation

Classification schemes should normally be amenable to independent evaluation. This means that they must be available and testable on the requirements above using an appropriate evaluation methodology. This will of course be more difficult for taxonomies that are restricted for various reasons (commercial, security, misuse prevention, etc).

Summing up…

In practice, it will not be possible to achieve anywhere near perfection on these criteria. Even where evaluation results are very positive (assuming there is any evaluation), experience in use will usually be different (and usually worse from the users’ points of view) and undocumented. Trade-offs must be made and some of the properties above will be more important than others, depending on the application. For instance, in some cases, the priority may be to help investigators to ensure that relevant issues have been considered, perhaps also to model the interactions between them (see Four Kinds of Human Factors: 4. Socio-Technical System Interaction). In other cases, the priority may be to help analysts understand prevalence and trends in very large data sets.  In still other cases, the priority may be to help users with little time or knowledge (‘casual users’) make basic inputs. These user groups have different needs and expectations.

It may also be necessary to use a taxonomy that is not adequate on some of the criteria above. In all cases, there is a need to understand the possible risks (e.g., time spent using the taxonomy; decisions made on the basis of the data) and to manage these risks (e.g., ignore data for categories that are know to be unreliable; merge categories; analyse data based on a hierarchically higher category/level up). However, three basic activities should be undertaken to help achieve adequate validity:

  1. Involve appropriate stakeholders in taxonomic development and evaluation, with a focus on understanding their needs the associated taxonomic requirements, and the trade-offs between requirements. This should include people who understand human-centred design, taxonomy and all relevant aspects of the scope of the classification scheme.
  2. Review relevant literature, analyse the work and system, and review other classification schemes (including ones previously used by any stakeholders).
  3. Test the classification scheme throughout its development and implementation.

Afterword

This post is based on a short briefing note that I produced for an Australian government agency meeting in 2004, not long after being awarded a PhD related to taxonomy (461 pages; reading not recommended, but available on request). Since I sometimes find it hard to find this note, I thought it might be useful to put online, also in the hope that it might help someone else. The post focusses on the properties of effective taxonomies that relate to development, and not so much on the use, mis-use and abuse of taxonomies. Another post, maybe.

Posted in Human Factors/Ergonomics, Safety, systems thinking | Leave a comment

Vive la Compétence !

Posted on 16/08/2018 by

The text in this post is from the Editorial of HindSight magazine, Issue 27, on Competency and Expertise, available for download in late August at SKYbrary here.

France_champion_of_the_Football_World_Cup_Russia_2018.jpg

Image: Kremlin.ru [CC BY 4.0 (https://creativecommons.org/licenses/by/4.0)%5D, via Wikimedia Commons

This summer, we have been entertained by the world’s best footballers – experts in the game. And it just so happens that Competency and Expertise is theme of this Issue of HindSight. What might we learn from World Cup 2018? Here are five observations.

1. Past performance does not determine future performance

Some world-leading teams, which were favourites to win, were knocked out early, or didn’t qualify. It just goes to show that we can’t rely on our record. Past success does not guarantee future success. The same tactics that worked in the past will not necessarily work in the future.

But we humans are creatures of habit. In his famous book Human Error, James Reason (1990) described two ways that we rely – or over-rely – on our past experience. The first  is similarity matching. When a situation is similar to one experienced previously, we use pattern patching and tend to respond in a similar way to how we did before. The second is frequency gambling. More frequent solutions in roughly similar conditions will tend to prevail. Most of the time, these are efficient ways of working, and efficiency is critical when seconds count. But sometimes, we need to be more thorough, especially when  preparing, practising and planning. In any case, we must always adapt to the situation.

Just as past success does not guarantee future success, past failure does not guarantee future failure. Penalties were a case in point. Far from being a lottery that is impossible to rehearse for, or an event for which some teams are ‘jinxed’, this year showed that extensive physical and psychological preparation for such high pressure scenarios pays off.

This is something that I am particularly interested in within ANSPs. Front-line safety-critical staff need and deserve world-class training, especially refresher training. This isn’t a luxury. It’s a necessity, but the sort of necessity that sometimes becomes obvious only in hindsight. The same applies to team resource management training, and other training that integrates lessons from the past. The lessons that stick often come from past failures, but we need to learn those lessons in the right way, in the right context.

2. Teams are more than the sum of their parts…and success runs deep

It became clear in this World Cup that individual expertise does not equal team competence. Teams can suffer through overreliance on star players, but can benefit greatly from teamwork bonded with trust, respect, and an understanding of how each player will respond in a given situation. The same applies in air traffic management. Here, we have procedures to help us predict how others will respond. But procedures do not determine how someone will respond. They do not even apply to all situations, nor prescribe all responses. In this case, trust built from working together helps us to succeed.

In the World Cup, the team is not just the players on the pitch. The best managers set up their teams to win, using all necessary resources, and adapting their style to whatever will bring out the best from each player. Everything is designed and managed for human performance. Hundreds more, including psychologists, dietitians,
physiotherapists, etc, help players to perform at their peak. It is similar with ANSPs. While all have similar basic kinds of front-line support staff, some ANSPs have teams of qualified human factors/ ergonomics specialists, psychologists, TRM facilitators, CISM peers, educational specialists, etc. Human performance is what we do, but to be sustainably successful, it needs a strong support network.

3. Technology changes the nature of work

The introduction of video assistant referee showed how technology changes the nature of work. Referees now have to use their expertise to decide when to use the technology. Over-reliance ruins the spontaneity of play. Under-use may bring criticism that not only did a referee not spot a foul or offside, but that they didn’t use a tool that could have shown this: two mistakes, where previously there would have been only one

In The ETTO Principle, Erik Hollnagel discusses a fundamental trade-off that underlies human performance: the efficiency-thoroughness trade-off. Referees must balance efficiency against thoroughness to harmonise fluidity and fairness. Footballers do the same. If there is time to be thorough to set up a shot, then they will. If not, then they need to strike roughly on target. The right balance is clear in hindsight. For controllers, a very thorough approach to flight data recording with an electronic solution may result in too much head-down time. A very efficient approach may result in over-reliance on memory. The efficiency-thoroughness trade-off is a constant balancing act that is fundamental to the development of expertise.

4. Positivity helps (a lot)

Some teams, such as Belgium and Croatia, played with incredible self–belief and confidence. Positivity permeates effective teams, on and off the pitch, even when things are difficult. Having spent hundreds of hours with different fixed ATC teams, and in different units, it is clear that different teams and units develop particular cultures or personalities. For some, fun, friendliness and positivity are hallmarks.. This is something one can see and feel, as an outsider. We all know intuitively that working in a positive, joyful environment brings out the best in us. We all need to work on creating joy in work.

5. Respect is an attitude…and a non-technical skill

For me, two of the highlights of the World Cup were about respect. When England Won against Colombia on penalties, Manager Gareth Southgate consoled Colombia’s Mateus Uribe, who missed his shot. Southgate was perhaps mindful of the penalty that he missed as an England player. Southgate’s overall demeanour was not only respectful, but empathic, supportive, and measured: a great role model for managers.

Respectful people carry their respect with them wherever they go. The Japanese team – consistent with their culture – cleaned their own dressing room, and left a handwritten note of thanks – in Russian. This courtesy is also a sign of pride in work. Even the Japanese fans helped to clean the stadium after their side was knocked out. Perhaps there should be a separate trophy for the most respectful team and supporters. This year, Japan would have won that trophy.

But France won the World Cup after a superb run of matches. Writing this Editorial from France, it was a pleasure to see the French people celebrate their victory, against a strong and dynamic Croatian team.

Perhaps we can learn from the preparation, planning and practice that went into the World Cup, supporting such expert performances. Vive la compétence !

 

Posted in Culture, Human Factors/Ergonomics, Safety | Tagged , , , , , , , , , | Leave a comment