HindSight 27 on Competency and Expertise is out now!

HindSight Issue 27 is now available in print and online at SKYbrary and on the EUROCONTROL website. You can download the full issue, including an online supplement, and individual articles. HindSight magazine is free and published twice a year, reaching tens of thousands of readers in aviation and other sectors worldwide. You will find an introduction to this Issue below, along with links to the magazine and the individual articles.

HS27 cover_Page_01


Welcome

“Welcome to Issue 27 of HindSight magazine. The theme of this issue is ‘Competency and Expertise’. It is a topic that links to all previous Issues of HindSight.

Our ability to work effectively depends on the competency and expertise front-line practitioners and all involved in the operational, technical, support, and management functions. Safety isn’t something that is just ‘there’ in the aviation system. People actively create safety. But how do we create safety? And what do we need to do to help ensure that we can continue to do so? Competency and expertise is an important part of the answer.

In this issue, we have articles from operational, safety, human factors and psychology specialists. This is part of what makes HindSight unique – it brings together those who do the operational work, those who support operational work in a variety of ways, and those who study operational work to help better understand it. We are proud to give a voice to some of the world’s leading academic thinkers, and to operational and support specialists who have stories, experience and practical insights to convey. The key is that the articles are interesting and useful to the primary readers of HindSight: air traffic controllers and professional pilots, and hopefully to others who support operational work. Do we succeed? Let us know! In this Issue we explore the nature of competency and fundamental applications and implications for operational training, selection, and procedures, including non-technical skills and contingency. We then zoom out to regulatory and future issues. The regular feature on ‘Views from Elsewhere’ continues with articles from surgery and rail. These articles raise questions for us in aviation, and provide some practical ideas. And in this issue we have articles drawing from the world of sport. HindSight continues online over at SKYbrary with further articles in the online supplement, from aviation and other industries, on the theme of competency and expertise.

We also have ‘What we do’ good practice snippets. We’d particularly like to hear from more readers for this section. And this brings me to the next Issue, which will feature articles on ‘Change’. All readers have been affected by changes, in procedures, regulations, technology, people, incentives, organisation, etc. The pace of change will only increase. How do we change to adapt to the dynamic world of air traffic management? And how do we as individuals, teams, and organisations adapt to these changes? Let us know, in a few words or more, for your magazine on the safety or air traffic management – HindSight.”

HindSight 27 Articles

Foreword

Editorial

Op-ed

Fundamental Issues

Non-technical Skills

Contingency

View from the Air

Regulatory Issues

Future Issues

Views from elsewhere

What we do

Interview

HindSight 27 On-line Supplement

See all editions of HindSight magazine

Advertisements
Posted in Human Factors/Ergonomics, Safety | Tagged , , , , , , , , , , , , , | Leave a comment

Twelve Properties of Effective Classification Schemes

Most organisations seem to use a classification system (or taxonomy) of some sort, for instance for safety classification, and much time is spent developing and using such taxonomies. Importantly, decisions may be made on the basis of the taxonomy and associated database outputs (or it may be that much time is spent on development and use, but little happens as a result). There is therefore a risk of time and money spent unnecessarily, with associated opportunity costs. Still, taxonomies are a requirement in all sorts of areas, and several things should be kept in mind when designing and evaluating a taxonomy. This posts introduces twelve properties of effective classification systems.


Effective classification schemes are difficult to develop. The following properties
need to be considered to develop a valid classification scheme that is accepted and produces the desired results.

1. Reliability

A classification scheme must be used reliably by different users (inter-coder reliability or consensus) and by the same users over time (intra-coder reliability or consistency). Reliability will depend on many factors, including the degree of true category differentiation, the adequacy of definitions, the level of hierarchical taxonomic description being evaluated, the adequacy of the material being classified, the usability of the method, the adequacy of understanding of the scheme and method, and the suitability of reliability measurement. Adequate reliability can be very difficult to achieve (see Olsen and Shorrock, 2010 $$), and the heterogeneity of methodologies employed by researchers measuring reliability of incident coding techniques make it more difficult to to critically compare and evaluate different schemes (see Olsen, 2013 $$). However, if a classification scheme cannot be used reliably, then it is usually fair to say that it is not fit for purpose, especially for analysing large data sets (though it may be that reliability is achieved for certain users in certain contexts)

2. Mutual exclusivity

Categories should be mutually exclusive on the same horizontal level, so that it is only possible to place subject matter into one category. This relates to reliability. There are varying degrees of mutual exclusivity, since categories often have things in common, or overlap to some degree, depending on the criteria. Mutual exclusivity tends to be lower for abstract or unobservable concepts. This is especially true for psychological labels, and even more so those that are all-consuming (such as ‘situation awareness’, ‘mental model’, or ‘information processing’). For properly differentiated categories with clear definitions, appropriate guidance can reduce sources of confusion (see Olsen and Williamson, 2017 $$).

3. Comprehensiveness (or ‘content validity’)

It should be possible to place every sample or unit of subject matter somewhere. However, choices must be made about the granularity of categories. Highly detailed classification schemes and classification schemes that offer little granularity suffer from different problems concerning mutual exclusivity, usability, face validity, usefulness, etc.

4. Stability 

The codes within a classification system should be stable. If the codes change, prior classification may be unusable, making comparison difficult. On the other hand, it should be possible to update a classification scheme as developments occur that truly affect the scope and content (e.g., new technology). Ideally, changes should have minimal impact.

5. Face validity 

A classification system should ‘look valid’ to people who will use it or the results emanating from it. An industry classification scheme should incorporate contextual and domain-specific information (‘contextual validity’), but should also sit comfortably with pertinent theory and empirical data (‘theoretical validity’). The best approach here is to stick with what is well-understood and accepted.

6. Diagnosticity (or ‘construct validity’)

A classification scheme should help to identify the interrelations between categories and penetrate previously unforeseen trends. This may relate more to the database and method than the taxonomy itself.

7. Flexibility

A classification scheme should enable different levels of analysis according to the needs of a particular query and known information. This is often achieved by a modular and hierarchical approach. Shallow but wide taxonomies tend to suffer from low flexibility.

8. Usefulness

A classification scheme should provide useful insights into the nature of the system under consideration, and provide information for the consideration of practical measures (e.g., for improvement).

9. Resource efficiency

The time taken to become proficient in the use of a classification scheme, collect supporting information, etc., should be reasonable. Continued difficulties in using a classification scheme, after initial training and supervised practice, usually indicate a design problem and signal the need for (re-)testing.

10. Usability

A classification scheme should be easy to use in the applied setting. This means that the developers should be able to demonstrate a human-centred design process akin to ISO 9241-210. The most relevant aspects of usability should be determined. For instance, some users may have formal training in the use of the classification scheme, little time to make inputs, limited understanding of terms and acronyms, etc.

11. Trainability

It should be possible to train others how to use the classification scheme and achieve stated training objectives, including any required levels of reliability. In some cases, there may be valid reasons to go to only to the original developers for training (e.g., the taxonomy is sensitive or commercialised). In such cases, there is a need to consider why this is the case, and the possible related implications (e.g., lack of peer reviewed, public domain accounts of development; lack of independent testing).

12. Evaluation

Classification schemes should normally be amenable to independent evaluation. This means that they must be available and testable on the requirements above using an appropriate evaluation methodology. This will of course be more difficult for taxonomies that are restricted for various reasons (commercial, security, misuse prevention, etc).

Summing up…

In practice, it will not be possible to achieve anywhere near perfection on these criteria. Even where evaluation results are very positive (assuming there is any evaluation), experience in use will usually be different (and usually worse from the users’ points of view) and undocumented. Trade-offs must be made and some of the properties above will be more important than others, depending on the application. For instance, in some cases, the priority may be to help investigators to ensure that relevant issues have been considered, perhaps also to model the interactions between them (see Four Kinds of Human Factors: 4. Socio-Technical System Interaction). In other cases, the priority may be to help analysts understand prevalence and trends in very large data sets.  In still other cases, the priority may be to help users with little time or knowledge (‘casual users’) make basic inputs. These user groups have different needs and expectations.

It may also be necessary to use a taxonomy that is not adequate on some of the criteria above. In all cases, there is a need to understand the possible risks (e.g., time spent using the taxonomy; decisions made on the basis of the data) and to manage these risks (e.g., ignore data for categories that are know to be unreliable; merge categories; analyse data based on a hierarchically higher category/level up). However, three basic activities should be undertaken to help achieve adequate validity:

  1. Involve appropriate stakeholders in taxonomic development and evaluation, with a focus on understanding their needs the associated taxonomic requirements, and the trade-offs between requirements. This should include people who understand human-centred design, taxonomy and all relevant aspects of the scope of the classification scheme.
  2. Review relevant literature, analyse the work and system, and review other classification schemes (including ones previously used by any stakeholders).
  3. Test the classification scheme throughout its development and implementation.

Afterword

This post is based on a short briefing note that I produced for an Australian government agency meeting in 2004, not long after being awarded a PhD related to taxonomy (461 pages; reading not recommended, but available on request). Since I sometimes find it hard to find this note, I thought it might be useful to put online, also in the hope that it might help someone else. The post focusses on the properties of effective taxonomies that relate to development, and not so much on the use, mis-use and abuse of taxonomies. Another post, maybe.

Posted in Human Factors/Ergonomics, Safety, systems thinking | Leave a comment

Vive la Compétence !

The text in this post is from the Editorial of HindSight magazine, Issue 27, on Competency and Expertise, available for download in late August at SKYbrary here.


France_champion_of_the_Football_World_Cup_Russia_2018.jpg

Image: Kremlin.ru [CC BY 4.0 (https://creativecommons.org/licenses/by/4.0)%5D, via Wikimedia Commons

This summer, we have been entertained by the world’s best footballers – experts in the game. And it just so happens that Competency and Expertise is theme of this Issue of HindSight. What might we learn from World Cup 2018? Here are five observations.

1. Past performance does not determine future performance

Some world-leading teams, which were favourites to win, were knocked out early, or didn’t qualify. It just goes to show that we can’t rely on our record. Past success does not guarantee future success. The same tactics that worked in the past will not necessarily work in the future.

But we humans are creatures of habit. In his famous book Human Error, James Reason (1990) described two ways that we rely – or over-rely – on our past experience. The first  is similarity matching. When a situation is similar to one experienced previously, we use pattern patching and tend to respond in a similar way to how we did before. The second is frequency gambling. More frequent solutions in roughly similar conditions will tend to prevail. Most of the time, these are efficient ways of working, and efficiency is critical when seconds count. But sometimes, we need to be more thorough, especially when  preparing, practising and planning. In any case, we must always adapt to the situation.

Just as past success does not guarantee future success, past failure does not guarantee future failure. Penalties were a case in point. Far from being a lottery that is impossible to rehearse for, or an event for which some teams are ‘jinxed’, this year showed that extensive physical and psychological preparation for such high pressure scenarios pays off.

This is something that I am particularly interested in within ANSPs. Front-line safety-critical staff need and deserve world-class training, especially refresher training. This isn’t a luxury. It’s a necessity, but the sort of necessity that sometimes becomes obvious only in hindsight. The same applies to team resource management training, and other training that integrates lessons from the past. The lessons that stick often come from past failures, but we need to learn those lessons in the right way, in the right context.

2. Teams are more than the sum of their parts…and success runs deep

It became clear in this World Cup that individual expertise does not equal team competence. Teams can suffer through overreliance on star players, but can benefit greatly from teamwork bonded with trust, respect, and an understanding of how each player will respond in a given situation. The same applies in air traffic management. Here, we have procedures to help us predict how others will respond. But procedures do not determine how someone will respond. They do not even apply to all situations, nor prescribe all responses. In this case, trust built from working together helps us to succeed.

In the World Cup, the team is not just the players on the pitch. The best managers set up their teams to win, using all necessary resources, and adapting their style to whatever will bring out the best from each player. Everything is designed and managed for human performance. Hundreds more, including psychologists, dietitians,
physiotherapists, etc, help players to perform at their peak. It is similar with ANSPs. While all have similar basic kinds of front-line support staff, some ANSPs have teams of qualified human factors/ ergonomics specialists, psychologists, TRM facilitators, CISM peers, educational specialists, etc. Human performance is what we do, but to be sustainably successful, it needs a strong support network.

3. Technology changes the nature of work

The introduction of video assistant referee showed how technology changes the nature of work. Referees now have to use their expertise to decide when to use the technology. Over-reliance ruins the spontaneity of play. Under-use may bring criticism that not only did a referee not spot a foul or offside, but that they didn’t use a tool that could have shown this: two mistakes, where previously there would have been only one

In The ETTO Principle, Erik Hollnagel discusses a fundamental trade-off that underlies human performance: the efficiency-thoroughness trade-off. Referees must balance efficiency against thoroughness to harmonise fluidity and fairness. Footballers do the same. If there is time to be thorough to set up a shot, then they will. If not, then they need to strike roughly on target. The right balance is clear in hindsight. For controllers, a very thorough approach to flight data recording with an electronic solution may result in too much head-down time. A very efficient approach may result in over-reliance on memory. The efficiency-thoroughness trade-off is a constant balancing act that is fundamental to the development of expertise.

4. Positivity helps (a lot)

Some teams, such as Belgium and Croatia, played with incredible self–belief and confidence. Positivity permeates effective teams, on and off the pitch, even when things are difficult. Having spent hundreds of hours with different fixed ATC teams, and in different units, it is clear that different teams and units develop particular cultures or personalities. For some, fun, friendliness and positivity are hallmarks.. This is something one can see and feel, as an outsider. We all know intuitively that working in a positive, joyful environment brings out the best in us. We all need to work on creating joy in work.

5. Respect is an attitude…and a non-technical skill

For me, two of the highlights of the World Cup were about respect. When England Won against Colombia on penalties, Manager Gareth Southgate consoled Colombia’s Mateus Uribe, who missed his shot. Southgate was perhaps mindful of the penalty that he missed as an England player. Southgate’s overall demeanour was not only respectful, but empathic, supportive, and measured: a great role model for managers.

Respectful people carry their respect with them wherever they go. The Japanese team – consistent with their culture – cleaned their own dressing room, and left a handwritten note of thanks – in Russian. This courtesy is also a sign of pride in work. Even the Japanese fans helped to clean the stadium after their side was knocked out. Perhaps there should be a separate trophy for the most respectful team and supporters. This year, Japan would have won that trophy.

But France won the World Cup after a superb run of matches. Writing this Editorial from France, it was a pleasure to see the French people celebrate their victory, against a strong and dynamic Croatian team.

Perhaps we can learn from the preparation, planning and practice that went into the World Cup, supporting such expert performances. Vive la compétence !

 

Posted in Culture, Human Factors/Ergonomics, Safety | Tagged , , , , , , , , , | Leave a comment

Human Factors at the Fringe: BaseCamp

A legendary rivalry: one mountain and two climbers seeking to be the best. We join them at basecamp as they prepare for the challenges of the ascent. Invited into separate tents to join just one of the two climbers, audiences experience the subjective and different sides of this rivalry, sharing only one side of the story. As time passes, the voices travel through the camp and the line between truth and lies, fact and fiction, begin to blur. Award-winning Fever Dream Theatre return after their 2016 sell-out hit Wrecked. ‘Stays with you long after you’ve left’ (NME).

Basecamp, Fever Dream Theatre, C South  (Venue 58), Edinburgh, 4-13 & 15-27 August 2018

(See Human Factors at The Fringe for an introduction to this series of posts.)

As you meet the two climbers at the venue – ‘BaseCamp’ – you are taken into one of two tents. The climbers are raising money for their next climb, and you will hear about one of their climbing lives.

You are taken into a canvas tent and the climber starts to talk about climbing – her passion. You noticed on being introduced to the two climbers initially that there was tension between the two, and as your host continues her story, the knotty relationship between her and her friend in the other tent surfaces. Your host seems honest and credible. In the other tent, people are hearing from the other climber. You don’t know what she’s saying, and perhaps you never will. You will only hear one side of the story. Do you get the feeling that you’re not hearing the whole story, that you are missing part of the picture? Are you curious to find out? Or are you content with the version of events that you have heard?

In many work situations, we rely on the accounts that people provide. This is what I call Work-as-Disclosed.

“This is what we say or write about work, and how we talk or write about it. It may be simply how we explain the nitty-gritty or the detail of work, or espouse or promote a particular view or impression of work (as it is or should be) in official statements, etc. Work-as-disclosed is typically based on a partial  version of one or more of the other varieties of human work: Work-as-imagined, work-as-prescribed, and work-as-done. But the message (i.e., what is said/written, how it is said/written, when it is said/written, where it is said/written, and who says/writes it) is tailored to the purpose or objective of the message (why it is said/written), and, more or less deliberately, to what is thought to be palatable, expected and understandable to the audience. It is often based on what we want and are prepared to say in light of what is expected and imagined consequences.” From The Varieties of Human Work

BaseCamp provides two versions of Work-as-Disclosed. To some extent, each may contain P.R. and Subterfuge

“This is what people say happens or has happened, when this does not reflect the reality of what happens or happened. What is disclosed will often relate to what ‘should’ happen according to policies, procedures, standards, guidelines, or expected norms, or else will shift blame for problems elsewhere. What is disclosed may be based on deliberate deceit (by commission or omission), or on Ignorance and Fantasy, or something in between… The focus of P.R. and Subterfuge is therefore on disclosure, to influence what others think.” From The Archetypes of Human Work: 6. P.R. and Subterfuge

Each version of events seems credible, and as you listen to the story, for nearly an hour, you develop a felt rapport with the reporter. How much do you want to hear a second account? And if you do hear another account, how will you respond to conflicts with the account that you have heard, and trusted?

In these sorts of situations, at home, in organisations, in courtrooms, we often hear and accept the stories that we want to hear. Sometimes we choose not to hear the stories that we don’t want to hear. We may also choose the sequence of the stories that we hear, or else this might be forced upon us by others or by circumstance. In safety investigations, formal inquires, court cases and disputes of all kinds, who you chose to (or are able to) listen to, and the order in which you listen, will affect the story that you create about what happened. By hearing only from clinician(s), but not the patient and family, for example, your story will lack the perspectives and details that are required for a more thorough understanding. And the order in which you listen to people, even when you listen to many, will affect what you hear in subsequent accounts because it will affect your questions, your mental set and perceptual filter. This is an ‘anchoring’ heuristic that has been researched extensively in the context judgement. Mostly, people think about anchoring in the context of quantitative judgement:

‘In many situations, people make estimates by starting from an initial value that is adjusted to yield the final answer. The initial value, or starting point, may be suggested by the formulation of the problem, or it may be the result of a partial computation. In either case, adjustments are typically insufficient (Slovic & Lichtenstein, 1971). That is, different starting points yield different estimates, which are biased toward the initial values. We call this phenomenon anchoring.” Tversky & Kahneman (1974)

Anchoring can also affect our understanding of stories, by anchoring our expectations, questions, and desire for certainty.

There may indeed be misunderstandings between different parties to an event, because they each has partial knowledge and information, because each has different goals and expectations, and because each sees things from different perspectives and resolutions. This is the case with BaseCamp. Not only are there inconsistencies between the accounts, there is a crucial unspoken aspect to each of their thinking about the relationship and the factual and counterfactual aspects of a critical event. They don’t know because it is a Taboo, and you will only know if you hear both stories, or if you can, as two listeners, piece together the aspects of the stories.

In the EUROCONTROL ‘Systems Thinking for Safety: Ten Principles‘ White Paper, the term field experts was used to describe people who possess expertise relative to their own work-as-done.

“The perspectives of field experts need to be synthesised via the closer integration of relevant system actors, system designers, system influencers and system decision makers, depending on the purpose. The demands of work and various barriers (organisational, physical, social, personal) can seem to prevent such integration. But to understand work-as-done and to improve the system, it is necessary to break traditional boundaries.” From: Systems Thinking for Safety/Principle 1. Field Expert Involvement

There are many influences on who speak to, how, for how long, and when, for example:

  • Desire for certainty – by introducing new accounts, we may well introduce uncertainty, which may bring us anxiety.
  • Prejudice and confirmation bias – we may have a predetermined goal to achieve, or a preconceived idea about what happened and who is responsible for an outcome, and choose (more or less consciously) who and how we speak to people in order to confirm our hypothesis.
  • Time – listening to different accounts takes time, which is always limited. Even when there is time, we may perceive as better spent on something else (e.g., analysis, reporting, action). Sometimes, system constraints such as regulations can force the issue (see the example here).
  • Theory of causation – we may perceive that that those closest to an event (e.g, an air traffic controller) are ‘causal’ to it, and therefore important to hear, while those less close to an event (e.g,, a procedure writer) are merely ‘contributory’ to it (and therefore less important to hear). The second group are rarely interviewed, and so we tend to hear the first story, and not the second story (see talk here).
  • Expertise – we may simply lack the competency to investigate an issue appropriately.

Broadly these and other influences relate to barriers to new thinking about systems and safety, outlined here.

Multiple perspectives are not a sources of weakness. Diversity is a source of resilience, even – or especially – when accounts do not agree. This is counterintuitive for those who wish to have a straightforward, perhaps mechanistic, account.

This advice might help (adapted from Systems Thinking for Safety Ten Principles White Paper and Learning Cards):

  • Listen to people’s stories. Consider how people can best tell their stories from the point of view of how they experienced events at the time. Try to understand the person’s situation and world from their point of view, both in terms of the context and their moment-to-moment experience.
  • Understand their local rationalities. Be curious about how things make sense to people at the time. Listen to people’s individual goals, plans and expectations, in the context of the flow of work and the system as a whole. Focus on their ‘knowledge at the time’, not your knowledge now. Understand the various activities and focus of attention, at a particular moment and in the general time-frame.
  • Seek multiple perspectives. Don’t settle for the first explanation; seek alternative perspectives. Discuss different perceptions of events, situations, problems and opportunities, from different people and perspectives, including those who you might think are not directly involved. Consider the implications of these differential views. One way to do this is to adopt a group approach to debriefing, as explained in this Etsy Debriefing Facilitation Guide on leading groups to learn from accidents, by John Allspaw @allspaw, Morgan Evans @NeonMorgan, and Daniel Schauenburg @mrtazz.

I will leave you with this – an advertisement of my childhood, which remains my favourite of all time. I talk about it here.

“An event seen from one point of view gives one impression. Seen from another point of view, it gives quite a different impression. It’s only when you get the whole picture that you fully understand what’s going on.”

You may well have to accept that you can never fully understand what went on. But you can get past the basecamp of understanding.


See also:

Human Factors at The Fringe

Human Factors at The Fringe: The Girl in the Machine

Human Factors at The Fringe: My Eyes Went Dark

Human Factors at The Fringe: Nuclear Family

Human Factors at the Fringe: Lemons Lemons Lemons Lemons Lemons

Posted in Human Factors/Ergonomics, Safety, systems thinking | Tagged , , , , , , , , , , , | 1 Comment

The Safety-II Dance: A Podcast by Greater Than Code

A few weeks ago, I had a chat with Jamey HamptonJessica KerrJohn K. Sawers of Greater Than Code. Here is the podcast that resulted, expertly produced by Mandy Moore.

In the podcast, we roamed around topics of human factors/ergonomics, system performance and human wellbeing, empathy, appreciative inquiry, asset-based community development (ABCD), and Safety-II.

All Greater Than Code podcasts are on their website and on iTunes.

GmmVl5De_400x400

Posted in Culture, Human Factors/Ergonomics, Safety | Tagged , , , , , , , | 1 Comment

Suitably Qualified and Experienced? Five Questions to ask before buying Human Factors training or consultancy

Ergonomics (or human factors) is the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance. (IEA, 2018)

This definition – accepted by human factors and ergonomics (HF/E) societies worldwide – emphasises that HF/E is a discipline and profession. A discipline is “a branch of knowledge, typically one studied in higher education”. A profession is “a paid occupation, especially one that involves prolonged training and a formal qualification” (Oxford dictionaries).

Practitioners of ergonomics and ergonomists contribute to the design and evaluation of tasks, jobs, products, environments and systems in order to make them compatible with the needs, abilities and limitations of people. (IEA, 2018)

This contribution tends to be made by HF/E practitioners in two ways:

  1. as an external human factors consultant/trainer
  2. as an in-house human factors specialist (a typical job description is here).

But how do we assess whether a practitioner is a ‘suitably qualified and experienced person’ (SQEP)?

This is an important question because there is so much at stake for system performance and human well-being, but it is not straightforward to answer. In this post, I provide five questions that will help. The questions are for reflection and discussion. They are not definitive. In considering these questions, the point is not necessary to answer “yes’ to every question. Some will be more relevant than others, and there will be exceptions. But especially where the answer to two or more questions is “no”, there should be careful consideration as to why this is the case.

The emphasis of this post is not on those who fulfill specific HF/E roles in-house (e.g., HF/E in medical simulation). In such cases, internal practitioners with HF/E-related roles may well have education and experience in a specific area of HF/E, and use this in their role. But they would probably not describe themselves as ‘HF/E specialists’ (just as I have education in counselling but would not call myself a counsellor). This post does not cover these in-house practitioners, though they may wish to consider the questions and what support they might need.

Rather, the post concerns paid-for HF/E consultancy and training, and also employment as an HF/E specialist, where one has to abide by ethical professional standards in the practice of HF/E.

3534516458_48e4e8595f_b

Marco Bellucci CC BY 2.0 https://flic.kr/p/6okjAW

1. Qualification

Do they have a recognised qualification in HF/E?

There are several academic programmes in HF/E in the UK, USA, and other countries, which you can find via the relevant Society or Association in your country. Some of these programmes will be accredited by your national HF/E Society (the Centre for Registration of European Ergonomists offers is a guide to such courses in Europe).

An HF/E qualification gives reassurance that the person has undertaken an approved programme of study in HF/E, which addresses the relevant competencies (e.g., the CIEHF Professional Competency Guidance, or the Requirements for Registration of European Ergonomists in Europe). (But note that some qualifying courses are no longer offered and so may not be listed.)

Others academic programmes will not be accredited, but will offer a substantial component of HF/E as part of a mixed programme, or as a substantial part of (e.g., a major in) a programme in experimental psychology, industrial engineering, systems engineering, patient safety, occupational health and safety, etc. This is especially true in the USA), which only a small minority of the programmes listed on the Human Factors and Ergonomics Society website are accredited by the Human Factors and Ergonomics Society. Most Human Factors practitioners (HF being the dominant term used in the US) tend to have academic qualifications in psychology.

For specialist external HF/E consultancy and commercial HF/E training support, a university degree in HF/E (or closely related discipline, as listed by the HFES in the USA) will usually be necessary, and perhaps a higher postgraduate (e.g., Doctorate) degree in very specific circumstances (e.g., expert witness work).

2. Accreditation and Membership

Do they have an appropriate level of accreditation or membership of an HF/E related professional organisation?

Unlike some professions, the terms ‘human factors specialist’ or ‘ergonomist’ are not legally protected, or regulated (e.g., by the Health and Social Care Professions Council in the UK), and so are not regulated with titles that are legally protected (e.g. Registered Occupational Psychologist, Registered Dietitian, Registered Physiotherapist).

However, HF/E is subject to accreditation (e.g., registration, certification, and chartership) in many countries (e.g., UK, USA, Canada, Australia, NZ, and Europe as a whole). So perhaps the easiest way to have confidence in the competency of an HF/E consultancy, training provider, or individual practitioner is to check for accreditation. This varies throughout different countries. In the UK, the Chartered Institute of Ergonomics and Human Factors provides various accreditations via Chartership, which is conferred to those members who fulfil certain criteria. This includes “having a high level of qualification and experience and being able to demonstrate continuing professional development”. Additionally, different grades of membership of the CIEHF – Fellow, Registered Member, Graduate Member, Technical Member – reflect competency, proficiency and experience.

Member and consultancy directories of HF/E Societies and Associations are available to help. For instance, Members of the HFES can be seen here. Chartered Members of the CIEHF can be seen here. Registered Consultancies that are accredited by the CIEHF can be seen here. You can find other directories of individuals and organisations via the relevant Society or Association in your country. (Note that ‘Associate’ or ‘Affiliate’ Membership is, in most cases, available to anyone and indicates interest and commitment – since all members have to abide by the Code of Conduct – but does not provide assurance of qualifications or experience. Therefore a minimum membership grade for paid support should typically be Graduate or Technical Member.)

In some cases, those who identify as ‘human factors specialists’ will have accreditation via other professional organisations. Typically, these relate to psychology and engineering. Some human factors specialists will be Chartered Psychologists in the UK. (There are other organisations relating to psychology and human factors, often in specific sectors, but these are not recognised by the International Ergonomics Association, which is the umbrella organisation for Human Factors and Ergonomics worldwide. These other organisations also sometimes require members to purchase the organisations’ own training for accreditation, which raises questions that are beyond the scope of this post.) The point is that many who are accredited via another route (e.g., Chartered Psychologist or Chartered Engineer) may well be competent HF/E practitioners, but perhaps for specific aspects of HF/E and not in the whole score of HF/E, and may have a different perspective (e.g., more aligned with psychology) and different approach (e.g., more cognitive-behavioural, social-organisational).

Accreditation will require that the person undertakes appropriate continued professional development, and submits evidence of this. The is important, but difficult for buyers of consultancy and training services to assess. Accreditation and membership removes some of that burden, because the Society does this as a requirement of the person’s membership.

3. Code of Ethics

Do they abide by a code of ethical conduct from an HF/E-related society or association?

This issue is covered by Accreditation above, but it its worth considering specifically because it is so important. A person offering HF/E consultancy or training services who is a member of an IEA Federated Society will have to abide by the Code of Conduct of that Society. The person should be aware of the Code. In any case, the Code (e.g. the CIEHF Code of Conduct) will cover such ethical standards, such as:

  • working within limits of competence
  • representation and claims of effectiveness
  • supervision
  • respect for evidence
  • confidentiality
  • impartiality
  • probity
  • considerations of religion, gender, race, age, nationality, class, politics or extraneous factors.

Professional societies of other disciplines and professions (e.g., psychology, engineering, health and safety) will also have codes of ethical conduct, and while these will not reference ergonomics, they will refer to similar sorts of issues mentioned above, and so working within competence would normally be formally recognised as an ethical issue.

This is an important question to ask anyone offering HF/E services and training, or seeking a job as an HF/E specialist.

If the person is not operating under the Code of Conduct of a professional organisation, then the protections available are limited to those under the law.

4. Experience

Do they have experience in the HF/E work and in the domain of interest?

The question here is whether the person has relevant experience in:

  • the kind of HF/E work (e.g., interface design, fatigue assessment, human error identification, cognitive work analysis, manual handing assessment), and
  • the sector of application (e.g., manufacturing, oil and gas, aviation, healthcare).

The first is the more important of the two, since HF/E – more than many other disciplines and professions – applies across sectors. HF/E practitioners tend to spend time in several sectors in their career. However, sector knowledge is important and HF/E specialists with a deep knowledge of one sector will have a greater understanding of the stakeholders, activities, procedures, technologies, regulations, cultures, etc. So at a micro level of application (e.g., the design of display elements or manual handling), much in HF/E crosses sectors. But at a macro level (e.g., the integration of HF/E throughout an organisation), this is not the case. When it comes to training others in aspects of HF/E (e.g., short courses), experience in the sector is a huge advantage, if not essential.

If the HF/E specialist offering consultancy or training services is accredited, then issues will be covered by the Code of Conduct or Ethics of their HF/E Society or Association, and the person will have to abide the relevant requirements (it is the focus of several items of the CIEHF Code of Conduct).

5. Social recognition

Is the person recognised as an HF/E specialist by other qualified HF/E specialists?

It can be hard to know if a person is suitably qualified and experienced, though answering ‘yes’ to the above will suggest that the person is. But there will be occasions when people fall outside of one or more of the criteria above, but where HF/E colleagues and associated would say that the person is an HF/E specialist. This will tend to involve those who specialise in a specific aspect of HF/E, but perhaps do not call themselves human factors specialists or ergonomists (and perhaps use other terms, such as UX designer, interaction designer, etc), and who are not a member of an HF/E Society or Association (e.g., a Technical Member of the CEIHF). Such people may well use HF/E theory and methods appropriately, and may even be an recognised expert in the specialism. In this case, social recognition by experienced HF/E specialists will give a good indication.

Summing up

To sum up, here are the five criteria and questions that apply to paid-for human factors and ergonomics (HF/E) consultancy and training support and employment, that may help with reflection and discussion.

1. Qualification – Do they have a recognised qualification in HF/E?

2. Accreditation – Do they have an appropriate level of membership of an HF/E related professional organisation?

3. Code of Ethics – Do they abide by a code of ethical conduct from an HF/E related society or association?

4. Experience – Do they have experience in the HF/E work and the domain of interest?

5. Social recognition – Is the person recognised as an HF/E specialist by other qualified HF/E specialists?

The aim of these criteria and questions is to ensure that professional standards – including ethical standards – are met. The criteria and questions are frames above in the context of HF/E, but in fact they apply to any professions, such as psychology, dietetics, or physiotherapy. Proper consideration of the criteria and questions should help to protect organisations, individuals, and the integrity of the profession.

Further Reading

Education and application is discussed practically (in the context of aviation, but applicable more generally), in:

Hawkins, F. H. (1987). Human factors in flight.. Gower Technical Press, pp. 326-341.

Posted in Human Factors/Ergonomics | Tagged , , , , , , | 1 Comment

The Loneliest Profession in Healthcare

IMG_20180511_114240 3.jpg

Steven Shorrock CC BY-NC-SA 2.0 https://flic.kr/p/HoF2kK

Health and social care is one of the biggest employers in developed countries, and the National Health Service (NHS) in the United Kingdom is one of the largest employers in the world. By some calculations, the NHS is the fifth biggest organisation in the world in terms of number of staff, calculated at around 1.5 million, or 1.25 million full-time equivalent. That figure does not even include temporary staff, general practitioners, dentists, optometrists, and other staff in the independent sector or private hospitals.

It is an organisation facing enormous demand. Looking at just a few headline aspects, the NHS deals with over 1 million patients every 36 hours, with over 16 million hospital admissions in 2015/16, over 23 million attendances at Accident & Emergency departments in 2016/17, and over 89 million outpatient attendances in 2015/16. In terms of NHS net expenditure, meeting this demand cost over £120 billion in 2016/17, and is expected to rise to over £126bn in 2018/19.

Not surprisingly, it is also a bafflingly complex organisation, in terms of: the variable and unpredictable nature of demand; the huge variety of staff roles and competencies; the incalculable number of different types of equipment (which are very often not designed according to ergonomic standards) and medicines (which often look or sound alike); the tens of regulators, professional bodies, and associations; the thousands of laws, regulations, diktats, policies, procedures, guidelines, and good practice documents for clinical and non-clinical staff; the complicated record-keeping and communication channels; the links to government agencies, local authorities, police and fire services, suppliers, independent providers, universities; and the interactions between all of these that make it such a complex sociotechnical system of systems.

Added to this are the elements of the system that cannot easily be seen, let alone counted, but strongly affect human behaviour, including: shifting goals, incentives, punishments, subcultures, and pressures from the public, media, regulatory and professional bodies, politicians, and associations.

You’d expect, then, that Human Factors/Ergonomics would be very relevant to the NHS (the two terms are seen as equivalent within the discipline, though the terms are used in different contexts). After all, HF/E is:

“the scientific discipline concerned with the understanding of interactions among humans and other elements of a system, and the profession that applies theory, principles, data and methods to design in order to optimize human well-being and overall system performance” (International Ergonomics Association).

Healthcare organisations should naturally be interested in human well-being, of patients and staff. It is obviously important that the NHS, as a system of systems, performs effectively.

So you’d also expect that many HF/E practitioners would be embedded in the NHS, directly employed by the NHS and its Trusts, just like quality improvement specialists, performance management specialists, human resources specialists, and indeed even staff who work directly with patients, such as Dietitians. After all, the role of HF/E practitioners is directly relevant to the effective provision of health services, as per the ‘7 key principles that guide the NHS in all it does‘.

“Practitioners of ergonomics and ergonomists contribute to the design and evaluation of tasks, jobs, products, environments and systems in order to make them compatible with the needs, abilities and limitations of people.”  (International Ergonomics Association).

HF/E practitioners are well-embedded in a number of sectors, notably aviationraildefenceoil and gasnuclearmanufacturingregulationproduct designinclusive design, and UX.  NATS – a provider of air traffic services in the UK and international airports, airlines and governments – employs around 25 qualified Human Factors practitioners (comprising Human Factors/Ergonomics practitioners and Psychologists). The Rail Safety and Standards Board (RSSB), Network Rail, London Underground, BAe SystemsQinetiQ (formerly part of the Defence Evaluation and Research Agency [DERA]), and other large organisations and regulators such as the Health and Safety Executive, all have long-established teams of qualified HF/E practitioners, who help to optimise system performance and human well-being by contributing to the design and evaluation of tasks, jobs, products, environments and systems in order to make them compatible with the needs, abilities and limitations of people.

Compared to any of these organisations, the NHS is enormous. To make one comparison, NATS is an organisation of around 4,500 staff – a third of the staff in some of Britain’s biggest hospital trusts. The NHS is 333 times bigger than NATS in terms of staff. Even individual hospital Trusts are enormous, with staff counted in the thousands, and up to 15,000 – three times bigger than the whole of NATS.

Of the 1.5 million members of staff, professionally qualified staff make up over half (53.8 per cent) of the Hospital and Community Health Service (HCHS) workforce (based on FTE). Healthcare obviously requires professionally qualified and accredited staff in order to provide effective services.

So how many professionally qualified HF/E practitioners are there in the NHS? It is not straightforward to answer, because ‘human factors practitioner’ (or human factors specialist, etc) and ‘ergonomist’ are not protected titles. Anyone may describe themselves as such (much as anyone can call themselves a ‘psychologist’, which is not a specifically protected title). But the profession of Human Factors/Ergonomics is Chartered in the UK, like others such as Chartered Psychologist, Chartered Engineer and Chartered Accountant. Chartered status in HF/E is conferred by the Chartered Institute of Ergonomics and Human Factors to those members who fulfil certain criteria. This includes “having a high level of qualification and experience and being able to demonstrate continuing professional development”, and operating under a Code of Conduct.

We can therefore count the number of ‘Chartered Ergonomists and Human Factors Specialists’ (CErgHF) employed by NHS Trusts, since a list of CErgHFs is published here. At the time of writing, there are 429 CErgHFs on the register. While there are others with various qualifications, and those who may have ‘human factors’ in their job title, the CIEHF is the only arbiter that provides a countable, unarguable category (as is the case for a Surveyor or Accountant) for the purposes of determining how many work in NHS Trusts.

From my own research, I have determined the number of NHS Trusts that directly employ a Chartered Ergonomics and Human Factors Specialist, and the number CErgHFs in the 233 NHS Trusts.

That number is 1.

Whichever way you look at it, the number is one.

To my knowledge, after researching the network of CErgHFs, one Trust employs a CErgHF as a Human Factors/Ergonomics specialist, and that one Trust employs – at present – one CErgHF.

Let that sink in for a moment.

An organisation of 1,500,000 staff and £120 billion expenditure.

If the number of HF/E specialists in NATS (which works in an ultrasafe sector – commercial aviation) were scaled up to the number of staff in the NHS, there would be over 8,000 HF/E specialists. Obviously, that is neither feasible nor necessary. But even if there were just one CErgHF per Trust, then there would be 233 CErgHFs in Trusts, plus those who should certainly be in other central bodies, such as NHS Education (NHS Education Scotland has a CErgHF working in an HF/E role), NHS Improvement, NHS Digital, etc.

Even one CErgHF for a Trust of up to 15,000 staff is, however, inadequate, especially given the thousands of so-called ‘excess deaths’ every year, and the apparent focus on ‘Human Factors’ in bodies such as:

The tens of thousands of so-called ‘excess deaths‘ should lead us to question why the NHS Trusts have only one Chartered Ergonomist and Human Factors Specialist. Even if HF/E could help prevent just a relatively small number of deaths and injuries, then we must ask why it is not being integrated professionally. HF/E focuses on many aspects of healthcare that must be designed properly in order to deliver safe, effective, and cost-efficient services, and relates directly to the work of the NHS, especially (NHS England):

In other industries, HF/E contributes to to the design and evaluation of tasks, jobs, products, environments and systems, for improved system performance and human well-being, in terms of:

  • the design and evaluation of equipment
  • the design of tasks and jobs
  • the design of physical and ambient environments
  • the design of policies, procedures, checklists, guidelines and job aids
  • human factors integration into management systems
  • human performance assessment and support
  • safety assessment (and risk assessment generally)
  • incident and accident investigation and analysis
  • staffing and manpower planning
  • shift design and fatigue risk assessment and management
  • stress management
  • communication design
  • safety culture and organisational culture evaluation
  • non-technical skills, team resource management, and (simulation) training.

Of these functions, the main focus of human factors integration in the NHS has been the latter. Many clinicians and educators have embraced human factors and integrated it into their non-technical skills, team resource management, and simulation training. It’s perhaps an obvious place to start, it’s vitally important, and it’s very well done (in fact, simulation training in some Trusts could teach aviation a thing or two – and has here and here). There are also a few individuals in the NHS with HF/E qualifications and experience, but who are not Chartered. Aside from a small number (mostly in Scotland), these do not work as HF/E specialists per se and/or are not performing the activities above for Trusts.

And there are Chartered HF/E specialists in medical device design companies outside of the NHS, and of course in consultancies and universities, who consult or conduct research in healthcare organisations.

More generally, there is significant interest in HF/E from clinicians; so much so that one would expect that it was actually integrated into the NHS. The term ‘human factors’ regularly crops up on social media and in conferences (though it could mean anything).

But none of this embeds a systematic consideration of designing for humans in healthcare, which is part of normal business in many high-risk industries. This can only really be done with trained and qualified practitioners, just as is the case with Physiotherapists, Dietitians, and Counselling Psychologists. In the absence of the HF/E equivalent in the NHS, front-line staff are having to do what they can, with the time they have, and otherwise work around and patch up problems in resources, constraints and environments that are:

  • not understood at the blunt end
  • always changing
  • not implemented or functioning as originally designed or imagined
  • often not ‘designed’ at all, and
  • degraded and stretched beyond design intent.

When things go wrong, it is patients, families, and clinicians who suffer the consequences.

And yet, of the 233 NHS Trusts, only one Trust employs a Chartered Ergonomist and Human Factors Specialist in an HF/E role, and that Trust employs – at present – one CErgHF.

It’s incredible. But it’s true.

Posted in Human Factors/Ergonomics, Safety | Tagged , , , , , , | Leave a comment