How to measure psychological safety

Psychological safety is a big concept in HR and management science. It’s about trusting each other, being able to speak up and saying “I fucked up” or discussing problems in the team openly. Sometimes it is presented as the holy grail of teamwork: If there is psychological safety performance increases, people are happy, etc. This post will describe how to measure psychological safety.

Key points

• Psychological safety measures your team openness to discuss errors and test existing processes and norms
• Psychological safety is most often measured using a survey with 7 questions.
• Using text data (e.g., emails, video transcripts, chat data) is an innovative way to measure psychological safety.

What is psychological safety?

Psychological safety was first coined by Professor Amy Edmondson. As a side note, she’s an incredibly humble person for the success she has achieved. Psychological safety is also discussed in Google's Aristoteles project.

Psychological safety is based on trust but goes beyond trust. It is a belief that describes if a team is a safe space for learning. Learning doesn't just happen but is an iterative, ongoing process alternating between action and reflection. Reflecting on the action by asking questions, seeking feedback, or discussing errors or unexpected outcomes is necessary. When team members are afraid to be embarrassed, rejected, or punished for speaking up, then there is no psychological safety.

Amy Edmondson specified that psychological safety should be shared among team members. This means that all team members should have a similar opinion about the level of psychological safety in the team. You should not have some people who think it is high, while others think it is low.

Side note: If team members have different opinions about the level of psychological safety in the team, this can indicate a lack of inclusion. For example, if in your company those from minority backgrounds consistently evaluate teams as less safe than those from the dominant background, you have pretty good evidence that your teams are diverse, but not inclusive.

Studies have shown that teams with a high level of psychological safety are able to learn more. Team learning is important if teams want to be able to adapt to changes. Psychological safety also has a positive impact on performance. Psychological safety opens the door to discuss errors or sub-optimal processes. This leads to improved practices and through this higher level of performance. More information about the importance of psychological safety is included in Amy Edmondson's book Teaming.

How to measure psychological safety?

When it comes to measuring psychological safety the most often used method, at least in scientific papers, is a survey. Amy Edmondson created the survey based on the interview she conducted. The questions have been used in many different settings. The answers range from strongly disagree (1) to strongly agree (5). If you plan to use this survey, the best is to use all of the questions as this gives you the most reliable measurement. I’ve included the questions below.

As team members should have similar opinions about the level of psychological safety, you should check for differences in opinion. The easiest way to do this is to take the average of all 7 questions for all individuals. Once you have that, look if the average is more or less the same (bar charts are great for this). A more accurate way is to  calculate the within-team level of agreement using the Inter-class correlation coefficient.

If the variation is too much, don't even calculate the average level of psychological safety for your team. There is none. That would be the time to talk with members of the team to find out what the problem is. What too much is, depends on your selected method to calculate the differences in team members’ opinions.

Psychological safety survey

1. If you make a mistake on this team, it is often held against you
2. Members of this team are able to bring up problems and tough issues.
3. People on this team sometimes reject others for being different. (This question is reverse-coded. This means that low scores on this question, indicate high levels of psychological safety. You have to convert the scores to be in line with the scores from the other questions.)
4. It is safe to take a risk on this team.
5. It is difficult to ask other members of this team for help. (This question is reverse-coded. This means that low scores on this question, indicate high levels of psychological safety. You have to convert the scores to be in line with the scores from the other questions.)
6. No one on this team would deliberately act in a way that undermines my efforts.
7. Working with members of this team, my unique skills and talents are valued and utilized.

Other methods for measuring psychological safety

Another way to measure psychological safety is to use chat messages or other communication data. Of course, this means that the communication is analyzed. Before I go ahead and explain how to do this, we need to talk about privacy and ethics.

You need to spend time thinking about the consequences of analyzing communication data. Do your employees trust you with their data? How do you make sure that the data is processed the right way? Who has access to the text data? Keep in mind that the person analyzing the text data will know exactly what others said. Is your company ready for that?

To make sure that the privacy of employees is not compromised, one method is to delete as much personally identifiable information as possible. For example, if your team uses Slack, all users have a username and a user id. For the analysis, you don't need to know the names, the user id is enough.

using communication data to measure psychological safety

Ideally, the team's communication data is analyzed using text analytical methods. For example, psychological process dictionaries can be used to count the frequency of positive or negative emotions. That can give an indication of how much love or hate there is in a team. You could also count the number of questions, or simply question marks that appear in a team's chat, and what the response is. A response could be a detailed explanation, a joke, or questions could simply be ignored.

Finally, you could create your own specific dictionary with words that describe psychological safety.  This dictionary could include words such as mistake, error, that's ok, I'll explain, you should have known better, you idiot. Words could have a value, with high values indicating that this word signals high levels of psychological safety. The words that's ok could have a value of 10 as it shows forgiveness, whereas the words you should have known better would have a value of 1.

I haven't yet seen an example of a team using their chat data as a measure for psychological safety. Sike Insights goes in the right direction, guiding employees to communicate properly. But they are not measuring psychological safety. I've spoken with Corine Tan recently to better understand their product and give advice. They are looking for partners to further test their product. Give it a try, it can help any teamwork better together by improving communication,

Next step

Measuring your team’s level of psychological safety is the first step. After that, the real work begins. What can you do to increase psychological safety? What aspect of psychological safety do you think needs to be increased?