FAQs: Reporting Tools & Function

In this article: 

 

Key Drivers of Engagement

What is a key driver of engagement?

  • A key driver of engagement is a survey question (aka survey item) that has a larger impact on employee engagement compared to other survey questions.

Why are key drivers of engagement valuable?

  • An employee engagement survey measures perceptions around important workplace topics. The feedback from these surveys provide excellent insight into what employees find favorable unfavorable about the company.

    Key drivers of engagement are valuable in that they help narrow the focus on what really matters to the employees. These can be relative strengths of the organization or relative opportunities to. By improving perceptions on these questions, employee engagement is much more likely to increase and be sustained. 

    For example, an organization's least favorable question could be, "I am paid fairly." The organization may then decide to give everyone a pay raise and assume the problem solved. However, if the question isn't a key driver of engagement the pay raise is unlikely to make employees stronger advocates, show more discretionary effort or improve the chances of them staying with your organization. 

    While it is important that employees aren't paid below market standards and that they understand their compensation plan, solving your least favorable survey question is not likely to improve employee engagement in your organization. 

How are key drivers of engagement determined? 

  • Quantum Workplace's employee engagement surveys are created with two types of questions- Outcomes and Drivers. 
    • Outcomes are at the core of how we measure employee engagement and help reveal the currents state of employee engagement at an organization. Outcomes are not actionable because they are targets that organizations should strive to maintain or enhance. 

      Outcome questions can look like, "I recommend this organization as a great place to work." When outcome questions are consistently favorable, employee advocacy of the organization follows. 
    • Drivers (sometimes known as Cultural Diagnostics) help organizations identify ways to influence engagement outcomes. 

      Drivers often assess perceptions of career growth and development, communication, trust in leadership, future outlook and individual needs. A driver analysis determines which questions are key drivers of engagement to help tailor surveys with actionable questions based on feedback. 
  • For those wanting a detailed explanation on the statistical analysis - key drivers are calculated in a two-step process.
    1. Calculate overall engagement of each respondent. 

      Overall Engagement = Sum of scores for all engagement outcome survey questions. This is typically nine outcomes, each rated on a 1-6 favorability scale. 

      For example- if a respondent rates all nine outcomes as "1" or least favorable, we add the nine scores of "1" for a sum of nine. If all nine outcomes are scored with "6," the sum is 54. Most overall engagement scores will fall in between. 

      Note- If your organization used more than nine outcome questions or prefer a different scale, the numbers in the calculation will change accordingly. 
    2. Determine which questions (non-engagement outcomes) are most related to employees' overall engagement ratings using a Pearson product-moment correlation, or "Pearson Correlation."

      Higher Pearson Correlation coefficient = survey question is more related to overall engagement. 

      Key drivers of engagement are the questions with the highest correlation coefficients. 

How many key drivers of engagement are calculated?

  • Quantum Workplace typically calculates the top ten key drivers of engagement.

    However, if your organization prefers shorter employee engagement surveys, a shorter list such as only the top five can also be calculated. 

Does each team receive a unique set of key drivers of engagement?

  • No- Quantum Workplace typically calculates one set of key drivers of engagement for an entire organization. This is for both practical and statistical reasons. 

    Group size can strongly influence the statistical analysis. Practically, a list of ten key drivers allow for consistent effort internally and provide different teams with different areas to focus as strengths (more favorable) and opportunities (less favorable) vary across teams. 

Will our key drivers of engagement change year-to-year? 

  • Quantum Workplace conducts the driver analysis after each census engagement survey. While an organization's key drivers of engagement may change, we tend to see more consistency than change year-to-year. 

    We believe it is important that we calculate key drivers after each census survey to capture what is most important to your employees' engagement as your organization and workface evolves over time. 

Are my organization's key drivers of engagement typical?

  • While your organization's list of key drivers of engagement are unique to your organization, we do see common themes across different organizations. The following survey questions commonly appear in an organization's list of key drivers:
    • I find my job interesting and challenging.
    • My job allows me to utilize my strengths.
    • This job is in alignment with my career goals.
    • My opinions seem to count at work.
    • I see professional growth and career development opportunities for myself in this organization.
    • I believe this organization will be successful in the future.
    • I trust our senior leaders to lead the company to future success.
    • If I contribute to the organization’s success, I know I will be recognized.
    • The senior leaders of the organization value people as their most important resource.
    • I know how I fit into the organization’s future plans.

    As you can see, there are strong themes of career growth and development, feeling valued and future outlook common across different organizations. 

Which key drivers of engagement should we focus on?

  • Rather than trying to focus on all ten key drivers of engagement, we encourage teams to select one or two key drivers to focus on in the months after the engagement survey. Within your list of key drivers, we recommend choosing:
    • Key drivers that have opportunity for improvement, to further strengthen the team's engagement. 
    • Key drivers that are in a team's control of improving, to set the team up for success.
    • Key drivers that a team is passionate about focusing on, to ensure buy-in and shared ownership within the team.

What is favorability?

Favorability is the combination of responses that are either "Agree" or "Strongly Agree" on our six-point rating scale. 

For example, if a survey question has a favorability rating of 80% across an entire organization, then 80% of all respondents rated that item with either a 5 (Agree) or 6 (Strongly Agree).

Why are the numbers of respondents different for different questions? 

The number of respondents next to a question tells you how many employees responded to that specific question.

The number can vary by question if survey takers chose not to answer a certain question, or if they closed their browser before submitting their survey. Our reporting captures responses for all answered items, whether the survey was submitted or not. 

Why is favorability the best way to look at the data?

Favorability refers to the number of Agree and Strongly Agree responses on your survey out of the total number of responses. 

We focus on favorability for several reasons:

  • Favorability is inherently goal-oriented. Ideally all employees would respond to each question favorably, so an important goal for engagement survey programs is to focus on the metric that is tied to an end-goal of enhancing favorable perceptions. 
  • Differences between favorability ratings are easy to act on and interpret because we know the number the employees associated with that rating. As an example- if a 100-person team achieved 75% favorability on a question, then we know that 75 people responded with a 5 or 6. Therefore, we also know that we need to improve perception for the remaining 25 team members. 
  • Favorability ratings are easy to discuss. Whether it's communicating survey results to an organization or managers discussing results within their teams, favorability is an easily-understood metric regardless of an employee's statistical or analytical background. Results that are easy to interpret and discuss promote more meaningful discussions around team, departmental and organizational improvement. 
  • Focusing on one number reduces confusion. Comparing, analyzing, sharing and discussing one metric is easier and more consistent than focusing on multiple metrics. 

What does "Insufficient Response" mean?

"Insufficient Response" is labeled on reports where the minimum response threshold was not met-this is to protect employee anonymity. For data to be revealed for any group, five or more people within that group must respond for the data to be visible. 

What is the minimum response required?

To see results within our reporting site- we require at least five unique employee responses. This helps protect employee confidentiality and prevent users from filtering or slicing results down to only one employee.

What are Engagement Outcomes and Drivers?

Our engagement survey is comprised of two types of items: Outcomes and Drivers. 

  • Outcomes are at the core of how we measure employee engagement. They help to reveal the current state of employee engagement within organizations. Outcomes are not actionable because they are targets that organizations should strive to maintain or improve. There are nine Engagement Outcome questions, although you are not required to include all nine questions in your survey.
  • Drivers are the actionable questions that help determine what can be done to maintain, improve or otherwise drive outcomes towards higher favorability. 

Why do we not focus on uncertainty and unfavourability? 

Consider a survey question with 50% favorability, 40% uncertainty and 10% unfavourability. Another question has 50% favorability and 50% unfavourability. Despite favorability being the exact same in both questions, the first has more uncertainty. The second question is polarized between favorable and unfavorable perceptions. These differences suggest that focusing on the entire spectrum of ratings is more important than focusing on just one metric. 

One metric can never show the full complexity of results. However, it also overlooks three important aspects of focusing just on favorability:

  • Simplicity and consistency are crucial for effective discussions around survey results. Focusing on one metric not only reduces complexity, but also makes it easier to interpret and increase the chance that more employees will contribute to discussion around survey results. Some teams may prefer more complexity, but these are decisions at the discretion of each leader. 
  • Responses that are less than favorable are less than ideal. Regardless of uncertainty and unfavourability, these become irrelevant as the goal is to maximize favorability.
  • It's easy to over-analyze and get stuck in "analysis paralysis." A lot of time and effort can be spent trying to figure out which item takes priority, why a question had high uncertainty, etc., all before talking with employees. This strategy is well-intentioned but is often a distraction from the most important activity- creating open and consistent dialogue about questions/topics with lower favorability. 

Is there ever a situation where it's useful to focus on the six-point average?

Yes- focusing on the six-point average can be helpful if multiple groups have identical favorability ratings. For example, if two teams each have an 80% favorability, looking at the six-point average could show if there is a difference in that rating. These cases are more likely when analyzing results for small organizations (e.g. 50 or fewer employees) or small teams/departments (e.g. 10 or fewer employees). 

What do I do if I forget my password?

Forgot your password? Go to https://login.quantumworkplace.com and click Forgot Password?, enter your username (often your email address) and click Submit. You will receive an email to reset your password shortly afterward. 

2-1


If you're still having trouble logging in, contact us at support@quantumworkplace.com. 

What is the difference between item-level favorability and group-level favorability? 

  • Item-level favorability: if a survey item (question) has a favorability rating of 80% across an entire organization, then 80% of all respondents rated that item with a 5 or 6. 
  • Group-level favorability: if a team has overall favorability of 75%, then we can assume that each employee, on average, responded with a 5 or 6 to 75% of survey questions. 

The underlying calculations in both of those examples are technically the same- they include a combination of responses that are either "Agree" or "Strongly Agree" on our six-point agreement rating scale.

Item-level favorability focuses on a specific item across employees. It's representative of the proportion of employees who responded favorably to that specific item.

Group-level favorability focuses on a specific group of employees across items. It's representative of the proportion of items that were responded to favorably by a specific group of employees.

What do the engagement profiles mean? 

  • Engagement Profile Calculations: every survey taker is categorized into one of four profiles based on their average response to the six-point agreement questions on a survey:
    • Highly Engaged: average response of 5 or greater
    • Moderately Engaged: average response between 4- 4.99
    • Barely Engaged: average response between 3- 3.99
    • Disengaged: average response is less than 3
  • Engagement Profile Definitions
    • Highly Engaged: Employees are strongly connected to their places of work. They are often advocates of the organization who stick with the company through its ups and downs, and make an extra effort to achieve great results.
    • Moderately Engaged: Employees are favorable towards the overall workplace experience, but something is holding them back from being highly engaged. 
    • Barely Engaged: Employees have an indifferent attitude toward their workplace. These employees generally have decreased motivation to perform optimally and are at risk for retention. 
    • Disengaged: Employees are mentally and emotionally disconnected from their places of work. These employees may be negative and even disruptive with their concerns. Their critiques typically go beyond healthy feedback and don't necessarily share their concerns with the intention of improving the workplace. While disengaged employees are relatively rare, they can hinder the productivity of others. 

Why doesn't Quantum Workplace calculate statistical significance?

Statistical significance, numerically represented as the p-value (e.g. p=0.03), is not reported for engagement surveys or pulse surveys for the sake of simplicity and statistical soundness. Here are a few of our top reasons for not including this element in our survey reporting:

  • The average employee does not have a statistical background. As such, a large portion of users would not know how to accurately interpret the meaning of a p-value, making it a number that causes confusion and misinterpretation rather than clarity and guidance.
  • Statistical significance is not the same as practical importance. Reporting p-values could give users a false sense of direction or security, with the misunderstanding that p-values under or above a certain threshold are less or more valuable to pursue for positive organizational change.
  • Organizations are populations. Statistical significance is part of the branch of statistics known as inferential statistics. These statistics focus on being able to infer results about a population (e.g., a country, an industry) from a smaller group of that population because it’s rarely possible to get data from an entire population. However, in the case of census engagement surveys, the organization is the population. Response rates for our engagement surveys tend to be quite high ( > 80%), so there is little room, and therefore little need, for statistical inference. 
  • Statistical significance can be strongly impacted by group size. In particular, results are less likely to be “statistically significant” with smaller group sizes, and more likely to be “statistically significant” with larger group sizes. This relates directly to the second reason listed above; users could be misguided by an artifact of statistical significance (e.g., group size) rather than being guided by practically important differences.
  • The p-value as a measurement has come under severe scrutiny within the broader scientific community, and its use is becoming increasingly controversial. 

Instead of relying on statistical significance to determine whether a difference or change is important, we recommend focusing on relative differences or changes within your data. 

For example, if most survey questions increased in favorability yet a few decreased since the previous survey, then relatively speaking, those questions that decreased in favorability are more practically important to focus on. And more specifically, those items that decreased the most should receive highest priority. Likewise, if one department has especially low overall favorability, then that department is most important to focus on. And if all departments have similar yet low favorability, then that suggests a strong organization-wide effort is required.

What is a Commitment Plan Owner vs a Plan Observer?

Plan Owners are individuals directly responsible for creating, assigning and completing commitments. They have access to reports associated with the group of employees they're creating plans for. 

If you are a plan owner, you can add and remove plan owners and observers, as well as assign commitments to any employee.

Tables can't be imported directly. Insert an image of your table which can be found here.

Plan Observers get visibility to commitment plans through regular email notifications on plan progress. They do not have access to the reporting site. 

Commitment Holders are assigned a specific commitment within a plan.

What is the difference between Engagement and Best Places to Work benchmarks?

Within your Engagement Survey reporting site- you will see two categories of benchmarks: Engagement and Best Places to Work (BPTW).

Engagement benchmarks represent data from organizations that partner with Quantum Workplace to conduct an annual Engagement Survey. 

We recommend focusing on these benchmarks rather than BPTW benchmarks. 

BPTW benchmarks represent data from companies that participate in Best Places to Work contests in cities across the United States. This data tends to skew higher than Engagement benchmarks, partly because companies participating in BPTW are self-selected to be higher scorers on Engagement surveys. These benchmarks are often aspirational.