Comments: Theme and Sentiment Text Analytics

Note: this help doc applies to Pulse Survey analytics. Learn more about sentiment analysis in our Engagement analytics here. 

Overview 

Survey comments offer color and narrative to your survey analytics. But unstructured data is notoriously difficult to turn into insight. And as your comment count increases, this problem just gets worse.

Theme analysis and sentiment analysis offer powerful tools to help users find the story in survey comments. Quantum Workplace’s proprietary algorithms use natural language processing (NLP) to classify comment themes and sentiment. With the help of machine learning, these algorithms get smarter as administrators modify the automated theme and sentiment classifications.

Text Analytics brings together sentiment and themes to help users detect patterns and insights in their unstructured data.

Theme analysis 

As comments are submitted, our NLP engine classifies comments into one of X universal workplace themes. 

These labels are clearly displayed for each comment in the comments report.

Additionally, the “Themes” list in the lower right of the comments report displays all themes within your current comments, sorted by count. (You can also look at the sentiment distribution for any theme by clicking the “Sentiment” button.)

Modifying themes

Administrators can override any theme by clicking the “Edit Themes” link and typing a theme (an auto-complete feature will offer to help you select from existing themes). After editing a comment’s theme(s), you’ll need to refresh your comments report for the updated theme count to be reflected.

Note that manually edited themes will appear in a gray tag (whereas automated themes appear inside a blue tag) and on hover, attribution identifies which administrator applied that theme. 

Sentiment analysis

Currently, sentiment analysis is an optional feature and must be enabled for individual surveys. Contact your Customer Success Manager to enable sentiment analysis for any Pulse or Lifecycle survey. 

At survey close, our NLP engine classifies comments with one of four sentiment classes: positive, mixed, negative, or unclear. These labels are clearly displayed for each comment in the comments report.

Additionally, the sentiment distribution displays the breakdown of sentiment within your current comments (both count and percent).

Modifying sentiment

Administrators can override this sentiment by clicking the sentiment button and selecting a sentiment from the list. After modifying comment sentiment, you’ll need to refresh your comments report for the updated sentiment distribution reflected.

Text Analytics 

Text Analytics brings together comment themes and sentiment to help users find the story in unstructured data.

In the right pane, users can get a quick look at the “Most Positive” and “Most Negative” themes for any set of comments by looking at Theme Analytics. Users that want to dive deeper can click the “Full Text Analytics” button.

Theme Sentiment Bubble Chart 

When a user clicks “Full Text Analytics”, they’ll be presented with a Theme Sentiment bubble chart. This chart plots the survey comment themes based on their frequency and net sentiment.

Net Sentiment (X-axis)

The X-axis represents the “net sentiment” of each theme. Net Sentiment is the result of taking % positive - % negative. The more negative a theme’s net sentiment, the further left it will appear on the X axis. The more positive a theme’s net sentiment, the further right it will appear. 

Theme Frequency (Bubble Size)

The size of each bubble represents the frequency this theme appears in the current comments. Bigger bubbles represent themes that appear more often.

(Y-axis)

Note: for now, the height of the bubbles (the Y-axis) is randomized. If you refresh your page (or refresh the chart by closing and re-opening “Full Text Analytics” the Y-values will update). Soon, we hope to modify the Y-values to better use the space (by limiting the amount of overlapping bubbles.

Frequent questions

How accurate is your automated theme and sentiment analysis?

We are currently seeing 80% or higher accuracy. Over time, as the algorithm gets feedback from users overriding its classification, it will learn from these signals and accuracy should only get better.

 

Does theme and sentiment analysis honor filters?

Yes, the sentiment distribution and theme lists respond to filtering by demographic, by question, by theme, or by keyword will adjust your comment count and sentiment distribution. (Please note that the Theme Sentiment bubble chart currently only responds to demographic filters.)

 

Can sentiment analysis be turned off?

Yes, contact your Customer Success Manager to turn theme and/or sentiment analysis off for any survey.