Human vs Machine: How to work in harmony
On this blog we have often made the case for human analysis as machines cannot generate the quality insights needed to inform strategic decisions.
Yet automation can still unearth interesting data and serve as a starting point for a human-led research process. Referred to as the “coding and thinking” method, keyword analysis is one of its applications. This usually involves setting up automated searches within bodies of text to generate stats on the prevalence of specific words or thematic language groupings.
In a recent blog post, we showcased how findings around frizzy-haired consumers derived via human research were enriched by insights from natural language processing. This enabled us to recommend the use of specific language and messaging to target this consumer group.
We have recently worked on two further projects and tested the value of keyword analysis. The first we applied to a social media dataset, the second to news content.
Uncovering the what: prevalence of key themes in US vs. UK discussions around haircare
Could social media shed light on whether consumers in these markets use and/or perceive shampoo product types in the same way?
We knew that arriving at comprehensive answers would require reading through large content samples and devising a framework for categorising user motivations and experiences. However, keyword analysis offered a shortcut for arriving at some initial hypotheses.
We captured mentions of a specific shampoo type in the US and UK during a one-year period. Through scanning results we excluded brand-generated content and compiled a list of conversational themes that were present, for example around hair type, health and ethos. We then set up sophisticated keyword searches for each theme, extracted stats on their recurrence (i.e. the % share of each thematic language grouping within each content set), and organised the data so as to easily compare the two markets.
There were substantial differences between the US and UK in consumer discussions about shine, scalp issues, frizz and dyed hair, yet some themes were equally present (coconut oil, cleaning/cleansing and moisture). This could be significant in helping inform how a product is marketed, or else guide further research into consumer preferences to drive innovation.
For example, in the UK a higher share of consumers mentioned scalp issues and frizz, indicating a desire to remedy these conditions by using a specific shampoo type. This could be connected to demographics, geography or brand exposure; the raw stats can’t say. Nor do they tell us about the nature of these consumers’ experiences or perceptions. But they provide a good starting point to further explore the content.
Likewise, our automated searches showed that themes such as moisture and cleaning/cleansing were mentioned just as often in both markets. This suggests that certain motivations for using specific shampoos may apply to consumers on both sides of the Atlantic, which could have implications for marketers and R&D teams. Again, finding out why requires a qualitative approach – but keyword analysis has served a purpose in identifying the what.
Tracking perceptions: using keyword analysis to measure shifts in a brand’s reputation
A top city firm embarked on a brand refresh five years ago and asked us whether there was a tangible way of measuring its success. Happily, a significant part of this re-launch revolved around establishing positive associations with specific buzzwords, or “reputational attributes”, via targeted PR campaigns. The company wanted to be seen as innovative, creative and commercial, to give three examples.
Our approach was to collate the company’s news coverage from the past five years and set up basic keyword searches to identify the recurrence of each attribute. In order to cut down on “noise” we restricted the searches to top-tier publications and key business titles such as the Financial Times.
Of course, relying on automated word count alone has its drawbacks. It proves impossible to measure whether the word “commercial” appearing in the same article as that of a company name is a good, bad, or indifferent indicator based on stats alone. And as we quickly found out, “commercial” can apply to many areas of business, which resulted in the word appearing in hundreds of articles.
The lesson here is that automated analysis is highly useful for providing a landscape view of the data but a layer of human vetting and digging will always be required. In this case, we skimmed the articles, totting up the number of positive associations. This allowed us to fine-tune the results considerably. We whittled down recurrences of the word “commercial” to just three that were actually relevant.
The next step was to plot results and look at the overall linear trend of each attribute and see if a shift in perceptions could be established.
The conclusions were surprising. Of the ten buzzwords we tracked, only three showed a distinct upward trend – suggesting increased media associations of the brand with those attributes but decreasing associations with the others.
These findings proved valuable to the client and, as it turns out, the attribute which increased the most (“innovative”, shown in the chart above) was the one they had been pushing the hardest in their messaging