October 9, 2019
Artificial intelligence (AI) and machine learning are discussed in the tech media daily. These technologies have the potential to change or even revolutionise existing businesses, systems, and processes. AI can broadly be thought of as a system, program, or machine which carries out tasks often associated with human intelligence.
As a narrower application of AI, machine learning “is where a computer system is fed large amounts of data, which it then uses to learn how to carry out a specific task, such as understanding speech or captioning a photograph” (ZDNet). Think of voice-search technology or product recommendations you see based on past purchases. As the understanding of AI and machine learning continually evolves, so do their applications in the business world.
How does this relate to the LEWIS research team?
It is important for us to stay aware of these new technologies and learn how they can help us derive better insights from the research we carry out for clients. If you have ever taken a survey, you may remember answering multiple-choice, ranking, or even open-end questions. We compile, examine, and analyse this information, and then use it to formulate insights that can help companies recognise potential weaknesses in brand image, drive coverage around topics of interest, and identify areas of growth. AI and machine learning create new ways to help us analyse this information.
So where is the big opportunity? It lies in unstructured data.
Unstructured data can be thought of as information floating around in mediums that can’t be quantified such as long-form text, images, audio, and video. On the other hand, structured data can be thought of as information organised in a table or relational database. Most of market research comes in the form of structured data: tables full of numbers, ratings, and rankings, all of which we can easily analyse. So if the majority of the data we collect is quantitative and structured, why is unstructured data important?
Some of the most powerful and nuanced information we can collect comes in the form of unstructured text. For example, if we ask a survey participant an open-ended question like “Please explain your previous answer” and allow the participant to type anything they want to into a blank box, the answer won’t be a number. It could be anything from a few words that define their specific perception, or they may provide a lengthy description of a past interaction or experience with that brand. But sentences or even paragraphs of text are difficult to organise and evaluate, even if a researcher personally reads through thousands of individual responses. AI and machine learning can help.
By implementing AI and machine learning models, these systems can analyse and recognise key words, thoughts, and emotions participants may have, and condense and organise these pieces into digestible insights and trends. This unique information allows us to get a deeper understanding of what respondents feel and helps us provide a more informed perspective.
As beneficial as AI is expected to be, there is one potential negative. Bots. Survey bots are programs or algorithms which automatically complete online surveys in order to access rewards and incentives. Bots lead to bad data, as they don’t reflect the opinions or responses of legitimate participants. This bad data negatively impacts findings and can lead to confusing or contradictory insights.
As AI continues to develop and mature, it will most likely lead to an increase in bots. While the responses may initially seem qualified, a deeper look often reveals questionable responses if you know what to look for. But there’s hope! Putting in reCAPTCHA widgets, utilising quality control questions, and partnering with high-quality sample providers are all best practices to implement when conducting research to help minimise or reduce the chance of poor or fake data and the LEWIS research team uses these measures to defend against bots and provide higher-quality, useful research.
So, while AI in research may create a couple more variables to account for in the research process, the benefits it can create far outweigh potential issues. Using AI and machine learning in processing large amounts of unstructured data has the potential to drive a greater understanding of the questions we are asking and the “why?” or the “how?” that make up the big picture. While AI and machine learning will not take away the necessary eyes and skills of a research team, they hold great promise for the future of research.
Want to learn more about the LEWIS research team? Reach out to us today.