At InnovFast, we blur the lines between qualitative and quantitative research, offering an exciting new path for data collection and interpretation. Here’s how we approach this transformative method with synthetic users, and what it means for the future of user experience and market research, but especially for the future of businesses.
Effort Equivalence: Qualitative vs. Quantitative
Traditionally, qualitative research, characterized by interviews, focus groups, and case studies, is considered labor-intensive and time-consuming. On the other hand, quantitative research, known for its surveys, questionnaires, and numerical data, was seen as more scalable but sometimes lacking in depth.
The distinctions between qualitative and quantitative research were well-defined by Creswell, in “Research Design: Qualitative, Quantitative, and Mixed Methods Approaches”. He explained how qualitative methods allow for in-depth probing of human behavior, emotions, and motivations, while quantitative methodologies provide generalizable insights through statistical measurements.
With synthetic users, this differentiation begins to blur. Conducting 10 interviews to assess desirability or deploying 5,000 surveys can now require roughly the same effort.
Finding the Sweet Spot: the Intersection of Quality and Quantification
As we transition from strictly qualitative datasets to more quantitative ones, an ideal intersection exists—a sweet spot where the depth of qualitative insights meets the breadth of quantitative data.
This point represents a balance that allows Product Managers to gain rich, in-depth insights while leveraging the robustness of large-scale data. This balance means that clients and stakeholders can then focus on synthetic reports, benefiting from the advantages of both research types.
Synthetic users allow Nielsen’s qualitative insights to align with quantitative scalability. This fusion leads to what Nielsen and Norman called “triangulation”—a comprehensive understanding of user needs and behaviors through the use of multiple methods.
Nassim Taleb’s “black swan” concept truly comes into its own here: “Having more data does not necessarily make you more informed, but it does make you more confident.” The blend of qualitative and quantitative methodologies powered by synthetic users should, ideally, offer both—insight and confidence.
John W. Tukey, a data pioneer, once remarked: “The combination of some data and a strong desire for an answer does not guarantee that a reasonable answer can be extracted from a given set of data.” This underscores the importance of integrating qualitative and quantitative methodologies. By doing so, through synthetic users, researchers achieve a more holistic and reliable understanding, ensuring both insight and confidence.
Understanding the Blend: Getting the Best of both Worlds
Synthetic users are tools that help us gain clarity in a world where we, as humans, overemphasize a single story or experience, even if it’s not common. Daniel Kahneman discussed this in his book “Thinking, Fast and Slow.” Robert Cialdini did the same in “Influence: The Psychology of Persuasion,” when he introduced the concept of scarcity, where people assign more value to opportunities when they are perceived as rare.
By combining detailed stories (qualitative data) with numbers (quantitative data), researchers and anyone within the company can obtain a clearer, less biased picture. This is part of our mission: to clarify in order to foster better decision-making.
When research involves human beings, whether subjects or researchers, we are aware that biases emerge. Our InnovFast platform is able to mitigate the following:
– Interaction biases are eliminated: In qualitative studies, the very interaction between the researcher and the participant can introduce bias. For example, a participant might provide answers they believe the researcher wants to hear (social desirability bias).
– Observer-expectancy effect is neutralized: Sometimes, a researcher’s expectations can subtly influence participant behavior, leading to results that align with the researcher’s beliefs.
– Sampling bias is corrected by synthetic users to offer greater diversity by default: Even the participant selection process can introduce bias. If a sample is not representative of the entire population, conclusions can be skewed.
Synthetic users can sometimes overemphasize predefined user pain points and goals. We strive to counteract this phenomenon through dynamic interviews, where the focus is on study objectives rather than observed user pain points.
Synthetic users are trained on large quantities of data and generate results based on patterns within that data. If the training data exhibits certain common biases or if the questions posed to the model steer it in a particular direction, the model may produce results that appear biased or emphasize certain patterns over others. We consider this a feature, not a bug, when ensuring organic/synthetic parity.
Nevertheless, much is being done to mitigate this latter bias.
– Data Diversity: We ensure that the training data for the models we use is diverse and well-represented to produce more balanced results.
– Objective Questions: We design questions that offer a degree of neutrality and do not necessarily lean towards a particular outcome.
– Regular Evaluation: We periodically evaluate the model’s performance and results to check for any unintended trends or biases.
Thus, no, synthetic users do not have human-like biases, but their results can still reflect biases present in their training data or in how they are used. Any tool, regardless of its advancement, must be regularly evaluated and refined to ensure it meets its intended purpose without introducing unintended biases.
a New, more Efficient World with Better Products and Businesses
By deploying synthetic users in user experience and market research, combining the depth of qualitative insights with the breadth of quantitative data, we can anticipate a myriad of potential discoveries and advancements:
– Richer User Profiles: We are already noticing them. By merging vast datasets with detailed user experiences, synthetic users generate comprehensive user profiles that capture both broad behavioral trends and individual nuances. You will notice that they are much richer than most organic research results.
– Predictive Insights: The union of qualitative and quantitative methods allows for more accurate prediction of future user behaviors, preferences, and needs, facilitating proactive adjustments in product design or market strategy. This is what excites us the most.
– Refined User Journeys: Thanks to synthetic users who mimic potential real-world users, companies can map complex user journeys, identifying potential touchpoints, challenges, and opportunities for engagement improvement. This is an area where we need to work more.
– Hybrid Research Methods: We believe this blend could lead to the evolution of new research methodologies that leverage the strengths of both qualitative and quantitative approaches, thereby optimizing the research process.
– Diverse Perspectives: Synthetic users, trained on a vast range of data, can simulate a broader spectrum of user perspectives than traditional research allows. This will inevitably lead to more inclusive and accessible products and services.
– Bridging Cultural Gaps: With access to global data, synthetic users can provide insights into cultural nuances, preferences, and behaviors, helping businesses adapt their offerings to different geographical or demographic areas.
– Faster Iteration: The ability of synthetic users to rapidly process large amounts of data and provide feedback means businesses can iterate designs, campaigns, or strategies more quickly, responding more agilely to market demands.
– Deeper Emotional Understanding: While synthetic users do not possess emotions, their ability to analyze vast quantities of qualitative data can help businesses better understand the emotional drivers underlying user behaviors.
Examples of Synthetic User Use Cases
More Than Just Skincare:
Scenario: A company selling skincare products receives thousands of online reviews each month.
Action:Synthetic users are trained using thousands of organic online reviews.
Result:The company observes that words like “trust,” “disappointment,” or “relief” are frequently used. Synthetic users indicate that they are not just seeking skincare, but also a boost in self-esteem. Products can therefore be improved by considering this emotional need.
Airlines Facing Anxiety:
Scenario:An airline wants to improve its in-flight experience.
Action:Synthetic users are created from real passenger survey data.
Result: Synthetic users quickly reveal that they often feel “anxious” about having overly short layovers. In response, they suggest that the airline introduce more reassuring notifications about connecting flights.
Evolving Ethical Considerations: As we delve deeper into our use of synthetic users, we will make new discoveries in the field of ethics, data privacy, and data usage. The way synthetic users are deployed, the data they access, and the knowledge they generate will require continuous discussions and ethical guidelines.
The use of synthetic users in user experience and market research means we can combine in-depth personal insights with large-scale data.
Join us to explore this new approach. The balance point between qualitative and quantitative can give rise to new concepts yet to be discovered.
We believe it is essential to balance both aspects and monitor for potential issues. It is by doing things well that real benefits can be derived.
