If you've worked in the market research industry for a while, you must have had a situation where you were defending your sample methodology to a client. How will that change with AI synthetic data?
We have all been there. The results of the research are not what the client was expecting.
Of course, that's a problem within a problem, but the client is expecting your reassurance that the provided data is reliable and can be trusted.
In an ideal situation, you defend the research by:
- providing more information on the profile of the respondent
- total sample and quotas selected
- representation of the sample in the total population or targeted audience
- data quality checks done throughout the research
- statistical significance and margin of error
- panel provider reputation and references
If that all doesn't work, you should probably accept that your client doesn't appreciate research as a process.
But how does this all work with AI synthetic data?
How are you going to defend your results when the results of the research are not what the client was expecting?
I can assure you that will happen sooner or later.
If you plan to say; "Oh, it's AI and it's trained on X data points." - I don't think that will lead you far, at least not in the market research industry.
We need to be more sophisticated than that.
If we want AI synthetic data to become mainstream in the market research industry, we'll need to be much more transparent.
- On which data sets the model is trained, from when and how big
- Is it your own model or a combination with LLMs (which LLMs) or something else
- How many cross-researches have you done with real humans to prove the reliability of the data
Transparency on models and constant cross-research. That's how we plan to present our AI synthetic data appliance inside the Peekator platform later this year.
At the end of the day, if you can't defend your research, you better not even start with it. 🚶🏼