What does the AI vibe shift mean for user research?
First came vibe coding, and right now vibe marketing it’s heralded as the next evolutionary step in marketing. A paradigm shift that “marries the agility of vibe coding with the power of AI - delivering a framework where speed, experimentation, and automation reign supreme”, according to Forbes VCs love it for exactly that reason.
Milly Tamati wrote about “vibe work” in her Generalist newsletter along the same lines - the shift towards setting things up to generate and execute, while you’re having a coffee to come back to it and refocus and adjust.
I think there’s definitely a big paradigm shift happening and has been for a couple of months or years - depending on when you’re joining the parade. But I wonder if cost-savings, speed and relative ease of use are the right ways to measure what matters. Because as so often it really matters what you measure -
For example - while you’re saving the human cost of having large teams (fill in the discipline of your choice), we’re currently not measuring (therefore not paying for) the cost in water and energy use.
Moreover, at the very heart of this current vibe shift is the fact that team sizes are reduced because together with AI individuals can produce way more and often faster. If the prompts are right and connections well drawn the execution becomes the easy part.
But is easier always better?
In the Product world (including Design and UX research) we’re seeing similar discussions around how the craft is evolving, with fears of job losses taking up one side of the conversation (especially after the “we’re all a big family” work culture myth got disbanded) and the positive take on that: we can finally focus on what we love while AI does the tedious bits, taking up the other half.
And don’t get me wrong I’m seeing many fantastic applications and true problem fixes through AI - especially around combining and summarizing vast amounts of data (from Product, to Design or Science in particular).
However - There’s a real conundrum here.
On the one hand VIBES are all about capturing what’s happening and executing fast on it.
On the other hand, you need to understand context well and interpret signals correctly for the outcomes to matter. The more you’re doing that vibe marketing/coding/research/design (you name it) alone - the harder it is to be connected to the wider societal context.
That goes for teams and organizations, where joint sense making through discussion or deep dives are what provides the glue for everyone to move towards the same goals. And it goes for understanding your customers and users well.
As Vitaly Friedman argues in an article about the downsides of AI-first user research - a belief in “any user research is better than nothing” can be detrimental to truly understanding what is going on. Synthetic user testing without any actual customers involved gives the illusion of having done research but without the benefits of:
Seeing real people interact with your products
Inviting stakeholders and other teams to see the testing first hand
Spending the time discussing as a team about what you’re trying to figure out
All of these interactions help us make sense and create the glue that moves teams forward. These are the parts where learning occurs, where insights stick and where you create buy-in from across the organization.
VIBES without people means you’re placing all your bets into very few baskets (people) and hope the execution is right. It can work, but it can just as well lead to a copy-paste activity of what your competition is doing without the benefit of moving forward as a team.
So when it comes to weighing up whether or not to do user research with real customers, instead of synthetic testing I would choose real usability testing (with AI triangulation afterwards, or across your entire existing research library) any time. Because you’re not just doing user research to get straight back into execution and sending mode - a large part of it is to help your organization learn and see what is happening.
It’s about understanding and joint sense making and for that to happen, you need to have the real thing and put in the work of creating these learnings and insights together, otherwise they just won’t stick. And then all you have is yet more data - but without the insights.
#vibeshift #synthetictesting #AIresearch #customercentricity #userresearch #sensemaking
If you want to read more deeply into how the term “vibe shift” (describing the exact timeframe in which the cultural moment changes), originally coined by LA-based trend forecaster Sean Monahan, came about - there’s a great article by Alisson Davis in the Cut. This is not about AI as such, but the post-pandemic change and vibe shifts in general - when suddenly the societal mood changes, beyond the current style of denim.
