Usability Testing of Vibe-Coded Marketing Page
Example of actionable recommendations and implementation
The Challenge
Revenue was declining for a locally dominant brand selling licensing education in real estate and construction professions. Feedback from customers had indicated that confusion and uncertainty arising during the purchase experience were the biggest blockers. The conversion rate optimization team had an idea for a significantly redesigned marketing page featuring a guided quiz to help users select the right package for their needs, a simplified product comparison experience, and content reorganization. Before moving the vibe-coded design into production for A/B testing, they asked me to conduct an unmoderated usability test.
Approach
An unmoderated usability test conducted via UserTesting.com. Users were presented with a series of talk-aloud tasks and verbal response questions designed to assess first impressions, product comparison and purchase decisions, and functionality and usefulness of key page features. Participants consisted of 6 prospects (individuals looking to become licensed contractors) and 4 licensed contractors recruited through UserTesting.com. Videos and transcripts were analyzed for common themes.
Research Questions:
What are initial impressions of the site?
Do the guidance questions help users make a confident purchasing decision?
How do users make sense of the available offerings? How difficult or easy is it to compare offerings? What would they purchase and why?
What are impressions of the brand?
What do users expect to happen when clicking on CTAs?
What are user impressions of the "stories"? How do they compare to other forms of social proof (e.g., star ratings, product reviews, etc.)?
Impact
The marketing team adopted 90% of recommendations resulting from the usability test analysis, including optimization to the hero and product comparison experience, investment in 3rd party reviews, and overall content reorganization.
After optimizing the page, the team moved it into A/B testing. The new variant outperformed the control in AOV (+251%), CVR (191%), and revenue (+672%) during 3 weeks of testing. The variant is now live on the site for 100% of page traffic and teams are continuing to monitor results. (Check back for updates!)