Get in touch 👉🏽
hello@aperture.london

Why you should experiment: ​​Insights from product testing and user experience

Written by
Lea Samrani,
Product Consultant at Aperture

🗞️ Product experiments proved me wrong more times than I can count.

I am a big partisan of lean UI. I have a strong personal dislike for pop-ups, overlays, invasive notifications, and anything else that would interrupt the core user experience. I am not talking about dark UX patterns, just your regular annoying ones.

It feels salesy, pushy, and even sometimes spammy. It reduces my trust in the product and my likeliness to engage. So, for the longest time, I believed that a cleaner, nicer-looking UI would naturally result in better product engagement. Think Booking vs Airbnb.

‍

The experiment

A few years back, after running some user testing, my activation team concluded that the first-time experience on our app felt spammy. They watched users closing overlays without reading them and decided to prioritize a leaner, cleaner onboarding flow. I naturally jumped on board, led by my bias, and embarked on a mission to “simplify” onboarding and “make it look & feel better" to increase user engagement.

Our hypothesis was: “If we stop interrupting the user experience with our annoying and intrusive overlays, then they will be more likely to be engaged.”

We ran a series of tests:

🧞‍♂️ Seamless onboarding
🧞‍♀️ Smart onboarding (whatever that meant)
🧞 Inline onboarding
🧞‍♂️ Replacing all overlays with levers or full screens
🧞 Reducing the amount of push notifications sent
🧞‍♀️ Simplifying the UI, removing unnecessary elements
🧞‍♂️ Showing bigger images, less text

‍

All failed to increase:

❌ User engagement
❌ User retention
❌ Subscriptions

In one specific test, we edited an onboarding overlay, made it full screen, simplified the copy, and added a beautiful animation on it, which looked super smooth. The kind of design you put on your pitch deck because it’s pretty to look at.

‍

What we failed to realise

Users were paid to take part in our user testing and were given a specific action to perform down the funnel. Dismissing anything coming up in their way wasn’t a natural behavior but a manufactured one.

The lean experience might have looked better, made people like our brand better, or generated a better perception of the product, but it affected engagement negatively.

‍

đź’ˇ Turns out:

❌ Sending fewer notifications will not make the ones received more likely to be opened.
❌ Removing the super annoying overlay interrupting the experience will not make users more likely to engage.
❌ Nicer-looking UI will not increase retention and so on….

‍

‍

My conclusion

Today, my personal preference is unchanged. I still think leaner, cleaner designs create more trust. And I do not accept that the worst user experience will have better results in the long run.

However, I’ve been proven wrong so many times that I can now differentiate between “what looks nice” and “what works”. And I have generally accepted that users never behave as we expect them to, so testing is essential.
‍
‍

What is Aperture?

Hello, you, reading this. Aperture is a growth agency that specialises in creative advertising and product optimisation. That means we can develop your creative strategy, run your ads, and optimise your product.

Some of the companies we've worked with are Memrise, TomTom, The Smurfs, Juno, and Mindful Mamas.

Led by Hannah Parvaz (App Marketer of the Year, 4x Head of Growth), Aperture is a team of experts assembled from the best companies such as Bumble, DICE, Busuu, Luno, Bauer Media to bring in-house knowledge to your team and help you find your focus.

Snapshot by Aperture

Snapshot is our newsletter. Once a fortnight, we'll tell you stories like how we reduced a Meta CPA by 94%, produced UGC with 92% thumbstop rates, and experimented our way to a 59% increase in MRR.

Come on then — take a look behind the scenes. You're invited.

✍️ By Léa Samrani. Léa is a Product Growth Consultant at Aperture that specialises in subscription apps and product led growth. She has over a decade of experience growing top-grossing apps, such as Bumble, Badoo, Busuu and Uptime. Find her on LinkedIn here.

‍

‍

Previous Post

Next Post

There is no previous post.
Back to all posts.
There is no next post.
Back to all posts.