Why We Need to Create Experiment Plans

It is far too easy to fall into the trap of believing that customers will love what we are building. That is why we go to the effort of breaking down our solutions, identifying assumptions, and then trying to evaluate them. But there is still one hurdle left to overcome - our bias.

There are two big traps that we need to avoid: confirmation bias and hindsight bias. We fall in love with our ideas so we will be quick to discount any negative feedback that we get and continue to push forward with flawed ideas.

And then there is hindsight bias - of course this is what would happen. Everything is always obvious in hindsight but unless we have a written record of what we expected to happen we will limit our learning and could end up making similar mistakes again in future.

Experiment plans help us avoid these pitfalls by explicitly documenting upfront what we expect to happen and why. We also say who will will test with and how so that we can't argue with the approach afterwards. In my article below I go deeper into the components of an experiment plan and provide a worked example.

This Week’s Updates

Enabling the Team

Leaders Can Move Fast And Fix Things by Frances X. Frei & Anne Morriss
Leaders must embrace agility to tackle problems quickly. By empowering teams and fostering collaboration, they can drive change effectively, while balancing speed with thoughtful decision-making.

Leadership Over Measureship by Paul Worthington
Measureship, focusing on metrics over people, leads to a directionless, anti-human work culture. Leadership, in contrast, prioritizes empathy, qualitative insights, and innovation.

Product Direction

Hype-Driven CEOs: When Innovation Misses The Point by Raphael Dias
There are drawbacks of hype-driven innovation pushed by CEOs who prioritize trends over customer needs. Align innovation with the user value rather than following the latest fads.

Continuous Research

The Real Problem With Research by Maxim Mestovsky
Research is often misapplied, especially when it fails to align with real-world goals, urging a shift towards more actionable and impactful research practices.

Evaluating Your MVP With Mixed Methods Research by Jack Holmes
Get fresh perspectives from defining what early data you’ll collect, to evaluating from multiple angles and incorporating your learnings. [Sponsored Content]

Continuous Design

How To Create An Experiment Plan (And Why) by Rory Madden
The goal isn't just to run experiments - it's to learn from them. A well-crafted experiment plan ensures that learning is based on evidence rather than post-hoc rationalisation, leading to better product decisions and more valuable insights.

Prescriptive And Descriptive Information Architectures by Jorge Arango
Try using user-friendly, descriptive language, but a prescriptive approach when introducing new features or brand-specific terms

Continuous Development

The End Of Programming As We Know It by Tim O’Reilly
As LLMs evolve, traditional programming gives way to prompting and orchestration, redefining developer roles and the future of software creation.

Evolutionary Architecture By Example by Maciej "MJ" Jedrzejewski & Kamil Baczek
A step-by-step guide to navigate complex .NET software architecture. You can implement evolutionary architecture patterns that adapt to change, improve modularity, and support long-term system evolution.

Still Looking for an Opportunity to Join UXDX EMEA?
Volunteer with Us!

If you didn’t get your budget approved for this year’s UXDX EMEA conference, here’s your chance to get involved! We’re still looking for enthusiastic volunteers to help out at the event in exchange for the full conference experience. As a volunteer, you’ll have the opportunity to contribute to the success of the event while enjoying all the amazing talks, workshops, and networking opportunities.

Don’t miss out on the chance to be part of this incredible community and make the most of the UXDX experience! Sign up to volunteer today and help us create an unforgettable event for everyone: https://airtable.com/appZDHX5d901bzueE/pagyPl3NOceuj3z5K/form

FREE COMMUNITY EVENTS 

Check out more upcoming community events 

Video of the Week
AI in Design and Research: Revolution or Regression?

Should UX designers embrace AI to drive efficiency, or focus on maintaining creativity and originality? In this debate, John Cleere from Unmake and Kevin Hawkins from Amenitiz explore the impact of AI on design and research.

Watch now as they tackle key questions: Can AI-driven research build impactful products? How much should designers integrate AI into their process? And, will AI enhance job security or make designers obsolete? Tune in for a thought-provoking discussion.👇

The Results of Last Week’s Poll

The question: When does your team typically conduct usability testing?

This week’s poll results highlight that most teams focus their usability testing primarily at the prototype stage, with 52% of respondents conducting tests exclusively at this phase. This shows that teams are prioritizing testing early, when concepts are still fluid, allowing for valuable feedback before development moves forward.

Interestingly, 37% of teams conduct usability testing at both the prototype and development stages, reflecting a broader approach to refining the user experience. A smaller 11% test solely during the development phase, and none of the respondents rely only on demos or skip testing altogether. Usability testing remains a vital part of the design process, ensuring that products are user-friendly and aligned with user needs.

If you’re looking to enhance your usability testing strategies or better understand when and how to integrate it into your workflow, check out our Continuous Design course for practical insights and techniques!

Seen an interesting article online? Share it with us and we might feature it in our next issue!
Click here to share an article