3-Bullet Thursday – 15/9/2022
Lessons from burning out, how pre-mortems can save your project before it's started, and 2 tips from Booking.com about running better A/B tests "below the fold"
Hi friends 👋
I hope you find these as interesting as I did!
Lessons from burning out
Molly G burned out at Facebook, but her lessons might be useful to any one of you.
This quote from Dan Rose is worth framing:
You are working as if the product launch is the end of this journey when actually it’s the beginning.
Another highlight of mine, and something I’ve talked about a lot with my manager:
What you do in 24 hours matters way less than what you do over the course of a week or a month or a quarter.
Pre-mortems: How a Stripe Product Manager predicts & prevents problems before launch
You’ve probably all heard of post-mortems: “What went wrong?”.
Pre-mortems take the alternative angle of asking, “What could go wrong? How could this fail?”.
Shreyas lays out how he ran successful pre-mortems at Stripe, along with the memorable analogies of tigers, paper tigers, and elephants.
It’s a short but very worthwhile watch if you’re ever involved in running larger projects.
2 better ways to run tests “below the fold”
“If a visitor doesn't see the change you have made to a page, you shouldn't count them in your experiments.”
Booking.com explored two options for tackling this, both with pros and cons.
Use scroll depth % for triggering.
Use scroll depth % for segmentation.
The first option means that only customers who scroll a certain depth are even shown the experiment. The second option is more useful for post-analysis, whereby only those who actually saw the experiment are counted.
You can and should consider using one of these methods, both for experiments below the fold and also those hidden in sub-menus.