It's the cool new thing
Experimentation at Shopify is rapidly growing. In 2021, the number of experiments across the company saw a 311% jump from the year prior. At the time, Shopify was expanding into international markets, introducing different pricing plans, and exploring new business verticals. Unfortunately, our platform has been largely untouched since 2014 so the product wasn't scaling with the growing demand and user requirements.
I joined the Experiments team to lead design for this mission. Since then, our team has grown 6x and I was responsible for:
1.
Developing the North Star product vision and insights strategy for experiments at Shopify
2.
Collaborating with internal platform teams to integrate experiments into their products
3.
Contributing to a new Internal Platform Design System based on Material UI
4.
Improving the outdated user experience through redesigns and new features (ie. sample size integration)
2022 — Results table redesign
Team <3 content
The Experimentation Platform (EXP) faces a unique content problem: how might we present complex statistical concepts to non-technical users so that we can lower the barrier the entry, make education more accessible, and grow the experimentation culture?
While this is one of the few times I thank my younger self for graduating with a Stats Minor, it ultimately comes down to close collaboration with Data Scientists and users — acting as the translator in-between.
When I joined the team, we prioritized fixing the language and tone on EXP. By replacing technical terms with familiar words, we reduced our support queue and even helped our Engineers decide how to name our API parameters.
Colors mean business
Accessibility in EXP was a concern we advocated for. We'd fail as a data product if our colors and visualizations couldn't convey important information.
Recently, we adopted Material UI for our Internal Platform Design System. Since it was still new, we used our accessibility concerns in EXP as an opportunity to layer on a color system so that our solution can be shared across all internal platforms.
While user tests are still ongoing, we strived to meet a minimum of 4.5:1 contrast ratio to meet AA conformance levels (WCAG 2.0) and a Delta E value greater than 2 between any two shades for color perception.
2022 — Testing color-blind palettes
2022 — Shopify Internal Platform Color System
Beyond the platform
While there's a ton of work to be done on the platform, the experiment lifecycle extends beyond that. It's a story about discovery and determination. This is why our roadmap has shifted to build EXP into a learning platform.
We're exploring the experiment loop and how insights from concluded experiments can feed into the creation of future experiments. As a part of this, I'm currently championing our communication strategy for notifications and insights sharing.
If you'd like to learn more about the North Star project, please get in touch↗.
