Qubit Lens
Qubit
2020
Product Design
SUMMARY
Scope
Working on a differentiator capability to increase our customer retention
Process
A user-research focused process that over multiple research rounds and iterations helped us build our weekly insights product - first in Google Sheets, and then migrate it to our existing product - Qubit for Merchandising
Impact
Reached over 40+ clients and 80+ merchandisers
40% average CTR in weekly reports
11 actions taken through weekly insights
3 clients adopting early access program - and repeatedly taking weekly actions
THE CONTEXT
Background
Qubit pivoted a few times in the past years. In the recent years, we focused on personalisation and digital merchandising, and built an easy to use tool for those purposes - Q4M.
Our ideal key customer would have different titles in different business but what they’d be doing is digital merchandising. They’d always be on their desks, have Monday meetings with other teams (optimisation, email, trade) constantly keep eye on things, work in sheets, Google Analytics.
Why?
We were refreshing our GTM strategy for our new product, Q4M, and more specifically, looking into how we can differentiate between Qubit Start and Qubit Grow packages.
Qubit Start had all the personalization tools to create campaigns: Personalised content, product recommendations, and product badging.
For Qubit Grow - we were looking into adding something to our product that would leverage all the product and the (end) customer data we had access to. This would also create stickiness indirectly as it’d give the merchandisers something to start from.
USER RESEARCH:
INITIAL CONCEPTS
Early user research
This early concept creation and research were done in collaboration with Idean - a product design agency.
I joined this project around this time - and took ownership of the concept creation and research as this step was the agency’s last piece of work on the problem.
FIRST PROTOTYPE
Knowing “the right products”
Once we knew enough about the missing opportunity - we built a prototype to test it.
REVIEW PROCESS
Usability risks
After an initial research round with 10 customers, we noticed that customers found this feature powerful but at the same time, really overwhelming. In other words, there were either too many actions or many products and categories to pick from.
More bad news: Feasibility risks
As part of the product design process, I also started having conversations with engineers and data scientists on the team to understand the feasibility of that prototype. It became clear that building a fully-fledged product like this that relied heavily on the customer’s data structure would require a lot of time and resources that we weren’t able to afford.
NEXT ITERATION:
GOOGLE SHEETS
Based on that evidence (and a lot more conversations), we decided on generating a ML generated weekly report of 10 products with highest revenue opportunities.
Since there were many ways of getting those opportunity-action pairs, we picked the models that clients were interested in most:
Troubleshoot - Products that need review as they’ve started to underperform
Trending - Products that are performing better than they used to in last 5 weeks.
For us, quickest way getting this in the hands of the users were building something using Google Sheets and see how valuable it is before committing to building it into our product.
V1 TO… V6
ITERATIONS
Over the next 3 months, we kept sending and iterating on these reports - almost at a prototype speed, from v1 to v6.
A few highlights:
Let us access to Merchandisers, our target audience, that we didn’t have access to before
11 actions were taken based on these reports using other tools
Another model - Overperforming
But we always knew that we wanted to integrate this with our product - after all, some of the suggested actions were creating content, promoting products and badging them.
SECOND PROTOTYPE:
INTEGRATING WITH OUR PRODUCT
We had to pick a model for the next step and it was an iteration of Overperforming model - or in simpler terms: “Increase exposure”
As we were gaining more confidence, I started designing the next prototype - this time, it was about imagining what it would like if we added this feature to Q4M.
In addition to that it also had a story view concept - a flow that had a summarised context of - what, why, and how.
💡 Insight = Product + Action + Revenue Opportunity
INTERPRETING FEEDBACK & DESIGNING FOR PRODUCTION
Testing that prototype (rigorously!) helped us hone in on the details of what the actual feature could look like.
Some of the key feedback that steered us were:
Story view: Simply put, the clients would need further evidence before “trusting the AI”
“Where did you get this?” We also got some raised eyebrows with the detailed context - since some of the stock/margin data, on a SKU level wasn’t something most of our users had access to.
the DECISION
To help the company’s business goals for 2021, we had to make a decision on what’s going to be in the MVP version of this feature - that we are adding to the Qubit app.
After a few rounds of discussions both with team members (PM, engineers, data scientists) and the rest of business, we knew there would be a trade-off.
The two options were:
1. We could either focus on the insight context and expect users to take action using their merchandising/CMS tools
2. We could let them take action easily with Qubit but with limited context.
We picked the latter.
REASONING
The first reason was that letting users take action using Qubit app would remove the dependency risk (therefore business risk) and increase stickiness for the product.
The second reason was that we knew enough about building this. The context varied a lot - and there were feasibility risks that we may not be able to mitigate.
And last but not least - we knew this wouldn’t have too much usability risk as we “re-used” the create and review journeys we already had.
DESIGN ITERATIONS
While we were able to stick to the initial production designs generally, we had to make a few changes along the way.
But more importantly, we wanted to deliver value incrementally, therefore throughout the sprints, I sat down with the PM and product engineers to sequence the designs.
Our rough plan was to build insights on the homepage first, then the insight context and finally finishing up with the create flow.
The key challenge was here to strike that balance of re-using or inventing new systems.
PROJECT RECAP
MEASURING SUCCESS
out of two objectives we set:
✅ We had 7 customers who had taken an action when we closed off the early access program. (Requirement was 6)
❌ We had 2 customers who had taken repeated actions (multiple insights) when we’ve finished the early access program. (Requirement was 3)
However, this was enough to have the green light from the business, and we went into Closed beta and more recently we have started working on releasing it to general availability (GA)! 🎉🎉 ..
and shortly after, a 3rd client also took a repeated action and brought that second objective number to 3. Classic.
REFLECT & REWIND
A few of the learnings for me in this project were:
Difficulties of designing for Google Sheets
A new product requires trust - even with proof
Working in one week sprints
…and how much we relied on customers’ data layers
Although the feature was shaping up to be Qubit’s one of most successful initiatives, we had a few bumps in the road.
So what would I have done differently?
We invested in the weekly reports too much and that made it harder to build them in the app.
Although iterating on them was pretty fast, I wish I had started on app-integrated prototypes earlier.
Since we took the “actioning insight” path, the initial scope of this restricted users take 1 action at a time.
This stopped a few customers from taking repeated action for different reasons - live insight performing well, not having data confidence in time. I did not put too much thought on this experience. I wish we made it clearer to the business how important that is.
The type of insight we chose still required the customer’s data layer to be “good enough” - while it did great when it worked, it prevented us from enabling to a wider audience.
We could have done either a better scoping on this by shortlisting potential customers or have a plan-B design.