Context
ezCater is a two-sided marketplace for restaurants and those in need of catering. Our large team of customer service “ninjas” were swamped by frequent requests to make small account changes for our catering buyers and restaurants. Ninjas could make many of these setting changes in an outdated internal tool, while some requests required tickets and engineering time to complete.
Goals
What would be possible if we made it easy for our catering buyers and restaurants to control their own settings?
- Increase partner satisfaction, retention, and adoption.
- Help partners set their settings to get the best search results and business performance
- Reduce ninja hours spent on low-value tasks
- Reduce or remove engineering time spent on settings changes
Laying the Foundation
We audited our internal tool and made a spreadsheet of all settings that existed. We worked with support engineers to add any settings that were managed programmatically. We removed a surprising number of settings that no longer had any effect in the product.
Of the remaining settings, we split out settings that were intended to be internal only, usually settings that were used to implement our business model decisions, or occasionally settings that had the potential for catastrophic effects on the partner’s business.
Research
As our team listened to support calls that had been tagged as settings-related, we realized that a third of those calls were partners asking what a setting was set to. To validate our assumption that providing partners with even a read-only version of their settings would help reduce call volume, and armed with a few quick mockups, we set up interviews with 5 partners who had made support calls recently. Our interviews confirmed our assumption.
First Iteration
With our assumption validated, we implemented and rolled out a new read-only settings page over our next 2-week sprint. Our plan was to iterate on the UI to manage each setting one-by-one, using the first “edit setting” page as a template to speed up our development time. We weighed the settings in our spreadsheet by settings most frequently requested by the partner, with the highest impact on their business, and the level of effort required to implement. We would release settings iteratively, slowly transforming the page from read-only to editable.
To test our template, we released the simplest setting, the ability to edit a store’s name, as a fast-follow, and used the next sprint to work through a handful of settings on the list that were easy to implement.
Highest Impact
With the framework in place to roll out new settings quickly, we moved to delivering the individual setting that would have the highest impact on both the catering partner’s businesses and team’s time spent resolving account editing issues.
We used two criteria to make our selection:
- The setting that ninjas received the most calls to change.
- The setting that took the longest for the ninja or an engineer to change.
The setting of “Lead Time” (the amount of time before an event that an order could be placed) was our choice based on those criteria. Not only was it the most frequently requested change that ninjas received, it was one of the settings that had the biggest potential impact on a catering partners business given its effect on how a restaurant ranked in customer searches.
If we exposed this setting to catering partners, would it reduce the number of calls ninjas received?
Could the ninja’s high-touch advice be conveyed to the catering partner on our interface in such a way that they could set their own lead time and improve their search rankings?
A Complex Setting
Catering orders are typically large and take some notice to prepare, and some restaurants had dishes like lasagna that they required 24-hour lead times on. Partners wanted to adjust the lead time based on how busy they were that day or that week, and set it back to their default when business slowed down.
There were two settings under the hood that controlled how this worked in practice, and the logic was somewhat complicated to explain to partners.
If a store had an 8 hour lead time set, and an event needed food delivered at noon, the customer couldn’t place an order after 4am. That falls into the Nighttime Gap, which is based on the hours the store is open to receive and start preparing the order. So for this particular store, the customer would need to place an order before 10pm when the store closed the day before.
An optional setting for partners allowed them to specify the time the day before an event when they’d stop taking catering orders. For this store, they set it to 7pm, which reduced the window a customer could place an order even further.
Simplifying the Decision
I met with my Product Manager and Engineering Manager in a room with a whiteboard, and we pondered how we could simplify the decision-making here. We realized that with a very small set of exceptions, this logic would work.
To keep our team moving on this, we provided the engineering team basic ideas for a placeholder page and added the respective routes in our internal tool to prepare for launch. I iterated on the simplest interface for helping the partner through the decision tree while providing advice on how and why to set their lead times.
We started with a two-question interface to help frame the decision for the user, including a tip message to help reinforce that they would be giving up access to a portion of orders by opting out of same day orders.
We realized we could put the main decision that required the most thought first, and have follow-ups to tune the setting. Each option had conditional warnings or tips to help the user understand the effects of each option.
After sharing the design with an internal partner advocate who we used as a proxy for scrappy testing, we made the main choice even more prominent and tuned the copy for our tips and warnings.
Rollout
We rolled the feature out internally to a specialized support team to pressure test our design before opening it up to 10% of our partners. We watched the changes made by monitoring the data on a dashboard we created in our analytics tool, as well as watching FullStory session recordings of partners using the interface. After 24 hours with no red flags raised, we rolled the feature out to all partners.
Results
Our modular component-based interface and settings framework made it simple for us or any team to expose a setting to a catering partner.
The week of work (from idea to implementation) exposing Lead Time led to 100 unique partners each making on average 4 changes a week. Since a typical setting phone call required ~10 minutes of call time, we estimated we saved 4,000 ninja minutes a week. Additionally, our average lead time dropped from 8 hours to 5.5 hours, resulting in better search results for customers and potentially more business for our partners.
Takeaways
Given how complex the technical constraints of our existing settings systems were, we pulled our entire squad into the process very early, and discussed, whiteboarded, diagrammed, and wrote pseudocode for the problem as a team before we landed on our solution. This project was a reminder that while there’s a balancing act between iterating on the current problem as a team and having engineers working ahead of the designer, product manager, and engineering lead, the former can be incredibly productive.
ezCater has a culture of launching and learning, and that sometimes comes with pressure to bias towards speed. Investing in a framework or modular system early can yield savings down the road, either through making it easy for future teams to revisit and iterate on your work as they learn something new, or making it easier to roll out a feature incrementally and validate each portion’s value.