Customer Service by Amazon (CSBA)
Insights Dashboard
ux design | user research | prototyping | usability testing
Role: UX Designer and Usability Tester
Duration: 12 months and on-going
Tools: Figma design and prototyping, user research, usability interviews
Customer Service by Amazon (CSBA) background
Customer Service by Amazon is a program offered to selling partners (SP) that fulfill their own orders. We refer to this as the Merchant fulfilled network (MFN) as opposed to Fulfilled by Amazon (FBA) where SPs send their products to an Amazon warehouse and Amazon takes care of all sales, shipments, refunds and customer service on their behalf. With CSBA, MFN sellers can save time and effort by having Amazon handle all of their post-purchase customer inquiries on their behalf.
The minimum lovable product (MLP) version of the CSBA Insights dashboard provides SPs with visibility into the customer services that Amazon is providing on their behalf. Prior to the dashboard, SPs would only receive a monthly email detailing their fee rate for the upcoming month and the amount of contacts handled for them in the previous three months.
Getting Started
I joined CSBA in January of 2023, just as high fidelity designs were about to be locked and front-end engineering was about to start. I received designs for the CSBA home page and dashboard from the previous designer and began to comb through them to understand the program and the product better.
The first thing I did was convert all the elements of the designs into components of the Amazon Katal design system. This would make it easier for the engineers to build using standardized components, rather than recoding every element on the page.
On the home page, I added a pricing section into the main body of the page to be more upfront about the cost to some SPs. While 80-90% of SPs will enjoy the program for free I thought it was important not to bury this possible cost in the FAQ section. I also made the fee table more visually pleasing and after testing with users I was confident that it was easier to understand.
Optimizing the home page
After converting all the page elements to the design system I was also sure to fix the page hierarchy. On the left you can see that all the headings were different sizes and colors and the eye was not sure where to go.
I also adding two banners to entice SPs more with our value propositions as well as fixed the 'How it works' section to better describe the program and its benefits.
After the launch of the CSBA home page in July 2023, we saw a net adoption impact of +18bps, corresponding to roughly 1.7 MM annual CSBA units shipped.
Voice of Seller (VoS) research campaign
In April of 2023 we started a VoS research campaign to get more insights into what is working with CSBA as well as where the seller pain points were. We wanted to uncover what major factors were leading to attrition among sellers as well as find the blockers for prospective SPs not joining CSBA.
Over the next month we interviewed 12 active and attritted sellers of various size companies. I was able to log the anecdotes from the interviews and create an affinity board. As far as what was going well, we found that there was no issues with costs, most sellers were getting the service for free or thought the fees were reasonable. Additionally, three quarters of sellers reported a noticeable workload reduction.
The top issues causing attrition among current and former sellers were:
• Too many refunds and returns
• Concerns about the quality and abilities of Amazon CS
• Buyer seller messaging leakage, in which sellers were still receiving messages that should've been handled by Amazon
• Lack of visibility into customer contacts
The top blockers of adoptions were:
• SPs having their own CS departments or not enough contacts to justify joining
• Concerns about giving away control of refunds and returns to Amazon
• Past bad experiences with other programs or aspects of Amazon.
Insights dashboard
Before joining CSBA, sellers were used to getting valuable information from their buyers during customer service contacts. After joining CSBA, some sellers were describing it as a black box. Now knowing how much of an issue the lack of visibility was to sellers, it was going to be very important to deliver the dashboard to them. This would be the first time they would again hear the voice of their customers. The most important feature of the new dashboard would be the customer contacts details table. Sellers would be able to see which orders made contact with Amazon and for what reasons.
I like to put myself in the shoes of the user when I can to better understand how they might use our product. I could see that the designs I received from the previous designer were not going give the user the best experience when looking through their contacts to see what issues were arising. The contacts were being placed in separate buckets, one each the top three categories and everything else lumped into 'others.'
To follow more along with the standards of Seller Central, as well as give sellers the most control over their data, I altered the page to be one unified table. I moved the top three categories to the top for a quick view. Now it would be easier to search through the table and see all the categories. Rather than having to look through four different tables. This would also work better for upcoming features such as downloading the table.
Beta launch and collecting feedback
Since this was the first time we would be surfacing this data to external stakeholders we wanted to make sure it would be reliable and accurate. The contact reasons we would be showing to sellers are manually input by customer service associates and a deep dive had revealed that around 20% of categories and reasons could be inaccurate. We decided to launch the dashboard as a beta test for 52 sellers that our account managers worked closely with. This way we could see if there were any red flags before we launched to the entire seller pool of around 400,000 sellers. We used this opportunity to also test the usability of the dashboard and plan the roadmap for upcoming features.
The dashboard was very well received by sellers in the initial beta interviews. Right away sellers told me they loved to get insights such as "Where's my stuff?" being their most contacted issue. They said that they would have to look into aspects of their shipping such as re-evaluate their carrier and fix issues with tracking numbers.
After the launch of the dashboard in October 2023, we saw a net adoption impact of +28bps, corresponding to approximately 2.6MM annual CSBA units shipped
A few things came up in interviews that we overlooked and could easily fix in future releases namely: more contacts per page for the pagination, granting access to secondary sellers, seeing past data from further than the previous month and the ability to download the contacts for in house analysis. I also made the mistake of designing the table to default to 'sort by category' and after testing changed the default to 'sort by date.' We fixed the order of the table as well because it made more sense to have the order IDs and dates of contact first.
Dashboard Refunds Update
From seller interviews we knew the the second biggest issue facing sellers was the amount of refunds and returns. Sellers told us they had no idea how much being a part of CSBA was actually costing them. We had to show the refunds amount to the seller so they would know a little bit more of what happen with the contact. Currently they only resolution details the sellers would see on the dashboard was whether the contact was open or resolved.
The previous designer and the tech team wanted to create another tab with a separate table to show refunds to the seller. Their justification was that the data was coming from different sources and it would not work well to combine these sources, with different refresh schedules, into one table.
Once again I put myself in the sellers shoes and could see that there was no way this could be the best option for the seller. It would be so frustrating for the seller to keep going back and forth to match up contacts in one table with refunds in the other. CSBA is a program about contacts so we needed to give the seller a complete end-to-end view of the contact, including the resolution and how much it was actually costing them.
I added the refunds details to the contacts table, added more filters to give the seller more control of their data and consolidated the columns to make it more compact and easier to read through. From usability testing, we saw no issues with the updates and sellers expressed great excitement for these upcoming changes.
%201.png){kind=small}
