Revamping the 5-star evaluation process for sellers providing services on online platform. The project's scope enabled us to introduce features for sellers to present improvements based on shoppers feedback.
UX Research: Secondary Research, User Interviews, User Surveys, User persona development, User Scenario Storyboard, Affinity Diagramming, Video-making
UX Design: Ideation, Lo-fi Wireframes, Mid-fi Mockups, User Testing, Hi-fi Prototypes, Video making
Duration: 1 months, Aug '23 - Sept '23
Type: Academic Design Project
Team: Harshavardhan Thokala, Graham Allgood, Prithvi Manjunatha, Sowymyashree Bharathan
As part of the UX Design Graduate Studio, a core course in Purdue University's UX Design master's program, we were tasked with redesigning the evaluation process for freelancers on online platforms. The goal was to create a fair and balanced evaluation system that benefits both buyers and sellers, by reimagining the conventional 5-star rating system.
The main objectives of this project were to ensure potential sellers understand the seller’s journey. Design a evaluation system to assess sellers’ work fairly and enable them to use ratings for improvement and demonstrate progress to potential future sellers.
People are increasingly accepting the idea of availing services from sellers through online platforms. However, the method of evaluating sellers through reviews and ratings appears inconsistent, lacks detail, and does not accurately reflect the received service, thereby impacting the sellers' business.
The goal was to evaluate and propose a redesign of the effort required of the buyers to review sellers fairly while still providing constructive and accurate information about their service. Additionally, there is a need to anticipate how the shared information could potentially be exploited against the sellers.
The goal of the 'Comprehensive Rating Breakdown' feature is to enhance the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process, such as Product Description, Shipping, Product Quality, and Customer Service.
The goal of the 'Guided Feedback' feature is to educate and empower users about the vital role their reviews play in the Etsy community, aiming to guide buyers toward a more thoughtful and precise evaluation of their purchases. This is achieved by providing clear and contextual guidelines for each rating level within the feedback pop-up.
The goal of this feature is to create a toolkit within the Etsy platform that offers comprehensive guidance and strategies to help sellers enhance their ratings. This toolkit could include tips on improving product quality, enhancing customer service, optimizing shipping processes, and engaging effectively with buyers. Additionally, it could provide insights into interpreting customer feedback to identify areas for improvement, empowering sellers to take targeted actions and ultimately boost their ratings on Etsy.
The goal of this feature is to demonstrate the improvement in a seller's ratings over time. For example, it highlights enhancements in shipping ratings, such as from 3 to 4, over a designated time span. When a buyer views a product from that seller, a banner will display the shipping rating improvement achieved within the last 6 months, showcasing the seller's commitment to enhancing their service.
Through a 2-step primary and secondary research process, we identified key issues that sellers and buyers typically face on freelancing platforms. Our initial insights into the problem space came from secondary research, where research papers and case studies in this domain provided a strong understanding of the prevalent issues on online freelancing websites.
We conducted a review of existing academic research and industry case studies to gain insights into the problem space.
Through interviews with four Etsy sellers from across the United States, we delved deeper into Etsy's current review system and Star Seller Program.
Through user surveys, we gathered
insights from sellers across the world, which helped us comprehend issues from a wider perspective.
Our findings revealed that the 5-star rating system is still considered the most effective option for purchasing a product compared to binary and 10-star ratings. This is because it has a strong cognitive fit, offering users a clear understanding of the rating system and its associated meanings. Additionally, we explored weaknesses of the 5-star rating system, such as differences in mental models, and proposed information guidelines to ensure that buyers, sellers, and the platform share a common understanding of what each star rating represents.
After conducting interviews and surveys with the main target user group, we affinity-mapped the insights to categorize the data and identify common issues that sellers were experiencing with the platform. We then synthesized the data to develop themes aimed at solving user problems and improving the rating and reviewing experience for both the seller and buyer user groups.
Affinity Diagram of User Interviews
After gathering information on the most commonly experienced pain points by users, we focused on identifying the main reasons causing these issues.
Lack of Motivation from Buyers to Rate their Experience
There is a lack of motivation to provide ratings or reviews, as buyers perceive it as an additional task.
Buyers and Sellers View 5-Ratings Scale Differently
Perceptions of ratings vary depending on the buyer's background and previous experiences.
Absence of Way to Demonstrate Progress Based on Reviews
Etsy focuses exclusively on enhancing the buyer experience.
Buyers Unaware of the Impact of their Ratings on Sellers
Etsy’s primary emphasis has been on encouraging buyers to provide a rating rather than educating them about its impact.
Unclear Guidelines on how to Improve Ratings
Lack of assistance from Etsy leaves sellers to explore the solution to improve their ratings.
One Aspect of Buying Experience Affecting the Overall Rating
Negative experiences have a stronger impact on ratings than positive ones.
Mental Model Diagram
From our insights and analysis of research data, we identified two main user groups: sellers (small business owners) and buyers. To better understand their needs, we developed personas for these stakeholders, which were crucial for efficiently identifying and communicating user needs. The personas, along with user scenario storyboard, further guided our process in identifying user needs and pinpointing key issues.
Sellers Use Persona
Buyers Use Persona
To visualize the steps sellers and buyers take to achieve their goals on the online freelancing platform, we developed storyboards for each of the involved stakeholders. This process helped pinpoint issues that users might encounter while interacting with the product, allowing us to learn about user behaviors, motivations, and opportunities for our redesign.
Storyboard - Seller's Perspective
Storyboard - Buyer's Perspective
To translate our findings from primary and secondary research, we developed solutions for the identified pain points through low-fidelity wireframes. Narrowing down to the most feasible, functional, and intuitive design, the wireframes presented above showcase a few of our redesigns.
Mock-up of the redesigned review and ratings interface
Pop-up message prompting buyers to communicate their issue with seller
Left: Etsy providing guidelines for sellers to improve their ratings
Right: Etsy informing buyers about sellers imorovement based on reviews
To validate our proposed redesign and further discover opportunites fro improvements, we conducted an unmoderated usability testing with sellers on Etsy platform using the mid fidelity mock-ups. The following insights were provided by the participants influencing our final design proposal:
- Breaking down the initial review screen based on categories and progressively disclosing the rating and review tasks.
- Including a progress bar informing buyers about their current state in the reviewing process.
- Reorder reviews interface to categorize tasks better.
- Users feared losing review progress by selecting "Contact Seller."
- Users were unsure about the next steps after the star information screen.
The goal of the "Comprehensive Rating Breakdown" feature is to elevate the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process such as Product Description, Shipping, Product Quality, and Customer Service.
Comprehensive Rating Breakdown
The goal of the "Guided Feedback" feature is to educate and empower users about the vital role their reviews play in the Etsy community aiming to steer buyers towards a more thoughtful and precise evaluation of their purchases. By providing clear and contextual guidelines for each rating level within the feedback pop-up.
Guided Feedback
The goal of this feature is to Create a toolkit within the Etsy platform that offers comprehensive guidance and strategies to help sellers enhance their ratings. This toolkit could include tips on improving product quality, enhancing customer service, optimizing shipping processes, and engaging effectively with buyers. Additionally, it could provide insights into interpreting customer feedback to identify areas for improvement, empowering sellers to take targeted actions and ultimately boost their ratings on Etsy.
Rating Optimization
The goal of this feature is to exhibit the improvement in a seller's ratings over time. For example, it highlights enhancements in shipping ratings from for instance 3 to 4 over a designated time span. When a buyer views a product from that seller, a banner will display the shipping rating improvement achieved within the last 6 months, showcasing the seller's commitment to enhancing their service.
Performance Progress Banner