An academic project focused on revamping the evaluation process for sellers offering services on online platforms. The scope of the project also offered an opportunity to introduce features that allow sellers to present their improvements based on the feedback they receive from shoppers.
UX Research: Secondary Research, User Interviews, User Surveys, User persona development, User Scenario Storyboard, Affinity Diagramming, Video-making
UX Design: Ideation, Lo-fi Wireframes, Mid-fi Mockups, User Testing, Hi-fi Prototypes
Duration: 1 month, Aug 23 - Sept 23
Type: Academic Project
Team: Harshavardhan Thokala, Graham Allgood, Prithvi Manjunatha, and Sowymyashree Bharathan
As part of the UX Design Graduate Studio, a core course in Purdue University's UX Design master's program, we were assigned the task of redesigning the evaluation process for freelancers on online platforms, with a specific focus on reimagining the conventional 5-star rating system.
The main objectives of this project were to:
- Ensure the journey of working with a seller is clear to potential buyers.
- Provide sellers with a fair and balanced assessment of their work.
- Enable sellers to use ratings for improvement and to demonstrate their progress over time.
People are increasingly accepting the idea of availing services from sellers through online platforms. However, the method of evaluating sellers through reviews and ratings appears inconsistent, lacking in detail about the provided service, and not accurately reflecting the received service, thereby impacting the sellers' business.
The challenges include evaluating the effort required of the buyers and the ease of reviewing sellers while still providing constructive and accurate information about the service. Additionally, there is a need to anticipate how the shared information could potentially be exploited against the sellers.
The goal of the 'Comprehensive Rating Breakdown' feature is to enhance the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process, such as Product Description, Shipping, Product Quality, and Customer Service.
The challenges include evaluating the effort required of the customers, and the ease of reviewing freelancers while still providing constructive and accurate information about the service. Secondly, anticipating how the shared information could potentially be exploited against the freelancers.
We conducted a review of existing academic research and industry case studies to gain insights into the problem space.
Through interviews with four Etsy sellers from across the United States, we delved deeper into Etsy's current review system and Star Seller Program.
Through user surveys, we gathered insights from sellers worldwide, which helped us comprehend issues from a global perspective.
Through a 2-step primary and secondary research process, we identified key issues that sellers and buyers typically face on freelancing platforms. Our initial insights into the problem space came from secondary research, where research papers and case studies in this domain provided a strong understanding of the prevalent issues on online freelancing websites.
Our findings revealed that the 5-star rating system is still considered the most effective option for purchasing a product compared to binary and 10-star ratings. This is because it has a strong cognitive fit, offering users a clear understanding of the rating system and its associated meanings. Additionally, we explored weaknesses of the 5-star rating system, such as differences in mental models, and proposed information guidelines to ensure that buyers, sellers, and the platform share a common understanding of what each star rating represents.
After conducting interviews and surveys with the main target user group, we affinity-mapped the insights to categorize the data and identify common issues that sellers were experiencing with the platform. We then synthesized the data to develop themes aimed at solving user problems and improving the rating and reviewing experience for both the seller and buyer user groups.
After gathering information on the most commonly experienced pain points by users, we focused on identifying the main reasons causing these issues.
Lack of Motivation
There is a lack of motivation to provide ratings or reviews, as buyers perceive it as an additional task.
Diverse Perspectives
Perceptions of ratings vary depending on the buyer's background and previous experiences.
Absence of Way to Demonstarte Progress
Etsy focuses exclusively on enhancing the buyer experience.
Buyers unaware of the impact of their ratings on Sellers
Etsy’s primary emphasis has been on encouraging buyers to provide a rating rather than educating them about its impact.
Unclear Guidelines on how to Improve Ratings
Lack of assistance from Etsy leaves sellers to explore the solution to improve their ratings.
One Aspect of Buying Experience Affecting the Overall Rating
Negative experiences have a stronger impact on ratings than positive ones.
From our insights and analysis of research data, we identified two main user groups: sellers (small business owners) and buyers. To better understand their needs, we developed personas for these stakeholders, which were crucial for efficiently identifying and communicating user needs. The personas, along with user scenario storyboard, further guided our process in identifying user needs and pinpointing key issues.
To visualize the steps sellers and buyers take to achieve their goals on the online freelancing platform, we developed storyboards for each of the involved stakeholders. This process helped pinpoint issues that users might encounter while interacting with the product, allowing us to learn about user behaviors, motivations, and opportunities for our redesign.
Translating the findings from primary and secondary research to develop solutions for the identified pain points, we crafted low-fidelity wireframes to explore various options. Narrowing down to the most feasible, functional, and intuitive design, the wireframes presented above showcase a few of our redesigns.
The following mock-ups represent the mid fidelity wireframes used for conducting user evaluation
Mock-up of the redesigned review and ratings interface
Pop-up message prompting buyers to communicate their issue with seller
Left: Etsy providing guidelines for sellers to improve their ratings
Right: Etsy informing buyers about sellers imorovement based on reviews
To validate our proposed redesign and further discover opportunites fro improvements, we conducted an unmoderated usability testing with sellers on Etsy platform using the mid fidelity mock-ups. The following insights were provided by the participants influencing our final design proposal:
- Breaking down the initial review screen based on categories and progressively disclosing the rating and review tasks.
- Including a progress bar informing buyers about their current state in the reviewing process.
- Reorder reviews interface to categorize tasks better.
- Users feared losing review progress by selecting "Contact Seller."
- Users were unsure about the next steps after the star information screen.
The goal of the "Comprehensive Rating Breakdown" feature is to elevate the feedback experience by introducing sub-categories within ratings for purchases. These sub-categories encompass critical elements of the buying process such as Product Description, Shipping, Product Quality, and Customer Service.
The goal of the "Guided Feedback" feature is to educate and empower users about the vital role their reviews play in the Etsy community aiming to steer buyers towards a more thoughtful and precise evaluation of their purchases. By providing clear and contextual guidelines for each rating level within the feedback pop-up.
The goal of this feature is to Create a toolkit within the Etsy platform that offers comprehensive guidance and strategies to help sellers enhance their ratings. This toolkit could include tips on improving product quality, enhancing customer service, optimizing shipping processes, and engaging effectively with buyers. Additionally, it could provide insights into interpreting customer feedback to identify areas for improvement, empowering sellers to take targeted actions and ultimately boost their ratings on Etsy.
The goal of this feature is to exhibit the improvement in a seller's ratings over time. For example, it highlights enhancements in shipping ratings from for instance 3 to 4 over a designated time span. When a buyer views a product from that seller, a banner will display the shipping rating improvement achieved within the last 6 months, showcasing the seller's commitment to enhancing their service.
