Usability Testing: IKEA
This was in no way affiliated with IKEA, it was a project for a class. This was a 10 week group project in which I learned the steps to create a usability test. Using all of the skills I gained, I did a usability test with 8 participants. After testing I analyzed the results and created a final presentation and paper. Below is the process and what I learned during each step.
Project Declaration
DESIGN GOAL: Aside from testing to see if the IKEA website achieves company goals, our team also wanted to see how effectively the website appeals to college students. College students often shop on a budget and are bound by a lack of transportation. If the IKEA website can appeal to the budgeting college shopper, then we felt that the website would be achieving IKEA’s mission statement. Additionally, catering to the college student will help IKEA appeal to a younger generation, which can establish a loyal customer base for the future. The main goal of this study was to assess how the IKEA website caters to the budgeting college student shopper.
WHEN:Testing occurred in Fall of 2018 during a 10 week period
WHO: College students who have moved within the last six months into an unfurnished place and IKEA
WHY: Students need cheaper and a easy way to buy furniture and IKEA wants to gain new customers while also gaining lifetime customers.
WHAT: Provided design recommendations based on the usability tests.
HOW: Scroll down to see more information
Usability Observation
For this assignment I needed to learn how to use Userzoom so that in the future when I run the usability tests I am able to use Userzoom correctly. In order to do this I found a website similar to IKEA and ran a 10 min test to understand how to run a task with a participant. This was recorded, but since I told them it would be kept private I can not share the video. These are the following things I learned:
-
How to create tasks on userzoom
-
How to use two computers during the sessions so that one person can obverse quietly on one while the particpant uses the other to do the tasks
-
How to save and look at the results after so that I am able to take more notes
Project Proposal
For this step the team identified our product, client, goals, target user group, key tasks, and key usability questions that we want to explore. All of this information can be found above in the project declaration, but here are the initial usability questions that were used for the website walk through and heuristic evaluation.
During this I learned that I need to be specific about who the target user is because the rest of the study relies on this. By also having the description of the target user, you can always look back if you start to go off topic.
Initial Usability Questions
1. Is the user able to complete the task
without further guidance/follow up
questions?
2. How comfortable is the user navigating
through the website?
3. What are the most common pathways
on the website that users take ?
4. What is the user’s impression of the
overall experience, and how can it be
improved?
a. How can we decrease the amount
of time that users spend on the
website?
b. Are there any distraction/confusion
on the website?
5. What issues does the user have when
attempting to complete the tasks?
6. How likely is a user to recommend this
website to a friend?
How willing is the user to use the
website again?
Cognitive Walkthrough and heuristic Evaluation
Here is the document for the cognitive walk through and heuristic evaluation
Here is a brief summary for each step during this walk through and evaluation
Step 1
For this step I created two tasks that I can run through and evaluate, which can be compared with my teammates findings.
The two tasks are:
-
Add a couch under $150 to shopping cart
-
Use a coupon to buy a set of products
Step 2
The next step was to do a cognitive walk through of the two tasks. I described each step and then gave the issue and design recommendation with a picture. I learned how to identify issues and how to identify what pages potentially can be used for testing.
Task 2
Step 3
For this step I created heuristic definitions and severity ratings so that anybody who were to look at this can understand what and how I am rating. I used these to rate each page during the two tasks. Below is a example of some of the definitions and evaluations.
Step 4
This is the final step for the eval and walk through. For this step I summarized my findings so that I can compare them with my teammates and decide what is most important for the usability test.
Summary of Findings
-
Should focus on finding specific items within a specific price range
-
Focus on how to use offers and find them
Usability Study Plan
Here is the document for this study plan
Here is a brief summary of the different components of the plan and what I learned from each one. This plan consists of purpose, research questions, participant characteristics, method, task list, test environment, facilitation approach, data collection, data evaluation, and reporting. For each part there will be a short summary of what it is and why it was important.
PURPOSE: This was an overview of why and who we will be testing. This shapes the rest of the categories in this plan. I learned it is important to be specific because you are creating the rest of the categories based on this. Also if someone else were to look at this then they should know exactly why you are doing this usability test.
RESEARCH QUESTIONS: For this section we revised research questions based on our usability tests and feedback from our teacher. I Iearned that research questions need to be revised throughout the process because different findings will occur before hand. Also it is best to have them looked at because you need to make sure others understand what you are testing.
PARTICIPANT CHARACTERISTICS: These characteristics are based on our target user and goals. For this I learned I need to include inclusions and exclusions. I also need to be specific because I need to make sure that I am answering my purpose and research questions.
METHOD: This is an overview of what we will be doing during the usability test which includes when questionnaires occur and how we will interact with the participant.
TASK LIST: This includes a summary of the three tasks we will do and how to know if it is a success or fail. In this section there is also a pre-test, study design, and post interview questions. I learned that it is important to include what makes a task successful or a failure because when you are evaluating you need to make sure you are being specific and be able to give recommendations that are possible.
TESTING ENVIRONMENT: This includes where, who, how many people, how to take notes, and what we will be using. From this I learned that you need to be specific because you need to have the same settings for each participant to keep the testing consistent and controlled.
FACILITATION APPROACH: This includes how the moderator and note taker will act during the testing. It is broken into three steps: pre-background test, moderated usability testing, and post-usability testing interview. I learned that it is important to know what the moderator and note taker will do so that good notes are taken.
DATA COLLECTION: This includes the quantitative and qualitative date we will be collecting. This is important because this will shape how we will be evaluating and note taking.
DATA EVALUATION: This explains how the data collected will be evaluated. This is important to do before testing because you are able to look for those things during the testing.
REPORTING: This includes who we will be reporting to and what we will be reporting. This is important because you need to make sure you are reporting the results the team wants and the client wants. This can also be used for a letter to let the client know what you will be presenting and when.
Usability Test Kit
Here is the document for the test kit
This test kit includes, screening questionnaire, study qualifications, consent form, scripts during facilitation, pre-test questionnaire, task scenarios, post task questionnaire, post test questionnaire, and data logging tables. For each section I will give a short summary and why it is important.
SCREENING QUESTIONNAIRE: This was sent out to college students through different forms of contact. The questions used were about their experiences with buying furniture and their living situation. We also needed to know who was willing to meet and what time. This was important because we needed to find people that were relevant to the study so that we followed our goal and helped our client.
STUDY QUALIFICATIONS: These were the participants that we wanted to test our website on. This was important to include because we needed to know who to include and exclude when looking at who filled out our screening questionnaire.
CONSENT FORM: During this study we recorded their voices and face so we needed to make sure that they were okay with that. This was important to have because we needed to be able to look back and take notes on anything we missed.
SCRIPTS DURING FACILITATION:A introductory and facilitation script were created so that every participant was told the same thing. This was important because we needed to have consistency across all tests to get accurate results.
PRE-TEST QUESTIONNAIRE: This was an online form that we had each participant fill out to expand on their experiences and living situation. This was important because we were able to use this information to analyze the difference between people who have used IKEA and who have not.
TASK SCENARIOS: We created five tasks that could test the usability and learnability of a few different aspects of the website. The scenarios are written so that they can picture themselves in that situation and they are written so that all five tasks are continuous. This is important so that it feels more natural for the participants to be using the website.
POST TASK QUESTIONNAIRE: Four questions were asked after each task to see how they felt about the difficulty and how they felt in general. This was important because we needed to know which parts of the website were usable.
POST TEST QUESTIONNAIRE: We had five questions for the participants to fill out which asked what parts of the website were most enjoyable, what they found difficult, if mistakes were easy to fix, and if they would use it again. This was important because we wanted to see how there overall experience was when using IKEA.
DATA LOGGING TABLES: These are used to keep our notes consistent across all participants. We wanted to take notes to know the numbers of errors, error types, and if they needed help. This was important so that we are able to know what parts of the website had the most errors and what participants were struggling with.
Testing
For this section I cannot share any videos due to privacy of our participants, but I will share some of the experiences I had with testing, userzoom, moderating, and note taking.
-
It was important for us to test out our kit to make sure that our tasks make sense to the user and so that we understood how to use userzoom
-
There were some complications as the test went on because the website changed half way through the week, but we were able to change our script to match what we still wanted to test.
-
It is hard to moderate and note take at the same time, so it was important that we always had two people in the room.
-
Even when you are just note taking, it is important to look back at the videos to make sure that you took note of anything you missed
-
It is also important to note that you should combine all of the notes right away so that you do not forget what went together
Preliminary Analysis Small Report
This is a short report about the product, participants, preliminary findings, severity, and our future work. This is a informal report that can be sent to the client. This is important so that they remember what we were looking for and what they are going to be seeing in the future from us.To the right is a picture of all the findings. In the report there are only the more severe findings. The informal report can be read by clicking on small report.
.png)
Final Paper and Presentation
For the final paper and presentation we looked at all of our notes to come up with four main findings and recommendations. The presentation includes a overview, methods, findings, and conclusions. We had a short clip with no faces or sound for the participants privacy.
For the paper we included everything we did throughout our study. For the findings and recommendations we have participants comments, pictures of the website, and our notes. Here is our some of the findings.
Key Findings:
-
Checkout Issues
-
Inconsistent Search Parameters
-
Couldn't find Coupons
-
Confused Wordings Buy Online
Key Recommendations
-
Our team recommends the IKEA website to prompt the user with the options to input an address at the beginning of the session and save it for the whole session (until the user exits out of the website)
-
Our recommendation is that every page should have an apply or filter button for when the user is changing what they are looking for and that there be consistent branding between the search filters.
-
Our team recommends that the IKEA team adds a page to the top bar of the website titled “Coupons” that displays all of the current coupons being offered for IKEA products.
-
Changing the word of “Buy Online” to “Add to (My) Cart”, participants will know exactly what to expect when clicking on the button.
What I Learned
Throughout all of this process there were five main things that I learned:
-
It is important to test out the usability test kit to make sure that everything makes sense for the participants and so that you are following your goals
-
Usability testing is important so that we are able to know if a website is usable for multiple groups of people, learnable, and mistakes are easily correctable
-
Heuristic evaluations and cognitive walk through can be useful to find initial problems that can be used to expand upon
-
User testing is not always going to go as planned because websites can change, so you should always be prepared with alternative tasks
-
It is not always going to be easy to find participants, so you need to find people on multiple platforms and try to incentivize others to share it as well