You may never be sure whether you have an effective user experience until you have tested it with users. In this course, you’ll learn how to design user-centered experiments, how to run such experiments, and how to analyze data from these experiments in order to evaluate and validate user experiences. You will work through real-world examples of experiments from the fields of UX, IxD, and HCI, understanding issues in experiment design and analysis. You will analyze multiple data sets using recipes given to you in the R statistical programming language -- no prior programming experience is assumed or required, but you will be required to read, understand, and modify code snippets provided to you. By the end of the course, you will be able to knowledgeably design, run, and analyze your own experiments that give statistical weight to your designs.
Designing, Running, and Analyzing Experiments
This course is part of Interaction Design Specialization
Instructors: Scott Klemmer
31,379 already enrolled
Included with
(590 reviews)
Skills you'll gain
Details to know
Add to your LinkedIn profile
17 assignments
See how employees at top companies are mastering in-demand skills
Build your subject-matter expertise
- Learn new concepts from industry experts
- Gain a foundational understanding of a subject or tool
- Develop job-relevant skills with hands-on projects
- Earn a shareable career certificate
Earn a career certificate
Add this credential to your LinkedIn profile, resume, or CV
Share it on social media and in your performance review
There are 9 modules in this course
In this module, you will learn basic concepts relevant to the design and analysis of experiments, including mean comparisons, variance, statistical significance, practical significance, sampling, inclusion and exclusion criteria, and informed consent. You’ll also learn to think of an experiment in terms of usability, its participants, apparatus, procedure, and design & analysis. This module covers lecture videos 1-2.
What's included
2 videos1 reading1 assignment
In this module, you will learn how to analyze user preferences (or other tallies) using tests of proportions. You will also get up and running with R and RStudio. Topics covered include independent and dependent variables, variable types, exploratory data analysis, p-values, asymptotic tests, exact tests, one-sample tests, two-sample tests, Chi-Square test, G-test, Fisher’s exact test, binomial test, multinomial test, post hoc tests, and pairwise comparisons. This module covers lecture videos 3-9.
What's included
7 videos2 assignments
In this module, you will learn how to design and analyze a simple website A/B test. Topics include measurement error, independent variables as factors, factor levels, between-subjects factors, within-subjects factors, dependent variables as responses, response types, balanced designs, and how to report a t-test. You will perform your first analysis of variance in the form of an independent-samples t-test. This module covers lecture videos 10-11.
What's included
2 videos2 assignments
In this module, you will learn about how to ensure that your data is valid through the design of experiments, and that your analyses are valid by understanding and testing for certain assumptions. Topics include how to achieve experimental control, confounds, ecological validity, the three assumptions of ANOVA, data distributions, residuals, normality, homoscedasticity, parametric versus nonparametric tests, the Shapiro-Wilk test, the Kolmogorov-Smirnov test, Levene’s test, the Brown-Forsythe test, and the Mann-Whitney U test. This module covers lecture videos 12-15.
What's included
4 videos2 assignments
In this module, you will learn about one-factor between-subjects experiments. The experiment examined will be a between-subjects study of task completion time with various programming tools. You will understand and analyze data from two-level factors and three-level factors using the independent-samples t-test, Mann-Whitney U test, one-way ANOVA, and Kruskal-Wallis test. You will learn how to report an F-test. You will also understand omnibus tests and how they relate to post hoc pairwise comparisons with adjustments for multiple comparisons. This module covers lecture videos 16-18.
What's included
3 videos2 assignments
In this module, you will learn about one-factor within-subjects experiments, also known as repeated measures designs. The experiment examined will be a within-subjects study of subjects searching for contacts in a smartphone contacts manager, including the analysis of times, errors, and effort Likert-type scale ratings. You will learn counterbalancing strategies to avoid carryover effects, including full counterbalancing, Latin Squares, and balanced Latin Squares. You will understand and analyze data from two-level factors and three-level factors using the paired-samples t-test, Wilcoxon signed-rank test, one-way repeated measures ANOVA, and Friedman test. This module covers lecture videos 19-23.
What's included
5 videos2 assignments
In this module, you will learn about experiments with multiple factors and factorial ANOVAs. The experiment examined will be text entry performance on different smartphone keyboards while sitting, standing, and walking. Topics include mixed factorial designs, interaction effects, factorial ANOVAs, and the Aligned Rank Transform as a nonparametric factorial ANOVA. This module covers lecture videos 24-27.
What's included
4 videos2 assignments
In this module, you will learn about analyses for non-normal or non-numeric responses for between-subjects experiments using Generalized Linear Models (GLM). We will revisit three previous experiments and analyze them using generalized models. Topics include a review of response distributions, nominal logistic regression, ordinal logistic regression, and Poisson regression. This module covers lecture videos 28-29.
What's included
2 videos2 assignments
In this module, you will learn about mixed effects models, specifically Linear Mixed Models (LMM) and Generalized Linear Mixed Models (GLMM). We will revisit our prior experiment on text entry performance on smartphones but this time, keeping every single measurement trial as part of the analysis. The full set of analyses covered in this course will also be reviewed. This module covers lecture videos 30-33.
What's included
4 videos2 assignments
Instructors
Offered by
Recommended if you're interested in Design and Product
Johns Hopkins University
Ball State University
University of Minnesota
Why people choose Coursera for their career
Learner reviews
590 reviews
- 5 stars
40.43%
- 4 stars
19.45%
- 3 stars
14.21%
- 2 stars
9.13%
- 1 star
16.75%
Showing 3 of 590
Reviewed on Jun 16, 2016
The instructor for this course was great. He was very responsive to students' questions concerns.
Reviewed on Feb 26, 2019
Without any background in R programming and experiment design, I am able to learn a lot of useful stuff in this course. I wish the last three lessons and quizzes are a little more beginner friendly.
Reviewed on Sep 8, 2020
While the course if useful as an introduction to spectrum of tests used in experiment design, heavy external reading is required to truly understand the concepts in depth
New to Design and Product? Start here.
Open new doors with Coursera Plus
Unlimited access to 10,000+ world-class courses, hands-on projects, and job-ready certificate programs - all included in your subscription
Advance your career with an online degree
Earn a degree from world-class universities - 100% online
Join over 3,400 global companies that choose Coursera for Business
Upskill your employees to excel in the digital economy
Frequently asked questions
Access to lectures and assignments depends on your type of enrollment. If you take a course in audit mode, you will be able to see most course materials for free. To access graded assignments and to earn a Certificate, you will need to purchase the Certificate experience, during or after your audit. If you don't see the audit option:
The course may not offer an audit option. You can try a Free Trial instead, or apply for Financial Aid.
The course may offer 'Full Course, No Certificate' instead. This option lets you see all course materials, submit required assessments, and get a final grade. This also means that you will not be able to purchase a Certificate experience.
When you enroll in the course, you get access to all of the courses in the Specialization, and you earn a certificate when you complete the work. Your electronic Certificate will be added to your Accomplishments page - from there, you can print your Certificate or add it to your LinkedIn profile. If you only want to read and view the course content, you can audit the course for free.
If you subscribed, you get a 7-day free trial during which you can cancel at no penalty. After that, we don’t give refunds, but you can cancel your subscription at any time. See our full refund policy.