Course Project

The course project is your chance to gain hands on HRI research experience. You are expected to complete this project in a group of 2 to 4. This project must involve conducting a human-subjects study that answers a novel research question focused on the interaction between a physically embodied robot and 1 or more people. This page contains details on the project deliverables, due dates, as well as examples of prior projects (at the bottom of the page).

Course Project

---

• Project Timeline
• Project Deliverables
• Project Grading
• Helpful Information for Running Your Human-Subjects Study
• Example Projects from Prior Topics in HRI Student Teams

Project Timeline

---

Project Deliverables

---

Submission of Project Deliverables

Put your project pitches in this Google doc. All other project deliverables will be submitted on Canvas.

Project Pitches

Before pitching project ideas, read this entire web page thoroughly to get the full picture of what this project will entail.

Although the project will be completed in teams of 2-4, these pitches are made by individuals. The purpose of these project pitches is to aid in the formation of project teams (your project proposal can and will deviate from these project pitches, again, the project pitches are meant to serve as a starting point for team formation). In this Google document you'll pitch 1-3 project ideas that you'd be interested in pursuing for the course project. It does not need to be fully formed, but enough to give your classmates a good idea of what the project will entail.

Draft Project Proposal

Your draft project proposal should introduce your motivation for your project, the research questions it tries to answer, proposed methods, and an evaluation plan. We encourage you to usethe Lab 1 HRI Study Design Worksheet as a resource and template for the draft project proposal. Your draft proposal should include:

• Motivation & Experimental Design
  • Motivation: One paragraph describing why you're motivated to pursue this project. Why is it important and interesting?
  • Research questions: What research questions does your work try to answer?
  • Hypotheses: Detail your hypotheses.
  • Experimental Conditions: Detail your experimental conditions.
  • Experimental Design: Is your study between or within subjects? How many experimental conditions will a participant experience and in which order?
• Methods
  • Procedure: In 1-2 paragraphs, describe the procedure your human subjects will undergo.
  • Measures: Detail the measures that you will collect. This includes questionnaire measures, behavioral/observational measures, and interview questions. For the draft project proposal, you do not need to attach a Qualtrics survey (but you will for the finalized project proposal), for this deliverable, a bullet-point list will suffice.
  • Sample Size: State your intended sample size, as a reminder, these are the expectations for full credit for the project:
    • Group of 2: at least 14 participants
    • Group of 3: at least 20 participants
    • Group of 4: at least 26 participants
• Evaluation Plan
  • Quantitative Analysis: List which dependent measures you plan to evaluate using statistical tests.
  • Qualitative Analysis: Detail what, if any, qualitative analyses you would like to do, which measures are involved, and the goal(s) of these analysis.

Finalized Project Proposal

The finalized project proposal, in addition to what is contained in the draft proposal, should include:

• Timeline: A detailed timeilne with, at minimum, weekly project milestones.
• Qualtrics questionnaire: A Qualtrics questionnaire including all of your questionnaire measures (attached to the main document as a PDF).

Before submitting your final project proposal, set up a meeting with a member of the teaching team in the range of Wed-Mon, April 8-13. You'll receive a Canvas announcement on how to sign up for a time to meet with us to discuss your project proposal.

Project Deliverables & Update #1

Project Deliverables: For this first project deliverable, you will write a first draft of your final paper's introduction and background sections (see detailed instructions on expectations for these sections below in the Final Report section). You are expected to use the the ACM conference template for this submission. You are also expected to include a section on your genAI use for this deliverable, in which you will describe whether you used genAI tools and if so, which tool(s) you used and how you used them.

Project Update: You will also submit a brief update on your project progress that includes:

• The accomplishments you've made since the last project deliverable (1 paragraph or a bullet point list)
• Any unexpected roadblocks you've encountered (1-2 paragraphs or a bullet point list)
• An updated timeline of how you plan to complete the project

Project Theoretical Contribution (CMSC 30630 ONLY)

Students enrolled in the graduate section of the course (30630) will write a 1-2 page theoretical contribution positioning of their course project. If multiple students enrolled in the graduate section are on the same project team, each will need to write and submit their own theoretical contribution.

The purpose of this theoretical contribution positioning is to deepen the conceptual grounding of your project. This assignment is not a summary of prior work. Instead, it should clearly articulate how your project is positioned within, extends, or challenges existing theory in Human-Robot Interaction and related fields. You should:

• Identify Relevant Theoretical Framework(s): Identify one or more existing theories, models, or conceptual frameworks relevant to your research question (e.g., trust, rapport, anthropomorphism, agency, etc.). These may come from HRI, HCI, psychology, communication, or related disciplines. Clearly name and describe the theory, cite foundational and/or widely used papers, and explain the core constructs and relationships proposed by the theory.
• Position Your Project Relative to Existing Theory: Explain how your project relates to the identified theory: Does your study apply the theory in a new context (e.g., a new robot behavior)? Does it extend the theory (e.g., adding a new variable, mechanism, or boundary condition)? Does it challenge or test assumptions of the theory? Be specific about what is new or different.
• Theory-Driven Hypotheses or Predictions: Re-articulate your hypotheses so that they are clearly derived from the theoretical framework, not just intuitive expectations. Explain why the theory predicts your expected outcomes. If multiple theories could apply, briefly justify your choice.
• Gaps, Limitations, or Opportunities for Theoretical Development: Identify at least one limitation or gap in the existing theory as it applies to your context. What does the current theory fail to explain in HRI? How might your study help address this gap?

Project Deliverables & Update #2

Project Deliverables: For this project deliverable, you will write a first draft of your final paper's methods section (see detailed instructions on expectations for these sections below in the Final Report section). We realize that you will not be able (yet) to include all the details required for the final paper (e.g., IRB number, preregistraion link, robot code repo link), however, we encourage you to include as many elements as you can at this deliverable deadline, and at minimum:

• Hypotheses or research questions
• Experimental conditions
• Interaction context/task description
• Details on robot implementation
• Experimental protocol
• Measures

You are expected to add the methods section to the the final paper draft you started in the last project deliverable submission (introduction + background). You are also expected to update the section on your genAI use for this deliverable. The intention for these deliverables is that they will help you to iteratively write your final paper throughout the course (rather than all at the end).

Project Update: You will also submit a brief update on your project progress that includes:

• The accomplishments you've made since the last project deliverable (1 paragraph or a bullet point list)
• Any unexpected roadblocks you've encountered (1-2 paragraphs or a bullet point list)
• An updated timeline of how you plan to complete the project

Project Presentation

You will present the work that you've done in your course project during finals period. You should aim for your presentation to take 10-12 minutes, leaving 3-5 minutes for Q&A and transitioning to the next group (I will plan to cut you off if your presentation goes beyond 12 minutes). All members of your team are expected to participate in the presentation. Your slides should be uploaded to Canvas by the start of the class meeting where you will present. Your presentation should highlight the following:

• Motivation: Detail the research question(s) you were interested in investigating and why it's important to explore.
• Background: Describe any relevant background literature and tell us what unique and novel contributions your study is making beyond what already exists in the literature.
• Methods: Describe your study design (experimental conditions), hypotheses, study protocol (videos are great for this), and the measures you used to assess your hypotheses.
• Video: Show video clip(s) giving example(s) of the interactions participants had with the robot in each of your experimental conditions.
• Results: Report on the results you found. Please include all of the main measures, regardless of whether or not you found significant differences.
• Discussion/Implications: Reflect on the broader implications and takeaways from your results.
• Learning Outcomes: Enumerate some "lessons learned" from this project, what you learned by going out and actually conducting HRI research, beyond what you've learned from reading the papers in this class.

Your presentation will also be graded for its design and style. Please ensure that your slides are NOT too text heavy. If the audience is spending their time trying to read a lot of text on a slide, it's hard for them to pay attention to you. Use visuals and limited text on your slides, describing the details verbally during your presentation.

Final Report

The project report should be written in the style of an academic conference paper (like the papers we've been reading for our class discussions), feel free to use the ACM templates for conferences. The report should be approximately 6-8 pages excluding references and contain the following sections:

• Abstract: A concise (150-250 word) summary of your project's motivation, methods, results, and broader implications.
• Introduction: Motivate why the research you are presenting is both important and novel compared with prior work. Introduce the main concepts that you are investigating with your user study and briefly summarize your approach. All references to prior work should be supported with appropriate citations.
• Background: Review the relevant prior work and briefly highlight how the work you are doing is novel in light of the prior work. For full credit, include at least 15 references to prior work in this section.
• Methods: Include the following details in this section:
  • Link to AsPredicted preregistration
  • UChicago SBS IRB number
  • Link to the UChicago Box folder that contains all of your data (video data, questionnaire data, etc.)
  • Hypotheses or research questions
  • Experimental conditions
  • Interaction context/task description
  • Details on robot implementation (including a link to a public GitHub repo with your robot code)
  • Experimental protocol
  • Measures (remember to cite any measures you adopt from prior work)
  • Participant information (numbers in each condition, demographics)
• Results: Present both quantitative results based on observed participant behavior and/or Likert questionnaire responses as well as qualitative results based on open-ended questionnaire responses and/or interviews. Key results and dependent variables should be displayed visually (e.g., graphs) with readable labels (e.g., axis labels) that are in a large enough font size to easily read. All quantitative results should be accompanied with the test statistic, effect size, and p-value. All qualitative results should report the inter-rater reliability (Cohen's kappa).
• Discussion: Discuss how your findings answer (or not) your research questions and/or hypotheses, the broader implications of your results, and any limitations of your work.
• Conclusion: In 1 paragraph, summarize the main findings and broader implications of your work.
• Contributions: Detail the contributions of each team member, describing which portions of the project were completed by each team member.
• GenAI Use: In one sentence, describe whether you used genAI tools with this reading reflection and if so, which tool(s) you used and how you used them. Please refer to the Use of Generative AI section of the course syllabus for the expectations of genAI use in this course.

Teaming Survey

Due at the same time as your final report, ensure that you've filled out the course project teaming survey. Assuming that all team members make approximately equal contributions on the course project, all team members will receive the same grade. However, if certain team member(s) do not contribute a sufficient amount of work to the course project, each team member may receive a different grade that reflects their individual contributions.

Data Sharing

If you have not done so already, please share all of the data you've collected in your HRI project in a UChicago Box folder and add the teaching team to the shared folder. This includes questionnaire data, video data, and any other data you collected in your project.

Project Grading

---

The grading for the course project will be as follows:

• 20% intermediary deliverables - project pitches, proposals, progress reports
• 10% final project presentation
• 30% final report
• 40% project work - your grade based on the work that you accomplished during the class project is not dependent on your project being a 'success', your project work will be graded based on the project's scope, novelty, and the investigative work that you put into pursuing your research questions

For further details on project grading, please check out the final project grading rubric.

Helpful Information for Running Your Human-Subjects Study

---

Suggested Timeline

For running an in-person human subjects study, here's an example timeline of what your project work might look like:

• Weeks 1-3: Form project team, finalize study design (main research questions, experimental conditions, protocol, measures)
• Week 4: Develop study protocol, questionnaires, consent forms, etc. and submit your IRB application
• Week 5: Program the robot and go back-and-forth with the IRB if they request any changes to your IRB application
• Week 6: Pilot the study
• Weeks 7-8: Run human subjects
• Week 9: Conduct data analysis
• Week 10: Work on presentation and final report

Please note that the quarter is extremely short, you will have to be on top of it from day 1.

Human Subjects Protection Training

Since you will be conducting a human subjects study in your class project, you will need to submit an IRB (Internal Review Board) Application and have completed Human Subjects Protection Training (← that link contains all the info you need to complete the training). Before you submit your IRB application, you must have already completed the human subjects protection training.

UChicago Internal Review Board (IRB) Application

In order to conduct human subjects research in affiliation with UChicago, you must submit and get approved an Internal Review Board (IRB) application. This means that BEFORE you run any human subjects, you MUST have your IRB application approved. For HRI related research, we submit our IRB applications to UChicago's Social and Behavioral Sciences Institutional Review Board (SBS IRB). Their main website, https://sbsirb.uchicago.edu/, contains helpful information, templates, and guidance for completing an IRB application.

The purpose of the IRB is to ensure that UChicago research is being conducted ethically and with proper respect to human participants. Your IRB application will include:

• a high-level description of the research you plan to conduct
• a protocol specifying what each human participant will do in your research study
• a consent form that each participant will read and agree to before beginning your research study
• a description of how you plan to protect any identifiable data you collect from human participants and how you plan to store the data
• any questionnaires you plan to administer to the human participants
• any recruitment methods you plan to use (e.g., flyers, social media posts, emails)
• details on how participants are to be paid and how much they will be paid

To create, edit, submit, and amend your IRB application go to https://aura.uchicago.edu/ and click on the "IRB" button.

It is most helpful to model your IRB application after one that has already been approved. Please ask the teaching staff to provide you with model IRB application examples to help you as you create your own.

When you complete your own IRB, please put me (Sarah) as the PI and all other teaching team members as collaborators on the personnel list. Since I (Sarah) will be the PI for the IRB, this means that I must be the one to hit the "submit" button whenever you submit the IRB application or revisions to your IRB application. Please email me when you are ready to make an IRB submission, and I will hit that "submit" button.

HRI Measurement Scales

HRI researchers at the Naval Research Lab and George Mason University have compiled a database of scales HRI researchers use in their studies. I highly encourage you to visit their website Finding the "Perfect" Scale (especially their database page) where you can find questionnaire scales you can use for your study as well as a rating of the quality of each scale.

Qualtrics

If your study involves asking human participants to fill out a questionnaire, I recommended that you use Qualtrics. In order to get all the nice bells and whistles that comes with Qualtrics, we highly recommend that you request a pro account. You can do this by completing this User Agreement Form (for more context, see this UChicago ITS post about using Qualtrics at UChicago).

Recruitment

You will be responsible for recruiting your own participants. You can post on social media, send emails, post on UChicago organization sites, etc. Just be sure that you include all forms of recruitment in your IRB application that you plan to use. We recommend you try to recruit the following minimum number of participants, dependent on your group size:

• Group of 2: at least 14 participants
• Group of 3: at least 20 participants
• Group of 4: at least 26 participants

Robot

Payment

For in-person human participants, we will pay them at the rate of $12.00 per hour in the form of Amazon or Tango gift cards. To facilitate this payment, please email the teaching team daily when you are running your study with a list of email addresses that correspond with your study participants from that day. I will ensure that they are sent their Amazon or Tango gift card for participating in your study.

Piloting Your Study

It is SO important before you start running real human subjects that you practice, practice, practice executing your human subjects study. We recommend:

1. Running the study several times yourselves from top to bottom, AND
2. Running the study one one or two friends (including having them fill out the questionnaires) and asking them if anything was confusing or if they noticed any typos in your questionnaire, AND
3. Running the study on members of the teaching team and getting their feedback.

Piloting is essential to work out any kinks and ensure that your study is set up to adequately answer your research questions.

Example Projects from Prior Topics in HRI Student Teams

---