Lab 4: Wizard-of-Oz GUI for Misty


Learning Goals

Getting Started

Preparing Your Files

Please organize your Lab 4 files the same way as Lab 3 so that your imports work correctly and the teaching team can debug issues more easily.

  1. Create your Lab 4 folder with this exact name: Inside hri_course_misty_programming, create a folder named lab_4_misty_woz_gui. 
    cd hri_course_misty_programming
mkdir lab_4_misty_woz_gui
  2. Name your main Lab 4 file exactly: The Python file for this lab should be named lab_4_misty_woz_gui.py. 
    touch lab_4_misty_woz_gui/lab_4_misty_woz_gui.py
    This filename matters because the lab instructions and example run command assume this exact name.
  3. Design and implement your own interface in that file: This year, you will not be given starter GUI code. Instead, you should create your own Wizard-of-Oz interface and decide how to organize the controls, layout, and interaction flow.
  4. Activate your virtual environment before running anything: Each time you start working on this lab, cd hri_course_misty_programming and run source venv/bin/activate (or venv\Scripts\activate on Windows). When you're done, run deactivate.
  5. Use this folder structure: To make debugging easier, everyone should organize their files in the same way: 
    hri_course_misty_programming/
├── venv/
├── Python-SDK/
├── lab_3_misty_introduction/
│   └── misty_introduction.py
└── lab_4_misty_woz_gui/
    └── lab_4_misty_woz_gui.py
    Important:
    • Python-SDK should be directly inside hri_course_misty_programming
    • Your Lab 4 folder should also be directly inside hri_course_misty_programming
    • Do not put Python-SDK inside your lab folder
  6. How to run: With your virtual environment activated and while inside lab_4_misty_woz_gui, run: 
    python3 lab_4_misty_woz_gui.py <MISTY-IP-ADDRESS>
    Replace <MISTY-IP-ADDRESS> with your robot's IP address.

Lab 4 Overview

In this lab, you will design and build your own GUI interface that allows a human to control Misty during a live human-robot interaction. The goal is to simulate Misty's conversational abilities and non-verbal behaviors in a controlled setting while giving you practice making interface design decisions yourself.

The interface will include:

Here is an example GUI to help you envision the kind of layout you might build:

Example Wizard-of-Oz GUI layout

Designing Your Own Interface

This year, we are not providing starter GUI code for Lab 4. Instead, your group should research how to build a Python-based interface for Misty and then create your own Wizard-of-Oz system from scratch in lab_4_misty_woz_gui.py.

Your team should decide what kind of interface will work best for a Wizard who needs to control Misty quickly and clearly during the interaction. For example, you might research tkinter, PySide6, PyQt6, wxPython, or another lightweight approach that you can run locally on your laptop. The key requirement is that your design is usable in real time and successfully controls the robot.

As part of this process, you should:

Note: Because you are designing the interface yourselves, you may need to look up documentation or tutorials for the GUI toolkit you choose. Keep your design simple and practical. A clear, reliable interface is much better than an overly complex one.

A useful starting point for comparing GUI libraries is this overview page: Which Python GUI library should you use?

Working in Groups

You will work with this same group for all of the robot programming labs (Labs 3-5). Each group will turn in one piece of code for their GUI design.

Lab 4 Deliverables & Submission

Your GUI must include:

We ask you to turn in the following:

Example requirements.txt 

requests==2.31.0
Pillow==10.3.0
websocket-client==0.57.0
PySide6==6.6.0

When the teaching team receives your requirements.txt, we will recreate your environment by activating our course virtual environment and running pip install -r requirements.txt from within hri_course_misty_programming.

Important: The teaching team needs to be able to run your GUI when grading. If your submitted code does not include enough information for us to reproduce your software environment, or if we cannot run your GUI because dependencies are missing or unclear, you may receive deductions.

Submit everything to Canvas by Thursday, April 16, 2026 at 11:59pm.

"Three Good Things" Interaction Flow

As we introduced in Lab 3, you will be programming the Misty robot to act as a facilitator of a positive psychology exercise called the "Three Good Things." This exercise is a well-known positive psychology intervention that has been shown to improve overall mental health and well-being. In this exercise, both the robot and the participant will take turns sharing three things that they are grateful for that have happened to them in the past week. Here's how the interaction should flow between the robot and human participant and which parts should be pre-scripted vs. dynamic for the wizard:

Tips & Resources

Extra Challenge

For an additional challenge, consider adding:

Common Errors & Troubleshooting

Before reaching out to the teaching team for help, please read through this section. Many Lab 4 issues come from the same setup and import problems that appeared in Lab 3.

HTTP Connection Errors

If your GUI runs but Misty does not respond, or if you see a connection error when trying to send a command to the robot, first check the following:

If you still cannot connect, refer back to the HTTP Connection Errors troubleshooting instructions from Lab 3. The same local network, firewall, and permissions issues can prevent your Lab 4 GUI from talking to Misty.

Module Not Found Errors

Even after installing packages, you might still see errors like these:

ModuleNotFoundError: No module named 'tkinter'
ModuleNotFoundError: No module named 'PySide6'
ModuleNotFoundError: No module named 'PyQt6'
ModuleNotFoundError: No module named 'wx'
ModuleNotFoundError: No module named 'requests'
ModuleNotFoundError: No module named 'PIL'

For this lab, the most common causes are:

The same debugging process from Lab 3 applies here, so confirm you are running the Python interpreter that has your GUI toolkit installed and that the toolkit is listed in requirements.txt.

Important: Lab 4 depends on the same environment setup as Lab 3. If you run into import problems, go back to the Module Not Found Errors troubleshooting steps from Lab 3 and apply the same process here.

Misty SDK Import Errors

If you see an error like this:

from mistyPy.Robot import Robot
ModuleNotFoundError: No module named 'mistyPy.Robot'

This usually means Python cannot find the Misty Python SDK. Double-check that your folder structure matches the required structure shown earlier on this page and that Python-SDK is directly inside hri_course_misty_programming, not inside your Lab 4 folder.

If needed, reuse the same path debugging strategy from Lab 3 to confirm the path your code is appending for the SDK.


© 2026 Sarah Sebo