Class Meeting 13: Partially Observable Markov Decision Processes (POMDPs)
Today's Class Meeting
- Learning about partially observable Markov decision processes (POMDPs) and an application of POMDPs to child-robot tutoring (here's a link to the slides)
- Q-learning project studio (work) time
Resources where you can learn more about POMDPs
- Hossein Kamalzadeh and Michael Hahsler's POMDP: Introduction to Partially Observable Markov Decision Processes, which details the use of the pomdp R package
- pomdp.org - which contains tutorials, code, examples, and more on POMDPs
- The Wikipedia page on POMDPs
- Cassandra, A. R., Kaelbling, L. P., & Littman, M. L. (1994, July). Acting optimally in partially observable stochastic domains. In AAAI (Vol. 94, pp. 1023-1028).
- Kaelbling, L. P., Littman, M. L., & Cassandra, A. R. (1998). Planning and acting in partially observable stochastic domains. Artificial intelligence, 101(1-2), 99-134.
Q-learning Project Studio (Work) Time
Please feel free to use the remainder of class time to work on your Q-learning projects. It's up to you and your team how you want to use this time. You can stay or leave the class Zoom, it's up to you. We'll be opening up breakout rooms for project teams if you want to use them.
If you would like some debugging help or have a question you'd like to discuss with the teaching team, please use the Zoom "ask for help" feature or wait for a teaching team member to come by your group to check in with you.
Acknowledgments
The information delivered for today's lecture was informed by tutorial content on pomdp.org and images from Hossein Kamalzadeh and Michael Hahsler's POMDP: Introduction to Partially Observable Markov Decision Processes.