Class Meeting 08: Q-Learning in a 5 Room Building Class Exercise Solutions

This page contains solutions for the q-learning in a 5 room building class exercise in Class Meeting 08.

\(Q(s_t, a_t)\) after trajectory 0

	North	South	East	West
rm₀	0	0	0	0
rm₁	0	0	0	0
rm₂	0	0	0	0
rm₃	0	0	0	0
rm₄	0	100	0	0
rm₅	100	0	0	0

\(Q(s_t, a_t)\) after trajectory 1

	North	South	East	West
rm₀	0	0	0	0
rm₁	180	0	0	0
rm₂	0	0	0	0
rm₃	0	0	0	0
rm₄	0	100	0	0
rm₅	100	180	0	0

\(Q(s_t, a_t)\) after trajectory 2

	North	South	East	West
rm₀	0	80	0	0
rm₁	180	0	0	0
rm₂	0	0	0	0
rm₃	0	0	0	80
rm₄	0	244	0	0
rm₅	100	180	0	0

\(Q(s_t, a_t)\) after trajectory 3

	North	South	East	West
rm₀	0	80	0	0
rm₁	244	0	0	0
rm₂	0	0	0	64
rm₃	144	0	0	195.2
rm₄	0	244	156.16	0
rm₅	100	180	0	0

\(\pi(s)\) after trajectory 3

	\(a\)
rm₀	South
rm₁	North
rm₂	West
rm₃	West
rm₄	South
rm₅	South