MIT 6.390 Intro to Machine Learning: 11 Markov Decision Processes (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 11: Markov Decision Processes of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

Consider a robot learning to navigate through a maze, a game-playing AI developing strategies through self-play, or a self-driving car making driving decisions in real-time. These problems share a common challenge: the agent must make a sequence of decisions where each choice affects future possibilities and rewards. Unlike static prediction tasks where we learn a one-time mapping from inputs to outputs, these problems require reasoning about the consequences of actions over time.

Read more

MIT 6.390 Intro to Machine Learning: 12 Reinforcement Learning (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 12: Reinforcement Learning of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

Reinforcement learning (RL) is a type of machine learning where an agent learns to make decisions by interacting with an environment. Unlike other learning paradigms, RL has several distinctive characteristics:

Read more

MIT 6.390 Intro to Machine Learning: Appendix A Matrix Calculus (Repost)

  6.390 - Intro to Machine Learning

Reposted from Appendix A: Matrix Calculus of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

A.1 Single-variable Calculus

We all know how to calculate a derivative in single-variable calculus. If we have an input variable \(x \in \mathbb{R}\) and a function \(f(x): \mathbb{R} \rightarrow \mathbb{R}\), then:

Read more

MIT 6.390 Intro to Machine Learning: Appendix B Optimizing Neural Networks (Repost)

  6.390 - Intro to Machine Learning

Reposted from Appendix B: Optimizing Neural Networks of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

B.1 Momentum and adaptive learning rates

B.1.1 Running averages

We’ll start by looking at the notion of a running average. It’s a computational strategy for estimating a possibly weighted average of a sequence of data. Let our data sequence be \(c_1, c_2, \ldots\); then we define a sequence of running average values, \(C_0, C_1, C_2, \ldots\) using the equations

Read more

MIT 6.390 Intro to Machine Learning: Appendix C Supervised learning in a nutshell (Repost)

  6.390 - Intro to Machine Learning

Reposted from Appendix C: Supervised Learning in a Nutshell of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

In which we try to describe the outlines of the “lifecycle” of supervised learning, including hyperparameter tuning and evaluation of the final product.

Read more