MIT 6.390 Intro to Machine Learning: 1 Introduction (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 1: Introduction of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site; the course logos left out. This post is shared under the same license.

The main focus of machine learning (ML) is making decisions or predictions based on data. There are a number of other fields with significant overlap in technique, but difference in focus: in economics and psychology, the goal is to discover underlying causal processes and in statistics it is to find a model that fits a data set well. In those fields, the end product is a model. In machine learning, we often fit models, but as a means to the end of making good predictions or decisions.

Read more

MIT 6.390 Intro to Machine Learning: 2 Regression (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 2: Regression of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

Regression is an important machine-learning problem that provides a good starting point for diving deeply into the field.

Read more

MIT 6.390 Intro to Machine Learning: 3 Gradient Descent (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 3: Gradient Descent of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

In the previous chapter, we showed how to describe an interesting objective function for machine learning, but we need a way to find the optimal \(\Theta^* = {\rm arg}\min_{\Theta} J(\Theta)\), particularly when the objective function is not amenable to analytical optimization. For example, this can happen when \(J(\Theta)\) involves a more complex loss function or more general forms of regularization. It can also be the case when there are simply too many parameters to learn for it to be computationally feasible.

Read more

MIT 6.390 Intro to Machine Learning: 4 Classification (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 4: Classification of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site; the interactive demo linked instead of embedded. This post is shared under the same license.

4.1 Problem formulation

Classification is a machine learning problem seeking to map from inputs \(\mathbb{R}^d\) to outputs in an unordered set.

Read more

MIT 6.390 Intro to Machine Learning: 5 Feature Representation (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 5: Feature Representation of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

Linear regression and classification are powerful tools, but in the real world, data often exhibit non-linear behavior that cannot immediately be captured by the linear models which we have built so far. For example, suppose the true behavior of a system (with \(d=2\)) looks like this wavelet:

Read more

MIT 6.390 Intro to Machine Learning: 6 Neural Networks (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 6: Neural Networks of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

You’ve probably been hearing a lot about “neural networks.” Now that we have several useful machine-learning concepts (hypothesis classes, classification, regression, gradient descent, regularization, etc.), we are well equipped to understand neural networks in detail.

Read more

MIT 6.390 Intro to Machine Learning: 7 Convolutional Neural Networks (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 7: Convolutional Neural Networks of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site; a figure from MathWorks left out, since the course license doesn’t cover it. This post is shared under the same license.

So far, we have studied what are called fully connected neural networks, in which all of the units in one layer are connected to all of the units in the next layer. This is a good arrangement when we don’t know anything about the kind of mapping from inputs to outputs that we will be asking the network to approximate. However, if we do know something about our problem, it is better to build that knowledge into the structure of the neural network. Doing so can save computation time and significantly reduce the amount of training data required to arrive at a solution that generalizes robustly.

Read more

MIT 6.390 Intro to Machine Learning: 8 Representation Learning (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 8: Representation Learning of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

In previous chapters, we have largely focused on supervised learning, using training samples that have both features/inputs and corresponding labels/outputs, to learn models that predict labels for new data. A recurring theme, however, has been the importance of how data is represented. In Chapter 5, we saw that choosing the right feature transformation can make the difference between a model that works and one that doesn’t. In Chapter 7, we saw that convolutional neural networks learn hierarchical features automatically from data. This chapter steps back to ask a broader question: can we learn good representations of data directly, and why would we want to?

Read more

MIT 6.390 Intro to Machine Learning: 9 Transformers (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 9: Transformers of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site. This post is shared under the same license.

Transformers are a very recent family of architectures that were originally introduced in the field of natural language processing (NLP) in 2017, as an approach to process and understand human language. Since then, they have revolutionized not only NLP but also other domains such as image processing and multi-modal generative AI. Their scalability and parallelizability have made them the backbone of large-scale foundation models, such as GPT, BERT, and Vision Transformers (ViT), powering many state-of-the-art applications.

Read more

MIT 6.390 Intro to Machine Learning: 10 Non-parametric Models (Repost)

  6.390 - Intro to Machine Learning

Reposted from Chapter 10: Non-parametric Models of the MIT 6.390 Intro to Machine Learning course notes, © Massachusetts Institute of Technology and the 6.390 (formerly 6.036) course staff, licensed under CC BY-NC-SA 4.0. Changes made on 2026-11-20: converted from the course website to Markdown; side notes and boxed notes shown as quotes; links to other chapters point to the original site; a figure reproduced from Breiman et al. (1984) left out, since the course license doesn’t cover it. This post is shared under the same license.

Neural networks have adaptable complexity in the sense that we can try different architectures and use cross-validation to pick one that works well on our data. In this chapter we turn to models that adapt their complexity automatically to the training data: the name non-parametric is misleading: these methods do have parameters, but the size of the parameterization can grow as we acquire more data, rather than being fixed in advance.

Read more