Machine-Learning on Ty Scales

ISLP Notes: Linear Regression

Sat, 10 Oct 2026 00:00:28 -0700

Simple Linear Regression

It’s really convenient when all data falls on a straight line. For any input X, you get an exact output Y. Here’s a plot for Y = 2 + 3X

import numpy as np
import matplotlib.pyplot as plt

X = np.arange(1,11)
Y = 2 + 3 * X
plt.scatter(X,Y, color='gray', alpha=0.7, facecolors='none', edgecolors='black')

We are rarely so lucky. There is often some unaccountable variance we can’t explain. Here the variance is represented by the variable e

Finding the minimizers of RSS

Sun, 27 Sep 2026 20:20:26 -0700

In chapter 3 of ISLP The authors introduce the minimizers of the residual sum of squares with respect to $\hat{B_0}$ and $\hat{B_1}$ for simple linear regression.

The least squares approach chooses $\hat{B_0}$ and $\hat{B_1}$ to minimize the RSS. Using some calculus, one can show that the minimizers are

$\hat{B_1} = \frac{\sum_{i=1}^{n} (x_i - \bar{x})(y_i - \bar{y})}{\sum_{i=1}^{n} (x_i - \bar{x})^2}$

$\hat{B_0} = \bar{y} - \hat{B_1}\bar{x}$

I’m going to walk through how these functions are derived from the RSS.

ISLP Notes - Chapter 2

Tue, 07 Jul 2026 19:29:08 -0700

The essence of statistical learning is crafting functions that predict or classify things about the world based on a set of inputs.

How do we craft these functions? You start with one of two options.

Make an assumption on the form of your function. Try assuming its a straight line, and then use a procedure to calculate the best slope for that line.
Make no assumptions of the data. Algorithms like K nearest neighbors just observe the data nearby to make their predictions.