Hey, Great article. It was effortless to understand. But I have some suggested changes:
The error message I received running your code:
The error message "UnboundLocalError: local variable 'epoch_loss' referenced before assignment" arises in the stochastic_gradient_descent function. This occurs because you're trying to increment epoch_loss with epoch_loss += compute_loss(yi, predictions) before it has been assigned an initial value. In Python, you need to define a variable before you can use it in an operation that modifies its value.
Suggested changes:
def stochastic_gradient_descent(X, y, epochs, lr=0.01):
m, n = X.shape
X = np.c_[np.ones((m, 1)), X] # Add bias term
theta = np.random.randn(n + 1, 1)
losses = []
for epoch in range(epochs):
epoch_loss = 0 # Initialize epoch_loss to 0 at the start of each epoch
Hey, Great article. It was effortless to understand. But I have some suggested changes:
The error message I received running your code:
The error message "UnboundLocalError: local variable 'epoch_loss' referenced before assignment" arises in the stochastic_gradient_descent function. This occurs because you're trying to increment epoch_loss with epoch_loss += compute_loss(yi, predictions) before it has been assigned an initial value. In Python, you need to define a variable before you can use it in an operation that modifies its value.
Suggested changes:
def stochastic_gradient_descent(X, y, epochs, lr=0.01):
m, n = X.shape
X = np.c_[np.ones((m, 1)), X] # Add bias term
theta = np.random.randn(n + 1, 1)
losses = []
for epoch in range(epochs):
epoch_loss = 0 # Initialize epoch_loss to 0 at the start of each epoch
random_index = np.random.randint(m)
xi = X[random_index:random_index + 1]
yi = y[random_index:random_index + 1]
predictions = xi.dot(theta)
errors = predictions - yi
gradients = 2 * xi.T.dot(errors)
theta -= lr * gradients
epoch_loss += compute_loss(yi, predictions) # Now you can increment epoch_loss
losses.append(epoch_loss)
return theta, losses