Deep Learning · November 2024 · 1 min read

RNN Sequence Modeling

Recurrent networks from scratch — forward pass, backpropagation through time, and gradient flow analysis. Vectorized NumPy implementation validated to 5e-5 tolerance.

PythonNumPyPyTorchSequence Models

RNN forward and backward passes, hand-derived and vectorized. Gradient-checked implementation with 5e-5 max error. Sequence targets (x, y) predicted across 10 timesteps, validated across four sample batches.

implementationForward pass (all_h, last_h), BPTT, gradient check validationMax error 5e-5 across all components architecturesVanilla RNN, LSTM, GRU — compared head-to-head

Gradient Flow Analysis

The vanishing gradient problem in action. Vanilla RNN gradients collapse to 1e-7 at 50 timesteps — LSTM maintains 1e-1. Hidden state evolution shows how cell gates preserve long-range information that vanilla RNNs lose.

#rnn#lstm#sequence-modeling#backpropagation

Related projects

Deep Learning · 1 min read

Deep Learning from Scratch

Backpropagation, BatchNorm, Dropout, and CNNs implemented from first principles in NumPy — then PyTorch deployment achieving 74.8% on CIFAR-10.

PythonNumPyPyTorchDeep Learning

Deep Learning · 1 min read

Vision Transformer + Masked Autoencoder

ViT classifier achieving 73.5% on CIFAR-10, then self-supervised MAE pretraining boosts finetuned accuracy to 76.8%. Full implementation of patchify, attention pooling, and mask reconstruction.

PythonPyTorchVision TransformersSelf-Supervised Learning

Deep Learning · 1 min read

Transformer for News Summarization

Self-attention, multi-head attention, and encoder-decoder architecture implemented from scratch. Trained on CNN/DailyMail achieving 35.1 ROUGE-L, outperforming LSTM baseline by 60%.

PythonPyTorchTransformersNLP