RNN Basics
A simple RNN processes a sequence step by step. At each step, it takes an input and the previous hidden state, computes a new hidden state, and optionally produces an output.
Unrolled RNN
Unrolling means writing the loop as a chain of repeating modules. Each time step shares the same weights (W_h, W_x, b).
h_0 (initial) → (x_1, h_0) → h_1 → (x_2, h_1) → h_2 → ... → h_TRNN Cell Operations
At each time step t:
1. Combine input x_t and previous hidden state h_{t-1} (concatenate or add).
2. Apply linear transformation: W * [h_{t-1}, x_t] + b.
3. Apply activation function (usually tanh or ReLU).
4. Output hidden state h_t (and optionally y_t = softmax(W_out * h_t)).
Input and Output Modes
- One‑to‑one: standard network (not sequential).
- One‑to‑many: single input (e.g., image captioning).
- Many‑to‑one: sequence → single output (e.g., sentiment classification).
- Many‑to‑many: sequence → sequence (e.g., machine translation, video classification).
- Many‑to‑many (same length): e.g., part‑of‑speech tagging.
Two Minute Drill
- RNN shares weights across time steps.
- Hidden state passes information forward.
- Different input/output modes for various tasks.
- Unrolling shows the chain structure.
Need more clarification?
Drop us an email at career@quipoinfotech.com