Training report — char_transformer_tiny
Generated by src/ct_train.py (seed 42, deterministic).
Config
| Setting | Value |
|---|---|
| vocab_size | 58 |
| block_size | 32 |
| d_model | 48 |
| d_ff | 96 |
| batch_size | 64 |
| steps | 400 |
| optimizer | Adam (ml_first_principles.optimizers), lr=0.003 |
| parameters | 22896 |
| uniform baseline \(\ln V\) | 4.0604 |
| wall time | 9.7 s |
Loss trace
| Step | Train loss |
|---|---|
| 1 | 4.0593 |
| 25 | 3.1559 |
| 50 | 3.0761 |
| 75 | 2.9589 |
| 100 | 2.8190 |
| 125 | 2.6628 |
| 150 | 2.6146 |
| 175 | 2.4926 |
| 200 | 2.4783 |
| 225 | 2.4401 |
| 250 | 2.4264 |
| 275 | 2.3115 |
| 300 | 2.3767 |
| 325 | 2.2828 |
| 350 | 2.2204 |
| 375 | 2.2065 |
| 400 | 2.2307 |
Loss falls from 4.0593 (near the uniform baseline 4.0604) to 2.2307.
Sample generations
Prompt: "Shall I ", 120 new characters each, generation seed 42.
greedy
Shall I ther the thare than than the thar the thar the thar the thar the thar the thar the thar the thar the thar the thar the t
temperature 0.8
Shall I thos wis methin pathe wor neastt in blemess ton sea toth paireng,
I an mon ferest thais ck shand akstante,
Fese fonen mo
temperature 1.0
Shall I or land bofse's wis mith fipe trar shisids aonnk wen wablyenst for as; dat cou fswe I brarlf tot bund suls or blurtoou b