Cogito Capability Datasets: Long-Context Tests for Small Models
·2338 words·11 mins
Five open Hugging Face datasets that test whether small (5M–100M) models actually use facts from far back in their context: recall, binding, structure, multi-hop, counting and deduction, on a length ladder from 1k to 128k tokens.