Bendtorch.CSM: type of bendtorch@1.3.0.0 Import: import bendtorch@1.3.0.0/main.bend as Bendtorch Lines 3164-3169 of bendtorch@1.3.0.0/main.bend: https://jend.leeeet.dev/read.txt?path=bendtorch@1.3.0.0/main.bend&start_line=3164&end_line=3169 import Base # Cross-entropy's softmax over a row of c classes: as psm, but the row's exponentials are summed as a balanced tree # (tree_sum, on a copy), since a running F32 sum of SmolLM2's 49152 classes was 3.4 tolerances off PyTorch's loss # (NOTES, M10). psm itself keeps its order: attention's rows are short and the law causal reads its count. # a row's probabilities, its max m and log of its sum of exp(x - m) (log_softmax's pieces) type CSM is Type: CSM{a: Array, m: F32, ls: F32} Uses (2): https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=Array https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=F32