Bendtorch.fu3: def of bendtorch@1.3.0.0 Import: import bendtorch@1.3.0.0/main.bend as Bendtorch Lines 2742-2748 of bendtorch@1.3.0.0/main.bend: https://jend.leeeet.dev/read.txt?path=bendtorch@1.3.0.0/main.bend&start_line=2742&end_line=2748 import Base # a unit of the parallel work is one sequence's key/value group j: its g query heads' columns of Q (t x g·hd), head j's # columns of K and V (t x hd), and an output of its own (t x g·hd). Inside a unit the heads keep their order and their # arithmetic (head_cols with j = 0: the law kv_cols), so the units give the bits of the whole sequence computed at once. def fu3(+t: Nat, +gw: Nat, qq: Array, kk: Array, xs: List<&1, FS>, q: Array, k: Array, r: A2) -> FL: match r: case A2{vv, v}: FL{qq, kk, vv, FS{q, k, v, zeros_a(Nat.mul(t, gw))} <> xs} Uses (9): https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=Array https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=F32 https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=List https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=Nat https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=Nat.mul https://jend.leeeet.dev/definition.txt?path=bendtorch@1.3.0.0/main.bend&name=A2 https://jend.leeeet.dev/definition.txt?path=bendtorch@1.3.0.0/main.bend&name=FL https://jend.leeeet.dev/definition.txt?path=bendtorch@1.3.0.0/main.bend&name=FS https://jend.leeeet.dev/definition.txt?path=bendtorch@1.3.0.0/main.bend&name=zeros_a