bendtorch-fast

Releases (6)

• 1.3.0.0 indexed
• 1.2.0.0 unindexed
• 1.1.1.0 unindexed
• 1.1.0.0 unindexed
• 1.0.1.0 unindexed
• 1.0.0.0 unindexed

Definitions (66)

main.bend

• def ew_d2log2 of the blocks for cnt numbers: one block below 131072 (MNIST's and the tiny Llama's matrices; at 100352 the blocks cost MNIST more than they gave on 16 threads), else one per 16384, at most 64
• def ew_depth
• def p2_b2
• def sgd_leaf
• def mom_b
• def mom_leaf
• def q4_b
• def opt_leaf
• def sgd_fin
• def mom_fin
• def opt_fin
• type PE
• def ph_cntthe elements of a block of cnt words from element e0, the last block stopping at n
• def pq_b
• def pqv_b
• def ph_go
• def ph_leaf
• def phv_go
• def phv_leaf
• def ph_fin
• def phv_fin
• def ph_nwthe words of r·c elements, ceil(r·c / 2)
• def ph_leafs
• def ph_mat
• def opt_matSGD splits two arrays, momentum three, Adam four.
• def mat_step
• def lin_step2
• def lin_step
• def ew1_leaf
• def map
• def ew2_leaf
• def zip_fin
• def zipa[i] = f(a[i], b[i]); b comes back unchanged
• def a2_b2
• def copy_leafnumbers i0 ..
• def dup_fin
• def Mat.duptwo copies of a matrix (T.Mat.dup)
• def mpair_a
• def Mat.sum2a + b, dropping b (T.Mat.sum2)
• def lin_fwd
• def lin_b2
• def lin_bwd
• def LinearF
• def linf_bwd
• def LinearFirstF
• def lnb_b2
• def LinearNBF
• def acti_f3
• def ActInT.ActIn: f on a copy, the input saved
• def SiLU
• def add2_f
• def Add2
• def mul2_f2
• def mul2_f
• def mul2_b2
• def Mul2
• def ResidualT.Residual with these copies and sums (T.ResidualOf, the composition the law residual is about)
• def RMSNormmain.bend's RMSNorm, RoPE, Attention, MSE and CrossEntropy under their own names (Embedding below, with its step in blocks), so that a program that takes its layers from here can take device versions of the same names from another file by its import line alone (they run as in main.bend)
• def RoPE
• def Attention
• def MSE
• def Embeddingmain.bend's Embedding with its optimizer step in blocks (mat_step): the table is SmolLM2's largest matrix (49152 x 576, tied to the output head), and main.bend's step updates it on one thread
• def CrossEntropy
• def Linear
• def LinearFirst
• def LinearNB