bendtorch-gpu

Search uses release 1.3.0.0 · 324 definitions

Releases (6)

1.3.0.0 indexed
1.2.0.0 unindexed
1.1.1.0 unindexed
1.1.0.0 unindexed
1.0.1.0 unindexed
1.0.0.0 unindexed

Definitions (324)

main.bend

def gdw_put
type EGW
def gdw_if
def lf_gdwx1 = x, x2 = dy, x3 = the blocks' means and 1/sqrt(var + eps); the gain's and the bias's gradients, in place: a leaf a channel
def gn_b3
def gn_b2
def gn_b
def ln_step2the gain and the bias take this file's optimizer step, as Linear's two matrices do
def ln_step
def GroupNormGroupNorm over n images of hw pixels and ng groups of cg channels, channels last (main.bend's GroupNorm)
type ERO
def rope_loop
def rs3_x
def lf_ropex1 = cos, x2 = sin; x rotated in place
def rope_go2
def rope_go
def RoPERoPE on n = b·t rows of h heads of 2·half (main.bend's RoPE)
type S4
type EACthe sizes; n is how many cells (or rows) the bang has, nw and nk the cells of dQ and of dK, tr = t(t + 1)/2 the cells of a head's P
def toffwhere row i of head bh starts in P and G, which keep only the lower triangle (j <= i), row after row: main.bend's P and G are t·t with zeros above the diagonal, and a product by one of those zeros adds +0.0 or -0.0 to a sum that started at +0.0, which leaves it as it is; so the sums below skip them and give the same bits (for finite numbers: an infinity or a NaN in a masked key's row would reach main.bend's sums as 0·inf = NaN, and does not reach these)
def eac_n
type S3
def s3_c
def sc_fin
def pv_valthe cell e = row·w + col of o (a = P, b = V) or of dQ (a = G, b = K): row i of P (or G) for the query's head, its i + 1 positions, · V's (or K's) column of the head's key/value head
def tdotacc + a[ia]·b[ib] + ..
def tsum_hacc + the group's heads' sums, in head order (main.bend's pacc): a's column j (tr apart a head) · b's column (hd apart a head), rows j..t-1
def kv_valthe cell e = row·kvw + col of dK (a = G, b = Q) or of dV (a = P, b = dO): key j = row % t, over the group's g query heads: the head's column j of a (rows j..t-1) · the head's column of b
def tk_srcK and V transposed for the score cells (kᵀ: b·kv·hd·t, by sequence, key/value head, column c, position j), so that neighbouring lanes, which take neighbouring keys j, read neighbouring cells.
def tk_put
def tk_loop
def o2_b
def lf_tkx1 = k; kᵀ written (made here)
def tkv_kboth: cells [0, nk) are kᵀ's from k, the next nk vᵀ's from v (S4{k, v, kᵀ, vᵀ})
def tkv_v
def tkv_sel
def tkv_loop
def tkv_o
def lf_tkvx1 = k, x2 = v; outputs kᵀ and vᵀ (written), dQ (made here, written by backward 3)
type IJthe score cells are P's lower triangle (j <= i), dealt in pairs of rows i and t - 1 - i (t + 1 cells a pair), so every lane gets real work: cell e is pair p of head bh, place c; c <= p is (p, c), the rest (t - 1 - p, c - p - 1); in the middle row of an odd t the second half repeats the first and is skipped (ok false)
def ij_of
def sc_putforward 1: P's cell (i, j) = (q's row i · k's row j) * sc, as qrow's dot and pw's product (S3{q, kᵀ, p})
def sc_if
def sc_ij
def sc_cell
def sc_loop
def lf_scx1 = q, x2 = k; outputs P (written) and o (made here, written by forward 3)
def sm_loopforward 2: P's row r (query i = r % t) by main.bend's softmax over its i + 1 positions
def lf_sm
def put_cforward 3 (and backward 3's dQ): o[e] from pv_val (S3{a, b, o})
def pv_loop
def lf_pvx1 = P, x2 = v; o written in place
type S6backward 1: P's score cell as forward 1, and G's cell dP = dO's row · v's row, as drow's dot (S6{q, k, v, d, p, g})
def b1_g
def b1_p
def b1_if
def b1_ij
def b1_cell
def b1_loop
def b1_o
def lf_b1x1 = q, x2 = kᵀ, x3 = vᵀ, x4 = dO; outputs P and G (made here, written)
def gr_2backward 2: row r of P by the softmax, then G's row: s = dP·P over the i + 1 positions and G = (P * (dP - s)) * sc, main.bend's gr3 and g_set2
def gr_loop
def a2_o2
def lf_gr
type S8backward 3: cells [0, nw) are dQ's, [nw, nw + nk) dK's, the next nk dV's (S8{q, k, d, g, p, dq, dk, v})
def b3_q
def b3_k
def b3_v
def b3_sel
def b3_cell
def b3_loop
def b3_o
def lf_b3x1 = q, x2 = k, x3 = dO, x4 = G; P read from the first output (unchanged), dQ, dK and dV (over V) written
def eacthe sizes of a bang of cnt cells (or rows)
def pcellsP's cells (b·h·t(t + 1)/2), the score cells dealt in pairs of rows (b·h·ceil(t/2)·(t + 1)) and P's rows (b·h·t)
def tcells
def prows
def af3
def af2
def af1
def af0
def attn_f
def ab4
def ab3backward 3 writes dK over kᵀ and dV over v (neither read any more)
def ab2
def ab1
def attn_b
def toff_f
def pcf
def eacf
def ij_of_f
def sc_ij_f
def sc_cell_f
def sc_loop_f
def lf_sc_f
def sm_loop_fsoftmax over all t of row i, at the dense row start
def lf_sm_f
def pv_val_f
def pv_loop_f