bendtorch-gpu

Search uses release 1.3.0.0 · 324 definitions

Releases (6)

1.3.0.0 indexed
1.2.0.0 unindexed
1.1.1.0 unindexed
1.1.0.0 unindexed
1.0.1.0 unindexed
1.0.0.0 unindexed

Definitions (324)

main.bend

def lf_pv_f
def af3_f
def af2_f
def af1_f
def af0_f
def attn_f_f
def b1_ij_fbackward 1, dense: the score cell as forward 1, and G's dP = dO's row · v's row, at the dense offsets
def b1_cell_f
def b1_loop_f
def lf_b1_fx1 = q, x2 = kᵀ, x3 = vᵀ, x4 = dO; outputs P and G-dP (made here, written)
def gr_2_fbackward 2, dense: row r's softmax over all t, then G's row: s = dP·P over all t and G = (P * (dP - s)) * sc.
def gr_loop_f
def lf_gr_f
def ddotacc + a's cells ia, ia + sa, ..
def dsum_hacc + the group's heads' sums, in head order (main.bend's pacc): a's column j from row 0 (rows t32 apart) · b's column (rows w apart), all tn rows; the dense form of tsum_h and tdot
def kv_val_fthe cell e = row·kvw + col of dK (a = G, b = Q) or of dV (a = P, b = dO), dense: key j = row % t, over the group's g query heads, each the head's column j of a from row 0 (t apart a row) · the head's column of b (w apart a row)
def b3_sel_fbackward 3, dense: cells [0, nw) are dQ's, [nw, nw + nk) dK's, the next nk dV's (S8 as the causal lane's)
def b3_cell_f
def b3_loop_f
def lf_b3_fx1 = q, x2 = k, x3 = dO, x4 = G; P read from the first output (unchanged), dQ, dK and dV (over V) written
def ab3_fbackward 3's bang: dense cells, dK over kᵀ and dV over V (neither read any more)
def ab2_fbackward 2's bang: the softmax over all t, then G, in place
def ab1_fbackward 1's bang: dense P and G-dP (made here), kᵀ and vᵀ read
def attn_b_fx1 = q, x2 = k, x3 = v, x4 = dO; kᵀ and vᵀ made here, then dQ, dK and dV
def Attentioncausal attention with grouped heads over b sequences of t positions (main.bend's Attention)
def FullAttentionthe same layer with every key visible (main.bend's FullAttention on the device); Attention is untouched, so the causal lane's bits stay as they are
type EMS
type MS
def ms_add
def ms_then
def ms_cell
def ms_blkthe squares of the block of 2^p cells from i summed as a balanced tree of pairs (h = 2^(p-1)), gradients written
def ms_put
def ms_if
def lf_msex1 = the target; y written in place with the gradient, ps[t] = leaf t's block sum (one leaf a block)
def ms_pthe largest p <= k with 2^p dividing m (m > 0)
def mse_fin
def mse_go
def MSEmean squared error over every number (main.bend's MSE, the same bits)
def u2f
def f2u
type IA
def ia_go
def ids_arr
type EEM
type ES3
def ef_3
def ef_2
def ef_cello[i, j] = e[ids[i], j]
def ef_loop
def es3_o
def ef_if
def lf_embfx1 = the table, x2 = the ids; o (made on the device, zeros) gets the rows
def eb_4
def eb_3
def eb_2
def eb_cellg[ids[i], j] = g[ids[i], j] + dy[i, j]
def eb_loop
def eb_if
def lf_embbx1 = dy, x2 = the ids; the gradient g, in place
def emb_f3
def emb_f2
def emb_f
def emb_b3
def emb_b2
def emb_bwd
def Embeddinga table of v rows of d numbers, looked up by n ids (main.bend's Embedding, the same bits); its step on the device
type TS
def ts_leaf
def ts_plus
def ts_then
def tnodetree_sum of the len cells from lo, read in place: a node l passes above the cells sums its left child (the first h = 2^(l-1) cells) and its right child (the rest) when there is a rest (big = h < len), and is its left child otherwise (tree_sum's odd last cell carried up a pass)
def ts_lvthe passes tree_sum makes over m cells: the smallest l with 2^l >= m (f is fuel)
type LSthe host's part, row by row in order: loss - ((x[y] - m) - log(s)), or main.bend's clamped term for a label past the last class
def ce_t2
def ce_g4
type EC11.
def c1_3
def c1_2
def c1_1
def c1_if
def lf_c1
def tsx_leaf2.
def tsx
def x8_7the same over 8 cells in straight-line code (no closure a node): e0..e7 written, then ((e0 + e1) + (e2 + e3)) + ((e4 + e5) + (e6 + e7)), the balanced tree's three levels
def x8_6
def x8_5
def x8_4
def x8_3
def x8_2
def x8_1
def x8_0
def tsx8tsx over 2^(l+3) cells, its last three levels in blocks of 8
type EC2a leaf takes the blocks b, b + l, ..
type C3S
def c2_put
def c2_m
def c2_cell
def c2_loop
def c2_o