bendtorch

Releases (6)

• 1.3.0.0 indexed
• 1.2.0.0 unindexed
• 1.1.1.0 unindexed
• 1.1.0.0 unindexed
• 1.0.1.0 unindexed
• 1.0.0.0 unindexed

Definitions (1,049)

spec.bend

• def dotnthe dot product of the common prefix
• def vaddvector sum; the shorter vector counts as padded with zeros
• def vscale
• def mvM·v: one dot product per row (a linear layer's forward pass, y = W·x)
• def mtvMᵀ·g: the rows weighted by g and summed (a linear layer's backward pass to its input, dx = Wᵀ·dy)
• def outerg ⊗ v: row i is g[i]·v (a linear layer's weight gradient, dW = dy ⊗ x)
• def frobthe sum of the products of matching entries of two matrices (Frobenius inner product)
• def lea <= b, by recursion (simpler to reason about than Nat.is_le, which goes through Nat.cmp)
• def maddrow-wise sum of two matrices (the shorter counts as padded)
• def lin_fwdy = W·x, saving x
• def lin_bwddx = Wᵀ·dy, and dW += dy ⊗ x
• def LinL
• def SeqLtwo linear layers in sequence, and side by side on one input, through BendTorch's combinators
• def dupv
• def ForkL
• def BothLtwo linear layers on the two vectors of a pair, through T.Both
• def add_fwda + b for a pair of vectors; the backward pass hands dy to both (T.Add2 over vectors)
• def AddL
• def ResLx + W·x through T.ResidualOf, the code of T.Residual: Seq(Fork(Id, linear), AddL)
• def fy
• def fsv
• def bdx
• def pdotthe dot product of two pairs of vectors
• type Sum
• def pw2^p
• def balthe 2^p cells from cell i summed as a balanced tree of pairs: how a GPU leaf sums its block (the device's MSE and cross-entropy block sums)
• def blocksk blocks of 2^p cells, the first from cell lo, each summed by bal; with p = 0, a row of k cells
• def spassone pass of main.bend's tree_sum (ts_pass): neighbours added in pairs, an odd last cell carried up as it is
• def passesr passes of tree_sum; tree_sum (ts_go) stops once one cell is left, and a pass leaves one cell as it is, so the sum of m cells is passes(r, row) for any r at least the number of passes it makes

tensor.bend

• type Tensor
• def numelthe number of elements of a tensor of shape s
• def at1size of axis k (1 past the last axis, so a missing axis counts as a size-1 one)
• def removes without axis k (s itself past the last axis)
• def ones_thenn copies of 1, then s: s seen with n more leading axes
• def pads aligned to rank r by leading size-1 axes, as broadcasting aligns shapes from the right
• def bc_pairone axis: z is x, and y is x or 1
• def bc_dimone axis, either way round
• def bc_eqthree shapes of one rank, axis by axis
• def bc_rankthe ranks: o's is the larger of a's and b's
• def bc_oka and b broadcast to o
• def Bool.checka proof that b is True, when it is: where shapes read at run time enter, a program checks a condition once
• def nat_eq_succ
• def Nat.check_eqa proof that a == b, when it holds (reshape's count, read at run time)
• type Axone axis of an index map: its size, and the stride of the tensor read along it (0 where it is broadcast)
• type IMap
• def stride_of
• def axesthe axes of reading a tensor of shape a (aligned to o, same rank) at every position of o; the last axis first
• def bmapreading a (any rank up to o's) along o
• def strided
• def imapthe position m(j)
• type P2
• type P3
• def gather2_lo[j] = f(x[m(j)], y[n(j)]) for k positions from j; rx and ry hold x[m(j)] and y[n(j)], already read
• def gather_lo[j] = x[m(j)] for k positions from j
• def scatter_lg[m(j)] = g[m(j)] + x[j] for k positions from j; rx holds x[j], rg holds g[m(j)]
• def new
• def zeros
• def of_mat
• def to_mat
• type T3a binary operation, both operands handed back with the result
• def t3_of
• def f32_add
• def f32_sub
• def f32_mul
• def f32_div
• def map2z = f(x, y) element by element, x and y broadcast to o
• def add
• type TPa tensor and the gradient that comes back with it
• def tp_of
• def sum_tog + (dz summed down to a's shape): the gradient of an operand that was broadcast from a to o; dz comes back
• def innerthe product of the sizes after axis k
• def drop_map
• def sum_axisx summed over axis k: shape remove(s, k); x comes back
• def sum_axis_bwdthe gradient of sum_axis: dy (shape remove(s, k)) copied along axis k; dy comes back
• def setds with the size of axis k replaced by v (s itself past the last axis)
• def rank
• def win_axesthe index map of reading a tensor of strides st (given by its shape st) at every position of a tensor of shape o, from a base: axis e has o's size and st's stride (inner(st, e)); the last axis first
• def window
• def place_lg[m(j)] = x[j] for k positions from j (a copy into a window of g; no addition, so -0 stays -0)
• def ewc_la[j] = f(a[j]) for k positions from j, with a constant c
• def ew_div
• def n_f32
• def grad2_lg[mg(j)] = g[mg(j)] + f(dz[j], x[ma(j)], y[mb(j)]) for k positions from j: the gradient of one operand of map2, with both operands at hand (mul and div need them)
• type P4
• type G2dz, x, y and the gradient, handed back
• def g2_of
• def grad2g + the gradient, through f, of the operand of shape gs (a or b) of z = op(x, y): f(dz, x, y) at each position of o
• def d_idthe partial derivatives, dz times d op / d operand (PyTorch's formulas and order of operations)
• def d_neg
• def d_mul_a
• def d_mul_b
• def d_div_a
• def d_div_b-dz * ((x / y) / y), as PyTorch's div backward for the divisor
• def sub
• def mul
• def div
• def sum_axis_keepthe sum over axis k with the axis kept (size 1): the same numbers as sum_axis, in the same layout
• def tp_div
• def mean_axisthe mean over axis k: the sum divided by the axis' size
• def mean_axis_bwdthe gradient of mean_axis: dy divided by the axis' size, copied along it