bendtorch

Releases (6)

• 1.3.0.0 indexed
• 1.2.0.0 unindexed
• 1.1.1.0 unindexed
• 1.1.0.0 unindexed
• 1.0.1.0 unindexed
• 1.0.0.0 unindexed

Definitions (1,049)

main.bend

• def ph_f2u
• def ph_u2f
• def ph_pick
• def ph_of_bitsF32 -> BF16 bits (dtype.bend's BF16.of_f32): add 0x7FFF plus the lowest kept bit, keep the top half; NaN -> 0xFFFF
• def ph_of
• def ph_toBF16 bits -> F32, exact
• def ph_halfthe half of word w that element t uses (odd t: the high half), and w with that half replaced by h
• def ph_put
• type PQ
• type PQV
• def adamh_lm and v packed.
• def adamhv_lm packed, v in F32 (one cell an element, as adam_l's)
• def ph_wordsthe words of n packed values: ceil(n / 2), in a block of the next power of two
• def ph_fin
• def phv_fin
• def adamh_leafone matrix's step on its leaf; anything else leaves the values untouched, as Opt.mat's fallback
• def ph_newthe leaf of a matrix on its first step: zeros (BF16 0x0000 is +0.0)
• def adamh_mat
• def Opt.sgd_finOne matrix step, with only the buffers this optimizer reads.
• def Opt.mom_fin
• def Opt.fin
• def Opt.matA mismatched state is initialized only at the training boundary; Opt.mat requires the matching variant.
• type OptCfgWhat the user picks; `Opt.at` turns it into the step's `Opt` from q1 = 1 - b1^t and q2 = 1 - b2^t (t = 1 on the first step).
• def Opt.at
• def OptCfg.b1
• def OptCfg.b2
• def OptCfg.c1
• def OptCfg.c2
• def beta_okc = 1 - b up to F32's rounding, with 0 < c <= 1 (PyTorch's 0 <= beta < 1).
• def OptCfg.okTrue unless an Adam configuration's c1, c2 disagree with its b1, b2.
• def cfg_stop
• def OptCfg.check
• def mat_dump_findump and load for a matrix: its r*c numbers, row by row
• def Mat.dump
• def load_cells
• def mat_restore_fin
• def Mat.restore
• def store_cellsload_cells' inverse: copies the matrix's cells i..
• def mat_store_fin
• def Mat.store
• def pa_second
• def pl_pair
• def pl_second
• type Lin
• def lin_f3
• def lin_f2
• def lin_fwd
• def lin_b4
• def lin_b3
• def lin_b2
• def lin_bwddX = dY·W, dW += dYᵀ·X, db += the column sums of dY; the two sums are added straight into the gradient (the accumulating products of bend-ml-tensor-array 0.1.10.0), with the bits of computing them apart and adding them
• def pair_s3
• def lin_pair
• def lin_unpair
• def lin_s2
• def lin_step2
• def lin_step
• def lin_dump2
• def lin_dump
• def lin_load3
• def lin_load2
• def lin_load
• def lin_store2
• def lin_store
• def lin_zeros
• def Lineara linear layer from i to o features on batches of n rows; its products run in 2^par blocks of rows
• def linf_b3the first layer of a model, whose input (the data) needs no gradient, as an input without requires_grad in PyTorch: the backward pass computes dW and db only, skipping dX = dY·W (a third of a linear layer's work), and returns zeros as dX.
• def linf_bwd
• def LinearFirst
• def pick
• def no_step
• def act_f2an element-wise activation that saves a copy of its output (relu, sigmoid, tanh: their derivative is a function of the output).
• def act_b2
• def ActOut
• def relu_vrelu(x) = x unless x < 0 (so relu(NaN) = NaN and relu(-0) = -0); its gradient is +0 where the output is <= 0 and passes everywhere else, NaN included (PyTorch's threshold_backward)
• def relu_d
• def ReLU
• def seq_f3
• def seq_f2
• def seq_fwd
• def seq_b3
• def seq_b2
• def seq_bwd
• def seq_s3
• def seq_s2
• def seq_step2
• def seq_step
• def pair_zerosdump and load for a pair of layers: the first layer's numbers, then the second's a pair's zeros, the two halves in a parallel let: a model's gradient and optimizer state are many arrays, and making them one after another left the training state's zeroing (page faults, mostly) on one thread
• def pair_dump
• def pair_load2
• def pair_load
• def pair_store2
• def pair_store
• def Seq
• def seq_at_f2Owned layer composition for dimensions read at run time.
• def seq_at_f
• def seq_at_b2
• def seq_at_b
• def seq_at_step
• def seq_at_dump