bendtorch-core

Search uses release 0.1.0.0 · 5,204 definitions

Releases (1)

0.1.0.0 indexed

Definitions (5,204)

src/autograd/tensor_backward_PROOF.bend

def other_operand
def alone_spared
def right_spared
def first_spared
def right_alone_spared
def frozen_nt_spared
def and_true
def pair_bounded
def operands_bounded
def truth
def false_true
def sum_result
def sum_contractA successful broadcast_to hands its output to the Sum VJP's one contribution unchanged.
def restored_contractrestored only puts g back in its own shape: the contributions are the pass's.
def sum_dims_decided

src/autograd/tensor_chain_LAWS.bend

def Slots
def LogContributions sent so far, newest first.
def TakenThe gathered slot of one id, and the contributions left for the others.
def settled
def poured
def sorted
def takeTakes out of a newest-first log every contribution to id and accumulates them into slot, oldest first; the others stay in the log, in order.
def joined
def gather_step
def gatherEach slot of a newest-first list of count slots, the one with k older slots for id k, with what the log sends to it.
def firedAn entry that holds g after gathering: its VJP's contributions join the log in operand order, so the first operand's arrives first.
def reached
def chainedThe working gradients by the chain rule, pulled, for entries newest first that start with slots and receive the contributions in log: the entry with id k gathers every contribution sent to k into its starting slot; if it then holds g, the contributions of vjp(node, saved, g) join the log.
def resultThe working gradients of a sweep, without its entries.
law tensor_sweep_chain_ruleThe chain rule, for any recorded tape and any starting working gradients: when the sweep succeeds, the working gradient it leaves at every entry is the chained one, each slot the accumulation, in the order above, of the VJP values its consumers sent it at their own chained gradients.
def seedsThe starting working gradients of a backward from the entry with older entries below it and newer entries above it: no gradient for the newer entries, ones of the output's shape at the output, as PyTorch's backward(torch.ones_like(output)), and no gradient below.
law tensor_backward_chain_ruleThe chain rule for the whole backward: when the pass of a backward from the output with older entries below it succeeds, the working gradients it hands to the merge (tensor_backward_success, tensor_merge_accumulates) are the chained ones from the seeds, for any tape, newer entries included.

src/autograd/tensor_chain_PROOF.bend

type CellWhat the pull side holds for a push-side list of pending slots: each pending slot x, the one with k older slots, is what take gathers for id k from the log into the starting slot w, and the rest of the log serves the older slots.
def Pulls
def gatheredWith no entry left, the pull side gathers exactly the pending slots.
type StartedThe starting slots, handed back, with the pull side of an empty log.
def restarted
def started
def Pushed
type BumpedOne contribution bumped into the push side, the new state handed back.
type Hit
def hit
def hit_bumped
def took_hit
def took_miss
def missed
def bump_pullsBumping a contribution into the push side keeps the pull side in step, with the contribution at the head of the log.
type DeliveredA whole list of contributions delivered into any push-side state.
def relay
def rebump
def delivered
type StopWhat a sweep from a push-side state does, with the pull side's answer when it succeeds: done and gradients are the sweep's own outputs.
type Run
def Out
type FiredThe same for the entry whose VJP pass is known.
def fired_stop
def fired_go
def fail_fired
def done_fired
def held_step
def unset_step
def fired_run
def slot_step
def stepped
def swept_pullsThe sweep from any push-side state, in step with the pull side.
def is_done
def done_fail
def conclude
def started_pulls
def eq_reflId arithmetic for the entries newer than the output, as in tensor_backward_PROOF.bend, kept here so this file checks on its own.
def add_zero
def add_succ
def older_neA node with older nodes below it and extra + 1 newer ones above it never has the id of the target.
type PassedWhat a backward's pass does, with the chained answer when it succeeds.
def seeded_out
def seeded_run
def newer_step
def newer_chain
def passed_chain

src/autograd/tensor_vjp_LAWS.bend

def cellsThe words of a pointwise product in index order: word k is scalar(operation, left[left_map(plan, k)], right[right_map(plan, k)]), the value the pointwise family's broadcast instance writes at k (`binary_LAWS.bend` in the tensor contracts).
def reducedWhat sum_to_size(contribution, target) holds at target cell index, for a contribution of `shape` whose words are `values`: the word itself when the target has the contribution's dimensions, the ordered fiber sum otherwise.
def ProductWithWhat the transport form of tensor_product_replayed asks of P: P holds for both inputs, replayed with any histories that keep their words, next to the output replayed over +0.0 with any history whose words are `cells`.
law tensor_product_replayedThe pointwise product the Sub and Mul VJPs compute, `Elementwise.binary` with its parallel build, writes the words of the pointwise family's broadcast instance, in transport form: for two replayed inputs whose shapes broadcast, for every P, if P holds for every result that hands back both inputs with their words and gives an output of the plan's shape whose words are `cells`, P holds for the product.
def first_ofThe two contributions of a VJP, when it sends exactly two.
def second_of
type PairContractWhat a VJP that sends two contributions promises: the first goes to node a with left's dimensions and holds left_cell(i) at every in-bounds i, the second goes to node b with right's dimensions and holds right_cell(i).
law tensor_add_vjp_valuesd(a+b)/da and d(a+b)/db: for a replayed gradient g, and operand shapes that each broadcast to exactly g's shape, the Add VJP sends g reduced to the left operand's shape to a, then g reduced to the right operand's shape to b.
law tensor_sub_vjp_valuesd(a-b)/da and d(a-b)/db: the Sub VJP sends g reduced to the left operand's shape to a, as Add does, then -g reduced to the right operand's shape to b.
law tensor_mul_vjp_valuesd(a*b)/da and d(a*b)/db: with the saved operand values x (left) and y (right), the Mul VJP sends g * y reduced to the left operand's shape to a, then g * x reduced to the right operand's shape to b.
def swappedThe axes transpose_dynamic(t, 0, 1) permutes t by, and the shape of the copy the plan gives.
def transposed
def right_transposed_factorsThe first `count` factor pairs of cell (row, column) of left @ right^T, where right^T is right permuted by `axes`: left[row * inner + k] and the word of right at permute_source(right, axes, k * columns + column), in increasing k.
def left_transposed_factorsThe first `count` factor pairs of cell (row, column) of left^T @ right, where left^T is left permuted by `axes`: the word of left at permute_source(left, axes, row * inner + k) and right[k * columns + column], in increasing k.
type MatmulPairContractWhat the Matmul VJP promises: the first contribution goes to node a with left's dimensions and holds left_cell(row, column) at row * left_columns + column for every row < left_rows and column < left_columns; the second goes to node b with right's dimensions and holds right_cell(row, column) likewise.
law tensor_matmul_vjp_valuesd(a@b)/da and d(a@b)/db: with the saved operand values x (left) and y (right), the Matmul VJP sends g @ y^T to a, then x^T @ g to b.
law tensor_matmul_nt_vjp_valuesd(x @ W^T)/dx and d(x @ W^T)/dW, the backward of Linear's product: with the saved operand values x (left, [batch, in]) and W (right, [out, in], PyTorch's layout) and the gradient g ([batch, out]), the MatmulNT VJP sends g @ W to a, then g^T @ x to b.
type MatmulSoleContractWhat a VJP that sends one contribution of a matrix promises: it goes to node b with target's dimensions and holds cell(row, column) at row * columns + column for every row < rows and column < columns.
law tensor_matmul_nt_frozen_vjp_valuesd(x @ W^T)/dW of a product whose input x is a constant: the VJP of Frozen{a, MatmulNT{a, b, left, right}}, for an input a that is not the weight b, sends one contribution, to b, with right's dimensions: g^T @ x, cell (j, l) the ordered dot product over i of g[i, j] * x[i, l] from +0.0, the words of the second contribution of tensor_matmul_nt_vjp_values.
law nt_reads_transposematmul_nt reads what matmul reads from the transposed copy: for a right operand of dimensions [columns, inner], the factor pairs of cell (row, column) of a product of two `Kept` operands (`matmul_nt`), left[row, k] and right[column, k], are the pairs `right_transposed_factors` reads through `permute_source` on right^T, the pairs `tensor_matmul_vjp_values` states for g @ y^T.
law tn_reads_transposematmul_tn likewise: for a left operand of dimensions [inner, rows], the pairs of a product of two `Swapped` operands (`matmul_tn`), left[k, row] and right[k, column], are the pairs `left_transposed_factors` reads through `permute_source` on left^T, the pairs `tensor_matmul_vjp_values` states for x^T @ g.