bendtorch-core

Search uses release 0.1.0.0 · 5,204 definitions

Releases (1)

0.1.0.0 indexed

Definitions (5,204)

src/autograd/tensor_backward.bend

def matmul_ntRecords a @ b^T, with b read in place: PyTorch's linear(a, b) without a bias.
def poppedThe tape without its newest entry.
def unrecordedA refused sum drops the product, the newest entry, so the tape comes back as it was before the product was recorded.
def biased
def linear_checked
def linear_counted
def linearLinear: records x @ W^T with W of [out, in], PyTorch's layout, then adds the bias b of [out] to every row.
def summed
def sum_shaped
def sum_found
def sumRecords the sum of every element of a, a rank-0 tensor.
def dims_error
def dims_summed
def dims_planned
def dims_shapedThe signed axes, each wrapped among the operand's axes (its rank, or 1 at rank 0), as PyTorch wraps them, before the plan checks them.
def dims_found
def sum_dimsRecords PyTorch's a.sum(dim=axes, keepdim=keepdim), SumDims.sum_dims of a copy of a's value: the kept shape when keepdim is True, else the dropped one.
def slice_kept
def slice_planned
def slice_taken
type SlicingWhich of the three a slice step is: PyTorch's a[..., start:stop:step, ...] at axis `dim`, a.narrow(dim, start, length) and a.select(dim, index).
def slice_extent
def slice_axis_plan
def slice_dim
def slice_planThe slice each one records: its plan, at the axis and positions the value takes (Signed.slice, narrow and select), and the value it computes.
def slice_value
def slice_shaped
def slice_found
def slicedRecords a slice of a copy of a's value (Signed.slice, narrow or select) as a Slice node with a's shape, the slice's shape with the axis kept and its plan.
def sliceRecords a[..., start:stop:step, ...] with the slice at axis `dim`.
def narrowRecords a.narrow(dim, start, length).
def selectRecords a.select(dim, index), which drops the axis.
def gather_kept
type IndexingWhich of the two an indexing step is: PyTorch's a.index_select(dim, index) with a vector of entries, and a.gather(dim, index) with an index tensor.
def indexing_plan
def indexing_value
def indexing_entries
def gather_planned
def gather_taken
def gather_shaped
def gather_found
def indexedRecords an index_select or a gather of a copy of a's value as a Gathered node with a's shape, the plan and the entries.
def index_selectRecords a.index_select(dim, index).
def gatherRecords a.gather(dim, index).
def relu_zeroed
def relu_shaped
def relu_found
def reluRecords relu(a), PyTorch's relu, of a's shape: the pointwise product of a copy of a's value with ReluLeft against a rank-0 +0.0, on the parallel elementwise build.
def mse_lossRecords mse_loss(a, b), PyTorch's mse_loss(input, target) with mean reduction, a rank-0 tensor, from copies of the operands' values.
def entropy_shaped
def entropy_found
def cross_entropyRecords cross_entropy(a, targets), PyTorch's cross_entropy with class indices and mean reduction, a rank-0 tensor, from a copy of a's value: a must be [rows, columns] logits and `targets` one class below `columns` per row.
def error_code

src/autograd/tensor_backward_LAWS.bend

def nodes_ofThe nodes of a list of entries, newest first.
def slots_of
def tape_nodes
def tape_slots
def targetsOperand bounds, stated without the engine's validator: a node may name only nodes older than itself, so each operand id must be below the number of older nodes.
def bounded
def well_formed
def relistedThe slots newer than a target, put back in front of the result.
def newer_unsetThe entries newer than the output, each with no gradient, in front of the sweep's result.
def soleThe only contribution of a VJP, when it sends exactly one.
def sole_target
def sole_dimensions
def sole_point
type SoleContractWhat a VJP that sends one contribution promises: it goes to node a, it has the target's dimensions, and every in-bounds index i holds cell(i).
law negate_left_scalarNegateLeft flips the sign bit of the left word, as PyTorch's neg does, and ignores the right word.
law tensor_vjp_leafA leaf sends nothing, and hands back its saved values and its gradient.
law tensor_vjp_constantA constant sends nothing, and hands back its saved values and its gradient.
law tensor_vjp_frozen_unsentFrozen{id, node}, the node of a recording whose operand id is a constant, sends node's contributions without the ones to id, in order, and hands back what node's VJP hands back.
def reachesWhether a list of contributions holds one to id.
def sends_toWhether a VJP's pass sends a contribution to id.
law tensor_vjp_frozen_sparesA frozen operand gets nothing: for any node, saved values and g, no contribution of the VJP of Frozen{id, node} goes to id, skipped product included.
law tensor_backward_unknownbackward of an output the tape does not hold is refused with UnknownNode, and the tape comes back unchanged.
law tensor_backward_danglingA tape with a node that refers to itself, to a newer node or to a missing node is refused with DanglingOperand, and comes back unchanged.
law tensor_backward_successOtherwise backward runs pass from the selected output and accumulates its working gradients into the slots.
law tensor_wired_matches_operand_boundsThe engine's wiring check is exactly the operand-bound specification above.
law tensor_seed_selected_outputThe seed is the selected output (C1): for any entries newer than it, those entries get no gradient, and the sweep starts at the output with ones of the output's shape, Core.new(shape, 1.0), and no gradient for the older entries.
law tensor_seeded_sweepThe sweep starts from the seed: the output gets ones of its shape and every older entry starts with no gradient.
law tensor_unreached_sends_nothingAn entry that received no gradient sends nothing: the older entries are swept with their working gradients unchanged.
law tensor_deliver_in_orderThe contributions of one VJP are delivered in list order, the first operand before the second.
law tensor_bump_targetA contribution to node target reaches exactly the slot of that node, whose Gradient.accumulate adds it to the held total (or moves it in); every other slot is kept, whatever the list holds.
law tensor_merge_accumulatesAfter the sweep, an entry that received a gradient g accumulates it into its slot with Gradient.accumulate, so a repeated backward adds; an entry that received nothing keeps its slot.
law tensor_merge_keeps_unreached
law tensor_zero_grad_emptieszero_grad empties every slot, like PyTorch's zero_grad(set_to_none=True), and keeps every node.
law tensor_zero_grad_keeps_nodes
law tensor_sum_vjp_pointwised(sum a)/da through the live broadcast_to_pointwise law: for a replayed gradient g whose shape broadcasts to exactly a's shape, the Sum VJP sends one contribution, to a, with a's dimensions, and every position i holds the gradient word at left_map(plan, i), its right-aligned broadcast projection; for the rank-0 g of a sum, that is g's only word at every position.
law tensor_sum_dims_vjp_pointwised(sum of a over some axes)/da: a gradient g of `shape`, the kept or the dropped shape, with as many elements as the kept shape, is read with the kept shape and broadcast as the Sum VJP broadcasts: for a replayed g whose words are a replay over the kept shape's count, and a kept shape that broadcasts to exactly a's shape, the VJP sends one contribution, to a, with a's dimensions, and every position i holds the word of g at left_map(plan, i), the kept shape's right-aligned projection of i.
law tensor_sum_dims_wrapsThe tape's sum over axes takes PyTorch's signed axes: they are wrapped among the operand's axes, its rank or 1 at rank 0 (SumDims.limit), before the plan checks them.

src/autograd/tensor_backward_PROOF.bend

def eq_refl
def add_zero
def add_succ
def older_neA node with older nodes below it and extra + 1 newer ones above it never has the id of the target.
def listed_back
def frozen_nt_unsent
def spared_onekept_others leaves no contribution to id.
def spared
def unsent_spared_result
def unsent_spared