bendtorch-core

Releases (1)

• 0.1.0.0 indexed

Definitions (5,204)

src/autograd/tensor_backward.bend

• def mse_shaped
• def vjp_msed mse_loss(a, b)/da: (2 / N * d) * g, and d/db: (2 / N * e) * g, PyTorch's mse_loss_backward, with the differences e = right - left and d = left - right saved at record time over the broadcast shape of N cells; each is reduced to its operand's shape.
• def entropy_second
• def entropy_first
• def vjp_cross_entropyd cross_entropy(a)/da, as two contributions to a that its slot adds in order.
• def kept_one
• def kept_othersThe contributions to every node but id, in order.
• def without
• def unsentA VJP's pass without its contributions to id.
• def skipsWhether the VJP of Frozen{id, node} skips a product instead of dropping it: node is x @ W^T and id is x and not W.
• def nt_frozen
• def batched_reducedA contribution for operand target from a product over the output's batch: the product reduced to the operand's shape, as a broadcast operand of an elementwise operation is.
• def batched_fourth
• def batched_rightleft^T @ g for the right operand, from copies of g and of the saved left operand.
• def batched_third
• def batched_second
• def batched_leftg @ right^T for the left operand, from copies of g and of the saved right operand, so a failure loses neither.
• def batched_first
• def batched_planned
• def vjp_batchedd(a@b)/da and d(a@b)/db of torch.matmul: g @ b^T and a^T @ g over the output's batch (Batched.left_gradient and Batched.right_gradient), each then summed to its operand's shape with sum_to_size, which sums the batch dimensions the operand was broadcast along and keeps a product that already has the operand's shape.
• def scatter_sent
• def slice_unsqueezed
• def vjp_sliced(a slice of a)/da: g read with the kept shape, the slice's output with the axis kept (PyTorch's unsqueeze for select), then scattered into +0.0 words of a's shape (PyTorch's slice_backward): each input cell the slice read gets the gradient word of the output cell it went to.
• def gather_sent
• def vjp_gatherd(an index_select or a gather of a)/da: g, of the output's shape, added into +0.0 words of a's shape at the entries (PyTorch's index_add and scatter_add): each input cell gets the sum of the gradient words of the output cells that read it, in output order.
• def vjpThe vector-Jacobian product of one node with gradient g: the contribution each operand receives, in operand order, already reduced to its shape.
• def placed
• def bumpAccumulates a contribution into the slot of node target, in a newest-first list of count slots; same says whether the head is that node.
• def deliver
• def sent
• def Swept
• def finished
• def passed
• def visited
• def sweepVisits entries newest first with their working gradients.
• def seeded
• def seeded_entryThe seed: ones of the output's shape.
• def passThe working gradient of every entry, newest first: Unset for the entries newer than output and for every entry output does not reach, ones of the output's shape at output, and the sweep's totals below it.
• def operands_below
• def wired
• def merged_entryAccumulates each working gradient into its entry's slot.
• def merge
• def merged_tape
• def ran
• def checked
• def planned
• def backwardRuns one backward of output and accumulates its gradients into the slots.
• def cleared
• def zero_gradEmpties every gradient slot, like PyTorch's zero_grad(set_to_none=True).
• def Found
• def cloned_value
• def found_value
• def prepend_found
• def value_atA copy of the value of node id, in a newest-first list of count entries.
• def counted_entry
• def countedThe number of entries, next to the entries.
• def value_counted
• def found_tape
• def valueA copy of the forward value of node id.
• def Graded
• def held_copy
• def copied_grad
• def graded_entry
• def found_grad
• def prepend_graded
• def grad_at
• def grad_counted
• def graded_tape
• def gradA copy of the gradient slot of node id: None until a backward reaches it.
• def pushed
• def done_id
• def is_constant
• def constant_atWhether node id of a newest-first list of count nodes is a constant.
• def operand_idsThe operand ids of a node, in operand order.
• def frozen_if
• def freezenode wrapped in Frozen{id, .} for each operand id that is a constant of the newest-first list of count nodes, the first operand outermost.
• def all_constantWhether every id of a list is a constant of the newest-first list of count nodes.
• def settledA node whose operands are all constants is a constant itself, as in PyTorch an operation on tensors that do not require grad gives one that does not; any other node is frozen at each operand that is a constant.
• def marked
• def pushThe new entry's id is the number of entries before it.
• def leafRecords a leaf.
• def constantRecords a constant: a leaf that takes no gradient, PyTorch's tensor with requires_grad=False.
• def recorded
• def multiplied_record
• def dropped
• def kept
• def loss_recorded
• def loss_dropped
• def loss_kept
• def batched_recordtorch.matmul on the tape: two rank-2 operands record `Matmul`, the product `matmul` records, and every other pair records `BatchedMatmul`.
• def operated
• def computed
• def right_found
• def left_found
• def binaryRecords `a op b` from copies of the operands' values.
• def add
• def sub
• def mul
• def matmul
• def batched_matmulRecords torch.matmul(a, b): batched products and 1-D promotion (Batched.matmul).