law adjoint

bendtorch@1.3.0.0 · LAWS.bend:11

import Base
import bendtorch@1.3.0.0/spec.bend as Spec

11# LAW (adjoint): the backward pass of a linear layer, dx = Wᵀ·dy, is the adjoint of its forward pass y = W·x: for any
12# W, dy and x, dy·(W·x) == (Wᵀ·dy)·x. This is what makes dx the gradient of the loss with respect to x.
13law adjoint:
14  for w: List<&2, List<&2, Nat>>
15  for g: List<&2, Nat>
16  for +v: List<&2, Nat>
17  {Spec.dotn(g, Spec.mv(w, v)) == Spec.dotn(Spec.mtv(w, g), v) : Nat}