bend-ml-tensor

Releases (3)

• 0.1.2.0 indexed
• 0.1.1.0 unindexed
• 0.1.0.0 unindexed

Definitions (78)

main.bend

• type Vec
• type Mat
• def zip_with.add
• def zip_with.sub
• def zip_with.mul
• def scale.l
• def dot.go
• def sum.go
• def relu.l
• def fill.l
• def Vec.zeros
• def Vec.fill
• def Vec.add
• def Vec.sub
• def Vec.mul
• def Vec.scale
• def Vec.dot
• def Vec.sum
• def Vec.relu
• def heads
• def tails
• def transpose.gotranspose of a matrix with `c` columns (c is the one that shrinks, so it comes first)
• def rows.zip_add
• def rows.zip_sub
• def rows.zip_mul
• def rows.scale
• def rows.relu
• def rows.col_sumscolumn sums
• def rows.add_row
• def matmul.row
• def matmul.rows
• def flatten.ljoins all the rows into a single list
• def chunk.lsplits a list into r rows of c numbers
• def Mat.zeros
• def Mat.fill
• def Mat.add
• def Mat.sub
• def Mat.mulelement-wise product (Hadamard)
• def Mat.scale
• def Mat.relu
• def Mat.transpose
• def Mat.matmul(n x k) · (k x m) = (n x m): the inner dimensions must be the same k
• def Mat.matmul_t(n x k) · (m x k)ᵀ = (n x m): the second operand arrives already TRANSPOSED (stored as m rows of k numbers).
• def Mat.add_rowadds a bias (a vector of m numbers) to each row of an n x m matrix
• def Mat.col_sumssum of each column: Mat<n, m> -> Vec<m>
• def Mat.matvecmatrix · vector: (n x k) · k = n
• def Mat.reshapeReshape: compiles only with the PROOF that the number of elements does not change.
• def max.go
• def max.l
• def exp_shift
• def div.l
• def softmax.finstable softmax: subtracts the maximum before exponentiating
• def softmax.l
• def rows.softmax
• def gelu.fGELU with the tanh approximation (the same as GPT-2)
• def gelu.l
• def rows.gelu
• def sub_const
• def sq_sum
• def layernorm.fin(x - mean) / sqrt(variance + eps), then * gamma + beta
• def layernorm.l
• def rows.layernorm
• def Mat.softmax
• def Mat.gelu
• def Mat.layernormnormalizes each row (d columns) and applies gamma and beta, vectors of d numbers
• def Vec.softmax
• def len.eq
• def rows.ok
• def Vec.of2
• def Vec.ofVec<n> from a list, only if it has exactly n numbers
• def Mat.of2
• def Mat.ofMat<r, c> from rows, only if they are exactly r rows of c numbers
• def Vec.to_list
• def Mat.to_rows
• law reshape_swapLAW (reshape_swap): r x c and c x r have the same number of elements (multiplication is commutative), so this reshape always exists.
• law reshape_flatLAW (reshape_flat): r x c has the same number of elements as 1 x (r*c) (multiplying by 1 changes nothing), so flattening a matrix always exists.
• def Mat.flattenMat<r, c> -> Mat<1, r*c>: the proof comes from the law above
• def Mat.reshape_swapMat<r, c> -> Mat<c, r> over the same data (it is NOT the transpose!): it only reinterprets