BendMlBpeTokenizer.encode.go: def of bend-ml-bpe-tokenizer@0.1.3.0
Import: import bend-ml-bpe-tokenizer@0.1.3.0/main.bend as BendMlBpeTokenizer
Lines 69-80 of bend-ml-bpe-tokenizer@0.1.3.0/main.bend: https://jend.leeeet.dev/read.txt?path=bend-ml-bpe-tokenizer@0.1.3.0/main.bend&start_line=69&end_line=80

import Base

# applies the rules from the oldest to the newest; `done` keeps the ones already applied
# (newest first), which is exactly the table that decode needs. Only the proofs read it, so it
# is erased (`-done`): the checker keeps it and the compiled program never builds it. Building
# it cost half of the time to encode (found by Mathews Fernando's bend-lint, dead_accumulator).
def encode.go(todo: List<&2, Rule>, -done: List<&2, Rule>, xs: List<&2, Tk>) -> List<&2, Tk>:
  match todo:
    case Nil{}:
      xs
    case Con{+r, rest}:
      match r:
        case Rule{+i, +a, +b}:
          encode.go(rest, Rule{i, a, b} <> done, merge(a, b, M{i}, xs))

Uses (4):
  https://jend.leeeet.dev/definition.txt?path=Base/base.bend&name=List
  https://jend.leeeet.dev/definition.txt?path=bend-ml-bpe-tokenizer@0.1.3.0/main.bend&name=Rule
  https://jend.leeeet.dev/definition.txt?path=bend-ml-bpe-tokenizer@0.1.3.0/main.bend&name=Tk
  https://jend.leeeet.dev/definition.txt?path=bend-ml-bpe-tokenizer@0.1.3.0/main.bend&name=merge