book Crafting Interpreters programming language compiler


  • First step to any compiler or interpreter
  • Scanner takes in raw source as a series of characters and groups it into tokens
    • Scanning or lexing (lexical analysis)

The Interpreter Framework

  • Can be run in two ways
    • Command line with a path to a Lox file
    • Interactively
  • Will be using exit codes defined in the UNIX “sysexits.h”

Lexemes and Tokens

  • Scan list of characters and group them into the smallest sequences that have meaning
  • Each group of meaningful characters is a lexeme
var language = "lox";
  • Lexeme from the above example:
    • var
    • language
    • =
    • "lox"
    • ;

Regular Languages and Expressions

  • Rules uses to determine how a language groups characters into lexemes is called its lexical grammar
  • If a language's rules can be defined by a regular expression then the language is a regular language