SLC - Lexical Analysis

SLC | Lexical Analysis

Like any programming language or most anyway, SkyLine's configuration language (SLC) has a lexer or scanner. This scanner is not like any normal scanner and instead of comparing byte positions, characters, peeking specific string tokens, it rather relies on a regular expression set to tokenize and catgeorize information. This is something good to mention as SkyLine also has a regular expression scanner behind it but it does not make it a primary option. Within SLC, regex is used because the goal of SLC is to be the most powerful engine for project development but also become the most lightweight engine out there that has 0 forms of bloat. To do so, everything was compacted down into a map of regular expressions which is shown in the block below.

var ScannerTokenizationRegularExpressions = map[*regexp.Regexp]TokenDataType{
	regexp.MustCompile(`^=$`):  ASSIGN_Token,
	regexp.MustCompile(`^;$`):  SEMICOLON_Token,
	regexp.MustCompile(`^\($`): LPAREN_Token,
	regexp.MustCompile(`^\)$`): RPAREN_Token,
	regexp.MustCompile(`^,$`):  COMMA_Token,
	regexp.MustCompile(`^{`):   LBRACE_Token,
	regexp.MustCompile(`^}`):   RBRACE_Token,
	regexp.MustCompile(`^\[$`): LBRACKET_Token,
	regexp.MustCompile(`^\]$`): RBRACKET_Token,
}