Commit graph

4 commits

Author SHA1 Message Date
Alam Ed Arias
da1b579cee Tokenizer_Open: pad the string with an NULL byte 2024-02-21 21:53:12 -05:00
Alam Ed Arias
8c7fe2ae95 Make tokenizer keep a copy of string in normal memory 2024-02-21 21:41:16 -05:00
Lactozilla
8bbc512b14 Tokenizer changes:
Capture '"' characters
Capture ';' tokens
Implement tracking of the current line
2024-01-19 16:40:32 -03:00
Lactozilla
9dbf30e20d Refactor tokenizer 2023-10-30 14:54:32 -03:00