This will be used for unifying preprocessing and parsing, the idea being that the tokens will be recorded for later expansion via macros, without the need to retokenize.
