The Lisp tokenizer works and it is 83 bytes
:)
You don't need lookahead to deal with lisp, you just need super fuckin lazy parsing.
So like, we don't deal with a number until we find a non-number character, then we execute both the number handling code and the non-number handling code.
This will break at places where you have like, numbers and symbols right next to each other but like... don't do that.
We're not going to have error handling or any nice conveniences because this is literally running in 512 bytes, what do you actually *need*, it should be able to cons two lists together fine. your fault if things crap out because you happened to not type a space
(ha ha only serious)
@alexandria so you can't have a literal negative number?
@efi but also yes a negative number will fuck up because we straight up do a shitty tiny hash on the input symbol, so it will start hashing the minus sign, and then multiply that by ten and add the number to it, because it would waste bytes checking the state beforehand