at the same time, i'll want to figure out how to make the scripts use
te tokenizer i trained. they're still using the extended vanilla
tokenizer. i actually trained a tokenizer on source code.

Reply via email to