big inhib! the nice thing with t5 is there are mt5 and t0pp and flan-t5 pretrained models that have been trained on a lot of tokens, not sure if they are in adapterhub.
i found the newer hf scripts can train decoder-only models on pure
text with linebreaks
whereas t5 models may take jsonlines of the format
{“translation”:{“key1”:”text1”,”key2”:”text2”}} where key1 and key2
are specified at launch time
