> But in my opinion we don't want to keep adding split code to every > gen* file individually > but concentrate that code in gensupport so ever gen* can use it.
The individual splits are there because I started out with one (genemit), then later realized genrecog is large as well. It was certainly not the initial intention to split everything individually. But I agree that by now we have reached a tipping point. BTW I also touched opinit before, splitting one function into several, because it would trigger pathological compiler behavior. So we already perform some kind of splitting. Are we sure we're not actually seeing pathological behavior again that could be helped by something else than file-level splitting? > Lastly I also don't think splitting on an iterative way is going to > give you the best compile > time increase. Because that tends to bias the output. > > In genmatch I use ftell to tell how big the files is so far an pick the > smallest file. This allows > me to balance out the compile time over all files. I believe Robin eventually > respun his > gen<something> match to do this a swell as that allows you to amortize the > compile time > over cores. Yeah, I added your suggestion to the initial genemit split. And at least the "which file to write to next"/ftell functionality is not local to genemit but already in gensupport. The function is choose_output and Kyryll's patch even uses it already so it shouldn't do iterative splitting if I'm not mistaken. Each gen* still keep track of its own files, though, and that part could be unified still. I don't think it's terribly difficult but will require some plumbing. -- Regards Robin
