I just want to chime in and say that we definitely want to continue to test the same number of Ruby protocols as before. Let test them with x86. I suspect that is the most common CPU ISA used with Ruby.
Thanks, Brad -----Original Message----- From: gem5-dev [mailto:[email protected]] On Behalf Of Steve Reinhardt Sent: Wednesday, June 08, 2016 4:01 PM To: gem5 Developer List <[email protected]> Subject: Re: [gem5-dev] Let's retire some ISAs Yes, I could quibble with your model, but I agree, I don't think it's worthwhile ;). I do think we've already outlined a number of steps (plus a couple others I thought of) that we could take immediately that are (I believe) totally uncontroversial: 1. Let's prune a lot of the long regressions that don't add much value, particularly for less-supported ISAs. Concretely, getting rid of all the long SE-mode ALPHA tests is pretty low-hanging fruit here. 2. Let's make sure the wiki is up-to-date on the level of support provided for various ISAs. 3. Let's add warnings, ideally at both compile time and run time, letting users know when they are using a less-supported ISA. 4. Let's make the default Ruby protocol something useful, so that we can do basic testing of both classic and Ruby memory systems with a single compile. 5. Let's only compile that one version of less-supported ISAs (read: ALPHA), instead of compiling it with four (!) different extra Ruby protocols. If we really want to test multiple Ruby protocols, we might as well at least do it using a well-supported ISA. 6. Let's give it another few days, but unless someone responds to my pleading EIO email from 6/3, let's go ahead and ditch EIO support. I suggest we go ahead and do the things we all agree on and see where that leaves us before we debate about additional steps. Steve On Wed, Jun 8, 2016 at 3:20 PM Andreas Hansson <[email protected]> wrote: > Hi Steve, > > In short: If we want to put in effort X to achieve something, we end > up having to put in 1.25 X (overhead times #ISAs), but it is only 1.05 > X that is actually useful (due to low utility). The 3 months was > merely me translating the wasted effort into a relatable number using > the current months commit count. > > Again, let’s not obsess about the numbers. Let us instead focus on > making the most of our efforts and decide what is needed and what is not. > > AndreasH > > On 08/06/2016, 23:05, "gem5-dev on behalf of Steve Reinhardt" > <[email protected] on behalf of [email protected]> wrote: > > >Hi Andreas, > > > >I don't know that I want a whole spreadsheet, but it's not clear to > >me how a 5% overhead turns into wasting 25% of our time (3 months out of 12). > >Partly that may be because I'm not sure what an "incremental cost of > >[...] 5% on average" means either. Can you elaborate please? > > > >Thanks, > > > >Steve > > > > > >On Wed, Jun 8, 2016 at 2:20 PM Andreas Hansson > ><[email protected]> > >wrote: > > > >> Hi all, > >> > >> It is worth remembering that the argument about “low" effort needs > >>to take long-term usefulness into account. Yes, the incremental > >>effort may seem small per commit, but if the incremental cost per > >>ISA is 5% on average, and the likelihood of that effort being > >>useful long term is 20%, then after one year we have effectively > >>wasted 3 months of gem5 contributions. > >> That is 90 commits. > >> > >> We can of course debate these numbers (let me know if you want the > >> spreadsheet...), but let’s not fool ourselves. Keeping things > >> around has noticeable implications on the overall progress. > >> > >> AndreasH > >> > >> > >> On 08/06/2016, 17:11, "gem5-dev on behalf of Steve Reinhardt" > >> <[email protected] on behalf of [email protected]> wrote: > >> > >> >On Wed, Jun 8, 2016 at 8:12 AM Andreas Sandberg > >><[email protected] > >> > > >> >wrote: > >> > > >> >> > > >> >> >Once the bar for reaching this point is raised by adding the > >> >> >steps > >>of > >> >> >determining that gem5 *used to* support the ISA of interest, > >> >> >finding > >> >>and > >> >> >extracting the old revision that still had that support in it, > >> >> >and > >>very > >> >> >likely needing to do who knows how much work to bring the ISA > >>support > >> >>back > >> >> >in sync with the head of the repo before being able to > >> >> >reasonably > >>make > >> >> >further enhancements, then the likelihood of anyone crossing > >> >> >that threshold is going to be dramatically lowered. > >> >> > >> >> You raise a valid point. However, I don¹t completely agree with > >> >> your > >>the > >> >> conclusion. > >> >> > >> > > >> >Hi Andreas, can you elaborate? I don't see anything you're saying > >> >that disagrees with what I think I said. > >> > > >> > > >> >> For complex work, say reworking register interfaces (e.g., > >>Nathanael¹s > >> >> patch), modifying every single ISA adds a significant > >> >> development overhead. > >> > > >> > > >> >Yes, I didn't say that incrementally maintaining all the ISAs is > >> >always easy. My point is that, first, however much effort was > >> >required of Nathanael, the effort required for someone else to > >> >come back later, identify his change as one that needs to be > >> >applied to a defunct ISA, reverse engineer what he did, and then > >> >apply it, is going to be > >>greater; > >> >and second, that greater effort, multiplied by the number of such > >>changes > >> >that accumulate, will significantly reduce the likelihood that > >> >anyone > >>ever > >> >does bother to resurrect an ISA, compared with the smaller effort > >> >of taking a functioning but incomplete ISA and addressing the > >> >gaps. > >> > > >> > > >> >> There is also a very real cost associated with having to > >> >> re-compile gem5 test all of the ISA/Ruby configuration we ³support". > >> > > >> > > >> >Yes, I didn't mean to imply that there weren't other costs. Your > >> >solutions below are some of the things I was thinking of when I > >> >said "some of the issues raised here could be addressed in other > >> >ways". > >> > > >> > > >> >> There > >> >> are two things we could do to improve the situation without > >>completely > >> >> retiring ISAs: > >> >> > >> >> * Improved build system reuse. We could significantly shorten > >> >> the build/test cycle by only compiling device models and the > >> >> core > >>framework > >> >> once instead of for each ISA/Ruby configuration. > >> >> > >> >> * We should get rid of long tests for unsupported ISAs and make > >> >>the remaining tests functional only (i.e., not diff stats or out files). > >> >>This > >> >> would make stat refreshes (which pretty much every single > >> >>non-trivial change requires) much more manageable. I¹m planning > >> >>to do similar > >>things > >> >> for some of the ARM tests that are more functional in nature. > >> >> > >> >> > >> >> >Which leads to #2: Basically the only way you win by retiring > >> >> >code > >>is > >> >>if > >> >> >that code is never resurrected. I'd say the lower bound on the > >> >> >cost > >>of > >> >> >resurrecting the code is the same as the incremental cost of > >> >> >keeping > >> >>the > >> >> >code alive, and that's a very optimistic lower bound. In > >> >> >practice, > >> >>whoever > >> >> >faces the task of resurrecting the code won't know which past > >>changes > >> >> >actually broke things, won't be the person who made those > >> >> >changes > >>and > >> >> >thus, > >> >> >once the guilty changes are identified, will have to > >>reverse-engineer > >> >>them > >> >> >to see how to apply them to the old ISA, and will be faced with > >> >> >the > >> >>task > >> >> >of > >> >> >fixing a number of things en masse rather than one at a time. > >> >> >In > >> >>addition, > >> >> >there's a good chance that someone considering taking this on > >> >> >is > >>new to > >> >> >gem5 and unfamiliar with the code to begin with, unlike the > >> >> >person > >>who > >> >>is > >> >> >committing patches and could perform the incremental updates. > >> >> > >> >> In that case, we Alpha would still be a candidate for removal. > >> >>The likelihood that someone will ever need it for research seems > >> >>very low and having to resurrect it seems extremely unlikely. A > >> >>simulator such as SimH seems more suitable for what can only be > >> >>described as retro computing (which I enjoy, but it¹s not gem5). > >> >> > >> > > >> >Yes, I don't dispute that Alpha looks like something we should > >>rationally > >> >retire. For sentimental reasons I'd hate to see that happen, but > >> >I > >>won't > >> >claim they are much more than that. Though perhaps there is > >> >pedagogical value in providing architecture students, who might > >> >not download SimH > >>on > >> >their own, the opportunity to look at the ISA and weep for what > >> >could > >>have > >> >been... > >> > > >> >>It would also be nice to keep in mind that some of the issues > >> >>raised > >>here > >> >> >could be addressed in other ways. For example, with respect to > >> >> >user expectations, there is already some documentation on the > >> >> >wiki about > >>the > >> >> >level of support for various ISAs: > >> >> >http://gem5.org/Architecture_Support > >> >> >http://gem5.org/Status_Matrix > >> >> >Regardless of the outcome here, it would be good to keep that > >> >>information > >> >> >up to date and maybe feature it more prominently so that new > >> >> >users > >>are > >> >> >aware of it. Perhaps less-supported ISAs could print a warning > >> >> >and > >>a > >> >>link > >> >> >to these pages when they are used. > >> >> > >> >> Updating these pages is definitely something we should do > >> >> regardless > >>of > >> >> whether we retire existing ISAs or not. We should make sure that > >> >>both the build system and the generated binary print a warning > >> >>if a user > >>wants to > >> >> use one of the unmaintained ISAs. This should help in setting > >> >> expectations. > >> >> > >> > > >> >Yes, I agree with this too. > >> > > >> >Steve > >> > > >> > > >> >> > >> >> //Andreas > >> >> > >> >> IMPORTANT NOTICE: The contents of this email and any attachments > >> >> are confidential and may also be privileged. If you are not the > >> >> intended recipient, please notify the sender immediately and do > >> >> not disclose > >>the > >> >> contents to any other person, use it for any purpose, or store > >> >> or > >>copy > >> >>the > >> >> information in any medium. Thank you. > >> >> > >> >> _______________________________________________ > >> >> gem5-dev mailing list > >> >> [email protected] > >> >> http://m5sim.org/mailman/listinfo/gem5-dev > >> >> > >> >_______________________________________________ > >> >gem5-dev mailing list > >> >[email protected] > >> >http://m5sim.org/mailman/listinfo/gem5-dev > >> > >> IMPORTANT NOTICE: The contents of this email and any attachments > >>are confidential and may also be privileged. If you are not the > >>intended recipient, please notify the sender immediately and do not > >>disclose the contents to any other person, use it for any purpose, > >>or store or copy the information in any medium. Thank you. > >> _______________________________________________ > >> gem5-dev mailing list > >> [email protected] > >> http://m5sim.org/mailman/listinfo/gem5-dev > >> > >_______________________________________________ > >gem5-dev mailing list > >[email protected] > >http://m5sim.org/mailman/listinfo/gem5-dev > > IMPORTANT NOTICE: The contents of this email and any attachments are > confidential and may also be privileged. If you are not the intended > recipient, please notify the sender immediately and do not disclose > the contents to any other person, use it for any purpose, or store or > copy the information in any medium. Thank you. > _______________________________________________ > gem5-dev mailing list > [email protected] > http://m5sim.org/mailman/listinfo/gem5-dev > _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev _______________________________________________ gem5-dev mailing list [email protected] http://m5sim.org/mailman/listinfo/gem5-dev
