Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hi devs, A lot has been said and it is difficult to address most of the interesting opinions that have been raised throughout the conversation. In general I am a moderate fan of releases and I strongly agree with Daniel's story: There is a lot of discussion about when users should be updating their gem5 versions, but how many even want to update? "You don't change a winning team" is something that I've heard a lot from teachers and colleagues during my former studies: why would you risk updating and breaking something, or reducing performance improvements? Throughout the years I've seen that students (me included) tend not to update anything (OS, compiler, software being used) until their work is done out of fear of having to reconfigure/fix things. I got asked several times which stable version should someone use. It is very weird and scary for a user who's willing to rebase his work to get a commit Id as an answer. Especially if there is no bug-fixing happening on a commitId. In fact, if you don't backport bug fixes, the entire concept of stableness is flawed, since you might discover a new bug later on and you are basically forcing a USER (not an active developer) to rebase his work in order to integrate the bug. (A user could cherry-pick the commit on his own, but first he needs to be aware that a specific commit is fixing his problem) I think we all agree that having a linear chunck of untagged and unbranched commits makes it difficult for common users to understand what they are getting when they rebase their work. It is not for some of us maybe, spending most of our engineering time reviewing patches and contributing to gerrit. I could more or less tell what's the difference in the simulator between today's gem5 and last month gem5, but that doesn't apply to the rest of the people. Giacomo From: gem5-dev on behalf of Daniel Carvalho Sent: 13 December 2019 11:32 To: gem5 Developer List Subject: Re: [gem5-dev] gem5 19.0.0 : New git branching proposal A bit late to the party, but here are my two cents. Regarding developing, based on previous experiences and questions I've received from users throughout these years, I couldn't agree more with Jason's sentence: "My experience with architecture grad students and researchers is that if you don't force them to use things like git most of them won't". I've also asked some of these users their preference, and the stable version on master with a development branch was an unanimous pick. However, it seems that this choice was made mostly because it'd require less learning effort; and I am positive not needing to know anything about git was appealing to some (i.e., "git clone" would be the first and last git command they'd use). Therefore, I am not sure this approach will achieve the intended goal of "directing to correct usage". There is a lot of discussion about when users should be updating their gem5 versions, but how many even want to update? "You don't change a winning team" is something that I've heard a lot from teachers and colleagues during my former studies: why would you risk updating and breaking something, or reducing performance improvements? Throughout the years I've seen that students (me included) tend not to update anything (OS, compiler, software being used) until their work is done out of fear of having to reconfigure/fix things. In the end I think that the focus of stable versions should be on reproducibility, not on updatability: researchers will inform which gem5 tag they are using, making it less difficult for readers to reproduce their results. There will obviously be users updating, but changing from possibly having to rebase on every commit to once or twice a year seems like a reasonable compromise given the benefits; the versioning will make updates start to feel more like upgrades, and API changes switch from a constant fear to a short term agreement. There is no need to say that in any case API changes must be made carefully and as infrequently as possible. Personally, I think major and minor differences should not differ as of now; this would complicate reviewing and merging, adding a "when should this go?" to an already complex process. I am not saying this is not possible, but maybe it'd be aiming too high and it would be better to take smaller steps. Overall, the ideas are excellent, and if they manage to leave the theory plane, it will be a great improvement! Looking forward to what's in the future. Regards, Daniel Em sexta-feira, 13 de dezembro de 2019 02:16:05 GMT+1, Jason Lowe-Power escreveu: Ok... so are you suggesting there be no difference between major and minor releases? How do you suggest communicating that API changes are coming? The most common way to do this is by marking functions as deprecated for some period of time. Thanks, Jason On Thu,
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
A bit late to the party, but here are my two cents. Regarding developing, based on previous experiences and questions I've received from users throughout these years, I couldn't agree more with Jason's sentence: "My experience with architecture grad students and researchers is that if you don't force them to use things like git most of them won't". I've also asked some of these users their preference, and the stable version on master with a development branch was an unanimous pick. However, it seems that this choice was made mostly because it'd require less learning effort; and I am positive not needing to know anything about git was appealing to some (i.e., "git clone" would be the first and last git command they'd use). Therefore, I am not sure this approach will achieve the intended goal of "directing to correct usage". There is a lot of discussion about when users should be updating their gem5 versions, but how many even want to update? "You don't change a winning team" is something that I've heard a lot from teachers and colleagues during my former studies: why would you risk updating and breaking something, or reducing performance improvements? Throughout the years I've seen that students (me included) tend not to update anything (OS, compiler, software being used) until their work is done out of fear of having to reconfigure/fix things. In the end I think that the focus of stable versions should be on reproducibility, not on updatability: researchers will inform which gem5 tag they are using, making it less difficult for readers to reproduce their results. There will obviously be users updating, but changing from possibly having to rebase on every commit to once or twice a year seems like a reasonable compromise given the benefits; the versioning will make updates start to feel more like upgrades, and API changes switch from a constant fear to a short term agreement. There is no need to say that in any case API changes must be made carefully and as infrequently as possible. Personally, I think major and minor differences should not differ as of now; this would complicate reviewing and merging, adding a "when should this go?" to an already complex process. I am not saying this is not possible, but maybe it'd be aiming too high and it would be better to take smaller steps. Overall, the ideas are excellent, and if they manage to leave the theory plane, it will be a great improvement! Looking forward to what's in the future. Regards, Daniel Em sexta-feira, 13 de dezembro de 2019 02:16:05 GMT+1, Jason Lowe-Power escreveu: Ok... so are you suggesting there be no difference between major and minor releases? How do you suggest communicating that API changes are coming? The most common way to do this is by marking functions as deprecated for some period of time. Thanks, Jason On Thu, Dec 12, 2019 at 4:26 PM Gabe Black wrote: > I think that's what the release vs. development branch split already > provides. If you want stable interfaces, then the release branch will stay > as it is for a year, or forever if you don't move to new releases. > > Gabe > > On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power > wrote: > > > Thanks Gabe! > > > > I think we're mostly in agreement. The key disagreement is the following: > > > > Note that I'm not advocating for a wild west free for all with no > > > accountability, but only being able to make changes to "interfaces" > > (which > > > pretty much everything could be considered) once a year is much too far > > in > > > the other direction, especially on a so called development branch. > > > > > > I believe, and I've heard from many gem5 users, that we need to have more > > stable interfaces. I think that once a year is a good compromise, but I'm > > open to a faster cadence. Either way, we must give a heads up of at least > > months to change an interface. > > > > Can you propose a different idea that will satisfy our need to both > > provided stability and allow developers to update code? > > > > Alternatively, maybe we need to come up with a better transition than > just > > "tomorrow you can't modify interfaces." Do you have any ideas on a more > > smooth transition to this development model of providing stability to our > > users? > > > > Thanks, > > Jason > > > > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black wrote: > > > > > I'll point out that nobody had to rebase O3 when, for example, the > > > ExecContext changed, because it was already in the tree and was changed > > > along with everything else. > > > > > > I think you have an important contradiction in your reasoning above, > > namely > > > that gem5 is simultaneously too unstable and too stable. You're saying > > it's > > > too unstable since the interfaces change too often, but then also say > > that > > > there have been very few new features in gem5 in the last few years. > > > > > > Fundamentally, new features require changing things. If we clamp down > on >
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
I didn't see anything in the email above about minor vs. major releases. Was that described somewhere else? As far as publicizing API deprecation, that could be something we patch into the preceding release some window before the new release. That could actually be a good point to grab a snapshot of the development branch so it can be tested and stabilized (ie fixed) before being released, since at that point we would know what APIs were changing. It would probably also be a good idea to send a heads up email to the user mailing list at that time since people probably won't pull down new versions of the releases very often, particularly if they don't change much (which is they point after all). Gabe On Thu, Dec 12, 2019 at 5:16 PM Jason Lowe-Power wrote: > Ok... so are you suggesting there be no difference between major and minor > releases? > > How do you suggest communicating that API changes are coming? The most > common way to do this is by marking functions as deprecated for some period > of time. > > Thanks, > Jason > > On Thu, Dec 12, 2019 at 4:26 PM Gabe Black wrote: > > > I think that's what the release vs. development branch split already > > provides. If you want stable interfaces, then the release branch will > stay > > as it is for a year, or forever if you don't move to new releases. > > > > Gabe > > > > On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power > > wrote: > > > > > Thanks Gabe! > > > > > > I think we're mostly in agreement. The key disagreement is the > following: > > > > > > Note that I'm not advocating for a wild west free for all with no > > > > accountability, but only being able to make changes to "interfaces" > > > (which > > > > pretty much everything could be considered) once a year is much too > far > > > in > > > > the other direction, especially on a so called development branch. > > > > > > > > > I believe, and I've heard from many gem5 users, that we need to have > more > > > stable interfaces. I think that once a year is a good compromise, but > I'm > > > open to a faster cadence. Either way, we must give a heads up of at > least > > > months to change an interface. > > > > > > Can you propose a different idea that will satisfy our need to both > > > provided stability and allow developers to update code? > > > > > > Alternatively, maybe we need to come up with a better transition than > > just > > > "tomorrow you can't modify interfaces." Do you have any ideas on a more > > > smooth transition to this development model of providing stability to > our > > > users? > > > > > > Thanks, > > > Jason > > > > > > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black > wrote: > > > > > > > I'll point out that nobody had to rebase O3 when, for example, the > > > > ExecContext changed, because it was already in the tree and was > changed > > > > along with everything else. > > > > > > > > I think you have an important contradiction in your reasoning above, > > > namely > > > > that gem5 is simultaneously too unstable and too stable. You're > saying > > > it's > > > > too unstable since the interfaces change too often, but then also say > > > that > > > > there have been very few new features in gem5 in the last few years. > > > > > > > > Fundamentally, new features require changing things. If we clamp down > > on > > > > changing what exists or add a bunch of onerous requirements, then > there > > > > will either be no change, or it will be side stepped by laying down a > > > layer > > > > of concrete and adding something new over the top, ie ifdefs, config > > > > options, "real" versions of APIs which live alongside the old ones. > > These > > > > are all things which have already happened in gem5 and which I've > spent > > > > some time partially cleaning up. In the real world, see the x86 ISA > as > > an > > > > example. Do I really need to be able to boot my huge workstation into > > > > little bitty 16 bit real mode? Probably not, but I still could, and > the > > > ISA > > > > is a bit of a mess because of it. > > > > > > > > Frankly, there is also a lot of technical debt in gem5. A lot of > things > > > > were accreted over time, often by well meaning but not necessary > expert > > > > programmers, including earlier (and even current) versions of me, and > > > those > > > > things need to be cleaned up. Fixing those things requires changing > > > > bad/obsolete/etc interfaces, restructuring code, and generally moving > > > > things around which have ripple effects throughout the simulator. You > > > can't > > > > make lemonade without cracking a few lemons. > > > > > > > > I think a few things are key: > > > > Tests make changes less dangerous. We need more of these. > > > > Warn people about changes so they have an opportunity to update and > > don't > > > > get blindsided. > > > > Change interfaces carefully, with consideration for the future so > they > > > > don't have to be changed excessively. > > > > > > > > Note that I'm not advocating for a wild west free for all
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Ok... so are you suggesting there be no difference between major and minor releases? How do you suggest communicating that API changes are coming? The most common way to do this is by marking functions as deprecated for some period of time. Thanks, Jason On Thu, Dec 12, 2019 at 4:26 PM Gabe Black wrote: > I think that's what the release vs. development branch split already > provides. If you want stable interfaces, then the release branch will stay > as it is for a year, or forever if you don't move to new releases. > > Gabe > > On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power > wrote: > > > Thanks Gabe! > > > > I think we're mostly in agreement. The key disagreement is the following: > > > > Note that I'm not advocating for a wild west free for all with no > > > accountability, but only being able to make changes to "interfaces" > > (which > > > pretty much everything could be considered) once a year is much too far > > in > > > the other direction, especially on a so called development branch. > > > > > > I believe, and I've heard from many gem5 users, that we need to have more > > stable interfaces. I think that once a year is a good compromise, but I'm > > open to a faster cadence. Either way, we must give a heads up of at least > > months to change an interface. > > > > Can you propose a different idea that will satisfy our need to both > > provided stability and allow developers to update code? > > > > Alternatively, maybe we need to come up with a better transition than > just > > "tomorrow you can't modify interfaces." Do you have any ideas on a more > > smooth transition to this development model of providing stability to our > > users? > > > > Thanks, > > Jason > > > > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black wrote: > > > > > I'll point out that nobody had to rebase O3 when, for example, the > > > ExecContext changed, because it was already in the tree and was changed > > > along with everything else. > > > > > > I think you have an important contradiction in your reasoning above, > > namely > > > that gem5 is simultaneously too unstable and too stable. You're saying > > it's > > > too unstable since the interfaces change too often, but then also say > > that > > > there have been very few new features in gem5 in the last few years. > > > > > > Fundamentally, new features require changing things. If we clamp down > on > > > changing what exists or add a bunch of onerous requirements, then there > > > will either be no change, or it will be side stepped by laying down a > > layer > > > of concrete and adding something new over the top, ie ifdefs, config > > > options, "real" versions of APIs which live alongside the old ones. > These > > > are all things which have already happened in gem5 and which I've spent > > > some time partially cleaning up. In the real world, see the x86 ISA as > an > > > example. Do I really need to be able to boot my huge workstation into > > > little bitty 16 bit real mode? Probably not, but I still could, and the > > ISA > > > is a bit of a mess because of it. > > > > > > Frankly, there is also a lot of technical debt in gem5. A lot of things > > > were accreted over time, often by well meaning but not necessary expert > > > programmers, including earlier (and even current) versions of me, and > > those > > > things need to be cleaned up. Fixing those things requires changing > > > bad/obsolete/etc interfaces, restructuring code, and generally moving > > > things around which have ripple effects throughout the simulator. You > > can't > > > make lemonade without cracking a few lemons. > > > > > > I think a few things are key: > > > Tests make changes less dangerous. We need more of these. > > > Warn people about changes so they have an opportunity to update and > don't > > > get blindsided. > > > Change interfaces carefully, with consideration for the future so they > > > don't have to be changed excessively. > > > > > > Note that I'm not advocating for a wild west free for all with no > > > accountability, but only being able to make changes to "interfaces" > > (which > > > pretty much everything could be considered) once a year is much too far > > in > > > the other direction, especially on a so called development branch. > > > > > > Also, I think everyone here is well intentioned and wants to make gem5 > > > available to as many people as possible and as useful to them as > > possible, > > > and we're just trying to figure out how to do that. That's why we're > > > hashing this out on a public mailing list and not in a private thread, > > > secret meetings, etc. > > > > > > Gabe > > > > > > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power > > > > wrote: > > > > > > > Hey Abhishek, > > > > > > > > Yes! Not only will we be releasing disk images (that we can depending > > on > > > > license for the benchmarks) and kernel images, we will also be > > releasing > > > > all of the documentation and the scripts describing how these images > > were > > > >
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
I think that's what the release vs. development branch split already provides. If you want stable interfaces, then the release branch will stay as it is for a year, or forever if you don't move to new releases. Gabe On Thu, Dec 12, 2019 at 4:18 PM Jason Lowe-Power wrote: > Thanks Gabe! > > I think we're mostly in agreement. The key disagreement is the following: > > Note that I'm not advocating for a wild west free for all with no > > accountability, but only being able to make changes to "interfaces" > (which > > pretty much everything could be considered) once a year is much too far > in > > the other direction, especially on a so called development branch. > > > I believe, and I've heard from many gem5 users, that we need to have more > stable interfaces. I think that once a year is a good compromise, but I'm > open to a faster cadence. Either way, we must give a heads up of at least > months to change an interface. > > Can you propose a different idea that will satisfy our need to both > provided stability and allow developers to update code? > > Alternatively, maybe we need to come up with a better transition than just > "tomorrow you can't modify interfaces." Do you have any ideas on a more > smooth transition to this development model of providing stability to our > users? > > Thanks, > Jason > > On Thu, Dec 12, 2019 at 4:03 PM Gabe Black wrote: > > > I'll point out that nobody had to rebase O3 when, for example, the > > ExecContext changed, because it was already in the tree and was changed > > along with everything else. > > > > I think you have an important contradiction in your reasoning above, > namely > > that gem5 is simultaneously too unstable and too stable. You're saying > it's > > too unstable since the interfaces change too often, but then also say > that > > there have been very few new features in gem5 in the last few years. > > > > Fundamentally, new features require changing things. If we clamp down on > > changing what exists or add a bunch of onerous requirements, then there > > will either be no change, or it will be side stepped by laying down a > layer > > of concrete and adding something new over the top, ie ifdefs, config > > options, "real" versions of APIs which live alongside the old ones. These > > are all things which have already happened in gem5 and which I've spent > > some time partially cleaning up. In the real world, see the x86 ISA as an > > example. Do I really need to be able to boot my huge workstation into > > little bitty 16 bit real mode? Probably not, but I still could, and the > ISA > > is a bit of a mess because of it. > > > > Frankly, there is also a lot of technical debt in gem5. A lot of things > > were accreted over time, often by well meaning but not necessary expert > > programmers, including earlier (and even current) versions of me, and > those > > things need to be cleaned up. Fixing those things requires changing > > bad/obsolete/etc interfaces, restructuring code, and generally moving > > things around which have ripple effects throughout the simulator. You > can't > > make lemonade without cracking a few lemons. > > > > I think a few things are key: > > Tests make changes less dangerous. We need more of these. > > Warn people about changes so they have an opportunity to update and don't > > get blindsided. > > Change interfaces carefully, with consideration for the future so they > > don't have to be changed excessively. > > > > Note that I'm not advocating for a wild west free for all with no > > accountability, but only being able to make changes to "interfaces" > (which > > pretty much everything could be considered) once a year is much too far > in > > the other direction, especially on a so called development branch. > > > > Also, I think everyone here is well intentioned and wants to make gem5 > > available to as many people as possible and as useful to them as > possible, > > and we're just trying to figure out how to do that. That's why we're > > hashing this out on a public mailing list and not in a private thread, > > secret meetings, etc. > > > > Gabe > > > > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power > > wrote: > > > > > Hey Abhishek, > > > > > > Yes! Not only will we be releasing disk images (that we can depending > on > > > license for the benchmarks) and kernel images, we will also be > releasing > > > all of the documentation and the scripts describing how these images > were > > > created. These will also be updated at every release as appropriate. > > > > > > Cheers, > > > Jason > > > > > > > > > > > > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh < > > > abhishek.singh199...@gmail.com> wrote: > > > > > > > Hello Jason, > > > > > > > > This is perfect ! > > > > > > > > I had one more question, for full system simulations will there be > > > release > > > > of images and kernel files for every architectures (arm , x86, etc)? > > > > > > > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power < >
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Thanks Gabe! I think we're mostly in agreement. The key disagreement is the following: Note that I'm not advocating for a wild west free for all with no > accountability, but only being able to make changes to "interfaces" (which > pretty much everything could be considered) once a year is much too far in > the other direction, especially on a so called development branch. I believe, and I've heard from many gem5 users, that we need to have more stable interfaces. I think that once a year is a good compromise, but I'm open to a faster cadence. Either way, we must give a heads up of at least months to change an interface. Can you propose a different idea that will satisfy our need to both provided stability and allow developers to update code? Alternatively, maybe we need to come up with a better transition than just "tomorrow you can't modify interfaces." Do you have any ideas on a more smooth transition to this development model of providing stability to our users? Thanks, Jason On Thu, Dec 12, 2019 at 4:03 PM Gabe Black wrote: > I'll point out that nobody had to rebase O3 when, for example, the > ExecContext changed, because it was already in the tree and was changed > along with everything else. > > I think you have an important contradiction in your reasoning above, namely > that gem5 is simultaneously too unstable and too stable. You're saying it's > too unstable since the interfaces change too often, but then also say that > there have been very few new features in gem5 in the last few years. > > Fundamentally, new features require changing things. If we clamp down on > changing what exists or add a bunch of onerous requirements, then there > will either be no change, or it will be side stepped by laying down a layer > of concrete and adding something new over the top, ie ifdefs, config > options, "real" versions of APIs which live alongside the old ones. These > are all things which have already happened in gem5 and which I've spent > some time partially cleaning up. In the real world, see the x86 ISA as an > example. Do I really need to be able to boot my huge workstation into > little bitty 16 bit real mode? Probably not, but I still could, and the ISA > is a bit of a mess because of it. > > Frankly, there is also a lot of technical debt in gem5. A lot of things > were accreted over time, often by well meaning but not necessary expert > programmers, including earlier (and even current) versions of me, and those > things need to be cleaned up. Fixing those things requires changing > bad/obsolete/etc interfaces, restructuring code, and generally moving > things around which have ripple effects throughout the simulator. You can't > make lemonade without cracking a few lemons. > > I think a few things are key: > Tests make changes less dangerous. We need more of these. > Warn people about changes so they have an opportunity to update and don't > get blindsided. > Change interfaces carefully, with consideration for the future so they > don't have to be changed excessively. > > Note that I'm not advocating for a wild west free for all with no > accountability, but only being able to make changes to "interfaces" (which > pretty much everything could be considered) once a year is much too far in > the other direction, especially on a so called development branch. > > Also, I think everyone here is well intentioned and wants to make gem5 > available to as many people as possible and as useful to them as possible, > and we're just trying to figure out how to do that. That's why we're > hashing this out on a public mailing list and not in a private thread, > secret meetings, etc. > > Gabe > > On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power > wrote: > > > Hey Abhishek, > > > > Yes! Not only will we be releasing disk images (that we can depending on > > license for the benchmarks) and kernel images, we will also be releasing > > all of the documentation and the scripts describing how these images were > > created. These will also be updated at every release as appropriate. > > > > Cheers, > > Jason > > > > > > > > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh < > > abhishek.singh199...@gmail.com> wrote: > > > > > Hello Jason, > > > > > > This is perfect ! > > > > > > I had one more question, for full system simulations will there be > > release > > > of images and kernel files for every architectures (arm , x86, etc)? > > > > > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power > > > > wrote: > > > > > > > Hey Abhishek, > > > > > > > > Emails will continue to gem5-dev on every changeset pushed to the > > develop > > > > branch as they do now to master :). We can discuss if we want the > same > > > for > > > > feature branches (if we end up using them). > > > > > > > > Your first interpretation on gem5 stable is correct (sorry if this > > wasn't > > > > clear). It will be much more heavily tested than the minute-by-minute > > > > releases from the develop branch. With this testing we will be >
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
I'll point out that nobody had to rebase O3 when, for example, the ExecContext changed, because it was already in the tree and was changed along with everything else. I think you have an important contradiction in your reasoning above, namely that gem5 is simultaneously too unstable and too stable. You're saying it's too unstable since the interfaces change too often, but then also say that there have been very few new features in gem5 in the last few years. Fundamentally, new features require changing things. If we clamp down on changing what exists or add a bunch of onerous requirements, then there will either be no change, or it will be side stepped by laying down a layer of concrete and adding something new over the top, ie ifdefs, config options, "real" versions of APIs which live alongside the old ones. These are all things which have already happened in gem5 and which I've spent some time partially cleaning up. In the real world, see the x86 ISA as an example. Do I really need to be able to boot my huge workstation into little bitty 16 bit real mode? Probably not, but I still could, and the ISA is a bit of a mess because of it. Frankly, there is also a lot of technical debt in gem5. A lot of things were accreted over time, often by well meaning but not necessary expert programmers, including earlier (and even current) versions of me, and those things need to be cleaned up. Fixing those things requires changing bad/obsolete/etc interfaces, restructuring code, and generally moving things around which have ripple effects throughout the simulator. You can't make lemonade without cracking a few lemons. I think a few things are key: Tests make changes less dangerous. We need more of these. Warn people about changes so they have an opportunity to update and don't get blindsided. Change interfaces carefully, with consideration for the future so they don't have to be changed excessively. Note that I'm not advocating for a wild west free for all with no accountability, but only being able to make changes to "interfaces" (which pretty much everything could be considered) once a year is much too far in the other direction, especially on a so called development branch. Also, I think everyone here is well intentioned and wants to make gem5 available to as many people as possible and as useful to them as possible, and we're just trying to figure out how to do that. That's why we're hashing this out on a public mailing list and not in a private thread, secret meetings, etc. Gabe On Thu, Dec 12, 2019 at 10:08 AM Jason Lowe-Power wrote: > Hey Abhishek, > > Yes! Not only will we be releasing disk images (that we can depending on > license for the benchmarks) and kernel images, we will also be releasing > all of the documentation and the scripts describing how these images were > created. These will also be updated at every release as appropriate. > > Cheers, > Jason > > > > On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh < > abhishek.singh199...@gmail.com> wrote: > > > Hello Jason, > > > > This is perfect ! > > > > I had one more question, for full system simulations will there be > release > > of images and kernel files for every architectures (arm , x86, etc)? > > > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power > > wrote: > > > > > Hey Abhishek, > > > > > > Emails will continue to gem5-dev on every changeset pushed to the > develop > > > branch as they do now to master :). We can discuss if we want the same > > for > > > feature branches (if we end up using them). > > > > > > Your first interpretation on gem5 stable is correct (sorry if this > wasn't > > > clear). It will be much more heavily tested than the minute-by-minute > > > releases from the develop branch. With this testing we will be > publishing > > > the following: > > > - What (common) workloads are supported (e.g., SPEC, parsec, etc.). > Which > > > workloads we use here will be discussed in the future, stay tuned. > > > - For all of the workloads, we will publish common statistics for a few > > > different systems (e.g., time to simulate, IPC, cache miss rates, > memory > > > bandwidth, etc). The systems used and the stats will be discussed in > the > > > future. > > > > > > The stable release will *not* be a continuous process. The only time > the > > > stable branch will be updated is 1) At releases or 2) if a "major" bug > is > > > encountered. For non-release updates (e.g., for bugs), we'll be very > > > careful to either re-validate our results or somehow know the results > > won't > > > change. > > > > > > Cheers, > > > Jason > > > > > > > > > > > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh < > > > abhishek.singh199...@gmail.com> wrote: > > > > > > > Hi Jason, > > > > > > > > Thanks for your email. It cleared a lot of misunderstanding which I > > had. > > > > > > > > Is it possible to have an email sent on every commit we make to > atleast > > > > gem5-dev? > > > > The email list could be different and can be sent to people
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hey Abhishek, Yes! Not only will we be releasing disk images (that we can depending on license for the benchmarks) and kernel images, we will also be releasing all of the documentation and the scripts describing how these images were created. These will also be updated at every release as appropriate. Cheers, Jason On Thu, Dec 12, 2019 at 10:04 AM Abhishek Singh < abhishek.singh199...@gmail.com> wrote: > Hello Jason, > > This is perfect ! > > I had one more question, for full system simulations will there be release > of images and kernel files for every architectures (arm , x86, etc)? > > On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power > wrote: > > > Hey Abhishek, > > > > Emails will continue to gem5-dev on every changeset pushed to the develop > > branch as they do now to master :). We can discuss if we want the same > for > > feature branches (if we end up using them). > > > > Your first interpretation on gem5 stable is correct (sorry if this wasn't > > clear). It will be much more heavily tested than the minute-by-minute > > releases from the develop branch. With this testing we will be publishing > > the following: > > - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which > > workloads we use here will be discussed in the future, stay tuned. > > - For all of the workloads, we will publish common statistics for a few > > different systems (e.g., time to simulate, IPC, cache miss rates, memory > > bandwidth, etc). The systems used and the stats will be discussed in the > > future. > > > > The stable release will *not* be a continuous process. The only time the > > stable branch will be updated is 1) At releases or 2) if a "major" bug is > > encountered. For non-release updates (e.g., for bugs), we'll be very > > careful to either re-validate our results or somehow know the results > won't > > change. > > > > Cheers, > > Jason > > > > > > > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh < > > abhishek.singh199...@gmail.com> wrote: > > > > > Hi Jason, > > > > > > Thanks for your email. It cleared a lot of misunderstanding which I > had. > > > > > > Is it possible to have an email sent on every commit we make to atleast > > > gem5-dev? > > > The email list could be different and can be sent to people who are > > > interested in this so that it does not spam to gem5-Dev list. > > > > > > I am talking about gem5 developer branch and not stable. > > > > > > This is because, it will keep all the interested community members well > > > informed about new features that are added, who like to keep note of > > latest > > > changes and merge in their projects as required. > > > > > > And the way, I understand that a stable gem5 when a user desires is: > > > It wants the conference accepted applications (standard workload for > > > example spec, parsec, etc to run without any errors on both SE and FS > > mode > > > for every architecture and cpu models. > > > > > > If that’s what the stable releases are going to be testing before > > releasing > > > it, then a stable release is much more helpful and will have the wide > > > reach. But when I read proposal this seems to be step by step process > > > reaching it as the final goal. Please correct me on this if I > understood > > it > > > wrong. > > > > > > > > > The aim of stable releases is a continuous process and will always be > > > doubtful unless tested with all the major conference accepted > application > > > on every stable releases. > > > > > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power > > > > wrote: > > > > > > > Hi all, > > > > > > > > First of all, let me say that I hope it's clear that gem5 is not > > > > "controlled" in any way by me! As laid out in our governance > document ( > > > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic, > > > > consensus-based community project". Through these emails, and as the > > > chair > > > > of the project management committee, I'm trying to *build* consensus. > > > > > > > > Importantly, there's a reason we're trying to make this push for > stable > > > > APIs. I've heard from dozens of current and potential gem5 users that > > > they > > > > want stable gem5 releases. By providing stable releases, we will be > > > > expanding the users of gem5 (and, IMO, making the research and papers > > > that > > > > use gem5 better). > > > > > > > > Could you please clarify the policy on breaking APIs? It makes sense > > for > > > > > releases to maintain stable APIs, but how does that apply to the > > > > > development branch? I'm worried that it will be very hard to make > > > changes > > > > > that don't change any interfaces, and we definitely don't want to > > > > encourage > > > > > a style of development where we just add and add and add without > ever > > > > > refactoring or consolidating things. If APIs can continue to change > > as > > > > > needed in the development branch and we just need to warn people > > before > > > > > they're released, that seems
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hello Jason, This is perfect ! I had one more question, for full system simulations will there be release of images and kernel files for every architectures (arm , x86, etc)? On Thu, Dec 12, 2019 at 12:57 PM Jason Lowe-Power wrote: > Hey Abhishek, > > Emails will continue to gem5-dev on every changeset pushed to the develop > branch as they do now to master :). We can discuss if we want the same for > feature branches (if we end up using them). > > Your first interpretation on gem5 stable is correct (sorry if this wasn't > clear). It will be much more heavily tested than the minute-by-minute > releases from the develop branch. With this testing we will be publishing > the following: > - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which > workloads we use here will be discussed in the future, stay tuned. > - For all of the workloads, we will publish common statistics for a few > different systems (e.g., time to simulate, IPC, cache miss rates, memory > bandwidth, etc). The systems used and the stats will be discussed in the > future. > > The stable release will *not* be a continuous process. The only time the > stable branch will be updated is 1) At releases or 2) if a "major" bug is > encountered. For non-release updates (e.g., for bugs), we'll be very > careful to either re-validate our results or somehow know the results won't > change. > > Cheers, > Jason > > > > On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh < > abhishek.singh199...@gmail.com> wrote: > > > Hi Jason, > > > > Thanks for your email. It cleared a lot of misunderstanding which I had. > > > > Is it possible to have an email sent on every commit we make to atleast > > gem5-dev? > > The email list could be different and can be sent to people who are > > interested in this so that it does not spam to gem5-Dev list. > > > > I am talking about gem5 developer branch and not stable. > > > > This is because, it will keep all the interested community members well > > informed about new features that are added, who like to keep note of > latest > > changes and merge in their projects as required. > > > > And the way, I understand that a stable gem5 when a user desires is: > > It wants the conference accepted applications (standard workload for > > example spec, parsec, etc to run without any errors on both SE and FS > mode > > for every architecture and cpu models. > > > > If that’s what the stable releases are going to be testing before > releasing > > it, then a stable release is much more helpful and will have the wide > > reach. But when I read proposal this seems to be step by step process > > reaching it as the final goal. Please correct me on this if I understood > it > > wrong. > > > > > > The aim of stable releases is a continuous process and will always be > > doubtful unless tested with all the major conference accepted application > > on every stable releases. > > > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power > > wrote: > > > > > Hi all, > > > > > > First of all, let me say that I hope it's clear that gem5 is not > > > "controlled" in any way by me! As laid out in our governance document ( > > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic, > > > consensus-based community project". Through these emails, and as the > > chair > > > of the project management committee, I'm trying to *build* consensus. > > > > > > Importantly, there's a reason we're trying to make this push for stable > > > APIs. I've heard from dozens of current and potential gem5 users that > > they > > > want stable gem5 releases. By providing stable releases, we will be > > > expanding the users of gem5 (and, IMO, making the research and papers > > that > > > use gem5 better). > > > > > > Could you please clarify the policy on breaking APIs? It makes sense > for > > > > releases to maintain stable APIs, but how does that apply to the > > > > development branch? I'm worried that it will be very hard to make > > changes > > > > that don't change any interfaces, and we definitely don't want to > > > encourage > > > > a style of development where we just add and add and add without ever > > > > refactoring or consolidating things. If APIs can continue to change > as > > > > needed in the development branch and we just need to warn people > before > > > > they're released, that seems reasonable. > > > > > > > > > Let's dig into this deeper. First, I'd like to remind everyone that the > > > current proposal was the community consensus reached in the gem5 > roadmap > > > document: > > > > > > > > > https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit > > > . > > > Of course, we can always change what we decided before :). > > > > > > The goal is to provide our users with a stable base to build of off. If > > we > > > are constantly changing interfaces, like we do today, it's impossible > to > > > build a project based off of gem5. There have been many projects that > > have > > > languished
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hey Abhishek, Emails will continue to gem5-dev on every changeset pushed to the develop branch as they do now to master :). We can discuss if we want the same for feature branches (if we end up using them). Your first interpretation on gem5 stable is correct (sorry if this wasn't clear). It will be much more heavily tested than the minute-by-minute releases from the develop branch. With this testing we will be publishing the following: - What (common) workloads are supported (e.g., SPEC, parsec, etc.). Which workloads we use here will be discussed in the future, stay tuned. - For all of the workloads, we will publish common statistics for a few different systems (e.g., time to simulate, IPC, cache miss rates, memory bandwidth, etc). The systems used and the stats will be discussed in the future. The stable release will *not* be a continuous process. The only time the stable branch will be updated is 1) At releases or 2) if a "major" bug is encountered. For non-release updates (e.g., for bugs), we'll be very careful to either re-validate our results or somehow know the results won't change. Cheers, Jason On Thu, Dec 12, 2019 at 9:38 AM Abhishek Singh < abhishek.singh199...@gmail.com> wrote: > Hi Jason, > > Thanks for your email. It cleared a lot of misunderstanding which I had. > > Is it possible to have an email sent on every commit we make to atleast > gem5-dev? > The email list could be different and can be sent to people who are > interested in this so that it does not spam to gem5-Dev list. > > I am talking about gem5 developer branch and not stable. > > This is because, it will keep all the interested community members well > informed about new features that are added, who like to keep note of latest > changes and merge in their projects as required. > > And the way, I understand that a stable gem5 when a user desires is: > It wants the conference accepted applications (standard workload for > example spec, parsec, etc to run without any errors on both SE and FS mode > for every architecture and cpu models. > > If that’s what the stable releases are going to be testing before releasing > it, then a stable release is much more helpful and will have the wide > reach. But when I read proposal this seems to be step by step process > reaching it as the final goal. Please correct me on this if I understood it > wrong. > > > The aim of stable releases is a continuous process and will always be > doubtful unless tested with all the major conference accepted application > on every stable releases. > > On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power > wrote: > > > Hi all, > > > > First of all, let me say that I hope it's clear that gem5 is not > > "controlled" in any way by me! As laid out in our governance document ( > > http://gem5.org/Governance#Overview), "gem5 is a meritocratic, > > consensus-based community project". Through these emails, and as the > chair > > of the project management committee, I'm trying to *build* consensus. > > > > Importantly, there's a reason we're trying to make this push for stable > > APIs. I've heard from dozens of current and potential gem5 users that > they > > want stable gem5 releases. By providing stable releases, we will be > > expanding the users of gem5 (and, IMO, making the research and papers > that > > use gem5 better). > > > > Could you please clarify the policy on breaking APIs? It makes sense for > > > releases to maintain stable APIs, but how does that apply to the > > > development branch? I'm worried that it will be very hard to make > changes > > > that don't change any interfaces, and we definitely don't want to > > encourage > > > a style of development where we just add and add and add without ever > > > refactoring or consolidating things. If APIs can continue to change as > > > needed in the development branch and we just need to warn people before > > > they're released, that seems reasonable. > > > > > > Let's dig into this deeper. First, I'd like to remind everyone that the > > current proposal was the community consensus reached in the gem5 roadmap > > document: > > > > > https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit > > . > > Of course, we can always change what we decided before :). > > > > The goal is to provide our users with a stable base to build of off. If > we > > are constantly changing interfaces, like we do today, it's impossible to > > build a project based off of gem5. There have been many projects that > have > > languished because of this including gem5+SST and dist-gem5. If we had > well > > defined interfaces *that weren't constantly changing* I believe it would > > make gem5 a more widely used project. > > > > The proposal is to have *one* API breaking release every year. This was > > decided based on community feedback in the roadmap document. > > > > I (personally) believe we need to slow down and be more deliberate about > > changing our interfaces. I think that it's a good thing
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hi Jason, Thanks for your email. It cleared a lot of misunderstanding which I had. Is it possible to have an email sent on every commit we make to atleast gem5-dev? The email list could be different and can be sent to people who are interested in this so that it does not spam to gem5-Dev list. I am talking about gem5 developer branch and not stable. This is because, it will keep all the interested community members well informed about new features that are added, who like to keep note of latest changes and merge in their projects as required. And the way, I understand that a stable gem5 when a user desires is: It wants the conference accepted applications (standard workload for example spec, parsec, etc to run without any errors on both SE and FS mode for every architecture and cpu models. If that’s what the stable releases are going to be testing before releasing it, then a stable release is much more helpful and will have the wide reach. But when I read proposal this seems to be step by step process reaching it as the final goal. Please correct me on this if I understood it wrong. The aim of stable releases is a continuous process and will always be doubtful unless tested with all the major conference accepted application on every stable releases. On Thu, Dec 12, 2019 at 12:12 PM Jason Lowe-Power wrote: > Hi all, > > First of all, let me say that I hope it's clear that gem5 is not > "controlled" in any way by me! As laid out in our governance document ( > http://gem5.org/Governance#Overview), "gem5 is a meritocratic, > consensus-based community project". Through these emails, and as the chair > of the project management committee, I'm trying to *build* consensus. > > Importantly, there's a reason we're trying to make this push for stable > APIs. I've heard from dozens of current and potential gem5 users that they > want stable gem5 releases. By providing stable releases, we will be > expanding the users of gem5 (and, IMO, making the research and papers that > use gem5 better). > > Could you please clarify the policy on breaking APIs? It makes sense for > > releases to maintain stable APIs, but how does that apply to the > > development branch? I'm worried that it will be very hard to make changes > > that don't change any interfaces, and we definitely don't want to > encourage > > a style of development where we just add and add and add without ever > > refactoring or consolidating things. If APIs can continue to change as > > needed in the development branch and we just need to warn people before > > they're released, that seems reasonable. > > > Let's dig into this deeper. First, I'd like to remind everyone that the > current proposal was the community consensus reached in the gem5 roadmap > document: > > https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit > . > Of course, we can always change what we decided before :). > > The goal is to provide our users with a stable base to build of off. If we > are constantly changing interfaces, like we do today, it's impossible to > build a project based off of gem5. There have been many projects that have > languished because of this including gem5+SST and dist-gem5. If we had well > defined interfaces *that weren't constantly changing* I believe it would > make gem5 a more widely used project. > > The proposal is to have *one* API breaking release every year. This was > decided based on community feedback in the roadmap document. > > I (personally) believe we need to slow down and be more deliberate about > changing our interfaces. I think that it's a good thing that "it will be > very hard to make changes that don't change any interfaces". This will make > gem5 more stable and easy for others to use. If you believe you need to > change an interface, we should have a discussion about it first, and this > should include a discussion about how the interface change will affect our > users. > > I strongly agree that "we definitely don't want to encourage a style of > development where we just add and add and add without ever refactoring or > consolidating things." The current proposal states that when changing an > interface, we first add a new function and mark the old version as > deprecated. Then, during the API changing merge window delete all of the > deprecated functions. This is going to take more maintenance from our side, > but we now have the resources to do this. > > I disagree that "we just need to warn people before [API changes] are > released" is enough. We need to give them significant (months) of lead > time, and we need to give them the opportunity to transition. In fact, I > believe this is what Abhishek was getting at in his follow up. > > In general (and not necessarily tied to this proposal), we should try very > > hard not to have silos where people develop incompatible features and > don't > > add to gem5 in a holistic way. It's easiest to wall off functionality > > behind ifdefs or config
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hi all,
First of all, let me say that I hope it's clear that gem5 is not
"controlled" in any way by me! As laid out in our governance document (
http://gem5.org/Governance#Overview), "gem5 is a meritocratic,
consensus-based community project". Through these emails, and as the chair
of the project management committee, I'm trying to *build* consensus.
Importantly, there's a reason we're trying to make this push for stable
APIs. I've heard from dozens of current and potential gem5 users that they
want stable gem5 releases. By providing stable releases, we will be
expanding the users of gem5 (and, IMO, making the research and papers that
use gem5 better).
Could you please clarify the policy on breaking APIs? It makes sense for
> releases to maintain stable APIs, but how does that apply to the
> development branch? I'm worried that it will be very hard to make changes
> that don't change any interfaces, and we definitely don't want to encourage
> a style of development where we just add and add and add without ever
> refactoring or consolidating things. If APIs can continue to change as
> needed in the development branch and we just need to warn people before
> they're released, that seems reasonable.
Let's dig into this deeper. First, I'd like to remind everyone that the
current proposal was the community consensus reached in the gem5 roadmap
document:
https://docs.google.com/document/d/1fv01HavfkIIqfcgZoKUpojkUKkWujxspRCcvS5cTfkk/edit.
Of course, we can always change what we decided before :).
The goal is to provide our users with a stable base to build of off. If we
are constantly changing interfaces, like we do today, it's impossible to
build a project based off of gem5. There have been many projects that have
languished because of this including gem5+SST and dist-gem5. If we had well
defined interfaces *that weren't constantly changing* I believe it would
make gem5 a more widely used project.
The proposal is to have *one* API breaking release every year. This was
decided based on community feedback in the roadmap document.
I (personally) believe we need to slow down and be more deliberate about
changing our interfaces. I think that it's a good thing that "it will be
very hard to make changes that don't change any interfaces". This will make
gem5 more stable and easy for others to use. If you believe you need to
change an interface, we should have a discussion about it first, and this
should include a discussion about how the interface change will affect our
users.
I strongly agree that "we definitely don't want to encourage a style of
development where we just add and add and add without ever refactoring or
consolidating things." The current proposal states that when changing an
interface, we first add a new function and mark the old version as
deprecated. Then, during the API changing merge window delete all of the
deprecated functions. This is going to take more maintenance from our side,
but we now have the resources to do this.
I disagree that "we just need to warn people before [API changes] are
released" is enough. We need to give them significant (months) of lead
time, and we need to give them the opportunity to transition. In fact, I
believe this is what Abhishek was getting at in his follow up.
In general (and not necessarily tied to this proposal), we should try very
> hard not to have silos where people develop incompatible features and don't
> add to gem5 in a holistic way. It's easiest to wall off functionality
> behind ifdefs or config options or in separate repositories and then just
> ignore it when implementing new features, but that creates a lot of
> technical debt and complexity which really cripples both gem5 and future
> development and compounds exponentially over time.
I agree with this statement. We need to encourage development in the open
based on the mainline code. I strongly believe the current proposal *makes
this easier*, not harder. Today, many people do exactly what you're
describing. By implementing stable APIs that people can build off of and
allowing branches, we will be making it easier to add features to gem5 in a
holistic way.
Back to the branches, I'm still not a fan. I think it gives people an
> excuse not to make changes ("you want me to refactor 18 months of code for
> that?" (not an actual quote)), on top of other issues. Also there are more
> than two options. You can do your work on top of tree like I do, even when
> adding big new features. It ensures that you don't get too invested in bad
> paths and don't have a mountain of integration work to do when you finally
> decide to bridge back in. It also gives the community an appropriate
> opportunity to provide feedback through review which is not practical when
> dealing with dozens of files and thousands of lines of code with months of
> yet unseen history and iteration.
No one can do what you do, Gabe :D.
More seriously, a couple of points:
- Branches won't be "unseen." Their development
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hi Jason,
I agree with Gabe, it seems like gem5 is taking a way to get better but it
will very hard for PhD students to suddenly adapt and review thousand lines
of code with months of yet unseen history and iteration and the reason
behind every change.
And in the end it may end up to be used as a course project rather than
thesis project or project for architecture conferences as people will use
the commit or the branch which they are most familiar with.
With so less releases every year, personally it will take a lot of time to
relearn and understand the implementation in detail.
But as this fully controlled by you, we can only suggest you. It should not
be like people are creating another public gem5 as we have now and use that.
This issue was raised by Gabe on Nov 27 email and today also, so I hope we
take that into account.
It feels like gem5 is getting towards privatization...
It’s just a opinion, it may be wrong.
On Wed, Dec 11, 2019 at 10:12 PM Gabe Black wrote:
> Could you please clarify the policy on breaking APIs? It makes sense for
> releases to maintain stable APIs, but how does that apply to the
> development branch? I'm worried that it will be very hard to make changes
> that don't change any interfaces, and we definitely don't want to encourage
> a style of development where we just add and add and add without ever
> refactoring or consolidating things. If APIs can continue to change as
> needed in the development branch and we just need to warn people before
> they're released, that seems reasonable.
>
> In general (and not necessarily tied to this proposal), we should try very
> hard not to have silos where people develop incompatible features and don't
> add to gem5 in a holistic way. It's easiest to wall off functionality
> behind ifdefs or config options or in separate repositories and then just
> ignore it when implementing new features, but that creates a lot of
> technical debt and complexity which really cripples both gem5 and future
> development and compounds exponentially over time.
>
> Back to the branches, I'm still not a fan. I think it gives people an
> excuse not to make changes ("you want me to refactor 18 months of code for
> that?" (not an actual quote)), on top of other issues. Also there are more
> than two options. You can do your work on top of tree like I do, even when
> adding big new features. It ensures that you don't get too invested in bad
> paths and don't have a mountain of integration work to do when you finally
> decide to bridge back in. It also gives the community an appropriate
> opportunity to provide feedback through review which is not practical when
> dealing with dozens of files and thousands of lines of code with months of
> yet unseen history and iteration.
>
> Gabe
>
> On Wed, Dec 11, 2019 at 5:30 PM Jason Lowe-Power
> wrote:
>
> > Hi all,
> >
> > Here are the specific changes we're proposing. Please give us your
> > feedback! If you disagree with some part of this proposal, let us know
> > why and what you suggest instead.
> >
> > # What's changing now
> >
> > 1) We are going to release gem5 19 "soon" (two blocking issues: 1)
> > this proposal and 2) the website)
> > 2) When we release gem5 19 we are going to create a new branch called
> > "develop" which branches at the point we make the release.
> > 3) We are going to tag master with "19.0.0-beta"
> >
> > After this, all development will move to the develop branch instead of
> > master. This will allow the common case of people using gem5 for
> > research to be the default. When cloning gem5 you will get a "stable"
> > release.
> >
> > ## Changes for developers
> >
> > There will be two changes for developers:
> > 1) You'll have to checkout develop after cloning
> > 2) You'll have to push to "refs/for/develop" instead of "refs/for/master"
> >
> > # Next release
> >
> > When we get to our next release (gem5 20, yay!), we will merge the
> > develop branch into master. Assuming no hotfixes, this will be a
> > simple fast forward.
> >
> > We will have a major release once per year usually in April (after the
> > MICRO deadline but with enough lead time to put together an ISCA
> > tutorial). We will have two minor releases later in the year, one in
> > August and one in December.
> >
> > # Why are we going to releases?
> >
> > It's important for our *users* (remember, we're developing gem5 to be
> > used by other people, not just core developers!) to be able to build
> > off of stable APIs. Therefore, between major releases *we will not
> > change any APIs*. Code that we expect other people will build off of
> > are APIs. This includes, but is not limited to:
> > - Ports
> > - ExecContext
> > - Packet
> > - SimObject, ClockedObject, etc.
> > - Event, EventQueue, etc.
> > - Command line options
> > - And probably many more. We will work on finalizing this over the
> > next couple of months before the gem5-20 release.
> >
> > ## What does going to releases mean for developers?
> >
> > Thus,
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Could you please clarify the policy on breaking APIs? It makes sense for
releases to maintain stable APIs, but how does that apply to the
development branch? I'm worried that it will be very hard to make changes
that don't change any interfaces, and we definitely don't want to encourage
a style of development where we just add and add and add without ever
refactoring or consolidating things. If APIs can continue to change as
needed in the development branch and we just need to warn people before
they're released, that seems reasonable.
In general (and not necessarily tied to this proposal), we should try very
hard not to have silos where people develop incompatible features and don't
add to gem5 in a holistic way. It's easiest to wall off functionality
behind ifdefs or config options or in separate repositories and then just
ignore it when implementing new features, but that creates a lot of
technical debt and complexity which really cripples both gem5 and future
development and compounds exponentially over time.
Back to the branches, I'm still not a fan. I think it gives people an
excuse not to make changes ("you want me to refactor 18 months of code for
that?" (not an actual quote)), on top of other issues. Also there are more
than two options. You can do your work on top of tree like I do, even when
adding big new features. It ensures that you don't get too invested in bad
paths and don't have a mountain of integration work to do when you finally
decide to bridge back in. It also gives the community an appropriate
opportunity to provide feedback through review which is not practical when
dealing with dozens of files and thousands of lines of code with months of
yet unseen history and iteration.
Gabe
On Wed, Dec 11, 2019 at 5:30 PM Jason Lowe-Power
wrote:
> Hi all,
>
> Here are the specific changes we're proposing. Please give us your
> feedback! If you disagree with some part of this proposal, let us know
> why and what you suggest instead.
>
> # What's changing now
>
> 1) We are going to release gem5 19 "soon" (two blocking issues: 1)
> this proposal and 2) the website)
> 2) When we release gem5 19 we are going to create a new branch called
> "develop" which branches at the point we make the release.
> 3) We are going to tag master with "19.0.0-beta"
>
> After this, all development will move to the develop branch instead of
> master. This will allow the common case of people using gem5 for
> research to be the default. When cloning gem5 you will get a "stable"
> release.
>
> ## Changes for developers
>
> There will be two changes for developers:
> 1) You'll have to checkout develop after cloning
> 2) You'll have to push to "refs/for/develop" instead of "refs/for/master"
>
> # Next release
>
> When we get to our next release (gem5 20, yay!), we will merge the
> develop branch into master. Assuming no hotfixes, this will be a
> simple fast forward.
>
> We will have a major release once per year usually in April (after the
> MICRO deadline but with enough lead time to put together an ISCA
> tutorial). We will have two minor releases later in the year, one in
> August and one in December.
>
> # Why are we going to releases?
>
> It's important for our *users* (remember, we're developing gem5 to be
> used by other people, not just core developers!) to be able to build
> off of stable APIs. Therefore, between major releases *we will not
> change any APIs*. Code that we expect other people will build off of
> are APIs. This includes, but is not limited to:
> - Ports
> - ExecContext
> - Packet
> - SimObject, ClockedObject, etc.
> - Event, EventQueue, etc.
> - Command line options
> - And probably many more. We will work on finalizing this over the
> next couple of months before the gem5-20 release.
>
> ## What does going to releases mean for developers?
>
> Thus, if any developers want to make changes to the above code, we must
> Mark the API as "deprecated" at least by the *.2 December release to
> give users at least one quarter to prepare.
> AND either
> 1) Make backwards compatible changes (e.g., just add a new function to
> the API that we will switch to when the API merge window opens (after
> the December minor release))
> or 2) Create a new branch from develop that we wait to merge until the
> merge window.
>
> # Feature branches
>
> Despite the disagreement in this thread so far, we're still proposing
> feature branches. However, they should be *rare*, and only maintainers
> will have the ability to add new branches. The exact gerrit mechanisms
> for this need to be figured out.
>
> We *require* something like feature branches given this new release
> model for two reasons:
> 1) There will be API breaking changes that will be long-lived (at
> least between a major release and the next API change merge window).
> 2) Many people develop new features to incorporate into gem5 and
> frequently they take months of *iteration* before they are ready to be
> merged. AMD's GPU model in the gcn-staging
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Hi all, Here are the specific changes we're proposing. Please give us your feedback! If you disagree with some part of this proposal, let us know why and what you suggest instead. # What's changing now 1) We are going to release gem5 19 "soon" (two blocking issues: 1) this proposal and 2) the website) 2) When we release gem5 19 we are going to create a new branch called "develop" which branches at the point we make the release. 3) We are going to tag master with "19.0.0-beta" After this, all development will move to the develop branch instead of master. This will allow the common case of people using gem5 for research to be the default. When cloning gem5 you will get a "stable" release. ## Changes for developers There will be two changes for developers: 1) You'll have to checkout develop after cloning 2) You'll have to push to "refs/for/develop" instead of "refs/for/master" # Next release When we get to our next release (gem5 20, yay!), we will merge the develop branch into master. Assuming no hotfixes, this will be a simple fast forward. We will have a major release once per year usually in April (after the MICRO deadline but with enough lead time to put together an ISCA tutorial). We will have two minor releases later in the year, one in August and one in December. # Why are we going to releases? It's important for our *users* (remember, we're developing gem5 to be used by other people, not just core developers!) to be able to build off of stable APIs. Therefore, between major releases *we will not change any APIs*. Code that we expect other people will build off of are APIs. This includes, but is not limited to: - Ports - ExecContext - Packet - SimObject, ClockedObject, etc. - Event, EventQueue, etc. - Command line options - And probably many more. We will work on finalizing this over the next couple of months before the gem5-20 release. ## What does going to releases mean for developers? Thus, if any developers want to make changes to the above code, we must Mark the API as "deprecated" at least by the *.2 December release to give users at least one quarter to prepare. AND either 1) Make backwards compatible changes (e.g., just add a new function to the API that we will switch to when the API merge window opens (after the December minor release)) or 2) Create a new branch from develop that we wait to merge until the merge window. # Feature branches Despite the disagreement in this thread so far, we're still proposing feature branches. However, they should be *rare*, and only maintainers will have the ability to add new branches. The exact gerrit mechanisms for this need to be figured out. We *require* something like feature branches given this new release model for two reasons: 1) There will be API breaking changes that will be long-lived (at least between a major release and the next API change merge window). 2) Many people develop new features to incorporate into gem5 and frequently they take months of *iteration* before they are ready to be merged. AMD's GPU model in the gcn-staging branch is one example (https://gem5.googlesource.com/amd/gem5/+/refs/heads/agutierr/master-gcn3-staging), the FlexCPU model my group has been working on is another (https://github.com/darchr/gem5/tree/flexcpu). We need an improved method of merging these kinds of changes. The way I see it, we have two options: continue development locally in our own repos as we are doing right now. OR we could move the development *into the open* by creating feature branches on the main gem5 repo. I prefer the latter. There's little difference to Gerrit. The main difference is what you get when you do git pull. It will be up to the maintainers to make sure branches are merged in a timely fashion and we don't create too many branches. They should be kept up to date by their maintainer by frequently merging the develop branch. # Hotfixes to the stable/master branch It's likely we'll find bugs while developing gem5. If the maintainer for the component thinks the bug is severe enough (I don't want to make any specific guidelines right now), we will backport that bug to the master branch. We'll have to work on our process for this to ensure that develop will cleanly merge into master later, but this should be straightforward. # Feedback Please let us know what you think! We're going to go one more round of feedback on the gem5-dev list, then we'll post the finalized version on gem5-users for a last round of feedback. I'm hoping to begin to implement this early next year! Thanks for your patience on this. I know it's a lot to read through and think about, but it's important that we get this process right the first time. We don't want to have to change this every year :). Cheers, Jason On Tue, Dec 10, 2019 at 4:49 PM Bobby Bruce wrote: > > Thank you all for your comments on this. > > In regards to the master/develop branch idea, I'm a big advocate of Jason's > "make the most common use the easiest" philosophy. I think
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Thank you all for your comments on this. In regards to the master/develop branch idea, I'm a big advocate of Jason's "make the most common use the easiest" philosophy. I think he did a good job of describing why this is important. We use the git repo as our primarily way of distributing gem5, and, as such, I believe the default behaviour should be that when a user clones the repo, they are getting the latest stable release to build on top of. I strongly suspect users will disproportionately use whatever `git clone` gives them rather than move between branches or tags in order to get specific, stable releases. If most users end up working on the very latest "unstable" development version then our release system becomes pointless. I realize as everyone on this list is a gem5 contributor, there is a preference for the development branch to be default, but I'd hope we are in the minority of people who actually use gem5. My vote therefore still goes to having two branches: the master/"release" branch and the "develop" branch for day-to-day development. Hotfixes would be created on a separate branch and merged into both upon completion. Regarding feature branching --- My intention with this part of the proposal was not for feature branches to be the default manner in which people commit to gem5 (as they would be if we were to implement the GitFlow model in full), I consider them something we should allow, in some form, to improve the development process in certain circumstances, but that they should be a rather rare thing. For example, a new component may need several months of development work, consisting of many commits. Developers, at present, have two options: either A) gradually incorporate commits over the months on the master branch, or B) push all the commits to the master branch in one go. I don't think either are ideal solutions. A dedicated feature branch would quarantine these changes while in active development, to be merged at a later date. A second circumstance that could arise is when we wish to postpone the incorporation of a feature. A change that requires an API alteration, for example, should be held-off until we are ready to deploy a major release. Isolating such a change in a feature branch, to be merged later, may be necessary. This all being said, I acknowledge there is a risk in code getting lost indefinitely, or for long periods of time, in feature branches, and our repo getting messy as a result. After some discussion with Jason, I believe the best of both worlds can be achieved if we make feature branches "by request" --- anyone wanting to create a feature branch would need to ask a maintainer (or a PCM) to create one for them. Whether or not a feature branch is to be created would be decided upon on a case-by-case basis. I hope no one feels like I'm dictating anything here. I'm writing this primarily to prompt more discussion, so please feel free to counter any of my points or suggest alternatives. Simple messages of support, as well as criticism, would also be appreciated to help us get a feel for what is, or what is not, proving popular with the community. Kind regards, Bobby -- Dr. Bobby R. Bruce Room 2235, Kemper Hall, UC Davis Davis, CA, 95616 On Wed, Nov 27, 2019 at 4:49 PM Gabe Black wrote: > The idea of feature branches makes me very nervous. It sounds like a dark > corner for a large body of code to grow in, and then once it pops back into > the light there may be very ingrained and difficult to fix problems which > either have to be ignored or fixed. A long standing branch may also be very > hard to reconcile with ToT, and it may be very tempting to just let it die > rather than try to jump that energy barrier and get it back into mainline. > > This would basically be sanctifying the practice of perma-forking gem5, > which unfortunately happens anyway now and then. There have been a number > of times on the mailing list someone says something along the lines of "I > downloaded special version of gem5 foo from some github" and it turns out > it's X years old with a bunch of old and new bugs, nobody knows how it > works, etc. Let's not encourage that :-). > > Gabe > > On Wed, Nov 27, 2019 at 9:17 AM Vince Harron wrote: > > > I agree with Ciro that the de facto standard used by most projects is > that > > mainline development should happen in master and releases should be new > > branches created off of master. > > > > You could also create a branch “stable” or “latest” that gets updated to > > the latest release. > > > > The default download instructions could say something like > > > > git clone —branch stable > > > > > > On Wed, Nov 27, 2019 at 8:54 AM Jason Lowe-Power > > wrote: > > > > > Thanks for the input Ciro! > > > > > > My pushback here is that most people who use gem5 will be making > > > modifications to it (at least for the foreseeable future). I don't > think > > we > > > want to common use case to be 1) Download a tarball, 2) make local > > >
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
The idea of feature branches makes me very nervous. It sounds like a dark corner for a large body of code to grow in, and then once it pops back into the light there may be very ingrained and difficult to fix problems which either have to be ignored or fixed. A long standing branch may also be very hard to reconcile with ToT, and it may be very tempting to just let it die rather than try to jump that energy barrier and get it back into mainline. This would basically be sanctifying the practice of perma-forking gem5, which unfortunately happens anyway now and then. There have been a number of times on the mailing list someone says something along the lines of "I downloaded special version of gem5 foo from some github" and it turns out it's X years old with a bunch of old and new bugs, nobody knows how it works, etc. Let's not encourage that :-). Gabe On Wed, Nov 27, 2019 at 9:17 AM Vince Harron wrote: > I agree with Ciro that the de facto standard used by most projects is that > mainline development should happen in master and releases should be new > branches created off of master. > > You could also create a branch “stable” or “latest” that gets updated to > the latest release. > > The default download instructions could say something like > > git clone —branch stable > > > On Wed, Nov 27, 2019 at 8:54 AM Jason Lowe-Power > wrote: > > > Thanks for the input Ciro! > > > > My pushback here is that most people who use gem5 will be making > > modifications to it (at least for the foreseeable future). I don't think > we > > want to common use case to be 1) Download a tarball, 2) make local > > modifications to the source. I think we want people to 1) clone a > "stable" > > gem5, 2) make modifications to the stable branch for their research *in > > git*. > > > > My experience with architecture grad students and researchers is that if > > you don't force them to use things like git most of them won't. One of my > > (personal) goals is to make it easy/default to use gem5 "the right way". > > > > In fact, I almost don't want to provide tarballs at all. Unless we can > > provide pre-build gem5 binaries, I don't think it makes sense to provide > > tarballs. I don't think prebuilt binaries make sense for the research use > > case of gem5 since almost all research will require extending gem5 and > > there's no way to dynamically load in new modules (though we should > support > > this in the future!). For teaching with gem5 prebuilt binaries could make > > sense, but I think we'll hold off on that for now, TBH. > > > > I think this comes down to "make the common case fast". I believe the > > common case is researchers who want to make minor changes to gem5 (e.g., > > add a new SimObject or modify the cache protocol) and run experiments for > > their research. Developing gem5 for the community (using the development > > branch) is actually the uncommon case. > > > > BTW, I agree with you that we should try to follow whatever practices are > > most popular when it makes sense. However, in many ways gem5 is a unique > > project, so following others doesn't always make sense :). > > > > I'm open to discussion on this. Thoughts? > > > > Cheers, > > Jason > > > > On Wed, Nov 27, 2019 at 12:01 AM Ciro Santilli > > wrote: > > > > > Thanks Bobby for pushing this forward, having releases would be a good > > > thing for gem5. > > > > > > I would recommend against having both master and develop branch though, > > > because in the large majority of projects out there, master == the > latest > > > version, so I'm always confused when I have to switch to develop, > > specially > > > if it isn't the default HEAD (is is however possible to set the default > > > branch to develop as shown as: > > > > > > https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch > > > ) > > > > > > This is even more important if we start having prebuilt releases or > tgzs > > > with source (which are smaller than full repo) for the tags, at which > > point > > > basically everyone who clones wants the latest. > > > > > > I would instead recommend: > > > > > > - master: latest version > > > - v2019: a tag. The latest one can be easily found with: > > > > > > https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git > > > - 2019 or b2019: branch created at the same time as v2019 to which > > > backports are applied, and to which v2019.0.1 tags can also be > optionally > > > applied > > > > > > > > > > > > From: gem5-dev on behalf of Bobby Bruce < > > > bbr...@ucdavis.edu> > > > Sent: Wednesday, November 27, 2019 12:10 AM > > > To: gem5-dev@gem5.org > > > Subject: [gem5-dev] gem5 19.0.0 : New git branching proposal > > > > > > Dear all, > > > > > > As you know, at the end of this quarter we will be releasing gem5-19, > the > > > first official release of gem5. As part of this release we'd like to > > change > > > our git branching
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
I agree with Ciro that the de facto standard used by most projects is that mainline development should happen in master and releases should be new branches created off of master. You could also create a branch “stable” or “latest” that gets updated to the latest release. The default download instructions could say something like git clone —branch stable On Wed, Nov 27, 2019 at 8:54 AM Jason Lowe-Power wrote: > Thanks for the input Ciro! > > My pushback here is that most people who use gem5 will be making > modifications to it (at least for the foreseeable future). I don't think we > want to common use case to be 1) Download a tarball, 2) make local > modifications to the source. I think we want people to 1) clone a "stable" > gem5, 2) make modifications to the stable branch for their research *in > git*. > > My experience with architecture grad students and researchers is that if > you don't force them to use things like git most of them won't. One of my > (personal) goals is to make it easy/default to use gem5 "the right way". > > In fact, I almost don't want to provide tarballs at all. Unless we can > provide pre-build gem5 binaries, I don't think it makes sense to provide > tarballs. I don't think prebuilt binaries make sense for the research use > case of gem5 since almost all research will require extending gem5 and > there's no way to dynamically load in new modules (though we should support > this in the future!). For teaching with gem5 prebuilt binaries could make > sense, but I think we'll hold off on that for now, TBH. > > I think this comes down to "make the common case fast". I believe the > common case is researchers who want to make minor changes to gem5 (e.g., > add a new SimObject or modify the cache protocol) and run experiments for > their research. Developing gem5 for the community (using the development > branch) is actually the uncommon case. > > BTW, I agree with you that we should try to follow whatever practices are > most popular when it makes sense. However, in many ways gem5 is a unique > project, so following others doesn't always make sense :). > > I'm open to discussion on this. Thoughts? > > Cheers, > Jason > > On Wed, Nov 27, 2019 at 12:01 AM Ciro Santilli > wrote: > > > Thanks Bobby for pushing this forward, having releases would be a good > > thing for gem5. > > > > I would recommend against having both master and develop branch though, > > because in the large majority of projects out there, master == the latest > > version, so I'm always confused when I have to switch to develop, > specially > > if it isn't the default HEAD (is is however possible to set the default > > branch to develop as shown as: > > > https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch > > ) > > > > This is even more important if we start having prebuilt releases or tgzs > > with source (which are smaller than full repo) for the tags, at which > point > > basically everyone who clones wants the latest. > > > > I would instead recommend: > > > > - master: latest version > > - v2019: a tag. The latest one can be easily found with: > > > https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git > > - 2019 or b2019: branch created at the same time as v2019 to which > > backports are applied, and to which v2019.0.1 tags can also be optionally > > applied > > > > > > > > From: gem5-dev on behalf of Bobby Bruce < > > bbr...@ucdavis.edu> > > Sent: Wednesday, November 27, 2019 12:10 AM > > To: gem5-dev@gem5.org > > Subject: [gem5-dev] gem5 19.0.0 : New git branching proposal > > > > Dear all, > > > > As you know, at the end of this quarter we will be releasing gem5-19, the > > first official release of gem5. As part of this release we'd like to > change > > our git branching structure. Therefore, I'm writing to ask for feedback > on > > what we have planned and whether it can be improved upon. > > > > We'd like to have a git repo structure similar to that used in gitflow > > development: https://nvie.com/posts/a-successful-git-branching-model . > > I've > > seen this model work well before, as it has some worthwhile abstractors > for > > public, open source git projects with regular releases. What I'd like to > > incorporate from this model is the following: > > > > - Two permanent branches: master and develop. > > - "develop" would function as master does now. This would be the main > point > > in which changes are applied between gem5 releases. > > - Upon a new release of gem5, the develop branch would be merged into > > master and a new git tag added to master indicating the release version. > > Ergo, the master branch would always contain the latest release of gem5. > > - If a quick hotfix is needed, a new "hotfix" branch would be created and > > merged into both the develop and master branches upon completion. This > > would also require a new tag on the master branch. (I
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Thanks for the input Ciro! My pushback here is that most people who use gem5 will be making modifications to it (at least for the foreseeable future). I don't think we want to common use case to be 1) Download a tarball, 2) make local modifications to the source. I think we want people to 1) clone a "stable" gem5, 2) make modifications to the stable branch for their research *in git*. My experience with architecture grad students and researchers is that if you don't force them to use things like git most of them won't. One of my (personal) goals is to make it easy/default to use gem5 "the right way". In fact, I almost don't want to provide tarballs at all. Unless we can provide pre-build gem5 binaries, I don't think it makes sense to provide tarballs. I don't think prebuilt binaries make sense for the research use case of gem5 since almost all research will require extending gem5 and there's no way to dynamically load in new modules (though we should support this in the future!). For teaching with gem5 prebuilt binaries could make sense, but I think we'll hold off on that for now, TBH. I think this comes down to "make the common case fast". I believe the common case is researchers who want to make minor changes to gem5 (e.g., add a new SimObject or modify the cache protocol) and run experiments for their research. Developing gem5 for the community (using the development branch) is actually the uncommon case. BTW, I agree with you that we should try to follow whatever practices are most popular when it makes sense. However, in many ways gem5 is a unique project, so following others doesn't always make sense :). I'm open to discussion on this. Thoughts? Cheers, Jason On Wed, Nov 27, 2019 at 12:01 AM Ciro Santilli wrote: > Thanks Bobby for pushing this forward, having releases would be a good > thing for gem5. > > I would recommend against having both master and develop branch though, > because in the large majority of projects out there, master == the latest > version, so I'm always confused when I have to switch to develop, specially > if it isn't the default HEAD (is is however possible to set the default > branch to develop as shown as: > https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch > ) > > This is even more important if we start having prebuilt releases or tgzs > with source (which are smaller than full repo) for the tags, at which point > basically everyone who clones wants the latest. > > I would instead recommend: > > - master: latest version > - v2019: a tag. The latest one can be easily found with: > https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git > - 2019 or b2019: branch created at the same time as v2019 to which > backports are applied, and to which v2019.0.1 tags can also be optionally > applied > > > > From: gem5-dev on behalf of Bobby Bruce < > bbr...@ucdavis.edu> > Sent: Wednesday, November 27, 2019 12:10 AM > To: gem5-dev@gem5.org > Subject: [gem5-dev] gem5 19.0.0 : New git branching proposal > > Dear all, > > As you know, at the end of this quarter we will be releasing gem5-19, the > first official release of gem5. As part of this release we'd like to change > our git branching structure. Therefore, I'm writing to ask for feedback on > what we have planned and whether it can be improved upon. > > We'd like to have a git repo structure similar to that used in gitflow > development: https://nvie.com/posts/a-successful-git-branching-model . > I've > seen this model work well before, as it has some worthwhile abstractors for > public, open source git projects with regular releases. What I'd like to > incorporate from this model is the following: > > - Two permanent branches: master and develop. > - "develop" would function as master does now. This would be the main point > in which changes are applied between gem5 releases. > - Upon a new release of gem5, the develop branch would be merged into > master and a new git tag added to master indicating the release version. > Ergo, the master branch would always contain the latest release of gem5. > - If a quick hotfix is needed, a new "hotfix" branch would be created and > merged into both the develop and master branches upon completion. This > would also require a new tag on the master branch. (I suggest using the > standard "Version [Major].[Minor].[Hotfix]" version numbering system. I.e., > the first version would be V19.0.0, a hotfix to this would make it V19.0.1, > and a minor release would make it V19.1.0). > - The creation of feature branches would be permitted. These branches would > encapsulate the gradual development of large features (i.e., ones carried > out over many commits). When complete a feature branch would be merged into > the develop branch. They'd be no obligation to use feature branches though > we believe they could be of value in certain cases. For example, if a > developer
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
Thanks Bobby for pushing this forward, having releases would be a good thing for gem5. I would recommend against having both master and develop branch though, because in the large majority of projects out there, master == the latest version, so I'm always confused when I have to switch to develop, specially if it isn't the default HEAD (is is however possible to set the default branch to develop as shown as: https://gerrit-review.googlesource.com/Documentation/project-configuration.html#default-branch ) This is even more important if we start having prebuilt releases or tgzs with source (which are smaller than full repo) for the tags, at which point basically everyone who clones wants the latest. I would instead recommend: - master: latest version - v2019: a tag. The latest one can be easily found with: https://stackoverflow.com/questions/1404796/how-to-get-the-latest-tag-name-in-current-branch-in-git - 2019 or b2019: branch created at the same time as v2019 to which backports are applied, and to which v2019.0.1 tags can also be optionally applied From: gem5-dev on behalf of Bobby Bruce Sent: Wednesday, November 27, 2019 12:10 AM To: gem5-dev@gem5.org Subject: [gem5-dev] gem5 19.0.0 : New git branching proposal Dear all, As you know, at the end of this quarter we will be releasing gem5-19, the first official release of gem5. As part of this release we'd like to change our git branching structure. Therefore, I'm writing to ask for feedback on what we have planned and whether it can be improved upon. We'd like to have a git repo structure similar to that used in gitflow development: https://nvie.com/posts/a-successful-git-branching-model . I've seen this model work well before, as it has some worthwhile abstractors for public, open source git projects with regular releases. What I'd like to incorporate from this model is the following: - Two permanent branches: master and develop. - "develop" would function as master does now. This would be the main point in which changes are applied between gem5 releases. - Upon a new release of gem5, the develop branch would be merged into master and a new git tag added to master indicating the release version. Ergo, the master branch would always contain the latest release of gem5. - If a quick hotfix is needed, a new "hotfix" branch would be created and merged into both the develop and master branches upon completion. This would also require a new tag on the master branch. (I suggest using the standard "Version [Major].[Minor].[Hotfix]" version numbering system. I.e., the first version would be V19.0.0, a hotfix to this would make it V19.0.1, and a minor release would make it V19.1.0). - The creation of feature branches would be permitted. These branches would encapsulate the gradual development of large features (i.e., ones carried out over many commits). When complete a feature branch would be merged into the develop branch. They'd be no obligation to use feature branches though we believe they could be of value in certain cases. For example, if a developer wishes to postpone a developed feature for a given gem5 release (e.g., something more suited for a major release rather than a minor one), then they could submit their changes as a feature branch and wait to merge to the develop branch at a later date. I believe this setup would make our development process run smoother and give gem5 users more stability. Day-to-day development wouldn't change much as committing to the develop branch would work in the same way as submitting to master does now. If anyone has any thoughts about this, I'd be happy to hear from you. Kind regards, Bobby -- Dr. Bobby R. Bruce Room 2235, Kemper Hall, UC Davis Davis, CA, 95616 ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
Re: [gem5-dev] gem5 19.0.0 : New git branching proposal
It’s a nice plan On Tue, Nov 26, 2019 at 7:10 PM Bobby Bruce wrote: > Dear all, > > As you know, at the end of this quarter we will be releasing gem5-19, the > first official release of gem5. As part of this release we'd like to change > our git branching structure. Therefore, I'm writing to ask for feedback on > what we have planned and whether it can be improved upon. > > We'd like to have a git repo structure similar to that used in gitflow > development: https://nvie.com/posts/a-successful-git-branching-model . > I've > seen this model work well before, as it has some worthwhile abstractors for > public, open source git projects with regular releases. What I'd like to > incorporate from this model is the following: > > - Two permanent branches: master and develop. > - "develop" would function as master does now. This would be the main point > in which changes are applied between gem5 releases. > - Upon a new release of gem5, the develop branch would be merged into > master and a new git tag added to master indicating the release version. > Ergo, the master branch would always contain the latest release of gem5. > - If a quick hotfix is needed, a new "hotfix" branch would be created and > merged into both the develop and master branches upon completion. This > would also require a new tag on the master branch. (I suggest using the > standard "Version [Major].[Minor].[Hotfix]" version numbering system. I.e., > the first version would be V19.0.0, a hotfix to this would make it V19.0.1, > and a minor release would make it V19.1.0). > - The creation of feature branches would be permitted. These branches would > encapsulate the gradual development of large features (i.e., ones carried > out over many commits). When complete a feature branch would be merged into > the develop branch. They'd be no obligation to use feature branches though > we believe they could be of value in certain cases. For example, if a > developer wishes to postpone a developed feature for a given gem5 release > (e.g., something more suited for a major release rather than a minor one), > then they could submit their changes as a feature branch and wait to merge > to the develop branch at a later date. > > I believe this setup would make our development process run smoother and > give gem5 users more stability. Day-to-day development wouldn't change much > as committing to the develop branch would work in the same way as > submitting to master does now. > > If anyone has any thoughts about this, I'd be happy to hear from you. > > Kind regards, > Bobby > -- > Dr. Bobby R. Bruce > Room 2235, > Kemper Hall, UC Davis > Davis, > CA, 95616 > ___ > gem5-dev mailing list > gem5-dev@gem5.org > http://m5sim.org/mailman/listinfo/gem5-dev ___ gem5-dev mailing list gem5-dev@gem5.org http://m5sim.org/mailman/listinfo/gem5-dev
