Re: [Geoserver-devel] status of release scripts
To do so it uses the jira rest api (which is nice btw). This is something I would like feedback on. Bleck - tough call. We have a couple of options here on what to do with respect to jira. 1. don't do any checks against jira, and leave that a manual process In my experience going through the Jira list is educational; for two reasons: - as often I find items that are marked resolved but not yet closed as they are waiting confirmation. - often issues are fixed (usually due to issues being duplicated etc…) This is also our last chance to see what is going on before the issues are punted over to the next release. 2. halt the build if the release is not released in jira or there are unresolved issues, making releasing on a jira manually a precursor to running the script 3. mark the release as released in jira, and push back any unresolved issues to the next version I would go with this option (despite the temptation to halt the build for manual review). Reason is: a) we are shooting for a two hour release window b) we can update the release notes in Jira after the fact; and the page will update More options welcome. Can we promote any resolved' issues to closed if they have now been included in a release? Jody -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] status of release scripts
The second thing I would like to discuss is how we handle the README. Currently when we release we update the README on the main branch (and not the tag). I would kind of like to avoid committing anything to the main branch in an automated process. For example what happens if we run the release, it fails and we run it again. We have to check if the readme has been already updated, etc... While certainly possible with some script mojo it seems like an error prone process I would like to avoid. So what I would like to propose is the following. We don't ever update the README on the main release branch. We leave it generic with information about the project, where to find the docs, etc... On the tag we create we would generate a new file called RELEASE_NOTES.txt (or something to that effect) that contains the stuff we usually put in the README like revision info and the new and noteworthy info for that release. To generate the new and noteworthy for the release notes i was thinking we could come up with a heuristic. One such heuristic could be to pull out any new feature or improvement, and perhaps any issue that is a high priority one. Anyways, you get the idea. And naturally it would contain the link to the entire changelog from jira like we do now. Perhaps we should just stick to simple and put the revision info and the changelog link in the release notes. Probably what I will do for a first pass if people are into the idea. I would look a lot better if you could generate a README.html using: - the boiler plate README from the main branch - idea: pull the release description from JIRA (if it is available ) and use it for the release specific description. - a link to the release notes in JIRA; optionally include a list of the closed items (as a highlights) and then offer a link through to JIRA for more information - (Wish) the features are often best summed up in the associated blog posts that have been made; could we troll the various geoserver, opengeo, geo-solutions blogs for a tag matching the release number - and provide links out to that information. Thanks for the hard work on this justin; when I was on IRC I saw geotools patches still being applied - is the coast clear now? Jody -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] status of release scripts
You were very fast Justin :) Don't have time right now to take a deep look at the scripts but I will do during the we as a personal interest. Thanks for achieving this. About JIRA, I also think option 3 is the best one. Do you still need help to do something in order to close up this task? Regards, Alessio. --- Ing. Alessio Fabiani Founder / CTO GeoSolutions S.A.S. GeoSolutions S.A.S. Via Poggio alle Viti 1187 55054 Massarosa (LU) Italy phone: (+39) 0584 96.23.13 fax: (+39) 0584 96.23.13 mobile:(+39) 331 62.33.686 http://www.geo-solutions.it http://geo-solutions.blogspot.com http://www.linkedin.com/in/alessiofabiani https://twitter.com/alfa7961 http://twitter.com/geosolutions_it --- On Wed, May 16, 2012 at 9:54 AM, Jody Garnett jody.garn...@gmail.comwrote: The second thing I would like to discuss is how we handle the README. Currently when we release we update the README on the main branch (and not the tag). I would kind of like to avoid committing anything to the main branch in an automated process. For example what happens if we run the release, it fails and we run it again. We have to check if the readme has been already updated, etc... While certainly possible with some script mojo it seems like an error prone process I would like to avoid. So what I would like to propose is the following. We don't ever update the README on the main release branch. We leave it generic with information about the project, where to find the docs, etc... On the tag we create we would generate a new file called RELEASE_NOTES.txt (or something to that effect) that contains the stuff we usually put in the README like revision info and the new and noteworthy info for that release. To generate the new and noteworthy for the release notes i was thinking we could come up with a heuristic. One such heuristic could be to pull out any new feature or improvement, and perhaps any issue that is a high priority one. Anyways, you get the idea. And naturally it would contain the link to the entire changelog from jira like we do now. Perhaps we should just stick to simple and put the revision info and the changelog link in the release notes. Probably what I will do for a first pass if people are into the idea. I would look a lot better if you could generate a README.html using: - the boiler plate README from the main branch - idea: pull the release description from JIRA (if it is available ) and use it for the release specific description. - a link to the release notes in JIRA; optionally include a list of the closed items (as a highlights) and then offer a link through to JIRA for more information - (Wish) the features are often best summed up in the associated blog posts that have been made; could we troll the various geoserver, opengeo, geo-solutions blogs for a tag matching the release number - and provide links out to that information. Thanks for the hard work on this justin; when I was on IRC I saw geotools patches still being applied - is the coast clear now? Jody -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] status of release scripts
On Wed, May 16, 2012 at 1:48 AM, Jody Garnett jody.garn...@gmail.comwrote: To do so it uses the jira rest api (which is nice btw). This is something I would like feedback on. Bleck - tough call. We have a couple of options here on what to do with respect to jira. 1. don't do any checks against jira, and leave that a manual process In my experience going through the Jira list is educational; for two reasons: - as often I find items that are marked resolved but not yet closed as they are waiting confirmation. - often issues are fixed (usually due to issues being duplicated etc…) This is also our last chance to see what is going on before the issues are punted over to the next release. 2. halt the build if the release is not released in jira or there are unresolved issues, making releasing on a jira manually a precursor to running the script 3. mark the release as released in jira, and push back any unresolved issues to the next version I would go with this option (despite the temptation to halt the build for manual review). Reason is: a) we are shooting for a two hour release window b) we can update the release notes in Jira after the fact; and the page will update More options welcome. Can we promote any resolved' issues to closed if they have now been included in a release? I don't see why not... I believe resolved is meant to signify it has been fixed by a developer and closed is used more as a confirmation that the fix actually works. We don't really use the latter so i am ok with just marking stuff with closed when we release. Like to hear what others thing about this before proceeding though. Jody -- Justin Deoliveira OpenGeo - http://opengeo.org Enterprise support for open source geospatial. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
Re: [Geoserver-devel] status of release scripts
On Wed, May 16, 2012 at 1:53 AM, Andrea Aime andrea.a...@geo-solutions.itwrote: On Wed, May 16, 2012 at 7:55 AM, Justin Deoliveira jdeol...@opengeo.org wrote: The end result is all the artifacts wind up in a single directory matching the release name. For example: http://gridlock.opengeo.org/geoserver/release/2.2-beta2/ Looking good :-) The above list of tasks is an overview, the main release script actually does a bit more which is what i would like to discuss. The script also checks jira and determines two things: 1. is the release actually released on jira 2. does the release have any open/unresolved issues filed against it To do so it uses the jira rest api (which is nice btw). This is something I would like feedback on. We have a couple of options here on what to do with respect to jira. 1. don't do any checks against jira, and leave that a manual process 2. halt the build if the release is not released in jira or there are unresolved issues, making releasing on a jira manually a precursor to running the script 3. mark the release as released in jira, and push back any unresolved issues to the next version Hum... I don't remember ever doing a release where all the tickets assigned to it were resolved. I'd go for 3, the release is not happening by itself anyways, and hopefully there will be a bit of discussion on the ml before someone pushes the big red release button. To clarify i wasn't implying that we fix all issues assigned to the version being released, just move them. The moving being manual or automatic. Sounds like people like the automatic approach so will push on with that. The second thing I would like to discuss is how we handle the README. Currently when we release we update the README on the main branch (and not the tag). I would kind of like to avoid committing anything to the main branch in an automated process. For example what happens if we run the release, it fails and we run it again. We have to check if the readme has been already updated, etc... While certainly possible with some script mojo it seems like an error prone process I would like to avoid. So what I would like to propose is the following. We don't ever update the README on the main release branch. We leave it generic with information about the project, where to find the docs, etc... On the tag we create we would generate a new file called RELEASE_NOTES.txt (or something to that effect) that contains the stuff we usually put in the README like revision info and the new and noteworthy info for that release. To generate the new and noteworthy for the release notes i was thinking we could come up with a heuristic. One such heuristic could be to pull out any new feature or improvement, and perhaps any issue that is a high priority one. Anyways, you get the idea. And naturally it would contain the link to the entire changelog from jira like we do now. Perhaps we should just stick to simple and put the revision info and the changelog link in the release notes. Probably what I will do for a first pass if people are into the idea. Hum... this file is going to be inside the packages and on sourceforge, but wondering how many people are actually going to read it? A link to the release notes on Jira seems reasonable to me. Cool. I think the same... it would be interesting to know how many people actually do read that file. Alternatively we allow the releaser to compose a narrative that is going to be included in the README file, possibly the same message that is then going to be sent to the mailing lists and reused in the blogs. Yeah, i thought about this as well. I will look into some of the options here. Good point about using it for the notifications as well. Those are the two main things I need input on thus far. Aside from the above stuff the main thing left to do is to do the upload to sourceforge. I think this will be pretty straight forward. The idea is that there would be a second release job (called geoserver-release-publish or something) that would take the release tag as input and basically just upload all the artifacts from the distribution directory to sourceforge. The idea is that after the main release job (and the hudson installer jobs) complete the person doing the release would do a quick sanity check on the artifacts, fix issues, etc... Once completed the release publish job would do the final push. Yep, sounds good to me. Oh, and another TODO will be to take the associated geotools info (version, revision, etc...) and ensure that is what is included in the geoserver build. I guess same goes for geowebcache as well. Yep That is it for now. Looking forward to hearing feedback. Nothing in particular. Kudos to your bash-fu and pretty excited about this new work becoming real :) Cheers Andrea -- Ing. Andrea Aime GeoSolutions
Re: [Geoserver-devel] status of release scripts
On Wed, May 16, 2012 at 1:54 AM, Jody Garnett jody.garn...@gmail.comwrote: The second thing I would like to discuss is how we handle the README. Currently when we release we update the README on the main branch (and not the tag). I would kind of like to avoid committing anything to the main branch in an automated process. For example what happens if we run the release, it fails and we run it again. We have to check if the readme has been already updated, etc... While certainly possible with some script mojo it seems like an error prone process I would like to avoid. So what I would like to propose is the following. We don't ever update the README on the main release branch. We leave it generic with information about the project, where to find the docs, etc... On the tag we create we would generate a new file called RELEASE_NOTES.txt (or something to that effect) that contains the stuff we usually put in the README like revision info and the new and noteworthy info for that release. To generate the new and noteworthy for the release notes i was thinking we could come up with a heuristic. One such heuristic could be to pull out any new feature or improvement, and perhaps any issue that is a high priority one. Anyways, you get the idea. And naturally it would contain the link to the entire changelog from jira like we do now. Perhaps we should just stick to simple and put the revision info and the changelog link in the release notes. Probably what I will do for a first pass if people are into the idea. I would look a lot better if you could generate a README.html using: - the boiler plate README from the main branch - idea: pull the release description from JIRA (if it is available ) and use it for the release specific description. - a link to the release notes in JIRA; optionally include a list of the closed items (as a highlights) and then offer a link through to JIRA for more information - (Wish) the features are often best summed up in the associated blog posts that have been made; could we troll the various geoserver, opengeo, geo-solutions blogs for a tag matching the release number - and provide links out to that information. Thanks for the hard work on this justin; when I was on IRC I saw geotools patches still being applied - is the coast clear now? Yup, had a few patches yesterday i wanted to get in, all good to go know though. Jody -- Justin Deoliveira OpenGeo - http://opengeo.org Enterprise support for open source geospatial. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Geoserver-devel mailing list Geoserver-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/geoserver-devel
[Geoserver-devel] status of release scripts
Hi all, I have been working on the scripts to automate our release process. Here is where things are at. First, if you want to take a look at the scripts they are in my github repo in a branch called build_scripts: https://github.com/jdeolive/geoserver/tree/build_scripts/build Here is how they work. And I actually haven't setup all the hudson jobs yet, but the underlying scripts are working. So here is how it will work. On our main hudson server there is a job called geoserver-release: http://hudson.opengeo.org/hudson/job/geoserver-release/ The job takes the following parameters: * branch to release from * revision to tag * release name The underlying scripts actually take more parameters but i will get to those later. When run the above job does the following: 1. creates the tag from the specified parameters 2. checks out the tag and preps for release, updating version numbers, etc... 3. performs the main release build 4. assembles artifacts and uploads them to a distribution directory, currently located here: http://gridlock.opengeo.org/geoserver/release/ 5. Fires off two installer jobs (one for windows, one for osx) on remote hudsons. The two hudsons are currently on our internal network so i can't provide access to them. 6. When the installer jobs complete they upload the artifacts to the same distribution directory specified above. The end result is all the artifacts wind up in a single directory matching the release name. For example: http://gridlock.opengeo.org/geoserver/release/2.2-beta2/ The above list of tasks is an overview, the main release script actually does a bit more which is what i would like to discuss. The script also checks jira and determines two things: 1. is the release actually released on jira 2. does the release have any open/unresolved issues filed against it To do so it uses the jira rest api (which is nice btw). This is something I would like feedback on. We have a couple of options here on what to do with respect to jira. 1. don't do any checks against jira, and leave that a manual process 2. halt the build if the release is not released in jira or there are unresolved issues, making releasing on a jira manually a precursor to running the script 3. mark the release as released in jira, and push back any unresolved issues to the next version More options welcome. The second thing I would like to discuss is how we handle the README. Currently when we release we update the README on the main branch (and not the tag). I would kind of like to avoid committing anything to the main branch in an automated process. For example what happens if we run the release, it fails and we run it again. We have to check if the readme has been already updated, etc... While certainly possible with some script mojo it seems like an error prone process I would like to avoid. So what I would like to propose is the following. We don't ever update the README on the main release branch. We leave it generic with information about the project, where to find the docs, etc... On the tag we create we would generate a new file called RELEASE_NOTES.txt (or something to that effect) that contains the stuff we usually put in the README like revision info and the new and noteworthy info for that release. To generate the new and noteworthy for the release notes i was thinking we could come up with a heuristic. One such heuristic could be to pull out any new feature or improvement, and perhaps any issue that is a high priority one. Anyways, you get the idea. And naturally it would contain the link to the entire changelog from jira like we do now. Perhaps we should just stick to simple and put the revision info and the changelog link in the release notes. Probably what I will do for a first pass if people are into the idea. Those are the two main things I need input on thus far. Aside from the above stuff the main thing left to do is to do the upload to sourceforge. I think this will be pretty straight forward. The idea is that there would be a second release job (called geoserver-release-publish or something) that would take the release tag as input and basically just upload all the artifacts from the distribution directory to sourceforge. The idea is that after the main release job (and the hudson installer jobs) complete the person doing the release would do a quick sanity check on the artifacts, fix issues, etc... Once completed the release publish job would do the final push. Oh, and another TODO will be to take the associated geotools info (version, revision, etc...) and ensure that is what is included in the geoserver build. I guess same goes for geowebcache as well. That is it for now. Looking forward to hearing feedback. -Justin -- Justin Deoliveira OpenGeo - http://opengeo.org Enterprise support for open source geospatial. -- Live Security Virtual Conference Exclusive live
