Re: Changes Mess
Does anyone feel like there is consensus here? On Dec 8, 2010, at 1:42 PM, Chris Hostetter wrote: I'm going to side step the use jira to generate changes.txt debate, and focus on what i think is the broader problem with a simpler fix. FWIW, i like CHANGES.txt myself, i think the jira generate pages compliment it, and give you a differnet view of the same info, but i prefer CHANGES.txt because it hsa a human element to it... if we do decide to go the jira automated route, the problems (and ultimate decision making needed) that i point out below still needs to be dealt with *** : when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found : out that 3.x changes differ immense between the trunk changes.txt and the : 3.x changes.txt. Some entries are missing in the 3.x branch, but are : available in trunk's 3.x part or other entries using new trunk class names : are between 3.x changes in trunk. I tried to point this out a whle back when i discovered there even *was* a 3x section in trunk's CHANGES.txt http://search.lucidimagination.com/search/document/9a9b1327fe281305/solr_changes_3_1_4_0 The problem seems entirely of our own making by assuming that the 3x section of trunk/CHANGES.txt should be updated when things were commited to the 3x branch. as i said before... If we commit to the 3x branch, add to the 3x CHANGES.txt If we commit to the trunk, add to the trunk CHANGES.txt the 3x section of trunks CHANGES.txt should never have existed. There are two very simple solutions to cleaning up the current mess, depending on wether we really want to have CHANGES.txt list the full history of all changes to the product in all versions, ad infinitum; or if (as has been suggested before but i don't think ever decided on) we want to start pruning CHANGES.txt to focus only on hte most recent relase (or current branch of development) *** If we do want CHANGES.txt to be a historical record *** the 3x section of trunks CHANGES.txt should just be riped out now and replaced with the final 3.0.3 section of the CHANGES.txt that was included in the 3.0.3. the 2.9.4 section of the CHANGES.txt that was included in the 2.9.4 release should also be included in trunk's CHANGES.txt the process going forward on each release should be that after a release is done, cut/past the section of changes from that release to the HEAD of any branches that have higher release numbers. Arguably we could dedup identical entries so that each entry appears only once (i suggested this before and now think i was wrong), but that loses information: people who see that LUCENE-9876 was fixed in 2.9.4 have no context of wether that fix was in 3.0.2 or 3.0.3 -- to have that full context it makes sense for each fix commited in a branch to actually be listed in the first release on that branch that the fix was in. *** If we want CHANGES.txt to focus on the current version/branch *** We do nothing special at all. We rip out the 3x section of trunk's CHANGES.txt and forget about it. we rip out everything pre 4.x from trunk's CHANGES.txt and let people who are interested in the CHANGES.txt from those versions go look them up there. (we could/should start publishing them on the website in a more conspicous place to make them more accessible) we move forward only ever adding things to CHANGES.txt on a branch if that change is actually commited to the branch, and we do nothing special when releases happen. *** ...my vote is to start having CHANGES.txt focus solely on the changes commited to the branches that lead to that release (so 4.1.1's CHANGES.txt will list 4.0, 4.1, 4.2, and 4.2.1 - but not 3.1, or 4.1.1) but i could also be pursuaded that it should only be that exact release (4.1.1 only contains commits specific to 4.1.1) I think that general approach is the only sane thing to do (both from a developer sanity stand point, and from a clarity to users standpoint about what versions on what branches contain what bug fixes/features) now that we are knowingly and deliberately maintaining multiple branches under active development. if we publish releases containing CHANGES.txt file that contains a sequential list of versions like 2.9.4, 3.0.2, 3.1, 3.2, 4.0, 4.1 then people are going to (understandable) assume that if the 3.2 section says it fixes a bug, that means the bug fix was included in 4.0 -- and that won't always be true, it might have been fixed in 3.2 and 4.1, but 4.0 might still have it. (that scenerio has always been possible with our point releases, but with the multiple active branches the frequency of that scenerio is going to skyrocket) -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
On Dec 8, 2010, at 10:42 AM, Chris Hostetter wrote: *** ...my vote is to start having CHANGES.txt focus solely on the changes commited to the branches that lead to that release (so 4.1.1's CHANGES.txt will list 4.0, 4.1, 4.2, and 4.2.1 - but not 3.1, or 4.1.1) but i could also be pursuaded that it should only be that exact release (4.1.1 only contains commits specific to 4.1.1) I think that general approach is the only sane thing to do (both from a developer sanity stand point, and from a clarity to users standpoint about what versions on what branches contain what bug fixes/features) now that we are knowingly and deliberately maintaining multiple branches under active development. Makes sense to me. History is easily obtained from other places and people will likely have the context of older changes already. -Grant - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
I was going by what Uwe said :) But if indeed we can make our own fields, this seems like a great solution? We could add a CHANGES entry field, and a maybe a CHANGES category (New feature, Bug fix, API change, Back-compat exception, etc.)? Mike On Tue, Dec 7, 2010 at 8:53 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Mike real quick I'm not sure what you mean by not having full control over JIRA. I know that you can customize the field types, issue types etc. You just ask infra and they take care of enabling perms for whomever would like to manage this for the project. Cheers, Chris Sent from my iPhone On Dec 7, 2010, at 1:03 PM, Michael McCandless luc...@mikemccandless.com wrote: Both approaches have tradeoffs... I love that the CHANGES today is carefully edited and as a result very consumable to users. The verbiage we put into a CHANGES entry is necessarily different from the title we put into Jira since it's a different audience. But I don't like the confusion over which section we put an entry into on all our branches, the work the RM must do to dedup (thanks Uwe!), etc. And we can't expect the RM to have to editorialize the changes from raw Jira after the fact... Could we do something hybrid? Since we have no full control over Jira, could we eg append a comment on the the Jira issue starting with CHANGES: , if it needs a changes entry (many issues do not)? Then on releasing we can pull from all issues fixed on that release and coalesce the CHANGES? This would fix the confusion of duping CHANGES entries across releases, etc... it's just pushing a button. Mike On Mon, Dec 6, 2010 at 11:23 PM, Shai Erera ser...@gmail.com wrote: Jumping in late to this thread, though I've read most of it. As a user and committer, I find the current CHANGES very convenient! It's very easy for me to read what has changed in 3.0, and very easy for me to put a CHANGES entry whenever I work on something that warrants such entry. And if an issue is back ported all the way 'till 1.4, then IMO it should contain an entry in each CHANGES (of each release). Users who download 2.9.4 need to be able to read what has changed since 2.9.3, in a clear and concise way. Which as far as I'm concerned is the current situation and I'm happy with it. Back porting issues is usually a simple svn merge, and in more complex cases, even if it's done manually, the CHANGES entry is the easiest to copy over. I don't think we should work hard to make JIRA produce the CHANGES for us - in the end of the day, JIRA is our bug tracking system, and it should remain like that. The CHANGES entry need to summarize the change to the reader, and combined with the issue number it gives enough info. If one wants, one can load the issue in JIRA and read the full correspondence. So I'm +1 for keeping things as they are, and paying attention to put the entries in all applicable CHANGES. Shai On Monday, December 6, 2010, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Robert, I feel ya. +1 to releasing more often! :) Cheers, Chris On Dec 6, 2010, at 8:31 AM, Robert Muir wrote: On Mon, Dec 6, 2010 at 11:20 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Yeah in the end all I can say is that you basically get out of JIRA what you put into it. What you call extra work is just something that I would do anyways working on some of my projects. I'm not saying it's *not difficult* and super easy, but we've just decided in those cases to invest time into the issue management system so that we can get the reports we want out of it. I've seen this work both ways: in the early days of Nutch there were intense debates on simply moving everything to JIRA versus maintaining a disconnected CHANGES.txt file. I've heard all the arguments (many times over) on both sides including tidbits like oh I don't want to go to a separate URL as a consumer of software just to see what changed in it to what's so hard about doing a curl or wget on an Internet-connected system which most of our software assumes nowadays?, those types of things. When the dust cleared, I think I like the approach we use in Tika (and that I use in a number of projects at JPL) which is just to maintain the information in JIRA. It's worked for us since it's a single source to curate that type of information; it produces very useable reports (not perfect, but useable) that are good enough for us in terms of trading between the different properties we want to maximize (user contribution acknowledgement, change history, etc.) I agree with what you said, and as I mentioned before I'm not opposed to the idea at all. But if we are going to rely on JIRA more to produce this documentation, we need to make some major changes to how we use it, to avoid some of the problems I mentioned... The scariest part to me about this approach is
Re: Changes Mess
On Tue, Dec 7, 2010 at 3:14 PM, Steven A Rowe sar...@syr.edu wrote: On 12/7/2010 at 3:03 PM, Michael McCandless wrote: Could we do something hybrid? Since we have no full control over Jira, could we eg append a comment on the the Jira issue starting with CHANGES: , if it needs a changes entry (many issues do not)? Can't we add a field? I see we already have a couple of Lucene-only fields now: patch available (how is this helpful?) and new (what does this one mean, anyway?) I think for patch available the idea is to mark issues w/ a patch pending instead of the [PATCH] which is sometimes used? But, it seems sort of silly since it's not automagically set by whether a patch has been attached to the issue, and people don't know to set it. I have no idea why we have New! Mike - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
I'm going to side step the use jira to generate changes.txt debate, and focus on what i think is the broader problem with a simpler fix. FWIW, i like CHANGES.txt myself, i think the jira generate pages compliment it, and give you a differnet view of the same info, but i prefer CHANGES.txt because it hsa a human element to it... if we do decide to go the jira automated route, the problems (and ultimate decision making needed) that i point out below still needs to be dealt with *** : when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found : out that 3.x changes differ immense between the trunk changes.txt and the : 3.x changes.txt. Some entries are missing in the 3.x branch, but are : available in trunk's 3.x part or other entries using new trunk class names : are between 3.x changes in trunk. I tried to point this out a whle back when i discovered there even *was* a 3x section in trunk's CHANGES.txt http://search.lucidimagination.com/search/document/9a9b1327fe281305/solr_changes_3_1_4_0 The problem seems entirely of our own making by assuming that the 3x section of trunk/CHANGES.txt should be updated when things were commited to the 3x branch. as i said before... If we commit to the 3x branch, add to the 3x CHANGES.txt If we commit to the trunk, add to the trunk CHANGES.txt the 3x section of trunks CHANGES.txt should never have existed. There are two very simple solutions to cleaning up the current mess, depending on wether we really want to have CHANGES.txt list the full history of all changes to the product in all versions, ad infinitum; or if (as has been suggested before but i don't think ever decided on) we want to start pruning CHANGES.txt to focus only on hte most recent relase (or current branch of development) *** If we do want CHANGES.txt to be a historical record *** the 3x section of trunks CHANGES.txt should just be riped out now and replaced with the final 3.0.3 section of the CHANGES.txt that was included in the 3.0.3. the 2.9.4 section of the CHANGES.txt that was included in the 2.9.4 release should also be included in trunk's CHANGES.txt the process going forward on each release should be that after a release is done, cut/past the section of changes from that release to the HEAD of any branches that have higher release numbers. Arguably we could dedup identical entries so that each entry appears only once (i suggested this before and now think i was wrong), but that loses information: people who see that LUCENE-9876 was fixed in 2.9.4 have no context of wether that fix was in 3.0.2 or 3.0.3 -- to have that full context it makes sense for each fix commited in a branch to actually be listed in the first release on that branch that the fix was in. *** If we want CHANGES.txt to focus on the current version/branch *** We do nothing special at all. We rip out the 3x section of trunk's CHANGES.txt and forget about it. we rip out everything pre 4.x from trunk's CHANGES.txt and let people who are interested in the CHANGES.txt from those versions go look them up there. (we could/should start publishing them on the website in a more conspicous place to make them more accessible) we move forward only ever adding things to CHANGES.txt on a branch if that change is actually commited to the branch, and we do nothing special when releases happen. *** ...my vote is to start having CHANGES.txt focus solely on the changes commited to the branches that lead to that release (so 4.1.1's CHANGES.txt will list 4.0, 4.1, 4.2, and 4.2.1 - but not 3.1, or 4.1.1) but i could also be pursuaded that it should only be that exact release (4.1.1 only contains commits specific to 4.1.1) I think that general approach is the only sane thing to do (both from a developer sanity stand point, and from a clarity to users standpoint about what versions on what branches contain what bug fixes/features) now that we are knowingly and deliberately maintaining multiple branches under active development. if we publish releases containing CHANGES.txt file that contains a sequential list of versions like 2.9.4, 3.0.2, 3.1, 3.2, 4.0, 4.1 then people are going to (understandable) assume that if the 3.2 section says it fixes a bug, that means the bug fix was included in 4.0 -- and that won't always be true, it might have been fixed in 3.2 and 4.1, but 4.0 might still have it. (that scenerio has always been possible with our point releases, but with the multiple active branches the frequency of that scenerio is going to skyrocket) -Hoss - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
Both approaches have tradeoffs... I love that the CHANGES today is carefully edited and as a result very consumable to users. The verbiage we put into a CHANGES entry is necessarily different from the title we put into Jira since it's a different audience. But I don't like the confusion over which section we put an entry into on all our branches, the work the RM must do to dedup (thanks Uwe!), etc. And we can't expect the RM to have to editorialize the changes from raw Jira after the fact... Could we do something hybrid? Since we have no full control over Jira, could we eg append a comment on the the Jira issue starting with CHANGES: , if it needs a changes entry (many issues do not)? Then on releasing we can pull from all issues fixed on that release and coalesce the CHANGES? This would fix the confusion of duping CHANGES entries across releases, etc... it's just pushing a button. Mike On Mon, Dec 6, 2010 at 11:23 PM, Shai Erera ser...@gmail.com wrote: Jumping in late to this thread, though I've read most of it. As a user and committer, I find the current CHANGES very convenient! It's very easy for me to read what has changed in 3.0, and very easy for me to put a CHANGES entry whenever I work on something that warrants such entry. And if an issue is back ported all the way 'till 1.4, then IMO it should contain an entry in each CHANGES (of each release). Users who download 2.9.4 need to be able to read what has changed since 2.9.3, in a clear and concise way. Which as far as I'm concerned is the current situation and I'm happy with it. Back porting issues is usually a simple svn merge, and in more complex cases, even if it's done manually, the CHANGES entry is the easiest to copy over. I don't think we should work hard to make JIRA produce the CHANGES for us - in the end of the day, JIRA is our bug tracking system, and it should remain like that. The CHANGES entry need to summarize the change to the reader, and combined with the issue number it gives enough info. If one wants, one can load the issue in JIRA and read the full correspondence. So I'm +1 for keeping things as they are, and paying attention to put the entries in all applicable CHANGES. Shai On Monday, December 6, 2010, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Robert, I feel ya. +1 to releasing more often! :) Cheers, Chris On Dec 6, 2010, at 8:31 AM, Robert Muir wrote: On Mon, Dec 6, 2010 at 11:20 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Yeah in the end all I can say is that you basically get out of JIRA what you put into it. What you call extra work is just something that I would do anyways working on some of my projects. I'm not saying it's *not difficult* and super easy, but we've just decided in those cases to invest time into the issue management system so that we can get the reports we want out of it. I've seen this work both ways: in the early days of Nutch there were intense debates on simply moving everything to JIRA versus maintaining a disconnected CHANGES.txt file. I've heard all the arguments (many times over) on both sides including tidbits like oh I don't want to go to a separate URL as a consumer of software just to see what changed in it to what's so hard about doing a curl or wget on an Internet-connected system which most of our software assumes nowadays?, those types of things. When the dust cleared, I think I like the approach we use in Tika (and that I use in a number of projects at JPL) which is just to maintain the information in JIRA. It's worked for us since it's a single source to curate that type of information; it produces very useable reports (not perfect, but useable) that are good enough for us in terms of trading between the different properties we want to maximize (user contribution acknowledgement, change history, etc.) I agree with what you said, and as I mentioned before I'm not opposed to the idea at all. But if we are going to rely on JIRA more to produce this documentation, we need to make some major changes to how we use it, to avoid some of the problems I mentioned... The scariest part to me about this approach is that we unfortunately have very long release cycles. So i'm worried about this documentation being generated and fixed at release time versus incrementally where its fresh in our mind... thats a lot of editing and filtering to do. Obviously I feel this would be mitigated and other things much better if we released more often but thats a harder problem, this is just the situation as it is now. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory
RE: Changes Mess
I will checkout as I have an Project Admin account. But from my JIRA knowledge you need to at least have global admin rights to add the fields you can then activate in specific projects like LUCENE. - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de -Original Message- From: Steven A Rowe [mailto:sar...@syr.edu] Sent: Tuesday, December 07, 2010 9:15 PM To: dev@lucene.apache.org Subject: RE: Changes Mess On 12/7/2010 at 3:03 PM, Michael McCandless wrote: Could we do something hybrid? Since we have no full control over Jira, could we eg append a comment on the the Jira issue starting with CHANGES: , if it needs a changes entry (many issues do not)? Can't we add a field? I see we already have a couple of Lucene-only fields now: patch available (how is this helpful?) and new (what does this one mean, anyway?) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
CHANGES file: LUCENE-2658: Exceptions while processing term vectors enabled for multiple fields could lead to invalid ArrayIndexOutOfBoundsExceptions. JIRA description: LUCENE-2658: TestIndexWriterExceptions random failure: AIOOBE in ByteBlockPool.allocSlice So you see the story, i hit a random test failure and just opened an issue describing that the test randomly failed. Mike then went and fixed it and wrote up a CHANGES.txt entry thats significantly better to the users. In order for us to use JIRA here, we would have to do a lot of JIRA-editing and re-organizing I think, and probably create a lot of unnecessary issues. What's the difference between Mike going and writing up a more informative CHANGES.txt entry than say updating JIRA with the information from that entry to have a more descriptive title? Also, besides issue titles, there is also a way to capture anyone that's been involved in the JIRA cycle (comment, issue created, etc.) as part of the contribution report that's probably *even more* inclusive than what you guys are currently doing. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
Would you mind naming these Apache projects? I'd like to take a look. Tika, Nutch, OODT. Cheers, Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
On Mon, Dec 6, 2010 at 9:56 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: What's the difference between Mike going and writing up a more informative CHANGES.txt entry than say updating JIRA with the information from that entry to have a more descriptive title? Well, you are right, but its another modification to JIRA (an edit). And then there are more examples like this: CHANGES: * LUCENE-2650: Added extra safety to MMapIndexInput clones to prevent accessing an unmapped buffer if the input is closed JIRA: * LUCENE-2650: improve windows defaults in FSDirectory The jira is *CORRECT*. While working on the issue i discovered we could trivially add some extra safety. So i backported the extra safety to all branches. In this case i would have to split my patch in half and create another JIRA issue for this very trivial change? Just saying, to do what you are saying (by the way, I'm not opposed to the idea!), we would have to change the way we use JIRA and increase noise to the mailing list. There are quite a few examples like this: e.g. this jira release notes say this: [LUCENE-2055] - Fix buggy stemmers and Remove duplicate analysis functionality. But i certainly didn't do this in a bugfix release! what actually happened is in contrib/CHANGES.txt: * LUCENE-2055: Add documentation noting that the Dutch and French stemmers in contrib/analyzers do not implement the Snowball algorithm correctly, and recommend to use the equivalents in contrib/snowball if possible. So I don't know how jira would handle this case? because we merged contrib/snowball with contrib/analyzers in 3.1 i would have to create a separate jira issue just so that 3.1 has the correct description/path name in its release notes? and in 4.0 i'd have to create a third duplicate JIRA issue because we merged all the analyzers, so there it needs to refer to modules/analysis? - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
On Dec 6, 2010, at 9:58 AM, Mattmann, Chris A (388J) wrote: Would you mind naming these Apache projects? I'd like to take a look. Tika, Nutch, OODT. Add in Mahout. I believe Hadoop does too. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
On Dec 5, 2010, at 12:18 PM, Robert Muir wrote: On Sun, Dec 5, 2010 at 12:08 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Mark, RE: the credit system. JIRA provides a contribution report here, like this one that I generated for Lucene 3.1: My concern with this is that it leaves out important email contributors. I think we probably miss these as is too. br/ Note, however, in my proposal, one can still call out specific things. We could for instance have a Contributors section and just add names to it. I just think we put too much minutiae in CHANGES and it is a real burden to deal with it across branches b/c there are always massive conflicts and it requires you to look up every last change to recall which version it is in. IMO, JIRA should be the system of record for all bug discussions. Discussions that happen on email can easily be pointed to using any one of our many mail archive systems. Our new Changes could be structured like below. The important thing about this approach is that it can all more or less be written at release time other than the contributor list and perhaps the back compat section. /snip = Version X.Y = Brief Intro == Dependencies == Junit 4.4 == New Features == * Magic search was implemented == Backward Compatibility Breaks == * Blah, blah, blah == Significant Changes == * We've replaced the inverted index with a giant array == Contributors == (alphabetical order) Joe Schmoe (optionally cite an issue number) Jane Doe Optionally paste in the list from JIRA == Full Changes List == * LINK TO JIRA /snip -Grant - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
This is out of thread, but I realized that some entries for DIH are in solr/CHANGES.txt. These should go solr/contrib/dataimporthandler/CHANGES.txt (Some of them are my fault). I also found that solr/contrib/*/CHANGES.txt have 1.5-dev title. These should be 4.0-dev or 3.1-dev. I'll open a ticket. Koji -- http://www.rondhuit.com/en/ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
Jumping in late to this thread, though I've read most of it. As a user and committer, I find the current CHANGES very convenient! It's very easy for me to read what has changed in 3.0, and very easy for me to put a CHANGES entry whenever I work on something that warrants such entry. And if an issue is back ported all the way 'till 1.4, then IMO it should contain an entry in each CHANGES (of each release). Users who download 2.9.4 need to be able to read what has changed since 2.9.3, in a clear and concise way. Which as far as I'm concerned is the current situation and I'm happy with it. Back porting issues is usually a simple svn merge, and in more complex cases, even if it's done manually, the CHANGES entry is the easiest to copy over. I don't think we should work hard to make JIRA produce the CHANGES for us - in the end of the day, JIRA is our bug tracking system, and it should remain like that. The CHANGES entry need to summarize the change to the reader, and combined with the issue number it gives enough info. If one wants, one can load the issue in JIRA and read the full correspondence. So I'm +1 for keeping things as they are, and paying attention to put the entries in all applicable CHANGES. Shai On Monday, December 6, 2010, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hey Robert, I feel ya. +1 to releasing more often! :) Cheers, Chris On Dec 6, 2010, at 8:31 AM, Robert Muir wrote: On Mon, Dec 6, 2010 at 11:20 AM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Yeah in the end all I can say is that you basically get out of JIRA what you put into it. What you call extra work is just something that I would do anyways working on some of my projects. I'm not saying it's *not difficult* and super easy, but we've just decided in those cases to invest time into the issue management system so that we can get the reports we want out of it. I've seen this work both ways: in the early days of Nutch there were intense debates on simply moving everything to JIRA versus maintaining a disconnected CHANGES.txt file. I've heard all the arguments (many times over) on both sides including tidbits like oh I don't want to go to a separate URL as a consumer of software just to see what changed in it to what's so hard about doing a curl or wget on an Internet-connected system which most of our software assumes nowadays?, those types of things. When the dust cleared, I think I like the approach we use in Tika (and that I use in a number of projects at JPL) which is just to maintain the information in JIRA. It's worked for us since it's a single source to curate that type of information; it produces very useable reports (not perfect, but useable) that are good enough for us in terms of trading between the different properties we want to maximize (user contribution acknowledgement, change history, etc.) I agree with what you said, and as I mentioned before I'm not opposed to the idea at all. But if we are going to rely on JIRA more to produce this documentation, we need to make some major changes to how we use it, to avoid some of the problems I mentioned... The scariest part to me about this approach is that we unfortunately have very long release cycles. So i'm worried about this documentation being generated and fixed at release time versus incrementally where its fresh in our mind... thats a lot of editing and filtering to do. Obviously I feel this would be mitigated and other things much better if we released more often but thats a harder problem, this is just the situation as it is now. - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
Hi Mark, RE: the credit system. JIRA provides a contribution report here, like this one that I generated for Lucene 3.1: http://s.apache.org/BpL Just click on Reports Contribution Report in the upper right of JIRA on the main project summary page. We've been using this in Tika since the beginning to indicate contributions from folks and it's worked well. Cheers, Chris On Dec 4, 2010, at 10:03 PM, Mark Miller wrote: I like this idea myself - it would encourage better JIRA summaries and reduce duplication. It's easy to keep a mix of old and new too - keep the things that Grant mentions in CHANGES.txt (back compat migration, misc info), but you can also just export a text Changes from JIRA at release and add that (along with a link). Certainly nice to have a 'hard' copy. https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12315147styleName=TextprojectId=12310110Create=Create The only thing I don't like is the loss of the current credit system - I like that better than the crawl through JIRA method. I think prominent credits are a good encouragement for new contributors. Any comments on that? - Mark On 12/2/10 11:46 AM, Grant Ingersoll wrote: I think we should drop the item by item change list and instead focus on 3 things: 1. Prose describing the new features (see Tika's changes file for instance) and things users should pay special attention to such as when they might need to re-index. 2. Calling out explicit compatibility breaks 3. A Pointer to full list of changes in JIRA. Alternatively, I believe there is a way in JIRA to export/generate a summary of all issues fixed. #1 can be done right before release simply by going through #3 and doing the appropriate wordsmithing. #2 should be tracked as it is found. It's kind of silly that we have all this duplication of effort built in, not too mention having to track it across two branches. We do this over in Mahout and I think it works pretty well and reduces the duplication quite a bit since everything is already in JIRA and JIRA produces nice summaries too. It also encourages people to track things better in JIRA. #1 above also lends itself well as the basis of press releases/blogs/etc. -Grant On Dec 1, 2010, at 11:54 AM, Michael McCandless wrote: So, going forward... When committing an issue that needs a changes entry, where are we supposed to put it? EG if it's a bug fix that we'll backport all the way to 2.9.x... where does it go? If it's a new feature/API that's going to 3.x and trunk... only in 3.x's CHANGES? Mike On Wed, Dec 1, 2010 at 9:22 AM, Uwe Schindleru...@thetaphi.de wrote: Hi all, when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found out that 3.x changes differ immense between the trunk changes.txt and the 3.x changes.txt. Some entries are missing in the 3.x branch, but are available in trunk's 3.x part or other entries using new trunk class names are between 3.x changes in trunk. I copied over the 3.x branch CHANGES.txt over trunks 3.x section and attached a patch of this. What should we do? Its messy :( Most parts seem to be merge failures. We should go through all those diff'ed issues and check where they were really fixed (3.x or trunk) and move the entries accordingly. After that in the 3.x branch and in trunk's 3.x section of CHANGES.txt should be identical text! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++
Re: Changes Mess
On Sun, Dec 5, 2010 at 12:08 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Mark, RE: the credit system. JIRA provides a contribution report here, like this one that I generated for Lucene 3.1: My concern with this is that it leaves out important email contributors. For example if a user reports a bug, we typically include their name in CHANGES.txt The user who reports the bug does the hard work of finding that there is a bug and reporting it to us. Additionally sometimes they do extra stuff, boiling the problem down to a certain piece of code, into a test case, etc, even if they don't know how to fix the bug. Then again, maybe they are a solr user who doesn't even know the java programming language but finds a nasty bug in lucene. In all cases I think if a user finds a bug and we fix it, its important we credit them as we should encourage people to find bugs :) - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Changes Mess
On 12/5/2010 at 12:19 PM, Robert Muir wrote: On Sun, Dec 5, 2010 at 12:08 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Mark, RE: the credit system. JIRA provides a contribution report here, like this one that I generated for Lucene 3.1: My concern with this is that it leaves out important email contributors. I agree, this is a serious problem. My additional problems with JIRA-generated changes: 1. Huge undifferentiated change lists are frightening and nearly useless, regardless of the quality of the descriptions. JIRA's issue types are: Bug, New Feature, Improvement, Test, Wish, Task Even if we used JIRA's issue types to group issues, they are not the same as Lucene's CHANGES.txt issue types: Changes in backwards compatibility policy, Changes in runtime behavior, API Changes, Documentation, Bug fixes, New features, Optimizations, Build, Test Cases, Infrastructure (I left out Requirements, last used in 2006 under release 1.9 RC1, since Build seems to have replaced it.) 2. There are now four separate CHANGES.txt files in the Lucene code base, excluding Solr and its modules (each of which has one of them). This number will only grow as more Lucene contribs become modules. The JIRA project components list is outdated / incomplete / has different granularity than the CHANGES.txt locations, so using it to group JIRA issues would not work because they don't align with Lucene/Solr components. 3. Some of the CHANGES.txt entries draw from multiple JIRA issues. From dev/trunk/lucene/CHANGES.txt: Trunk: 9 out of 56 include multiple JIRA issues 3.X: 7/94 3.0.0: 3/29 2.9.0: 9/153 I'm assuming a JIRA dump can't do this. 4. Some JIRA issues appear under multiple change categories in CHANGES.txt. From dev/trunk/lucene/CHANGES.txt: Trunk: 3 out of 68 multiply categorized 3.X: 9/102 3.0.0: 1/53 2.9.0: 20/166 A JIRA dump would not allow for multiple issue categorization, since JIRA only allows a single issue type to be assigned - I guess they are assumed to be mutually exclusive. Maybe our use of JIRA could be changed to address some of these problems, through addition of new fields and/or modification of existing fields' allowable values? Steve
Re: Changes Mess
Hi Steven, Yep, like you state below JIRA *could* be configured to deal with this. In all honesty, putting tons of thought and effort into how to precisely deal with the changes you specify below might be somewhat overkill. Cheers, Chris On Dec 5, 2010, at 12:17 PM, Steven A Rowe wrote: On 12/5/2010 at 12:19 PM, Robert Muir wrote: On Sun, Dec 5, 2010 at 12:08 PM, Mattmann, Chris A (388J) chris.a.mattm...@jpl.nasa.gov wrote: Hi Mark, RE: the credit system. JIRA provides a contribution report here, like this one that I generated for Lucene 3.1: My concern with this is that it leaves out important email contributors. I agree, this is a serious problem. My additional problems with JIRA-generated changes: 1. Huge undifferentiated change lists are frightening and nearly useless, regardless of the quality of the descriptions. JIRA's issue types are: Bug, New Feature, Improvement, Test, Wish, Task Even if we used JIRA's issue types to group issues, they are not the same as Lucene's CHANGES.txt issue types: Changes in backwards compatibility policy, Changes in runtime behavior, API Changes, Documentation, Bug fixes, New features, Optimizations, Build, Test Cases, Infrastructure (I left out Requirements, last used in 2006 under release 1.9 RC1, since Build seems to have replaced it.) 2. There are now four separate CHANGES.txt files in the Lucene code base, excluding Solr and its modules (each of which has one of them). This number will only grow as more Lucene contribs become modules. The JIRA project components list is outdated / incomplete / has different granularity than the CHANGES.txt locations, so using it to group JIRA issues would not work because they don't align with Lucene/Solr components. 3. Some of the CHANGES.txt entries draw from multiple JIRA issues. From dev/trunk/lucene/CHANGES.txt: Trunk: 9 out of 56 include multiple JIRA issues 3.X: 7/94 3.0.0: 3/29 2.9.0: 9/153 I'm assuming a JIRA dump can't do this. 4. Some JIRA issues appear under multiple change categories in CHANGES.txt. From dev/trunk/lucene/CHANGES.txt: Trunk: 3 out of 68 multiply categorized 3.X: 9/102 3.0.0: 1/53 2.9.0: 20/166 A JIRA dump would not allow for multiple issue categorization, since JIRA only allows a single issue type to be assigned - I guess they are assumed to be mutually exclusive. Maybe our use of JIRA could be changed to address some of these problems, through addition of new fields and/or modification of existing fields' allowable values? Steve ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
RE: Changes Mess
Hi Chris, On 12/5/2010 at 10:36 PM, Chris Mattman wrote: Yep, like you state below JIRA *could* be configured to deal with this. In all honesty, putting tons of thought and effort into how to precisely deal with the changes you specify below might be somewhat overkill. I think dumping CHANGES.txt in favor of output from a badly misconfigured issue tracking system would be foolish. One way to deal with the problem is to stay with CHANGES.txt. (We've been down this road before, and this is where we landed in the past.) Another would be to fix the issue tracking system. Yet another way would be to declare the problem non-existent and screw our users by insulting them with a honking great mass of changes without any indication about what they are or how they are inter-related. (You won't be surprised at this point, I think, by my -1 to this.) Steve
Re: Changes Mess
Yet another way would be to declare the problem non-existent and screw our users by insulting them with a honking great mass of changes without any indication about what they are or how they are inter-related. (You won't be surprised at this point, I think, by my -1 to this.) Right, I'm one of those users (have been in the past and am somewhat still) as well as a former member of the PMC and so acting like I'm suggesting screwing them over (them which would include me) by simply suggesting that solving this mess in completeness is intractable so you just have to go with a heuristic (which I'd argue spending oodles of time on isn't worth it) is also a bit insulting. I suggested that JIRA can handle this. We're using it in oh, about 2-3 Apache projects I'm on and it's working great. If you think it's a mess for all the stuff you put in the email, great, that's your prerogative. I'm just saying in my experience it hasn't been that bad. Chris ++ Chris Mattmann, Ph.D. Senior Computer Scientist NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA Office: 171-266B, Mailstop: 171-246 Email: chris.a.mattm...@nasa.gov WWW: http://sunset.usc.edu/~mattmann/ ++ Adjunct Assistant Professor, Computer Science Department University of Southern California, Los Angeles, CA 90089 USA ++ - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
I like this idea myself - it would encourage better JIRA summaries and reduce duplication. It's easy to keep a mix of old and new too - keep the things that Grant mentions in CHANGES.txt (back compat migration, misc info), but you can also just export a text Changes from JIRA at release and add that (along with a link). Certainly nice to have a 'hard' copy. https://issues.apache.org/jira/secure/ReleaseNote.jspa?version=12315147styleName=TextprojectId=12310110Create=Create The only thing I don't like is the loss of the current credit system - I like that better than the crawl through JIRA method. I think prominent credits are a good encouragement for new contributors. Any comments on that? - Mark On 12/2/10 11:46 AM, Grant Ingersoll wrote: I think we should drop the item by item change list and instead focus on 3 things: 1. Prose describing the new features (see Tika's changes file for instance) and things users should pay special attention to such as when they might need to re-index. 2. Calling out explicit compatibility breaks 3. A Pointer to full list of changes in JIRA. Alternatively, I believe there is a way in JIRA to export/generate a summary of all issues fixed. #1 can be done right before release simply by going through #3 and doing the appropriate wordsmithing. #2 should be tracked as it is found. It's kind of silly that we have all this duplication of effort built in, not too mention having to track it across two branches. We do this over in Mahout and I think it works pretty well and reduces the duplication quite a bit since everything is already in JIRA and JIRA produces nice summaries too. It also encourages people to track things better in JIRA. #1 above also lends itself well as the basis of press releases/blogs/etc. -Grant On Dec 1, 2010, at 11:54 AM, Michael McCandless wrote: So, going forward... When committing an issue that needs a changes entry, where are we supposed to put it? EG if it's a bug fix that we'll backport all the way to 2.9.x... where does it go? If it's a new feature/API that's going to 3.x and trunk... only in 3.x's CHANGES? Mike On Wed, Dec 1, 2010 at 9:22 AM, Uwe Schindleru...@thetaphi.de wrote: Hi all, when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found out that 3.x changes differ immense between the trunk changes.txt and the 3.x changes.txt. Some entries are missing in the 3.x branch, but are available in trunk's 3.x part or other entries using new trunk class names are between 3.x changes in trunk. I copied over the 3.x branch CHANGES.txt over trunks 3.x section and attached a patch of this. What should we do? Its messy :( Most parts seem to be merge failures. We should go through all those diff'ed issues and check where they were really fixed (3.x or trunk) and move the entries accordingly. After that in the 3.x branch and in trunk's 3.x section of CHANGES.txt should be identical text! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
I think we should drop the item by item change list and instead focus on 3 things: 1. Prose describing the new features (see Tika's changes file for instance) and things users should pay special attention to such as when they might need to re-index. 2. Calling out explicit compatibility breaks 3. A Pointer to full list of changes in JIRA. Alternatively, I believe there is a way in JIRA to export/generate a summary of all issues fixed. #1 can be done right before release simply by going through #3 and doing the appropriate wordsmithing. #2 should be tracked as it is found. It's kind of silly that we have all this duplication of effort built in, not too mention having to track it across two branches. We do this over in Mahout and I think it works pretty well and reduces the duplication quite a bit since everything is already in JIRA and JIRA produces nice summaries too. It also encourages people to track things better in JIRA. #1 above also lends itself well as the basis of press releases/blogs/etc. -Grant On Dec 1, 2010, at 11:54 AM, Michael McCandless wrote: So, going forward... When committing an issue that needs a changes entry, where are we supposed to put it? EG if it's a bug fix that we'll backport all the way to 2.9.x... where does it go? If it's a new feature/API that's going to 3.x and trunk... only in 3.x's CHANGES? Mike On Wed, Dec 1, 2010 at 9:22 AM, Uwe Schindler u...@thetaphi.de wrote: Hi all, when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found out that 3.x changes differ immense between the trunk changes.txt and the 3.x changes.txt. Some entries are missing in the 3.x branch, but are available in trunk's 3.x part or other entries using new trunk class names are between 3.x changes in trunk. I copied over the 3.x branch CHANGES.txt over trunks 3.x section and attached a patch of this. What should we do? Its messy :( Most parts seem to be merge failures. We should go through all those diff'ed issues and check where they were really fixed (3.x or trunk) and move the entries accordingly. After that in the 3.x branch and in trunk's 3.x section of CHANGES.txt should be identical text! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org -- Grant Ingersoll http://www.lucidimagination.com - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
Re: Changes Mess
So, going forward... When committing an issue that needs a changes entry, where are we supposed to put it? EG if it's a bug fix that we'll backport all the way to 2.9.x... where does it go? If it's a new feature/API that's going to 3.x and trunk... only in 3.x's CHANGES? Mike On Wed, Dec 1, 2010 at 9:22 AM, Uwe Schindler u...@thetaphi.de wrote: Hi all, when merging changes done in 2.9.4/3.0.3 with current 3.x and trunk I found out that 3.x changes differ immense between the trunk changes.txt and the 3.x changes.txt. Some entries are missing in the 3.x branch, but are available in trunk's 3.x part or other entries using new trunk class names are between 3.x changes in trunk. I copied over the 3.x branch CHANGES.txt over trunks 3.x section and attached a patch of this. What should we do? Its messy :( Most parts seem to be merge failures. We should go through all those diff'ed issues and check where they were really fixed (3.x or trunk) and move the entries accordingly. After that in the 3.x branch and in trunk's 3.x section of CHANGES.txt should be identical text! Uwe - Uwe Schindler H.-H.-Meier-Allee 63, D-28213 Bremen http://www.thetaphi.de eMail: u...@thetaphi.de - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org - To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
