Re: DERBY-4331 and the 10.5.2.0 release
Thanks, Mike and Mamta for fixing this problem so quickly. Mike Matrigali wrote: Whoever is going to be release manager let me know if there is anything else that is needed for this issue. That would be me. I don't think anything else is needed. I will publish a webpage and schedule for 10.5.3.0 later this morning after the coffee kicks in and my mind unfogs. Thanks, -Rick
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Rick Hillegas wrote: Kathey Marsden wrote: I don't really care what we call it. There is no reason that the release notes have to describe the delta between the current release and the immediately preceding one. I can build a 10.5.3.0 whose release notes describes its delta from 10.5.1.1. I think that would be ok but would prefer if there were separate sections for each release. That way it would be clear where DERBY-4331 was fixed. Kathey I'm planning to highlight the small number of extra bugs which were fixed after 10.5.2.1--perhaps just by making them bold or maybe by separating them out into their own section as you suggest. That should be useful for the small number of users who will be upgrading from 10.5.2.1 and for our own archeology in the future. Regards, -Rick
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: Kathey Marsden wrote: I don't really care what we call it. There is no reason that the release notes have to describe the delta between the current release and the immediately preceding one. I can build a 10.5.3.0 whose release notes describes its delta from 10.5.1.1. I think that would be ok but would prefer if there were separate sections for each release. That way it would be clear where DERBY-4331 was fixed. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
I've checked in the 4331 fix into the 10.5 branch. It passed all tests in my environment. I would suggest waiting until it has passed automatic nightly regression runs on the various platforms and then cut a new apache release if all goes well. I will check tommorow and Saturday on the posted results of the 10.5 branch runs. The IBM nightly runs on trunk all passed last night with this change. The sun tinderbox on trunk also all passed. The other sun based trunk runs I checked looked to be in progress now. I am not sure when exactly the sun based jvm nightly's kick off, so it may take at least a couple of days. At this point I assume the new release would be just the current version of the 10.5 branch. I don't really care what the number is as long as it is a different bigger number than what someone may have already got their hands on. Whoever is going to be release manager let me know if there is anything else that is needed for this issue. /mikem
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: Kathey Marsden wrote: Jean T. Anderson wrote: The release happened, but there's no reason you can't quickly follow up with another release with bug fixes. Thanks Jean for the clarification that the release really did happen. I honestly wasn't sure it had. :-) I think since a release happened, I feel more strongly that we should keep release notes separate, not merge Jira versions, etc. Kathey I don't really care what we call it. There is no reason that the release notes have to describe the delta between the current release and the immediately preceding one. I can build a 10.5.3.0 whose release notes describes its delta from 10.5.1.1. I bet users would appreciate that -- they wouldn't have fetch just the release notes for each fix release. Get all the changes in one blast. -jean
Re: DERBY-4331 and the 10.5.2.0 release
Considering we aren't aware of how many people actually downloaded 10.5.2.0, then it might be wise to e-mail the user list and post a big warning with the 10.5.[2.1|3.0] release urging 10.5.2.0 users to upgrade to the new version asap. Tiago On Thu, Aug 6, 2009 at 9:56 PM, Jean T. Anderson wrote: > Kathey Marsden wrote: > >> Jean T. Anderson wrote: >> >>> The release happened, but there's no reason you can't quickly follow up >>> with another release with bug fixes. >>> >>> Thanks Jean for the clarification that the release really did happen. I >> honestly wasn't sure it had. :-) I think since a release happened, I feel >> more strongly that we should keep release notes separate, not merge Jira >> versions, etc. >> >> Kathey >> >> I just downloaded it -- it's already out on mirrors worldwide. :-) The > release happened, and was approved by the DB PMC. > > Now, given that it has been available for a week, if it's so broken that > folks shouldn't use it, it might be wise to remove it, maybe update the > website to indicate that those who have already downloaded should be aware > of the two key bugs that are (or will be) included in 10.5.2.1. > > -jean > >
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Jean T. Anderson wrote: The release happened, but there's no reason you can't quickly follow up with another release with bug fixes. Thanks Jean for the clarification that the release really did happen. I honestly wasn't sure it had. :-) I think since a release happened, I feel more strongly that we should keep release notes separate, not merge Jira versions, etc. Kathey I just downloaded it -- it's already out on mirrors worldwide. :-) The release happened, and was approved by the DB PMC. Now, given that it has been available for a week, if it's so broken that folks shouldn't use it, it might be wise to remove it, maybe update the website to indicate that those who have already downloaded should be aware of the two key bugs that are (or will be) included in 10.5.2.1. -jean
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Jean T. Anderson wrote: The release happened, but there's no reason you can't quickly follow up with another release with bug fixes. Thanks Jean for the clarification that the release really did happen. I honestly wasn't sure it had. :-) I think since a release happened, I feel more strongly that we should keep release notes separate, not merge Jira versions, etc. Kathey I don't really care what we call it. There is no reason that the release notes have to describe the delta between the current release and the immediately preceding one. I can build a 10.5.3.0 whose release notes describes its delta from 10.5.1.1.
Re: DERBY-4331 and the 10.5.2.0 release
Jean T. Anderson wrote: The release happened, but there's no reason you can't quickly follow up with another release with bug fixes. Thanks Jean for the clarification that the release really did happen. I honestly wasn't sure it had. :-) I think since a release happened, I feel more strongly that we should keep release notes separate, not merge Jira versions, etc. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden writes: > Current options as I see them. 1) Pull 10.5.2.0 off the > website, cut 10.5.2.1 with the fix and short vote with Rick as release > manager. > 2) Move 10.5.2.0 down to the deprecated section. cut 10.5.3.0 with > short vote with Rick as release manager. I am OK with either approach as long as we follow up with a quick new release. Thanks for the quick work on DERBY-4331, everyone! :) Dag
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Now that we have fixes for both DERBY-3926 and DERBY4331 in the trunk. I hope we can restart this discussion with better options, which hopefully will avoid the need for the funny JavaDB non-fork fork. Current options as I see them. 1) Pull 10.5.2.0 off the website, cut 10.5.2.1 with the fix and short vote with Rick as release manager. 2) Move 10.5.2.0 down to the deprecated section. cut 10.5.3.0 with short vote with Rick as release manager. I think I prefer option 2 as we don't try to erase the history of the vote or the release being on the website, but I can accept either approach. there's actually no way to erase the history of the vote -- that history remains forever in the public derby-dev mail archives at the ASF and at umpteen mirrors world wide. And the web site update gets a subversion commit record. What's the big deal? The release happened, but there's no reason you can't quickly follow up with another release with bug fixes. -jean
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: I vote for option (1). The next release vehicle looks like a second RC to me, not a new release. As I see it, we post releases on our website chiefly for the convenience of our users. What user value do we provide by continuing to post 10.5.2.0? I think the thing keeping the release notes on the website offers is an accurate history. Like the other deprecated releases the binaries of course would not be available. Since the release has been available on the mirrors, someone may be using it and wonder later where it came from and need a separate history of what was fixed in 10.5.2.0 vs the new release, But, as I said if you would like to go this way and the rest of the community agrees, I can go with it. You might want to advise the user list that they should upgrade from 10.5.2.0 to 10.5.2.1 if they are using it. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
I vote for option (1). The next release vehicle looks like a second RC to me, not a new release. As I see it, we post releases on our website chiefly for the convenience of our users. What user value do we provide by continuing to post 10.5.2.0? Thanks, -Rick Kathey Marsden wrote: Dag H. Wanvik wrote: Myrna van Lunteren writes: I personally don't care for a release that has either DERBY-3926 or DERBY-4331 in it. Now that we have fixes for both DERBY-3926 and DERBY4331 in the trunk. I hope we can restart this discussion with better options, which hopefully will avoid the need for the funny JavaDB non-fork fork. Current options as I see them. 1) Pull 10.5.2.0 off the website, cut 10.5.2.1 with the fix and short vote with Rick as release manager. 2) Move 10.5.2.0 down to the deprecated section. cut 10.5.3.0 with short vote with Rick as release manager. I think I prefer option 2 as we don't try to erase the history of the vote or the release being on the website, but I can accept either approach. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Tiago Espinha wrote: In my opinion we'd have a mix of both options. We could keep 10.5.2.0 on the site just for history's sake and introduce the 10.5.2.1 at the same time, with the DERBY-4331 fix. I'm just suggesting this because it seems a little extreme to bump the version up to 10.5.3 with virtually no relevant changes other than DERBY-4331. Typically we always bump the third digit for an official release, so I think is good to stick with that if 10.5.2.0 stays up there. The last digit is used generally for unofficial snapshots or respun release candidates. There are other fixes that have gone into 10.5 since the release, most notably DERBY-4312. https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=10594&fixfor=12314117&resolution=1&sorter/field=issuekey&sorter/order=DESC It might be nice to see a few more like DERBY-4310, but that can be ironed out after we develop a general strategy. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
In my opinion we'd have a mix of both options. We could keep 10.5.2.0 on the site just for history's sake and introduce the 10.5.2.1 at the same time, with the DERBY-4331 fix. I'm just suggesting this because it seems a little extreme to bump the version up to 10.5.3 with virtually no relevant changes other than DERBY-4331. Tiago On Thu, Aug 6, 2009 at 5:57 PM, Kathey Marsden wrote: > Dag H. Wanvik wrote: > >> Myrna van Lunteren writes: >> >> >> >>> I personally don't care for a release that has either DERBY-3926 or >>> DERBY-4331 in it. >>> >>> >> Now that we have fixes for both DERBY-3926 and DERBY4331 in the trunk. I > hope we can restart this discussion with better options, which hopefully > will avoid the need for the funny JavaDB non-fork fork. Current options > as I see them. 1) Pull 10.5.2.0 off the website, cut 10.5.2.1 with the fix > and short vote with Rick as release manager. > 2) Move 10.5.2.0 down to the deprecated section. cut 10.5.3.0 with short > vote with Rick as release manager. > > I think I prefer option 2 as we don't try to erase the history of the vote > or the release being on the website, but I can accept either approach. > > Kathey > > >
Re: DERBY-4331 and the 10.5.2.0 release
Dag H. Wanvik wrote: Myrna van Lunteren writes: I personally don't care for a release that has either DERBY-3926 or DERBY-4331 in it. Now that we have fixes for both DERBY-3926 and DERBY4331 in the trunk. I hope we can restart this discussion with better options, which hopefully will avoid the need for the funny JavaDB non-fork fork. Current options as I see them. 1) Pull 10.5.2.0 off the website, cut 10.5.2.1 with the fix and short vote with Rick as release manager. 2) Move 10.5.2.0 down to the deprecated section. cut 10.5.3.0 with short vote with Rick as release manager. I think I prefer option 2 as we don't try to erase the history of the vote or the release being on the website, but I can accept either approach. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: It seems to me that the reactions to this email thread were mostly negative. It is hard to tell without a formal vote (which of course would take a week) but I see a more mixed reaction on the issue of posting the release. Some people say complete the release process, some say abort and others think we should just stay in limbo a bit longer to research impact. I don't think there is a clear consensus. There also seems to be a similar divergence of opinions on whether and where to back out the DERBY-3926 fix. As Knut mentioned we have left releases with known wrong results regressions on the site before without even a mention of the problem on the download page. I feel like as release manager I should make a decision soon and there is not time for a 7 day revote on 10.5.2.0. My inclination is to go ahead with the process and include the warning on the download page. Then there can be more discussion and/or a vote on the fate of the current DERBY-3926 fix, separate from the release discussion. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Dag H. Wanvik [email protected] writes: >If there users who badly need DERBY-3926 *and* are not affected by >DERBY-4331 *and* are clamoring for it, they could download the patch >and spin it themselves while we ready 10.5.3. >My 0.02 cents; I agree it's a difficult call, though. I can live with >either outcome. I totally agree. Open source community should be a community for hard work and open communication on all aspect of issues. Thanks, Lily
Re: DERBY-4331 and the 10.5.2.0 release
Myrna van Lunteren writes: > I personally don't care for a release that has either DERBY-3926 or > DERBY-4331 in it. I agree. So the question is, do we want to announce the release now to let users benefit from the other fixes in 10.5.2? Following the release often credo, I would be tempted to say yes. But since I cannot weigh the relative importances of the two issues, I do think we should back out the fix for DERBY-3926 (from 10.5 branch), until we have settled DERBY-4331. That way 10.5.3 would show neither bug, since we can take the time we need to fix both satisfactorily. If there users who badly need DERBY-3926 *and* are not affected by DERBY-4331 *and* are clamoring for it, they could download the patch and spin it themselves while we ready 10.5.3. My 0.02 cents; I agree it's a difficult call, though. I can live with either outcome. Dag
Re: DERBY-4331 and the 10.5.2.0 release
It's a tough call. To give my 2c: - clearly DERBY-4331 needs to be fixed asap. - DERBY-3926 also needs to be fixed. - we need a new release. We voted and approved of 10.5.2.0, and there are fixes in it that people need. Now what do we do with it? I think there are 2 main questions: 1. Is the fix for DERBY-3926 "bad"? i.e. It clearly does fix a problem, and another problem shows up since that code change, but did it not take things "far enough", did it expose a bug that was previously masked, or was it altogether the wrong thing to do? 2. Do we care for a release without DERBY-3926? I personally don't care for a release that has either DERBY-3926 or DERBY-4331 in it. It seems to me we don't know the answer to question 1. yet. In the mean time, we have a release that we voted on, approved, that's been put on the mirrors, but not yet announced. And a release manager who's going on vacation. (Thanks Rick for volunteering to pick it up. I can help also). As we don't know the answer to question 1, and I don't care for a release with either problem, but we've gone almost to the end of the release process, I lean towards finishing the publishing work for 10.5.2.0, but with an explicit warning that we've found a serious regression, and are working to make another follow up release that fixes both issues. If someone is then desperate for one of the other fixes to try it out - hey, perhaps their application is having the type of queries that got fixed by the current fix for DERBY-3926. Other folks will choose to hold off. On top of the fact that I don't personally care for a release with either problem, there's work involved in backing out DERBY-3926 (and, it's been backported all the way to 10.1, so it should be backed out from the other branches too?), and I'd suggest we postpone that at least until we know what's going on. Myrna
Re: DERBY-4331 and the 10.5.2.0 release
I agree with all the data we have so far none of us can weigh whether DERBY-3962 is worse than DERBY-4331 or vice-versa. However, we know 10.5.2.0 fix DERBY-3962 and it has DERBY-4331. However, DERBY-4331 has a ugly work around by Calling SYSCS_UTIL.SYSCS_UPDATE_STATISTICS on the tables involved. We have a handle on simpler repro case with three tables join and one subquery join with two tables (repro script by Mamba). Do we have to make a decision now? I will think wait for one more day will give us time to have better handle on DERBY-4331. Thanks, Lily From: Rick Hillegas To: [email protected] Sent: Tuesday, August 4, 2009 10:05:30 AM Subject: Re: DERBY-4331 and the 10.5.2.0 release Hi Kathey, Thanks for release-managing 10.5.2.0 and for proposing a next step. Another option would be: 1) You can give 10.5.2.0 to your users who need the fix to DERBY-3962 2) But remove 10.5.2.0 from the Apache website 3) Back out the fix for DERBY-3962 and spin a 10.5.2.1 RC 3) Continue to work on 10.5.3, which would fix DERBY-3962 and DERBY-4331. Vetting for the RC could be expedited and take, say, a week. It seems to me that the reactions to this email thread were mostly negative. I don't know that any of us can weigh whether DERBY-3962 is worse than DERBY-4331 or vice-versa. My sense is that DERBY-3962 is not really fixed and the problem was simply shifted elsewhere--I do not believe this regression would have been allowed if it had been noticed earlier. Thanks, -Rick Kathey Marsden wrote: > Mike Matrigali wrote: >> Kathey Marsden wrote: >>> Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I >>> normally would not want to make a release with a known wrong results >>> regression, but am not quite sure what to do with 10.5.2.0. The vote >>> closed and passed. The release has been posted to the website, but no >>> announcement has been made. Should we continue with the release or try to >>> abort and make another release candidate? > 10.5.2.0 fixes 8 regression fixes that we know some users have already > picked up. > https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=10594&customfield_12310200=Regression&fixfor=12314116&resolution=1&sorter/field=issuekey&sorter/order=DESC > > > It also has many other useful fixes. > > If we restart 10.5.2, my guess is it will take about 4 weeks. Two to fix > DERBY-4331 and two to restart the vote and test cycle. The fix may take > longer since of course we don't know much yet. > > If we post 10.5.2.0 and start immediately on a 10.5.3.0, we could hopefully > get it out in about 6 weeks. Four weeks for bug fixing and two for the vote > and testing. > > I think a stern warning on the 10.5.2.0 will help mitigate any problems from > DERBY-4331 and allow users to make their own choice and possibly benefit from > the fixes in 10.5.2.0. Mike mentioned that perhaps there will be a > workaround for DERBY-4331, to add extra columns to the order by. If that is > the case we could post that too. > > I think we have educated users who given the proper information can decide > for themselves whether to wait for 10.5.3 if given sufficient information. > If I hear no objections I will continue with the 10.5.2.0 release process > this afternoon. > > Kathey > >
Re: DERBY-4331 and the 10.5.2.0 release
Hi Kathey, Thanks for release-managing 10.5.2.0 and for proposing a next step. Another option would be: 1) You can give 10.5.2.0 to your users who need the fix to DERBY-3962 2) But remove 10.5.2.0 from the Apache website 3) Back out the fix for DERBY-3962 and spin a 10.5.2.1 RC 3) Continue to work on 10.5.3, which would fix DERBY-3962 and DERBY-4331. Vetting for the RC could be expedited and take, say, a week. It seems to me that the reactions to this email thread were mostly negative. I don't know that any of us can weigh whether DERBY-3962 is worse than DERBY-4331 or vice-versa. My sense is that DERBY-3962 is not really fixed and the problem was simply shifted elsewhere--I do not believe this regression would have been allowed if it had been noticed earlier. Thanks, -Rick Kathey Marsden wrote: Mike Matrigali wrote: Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? 10.5.2.0 fixes 8 regression fixes that we know some users have already picked up. https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=10594&customfield_12310200=Regression&fixfor=12314116&resolution=1&sorter/field=issuekey&sorter/order=DESC It also has many other useful fixes. If we restart 10.5.2, my guess is it will take about 4 weeks. Two to fix DERBY-4331 and two to restart the vote and test cycle. The fix may take longer since of course we don't know much yet. If we post 10.5.2.0 and start immediately on a 10.5.3.0, we could hopefully get it out in about 6 weeks. Four weeks for bug fixing and two for the vote and testing. I think a stern warning on the 10.5.2.0 will help mitigate any problems from DERBY-4331 and allow users to make their own choice and possibly benefit from the fixes in 10.5.2.0. Mike mentioned that perhaps there will be a workaround for DERBY-4331, to add extra columns to the order by. If that is the case we could post that too. I think we have educated users who given the proper information can decide for themselves whether to wait for 10.5.3 if given sufficient information. If I hear no objections I will continue with the 10.5.2.0 release process this afternoon. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Mike Matrigali wrote: Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? 10.5.2.0 fixes 8 regression fixes that we know some users have already picked up. https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&&pid=10594&customfield_12310200=Regression&fixfor=12314116&resolution=1&sorter/field=issuekey&sorter/order=DESC It also has many other useful fixes. If we restart 10.5.2, my guess is it will take about 4 weeks. Two to fix DERBY-4331 and two to restart the vote and test cycle. The fix may take longer since of course we don't know much yet. If we post 10.5.2.0 and start immediately on a 10.5.3.0, we could hopefully get it out in about 6 weeks. Four weeks for bug fixing and two for the vote and testing. I think a stern warning on the 10.5.2.0 will help mitigate any problems from DERBY-4331 and allow users to make their own choice and possibly benefit from the fixes in 10.5.2.0. Mike mentioned that perhaps there will be a workaround for DERBY-4331, to add extra columns to the order by. If that is the case we could post that too. I think we have educated users who given the proper information can decide for themselves whether to wait for 10.5.3 if given sufficient information. If I hear no objections I will continue with the 10.5.2.0 release process this afternoon. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
I am looking at DERBY-4331. I just wanted to share though that I had made plans to be on vacation in August before this bug surfaced and hence will not be able to work on the bug full time for the most of August. Hopefully the bug will not take that long but wanted to give a heads up. Apologies for the bad timing of vacation plans and this bug Mamta On Mon, Aug 3, 2009 at 10:42 AM, Mike Matrigali wrote: > Kathey Marsden wrote: >> >> Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I >> normally would not want to make a release with a known wrong results >> regression, but am not quite sure what to do with 10.5.2.0. The vote closed >> and passed. The release has been posted to the website, but no announcement >> has been made. Should we continue with the release or try to abort and make >> another release candidate? If we do abort, I won't be able to be release >> manager as I am going out on vacation for a few weeks on August 15. >> >> Another issue is that DERBY-3926 which introduced the regression was not >> listed in the release notes because it was still open. One possible option >> is to add a release note to that issue now and manually update the release >> notes on the website (not in the distribution) to include DERBY-3926 and >> warn about DERBY-4331. Then start working on another 10.5 release to be >> released fairly soon (a couple months). >> >> Thoughts? >> >> Kathey >> > Since the vote passed, I lean toward posting the release with a warning > about 4331 - and also think we should fast track another release on top > of it if we can get someone to fix 4331. A lot of people reviewed 3926 and > it has been out there for awhile so I real rather not back it out as that > also will result in queries with wrong sort order. I may change > my opinion if we can identify that 3926 was a bad fix. > > I can volunteer some immediate time looking at the bad query plans and > trying to come up with some simpler test cases. Once I get set up I will > also post a workaround if the usual thing works, which is to add some > columns to the order by clause as I assume the problem is bad > sort avoidance rather than an actual sorter problem. > >
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? If we do abort, I won't be able to be release manager as I am going out on vacation for a few weeks on August 15. Another issue is that DERBY-3926 which introduced the regression was not listed in the release notes because it was still open. One possible option is to add a release note to that issue now and manually update the release notes on the website (not in the distribution) to include DERBY-3926 and warn about DERBY-4331. Then start working on another 10.5 release to be released fairly soon (a couple months). Thoughts? Kathey Since the vote passed, I lean toward posting the release with a warning about 4331 - and also think we should fast track another release on top of it if we can get someone to fix 4331. A lot of people reviewed 3926 and it has been out there for awhile so I real rather not back it out as that also will result in queries with wrong sort order. I may change my opinion if we can identify that 3926 was a bad fix. I can volunteer some immediate time looking at the bad query plans and trying to come up with some simpler test cases. Once I get set up I will also post a workaround if the usual thing works, which is to add some columns to the order by clause as I assume the problem is bad sort avoidance rather than an actual sorter problem.
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: I am concerned about declaring victory on DERBY-3926. It seems that the ordering problem may simply have been shifted from one family of queries to another. Well, the cases reported with DERBY-3926 and DERBY-4240 have been resolved as far as we know and I believe that users have picked up the fix. I think it is fine to leave it open if you think that is the right thing to do, but don't think it should be backed out without a replacement that fixes these issues. Thanks Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? If we do abort, I won't be able to be release manager as I am going out on vacation for a few weeks on August 15. Another issue is that DERBY-3926 which introduced the regression was not listed in the release notes because it was still open. One possible option is to add a release note to that issue now and manually update the release notes on the website (not in the distribution) to include DERBY-3926 and warn about DERBY-4331. Then start working on another 10.5 release to be released fairly soon (a couple months). I am concerned about declaring victory on DERBY-3926. It seems that the ordering problem may simply have been shifted from one family of queries to another. Regards, -Rick Thoughts? Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Lily Wei wrote: Do we have an estimate in turn of how long it takes to fix DERBY-4331 . I think this is an important point. If DERBY-4331 is hard to fix or nobody can look at it right now, I think we should go ahead and release 10.5.2.0 with the warning. and knowledge in turn whether the fix will cause any further regression? I think our testing for order by (and also group by) may be somewhat thin based on past issues and this regression popping up. Any ideas on improved testing would be welcome. Perhaps DERBY-4112 would be helpful in flushing out these cases where one plan works and another does not. I don't think implementing such testing would be a quick fix by any means, even if the fix for DERBY-4331 proves to be. To ensure adequate time to fix the bug and do the proper testing, I am still leaning on posting 10.5.2.0 with a warning. Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Lily Wei wrote: Do we have an estimate in turn of how long it takes to fix DERBY-4331 and knowledge in turn whether the fix will cause any further regression? I just want to make sure we iron out some of the unknown issues. Hi Lily, I don't think we have an estimate of that effort yet. One approach is to back out the DERBY-3926 fix which appears to have caused this regression. Regards, -Rick Thanks, Lily *From:* Rick Hillegas *To:* [email protected] *Sent:* Monday, August 3, 2009 7:58:40 AM *Subject:* Re: DERBY-4331 and the 10.5.2.0 release Knut Anders Hatlen wrote: > Rick Hillegas <mailto:[email protected]>> writes: > > >> As I see it, data corruptions are the worst kind of database bug, >> followed closely by wrong results. We have a precedent for pulling >> releases from our download site and I recommend that we pull the >> 10.5.2.0 release because of this regression. >> > > Although we have pulled releases with serious problems from our download > site once, I think we shouldn't make that the rule. The pulling did > cause some problems for us, like holes in the release notes when > upgrading over multiple releases in one go. Note that there won't be a hole in the release notes in this case. > And most of the releases > from the 10.3 and 10.4 branches did have wrong results regressions, but > they're still available for download. > Probably we would have spun new RCs for those distributions if our release testing had disclosed the regressions early on. > However, since we haven't yet announced 10.5.2.0, holding the > announcment (possibly pulling the release from the download page) and > quickly re-spinning a release with DERBY-4331 fixed sounds like a good > option. Also, I'd prefer a quick re-spin as soon as the known > regressions are fixed, rather than having a new maintenance release in > the autumn, since right now all our 10.5 releases have known regressions > from 10.4. > > That said, I won't be able to help with a release the next two weeks, so > I'm just expressing a wish for a quick fix, not actually offering to do > the work. :( > >
Re: DERBY-4331 and the 10.5.2.0 release
Do we have an estimate in turn of how long it takes to fix DERBY-4331 and knowledge in turn whether the fix will cause any further regression? I just want to make sure we iron out some of the unknown issues. Thanks, Lily From: Rick Hillegas To: [email protected] Sent: Monday, August 3, 2009 7:58:40 AM Subject: Re: DERBY-4331 and the 10.5.2.0 release Knut Anders Hatlen wrote: > Rick Hillegas writes: > > >> As I see it, data corruptions are the worst kind of database bug, >> followed closely by wrong results. We have a precedent for pulling >> releases from our download site and I recommend that we pull the >> 10.5.2.0 release because of this regression. >> > > Although we have pulled releases with serious problems from our download > site once, I think we shouldn't make that the rule. The pulling did > cause some problems for us, like holes in the release notes when > upgrading over multiple releases in one go. Note that there won't be a hole in the release notes in this case. > And most of the releases > from the 10.3 and 10.4 branches did have wrong results regressions, but > they're still available for download. > Probably we would have spun new RCs for those distributions if our release testing had disclosed the regressions early on. > However, since we haven't yet announced 10.5.2.0, holding the > announcment (possibly pulling the release from the download page) and > quickly re-spinning a release with DERBY-4331 fixed sounds like a good > option. Also, I'd prefer a quick re-spin as soon as the known > regressions are fixed, rather than having a new maintenance release in > the autumn, since right now all our 10.5 releases have known regressions > from 10.4. > > That said, I won't be able to help with a release the next two weeks, so > I'm just expressing a wish for a quick fix, not actually offering to do > the work. :( > >
Re: DERBY-4331 and the 10.5.2.0 release
Ok, it seems to be amicable then that we need to abort. We can go on with Rick as the release manager and get this fixed for a 10.5.2.1, skipping 10.5.2.0 altogether. I'm still under my GSoC period but I'm available to help with whatever has to be done. I would gladly offer to fix this issue but it feels like 'a little too much sand for my truck' and it would probably take me way too long and way too much of banging my head on the wall, so I'm leaving it up to someone more experienced than me. Tiago On Mon, Aug 3, 2009 at 3:58 PM, Rick Hillegas wrote: > Knut Anders Hatlen wrote: > >> Rick Hillegas writes: >> >> >> >>> As I see it, data corruptions are the worst kind of database bug, >>> followed closely by wrong results. We have a precedent for pulling >>> releases from our download site and I recommend that we pull the >>> 10.5.2.0 release because of this regression. >>> >>> >> >> Although we have pulled releases with serious problems from our download >> site once, I think we shouldn't make that the rule. The pulling did >> cause some problems for us, like holes in the release notes when >> upgrading over multiple releases in one go. >> > Note that there won't be a hole in the release notes in this case. > >> And most of the releases >> from the 10.3 and 10.4 branches did have wrong results regressions, but >> they're still available for download. >> >> > Probably we would have spun new RCs for those distributions if our release > testing had disclosed the regressions early on. > > However, since we haven't yet announced 10.5.2.0, holding the >> announcment (possibly pulling the release from the download page) and >> quickly re-spinning a release with DERBY-4331 fixed sounds like a good >> option. Also, I'd prefer a quick re-spin as soon as the known >> regressions are fixed, rather than having a new maintenance release in >> the autumn, since right now all our 10.5 releases have known regressions >> from 10.4. >> >> That said, I won't be able to help with a release the next two weeks, so >> I'm just expressing a wish for a quick fix, not actually offering to do >> the work. :( >> >> >> > >
Re: DERBY-4331 and the 10.5.2.0 release
Knut Anders Hatlen wrote: Rick Hillegas writes: As I see it, data corruptions are the worst kind of database bug, followed closely by wrong results. We have a precedent for pulling releases from our download site and I recommend that we pull the 10.5.2.0 release because of this regression. Although we have pulled releases with serious problems from our download site once, I think we shouldn't make that the rule. The pulling did cause some problems for us, like holes in the release notes when upgrading over multiple releases in one go. Note that there won't be a hole in the release notes in this case. And most of the releases from the 10.3 and 10.4 branches did have wrong results regressions, but they're still available for download. Probably we would have spun new RCs for those distributions if our release testing had disclosed the regressions early on. However, since we haven't yet announced 10.5.2.0, holding the announcment (possibly pulling the release from the download page) and quickly re-spinning a release with DERBY-4331 fixed sounds like a good option. Also, I'd prefer a quick re-spin as soon as the known regressions are fixed, rather than having a new maintenance release in the autumn, since right now all our 10.5 releases have known regressions from 10.4. That said, I won't be able to help with a release the next two weeks, so I'm just expressing a wish for a quick fix, not actually offering to do the work. :(
Re: DERBY-4331 and the 10.5.2.0 release
Kristian Waagan writes: > +1 from me. > > I'd like to see a maintenance release as soon as possible, but I think > we are best served if we fix the mentioned bug (DERBY-4331) and also > spend some time investigating the error reports from users on > 10.5.1.1. > The reason I'm not in favor of (2), is that I believe we have some > important fixes in place already, and that some of our users would > benefit from them. +1 Thanks for offering to do this, Rick! Please note that after 10.5.2.0 was spun, more stuff has gone into the 10.5 branch, I think Kathey lifted the "freeze" on July 16th. Dag
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas writes: > As I see it, data corruptions are the worst kind of database bug, > followed closely by wrong results. We have a precedent for pulling > releases from our download site and I recommend that we pull the > 10.5.2.0 release because of this regression. Although we have pulled releases with serious problems from our download site once, I think we shouldn't make that the rule. The pulling did cause some problems for us, like holes in the release notes when upgrading over multiple releases in one go. And most of the releases from the 10.3 and 10.4 branches did have wrong results regressions, but they're still available for download. However, since we haven't yet announced 10.5.2.0, holding the announcment (possibly pulling the release from the download page) and quickly re-spinning a release with DERBY-4331 fixed sounds like a good option. Also, I'd prefer a quick re-spin as soon as the known regressions are fixed, rather than having a new maintenance release in the autumn, since right now all our 10.5 releases have known regressions from 10.4. That said, I won't be able to help with a release the next two weeks, so I'm just expressing a wish for a quick fix, not actually offering to do the work. :( -- Knut Anders
Re: DERBY-4331 and the 10.5.2.0 release
Rick Hillegas wrote: As I see it, data corruptions are the worst kind of database bug, followed closely by wrong results. We have a precedent for pulling releases from our download site and I recommend that we pull the 10.5.2.0 release because of this regression. I think we should do one of the following also: 1) Produce a 10.5.2.1 RC with a fix for this regression or 2) Produce a 10.5.3 release in the autumn I vote for option (1) and am willing to serve as release manager for the remainder of 10.5.2. +1 from me. I'd like to see a maintenance release as soon as possible, but I think we are best served if we fix the mentioned bug (DERBY-4331) and also spend some time investigating the error reports from users on 10.5.1.1. The reason I'm not in favor of (2), is that I believe we have some important fixes in place already, and that some of our users would benefit from them. Thank you for volunteering to drive the release process, Rick. Regards, -- Kristian Regards, -Rick Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? If we do abort, I won't be able to be release manager as I am going out on vacation for a few weeks on August 15. Another issue is that DERBY-3926 which introduced the regression was not listed in the release notes because it was still open. One possible option is to add a release note to that issue now and manually update the release notes on the website (not in the distribution) to include DERBY-3926 and warn about DERBY-4331. Then start working on another 10.5 release to be released fairly soon (a couple months). Thoughts? Kathey
Re: DERBY-4331 and the 10.5.2.0 release
As I see it, data corruptions are the worst kind of database bug, followed closely by wrong results. We have a precedent for pulling releases from our download site and I recommend that we pull the 10.5.2.0 release because of this regression. I think we should do one of the following also: 1) Produce a 10.5.2.1 RC with a fix for this regression or 2) Produce a 10.5.3 release in the autumn I vote for option (1) and am willing to serve as release manager for the remainder of 10.5.2. Regards, -Rick Kathey Marsden wrote: Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I normally would not want to make a release with a known wrong results regression, but am not quite sure what to do with 10.5.2.0. The vote closed and passed. The release has been posted to the website, but no announcement has been made. Should we continue with the release or try to abort and make another release candidate? If we do abort, I won't be able to be release manager as I am going out on vacation for a few weeks on August 15. Another issue is that DERBY-3926 which introduced the regression was not listed in the release notes because it was still open. One possible option is to add a release note to that issue now and manually update the release notes on the website (not in the distribution) to include DERBY-3926 and warn about DERBY-4331. Then start working on another 10.5 release to be released fairly soon (a couple months). Thoughts? Kathey
Re: DERBY-4331 and the 10.5.2.0 release
Hello people, It's been a while since I've shown some appearance here... however, I'd favour an abort of the release. We use Derby in production and have applied a couple of patches in order to keep up with performance and new features... so far so good. We've anxiously been waiting for a few bugfixes, some of which I'd liked to have fixed myself if time had permitted me... though I haven't been that fortunate. This 'simple' (not meant to disrespect anything of the great work on Derby!) case of ordering can lead to disastrous results on certain kinds of applications. We *did* choose Derby as our main RDBMS after a lot of tests and comparisons to other open source and commercial databases. A lot of people using Derby are most likely unaware that, as Knut Anders alreay mentioned: the chosen plan for the optimizer *may* give the correct results when SYSCS_UTIL.SYSCS_UPDATE_STATISTICS is applied. Besides, try to do that on a database in production with millions of records and a size of an arbitrary number of Gb's. Phew. I don't think I need to explain any further, why wrong ordering results are really inexcusable. Please abort and re-release asap. Some fixes (f.e. CLOB-sorting (DERBY-4245) by Kristian) are really needed. Thanks for reading... and if and when I've got some spare time, I'll again dedicate some of it to making Derby even better! I *really* beg your pardon if this sounds like a rant. It isn't. -Stephan.
Re: DERBY-4331 and the 10.5.2.0 release
Since we don't seem to have a fix for DERBY-4331 yet, I suggest we go ahead with the 10.5.2.0 release and then perhaps we can make a 10.5.2.1 right after we fix this? I'm not totally aware of the versioning rules for Derby so I don't know if this would be feasible. It just seems like the best approach in light of how far the 10.5.2.0 release process is and considering the fact that you won't be around for a while. It's just my $0.02 Tiago On Sun, Aug 2, 2009 at 5:59 PM, Kathey Marsden wrote: > Knut identified a wrong results regression in 10.5.2.0, DERBY-4331. I > normally would not want to make a release with a known wrong results > regression, but am not quite sure what to do with 10.5.2.0. The vote closed > and passed. The release has been posted to the website, but no announcement > has been made. Should we continue with the release or try to abort and make > another release candidate? If we do abort, I won't be able to be release > manager as I am going out on vacation for a few weeks on August 15. > > Another issue is that DERBY-3926 which introduced the regression was not > listed in the release notes because it was still open. One possible option > is to add a release note to that issue now and manually update the release > notes on the website (not in the distribution) to include DERBY-3926 and > warn about DERBY-4331. Then start working on another 10.5 release to be > released fairly soon (a couple months). > > Thoughts? > > Kathey >
