[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Matrigali updated DERBY-4442: -- backporting svn #888311 from trunk to 10.5 branch. m105_jdk16:111>svn commit Sending. Sending java/engine/org/apache/derby/impl/sql/compile/ResultColumnList.java Transmitting file data . Committed revision 960910. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Mike Matrigali > Fix For: 10.6.1.0 > > Attachments: always_prn.diff, backout-derby-4425.diff, d4442-1a.diff, > d4442-1a.stat, d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Matrigali updated DERBY-4442: -- backporting change #885421 from trunk to 10.5 branch. m105_jdk16:76>svn commit Sending. Sendingjava/engine/org/apache/derby/impl/sql/compile/InsertNode.java Sendingjava/engine/org/apache/derby/impl/sql/compile/ResultSetNode.java Sending java/engine/org/apache/derby/impl/sql/compile/RowResultSetNode.java Sendingjava/engine/org/apache/derby/impl/sql/compile/UnionNode.java Sending java/testing/org/apache/derbyTesting/functionTests/master/intersect.out Sending java/testing/org/apache/derbyTesting/functionTests/tests/lang/DistinctTest.java Sending java/testing/org/apache/derbyTesting/functionTests/tests/lang/intersect.sql Transmitting file data ... Committed revision 960731. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Mike Matrigali > Fix For: 10.6.1.0 > > Attachments: always_prn.diff, backout-derby-4425.diff, d4442-1a.diff, > d4442-1a.stat, d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Matrigali updated DERBY-4442: -- Assignee: Mike Matrigali (was: Knut Anders Hatlen) working on backporting this issue to 10.5. DERBY-4419, DERBY-4413, DERBY-4425, and DERBY-4442 all seem related. I am going to apply and checkin the backported changes to these issues in order. I think the changes necessary are: DERBY-4413 #829410 DERBY-4419 #831304 DERBY-4425 #831319 DERBY-4442 #885421 DERBY-4413 #885659 DERBY-4442 #888311 > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Mike Matrigali > Fix For: 10.6.1.0 > > Attachments: always_prn.diff, backout-derby-4425.diff, d4442-1a.diff, > d4442-1a.stat, d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Issue & fix info: [Patch Available] > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Knut Anders Hatlen > Fix For: 10.6.0.0 > > Attachments: always_prn.diff, backout-derby-4425.diff, d4442-1a.diff, > d4442-1a.stat, d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Attachment: backout-derby-4425.diff The attached patch backs out the fix for DERBY-4425 (except the tests), since it's not needed anymore. All the regression tests ran cleanly. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Knut Anders Hatlen > Fix For: 10.6.0.0 > > Attachments: always_prn.diff, backout-derby-4425.diff, d4442-1a.diff, > d4442-1a.stat, d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Issue & fix info: (was: [Patch Available]) Affects Version/s: 10.5.3.0 Fix Version/s: 10.6.0.0 Thanks Dag. I've committed the patch to trunk with revision 885421, since it looks like DERBY-4426 is more or less ready for commit too. I'll leave this issue open until I've investigated if we can back out the fixes for some of the other issues linked to this one now. I think at least there were some asserts that were loosened to accept untyped nulls (placeholders for the identity values) at odd places in the code, and these could now be made stricter again to potentially catch other errors. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Affects Versions: 10.5.3.0 >Reporter: Dag H. Wanvik >Assignee: Knut Anders Hatlen > Fix For: 10.6.0.0 > > Attachments: always_prn.diff, d4442-1a.diff, d4442-1a.stat, > d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Issue & fix info: [Patch Available] The patch is ready for review, so I'm setting "Patch Available". We should probably wait for DERBY-4426 to be fixed before the patch is committed to prevent exposing the NPEs mentioned earlier. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Reporter: Dag H. Wanvik >Assignee: Knut Anders Hatlen > Attachments: always_prn.diff, d4442-1a.diff, d4442-1a.stat, > d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Attachment: d4442-1b.diff Here's a new revision of the patch (1b) with one small change: The comment in InsertNode.enhanceAndCheckForAutoincrement() only explained why it was OK to skip the call to forbidOverrides() on the top-level RCL for a multi-row table constructor. Now it also explains why it *should* be skipped, to make it clear that the check is actually needed and not some kind of optimization. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Reporter: Dag H. Wanvik >Assignee: Knut Anders Hatlen > Attachments: always_prn.diff, d4442-1a.diff, d4442-1a.stat, > d4442-1b.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Attachment: d4442-1a.diff d4442-1a.stat Here's an updated patch (d4442-1a.diff). I've added comments to the new methods, updated the existing comments to better match the current state of the code, updated the DERBY-3 test case to expect the new behaviour, and also added a test case for DERBY-4433. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Reporter: Dag H. Wanvik > Attachments: always_prn.diff, d4442-1a.diff, d4442-1a.stat, > insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dag H. Wanvik updated DERBY-4442: - Attachment: insert.diff Thanks for trying the patch with the DERBY-4 patch. Here is a slightly modified InsertNode.java which (with your patch + a modified contribution from DERBY-4) which seems to work. As you can see, I moved the ORDER BY pulling and binding to happen before the enhancement (not a problem any longer since we can't sort a VALUES in the insert context), and dropped the use of the column map. ij> select * from t4; I |J --- 1 |200 2 |100 3 |300 ij> insert into t3(i) select j from t4 order by j; ij> select * from t3; ID |I --- 34 |100 35 |200 36 |300 > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Reporter: Dag H. Wanvik > Attachments: always_prn.diff, insert.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
[jira] Updated: (DERBY-4442) Evaluation of default value and identity in an INSERT result set evaluated too early.
[ https://issues.apache.org/jira/browse/DERBY-4442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Knut Anders Hatlen updated DERBY-4442: -- Attachment: always_prn.diff I took the prn3 patch attached to DERBY-4433, which ensures that generated/identity columns are added in a ProjectRestrictNode on top of the source result set if the source is a SetOperatorNode, and moved the PRN generation from SetOperatorNode to ResultSetNode so that it applies to all kinds of result sets (except table constructors, which need special handling). See the attached always_prn patch. With that patch, the DERBY-3 repro does no longer produce gaps in the identity values, so the approach looks promising. I also tried it in combination with the DERBY-4 patch (derby-4_dhw), but then it unfortunately produced an assert error: ij> create table t3(id int generated always as identity, i int); 0 rows inserted/updated/deleted ij> insert into t3(i) select * from t1 order by x; ERROR XJ001: Java exception: 'ASSERT FAILED bindExpressions() is not expected to be called for class org.apache.derby.impl.sql.compile.ProjectRestrictNode: org.apache.derby.shared.common.sanity.AssertFailure'. > Evaluation of default value and identity in an INSERT result set evaluated > too early. > - > > Key: DERBY-4442 > URL: https://issues.apache.org/jira/browse/DERBY-4442 > Project: Derby > Issue Type: Bug > Components: SQL >Reporter: Dag H. Wanvik > Attachments: always_prn.diff > > > In contrast to generated column, which are evaluated when the next row from > the result set to be inserted, currently default values and identity columns > are generated "early", that is as part of avaluating the subquery (SELECT or > VALUES as the case may be). > This does not currently cause a user visible bug in Derby, but it lies behind > DERBY-3 and the effect Bryan observed in DERBY-4. > Additionally, "early" computation has given rise to much special handling and > ensuing bugs, cf. DERBY-1644, DERBY-4413, DERBY-4419, DERBY-4425 and others. > DERBY-4397 requires this fix for correct behaviour with INSERT. > See also > https://issues.apache.org/jira/browse/DERBY-4413?focusedCommentId=12769532&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12769532 -- This message is automatically generated by JIRA. - You can reply to this email to add a comment to the issue online.
