[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[ https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493987#comment-16493987 ] Vineet Garg commented on HIVE-18748: I just confirmed this doesn't break the build but still we should get a test result before pushing this patch to branch-3. > Rename table impacts the ACID behavior as table names are not updated in > meta-tables. > - > > Key: HIVE-18748 > URL: https://issues.apache.org/jira/browse/HIVE-18748 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Eugene Koifman >Priority: Critical > Labels: ACID, DDL > Fix For: 3.1.0 > > Attachments: HIVE-18748.01-branch-3.patch, > HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch, > HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch, > HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch, > HIVE-18748.05-branch-3.patch, HIVE-18748.05.patch, > HIVE-18748.06-branch-3.patch > > > ACID implementation uses metatables such as TXN_COMPONENTS, > COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to > manage ACID operations. > Per table write ID implementation (HIVE-18192) introduces couple of > metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids > allocated per table. > Now, when we rename any tables, it is necessary to update the corresponding > table names in these metatables as well. Otherwise, ACID table operations > won't work properly. > Since, this change is significant and have other side-effects, we propose to > disable rename tables on ACID tables until a fix is figured out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16493958#comment-16493958
]
Vineet Garg commented on HIVE-18748:
[~ekoifman] {{[^HIVE-18748.06-branch-3.patch]}} did not have test run (in fact
it had build failure according to ptest results). Not sure why this was
committed to branch-3 in this state?
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Fix For: 3.1.0
>
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch,
> HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch,
> HIVE-18748.05-branch-3.patch, HIVE-18748.05.patch,
> HIVE-18748.06-branch-3.patch
>
>
> ACID implementation uses metatables such as TXN_COMPONENTS,
> COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to
> manage ACID operations.
> Per table write ID implementation (HIVE-18192) introduces couple of
> metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids
> allocated per table.
> Now, when we rename any tables, it is necessary to update the corresponding
> table names in these metatables as well. Otherwise, ACID table operations
> won't work properly.
> Since, this change is significant and have other side-effects, we propose to
> disable rename tables on ACID tables until a fix is figured out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16491492#comment-16491492
]
Hive QA commented on HIVE-18748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924857/HIVE-18748.06-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11220/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11220/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11220/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/security/SecurityUtil.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/util/GenericOptionsParser.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar(org/eclipse/jetty/rewrite/handler/RedirectPatternRule.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-rewrite/9.3.8.v20160314/jetty-rewrite-9.3.8.v20160314.jar(org/eclipse/jetty/rewrite/handler/RewriteHandler.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar(org/eclipse/jetty/server/Handler.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar(org/eclipse/jetty/server/Server.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar(org/eclipse/jetty/server/ServerConnector.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-server/9.3.8.v20160314/jetty-server-9.3.8.v20160314.jar(org/eclipse/jetty/server/handler/HandlerList.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar(org/eclipse/jetty/servlet/FilterHolder.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar(org/eclipse/jetty/servlet/ServletContextHandler.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-servlet/9.3.8.v20160314/jetty-servlet-9.3.8.v20160314.jar(org/eclipse/jetty/servlet/ServletHolder.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-xml/9.3.8.v20160314/jetty-xml-9.3.8.v20160314.jar(org/eclipse/jetty/xml/XmlConfiguration.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/slf4j/jul-to-slf4j/1.7.10/jul-to-slf4j-1.7.10.jar(org/slf4j/bridge/SLF4JBridgeHandler.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/DispatcherType.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/Filter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterChain.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482017#comment-16482017
]
Hive QA commented on HIVE-18748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924288/HIVE-18748.05-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 14209 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_vector_dynpart_hashjoin_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_2]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insertsel_fail]
(batchId=95)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=225)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.exec.tez.TestWorkloadManager.testApplyPlanQpChanges
(batchId=293)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=236)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=236)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11094/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11094/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11094/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924288 - PreCommit-HIVE-Build
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch,
> HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch,
> HIVE-18748.05-branch-3.patch, HIVE-18748.05.patch
>
>
> ACID implementation uses metatables such as TXN_COMPONENTS,
> COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to
> manage ACID operations.
> Per table write ID implementation (HIVE-18192) introduces couple of
> metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids
> allocated per table.
> Now, when we rename any tables, it is necessary to update the corresponding
> table names in these metatables as well. Otherwise, ACID table operations
> won't work properly.
> Since, this change is significant and have other side-effects, we propose to
> disable rename tables on ACID tables until a fix is figured out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482004#comment-16482004
]
Hive QA commented on HIVE-18748:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 11s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-11094/patches/PreCommit-HIVE-Build-11094.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11094/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch,
> HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch,
> HIVE-18748.05-branch-3.patch, HIVE-18748.05.patch
>
>
> ACID implementation uses metatables such as TXN_COMPONENTS,
> COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to
> manage ACID operations.
> Per table write ID implementation (HIVE-18192) introduces couple of
> metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids
> allocated per table.
> Now, when we rename any tables, it is necessary to update the corresponding
> table names in these metatables as well. Otherwise, ACID table operations
> won't work properly.
> Since, this change is significant and have other side-effects, we propose to
> disable rename tables on ACID tables until a fix is figured out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481998#comment-16481998
]
Hive QA commented on HIVE-18748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12924287/HIVE-18748.04-branch-3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 14209 tests
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=253)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed
out) (batchId=253)
TestNonCatCallsWithCatalog - did not produce a TEST-*.xml file (likely timed
out) (batchId=217)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=253)
TestTxnExIm - did not produce a TEST-*.xml file (likely timed out) (batchId=286)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_stats]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_2]
(batchId=162)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[insertsel_fail]
(batchId=95)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=225)
org.apache.hadoop.hive.ql.TestAcidOnTez.testCtasTezUnion (batchId=228)
org.apache.hadoop.hive.ql.TestAcidOnTez.testNonStandardConversion01
(batchId=228)
org.apache.hadoop.hive.ql.TestTxnCommandsForMmTable.testInsertOverwriteWithUnionAll
(batchId=264)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=236)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=236)
org.apache.hive.jdbc.TestJdbcDriver2.testSelectExecAsync2 (batchId=241)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11093/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11093/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11093/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12924287 - PreCommit-HIVE-Build
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch,
> HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch, HIVE-18748.05.patch
>
>
> ACID implementation uses metatables such as TXN_COMPONENTS,
> COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to
> manage ACID operations.
> Per table write ID implementation (HIVE-18192) introduces couple of
> metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids
> allocated per table.
> Now, when we rename any tables, it is necessary to update the corresponding
> table names in these metatables as well. Otherwise, ACID table operations
> won't work properly.
> Since, this change is significant and have other side-effects, we propose to
> disable rename tables on ACID tables until a fix is figured out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481989#comment-16481989
]
Hive QA commented on HIVE-18748:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 8s{color}
| {color:red}
/data/hiveptest/logs/PreCommit-HIVE-Build-11093/patches/PreCommit-HIVE-Build-11093.patch
does not apply to master. Rebase required? Wrong Branch? See
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help.
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-11093/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch,
> HIVE-18748.04-branch-3.patch, HIVE-18748.04.patch, HIVE-18748.05.patch
>
>
> ACID implementation uses metatables such as TXN_COMPONENTS,
> COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to
> manage ACID operations.
> Per table write ID implementation (HIVE-18192) introduces couple of
> metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids
> allocated per table.
> Now, when we rename any tables, it is necessary to update the corresponding
> table names in these metatables as well. Otherwise, ACID table operations
> won't work properly.
> Since, this change is significant and have other side-effects, we propose to
> disable rename tables on ACID tables until a fix is figured out.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[ https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481106#comment-16481106 ] Eugene Koifman commented on HIVE-18748: --- same patch again - looks like pom has been updated > Rename table impacts the ACID behavior as table names are not updated in > meta-tables. > - > > Key: HIVE-18748 > URL: https://issues.apache.org/jira/browse/HIVE-18748 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Eugene Koifman >Priority: Critical > Labels: ACID, DDL > Attachments: HIVE-18748.01-branch-3.patch, > HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch, > HIVE-18748.03-branch-3.patch, HIVE-18748.03.patch, HIVE-18748.04.patch, > HIVE-18748.05.patch > > > ACID implementation uses metatables such as TXN_COMPONENTS, > COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to > manage ACID operations. > Per table write ID implementation (HIVE-18192) introduces couple of > metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids > allocated per table. > Now, when we rename any tables, it is necessary to update the corresponding > table names in these metatables as well. Otherwise, ACID table operations > won't work properly. > Since, this change is significant and have other side-effects, we propose to > disable rename tables on ACID tables until a fix is figured out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480084#comment-16480084
]
Hive QA commented on HIVE-18748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923949/HIVE-18748.02-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11039/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11039/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11039/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-05-18 03:21:51.772
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11039/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z branch-3 ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-05-18 03:21:51.775
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 0b6bea8 HIVE-19572: Add option to mask stats and data size in q
files (Jesus Camacho Rodriguez, reviewed by Prasanth Jayachandran) (addendum)
+ git clean -f -d
+ git checkout branch-3
Switched to branch 'branch-3'
Your branch is up-to-date with 'origin/branch-3'.
+ git reset --hard origin/branch-3
HEAD is now at 9cd6258 HIVE-19317 : Handle schema evolution from int like types
to decimal (Janaki Lahorani, reviewed by Vihang Karajgaonkar)
+ git merge --ff-only origin/branch-3
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-05-18 03:21:53.555
+ rm -rf ../yetus_PreCommit-HIVE-Build-11039
+ mkdir ../yetus_PreCommit-HIVE-Build-11039
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11039
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11039/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] Failed to execute goal on project hive-standalone-metastore: Could not
resolve dependencies for project
org.apache.hive:hive-standalone-metastore:jar:3.0.0: Could not find artifact
org.apache.hive:hive-storage-api:jar:2.6.0-SNAPSHOT in apache.snapshots
(https://repository.apache.org/snapshots) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-standalone-metastore
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923949 - PreCommit-HIVE-Build
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch,
> HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch, HIVE-18748.03.patch,
> HIVE-18748.04.patch, HIVE-18748.05.patch
>
>
> ACID
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[ https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479328#comment-16479328 ] Eugene Koifman commented on HIVE-18748: --- committed to master Thanks Sankar for the reriew > Rename table impacts the ACID behavior as table names are not updated in > meta-tables. > - > > Key: HIVE-18748 > URL: https://issues.apache.org/jira/browse/HIVE-18748 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Eugene Koifman >Priority: Critical > Labels: ACID, DDL > Attachments: HIVE-18748.01-branch-3.patch, > HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch, HIVE-18748.03.patch, > HIVE-18748.04.patch, HIVE-18748.05.patch > > > ACID implementation uses metatables such as TXN_COMPONENTS, > COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to > manage ACID operations. > Per table write ID implementation (HIVE-18192) introduces couple of > metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids > allocated per table. > Now, when we rename any tables, it is necessary to update the corresponding > table names in these metatables as well. Otherwise, ACID table operations > won't work properly. > Since, this change is significant and have other side-effects, we propose to > disable rename tables on ACID tables until a fix is figured out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[ https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16479323#comment-16479323 ] Eugene Koifman commented on HIVE-18748: --- rebased branch-3 patch > Rename table impacts the ACID behavior as table names are not updated in > meta-tables. > - > > Key: HIVE-18748 > URL: https://issues.apache.org/jira/browse/HIVE-18748 > Project: Hive > Issue Type: Sub-task > Components: HiveServer2, Transactions >Affects Versions: 3.0.0 >Reporter: Sankar Hariappan >Assignee: Eugene Koifman >Priority: Critical > Labels: ACID, DDL > Attachments: HIVE-18748.01-branch-3.patch, > HIVE-18748.02-branch-3.patch, HIVE-18748.02.patch, HIVE-18748.03.patch, > HIVE-18748.04.patch, HIVE-18748.05.patch > > > ACID implementation uses metatables such as TXN_COMPONENTS, > COMPLETED_TXN_COMPONENTS, COMPACTION_QUEUE, COMPLETED_COMPCTION_QUEUE etc to > manage ACID operations. > Per table write ID implementation (HIVE-18192) introduces couple of > metatables such as NEXT_WRITE_ID and TXN_TO_WRITE_ID to manage write ids > allocated per table. > Now, when we rename any tables, it is necessary to update the corresponding > table names in these metatables as well. Otherwise, ACID table operations > won't work properly. > Since, this change is significant and have other side-effects, we propose to > disable rename tables on ACID tables until a fix is figured out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18748) Rename table impacts the ACID behavior as table names are not updated in meta-tables.
[
https://issues.apache.org/jira/browse/HIVE-18748?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478935#comment-16478935
]
Hive QA commented on HIVE-18748:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12923851/HIVE-18748.01-branch-3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/11025/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/11025/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-11025/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-05-17 11:41:18.386
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-11025/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z branch-3 ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-05-17 11:41:18.389
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at b329afa HIVE-19572: Add option to mask stats and data size in q
files (Jesus Camacho Rodriguez, reviewed by Prasanth Jayachandran) (addendum)
+ git clean -f -d
+ git checkout branch-3
Switched to branch 'branch-3'
Your branch is up-to-date with 'origin/branch-3'.
+ git reset --hard origin/branch-3
HEAD is now at 9cd6258 HIVE-19317 : Handle schema evolution from int like types
to decimal (Janaki Lahorani, reviewed by Vihang Karajgaonkar)
+ git merge --ff-only origin/branch-3
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-05-17 11:41:20.159
+ rm -rf ../yetus_PreCommit-HIVE-Build-11025
+ mkdir ../yetus_PreCommit-HIVE-Build-11025
+ git gc
+ cp -R . ../yetus_PreCommit-HIVE-Build-11025
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-11025/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: git apply -p0
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
[ERROR] Failed to execute goal on project hive-standalone-metastore: Could not
resolve dependencies for project
org.apache.hive:hive-standalone-metastore:jar:3.0.0: Could not find artifact
org.apache.hive:hive-storage-api:jar:2.6.0-SNAPSHOT in apache.snapshots
(https://repository.apache.org/snapshots) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please
read the following articles:
[ERROR] [Help 1]
http://cwiki.apache.org/confluence/display/MAVEN/DependencyResolutionException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn -rf :hive-standalone-metastore
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12923851 - PreCommit-HIVE-Build
> Rename table impacts the ACID behavior as table names are not updated in
> meta-tables.
> -
>
> Key: HIVE-18748
> URL: https://issues.apache.org/jira/browse/HIVE-18748
> Project: Hive
> Issue Type: Sub-task
> Components: HiveServer2, Transactions
>Affects Versions: 3.0.0
>Reporter: Sankar Hariappan
>Assignee: Eugene Koifman
>Priority: Critical
> Labels: ACID, DDL
> Attachments: HIVE-18748.01-branch-3.patch, HIVE-18748.02.patch,
> HIVE-18748.03.patch, HIVE-18748.04.patch, HIVE-18748.05.patch
>
>
> ACID implementation uses metatables such as
