[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575742#comment-15575742 ] Aihua Xu commented on HIVE-14926: - Yeah. It won't work with Oracle and mysql at least. Seems we may have to leave like this. > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575670#comment-15575670 ] Chaoyu Tang commented on HIVE-14926: [~aihuaxu] Left comments in RB. AFAIK, DDLs would not work with the autocommit false for most RDBMS and their drivers. > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15575370#comment-15575370 ] Aihua Xu commented on HIVE-14926: - The tests are not related to the change. > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[
https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15574567#comment-15574567
]
Hive QA commented on HIVE-14926:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833201/HIVE-14926.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10522 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapLocalCliDriver - did not produce a TEST-*.xml file
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[acid_globallimit]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[order_null]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[reloadJar]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_fast_stats]
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_bulk]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJarWithoutAddDriverClazz[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[0]
org.apache.hive.beeline.TestBeelineArgParsing.testAddLocalJar[1]
org.apache.hive.jdbc.TestNoSaslAuth.org.apache.hive.jdbc.TestNoSaslAuth
org.apache.hive.jdbc.authorization.TestHS2AuthzContext.org.apache.hive.jdbc.authorization.TestHS2AuthzContext
org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext.org.apache.hive.jdbc.authorization.TestHS2AuthzSessionContext
org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth.org.apache.hive.jdbc.authorization.TestJdbcMetadataApiAuth
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthUDFBlacklist.testBlackListedUdfUsage
org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization.org.apache.hive.jdbc.authorization.TestJdbcWithSQLAuthorization
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1558/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1558/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1558/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833201 - PreCommit-HIVE-Build
> Keep Schema in consistent state where schemaTool fails or succeeds.
> -
>
> Key: HIVE-14926
> URL: https://issues.apache.org/jira/browse/HIVE-14926
> Project: Hive
> Issue Type: Improvement
> Components: Database/Schema
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch
>
>
> SchemaTool uses autocommit right now when executing the upgrade or init
> scripts. Seems we should use database transaction to commit or roll back to
> keep schema consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[
https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15573980#comment-15573980
]
Hive QA commented on HIVE-14926:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833201/HIVE-14926.2.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1547/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1547/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.908
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'M2_OPTS=-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ M2_OPTS='-Xmx1g -XX:MaxPermSize=256m -Dhttp.proxyHost=localhost
-Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1547/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:14.910
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
b9e8157..eacf9f9 hive-14535 -> origin/hive-14535
+ git reset --hard HEAD
HEAD is now at 10b866a HIVE-13798: Fix the unit test failure
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_ivyDownload (Aihua Xu,
reviewed by Ashutosh Chauhan) (addendum)
+ git clean -f -d
+ git checkout master
Switched to branch 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 5353161 HIVE-14835: Improve ptest2 build time (Prasanth
Jayachandran reviewed by Sergio Pena)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-10-14 03:02:17.719
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 44: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 82: patch: command
not found
/data/hiveptest/working/scratch/smart-apply-patch.sh: line 84: patch: command
not found
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaHelper.java: No
such file or directory
error: a/beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java: No such
file or directory
error:
a/itests/hive-unit/src/test/java/org/apache/hive/beeline/TestSchemaTool.java:
No such file or directory
error:
a/metastore/src/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12833201 - PreCommit-HIVE-Build
> Keep Schema in consistent state where schemaTool fails or succeeds.
> -
>
> Key: HIVE-14926
> URL: https://issues.apache.org/jira/browse/HIVE-14926
> Project: Hive
> Issue Type: Improvement
> Components: Database/Schema
>Reporter: Aihua Xu
>Assignee: Aihua Xu
> Attachments: HIVE-14926.1.patch, HIVE-14926.2.patch
>
>
> SchemaTool uses autocommit right now when executing the upgrade or init
> scripts. Seems we should use database transaction to commit or roll back to
> keep schema consistent.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572357#comment-15572357 ] Aihua Xu commented on HIVE-14926: - Seems I forgot to update TestSchemaTool.java. Will do that. > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[
https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572259#comment-15572259
]
Hive QA commented on HIVE-14926:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12833143/HIVE-14926.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1532/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1532/console
Test logs:
http://ec2-204-236-174-241.us-west-1.compute.amazonaws.com/logs/PreCommit-HIVE-Build-1532/
Messages:
{noformat}
This message was trimmed, see log for full details
[exec] + local md5File=spark-2.0.0-bin-hadoop2-without-hive.tgz.md5sum
[exec] + curl -Sso
./../thirdparty/spark-2.0.0-bin-hadoop2-without-hive.tgz.md5sum
http://d3jw87u4immizc.cloudfront.net/spark-tarball/spark-2.0.0-bin-hadoop2-without-hive.tgz.md5sum
[exec] + cd ./../thirdparty
[exec] + md5sum -c spark-2.0.0-bin-hadoop2-without-hive.tgz.md5sum
[exec] + cd -
[exec] + tar -zxf ./../thirdparty/spark-2.0.0-bin-hadoop2-without-hive.tgz
-C ./target
[exec] spark-2.0.0-bin-hadoop2-without-hive.tgz: OK
[exec]
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit
[exec] + mv ./target/spark-2.0.0-bin-hadoop2-without-hive ./target/spark
[exec] + cp -f ./target/../../..//data/conf/spark/log4j2.properties
./target/spark/conf/
[INFO] Executed tasks
[INFO]
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ hive-it-unit
---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @
hive-it-unit ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/main/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (define-classpath) @ hive-it-unit ---
[INFO] Executing tasks
main:
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:compile (default-compile) @ hive-it-unit
---
[INFO] Compiling 2 source files to
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/classes
[INFO]
[INFO] --- maven-resources-plugin:2.6:testResources (default-testResources) @
hive-it-unit ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/test/resources
[INFO] Copying 3 resources
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-test-dirs) @ hive-it-unit ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/tmp
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/warehouse
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/tmp/conf
[copy] Copying 15 files to
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/tmp/conf
[INFO] Executed tasks
[INFO]
[INFO] --- maven-antrun-plugin:1.7:run (setup-metastore-scripts) @ hive-it-unit
---
[INFO] Executing tasks
main:
[mkdir] Created dir:
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/tmp/scripts/metastore
[copy] Copying 337 files to
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/tmp/scripts/metastore
[INFO] Executed tasks
[INFO]
[INFO] --- maven-compiler-plugin:3.1:testCompile (default-testCompile) @
hive-it-unit ---
[INFO] Compiling 114 source files to
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/target/test-classes
[INFO] -
[WARNING] COMPILATION WARNING :
[INFO] -
[WARNING]
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java:
Some input files use or override a deprecated API.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/history/TestHiveHistory.java:
Recompile with -Xlint:deprecation for details.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
Some input files use unchecked or unsafe operations.
[WARNING]
/data/hive-ptest/working/apache-github-source-source/itests/hive-unit/src/test/java/org/apache/hadoop/hive/metastore/TestHiveMetaStore.java:
Recompile with -Xlint:unchecked for details.
[INFO] 4 warnings
[INFO] -
[INFO]
[jira] [Commented] (HIVE-14926) Keep Schema in consistent state where schemaTool fails or succeeds.
[ https://issues.apache.org/jira/browse/HIVE-14926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15572221#comment-15572221 ] Aihua Xu commented on HIVE-14926: - FYI: [~ychena] and [~ashutoshc] Can you help review the change since you are more familiar with it? > Keep Schema in consistent state where schemaTool fails or succeeds. > - > > Key: HIVE-14926 > URL: https://issues.apache.org/jira/browse/HIVE-14926 > Project: Hive > Issue Type: Improvement > Components: Database/Schema >Reporter: Aihua Xu >Assignee: Aihua Xu > Attachments: HIVE-14926.1.patch > > > SchemaTool uses autocommit right now when executing the upgrade or init > scripts. Seems we should use database transaction to commit or roll back to > keep schema consistent. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
