[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743852#comment-15743852
]
Hive QA commented on HIVE-15118:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12842887/HIVE-15118.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 10811 tests
executed
*Failed tests:*
{noformat}
TestVectorizedColumnReaderBase - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=135)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_based_fetch_decision]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=93)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2551/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2551/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2551/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12842887 - PreCommit-HIVE-Build
> Remove unused 'COLUMNS' table from derby schema
> ---
>
> Key: HIVE-15118
> URL: https://issues.apache.org/jira/browse/HIVE-15118
> Project: Hive
> Issue Type: Sub-task
> Components: Database/Schema
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch,
> HIVE-15118.3.patch
>
>
> COLUMNS table is unused any more. Other databases already removed it. Remove
> from derby as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743444#comment-15743444
]
Naveen Gangam commented on HIVE-15118:
--
[~ychena] It was done it {{008-HIVE-2246.derby.sql}} file. There is a
{{008-REVERT-HIVE-2246.derby.sql}} script that reverts this change where the
rename occurs, but it never gets called from any of the real upgrade scripts.
So I assume it was meant for folks who wanted to manually revert the change
> Remove unused 'COLUMNS' table from derby schema
> ---
>
> Key: HIVE-15118
> URL: https://issues.apache.org/jira/browse/HIVE-15118
> Project: Hive
> Issue Type: Sub-task
> Components: Database/Schema
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch,
> HIVE-15118.3.patch
>
>
> COLUMNS table is unused any more. Other databases already removed it. Remove
> from derby as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[ https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743435#comment-15743435 ] Yongzhi Chen commented on HIVE-15118: - [~aihuaxu], could you point to me where is the script that change columns to columns_old? > Remove unused 'COLUMNS' table from derby schema > --- > > Key: HIVE-15118 > URL: https://issues.apache.org/jira/browse/HIVE-15118 > Project: Hive > Issue Type: Sub-task > Components: Database/Schema >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Minor > Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch, > HIVE-15118.3.patch > > > COLUMNS table is unused any more. Other databases already removed it. Remove > from derby as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[ https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743365#comment-15743365 ] Naveen Gangam commented on HIVE-15118: -- LGTM pending tests .. +1 for me. > Remove unused 'COLUMNS' table from derby schema > --- > > Key: HIVE-15118 > URL: https://issues.apache.org/jira/browse/HIVE-15118 > Project: Hive > Issue Type: Sub-task > Components: Database/Schema >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Minor > Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch, > HIVE-15118.3.patch > > > COLUMNS table is unused any more. Other databases already removed it. Remove > from derby as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15743236#comment-15743236
]
Naveen Gangam commented on HIVE-15118:
--
[~aihuaxu] Functionally, the second version of the patch looks fine. Just a
couple of nits:
1) the {{UPDATE "APP".VERSION SET SCHEMA_VERSION=}} command should be the last
command in this file for a couple of reasons a) for consistency with other
upgrade schema files and more importantly 2) the version in this table is
pretty heavily relied upon to determine the schema version. So the upgrade will
not be considered complete until the version is set. If for some reason, the
upgrade script terminates abruptly before this version is set, the version
would not be set and would be and indication that something went wrong during
upgrade. With the current patch, if the script ends right after the version is
set, the drop table command will not be processed but the schema version would
indicate that it succeeded.
2) Can you move the {{DROP TABLE}} command to a separate file, say
037-HIVE-15118.derby.sql and then just call RUN on this file from the upgrade
script? Thanks
> Remove unused 'COLUMNS' table from derby schema
> ---
>
> Key: HIVE-15118
> URL: https://issues.apache.org/jira/browse/HIVE-15118
> Project: Hive
> Issue Type: Sub-task
> Components: Database/Schema
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch
>
>
> COLUMNS table is unused any more. Other databases already removed it. Remove
> from derby as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15736388#comment-15736388
]
Hive QA commented on HIVE-15118:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12842601/HIVE-15118.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 11 failed/errored test(s), 10792 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample2] (batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample4] (batchId=15)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=61)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=38)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[transform_ppr2]
(batchId=134)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[stats_based_fetch_decision]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_5]
(batchId=91)
org.apache.hive.beeline.TestSchemaTool.testSchemaUpgrade (batchId=209)
org.apache.hive.hcatalog.api.TestHCatClientNotification.createTable
(batchId=219)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/2527/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/2527/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-2527/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 11 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12842601 - PreCommit-HIVE-Build
> Remove unused 'COLUMNS' table from derby schema
> ---
>
> Key: HIVE-15118
> URL: https://issues.apache.org/jira/browse/HIVE-15118
> Project: Hive
> Issue Type: Sub-task
> Components: Database/Schema
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15118.1.patch, HIVE-15118.2.patch
>
>
> COLUMNS table is unused any more. Other databases already removed it. Remove
> from derby as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[ https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15735657#comment-15735657 ] Naveen Gangam commented on HIVE-15118: -- [~aihuaxu] The main schema file changes looks good but it will not work for upgrade scenarios. When you upgrade from hive 2.1, this table will still exist. Can you also add changes for upgrade scenario? Thanks > Remove unused 'COLUMNS' table from derby schema > --- > > Key: HIVE-15118 > URL: https://issues.apache.org/jira/browse/HIVE-15118 > Project: Hive > Issue Type: Sub-task > Components: Database/Schema >Affects Versions: 2.2.0 >Reporter: Aihua Xu >Assignee: Aihua Xu >Priority: Minor > Attachments: HIVE-15118.1.patch > > > COLUMNS table is unused any more. Other databases already removed it. Remove > from derby as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15637191#comment-15637191
]
Hive QA commented on HIVE-15118:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12837164/HIVE-15118.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10628 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[join_acid_non_acid]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_fast_stats]
(batchId=145)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3]
(batchId=90)
org.apache.hadoop.hive.metastore.TestHiveMetaStoreStatsMerge.testStatsMerge
(batchId=188)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1967/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1967/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1967/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12837164 - PreCommit-HIVE-Build
> Remove unused 'COLUMNS' table from derby schema
> ---
>
> Key: HIVE-15118
> URL: https://issues.apache.org/jira/browse/HIVE-15118
> Project: Hive
> Issue Type: Improvement
> Components: Database/Schema
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-15118.1.patch
>
>
> COLUMNS table is unused any more. Other databases already removed it. Remove
> from derby as well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (HIVE-15118) Remove unused 'COLUMNS' table from derby schema
[
https://issues.apache.org/jira/browse/HIVE-15118?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15633084#comment-15633084
]
Hive QA commented on HIVE-15118:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12836841/HIVE-15118.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/1944/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/1944/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-1944/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2016-11-03 15:22:37.782
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ MAVEN_OPTS='-Xmx1g -Dhttp.proxyHost=localhost -Dhttp.proxyPort=3128'
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-1944/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2016-11-03 15:22:37.785
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 345353c HIVE-15039: A better job monitor console output for HoS
(Rui reviewed by Xuefu and Ferdinand)
+ git clean -f -d
Removing
ql/src/java/org/apache/hadoop/hive/ql/io/parquet/ParquetRecordReaderBase.java
Removing ql/src/java/org/apache/hadoop/hive/ql/io/parquet/vector/
Removing
ql/src/test/org/apache/hadoop/hive/ql/io/parquet/TestVectorizedColumnReader.java
Removing ql/src/test/queries/clientpositive/parquet_types_vectorization.q
Removing ql/src/test/results/clientpositive/parquet_types_vectorization.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 345353c HIVE-15039: A better job monitor console output for HoS
(Rui reviewed by Xuefu and Ferdinand)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2016-11-03 15:22:38.710
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
Going to apply patch with: patch -p1
patching file metastore/scripts/upgrade/derby/hive-schema-2.2.0.derby.sql
+ [[ maven == \m\a\v\e\n ]]
+ rm -rf /data/hiveptest/working/maven/org/apache/hive
+ mvn -B clean install -DskipTests -T 4 -q
-Dmaven.repo.local=/data/hiveptest/working/maven
ANTLR Parser Generator Version 3.4
org/apache/hadoop/hive/metastore/parser/Filter.g
DataNucleus Enhancer (version 4.1.6) for API "JDO"
DataNucleus Enhancer : Classpath
>> /usr/share/maven/boot/plexus-classworlds-2.x.jar
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MDatabase
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MFieldSchema
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MType
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MTable
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MConstraint
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MSerDeInfo
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MOrder
ENHANCED (Persistable) :
org.apache.hadoop.hive.metastore.model.MColumnDescriptor
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MStringList
ENHANCED (Persistable) :
org.apache.hadoop.hive.metastore.model.MStorageDescriptor
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MPartition
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MIndex
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MRole
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MRoleMap
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MGlobalPrivilege
ENHANCED (Persistable) : org.apache.hadoop.hive.metastore.model.MDBPrivilege
ENHANCED
