[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-12 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360869#comment-16360869
 ] 

Hive QA commented on HIVE-17983:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  7m 
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 17m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  8m 
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  7m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  7m 
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 15m 
20s{color} | {color:red} root: The patch generated 430 new + 70776 unchanged - 
5 fixed = 71206 total (was 70781) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
17s{color} | {color:red} beeline: The patch generated 14 new + 676 unchanged - 
14 fixed = 690 total (was 690) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}1449m 
52s{color} | {color:red} itests/hive-unit: The patch generated 1 new + 0 
unchanged - 2729 fixed = 1 total (was 2729) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  2m 
25s{color} | {color:red} standalone-metastore: The patch generated 430 new + 
5475 unchanged - 5 fixed = 5905 total (was 5480) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
2s{color} | {color:red} The patch has 24 line(s) that end in whitespace. Use 
git apply --whitespace=fix <>. Refer 
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
4s{color} | {color:red} The patch 121 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m 
35s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 13m 
28s{color} | {color:green} root generated 0 new + 336 unchanged - 1 fixed = 336 
total (was 337) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
19s{color} | {color:green} beeline in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
38s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
16s{color} | {color:green} metastore in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
20s{color} | {color:green} packaging in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
16s{color} | {color:green} standalone-metastore generated 0 new + 61 unchanged 
- 1 fixed = 61 total (was 62) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red}  0m 
22s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black}1545m 19s{color} 
| {color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  
xml  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 9fdb601 |

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-11 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359958#comment-16359958
 ] 

Hive QA commented on HIVE-17983:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910058/HIVE-17983.6.patch

{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 23 failed/errored test(s), 13168 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
 (batchId=241)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=79)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
 (batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] 
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
 (batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] 
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
 (batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
 (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
 (batchId=180)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] 
(batchId=122)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1] 
(batchId=251)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=222)
org.apache.hadoop.hive.metastore.client.TestListPartitions.testListPartitionNamesByValuesNullDbName[Remote]
 (batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
 (batchId=206)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
 (batchId=206)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=257)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=235)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=235)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9158/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9158/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9158/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 23 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12910058 - PreCommit-HIVE-Build

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, 
> HIVE-17983.4.patch, HIVE-17983.5.patch, HIVE-17983.6.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-09 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16359180#comment-16359180
 ] 

Hive QA commented on HIVE-17983:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909832/HIVE-17983.5.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9128/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9128/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9128/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:55.786
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9128/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:55.789
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 89e75c7 HIVE-17837: Explicitly check if the HoS Remote Driver 
has been lost in the RemoteSparkJobMonitor (Sahil Takiar, reviewed by Rui Li)
+ git clean -f -d
Removing ${project.basedir}/
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 89e75c7 HIVE-17837: Explicitly check if the HoS Remote Driver 
has been lost in the RemoteSparkJobMonitor (Sahil Takiar, reviewed by Rui Li)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-10 01:20:58.437
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9128/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: 
beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java:159
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java' 
with conflicts.
error: patch failed: 
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:209
Falling back to three-way merge...
Applied patch to 
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java'
 with conflicts.
error: patch failed: 
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java:56
Falling back to three-way merge...
Applied patch to 
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java'
 with conflicts.
Going to apply patch with: git apply -p0
/data/hiveptest/working/scratch/build.patch:699: trailing whitespace.
You can download the Oracle driver at 
/data/hiveptest/working/scratch/build.patch:724: trailing whitespace.
same directory as this NOTICE file.  
/data/hiveptest/working/scratch/build.patch:1023: trailing whitespace.

/data/hiveptest/working/scratch/build.patch:3713: trailing whitespace.
elif [ "${METASTORE_AUX_JARS_PATH}" != "" ]; then 
/data/hiveptest/working/scratch/build.patch:3731: trailing whitespace.
export HADOOP_HOME_WARN_SUPPRESS=true 
error: patch failed: 
beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java:159
Falling back to three-way merge...
Applied patch to 'beeline/src/java/org/apache/hive/beeline/HiveSchemaTool.java' 
with conflicts.
error: patch failed: 
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java:209
Falling back to three-way merge...
Applied patch to 
'standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/MetaStoreSchemaInfo.java'
 with conflicts.
error: patch failed: 
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/tools/HiveSchemaHelper.java:56
Falling

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-07 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356210#comment-16356210
 ] 

Hive QA commented on HIVE-17983:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12909534/HIVE-17983.4.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9078/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9078/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9078/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-02-07 23:04:24.529
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-9078/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-02-07 23:04:24.532
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at ee2c814 HIVE-18642 : incorrect assertion in TezSessionPool for 
WM (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at ee2c814 HIVE-18642 : incorrect assertion in TezSessionPool for 
WM (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-02-07 23:04:27.095
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-9078/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: 
metastore/scripts/upgrade/mysql/hive-txn-schema-3.0.0.mysql.sql:1
error: metastore/scripts/upgrade/mysql/hive-txn-schema-3.0.0.mysql.sql: patch 
does not apply
error: patch failed: 
metastore/scripts/upgrade/oracle/hive-txn-schema-3.0.0.oracle.sql:1
error: metastore/scripts/upgrade/oracle/hive-txn-schema-3.0.0.oracle.sql: patch 
does not apply
error: patch failed: 
metastore/scripts/upgrade/postgres/hive-txn-schema-3.0.0.postgres.sql:1
error: metastore/scripts/upgrade/postgres/hive-txn-schema-3.0.0.postgres.sql: 
patch does not apply
error: patch failed: 
metastore/scripts/upgrade/oracle/hive-schema-3.0.0.oracle.sql:615
Falling back to three-way merge...
Applied patch to 
'standalone-metastore/src/main/sql/oracle/hive-schema-3.0.0.oracle.sql' with 
conflicts.
fatal: git apply: bad git-diff - inconsistent old filename on line 6283
fatal: git diff header lacks filename information when removing 2 leading 
pathname components (line 39)
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12909534 - PreCommit-HIVE-Build

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, 
> HIVE-17983.4.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-07 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16356164#comment-16356164
 ] 

Thejas M Nair commented on HIVE-17983:
--

+1 pending tests

Thanks for adding the docker support for being able to test scripts changes 
easily! That really helps with developer productivity!
It would be good to get this in soon, as there are some other db script changes 
in pipeline blocked on this (ie, they would conflict with this)

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, 
> HIVE-17983.4.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-02-06 Thread Alan Gates (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16354821#comment-16354821
 ] 

Alan Gates commented on HIVE-17983:
---

Patch 4 is identical to patch 3, uploaded again in order to re-run tests.

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, 
> HIVE-17983.4.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-30 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346058#comment-16346058
 ] 

Hive QA commented on HIVE-17983:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m  
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  5m 
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  6m 
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 16m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  7m 
38s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  1m 
40s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
19s{color} | {color:red} beeline in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
33s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
36s{color} | {color:red} standalone-metastore in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
18s{color} | {color:red} beeline in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
37s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 18s{color} 
| {color:red} beeline in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 37s{color} 
| {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 14m  
4s{color} | {color:red} root: The patch generated 434 new + 71012 unchanged - 7 
fixed = 71446 total (was 71019) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
17s{color} | {color:red} beeline: The patch generated 4 new + 690 unchanged - 4 
fixed = 694 total (was 694) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  1m 
16s{color} | {color:red} standalone-metastore: The patch generated 430 new + 
5443 unchanged - 3 fixed = 5873 total (was 5446) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
1s{color} | {color:red} The patch has 24 line(s) that end in whitespace. Use 
git apply --whitespace=fix <>. Refer 
https://git-scm.com/docs/git-apply {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
1s{color} | {color:red} The patch 123 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
6s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  4m 
51s{color} | {color:red} root in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  
2s{color} | {color:red} beeline in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  
1s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  
2s{color} | {color:red} metastore in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  
2s{color} | {color:red} packaging in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red}  0m  
1s{color} | {color:red} standalone-metastore in the patch failed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:blue}0{color} | {color:blue} asflicense {color} | {color:blue}  0m  
3s{color} | {color:blue} ASF License check generated no output? {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 71m 54s{color} | 
{color:black} {color} |
\\
\\
||

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-30 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16346055#comment-16346055
 ] 

Hive QA commented on HIVE-17983:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12908404/HIVE-17983.3.patch

{color:green}SUCCESS:{color} +1 due to 8 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 12875 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
 (batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook] 
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
 (batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb] 
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
 (batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
 (batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid] 
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
 (batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
 (batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb] 
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_input_format_excludes]
 (batchId=163)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5] 
(batchId=122)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut 
(batchId=221)
org.apache.hadoop.hive.ql.exec.TestOperators.testNoConditionalTaskSizeForLlap 
(batchId=282)
org.apache.hadoop.hive.ql.io.TestDruidRecordWriter.testWrite (batchId=256)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=188)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8935/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8935/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8935/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12908404 - PreCommit-HIVE-Build

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.3.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-26 Thread Vihang Karajgaonkar (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16341373#comment-16341373
 ] 

Vihang Karajgaonkar commented on HIVE-17983:


bq. Also on these scripts I have unrolled them so that scripts no longer invoke 
other scripts. For example, the 2.3->3.0 upgrade script now includes all the 
create table and alter table statement itself rather than calling run on the 
various 0XX-HIVE-X.rdbms.sql scripts. The main reason for this is that 
HiveSchemaTool went to a lot of work to do the unrolling on these scripts.

I think having separate patch scripts which a main upgrade scripts calls is a 
cleaner way to apply schema changes. It also helps with the backporting of the 
schema changes between branches (less conflicts and chances of errors). If it 
is possible we should probably try to maintain that in the standalone-metastore 
as well. Looks like sqlline does have {{!script}} and {{!run}} commands to run 
scripts. Can we use it for this usecase? [~ngangam] also has worked a lot on 
schema upgrade patches and he might have some thoughts as well.

Also, I agree with [~thejas] above and we should provide an upgrade path from 
atleast 2.0.0 so that it is easier for users to adopt it. Its not necessary 
that all the users will be at 2.3.0 when they want to adopt 
standalone-metastore.

bq. how Hive and the standalone metastore should be installed. Do we completely 
separate them out and require users to install the standalone metastore and 
then Hive? This is easier for devs but harder on ops and packagers. But it also 
gives users maximum flexibility. Or do we modify the Hive build process to pull 
in the standalone metastore packages and produce a distribution that includes 
the metastore? This is more work for us devs. It gives users a seamless 
experience between older and newer versions of Hive. It also matches user 
expectations (I can't think of any database that requires you to install its 
data catalog as a separate package). On the other hand it locksteps a version 
of Hive with a version of the metastore, which may not be what people want.

I think second option here is a better alternative considering we want to 
maintain backwards compatibility as much as possible and make it easier to 
migrate from current metastore to standalone-metastore. Hive is anyways in 
lockstep with metastore version and a lot of code in Hive assumes that. I am 
not sure we are there yet to have one version of hive talking to another 
version of metastore and that should be a separate effort.

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-22 Thread Vihang Karajgaonkar (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16334825#comment-16334825
 ] 

Vihang Karajgaonkar commented on HIVE-17983:


Thanks [~alangates] for the patch. Left some comments on the PR.

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-19 Thread Alan Gates (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333141#comment-16333141
 ] 

Alan Gates commented on HIVE-17983:
---

Second version of the patch.  One big change in this one, it comments out the 
changes to COMPLETED_TXN_COMPONENTS in the Oracle scripts because those changes 
fail.  I have talked to [~jcamachorodriguez] regarding this and he'll take a 
look.  (The issue in a nutshell is that identify columns aren't supported until 
Oracle 12c and the old method of using a trigger didn't seem to work, I suspect 
because the oracle in our docker container doesn't support stored procedures.)

Also I removed the hive-schema-3.0.0 and upgrade-2.3.0-3.0.0 scripts in this 
patch so that developers make sure and put any changes in the correct location.

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
> Attachments: HIVE-17983.2.patch, HIVE-17983.patch
>
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2018-01-16 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16327915#comment-16327915
 ] 

ASF GitHub Bot commented on HIVE-17983:
---

GitHub user alanfgates opened a pull request:

https://github.com/apache/hive/pull/291

HIVE-17983 Make the standalone metastore generate tarballs etc.

See JIRA for full comments.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/alanfgates/hive hive17983

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/291.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #291


commit 1ba9b62d9ef488355e1a97dbc7237c1472349a24
Author: Alan Gates 
Date:   2017-10-19T23:49:38Z

HIVE-17983 Make the standalone metastore generate tarballs etc.




> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>Priority: Major
>  Labels: pull-request-available
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2017-11-06 Thread Thejas M Nair (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16241135#comment-16241135
 ] 

Thejas M Nair commented on HIVE-17983:
--

bq. On the installation and upgrade scripts I have only copied the 2.3 and 3.0 
installation scripts and 2.3->3.0 upgrade. My assumption is that the standalone 
metastore will be used with Hive 3.0 or later, so it doesn't make sense to copy 
all the older scripts. I copied in the 2.3 installation scripts so that we 
could test the upgrade procedure.

While install scripts of old versions are not needed, I think it would be 
useful to have the upgrade scripts from old versions. We could limit it to 
starting from say 2.0.0 or 1.0.0, but given the 0 maintenance cost of older 
version upgrade files, we could go all the way.



> Make the standalone metastore generate tarballs etc.
> 
>
> Key: HIVE-17983
> URL: https://issues.apache.org/jira/browse/HIVE-17983
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Alan Gates
>Assignee: Alan Gates
>
> In order to be separately installable the standalone metastore needs its own 
> tarballs, startup scripts, etc.  All of the SQL installation and upgrade 
> scripts also need to move from metastore to standalone-metastore.
> I also plan to create Dockerfiles for different database types so that 
> developers can test the SQL installation and upgrade scripts.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (HIVE-17983) Make the standalone metastore generate tarballs etc.

2017-11-04 Thread Alan Gates (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16239245#comment-16239245
 ] 

Alan Gates commented on HIVE-17983:
---

I have committed a patch for this to the standalone-metastore branch.  There is 
much in this patch to comment on.

On the standalone-metastore side, this patch adds several things.  
# creation of tarballs for both both source and binary distributions, hopefully 
along with the necessary license information;
# install and upgrade scripts for the various RDBMS types (more on this below);
# a version of HiveSchemaTool (called MetastoreSchemaTool) for the standalone 
metastore;
# docker files for the various RDBMS types for testing (more on this below);
# shell scripts for running the schema tool and starting the metastore;
# log4j configuration for the metastore server process (this config is not what 
we want as it dumps the logfile in the current working directory, this should 
be changed to match Hive's behavior of where the file is written).

On the docker files, I have added ones for mysql and postgres.  I haven't done 
oracle or sqlserver yet.  The docker files are generated as part of the build, 
but the build does not call {{docker build}}.  This is partly because building 
the images is time consuming and eats up several G of disk space.  But for 
oracle there are also license issues that prevent the automatic inclusion of 
the docker images.  Right now the resulting containers just have the required 
RDBMS, Hadoop, and the metastore distribution tarball (unpacked).  My goal is 
to get to a point where different docker files are created for users to test 
against the metastore, and for automated testing of installation and upgrade of 
each RDBMS type.

On the installation and upgrade scripts I have only copied the 2.3 and 3.0 
installation scripts and 2.3->3.0 upgrade.  My assumption is that the 
standalone metastore will be used with Hive 3.0 or later, so it doesn't make 
sense to copy all the older scripts.  I copied in the 2.3 installation scripts 
so that we could test the upgrade procedure.  

Also on these scripts I have unrolled them so that scripts no longer invoke 
other scripts.  For example, the 2.3->3.0 upgrade script now includes all the 
create table and alter table statement itself rather than calling run on the 
various 0XX-HIVE-X.rdbms.sql scripts.  The main reason for this is that 
HiveSchemaTool went to a lot of work to do the unrolling on these scripts.  As 
part of copying HiveSchemaTool and I had to convert it to use SqlLine rather 
than Beeline (since the metastore does not have access to beeline) and I did 
not want to go through the work of making the unrolling work for SqlLine.  And 
I saw no advantage to having every DB change in a separate script.  Our tools 
only support upgrade between versions.  I suspect these separate updates are a 
holdover from the days when Facebook used to run Hive top of trunk internally 
and thus wanted to be able to apply each change discretely.

This patch does not remove the RDBMS scripts from metastore or HiveSchemaTool 
from beeline.  There are two reasons for this.  One, the Hive information 
schema depends on HiveSchemaTool to setup a series of tables in Hive via 
beeline.  The metastore version of SchemaTool can't do this, because it doesn't 
have access to beeline.

But the second and much large reason is this brings up the question of how Hive 
and the standalone metastore should be installed.  Do we completely separate 
them out and require users to install the standalone metastore and then Hive?  
This is easier for devs but harder on ops and packagers.  But it also gives 
users maximum flexibility.  Or do we modify the Hive build process to pull in 
the standalone metastore packages and produce a distribution that includes the 
metastore?  This is more work for us devs.  It gives users a seamless 
experience between older and newer versions of Hive.  It also matches user 
expectations (I can't think of any database that requires you to install its 
data catalog as a separate package).  On the other hand it locksteps a version 
of Hive with a version of the metastore, which may not be what people want.  I 
don't propose to answer these questions in this JIRA, but I wanted to bring 
them up so we can start discussing them.

Leaving copies of the installation and upgrade scripts in metastore and 
HiveSchemaTool in beeline that duplicate much code that's also in 
standalone-metastore is obviously not a viable long term solution.  We will 
need some combination of separating things cleanly and refactoring so that a 
minimum amount of code is duplicated.  But until we answer the questions above 
we won't know which way to go so I've left it like this for the moment.

> Make the standalone metastore generate tarballs etc.
> 
>
> Key: