[jira] [Commented] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline

2018-01-20 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333419#comment-16333419
 ] 

Hive QA commented on HIVE-18493:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906991/HIVE-18493.04.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8737/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8737/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8737/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-01-21 07:35:55.964
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8737/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-01-21 07:35:55.967
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 90d236a HIVE-18231 : validate resource plan - part 2 - validate 
action and trigger expressions (Harish Jaiprakash, reviewed by Sergey Shelukhin)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 90d236a HIVE-18231 : validate resource plan - part 2 - validate 
action and trigger expressions (Harish Jaiprakash, reviewed by Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-01-21 07:35:56.501
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8737/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: cannot apply binary patch to 'data/files/test.parquet' without full 
index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test.parquet' without full 
index line
error: data/files/test.parquet: patch does not apply
error: src/java/org/apache/hive/beeline/BeeLine.java: does not exist in index
error: src/java/org/apache/hive/beeline/BeeLineOpts.java: does not exist in 
index
error: src/java/org/apache/hive/beeline/Rows.java: does not exist in index
error: src/main/resources/BeeLine.properties: does not exist in index
error: src/test/org/apache/hive/beeline/cli/TestHiveCli.java: does not exist in 
index
error: src/java/org/apache/hadoop/hive/cli/CliDriver.java: does not exist in 
index
error: src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in 
index
error: cannot apply binary patch to 'files/test.parquet' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test.parquet' without full index line
error: files/test.parquet: patch does not apply
error: 
hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java: does 
not exist in index
error: util/src/main/java/org/apache/hive/beeline/QFileBeeLineClient.java: does 
not exist in index
error: java/org/apache/hive/beeline/BeeLine.java: does not exist in index
error: java/org/apache/hive/beeline/BeeLineOpts.java: does not exist in index
error: java/org/apache/hive/beeline/Rows.java: does not exist in index
error: main/resources/BeeLine.properties: does not exist in index
error: test/org/apache/hive/beeline/cli/TestHiveCli.java: does not exist in 
index
error: java/org/apache/hadoop/hive/cli/CliDriver.java: does not exist in index
error: java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index
error: cannot apply binary patch to 'test.parquet' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'test.parquet' without full index line
error: test.parquet:

[jira] [Commented] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline

2018-01-20 Thread Matt McCline (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333418#comment-16333418
 ] 

Matt McCline commented on HIVE-18493:
-

Hive QA seems broken right now.  Wait on patch #4 until it is fixed.

> Add display escape for CR/LF to Hive CLI and Beeline
> 
>
> Key: HIVE-18493
> URL: https://issues.apache.org/jira/browse/HIVE-18493
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline, Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18493.01.patch, HIVE-18493.02.patch, 
> HIVE-18493.03.patch, HIVE-18493.04.patch
>
>
> Add optional display escaping of carriage return and line feed so row output 
> remains one line.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline

2018-01-20 Thread Matt McCline (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18493:

Attachment: HIVE-18493.04.patch

> Add display escape for CR/LF to Hive CLI and Beeline
> 
>
> Key: HIVE-18493
> URL: https://issues.apache.org/jira/browse/HIVE-18493
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline, Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18493.01.patch, HIVE-18493.02.patch, 
> HIVE-18493.03.patch, HIVE-18493.04.patch
>
>
> Add optional display escaping of carriage return and line feed so row output 
> remains one line.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18505) Added external hive configuration to prepDb in TxnDbUtil

2018-01-20 Thread ASF GitHub Bot (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333411#comment-16333411
 ] 

ASF GitHub Bot commented on HIVE-18505:
---

GitHub user chandulal opened a pull request:

https://github.com/apache/hive/pull/293

HIVE-18505 : Adding prepDb method that accept hive configs from outside



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/chandulal/apache-hive master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/293.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #293


commit 08d8de29831c9bea975107d5264894812337803b
Author: Chandu Kavar 
Date:   2018-01-21T06:20:15Z

HIVE-18505 : Adding prepDb method that accept hive configs from outside




> Added external hive configuration to prepDb in TxnDbUtil
> 
>
> Key: HIVE-18505
> URL: https://issues.apache.org/jira/browse/HIVE-18505
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Chandu Kavar
>Assignee: Chandu Kavar
>Priority: Minor
>  Labels: pull-request-available
>
> In Hive Metastore, We have TxtDbUtil.java and it contains few utils required 
> for tests. 
> There is prepDb() method, it is creating connection and execute some system 
> queries in order to prepare db.  While creating connection it's create new 
> HiveConf object and not taking configs from outside.
> TxtDbUtil.java should also contains prepDb method that can accept external 
> hive configs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18505) Added external hive configuration to prepDb in TxnDbUtil

2018-01-20 Thread ASF GitHub Bot (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ASF GitHub Bot updated HIVE-18505:
--
Labels: pull-request-available  (was: )

> Added external hive configuration to prepDb in TxnDbUtil
> 
>
> Key: HIVE-18505
> URL: https://issues.apache.org/jira/browse/HIVE-18505
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Chandu Kavar
>Assignee: Chandu Kavar
>Priority: Minor
>  Labels: pull-request-available
>
> In Hive Metastore, We have TxtDbUtil.java and it contains few utils required 
> for tests. 
> There is prepDb() method, it is creating connection and execute some system 
> queries in order to prepare db.  While creating connection it's create new 
> HiveConf object and not taking configs from outside.
> TxtDbUtil.java should also contains prepDb method that can accept external 
> hive configs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-18505) Added external hive configuration to prepDb in TxnDbUtil

2018-01-20 Thread Chandu Kavar (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chandu Kavar reassigned HIVE-18505:
---


> Added external hive configuration to prepDb in TxnDbUtil
> 
>
> Key: HIVE-18505
> URL: https://issues.apache.org/jira/browse/HIVE-18505
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Reporter: Chandu Kavar
>Assignee: Chandu Kavar
>Priority: Minor
>
> In Hive Metastore, We have TxtDbUtil.java and it contains few utils required 
> for tests. 
> There is prepDb() method, it is creating connection and execute some system 
> queries in order to prepare db.  While creating connection it's create new 
> HiveConf object and not taking configs from outside.
> TxtDbUtil.java should also contains prepDb method that can accept external 
> hive configs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-18504) Hive is throwing InvalidObjectException(message:Invalid column type name is too long.

2018-01-20 Thread Jimson K James (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimson K James updated HIVE-18504:
--
Description: 
Hive 2.1.0 is still throwing InvalidObjectException(message:Invalid column type 
name is too long.

Please find attached the create table query. For more details please refer to 
HIVE-15249
{code:java}

FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:Invalid 
column type name length 2980 exceeds max allowed length 2000, type 
struct>,symbols:array...
{code}
 
{code:java}
[root@sandbox-hdp hive-json]# hive --version
Hive 1.2.1000.2.6.3.0-235
Subversion 
git://ctr-e134-1499953498516-254436-01-04.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos6/SOURCES/hive
 -r 5f360bda08bb5489fbb3189b5aeaaf58029ed4b5
Compiled by jenkins on Mon Oct 30 02:48:31 UTC 2017
>From source with checksum 94298cc1f5f5bf0f3470f3ea2e92d646
[root@sandbox-hdp hive-json]# beeline
Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive
beeline> !connect 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Connecting to 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
Enter username for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
 hive
Enter password for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
 
Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235)
Driver: Hive JDBC (version 1.2.1000.2.6.3.0-235)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://sandbox-hdp.hortonworks.com:2>
{code}

  was:
Hive 2.1.0 is still throwing InvalidObjectException(message:Invalid column type 
name is too long.

Please find attached the create table query. For more details please refer to 
HIVE-15249
{code:java}

FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:Invalid 
column type name length 2980 exceeds max allowed length 2000, type 
struct>,symbols:array...
{code}
 
{code:java}
[root@sandbox-hdp hive-json]# hive --version Hive 1.2.1000.2.6.3.0-235 
Subversion 
git://ctr-e134-1499953498516-254436-01-04.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos6/SOURCES/hive
 -r 5f360bda08bb5489fbb3189b5aeaaf58029ed4b5 Compiled by jenkins on Mon Oct 30 
02:48:31 UTC 2017 From source with checksum 94298cc1f5f5bf0f3470f3ea2e92d646 
[root@sandbox-hdp hive-json]# beeline Beeline version 1.2.1000.2.6.3.0-235 by 
Apache Hive beeline> !connect 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
 Connecting to 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
 Enter username for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
 hive Enter password for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
  Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive 
JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: 
TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://sandbox-hdp.hortonworks.com:2>
{code}


> Hive is throwing InvalidObjectException(message:Invalid column type name is 
> too long.
> -
>
> Key: HIVE-18504
> URL: https://issues.apache.org/jira/browse/HIVE-18504
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Jimson K James
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.3.0, 3.0.0
>
> Attachments: tweets.sql
>
>
> Hive 2.1.0 is still throwing InvalidObjectException(message:Invalid column 
> type name is too long.
> Please find attached the create table query. For more details please refer to 
> HIVE-15249
> {code:java}
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. 
> InvalidObjectException(message:Invalid column type name length 2980 exceeds 
> max allowed length 2000, type 
> struct>,symbols:array...
> {code}
>  
> {code:java}
> [root@sandbox-hdp hive-json]# hive --version
> Hive

[jira] [Updated] (HIVE-18504) Hive is throwing InvalidObjectException(message:Invalid column type name is too long.

2018-01-20 Thread Jimson K James (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimson K James updated HIVE-18504:
--
Description: 
Hive 2.1.0 is still throwing InvalidObjectException(message:Invalid column type 
name is too long.

Please find attached the create table query. For more details please refer to 
HIVE-15249
{code:java}

FAILED: Execution Error, return code 1 from 
org.apache.hadoop.hive.ql.exec.DDLTask. InvalidObjectException(message:Invalid 
column type name length 2980 exceeds max allowed length 2000, type 
struct>,symbols:array...
{code}
 
{code:java}
[root@sandbox-hdp hive-json]# hive --version Hive 1.2.1000.2.6.3.0-235 
Subversion 
git://ctr-e134-1499953498516-254436-01-04.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos6/SOURCES/hive
 -r 5f360bda08bb5489fbb3189b5aeaaf58029ed4b5 Compiled by jenkins on Mon Oct 30 
02:48:31 UTC 2017 From source with checksum 94298cc1f5f5bf0f3470f3ea2e92d646 
[root@sandbox-hdp hive-json]# beeline Beeline version 1.2.1000.2.6.3.0-235 by 
Apache Hive beeline> !connect 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
 Connecting to 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
 Enter username for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
 hive Enter password for 
jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
  Connected to: Apache Hive (version 1.2.1000.2.6.3.0-235) Driver: Hive 
JDBC (version 1.2.1000.2.6.3.0-235) Transaction isolation: 
TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://sandbox-hdp.hortonworks.com:2>
{code}

  was:
HIve 2.1.0 is throwing InvalidObjectException(message:Invalid column type name 
is too long

 If you go through MetaStoreUtils.java you can see MAX_MS_TYPENAME_LENGTH is 
hard coded to 2000, but this can be set to higher value in mysql.Table create 
doesn't happen , for tables with higher type length. Different users can change 
this variable based on different use cases/metastore if this can be a config 
variable.

  public static final int MAX_MS_TYPENAME_LENGTH = 2000;

  public static final String TYPE_FROM_DESERIALIZER = "";
  /**
   * validate column type
   *
   * if it is predefined, yes. otherwise no
   * @param name
   * @return
   */
  static public String validateColumnType(String type) {
if (type.equals(TYPE_FROM_DESERIALIZER)) return null;
if (type.length() > MAX_MS_TYPENAME_LENGTH) {
  return "type name is too long: " + type;
}


> Hive is throwing InvalidObjectException(message:Invalid column type name is 
> too long.
> -
>
> Key: HIVE-18504
> URL: https://issues.apache.org/jira/browse/HIVE-18504
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Jimson K James
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.3.0, 3.0.0
>
> Attachments: tweets.sql
>
>
> Hive 2.1.0 is still throwing InvalidObjectException(message:Invalid column 
> type name is too long.
> Please find attached the create table query. For more details please refer to 
> HIVE-15249
> {code:java}
> FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. 
> InvalidObjectException(message:Invalid column type name length 2980 exceeds 
> max allowed length 2000, type 
> struct>,symbols:array...
> {code}
>  
> {code:java}
> [root@sandbox-hdp hive-json]# hive --version Hive 1.2.1000.2.6.3.0-235 
> Subversion 
> git://ctr-e134-1499953498516-254436-01-04.hwx.site/grid/0/jenkins/workspace/HDP-parallel-centos6/SOURCES/hive
>  -r 5f360bda08bb5489fbb3189b5aeaaf58029ed4b5 Compiled by jenkins on Mon Oct 
> 30 02:48:31 UTC 2017 From source with checksum 
> 94298cc1f5f5bf0f3470f3ea2e92d646 [root@sandbox-hdp hive-json]# beeline 
> Beeline version 1.2.1000.2.6.3.0-235 by Apache Hive beeline> !connect 
> jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
>  Connecting to 
> jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2
>  Enter username for 
> jdbc:hive2://sandbox-hdp.hortonworks.com:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2:
>  hive Enter password for 
>

[jira] [Updated] (HIVE-18504) Hive is throwing InvalidObjectException(message:Invalid column type name is too long.

2018-01-20 Thread Jimson K James (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimson K James updated HIVE-18504:
--
Attachment: tweets.sql

> Hive is throwing InvalidObjectException(message:Invalid column type name is 
> too long.
> -
>
> Key: HIVE-18504
> URL: https://issues.apache.org/jira/browse/HIVE-18504
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Jimson K James
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.3.0, 3.0.0
>
> Attachments: tweets.sql
>
>
> HIve 2.1.0 is throwing InvalidObjectException(message:Invalid column type 
> name is too long
>  If you go through MetaStoreUtils.java you can see MAX_MS_TYPENAME_LENGTH is 
> hard coded to 2000, but this can be set to higher value in mysql.Table create 
> doesn't happen , for tables with higher type length. Different users can 
> change this variable based on different use cases/metastore if this can be a 
> config variable.
>   public static final int MAX_MS_TYPENAME_LENGTH = 2000;
>   public static final String TYPE_FROM_DESERIALIZER = " deserializer>";
>   /**
>* validate column type
>*
>* if it is predefined, yes. otherwise no
>* @param name
>* @return
>*/
>   static public String validateColumnType(String type) {
> if (type.equals(TYPE_FROM_DESERIALIZER)) return null;
> if (type.length() > MAX_MS_TYPENAME_LENGTH) {
>   return "type name is too long: " + type;
> }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-18504) Hive is throwing InvalidObjectException(message:Invalid column type name is too long.

2018-01-20 Thread Jimson K James (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jimson K James reassigned HIVE-18504:
-


> Hive is throwing InvalidObjectException(message:Invalid column type name is 
> too long.
> -
>
> Key: HIVE-18504
> URL: https://issues.apache.org/jira/browse/HIVE-18504
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Jimson K James
>Assignee: Naveen Gangam
>Priority: Major
> Fix For: 2.3.0, 3.0.0
>
>
> HIve 2.1.0 is throwing InvalidObjectException(message:Invalid column type 
> name is too long
>  If you go through MetaStoreUtils.java you can see MAX_MS_TYPENAME_LENGTH is 
> hard coded to 2000, but this can be set to higher value in mysql.Table create 
> doesn't happen , for tables with higher type length. Different users can 
> change this variable based on different use cases/metastore if this can be a 
> config variable.
>   public static final int MAX_MS_TYPENAME_LENGTH = 2000;
>   public static final String TYPE_FROM_DESERIALIZER = " deserializer>";
>   /**
>* validate column type
>*
>* if it is predefined, yes. otherwise no
>* @param name
>* @return
>*/
>   static public String validateColumnType(String type) {
> if (type.equals(TYPE_FROM_DESERIALIZER)) return null;
> if (type.length() > MAX_MS_TYPENAME_LENGTH) {
>   return "type name is too long: " + type;
> }



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline

2018-01-20 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1650#comment-1650
 ] 

Hive QA commented on HIVE-18493:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906977/HIVE-18493.03.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8736/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8736/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8736/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-01-21 00:14:16.236
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8736/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-01-21 00:14:16.239
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 90d236a HIVE-18231 : validate resource plan - part 2 - validate 
action and trigger expressions (Harish Jaiprakash, reviewed by Sergey Shelukhin)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 90d236a HIVE-18231 : validate resource plan - part 2 - validate 
action and trigger expressions (Harish Jaiprakash, reviewed by Sergey Shelukhin)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-01-21 00:14:18.500
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8736/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: cannot apply binary patch to 'data/files/test.parquet' without full 
index line
Falling back to three-way merge...
error: cannot apply binary patch to 'data/files/test.parquet' without full 
index line
error: data/files/test.parquet: patch does not apply
error: src/java/org/apache/hive/beeline/BeeLine.java: does not exist in index
error: src/java/org/apache/hive/beeline/BeeLineOpts.java: does not exist in 
index
error: src/java/org/apache/hive/beeline/Rows.java: does not exist in index
error: src/main/resources/BeeLine.properties: does not exist in index
error: src/test/org/apache/hive/beeline/cli/TestHiveCli.java: does not exist in 
index
error: src/java/org/apache/hadoop/hive/cli/CliDriver.java: does not exist in 
index
error: src/java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in 
index
error: cannot apply binary patch to 'files/test.parquet' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'files/test.parquet' without full index line
error: files/test.parquet: patch does not apply
error: 
hive-unit/src/test/java/org/apache/hive/beeline/TestBeeLineWithArgs.java: does 
not exist in index
error: util/src/main/java/org/apache/hive/beeline/QFileBeeLineClient.java: does 
not exist in index
error: java/org/apache/hive/beeline/BeeLine.java: does not exist in index
error: java/org/apache/hive/beeline/BeeLineOpts.java: does not exist in index
error: java/org/apache/hive/beeline/Rows.java: does not exist in index
error: main/resources/BeeLine.properties: does not exist in index
error: test/org/apache/hive/beeline/cli/TestHiveCli.java: does not exist in 
index
error: java/org/apache/hadoop/hive/cli/CliDriver.java: does not exist in index
error: java/org/apache/hadoop/hive/conf/HiveConf.java: does not exist in index
error: cannot apply binary patch to 'test.parquet' without full index line
Falling back to three-way merge...
error: cannot apply binary patch to 'test.parquet' without full index line
error: test.parquet:

[jira] [Updated] (HIVE-18493) Add display escape for CR/LF to Hive CLI and Beeline

2018-01-20 Thread Matt McCline (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Matt McCline updated HIVE-18493:

Attachment: HIVE-18493.03.patch

> Add display escape for CR/LF to Hive CLI and Beeline
> 
>
> Key: HIVE-18493
> URL: https://issues.apache.org/jira/browse/HIVE-18493
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline, Hive
>Affects Versions: 3.0.0
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-18493.01.patch, HIVE-18493.02.patch, 
> HIVE-18493.03.patch
>
>
> Add optional display escaping of carriage return and line feed so row output 
> remains one line.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (HIVE-18471) Beeline gives log4j warnings

2018-01-20 Thread Janaki Lahorani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Janaki Lahorani resolved HIVE-18471.

Resolution: Duplicate

Accidentally filed 2 Jiras for the same issue.  So, closing 18471 as duplicate 
of 18472.

> Beeline gives log4j warnings
> 
>
> Key: HIVE-18471
> URL: https://issues.apache.org/jira/browse/HIVE-18471
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
>
> Starting Beeline gives the following warnings:
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> ERROR StatusLogger No log4j2 configuration file found. Using default 
> configuration: logging only errors to the console. Set system property 
> 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show 
> Log4j2 internal initialization logging. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18471) Beeline gives log4j warnings

2018-01-20 Thread Janaki Lahorani (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1608#comment-1608
 ] 

Janaki Lahorani commented on HIVE-18471:


Thanks [~asherman].

> Beeline gives log4j warnings
> 
>
> Key: HIVE-18471
> URL: https://issues.apache.org/jira/browse/HIVE-18471
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
>
> Starting Beeline gives the following warnings:
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> ERROR StatusLogger No log4j2 configuration file found. Using default 
> configuration: logging only errors to the console. Set system property 
> 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show 
> Log4j2 internal initialization logging. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-18471) Beeline gives log4j warnings

2018-01-20 Thread Janaki Lahorani (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-18471?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Janaki Lahorani reassigned HIVE-18471:
--

Assignee: Janaki Lahorani

> Beeline gives log4j warnings
> 
>
> Key: HIVE-18471
> URL: https://issues.apache.org/jira/browse/HIVE-18471
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Janaki Lahorani
>Assignee: Janaki Lahorani
>Priority: Major
>
> Starting Beeline gives the following warnings:
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/log4j-slf4j-impl-2.8.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: Found binding in 
> [jar:file:/opt/cloudera/parcels/CDH-6.x-1.cdh6.x.p0.215261/jars/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an 
> explanation.
> SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
> ERROR StatusLogger No log4j2 configuration file found. Using default 
> configuration: logging only errors to the console. Set system property 
> 'org.apache.logging.log4j.simplelog.StatusLogger.level' to TRACE to show 
> Log4j2 internal initialization logging. 
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-10041) Set defaults for HBASE_HOME in a smarter way

2018-01-20 Thread Prasad Nagaraj Subramanya (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Nagaraj Subramanya reassigned HIVE-10041:


Assignee: (was: Prasad Nagaraj Subramanya)

> Set defaults for HBASE_HOME in a smarter way
> 
>
> Key: HIVE-10041
> URL: https://issues.apache.org/jira/browse/HIVE-10041
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog
>Affects Versions: 1.1.0
>Reporter: Mark Grover
>Priority: Major
> Fix For: 1.1.0
>
>
> Similar to SQOOP-2145, hcat binary script doesn't do smart detection of 
> HBASE_HOME. It assumes it's always located in /usr/lib/hbase and if not 
> HBASE_HOME variable needs to be exported. The reason is often times people 
> have tarballs like ~/hive-hcatalog, ~/hbase, etc. and it would be good to 
> have their hcat scripts work out of the book. 
> This doesn't regress anything because it only sets the HBASE_HOME if it's not 
> already set and the directory it's setting it to exists and is valid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (HIVE-10041) Set defaults for HBASE_HOME in a smarter way

2018-01-20 Thread Prasad Nagaraj Subramanya (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-10041?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasad Nagaraj Subramanya reassigned HIVE-10041:


Assignee: Prasad Nagaraj Subramanya  (was: hizero)

> Set defaults for HBASE_HOME in a smarter way
> 
>
> Key: HIVE-10041
> URL: https://issues.apache.org/jira/browse/HIVE-10041
> Project: Hive
>  Issue Type: Bug
>  Components: HCatalog
>Affects Versions: 1.1.0
>Reporter: Mark Grover
>Assignee: Prasad Nagaraj Subramanya
>Priority: Major
> Fix For: 1.1.0
>
>
> Similar to SQOOP-2145, hcat binary script doesn't do smart detection of 
> HBASE_HOME. It assumes it's always located in /usr/lib/hbase and if not 
> HBASE_HOME variable needs to be exported. The reason is often times people 
> have tarballs like ~/hive-hcatalog, ~/hbase, etc. and it would be good to 
> have their hcat scripts work out of the book. 
> This doesn't regress anything because it only sets the HBASE_HOME if it's not 
> already set and the directory it's setting it to exists and is valid.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules

2018-01-20 Thread Hive QA (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333278#comment-16333278
 ] 

Hive QA commented on HIVE-17751:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12906968/HIVE-17751.01-standalone-metastore.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/8735/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/8735/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-8735/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2018-01-20 18:04:13.517
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-8735/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z standalone-metastore ]]
+ [[ -d apache-github-standalone-metastore-source ]]
+ [[ ! -d apache-github-standalone-metastore-source/.git ]]
+ [[ ! -d apache-github-standalone-metastore-source ]]
+ date '+%Y-%m-%d %T.%3N'
2018-01-20 18:04:13.533
+ cd apache-github-standalone-metastore-source
+ git fetch origin
>From https://github.com/apache/hive
   d6c6d96..90d236a  master -> origin/master
+ git reset --hard HEAD
HEAD is now at b623000 HIVE-17990 Add Thrift and DB storage for Schema Registry 
objects
+ git clean -f -d
+ git checkout standalone-metastore
+ git reset --hard origin/standalone-metastore
HEAD is now at b623000 HIVE-17990 Add Thrift and DB storage for Schema Registry 
objects
+ git merge --ff-only origin/standalone-metastore
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2018-01-20 18:04:20.456
+ rm -rf ../yetus
+ mkdir ../yetus
+ git gc
+ cp -R . ../yetus
+ mkdir /data/hiveptest/logs/PreCommit-HIVE-Build-8735/yetus
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
fatal: git apply: bad git-diff - inconsistent old filename on line 1619
error: patch failed: 
standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java:30
Falling back to three-way merge...
Applied patch to 
'hive-metastore-client/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java'
 with conflicts.
error: 
standalone-metastore/src/gen/org/apache/hadoop/hive/metastore/annotation/package-info.java:
 does not exist in index
fatal: git diff header lacks filename information when removing 2 leading 
pathname components (line 4262)
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12906968 - PreCommit-HIVE-Build

> Separate HMS Client and HMS server into separate sub-modules
> 
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only 
> include HMSClient library instead of one big library containing server as 
> well. We should ideally have a thin client library so that cross version 
> support for external applications is easier. We should sub-divide the 
> standalone module into possibly 3 modules (one for common classes, one for 
> client classes and one for server) or 2 sub-modules (one for client and one 
> for server) so that we can generate separate jars for HMS client and server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules

2018-01-20 Thread Alexander Kolbasov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated HIVE-17751:
--
Attachment: HIVE-17751.01-standalone-metastore.patch

> Separate HMS Client and HMS server into separate sub-modules
> 
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only 
> include HMSClient library instead of one big library containing server as 
> well. We should ideally have a thin client library so that cross version 
> support for external applications is easier. We should sub-divide the 
> standalone module into possibly 3 modules (one for common classes, one for 
> client classes and one for server) or 2 sub-modules (one for client and one 
> for server) so that we can generate separate jars for HMS client and server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules

2018-01-20 Thread Alexander Kolbasov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated HIVE-17751:
--
Status: Open  (was: Patch Available)

> Separate HMS Client and HMS server into separate sub-modules
> 
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only 
> include HMSClient library instead of one big library containing server as 
> well. We should ideally have a thin client library so that cross version 
> support for external applications is easier. We should sub-divide the 
> standalone module into possibly 3 modules (one for common classes, one for 
> client classes and one for server) or 2 sub-modules (one for client and one 
> for server) so that we can generate separate jars for HMS client and server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules

2018-01-20 Thread Alexander Kolbasov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated HIVE-17751:
--
Attachment: (was: HIVE-17751.01-standalone-metastore.patch)

> Separate HMS Client and HMS server into separate sub-modules
> 
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only 
> include HMSClient library instead of one big library containing server as 
> well. We should ideally have a thin client library so that cross version 
> support for external applications is easier. We should sub-divide the 
> standalone module into possibly 3 modules (one for common classes, one for 
> client classes and one for server) or 2 sub-modules (one for client and one 
> for server) so that we can generate separate jars for HMS client and server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (HIVE-17751) Separate HMS Client and HMS server into separate sub-modules

2018-01-20 Thread Alexander Kolbasov (JIRA) 

 [ 
https://issues.apache.org/jira/browse/HIVE-17751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Alexander Kolbasov updated HIVE-17751:
--
Status: Patch Available  (was: Open)

> Separate HMS Client and HMS server into separate sub-modules
> 
>
> Key: HIVE-17751
> URL: https://issues.apache.org/jira/browse/HIVE-17751
> Project: Hive
>  Issue Type: Sub-task
>  Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Alexander Kolbasov
>Priority: Major
> Attachments: HIVE-17751.01-standalone-metastore.patch
>
>
> external applications which are interfacing with HMS should ideally only 
> include HMSClient library instead of one big library containing server as 
> well. We should ideally have a thin client library so that cross version 
> support for external applications is easier. We should sub-divide the 
> standalone module into possibly 3 modules (one for common classes, one for 
> client classes and one for server) or 2 sub-modules (one for client and one 
> for server) so that we can generate separate jars for HMS client and server.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18445) qtests: auto_join25.q fails permanently

2018-01-20 Thread Laszlo Bodor (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333224#comment-16333224
 ] 

Laszlo Bodor commented on HIVE-18445:
-

It seems like auto_join25 test passed, and it has also passed locally without 
error.

[~kgyrtkirk]

> qtests: auto_join25.q fails permanently
> ---
>
> Key: HIVE-18445
> URL: https://issues.apache.org/jira/browse/HIVE-18445
> Project: Hive
>  Issue Type: Bug
>  Components: Tests
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18445.01.patch
>
>
> org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_join25] 
> (batchId=72)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Comment Edited] (HIVE-18061) q.outs: be more selective with masking hdfs paths

2018-01-20 Thread Laszlo Bodor (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18061?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332825#comment-16332825
 ] 

Laszlo Bodor edited comment on HIVE-18061 at 1/20/18 3:26 PM:
--

It looks good for me now.
[~kgyrtkirk]


was (Author: abstractdog):
It looks good for me now.


> q.outs: be more selective with masking hdfs paths
> -
>
> Key: HIVE-18061
> URL: https://issues.apache.org/jira/browse/HIVE-18061
> Project: Hive
>  Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18061.01.patch, HIVE-18061.02.patch, 
> HIVE-18061.03.patch, HIVE-18061.04.patch, HIVE-18061.05.patch, 
> HIVE-18061.06.patch, HIVE-18061.07.patch, HIVE-18061.08.patch, 
> HIVE-18061.09.patch, HIVE-18061.10.patch, HIVE-18061.11.patch, 
> HIVE-18061.12.patch, HIVE-18061.13.patch, HIVE-18061.14.patch
>
>
> currently any line which contains a path which looks like an hdfs location is 
> replaced with a "masked pattern was here"...
> it might be releavant to record these messages; since even an exception 
> message might contain an hdfs location
> noticed in
> HIVE-18012



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (HIVE-18051) qfiles: dataset support

2018-01-20 Thread Laszlo Bodor (JIRA) 

[ 
https://issues.apache.org/jira/browse/HIVE-18051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333222#comment-16333222
 ] 

Laszlo Bodor commented on HIVE-18051:
-

The failures have nothing to do with the patch, so it's good.

[~kgyrtkirk]

> qfiles: dataset support
> ---
>
> Key: HIVE-18051
> URL: https://issues.apache.org/jira/browse/HIVE-18051
> Project: Hive
>  Issue Type: Improvement
>  Components: Testing Infrastructure
>Reporter: Zoltan Haindrich
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-18051.01.patch, HIVE-18051.02.patch, 
> HIVE-18051.03.patch, HIVE-18051.04.patch, HIVE-18051.05.patch, 
> HIVE-18051.06.patch
>
>
> it would be great to have some kind of test dataset support; currently there 
> is the {{q_test_init.sql}} which is quite large; and I'm often override it 
> with an invalid string; because I write independent qtests most of the time - 
> and the load of {{src}} and other tables are just a waste of time for me ; 
> not to mention that the loading of those tables may also trigger breakpoints 
> - which is a bit annoying.
> Most of the tests are "only" using the {{src}} table and possibly 2 others; 
> however the main init script contains a bunch of tables - meanwhile there are 
> quite few other tests which could possibly also benefit from a more general 
> feature; for example the creation of {{bucket_small}} is present in 20 q 
> files.
> the proposal would be to enable the qfiles to be annotated with metadata like 
> datasets:
> {code}
> --! qt:dataset:src,bucket_small
> {code}
> proposal for storing a dataset:
> * the loader script would be at: {{data/datasets/__NAME__/load.hive.sql}}
> * the table data could be stored under that location
> a draft about this; and other qfiles related ideas:
> https://docs.google.com/document/d/1KtcIx8ggL9LxDintFuJo8NQuvNWkmtvv_ekbWrTLNGc/edit?usp=sharing



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)