[hive] branch master updated (0d9dbfa -> 20b0012)

2021-02-18 Thread pravin
This is an automated email from the ASF dual-hosted git repository.

pravin pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.


from 0d9dbfa  HIVE-24348: Changes for running beeline with java (Naveen 
Gangam + Abhay Chennagiri via Naveen Gangam) 1) Changes to use java 
to execute beeline if HADOOP_HOME is not set 2) POM changes to 
build distributable tar for beeline. 3) Eliminates some of the not 
required jars in classpath for beeline.
 add 20b0012  HIVE-24733: Handle replication when db location and managed 
location is set to custom location on source (Pravin Kumar Sinha, reviewed by 
Aasha Medhi)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/hadoop/hive/conf/HiveConf.java |   2 +
 .../ql/parse/BaseReplicationAcrossInstances.java   |   6 +
 .../hive/ql/parse/TestReplicationScenarios.java|   4 +
 .../parse/TestReplicationScenariosAcidTables.java  |   1 +
 .../TestReplicationScenariosExclusiveReplica.java  | 216 +
 .../ql/parse/TestStatsReplicationScenarios.java|   1 +
 .../hadoop/hive/ql/exec/repl/ReplDumpTask.java |   2 +-
 .../hadoop/hive/ql/exec/repl/ReplLoadTask.java |   9 +
 .../ql/exec/repl/bootstrap/load/LoadDatabase.java  |  32 ++-
 .../hadoop/hive/ql/exec/repl/util/ReplUtils.java   |   4 +
 .../org/apache/hadoop/hive/ql/parse/EximUtil.java  |  28 ++-
 .../hive/ql/parse/ImportSemanticAnalyzer.java  |   9 +-
 .../repl/dump/events/CreateDatabaseHandler.java|   2 +-
 13 files changed, 307 insertions(+), 9 deletions(-)



[hive] branch master updated: HIVE-24348: Changes for running beeline with java (Naveen Gangam + Abhay Chennagiri via Naveen Gangam) 1) Changes to use java to execute beeline if HADOOP_HOME is not set

2021-02-18 Thread ngangam
This is an automated email from the ASF dual-hosted git repository.

ngangam pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git


The following commit(s) were added to refs/heads/master by this push:
 new 0d9dbfa  HIVE-24348: Changes for running beeline with java (Naveen 
Gangam + Abhay Chennagiri via Naveen Gangam) 1) Changes to use java 
to execute beeline if HADOOP_HOME is not set 2) POM changes to 
build distributable tar for beeline. 3) Eliminates some of the not 
required jars in classpath for beeline.
0d9dbfa is described below

commit 0d9dbfa3aa368f66058673ccadd9490b21468a52
Author: Naveen Gangam 
AuthorDate: Mon Jan 11 12:19:33 2021 -0500

HIVE-24348: Changes for running beeline with java (Naveen Gangam + Abhay 
Chennagiri via Naveen Gangam)
1) Changes to use java to execute beeline if HADOOP_HOME is not 
set
2) POM changes to build distributable tar for beeline.
3) Eliminates some of the not required jars in classpath for 
beeline.
---
 bin/ext/beeline.sh  |  17 +++-
 bin/hive| 143 +---
 packaging/pom.xml   |  17 +++-
 packaging/src/main/assembly/beeline.xml |  78 +
 4 files changed, 185 insertions(+), 70 deletions(-)

diff --git a/bin/ext/beeline.sh b/bin/ext/beeline.sh
index 96f8f60..d825db9 100644
--- a/bin/ext/beeline.sh
+++ b/bin/ext/beeline.sh
@@ -20,7 +20,6 @@ export SERVICE_LIST="${SERVICE_LIST}${THISSERVICE} "
 beeline () {
   CLASS=org.apache.hive.beeline.BeeLine;
 
-  # include only the beeline client jar and its dependencies
   beelineJarPath=`ls ${HIVE_LIB}/hive-beeline-*.jar`
   superCsvJarPath=`ls ${HIVE_LIB}/super-csv-*.jar`
   jlineJarPath=`ls ${HIVE_LIB}/jline-*.jar`
@@ -32,10 +31,20 @@ beeline () {
   export 
HADOOP_CLASSPATH="${hadoopClasspath}${HIVE_CONF_DIR}:${beelineJarPath}:${superCsvJarPath}:${jlineJarPath}"
   export HADOOP_CLIENT_OPTS="$HADOOP_CLIENT_OPTS 
-Dlog4j.configurationFile=beeline-log4j2.properties "
 
-  if [ -z $CLIUSER ] ; then
-exec $HADOOP jar ${beelineJarPath} $CLASS $HIVE_OPTS "$@"
+  if [ "$EXECUTE_WITH_JAVA" != "true" ] ; then
+# if CLIUSER is not empty, then pass it as user id / password during 
beeline redirect
+if [ -z $CLIUSER ] ; then
+  exec $HADOOP jar ${beelineJarPath} $CLASS $HIVE_OPTS "$@"
+else
+  exec $HADOOP jar ${beelineJarPath} $CLASS $HIVE_OPTS "$@" -n 
"${CLIUSER}" -p "${CLIUSER}"
+fi
   else
-exec $HADOOP jar ${beelineJarPath} $CLASS $HIVE_OPTS "$@" -n "${CLIUSER}" 
-p "${CLIUSER}"
+# if CLIUSER is not empty, then pass it as user id / password during 
beeline redirect
+if [ -z $CLIUSER ] ; then
+  $JAVAEXE -cp ${HADOOP_CLASSPATH} $CLASS $HIVE_OPTS "$@"
+else
+  $JAVAEXE -cp ${HADOOP_CLASSPATH} $CLASS $HIVE_OPTS "$@" -n "${CLIUSER}" 
-p "${CLIUSER}"
+fi
   fi
 }
 
diff --git a/bin/hive b/bin/hive
index 8a08630..731e3aa 100755
--- a/bin/hive
+++ b/bin/hive
@@ -25,14 +25,11 @@ bin=`cd "$bin"; pwd`
 
 . "$bin"/hive-config.sh
 
-if [ -f "${HIVE_CONF_DIR}/hive-env.sh" ]; then
-  . "${HIVE_CONF_DIR}/hive-env.sh"
-fi
-
 SERVICE=""
 HELP=""
 SKIP_HBASECP=false
 SKIP_HADOOPVERSION=false
+EXECUTE_WITH_JAVA=false
 
 SERVICE_ARGS=()
 while [ $# -gt 0 ]; do
@@ -89,13 +86,15 @@ if [ "$SERVICE" = "" ] ; then
   fi
 fi
 
+USE_BEELINE_FOR_HIVE_CLI="true"
 if [[ "$SERVICE" == "cli" && "$USE_BEELINE_FOR_HIVE_CLI" == "true" ]] ; then
   CLIUSER=`whoami`
   SERVICE="beeline"
 fi
 
 if [[ "$SERVICE" == "beeline" && ! -z "$BEELINE_URL_LEGACY" ]] ; then
-  SERVICE="beeline -c $BEELINE_URL_LEGACY"
+  SERVICE="beeline"
+  SERVICE_ARGS=("${SERVICE_ARGS[@]}" "-c" "${BEELINE_URL_LEGACY}")
 fi
 
 if [[ "$SERVICE" =~ 
^(help|version|orcfiledump|rcfilecat|schemaTool|cleardanglingscratchdir|metastore|beeline|llapstatus|llap)$
 ]] ; then
@@ -106,15 +105,21 @@ if [[ "$SERVICE" =~ ^(help|schemaTool)$ ]] ; then
   SKIP_HADOOPVERSION=true
 fi
 
-if [[ -z "$SPARK_HOME" ]]
-then
-  bin=`dirname "$0"`
-  # many hadoop installs are in dir/{spark,hive,hadoop,..}
-  if test -e $bin/../../spark; then 
-sparkHome=$(readlink -f $bin/../../spark)
-if [[ -d $sparkHome ]]
-then
-  export SPARK_HOME=$sparkHome
+if [ -f "${HIVE_CONF_DIR}/hive-env.sh" ]; then
+  . "${HIVE_CONF_DIR}/hive-env.sh"
+fi
+
+if [ "$SERVICE" != "beeline" ]; then
+  if [[ -z "$SPARK_HOME" ]]
+  then
+bin=`dirname "$0"`
+# many hadoop installs are in dir/{spark,hive,hadoop,..}
+if test -e $bin/../../spark; then 
+  sparkHome=$(readlink -f $bin/../../spark)
+  if [[ -d $sparkHome ]]
+  then
+export SPARK_HOME=$sparkHome
+  fi
 fi
   fi
 fi
@@ -123,34 +128,27 @@ 
CLASSPATH="${TEZ_CONF_DIR:-/etc/tez/conf}:${HIVE_CONF_DIR}"
 
 HIVE_LIB=${HIVE_HOME}/lib
 
-# needed for execution
-if [ ! -f ${HIVE_LIB}/hive-exec-*.jar ]; then
-  echo "Missing Hive Execution Jar: 

[hive] branch master updated (056fdaf -> fc93e54)

2021-02-18 Thread prasanthj
This is an automated email from the ASF dual-hosted git repository.

prasanthj pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.


from 056fdaf  HIVE-24715: Increase bucketId range (Attila Magyar, reviewed 
by Peter Varga)
 add fc93e54  HIVE-24786: JDBC HttpClient should retry for idempotent and 
unsent http methods (#1983)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/hive/jdbc/HiveConnection.java  | 114 +++--
 1 file changed, 107 insertions(+), 7 deletions(-)



[hive] branch master updated (6404587 -> 056fdaf)

2021-02-18 Thread pvarga
This is an automated email from the ASF dual-hosted git repository.

pvarga pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/hive.git.


from 6404587  HIVE-24710: Optimise PTF iteration for count(*) to reduce CPU 
and IO cost (#1940)
 add 056fdaf  HIVE-24715: Increase bucketId range (Attila Magyar, reviewed 
by Peter Varga)

No new revisions were added by this update.

Summary of changes:
 .../java/org/apache/hadoop/hive/conf/HiveConf.java |  2 +
 .../apache/hadoop/hive/ql/DriverTxnHandler.java|  8 +++-
 .../apache/hadoop/hive/ql/io/AcidOutputFormat.java | 10 +
 .../org/apache/hadoop/hive/ql/io/BucketCodec.java  | 28 +---
 .../hadoop/hive/ql/io/HiveFileFormatUtils.java |  1 +
 .../hadoop/hive/ql/io/orc/OrcRecordUpdater.java|  6 ++-
 .../apache/hadoop/hive/ql/plan/FileSinkDesc.java   | 10 +
 .../apache/hadoop/hive/ql/io/TestBucketCodec.java  | 52 ++
 8 files changed, 109 insertions(+), 8 deletions(-)