[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has submitted this change and it was merged. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Reviewed-on: http://gerrit.cloudera.org:8080/13542
Reviewed-by: Thomas Marshall 
Reviewed-by: Hao Hao 
Tested-by: Impala Public Jenkins 
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 21 insertions(+), 5 deletions(-)

Approvals:
  Thomas Marshall: Looks good to me, approved
  Hao Hao: Looks good to me, but someone else must approve
  Impala Public Jenkins: Verified

--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: merged
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 5
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 4: Verified+1


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Fri, 07 Jun 2019 02:18:37 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Hao Hao (Code Review)
Hao Hao has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 4: Code-Review+1


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 21:20:08 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 3:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3524/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 21:05:47 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13544 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 1:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3525/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13544
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5
Gerrit-Change-Number: 13544
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:50:25 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 2:

Build Successful

https://jenkins.impala.io/job/gerrit-code-review-checks/3523/ : Initial code 
review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun 
to run full precommit tests.


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:34:10 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Thomas Marshall has abandoned this change. ( 
http://gerrit.cloudera.org:8080/13544 )

Change subject: Fix integration of kudu-hive.jar
..


Abandoned

pushed by mistake
--
To view, visit http://gerrit.cloudera.org:8080/13544
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: abandon
Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5
Gerrit-Change-Number: 13544
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall 


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Impala Public Jenkins (Code Review)
Impala Public Jenkins has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 4:

Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4424/ 
DRY_RUN=false


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:20 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Thomas Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 4: Code-Review+2

carrying forward


--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:33:07 +
Gerrit-HasComments: No


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Thomas Marshall has uploaded this change for review. ( 
http://gerrit.cloudera.org:8080/13544


Change subject: Fix integration of kudu-hive.jar
..

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 21 insertions(+), 5 deletions(-)



  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/44/13544/1
--
To view, visit http://gerrit.cloudera.org:8080/13544
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newchange
Gerrit-Change-Id: Ia874efa63d9114716da3b2ac8921e78b518bdae5
Gerrit-Change-Number: 13544
Gerrit-PatchSet: 1
Gerrit-Owner: Thomas Marshall 


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#4).

Change subject: Fix integration of kudu-hive.jar
..

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 21 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/4
--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 4
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Lars Volker (Code Review)
Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 3: Code-Review+2

(1 comment)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
> Both the 'debug' and the 'release' directories will be present regardless o
Thx for the explanation, might be good to leave that in a comment here for 
future reference.



--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:26:52 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Hello Lars Volker, Hao Hao, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#3).

Change subject: Fix integration of kudu-hive.jar
..

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 19 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/3
--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Thomas Marshall has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 3:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
> does this affect release builds?
Both the 'debug' and the 'release' directories will be present regardless of 
the build type because we always just unpack and keep everything in the tarball.


http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:

http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
> I think this could just be
Done



--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 3
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Reviewer: Thomas Marshall 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:24:54 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Lars Volker (Code Review)
Lars Volker has posted comments on this change. ( 
http://gerrit.cloudera.org:8080/13542 )

Change subject: Fix integration of kudu-hive.jar
..


Patch Set 2:

(2 comments)

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py
File bin/bootstrap_toolchain.py:

http://gerrit.cloudera.org:8080/#/c/13542/2/bin/bootstrap_toolchain.py@433
PS2, Line 433: debug
does this affect release builds?


http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh
File testdata/bin/run-hive-server.sh:

http://gerrit.cloudera.org:8080/#/c/13542/2/testdata/bin/run-hive-server.sh@105
PS2, Line 105: ${IMPALA_KUDU_JAVA_HOME}/*
I think this could just be

  for file in ${IMPALA_KUDU_JAVA_HOME}/*kudu-hive*jar; do

or you could use find | while.



--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins 
Gerrit-Reviewer: Lars Volker 
Gerrit-Comment-Date: Thu, 06 Jun 2019 20:11:13 +
Gerrit-HasComments: Yes


[Impala-ASF-CR] Fix integration of kudu-hive.jar

2019-06-06 Thread Thomas Marshall (Code Review)
Hello Hao Hao, Csaba Ringhofer, Impala Public Jenkins,

I'd like you to reexamine a change. Please visit

http://gerrit.cloudera.org:8080/13542

to look at the new patch set (#2).

Change subject: Fix integration of kudu-hive.jar
..

Fix integration of kudu-hive.jar

IMPALA-8503 added downloading kudu-hive.jar and adding it to
HADOOP_CLASSPATH in run-hive-server.sh to allow the Hive Metastore to
start with Kudu's HMS plugin.

There are two problems with this that are fixed by this patch:
- Previously, we fully specify the expected jar filename based on the
  value of IMPALA_KUDU_JAVA_VERSION when adding it to HADOOP_CLASSPATH
  but this is overly restrictive for users who may wish to override
  this value in impala-config-branch.sh to build their own branch with
  a different version of the kudu-hive.jar This patch relaxes this
  restriction by adding any jar containing the string kudu-hive in
  IMPALA_KUDU_JAVA_HOME to HADOOP_CLASSPATH
- In bootstrap_toolchain, we don't download a package if its directory
  already exists. Since the 'kudu' and 'kudu-java' packages download
  to the same directory, this led to a race condition where
  'kudu-java' might not be downloaded if 'kudu' had already been
  unpacked when it started. This patch fixes this by inspecting the
  contents of the Kudu package directory to look for specific files
  expected for each Kudu package.

Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
---
M bin/bootstrap_toolchain.py
M bin/impala-config.sh
M testdata/bin/run-hive-server.sh
3 files changed, 23 insertions(+), 5 deletions(-)


  git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/42/13542/2
--
To view, visit http://gerrit.cloudera.org:8080/13542
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings

Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I4ac79c3e9b8625ba54145dba23c69fd5117f35c7
Gerrit-Change-Number: 13542
Gerrit-PatchSet: 2
Gerrit-Owner: Thomas Marshall 
Gerrit-Reviewer: Csaba Ringhofer 
Gerrit-Reviewer: Hao Hao 
Gerrit-Reviewer: Impala Public Jenkins