[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=568329=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-568329 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 18/Mar/21 13:18 Start Date: 18/Mar/21 13:18 Worklog Time Spent: 10m Work Description: nrg4878 closed pull request #1852: URL: https://github.com/apache/hive/pull/1852 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 568329) Time Spent: 1.5h (was: 1h 20m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Abhay >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1.5h > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=568328=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-568328 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 18/Mar/21 13:18 Start Date: 18/Mar/21 13:18 Worklog Time Spent: 10m Work Description: nrg4878 commented on pull request #1852: URL: https://github.com/apache/hive/pull/1852#issuecomment-801919129 This change has been re-submitted as PR#1906 and merged. Closing this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 568328) Time Spent: 1h 20m (was: 1h 10m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Abhay >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=568033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-568033 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 18/Mar/21 00:51 Start Date: 18/Mar/21 00:51 Worklog Time Spent: 10m Work Description: github-actions[bot] commented on pull request #1852: URL: https://github.com/apache/hive/pull/1852#issuecomment-801535404 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Feel free to reach out on the d...@hive.apache.org list if the patch is in need of reviews. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 568033) Time Spent: 1h 10m (was: 1h) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Abhay >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=536891=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-536891 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 16/Jan/21 22:08 Start Date: 16/Jan/21 22:08 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1852: URL: https://github.com/apache/hive/pull/1852#issuecomment-761687399 Somewhat related (dovetails nicely) with my work on #1833 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 536891) Time Spent: 1h (was: 50m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Time Spent: 1h > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=534370=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534370 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 11/Jan/21 17:46 Start Date: 11/Jan/21 17:46 Worklog Time Spent: 10m Work Description: nrg4878 commented on a change in pull request #1852: URL: https://github.com/apache/hive/pull/1852#discussion_r555229243 ## File path: beeline/pom.xml ## @@ -197,6 +197,25 @@ + +org.apache.maven.plugins +maven-assembly-plugin +3.1.1 Review comment: Thanks David. These changes are not final. I am having a new team member look at these changes. There is more work to be done in terms of building a beeline standalone distributable (.zip file). I am just transitioning these code changes for him to take a look. But your comment is noted. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534370) Time Spent: 50m (was: 40m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Time Spent: 50m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=534365=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534365 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 11/Jan/21 17:36 Start Date: 11/Jan/21 17:36 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1852: URL: https://github.com/apache/hive/pull/1852#discussion_r555222070 ## File path: beeline/pom.xml ## @@ -197,6 +197,25 @@ + +org.apache.maven.plugins +maven-assembly-plugin +3.1.1 Review comment: I don't think you need to include an explicit version number here. It should just inherit from parent POM. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534365) Time Spent: 40m (was: 0.5h) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Time Spent: 40m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=534364=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534364 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 11/Jan/21 17:35 Start Date: 11/Jan/21 17:35 Worklog Time Spent: 10m Work Description: belugabehr commented on a change in pull request #1852: URL: https://github.com/apache/hive/pull/1852#discussion_r555222070 ## File path: beeline/pom.xml ## @@ -197,6 +197,25 @@ + +org.apache.maven.plugins +maven-assembly-plugin +3.1.1 Review comment: I don't think you need to include an explicit version number here. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534364) Time Spent: 0.5h (was: 20m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=534362=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534362 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 11/Jan/21 17:33 Start Date: 11/Jan/21 17:33 Worklog Time Spent: 10m Work Description: belugabehr commented on pull request #1852: URL: https://github.com/apache/hive/pull/1852#issuecomment-758106837 @nrg4878 Can you also take a look at my PR #1833 since you seem to be looking at beeline improvements? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534362) Time Spent: 20m (was: 10m) > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Labels: pull-request-available > Time Spent: 20m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Work logged] (HIVE-24348) Beeline: Isolating dependencies and execution with java
[ https://issues.apache.org/jira/browse/HIVE-24348?focusedWorklogId=534359=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-534359 ] ASF GitHub Bot logged work on HIVE-24348: - Author: ASF GitHub Bot Created on: 11/Jan/21 17:29 Start Date: 11/Jan/21 17:29 Worklog Time Spent: 10m Work Description: nrg4878 opened a new pull request #1852: URL: https://github.com/apache/hive/pull/1852 1) Changes to use java to execute beeline if HADOOP_HOME is not set 2) POM changes to build an uber beeline-with-dependencies jar 3) Eliminates some of the not required jars in classpath for beeline. ### What changes were proposed in this pull request? 1) Changes to use java to execute beeline if HADOOP_HOME is not set 2) POM changes to build an uber beeline-with-dependencies jar 3) Eliminates some of the not required jars in classpath for beeline. ### Why are the changes needed? Currently beeline has a bunch of dependencies on other jars, some required and some unnecessary. beeline script also uses HADOOP_HOME to execute beeline class. This adds some other jars to the classpath that makes it hard to run beeline on an edge node. This fix is a first pass at building a standalone jar with all needed dependencies and also make it work using "java" when HADOOP_HOME is not set. ### Does this PR introduce _any_ user-facing change? NO ### How was this patch tested? Manually. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 534359) Remaining Estimate: 0h Time Spent: 10m > Beeline: Isolating dependencies and execution with java > --- > > Key: HIVE-24348 > URL: https://issues.apache.org/jira/browse/HIVE-24348 > Project: Hive > Issue Type: Improvement > Components: Beeline >Affects Versions: 3.1.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Currently, beeline code, binaries and executables are somewhat tightly > coupled with the hive product. To be able to execute beeline from a node with > just JRE installed and some jars in classpath is impossible. > * beeline.sh/hive scripts rely on HADOOP_HOME to be set which are designed to > use "hadoop" executable to run beeline. > * Ideally, just the hive-beeline.jar and hive-jdbc-standalone jars should be > enough but sadly they arent. The latter jar adds more problems than it solves > because all the classfiles are shaded some dependencies cannot be resolved. > * Beeline has many other dependencies like hive-exec, hive-common. > hadoop-common, supercsv, jline, commons-cli, commons-io, commons-logging etc. > While it may not be possible to eliminate some of these, we should atleast > have a self-contains jar that contains all these to be able to make it work. > * the underlying script used to run beeline should use JAVA as an alternate > means to execute if HADOOP_HOME is not set -- This message was sent by Atlassian Jira (v8.3.4#803005)
