[jira] [Commented] (HIVE-7115) Support a mechanism for running hive locally that doesnt require having a hadoop executable.
[
https://issues.apache.org/jira/browse/HIVE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025735#comment-14025735
]
Sushanth Sowmyan commented on HIVE-7115:
Hi Jay,
There are two things I can think of that might help development of such a tool:
a) One possible place to look into for this is to look at the test framework
section. For example, look at our TestCli framework over at
ql/src/test/templates/TestCliDriver.vm and
./ql/src/test/templates/TestNegativeCliDriver.vm which manage our .q and .q.out
tests.
b) The other route I'd suggest is by directly launching a Hive Driver, as done
in tests like
hcatalog-pig-adapter/src/test/java/org/apache/hive/hcatalog/pig/TestE2EScenarios.java
Support a mechanism for running hive locally that doesnt require having a
hadoop executable.
Key: HIVE-7115
URL: https://issues.apache.org/jira/browse/HIVE-7115
Project: Hive
Issue Type: Improvement
Components: Testing Infrastructure, Tests
Reporter: jay vyas
Mapreduce has a local mode by default, and likewise, tools such as pig and
SOLR do as well, maybe we can have a first class local mode for hive
also.
For local integration testing of a hadoop app, it would be nice if we could
fire up a local hive instance which didnt require bin/hadoop for running
local jobs. This would allow us to maintain polyglot hadoop applications
much easier by incorporating hive into the integration tests. For example:
{noformat}
LocalHiveInstance hive = new LocalHiveInstance();
hive.set(course,crochet)l
hive.runScript(hive_flow.ql)l
{noformat}
Would essentially run a local hive query which mirrors
{noformat}
hive -f hive_flow.ql -hiveconf course=crochet
{noformat{
It seems like thee might be a simple way to do this, at least for small data
sets, by putting some kind of alternative (i.e. in memory) execution
environment under hive, if one is not already underway ?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7115) Support a mechanism for running hive locally that doesnt require having a hadoop executable.
[
https://issues.apache.org/jira/browse/HIVE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025745#comment-14025745
]
Sushanth Sowmyan commented on HIVE-7115:
Oh, and one more thing : there is a miniHS2 implementation in the tests :
hive-unit/src/main/java/org/apache/hive/jdbc/miniHS2/MiniHS2.java
I would recommend experimenting with this more than the other approaches,
because it's cleaner from an integration standpoint. You can then use jdbc or
beeline to connect to it.
Support a mechanism for running hive locally that doesnt require having a
hadoop executable.
Key: HIVE-7115
URL: https://issues.apache.org/jira/browse/HIVE-7115
Project: Hive
Issue Type: Improvement
Components: Testing Infrastructure, Tests
Reporter: jay vyas
Mapreduce has a local mode by default, and likewise, tools such as pig and
SOLR do as well, maybe we can have a first class local mode for hive
also.
For local integration testing of a hadoop app, it would be nice if we could
fire up a local hive instance which didnt require bin/hadoop for running
local jobs. This would allow us to maintain polyglot hadoop applications
much easier by incorporating hive into the integration tests. For example:
{noformat}
LocalHiveInstance hive = new LocalHiveInstance();
hive.set(course,crochet)l
hive.runScript(hive_flow.ql)l
{noformat}
Would essentially run a local hive query which mirrors
{noformat}
hive -f hive_flow.ql -hiveconf course=crochet
{noformat{
It seems like thee might be a simple way to do this, at least for small data
sets, by putting some kind of alternative (i.e. in memory) execution
environment under hive, if one is not already underway ?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
[jira] [Commented] (HIVE-7115) Support a mechanism for running hive locally that doesnt require having a hadoop executable.
[
https://issues.apache.org/jira/browse/HIVE-7115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14025767#comment-14025767
]
Edward Capriolo commented on HIVE-7115:
---
That would be really nice especially if it could be extended to dependent
projects
https://github.com/edwardcapriolo/hive_test requires lots of trickery to launch
a hive process.
Support a mechanism for running hive locally that doesnt require having a
hadoop executable.
Key: HIVE-7115
URL: https://issues.apache.org/jira/browse/HIVE-7115
Project: Hive
Issue Type: Improvement
Components: Testing Infrastructure, Tests
Reporter: jay vyas
Mapreduce has a local mode by default, and likewise, tools such as pig and
SOLR do as well, maybe we can have a first class local mode for hive
also.
For local integration testing of a hadoop app, it would be nice if we could
fire up a local hive instance which didnt require bin/hadoop for running
local jobs. This would allow us to maintain polyglot hadoop applications
much easier by incorporating hive into the integration tests. For example:
{noformat}
LocalHiveInstance hive = new LocalHiveInstance();
hive.set(course,crochet)l
hive.runScript(hive_flow.ql)l
{noformat}
Would essentially run a local hive query which mirrors
{noformat}
hive -f hive_flow.ql -hiveconf course=crochet
{noformat{
It seems like thee might be a simple way to do this, at least for small data
sets, by putting some kind of alternative (i.e. in memory) execution
environment under hive, if one is not already underway ?
--
This message was sent by Atlassian JIRA
(v6.2#6252)
