Could you please raise an issue with some type of example? Due to the way
Hadoop (and Hive) works,
things tend to be tricky in terms of configuring a job.
The configuration needs to be created before a job is submitted which in practice means
"dynamic configurations"
are basically impossible (this also has some security implications which are
simply avoided this way).
Thus either one specifies the configuration manually or loads a known location
file (hive-site.xml, core-site.xml...)
upfront, before the job is submitted.
This means when dealing with Hive, Pig, Cascading, etc... unless one adds a pre-processor to the job content (script,
flow, etc...)
by the time es-hadoop kicks in, the job is already running and thus its changes
discarded.
Cheers,
On 6/14/14 1:57 AM, Jinyuan Zhou wrote:
Hi,
I am playing with elasticsearch and hive integration. The documentation says
to set configuration like es.nodes, es.port in TBLPROPERTIES. It works.
But it can cause many reduntant codes. If I have ten data set to index to the
same es cluster,
I would have to repeat this information ten times in TBLPROPERTIES. Even if
I use var substitution I still have to rwrite this subtititiov var for each
table definition.
What I am looking for is to put these info in say one file and pass the
location, in some way, to hive cli
so hive elasticsearch will get these settings when trying to find es server to
talk to.
I am not looking into put these info into files like hive-site.xml.
Thanks,
Jack
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/7040c805-e845-4b3d-a9fe-5e18d8445f7f%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/7040c805-e845-4b3d-a9fe-5e18d8445f7f%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/539D6507.3080207%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
