Thanks for the detailed report. The user authentication triggered a bug [1] which has been fixed in master - I've
already pushed the nightly builds which you can try out [2]. It would be great if you could try it out and report back
if it works for you.
If you want to stick with M3 until RC1 is released, disabling user
authentication should fix the issue.
Cheers,
[1] https://github.com/elasticsearch/elasticsearch-hadoop/issues/195
[2]
http://www.elasticsearch.org/guide/en/elasticsearch/hadoop/current/install.html
On 4/23/14 12:10 PM, baskaran vaithiyanathan wrote:
Hi Costin,
Thanks for your reply.
Please find stackrace and other information that you have mentioned.
*1. Stacktrace*
*
*
java.lang.NullPointerException
at
org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.setupHttpProxy(CommonsHttpTransport.java:169)
at
org.elasticsearch.hadoop.rest.commonshttp.CommonsHttpTransport.<init>(CommonsHttpTransport.java:132)
at
org.elasticsearch.hadoop.rest.NetworkClient.selectNextNode(NetworkClient.java:67)
at org.elasticsearch.hadoop.rest.NetworkClient.<init>(NetworkClient.java:50)
at org.elasticsearch.hadoop.rest.RestClient.<init>(RestClient.java:69)
at
org.elasticsearch.hadoop.rest.InitializationUtils.discoverNodesIfNeeded(InitializationUtils.java:59)
at
org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:180)
at
org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.write(EsOutputFormat.java:157)
at org.elasticsearch.hadoop.pig.EsStorage.putNext(EsStorage.java:191)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:139)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigOutputFormat$PigRecordWriter.write(PigOutputFormat.java:98)
at
org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:638)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigMapOnly$Map.collect(PigMapOnly.java:48)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.runPipeline(PigGenericMapBase.java:284)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:277)
at
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.map(PigGenericMapBase.java:64)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:145)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:364)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
at org.apache.hadoop.mapred.Child.main(Child.java:249)
*2. I have checked the script without proxy and authentication. it works fine.*
*
*
*3. Version details*
* OS - Ubuntu 12.04.3 LTS *
* ElasticSearch - 1.0.0*
* Hadoop - 1.2.1*
* ElasticSearch- Hadoop - elasticsearch-hadoop 1.3 M3 *
* Pig - 0.12.0*
* Nginx - 1.1.19***
Thanks.
On Wednesday, April 23, 2014 4:01:32 AM UTC-4, Costin Leau wrote:
Hi,
Can you post the stacktrace you are getting, I'm not sure what causes the
NPE. Can you also confirm the Hadoop/Pig/OS
version used?
Additionally, could you check your script without the proxy or potentially
without authentication - just curious
whether
it makes any difference.
Cheers,
On 4/23/14 9:23 AM, baskaran vaithiyanathan wrote:
> i am a beginner in elasticsearch and hadoop. i am having a problem with
moving data from hdfs into elasticsearch server
> using es.net.proxy.http.host with credentials. Server secured with
credentials using nginx proxy configuration. But when
> i am trying to move data using pig script it shows null pointer exception.
>
> My pig script is
>
> *REGISTER
elasticsearch-hadoop-1.3.0.M3/dist/elasticsearch-hadoop-1.3.0.M3.jar*
>
> *A = load 'date' using PigStorage() as (date:datetime);*
>
> *store A into 'doc/id' using
>
org.elasticsearch.hadoop.pig.EsStorage('es.net.proxy.http.host=ipaddress','es.net.proxy.http.port=portnumber','es.net.proxy.http.user=username','es.net.proxy.http.pass=password');*
>
> I don't understand where is the problem with my script. Can anyone please
help me?
>
> Thanks in Advance.
>
> --
> You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
email to
>[email protected] <javascript:>
<mailto:[email protected] <javascript:>>.
> To view this discussion on the web visit
>https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com>
>
<https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer
<https://groups.google.com/d/msgid/elasticsearch/243c914a-dd48-4575-8adc-7b1b4fe3067c%40googlegroups.com?utm_medium=email&utm_source=footer>>.
> For more options, visithttps://groups.google.com/d/optout
<https://groups.google.com/d/optout>.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/688adc69-b4a7-42cb-bbea-92baebfd7f50%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5357891C.4020705%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
