(Much delayed) thank you Costin.
Indeed, on Ubuntu, changing ES_CLASSPATH to include hadoop and hadoop/lib
directories in /etc/default/elasticsearch (and exporting it in
/etc/init.d/elasticsearch) and installing light plugin version did work.
On Thursday, 14 August 2014 20:59:39 UTC, Costin Leau wrote:
>
> Hi,
>
> The hdfs repository relies on vanilla Hadoop 2.2 since that's the official
> stable version of Yarn. Since you are using a
> different
> Hadoop version, use the 'light' version as explained in the docs - this
> contains only the repository-hdfs, without the
> Hadoop dependency
> (since you already have it installed).
>
> In other words, both the error that you see as well as the 2.0.1
> (regarding JobLocalizer) seems to be related to the
> differences between
> vanilla Hadoop 2.2 and the one you are using.
>
> Hope this helps,
>
> On 8/14/14 7:36 PM, Mateusz Kaczynski wrote:
> > I'm trying to get es-hadoop repository plugin working on our hadoop
> 2.0.0-cdh4.6.0 distribution and it seems like I'm
> > quite lost.
> >
> > I installed plugin's -hadoop2 version on the machines on our hadoop
> cluster (which also run our stage elasticsearch nodes).
> >
> > When attempting to create a repository on one of the datanodes with:
> >
> > curl -XPUT 1.0.0.1:9200/_snapshot/hdfs -d '{"type":"hdfs",
> "settings": {"uri": "hdfs://1.0.0.10:54310",
> > "path":"/es_backup"}}'
> >
> >
> > I end up with the logs being filled with the following error:
> > Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol
> message contained an invalid tag (zero).
> > at
> com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:89)
>
>
> > at
> com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:108)
> > at
> org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.<init>(RpcHeaderProtos.java:1398)
>
>
> > at
> org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.<init>(RpcHeaderProtos.java:1362)
>
>
> > at
> org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto$1.parsePartialFrom(RpcHeaderProtos.java:1492)
>
>
> > at
> org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto$1.parsePartialFrom(RpcHeaderProtos.java:1487)
>
>
> > at
> com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
>
> > at
> com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:241)
>
>
> > at
> com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:253)
>
>
> > at
> com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:259)
>
>
> > at
> com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:49)
>
> > at
> org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcHeaderProtos.java:2364)
>
>
> > at
> org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:996)
> > at org.apache.hadoop.ipc.Client$Connection.run(Client.java:891)
> >
> > Is it possible that this is caused by the incompatible hadoop versions
> (2.2 used by plugin with 2.0 being installed) and
> > it is necessary to get it build with downgraded version?
> >
> > Also, to build the jar, is it just
> >
> > gradle build -Pdistro=hadoopYarn ?
> >
> > Because release 2.0.1 does not quite work for me as it fails to find
> JobLocalizer.class.
> >
> > Regards,
> > Mateusz
> >
> > --
> > You received this message because you are subscribed to the Google
> Groups "elasticsearch" group.
> > To unsubscribe from this group and stop receiving emails from it, send
> an email to
> > [email protected] <javascript:> <mailto:
> [email protected] <javascript:>>.
> > To view this discussion on the web visit
> >
> https://groups.google.com/d/msgid/elasticsearch/5aff7e2a-eb3e-4bb8-8698-05fec6a67e87%40googlegroups.com
>
> > <
> https://groups.google.com/d/msgid/elasticsearch/5aff7e2a-eb3e-4bb8-8698-05fec6a67e87%40googlegroups.com?utm_medium=email&utm_source=footer>.
>
>
> > For more options, visit https://groups.google.com/d/optout.
>
> --
> Costin
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/4ae0b343-72aa-459e-930e-559852c5d310%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
