Hi,
The hdfs repository relies on vanilla Hadoop 2.2 since that's the official stable version of Yarn. Since you are using a
different
Hadoop version, use the 'light' version as explained in the docs - this contains only the repository-hdfs, without the
Hadoop dependency
(since you already have it installed).
In other words, both the error that you see as well as the 2.0.1 (regarding JobLocalizer) seems to be related to the
differences between
vanilla Hadoop 2.2 and the one you are using.
Hope this helps,
On 8/14/14 7:36 PM, Mateusz Kaczynski wrote:
I'm trying to get es-hadoop repository plugin working on our hadoop
2.0.0-cdh4.6.0 distribution and it seems like I'm
quite lost.
I installed plugin's -hadoop2 version on the machines on our hadoop cluster
(which also run our stage elasticsearch nodes).
When attempting to create a repository on one of the datanodes with:
curl -XPUT 1.0.0.1:9200/_snapshot/hdfs -d '{"type":"hdfs", "settings": {"uri":
"hdfs://1.0.0.10:54310",
"path":"/es_backup"}}'
I end up with the logs being filled with the following error:
Caused by: com.google.protobuf.InvalidProtocolBufferException: Protocol message
contained an invalid tag (zero).
at
com.google.protobuf.InvalidProtocolBufferException.invalidTag(InvalidProtocolBufferException.java:89)
at com.google.protobuf.CodedInputStream.readTag(CodedInputStream.java:108)
at
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.<init>(RpcHeaderProtos.java:1398)
at
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.<init>(RpcHeaderProtos.java:1362)
at
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto$1.parsePartialFrom(RpcHeaderProtos.java:1492)
at
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto$1.parsePartialFrom(RpcHeaderProtos.java:1487)
at com.google.protobuf.AbstractParser.parsePartialFrom(AbstractParser.java:200)
at
com.google.protobuf.AbstractParser.parsePartialDelimitedFrom(AbstractParser.java:241)
at
com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:253)
at
com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:259)
at com.google.protobuf.AbstractParser.parseDelimitedFrom(AbstractParser.java:49)
at
org.apache.hadoop.ipc.protobuf.RpcHeaderProtos$RpcResponseHeaderProto.parseDelimitedFrom(RpcHeaderProtos.java:2364)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:996)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:891)
Is it possible that this is caused by the incompatible hadoop versions (2.2
used by plugin with 2.0 being installed) and
it is necessary to get it build with downgraded version?
Also, to build the jar, is it just
gradle build -Pdistro=hadoopYarn ?
Because release 2.0.1 does not quite work for me as it fails to find
JobLocalizer.class.
Regards,
Mateusz
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected]
<mailto:[email protected]>.
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/5aff7e2a-eb3e-4bb8-8698-05fec6a67e87%40googlegroups.com
<https://groups.google.com/d/msgid/elasticsearch/5aff7e2a-eb3e-4bb8-8698-05fec6a67e87%40googlegroups.com?utm_medium=email&utm_source=footer>.
For more options, visit https://groups.google.com/d/optout.
--
Costin
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/53ED232C.7080201%40gmail.com.
For more options, visit https://groups.google.com/d/optout.
