Looks like a bug in hive which passes a null progressable to the underlying
task. I would recommend upgrading hive to 0.10 or better ( and hadoop to
1.2.1 while at it).
On Sep 16, 2014 8:27 PM, "ibmuser1" <[email protected]> wrote:
> Hi, my hadoop version is 1.1.1 and hive version is 0.9.0 (biginsights
> installation). I am trying to push data from existing hive table(s) into
> elasticsearch. My job fails with the following error. I copied hive
> script as well below the error. Not sure what I am doing wrong. Can you
> help? Many thanks. The es table I am trying to create and the existing
> hive table have exactly the same schema.
>
> java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while
> processing row
> {"id":1533,"17qnigvs":981,"cpdgolct":4,"8kzzi4te":20,"atbvkhii":174,"xyygmjew":93,"cjtf65bx":35.237512742099895,"fj81mxmy":0.0,"zwewms3g":3386.0}
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:161)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:50)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:435)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:371)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(AccessController.java:310)
> at javax.security.auth.Subject.doAs(Subject.java:573)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1149)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime
> Error while processing row
> {"id":1533,"17qnigvs":981,"cpdgolct":4,"8kzzi4te":20,"atbvkhii":174,"xyygmjew":93,"cjtf65bx":35.237512742099895,"fj81mxmy":0.0,"zwewms3g":3386.0}
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:548)
> at org.apache.hadoop.hive.ql.exec.ExecMapper.map(ExecMapper.java:143)
> ... 8 more
> Caused by: org.elasticsearch.hadoop.EsHadoopIllegalArgumentException: a
> valid progressable is required to report status to Hadoop
> at org.elasticsearch.hadoop.util.Assert.notNull(Assert.java:40)
> at org.elasticsearch.hadoop.mr.HeartBeat.<init>(HeartBeat.java:44)
> at
> org.elasticsearch.hadoop.mr.EsOutputFormat$EsRecordWriter.init(EsOutputFormat.java:189)
> at
> org.elasticsearch.hadoop.hive.EsHiveOutputFormat$EsHiveRecordWriter.write(EsHiveOutputFormat.java:58)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.processOp(FileSinkOperator.java:588)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at
> org.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:84)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:83)
> at org.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:471)
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:762)
> at org.apache.hadoop.hive.ql.exec.MapOperator.process(MapOperator.java:529)
> ... 9 more
>
>
> hive script
> ---------------
>
> add JAR
> /homes/ibmusr/elasticsearch-hadoop-2.0.1/dist/elasticsearch-hadoop-2.0.1.jar;
> add JAR
> /homes/ibmusr/elasticsearch-hadoop-2.0.1/dist/elasticsearch-hadoop-hive-2.0.1.jar;
>
> drop table alarmdata2_gba_ID_es;
>
> create external table alarmdata2_gba_ID_es(
> ID int,17qNigvs bigint, cPdgOLct bigint, 8KzZi4Te bigint, ATBVkHiI bigint,
> xyYGMjEw bigint, CjTF65Bx double, FJ81mxMy double, zWEwms3G double)
>
> STORED BY 'org.elasticsearch.hadoop.hive.EsStorageHandler'
> TBLPROPERTIES('es.resource' = 'alarmdata/gbid',
> 'es.mapping.id' = 'id',
> 'es.index.auto.create' = 'true');
>
> insert overwrite table alarmdata2_gba_ID_es
> select * from alarmdata2_gba_ID;
>
> --
> You received this message because you are subscribed to the Google Groups
> "elasticsearch" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email protected].
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/elasticsearch/fcedbc80-3ef6-4001-b30a-5068b48f3481%40googlegroups.com
> <https://groups.google.com/d/msgid/elasticsearch/fcedbc80-3ef6-4001-b30a-5068b48f3481%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/CAJogdmebSuTUnOkEnqHNM3Ey8yBDGgohTG_pwX%3D8TnC3BaGuqQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
