Hello! I have problem with writing data to elasticsearch as described in: http://www.elastic.co/guide/en/elasticsearch/hadoop/2.0/hive.html#_writing_data_to_elasticsearch_2
I use cloudera-quickstart-vm-5.3.0-0-vmware to test how working with eslasticsearch in hadoop. The queries from hive to elasticsearch working as well but writing data as describe in tutorial is every time failed. Log for this task is: Diagnostics: Application application_1428135716222_0003 failed 2 times due to AM Container for appattempt_1428135716222_0003_000002 exited with exitCode: 1 due to: Exception from container-launch. Container id: container_1428135716222_0003_02_000001 Exit code: 1 Stack trace: ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Container exited with a non-zero exit code 1 .Failing this attempt.. Failing the application. in yarn-yarn-nodemanager-quickstart.cloudera.log: 2015-04-04 13:14:59,618 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.ContainerManagerImpl: Start request for container_1428135716222_0003_02_000001 by user cloudera 2015-04-04 13:14:59,618 INFO org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=cloudera IP=127.0.0.1 OPERATION=Start Container Request TARGET=ContainerManageImpl RESULT=SUCCESS APPID=application_1428135716222_0003 CONTAINERID=container_1428135716222_0003_02_000001 2015-04-04 13:14:59,618 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Adding container_1428135716222_0003_02_000001 to application application_1428135716222_0003 2015-04-04 13:14:59,619 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1428135716222_0003_02_000001 transitioned from NEW to LOCALIZING 2015-04-04 13:14:59,619 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_INIT for appId application_1428135716222_0003 2015-04-04 13:14:59,619 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1428135716222_0003_02_000001 transitioned from LOCALIZING to LOCALIZED 2015-04-04 13:14:59,636 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1428135716222_0003_02_000001 transitioned from LOCALIZED to RUNNING 2015-04-04 13:14:59,642 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: launchContainer: [nice, -n, 0, bash, /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/appcache/application_1428135716222_0003/container_1428135716222_0003_02_000001/default_container_executor.sh] 2015-04-04 13:15:00,625 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exit code from container container_1428135716222_0003_02_000001 is : 1 2015-04-04 13:15:00,625 WARN org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Exception from container-launch with container ID: container_1428135716222_0003_02_000001 and exit code: 1 ExitCodeException exitCode=1: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) at org.apache.hadoop.util.Shell.run(Shell.java:455) at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exception from container-launch. 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Container id: container_1428135716222_0003_02_000001 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Exit code: 1 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: Stack trace: ExitCodeException exitCode=1: 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell.runCommand(Shell.java:538) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell.run(Shell.java:455) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:702) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:197) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:299) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:81) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.FutureTask.run(FutureTask.java:262) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.ContainerExecutor: at java.lang.Thread.run(Thread.java:745) 2015-04-04 13:15:00,626 WARN org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Container exited with a non-zero exit code 1 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1428135716222_0003_02_000001 transitioned from RUNNING to EXITED_WITH_FAILURE 2015-04-04 13:15:00,626 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch: Cleaning up container container_1428135716222_0003_02_000001 2015-04-04 13:15:00,644 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/appcache/application_1428135716222_0003/container_1428135716222_0003_02_000001 2015-04-04 13:15:00,644 WARN org.apache.hadoop.yarn.server.nodemanager.NMAuditLogger: USER=cloudera OPERATION=Container Finished - Failed TARGET=ContainerImpl RESULT=FAILURE DESCRIPTION=Container failed with state: EXITED_WITH_FAILURE APPID=application_1428135716222_0003 CONTAINERID=container_1428135716222_0003_02_000001 2015-04-04 13:15:00,644 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.container.Container: Container container_1428135716222_0003_02_000001 transitioned from EXITED_WITH_FAILURE to DONE 2015-04-04 13:15:00,644 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Removing container_1428135716222_0003_02_000001 from application application_1428135716222_0003 2015-04-04 13:15:00,644 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Considering container container_1428135716222_0003_02_000001 for log-aggregation 2015-04-04 13:15:00,644 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event CONTAINER_STOP for appId application_1428135716222_0003 2015-04-04 13:15:02,203 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Starting resource-monitoring for container_1428135716222_0003_02_000001 2015-04-04 13:15:02,203 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl: Stopping resource-monitoring for container_1428135716222_0003_02_000001 2015-04-04 13:15:02,600 INFO org.apache.hadoop.yarn.server.nodemanager.NodeStatusUpdaterImpl: Removed completed containers from NM context: [container_1428135716222_0003_02_000001] 2015-04-04 13:15:02,602 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1428135716222_0003 transitioned from RUNNING to APPLICATION_RESOURCES_CLEANINGUP 2015-04-04 13:15:02,603 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting absolute path : /var/lib/hadoop-yarn/cache/yarn/nm-local-dir/usercache/cloudera/appcache/application_1428135716222_0003 2015-04-04 13:15:02,603 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.AuxServices: Got event APPLICATION_STOP for appId application_1428135716222_0003 2015-04-04 13:15:02,605 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.application.Application: Application application_1428135716222_0003 transitioned from APPLICATION_RESOURCES_CLEANINGUP to FINISHED 2015-04-04 13:15:02,606 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Application just finished : application_1428135716222_0003 2015-04-04 13:15:02,606 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Starting aggregate log-file for app application_1428135716222_0003 at /var/log/hadoop-yarn/apps/cloudera/logs/application_1428135716222_0003/quickstart.cloudera_47277.tmp 2015-04-04 13:15:02,641 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1428135716222_0003_01_000001. Current good log dirs are /var/log/hadoop-yarn/containers 2015-04-04 13:15:02,657 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Uploading logs for container container_1428135716222_0003_02_000001. Current good log dirs are /var/log/hadoop-yarn/containers 2015-04-04 13:15:02,658 INFO org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor: Deleting path : /var/log/hadoop-yarn/containers/application_1428135716222_0003 2015-04-04 13:15:02,763 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.AppLogAggregatorImpl: Finished aggregate log-file for app application_1428135716222_0003 2015-04-04 13:20:26,071 INFO SecurityLogger.org.apache.hadoop.ipc.Server: Auth successful for appattempt_1428135716222_0004_000001 (auth:SIMPLE) Anybody have some idea how to fix it? Thanks Lev -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to elasticsearch+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/ee24a950-2fbf-4961-8486-f372c3dfae03%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
