Hi,
Still struggling to get Es + Hadoop smoothly working together.
Anyone facing this type of issue ?
ERROR security.UserGroupInformation: PriviledgedActionException as:hue
(auth:SIMPLE) cause:BeeswaxException
14/02/26 05:42:34 INFO Configuration.deprecation:
mapred.input.dir.recursive is deprecated. Instead, use
mapreduce.input.fileinputformat.input.dir.recursive
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=Driver.run>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=TimeToSubmit>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=compile>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=parse>
14/02/26 05:42:34 INFO parse.ParseDriver: Parsing command: use default
14/02/26 05:42:34 INFO parse.ParseDriver: Parse Completed
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=parse start=1393422154484
end=1393422154485 duration=1>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=semanticAnalyze>
14/02/26 05:42:34 INFO ql.Driver: Semantic Analysis Completed
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=semanticAnalyze
start=1393422154485 end=1393422154485 duration=0>
14/02/26 05:42:34 INFO ql.Driver: Returning Hive schema:
Schema(fieldSchemas:null, properties:null)
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=compile start=1393422154483
end=1393422154485 duration=2>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=Driver.execute>
14/02/26 05:42:34 INFO ql.Driver: Starting command: use default
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=TimeToSubmit
start=1393422154483 end=1393422154486 duration=3>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=runTasks>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=task.DDL.Stage-0>
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=task.DDL.Stage-0
start=1393422154486 end=1393422154493 duration=7>
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=runTasks
start=1393422154486 end=1393422154493 duration=7>
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=Driver.execute
start=1393422154485 end=1393422154493 duration=8>
OK
14/02/26 05:42:34 INFO ql.Driver: OK
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=releaseLocks>
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=releaseLocks
start=1393422154493 end=1393422154493 duration=0>
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=Driver.run
start=1393422154483 end=1393422154493 duration=10>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=compile>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=parse>
14/02/26 05:42:34 INFO parse.ParseDriver: Parsing command: INSERT OVERWRITE
TABLE eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent FROM logs s
14/02/26 05:42:34 INFO parse.ParseDriver: Parse Completed
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=parse start=1393422154494
end=1393422154495 duration=1>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=semanticAnalyze>
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Starting Semantic Analysis
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Completed phase 1 of
Semantic Analysis
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Get metadata for source
tables
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Get metadata for subqueries
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Get metadata for destination
tables
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Completed getting MetaData
in Semantic Analysis
14/02/26 05:42:34 INFO ppd.OpProcFactory: Processing for FS(9)
14/02/26 05:42:34 INFO ppd.OpProcFactory: Processing for SEL(8)
14/02/26 05:42:34 INFO ppd.OpProcFactory: Processing for TS(7)
14/02/26 05:42:34 INFO physical.MetadataOnlyOptimizer: Looking for table
scans where optimization is applicable
14/02/26 05:42:34 INFO physical.MetadataOnlyOptimizer: Found 0 metadata
only table scans
14/02/26 05:42:34 INFO parse.SemanticAnalyzer: Completed plan generation
14/02/26 05:42:34 INFO ql.Driver: Semantic Analysis Completed
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=semanticAnalyze
start=1393422154495 end=1393422154549 duration=54>
14/02/26 05:42:34 INFO ql.Driver: Returning Hive schema:
Schema(fieldSchemas:[FieldSchema(name:time, type:string, comment:null),
FieldSchema(name:ext, type:string, comment:null), FieldSchema(name:ip,
type:string, comment:null), FieldSchema(name:req, type:string, comment:null),
FieldSchema(name:res, type:int, comment:null), FieldSchema(name:agent,
type:string, comment:null)], properties:null)
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=compile start=1393422154493
end=1393422154549 duration=56>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=Driver.execute>
14/02/26 05:42:34 INFO ql.Driver: Starting command: INSERT OVERWRITE TABLE
eslogs SELECT s.time, s.ext, s.ip, s.req, s.res, s.agent FROM logs s
Total MapReduce jobs = 1
14/02/26 05:42:34 INFO ql.Driver: Total MapReduce jobs = 1
14/02/26 05:42:34 INFO ql.Driver: </PERFLOG method=TimeToSubmit
start=1393422154483 end=1393422154585 duration=102>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=runTasks>
14/02/26 05:42:34 INFO ql.Driver: <PERFLOG method=task.MAPRED.Stage-0>
Launching Job 1 out of 1
14/02/26 05:42:34 INFO ql.Driver: Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
14/02/26 05:42:34 INFO exec.Task: Number of reduce tasks is set to 0 since
there's no reduce operator
14/02/26 05:42:34 INFO ql.Context: New scratch dir is
hdfs://sandbox.hortonworks.com:8020/tmp/hive-beeswax-hue/hive_2014-02-26_05-42-34_494_4288001753889524446-3
14/02/26 05:42:34 INFO Configuration.deprecation:
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use
mapreduce.reduce.speculative
14/02/26 05:42:34 INFO mr.ExecDriver: Using
org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
14/02/26 05:42:34 INFO exec.Utilities: Processing alias s
14/02/26 05:42:34 INFO exec.Utilities: Adding input file
hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/logs
14/02/26 05:42:34 INFO exec.Utilities: Content Summary not cached for
hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/logs
14/02/26 05:42:34 INFO ql.Context: New scratch dir is
hdfs://sandbox.hortonworks.com:8020/tmp/hive-beeswax-hue/hive_2014-02-26_05-42-34_494_4288001753889524446-3
14/02/26 05:42:34 INFO exec.Utilities: <PERFLOG method=serializePlan>
14/02/26 05:42:34 INFO exec.Utilities: Serializing MapWork via kryo
14/02/26 05:42:34 INFO exec.Utilities: </PERFLOG method=serializePlan
start=1393422154741 end=1393422154807 duration=66>
14/02/26 05:42:34 INFO client.RMProxy: Connecting to ResourceManager at
sandbox.hortonworks.com/10.0.2.15:8050
14/02/26 05:42:34 INFO client.RMProxy: Connecting to ResourceManager at
sandbox.hortonworks.com/10.0.2.15:8050
14/02/26 05:42:34 WARN mapreduce.JobSubmitter: Hadoop command-line option
parsing not performed. Implement the Tool interface and execute your
application with ToolRunner to remedy this.
14/02/26 05:42:35 INFO io.CombineHiveInputFormat: <PERFLOG method=getSplits>
14/02/26 05:42:35 INFO io.CombineHiveInputFormat: CombineHiveInputSplit
creating pool for hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/logs;
using filter path hdfs://sandbox.hortonworks.com:8020/apps/hive/warehouse/logs
14/02/26 05:42:35 INFO Configuration.deprecation: mapred.min.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize
14/02/26 05:42:35 INFO Configuration.deprecation:
mapred.min.split.size.per.node is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.node
14/02/26 05:42:35 INFO Configuration.deprecation:
mapred.min.split.size.per.rack is deprecated. Instead, use
mapreduce.input.fileinputformat.split.minsize.per.rack
14/02/26 05:42:35 INFO Configuration.deprecation: mapred.max.split.size is
deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize
14/02/26 05:42:35 INFO input.FileInputFormat: Total input paths to process
: 1
14/02/26 05:42:35 INFO input.CombineFileInputFormat: DEBUG: Terminated node
allocation with : CompletedNodes: 1, size left: 0
14/02/26 05:42:35 INFO io.CombineHiveInputFormat: number of splits 1
14/02/26 05:42:35 INFO io.CombineHiveInputFormat: </PERFLOG method=getSplits
start=1393422155194 end=1393422155204 duration=10>
14/02/26 05:42:35 INFO mapreduce.JobSubmitter: number of splits:1
14/02/26 05:42:35 INFO Configuration.deprecation: mapred.reduce.tasks is
deprecated. Instead, use mapreduce.job.reduces
14/02/26 05:42:35 INFO mapreduce.JobSubmitter: Submitting tokens for job:
job_1393416170595_0002
14/02/26 05:42:35 INFO impl.YarnClientImpl: Submitted application
application_1393416170595_0002 to ResourceManager at
sandbox.hortonworks.com/10.0.2.15:8050
14/02/26 05:42:35 INFO mapreduce.Job: The url to track the job:
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
Starting Job = job_1393416170595_0002, Tracking URL =
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
14/02/26 05:42:35 INFO exec.Task: Starting Job = job_1393416170595_0002,
Tracking URL =
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1393416170595_0002
14/02/26 05:42:35 INFO exec.Task: Kill Command = /usr/lib/hadoop/bin/hadoop
job -kill job_1393416170595_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
14/02/26 05:42:42 INFO exec.Task: Hadoop job information for Stage-0:
number of mappers: 1; number of reducers: 0
14/02/26 05:42:42 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
2014-02-26 05:42:42,747 Stage-0 map = 0%, reduce = 0%
14/02/26 05:42:42 INFO exec.Task: 2014-02-26 05:42:42,747 Stage-0 map = 0%,
reduce = 0%
14/02/26 05:43:06 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
2014-02-26 05:43:06,683 Stage-0 map = 100%, reduce = 0%
14/02/26 05:43:06 INFO exec.Task: 2014-02-26 05:43:06,683 Stage-0 map =
100%, reduce = 0%
14/02/26 05:43:09 WARN mapreduce.Counters: Group
org.apache.hadoop.mapred.Task$Counter is deprecated. Use
org.apache.hadoop.mapreduce.TaskCounter instead
Ended Job = job_1393416170595_0002 with errors
14/02/26 05:43:09 ERROR exec.Task: Ended Job = job_1393416170595_0002 with
errors
14/02/26 05:43:09 INFO impl.YarnClientImpl: Killing application
application_1393416170595_0002
14/02/26 05:43:09 INFO ql.Driver: </PERFLOG method=task.MAPRED.Stage-0
start=1393422154585 end=1393422189912 duration=35327>
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
14/02/26 05:43:09 ERROR ql.Driver: FAILED: Execution Error, return code 2
from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
14/02/26 05:43:09 INFO ql.Driver: </PERFLOG method=Driver.execute
start=1393422154583 end=1393422189912 duration=35329>
MapReduce Jobs Launched:
14/02/26 05:43:09 INFO ql.Driver: MapReduce Jobs Launched:
14/02/26 05:43:09 WARN mapreduce.Counters: Group FileSystemCounters is
deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
14/02/26 05:43:09 INFO ql.Driver: Job 0: Map: 1 HDFS Read: 0 HDFS Write:
0 FAIL
Total MapReduce CPU Time Spent: 0 msec
14/02/26 05:43:09 INFO ql.Driver: Total MapReduce CPU Time Spent: 0 msec
14/02/26 05:43:09 ERROR beeswax.BeeswaxServiceImpl: Exception while
processing query
BeeswaxException(message:Driver returned: 2. Errors: OK
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1393416170595_0002, Tracking URL =
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1393416170595_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-02-26 05:42:42,747 Stage-0 map = 0%, reduce = 0%
2014-02-26 05:43:06,683 Stage-0 map = 100%, reduce = 0%
Ended Job = job_1393416170595_0002 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
, log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
handle:QueryHandle(id:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5), SQLState: )
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:351)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:337)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1471)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
14/02/26 05:43:10 ERROR security.UserGroupInformation:
PriviledgedActionException as:hue (auth:SIMPLE)
cause:BeeswaxException(message:Driver returned: 2. Errors: OK
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1393416170595_0002, Tracking URL =
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1393416170595_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-02-26 05:42:42,747 Stage-0 map = 0%, reduce = 0%
2014-02-26 05:43:06,683 Stage-0 map = 100%, reduce = 0%
Ended Job = job_1393416170595_0002 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
, log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
handle:QueryHandle(id:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5), SQLState: )
14/02/26 05:43:10 ERROR beeswax.BeeswaxServiceImpl: Caught unexpected
exception.
java.lang.reflect.UndeclaredThrowableException
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1504)
at
com.cloudera.beeswax.BeeswaxServiceImpl.doWithState(BeeswaxServiceImpl.java:772)
at
com.cloudera.beeswax.BeeswaxServiceImpl.fetch(BeeswaxServiceImpl.java:980)
at
com.cloudera.beeswax.api.BeeswaxService$Processor$fetch.getResult(BeeswaxService.java:987)
at
com.cloudera.beeswax.api.BeeswaxService$Processor$fetch.getResult(BeeswaxService.java:971)
at org.apache.thrift.ProcessFunction.process(ProcessFunction.java:39)
at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:39)
at
org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:206)
at
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: BeeswaxException(message:Driver returned: 2. Errors: OK
Total MapReduce jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_1393416170595_0002, Tracking URL =
http://sandbox.hortonworks.com:8088/proxy/application_1393416170595_0002/
Kill Command = /usr/lib/hadoop/bin/hadoop job -kill job_1393416170595_0002
Hadoop job information for Stage-0: number of mappers: 1; number of
reducers: 0
2014-02-26 05:42:42,747 Stage-0 map = 0%, reduce = 0%
2014-02-26 05:43:06,683 Stage-0 map = 100%, reduce = 0%
Ended Job = job_1393416170595_0002 with errors
FAILED: Execution Error, return code 2 from
org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Job 0: Map: 1 HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec
, log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
handle:QueryHandle(id:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5,
log_context:aaf7e696-5cbe-4d4d-82ee-88dc6c7154d5), SQLState: )
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState.execute(BeeswaxServiceImpl.java:351)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:609)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1$1.run(BeeswaxServiceImpl.java:598)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:337)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1471)
at
com.cloudera.beeswax.BeeswaxServiceImpl$RunningQueryState$1.run(BeeswaxServiceImpl.java:598)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
... 3 more
Thanks.
Cheers,
Yann
--
You received this message because you are subscribed to the Google Groups
"elasticsearch" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to
[email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/elasticsearch/9aa69a10-09f4-408c-87be-db1749485d6a%40googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.
