You shall find detail log in Hadoop Node Manager GUI. The console message does not tell much.
On Mon, Dec 22, 2014 at 11:51 PM, 李运田 <[email protected]> wrote: > I use hcatalog as metadata Storage .so I can use hive to create table and > manipulate data .I can also use pig to manipulate the table which Hive > create. > now, I can use > "a = load 'tableHIVE' using org.apache.hcatalog.pig.HCatLoader();" > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - > mapred.input.dir.recursive is deprecated. Instead, use > mapreduce.input.fileinputformat.input.dir.recursive > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - mapred.max.split.size is > deprecated. Instead, use mapreduce.input.fileinputformat.split.maxsize > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - mapred.min.split.size is > deprecated. Instead, use mapreduce.input.fileinputformat.split.minsize > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - > mapred.min.split.size.per.rack is deprecated. Instead, use > mapreduce.input.fileinputformat.split.minsize.per.rack > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - > mapred.min.split.size.per.node is deprecated. Instead, use > mapreduce.input.fileinputformat.split.minsize.per.node > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - mapred.reduce.tasks is > deprecated. Instead, use mapreduce.job.reduces > 2014-12-23 15:31:45,791 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - > mapred.reduce.tasks.speculative.execution is deprecated. Instead, use > mapreduce.reduce.speculative > 2014-12-23 15:31:46,318 [main] INFO hive.metastore - Trying to connect to > metastore with URI thrift://10.210.90.*:9083 > 2014-12-23 15:31:46,407 [main] INFO hive.metastore - Connected to > metastore. > 2014-12-23 15:31:46,702 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > but when I store the data . I always get error： > "store a into 'tablePIG' using org.apache.hcatalog.pig.HCatStorer(); " > 2014-12-23 15:36:54,051 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > 2014-12-23 15:36:54,164 [main] INFO hive.metastore - Trying to connect to > metastore with URI thrift://10.210.90.*:9083 > 2014-12-23 15:36:54,166 [main] INFO hive.metastore - Connected to > metastore. > 2014-12-23 15:36:54,216 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig features used in the > script: UNKNOWN > 2014-12-23 15:36:54,218 [main] INFO > org.apache.pig.newplan.logical.optimizer.LogicalPlanOptimizer - > {RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, > DuplicateForEachColumnRewrite, GroupByConstParallelSetter, > ImplicitSplitInserter, LimitOptimizer, LoadTypeCastInserter, MergeFilter, > MergeForEach, NewPartitionFilterOptimizer, PartitionFilterOptimizer, > PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter], > RULES_DISABLED=[FilterLogicExpressionSimplifier]} > 2014-12-23 15:36:54,219 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > 2014-12-23 15:36:54,225 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler - > File concatenation threshold: 100 optimistic? false > 2014-12-23 15:36:54,227 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size before optimization: 1 > 2014-12-23 15:36:54,227 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOptimizer > - MR plan size after optimization: 1 > 2014-12-23 15:36:54,255 [main] INFO org.apache.hadoop.yarn.client.RMProxy > - Connecting to ResourceManager at pdmdev00/10.210.90.101:8050 > 2014-12-23 15:36:54,266 [main] INFO > org.apache.pig.tools.pigstats.ScriptState - Pig script settings are added > to the job > 2014-12-23 15:36:54,272 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - mapred.job.reduce.markreset.buffer.percent is not set, set to default 0.3 > 2014-12-23 15:36:54,373 [main] INFO > org.apache.hadoop.conf.Configuration.deprecation - fs.default.name is > deprecated. Instead, use fs.defaultFS > 2014-12-23 15:36:54,380 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - creating jar file Job8750450960344518471.jar > 2014-12-23 15:36:59,084 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - jar file Job8750450960344518471.jar created > 2014-12-23 15:36:59,096 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler > - Setting up single store job > 2014-12-23 15:36:59,145 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 1 map-reduce job(s) waiting for submission. > 2014-12-23 15:36:59,150 [JobControl] INFO > org.apache.hadoop.yarn.client.RMProxy - Connecting to ResourceManager at > pdmdev00/10.210.90.101:8050 > 2014-12-23 15:37:00,123 [JobControl] INFO > org.apache.hadoop.mapred.FileInputFormat - Total input paths to process : 1 > 2014-12-23 15:37:00,123 [JobControl] INFO > org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil - Total input > paths (combined) to process : 1 > 2014-12-23 15:37:00,376 [JobControl] INFO > org.apache.hadoop.mapreduce.JobSubmitter - number of splits:1 > 2014-12-23 15:37:00,527 [JobControl] INFO > org.apache.hadoop.mapreduce.JobSubmitter - Submitting tokens for job: > job_1418807862295_0055 > 2014-12-23 15:37:00,772 [JobControl] INFO > org.apache.hadoop.yarn.client.api.impl.YarnClientImpl - Submitted > application application_1418807862295_0055 > 2014-12-23 15:37:00,782 [JobControl] INFO org.apache.hadoop.mapreduce.Job > - The url to track the job: > http://pdmdev00:8088/proxy/application_1418807862295_0055/ > 2014-12-23 15:37:00,783 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - HadoopJobId: job_1418807862295_0055 > 2014-12-23 15:37:00,783 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Processing aliases a > 2014-12-23 15:37:00,783 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - detailed locations: M: a[1,4] C: R: > 2014-12-23 15:37:00,827 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 0% complete > 2014-12-23 15:37:15,910 [main] WARN > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Ooops! Some job has failed! Specify -stop_on_failure if you want Pig to > stop immediately on failure. > 2014-12-23 15:37:15,910 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - job job_1418807862295_0055 has failed! Stop running all dependent jobs > 2014-12-23 15:37:15,910 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - 100% complete > 2014-12-23 15:37:15,971 [main] ERROR > org.apache.pig.tools.pigstats.PigStatsUtil - 1 map reduce job(s) failed! > 2014-12-23 15:37:15,971 [main] INFO > org.apache.pig.tools.pigstats.SimplePigStats - Script Statistics: > > > HadoopVersionPigVersionUserIdStartedAtFinishedAtFeatures > 2.3.0-bc1.0.00.12.0-cdh5.0.0root2014-12-23 15:36:542014-12-23 > 15:37:15UNKNOWN > Failed! > Failed Jobs: > JobIdAliasFeatureMessageOutputs > job_1418807862295_0055aMAP_ONLYMessage: Job > failed!hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161, > Input(s): > Failed to read data from "table1418909729552" > Output(s): > Failed to produce result in > "hdfs://pdmdev00:8020/tmp/temp418476715/tmp-1157967161" > Counters: > Total records written : 0 > Total bytes written : 0 > Spillable Memory Manager spill count : 0 > Total bags proactively spilled: 0 > Total records proactively spilled: 0 > Job DAG: > job_1418807862295_0055 > 2014-12-23 15:37:15,972 [main] INFO > org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher > - Failed! > 2014-12-23 15:37:15,979 [main] ERROR org.apache.pig.tools.grunt.Grunt - > ERROR 1066: Unable to open iterator for alias a > Details at logfile: /root/pig_1419302861874.log -- CONFIDENTIALITY NOTICE NOTICE: This message is intended for the use of the individual or entity to which it is addressed and may contain information that is confidential, privileged and exempt from disclosure under applicable law. If the reader of this message is not the intended recipient, you are hereby notified that any printing, copying, dissemination, distribution, disclosure or forwarding of this communication is strictly prohibited. If you have received this communication in error, please contact the sender immediately and delete it from your system. Thank You.
