The pi job does not leave any HDFS data. You can see its result on the console: "Estimated value of Pi is 3.14127500000000000000".
What are you trying to really test or do here? If you want data on HDFS, you can load it up via 'hadoop fs -put', or generate it with teragen, etc.? On Mon, Jun 23, 2014 at 12:06 PM, EdwardKing <[email protected]> wrote: > I use hadoop 2.2.0 under Redhat Linux,I run hadoop mapreduce examples like > follows: > > [yarn@localhost sbin]$ hadoop jar > /opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar > pi > -Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory > -libjars > /opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar > 16 10000 > Number of Maps = 16 > Samples per Map = 10000 > Wrote input for Map #0 > Wrote input for Map #1 > Wrote input for Map #2 > Wrote input for Map #3 > Wrote input for Map #4 > Wrote input for Map #5 > Wrote input for Map #6 > Wrote input for Map #7 > Wrote input for Map #8 > Wrote input for Map #9 > Wrote input for Map #10 > Wrote input for Map #11 > Wrote input for Map #12 > Wrote input for Map #13 > Wrote input for Map #14 > Wrote input for Map #15 > Starting Job > 14/06/22 19:57:58 INFO client.RMProxy: Connecting to ResourceManager at > /0.0.0.0:8032 > 14/06/22 19:57:59 INFO input.FileInputFormat: Total input paths to process : > 16 > 14/06/22 19:57:59 INFO mapreduce.JobSubmitter: number of splits:16 > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.classpath.files > is deprecated. Instead, use mapreduce.job.classpath.files > 14/06/22 19:57:59 INFO Configuration.deprecation: user.name is deprecated. > Instead, use mapreduce.job.user.name > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.jar is deprecated. > Instead, use mapreduce.job.jar > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapred.cache.files.filesizes is deprecated. Instead, use > mapreduce.job.cache.files.filesizes > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files is > deprecated. Instead, use mapreduce.job.cache.files > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapred.map.tasks.speculative.execution is deprecated. Instead, use > mapreduce.map.speculative > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.reduce.tasks is > deprecated. Instead, use mapreduce.job.reduces > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.value.class > is deprecated. Instead, use mapreduce.job.output.value.class > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapred.reduce.tasks.speculative.execution is deprecated. Instead, use > mapreduce.reduce.speculative > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapred.used.genericoptionsparser is deprecated. Instead, use > mapreduce.client.genericoptionsparser.used > 14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.map.class is > deprecated. Instead, use mapreduce.job.map.class > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.name is > deprecated. Instead, use mapreduce.job.name > 14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.reduce.class is > deprecated. Instead, use mapreduce.job.reduce.class > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapreduce.inputformat.class is deprecated. Instead, use > mapreduce.job.inputformat.class > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.input.dir is > deprecated. Instead, use mapreduce.input.fileinputformat.inputdir > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.dir is > deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapreduce.outputformat.class is deprecated. Instead, use > mapreduce.job.outputformat.class > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.map.tasks is > deprecated. Instead, use mapreduce.job.maps > 14/06/22 19:57:59 INFO Configuration.deprecation: > mapred.cache.files.timestamps is deprecated. Instead, use > mapreduce.job.cache.files.timestamps > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.key.class is > deprecated. Instead, use mapreduce.job.output.key.class > 14/06/22 19:57:59 INFO Configuration.deprecation: mapred.working.dir is > deprecated. Instead, use mapreduce.job.working.dir > 14/06/22 19:58:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: > job_1403492007236_0001 > 14/06/22 19:58:01 INFO impl.YarnClientImpl: Submitted application > application_1403492007236_0001 to ResourceManager at /0.0.0.0:8032 > 14/06/22 19:58:01 INFO mapreduce.Job: The url to track the job: > http://localhost:8088/proxy/application_1403492007236_0001/ > 14/06/22 19:58:01 INFO mapreduce.Job: Running job: job_1403492007236_0001 > 14/06/22 19:58:22 INFO mapreduce.Job: Job job_1403492007236_0001 running in > uber mode : false > 14/06/22 19:58:22 INFO mapreduce.Job: map 0% reduce 0% > 14/06/22 20:01:45 INFO mapreduce.Job: map 6% reduce 0% > 14/06/22 20:01:52 INFO mapreduce.Job: map 13% reduce 0% > 14/06/22 20:01:53 INFO mapreduce.Job: map 19% reduce 0% > 14/06/22 20:01:56 INFO mapreduce.Job: map 38% reduce 0% > 14/06/22 20:02:36 INFO mapreduce.Job: map 50% reduce 15% > 14/06/22 20:02:37 INFO mapreduce.Job: map 69% reduce 15% > 14/06/22 20:02:39 INFO mapreduce.Job: map 69% reduce 23% > 14/06/22 20:03:05 INFO mapreduce.Job: map 94% reduce 23% > 14/06/22 20:03:06 INFO mapreduce.Job: map 100% reduce 31% > 14/06/22 20:03:08 INFO mapreduce.Job: map 100% reduce 100% > 14/06/22 20:03:09 INFO mapreduce.Job: Job job_1403492007236_0001 completed > successfully > 14/06/22 20:03:09 INFO mapreduce.Job: Counters: 43 > File System Counters > FILE: Number of bytes read=358 > FILE: Number of bytes written=1387065 > FILE: Number of read operations=0 > FILE: Number of large read operations=0 > FILE: Number of write operations=0 > HDFS: Number of bytes read=4230 > HDFS: Number of bytes written=215 > HDFS: Number of read operations=67 > HDFS: Number of large read operations=0 > HDFS: Number of write operations=3 > Job Counters > Launched map tasks=16 > Launched reduce tasks=1 > Data-local map tasks=16 > Total time spent by all maps in occupied slots (ms)=1587720 > Total time spent by all reduces in occupied slots (ms)=71305 > Map-Reduce Framework > Map input records=16 > Map output records=32 > Map output bytes=288 > Map output materialized bytes=448 > Input split bytes=2342 > Combine input records=0 > Combine output records=0 > Reduce input groups=2 > Reduce shuffle bytes=448 > Reduce input records=32 > Reduce output records=0 > Spilled Records=64 > Shuffled Maps =16 > Failed Shuffles=0 > Merged Map outputs=16 > GC time elapsed (ms)=45172 > CPU time spent (ms)=107260 > Physical memory (bytes) snapshot=2563190784 > Virtual memory (bytes) snapshot=6111346688 > Total committed heap usage (bytes)=1955135488 > Shuffle Errors > BAD_ID=0 > CONNECTION=0 > IO_ERROR=0 > WRONG_LENGTH=0 > WRONG_MAP=0 > WRONG_REDUCE=0 > File Input Format Counters > Bytes Read=1888 > File Output Format Counters > Bytes Written=97 > Job Finished in 311.613 seconds > Estimated value of Pi is 3.14127500000000000000 > > Then I check hdfs like follows: > [yarn@localhost sbin]$ hdfs dfs -ls > [yarn@localhost sbin] > > Why hdfs don't show any information? How to do it? Thanks > > > > --------------------------------------------------------------------------------------------------- > Confidentiality Notice: The information contained in this e-mail and any > accompanying attachment(s) > is intended only for the use of the intended recipient and may be > confidential and/or privileged of > Neusoft Corporation, its subsidiaries and/or its affiliates. If any reader > of this communication is > not the intended recipient, unauthorized use, forwarding, printing, > storing, disclosure or copying > is strictly prohibited, and may be unlawful.If you have received this > communication in error,please > immediately notify the sender by return e-mail, and delete the original > message and all copies from > your system. Thank you. > --------------------------------------------------------------------------------------------------- -- Harsh J
