Yes. I have tried with *tcpdump -i lo* and * tshark -i lo* and i am able to monitor this
17:38:55.553775 IP localhost.54311 > localhost.39619: Flags [P.], seq 1499:1606, ack 4095, win 1020, options [nop,nop,TS val 446136 ecr 446136], length 107 17:38:55.553787 IP localhost.39619 > localhost.54311: Flags [.], ack 1606, win 685, options [nop,nop,TS val 446136 ecr 446136], length 0 17:38:55.855535 IP localhost.39619 > localhost.54311: Flags [P.], seq 4095:4368, ack 1606, win 685, options [nop,nop,TS val 446211 ecr 446136], length 273 17:38:55.855839 IP localhost.54311 > localhost.39619: Flags [P.], seq 1606:1713, ack 4368, win 1020, options [nop,nop,TS val 446211 ecr 446211], length 107 17:38:55.856061 IP localhost.39619 > localhost.54311: Flags [.], ack 1713, win 685, options [nop,nop,TS val 446212 ecr 446211], length 0 17:38:56.102755 IP localhost.37150 > localhost.domain: 1+ TXT? _kerberos.localdomain. (39) 17:38:56.104445 IP localhost.domain > localhost.37150: 1 NXDomain 0/1/0 (114) 17:38:56.157408 IP localhost.39619 > localhost.54311: Flags [P.], seq 4368:4641, ack 1713, win 685, options [nop,nop,TS val 446287 ecr 446211], length 2 I am not able to see the port number ( 13562) in the logs which i have mentioned in mapred.xml file. Also tcpdump -i lo port 13562 doesn't capture anything. Thanks Navaz On Sun, Dec 8, 2013 at 4:32 PM, Marco Shaw <[email protected]> wrote: > Try something like: > http://tutorialgenius.blogspot.ca/2012/03/tcpdump-localhost.html > > After you launch tcpdump it mentions "eth0", while your mapred-site.xml is > referring to "localhost" or "lo". Perhaps... > > (Sorry, I don't have a VM running and can't remember if it's "lo" = > "elle-o".) > > > On Sun, Dec 8, 2013 at 2:57 PM, navaz <[email protected]> wrote: > >> Thanks. I have tried with sudo command and command executed successfully. >> But nothing is captured. >> >> hduser@ubuntu:~/Downloads$ sudo tcpdump port 13562 >> [sudo] password for hduser: >> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >> listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes >> >> I am running the mapreduce program and this is the output. >> >> hduser@ubuntu:~/Downloads$ hadoop jar chiu-wordcount2.jar WordCount >> /user/hduser/getty/gettysburg.txt /user/hduser/getty/out8 >> Warning: $HADOOP_HOME is deprecated. >> >> 13/12/08 10:43:47 WARN mapred.JobClient: Use GenericOptionsParser for >> parsing the arguments. Applications should implement Tool for the same. >> 13/12/08 10:43:47 INFO input.FileInputFormat: Total input paths to >> process : 1 >> 13/12/08 10:43:47 INFO util.NativeCodeLoader: Loaded the native-hadoop >> library >> 13/12/08 10:43:47 WARN snappy.LoadSnappy: Snappy native library not loaded >> 13/12/08 10:43:48 INFO mapred.JobClient: Running job: >> job_201312081040_0001 >> 13/12/08 10:43:49 INFO mapred.JobClient: map 0% reduce 0% >> 13/12/08 10:43:54 INFO mapred.JobClient: map 100% reduce 0% >> 13/12/08 10:44:02 INFO mapred.JobClient: map 100% reduce 33% >> 13/12/08 10:44:03 INFO mapred.JobClient: map 100% reduce 100% >> 13/12/08 10:44:04 INFO mapred.JobClient: Job complete: >> job_201312081040_0001 >> 13/12/08 10:44:04 INFO mapred.JobClient: Counters: 29 >> 13/12/08 10:44:04 INFO mapred.JobClient: Job Counters >> 13/12/08 10:44:04 INFO mapred.JobClient: Launched reduce tasks=1 >> 13/12/08 10:44:04 INFO mapred.JobClient: SLOTS_MILLIS_MAPS=5478 >> 13/12/08 10:44:04 INFO mapred.JobClient: Total time spent by all >> reduces waiting after reserving slots (ms)=0 >> 13/12/08 10:44:04 INFO mapred.JobClient: Total time spent by all maps >> waiting after reserving slots (ms)=0 >> 13/12/08 10:44:04 INFO mapred.JobClient: Launched map tasks=1 >> 13/12/08 10:44:04 INFO mapred.JobClient: Data-local map tasks=1 >> 13/12/08 10:44:04 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=8804 >> 13/12/08 10:44:04 INFO mapred.JobClient: File Output Format Counters >> 13/12/08 10:44:04 INFO mapred.JobClient: Bytes Written=1106 >> 13/12/08 10:44:04 INFO mapred.JobClient: FileSystemCounters >> 13/12/08 10:44:04 INFO mapred.JobClient: FILE_BYTES_READ=3059 >> 13/12/08 10:44:04 INFO mapred.JobClient: HDFS_BYTES_READ=1606 >> 13/12/08 10:44:04 INFO mapred.JobClient: FILE_BYTES_WRITTEN=108770 >> 13/12/08 10:44:04 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=1106 >> 13/12/08 10:44:04 INFO mapred.JobClient: File Input Format Counters >> 13/12/08 10:44:04 INFO mapred.JobClient: Bytes Read=1486 >> 13/12/08 10:44:04 INFO mapred.JobClient: Map-Reduce Framework >> 13/12/08 10:44:04 INFO mapred.JobClient: Map output materialized >> bytes=3059 >> 13/12/08 10:44:04 INFO mapred.JobClient: Map input records=6 >> 13/12/08 10:44:04 INFO mapred.JobClient: Reduce shuffle bytes=3059 >> 13/12/08 10:44:04 INFO mapred.JobClient: Spilled Records=544 >> 13/12/08 10:44:04 INFO mapred.JobClient: Map output bytes=2509 >> 13/12/08 10:44:04 INFO mapred.JobClient: Total committed heap usage >> (bytes)=131600384 >> 13/12/08 10:44:04 INFO mapred.JobClient: CPU time spent (ms)=1010 >> 13/12/08 10:44:04 INFO mapred.JobClient: Combine input records=0 >> 13/12/08 10:44:04 INFO mapred.JobClient: SPLIT_RAW_BYTES=120 >> 13/12/08 10:44:04 INFO mapred.JobClient: Reduce input records=272 >> 13/12/08 10:44:04 INFO mapred.JobClient: Reduce input groups=138 >> 13/12/08 10:44:04 INFO mapred.JobClient: Combine output records=0 >> 13/12/08 10:44:04 INFO mapred.JobClient: Physical memory (bytes) >> snapshot=182116352 >> 13/12/08 10:44:04 INFO mapred.JobClient: Reduce output records=138 >> 13/12/08 10:44:04 INFO mapred.JobClient: Virtual memory (bytes) >> snapshot=696483840 >> 13/12/08 10:44:04 INFO mapred.JobClient: Map output records=272 >> hduser@ubuntu:~/Downloads$ >> >> >> >> mapred-site.xml file is given below. >> >> <?xml version="1.0"?> >> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?> >> >> <!-- Put site-specific property overrides in this file. --> >> >> <configuration> >> <property> >> <name>mapred.job.tracker</name> >> <value>localhost:54311</value> >> <description>The host and port that the MapReduce job tracker runs >> at. If "local", then jobs are run in-process as a single map >> and reduce task. >> </description> >> </property> >> <property> >> <name>mapreduce.shuffle.port</name> >> <value>localhost:13562</value> >> <description>To Monitor traffic on this port >> </description> >> </property> >> </configuration> >> >> >> >> >> I tried to run again after changing the port number to 54382 still no >> output. Can anybody tell me why I am not able to monitor traffic? Thanks in >> advance. >> >> >> Thanks & Regards >> Abdul Navaz >> >> >> On Fri, Dec 6, 2013 at 12:39 PM, Lars Nilsson <[email protected]>wrote: >> >>> On Dec 6, 2013 11:08 AM, "navaz" <[email protected]> wrote: >>> > I have tried tcpdump using "tcpdump port 13562" but not succeeded. >>> Could you please tell me am i following the step correctly. >>> > >>> > hduser@ubuntu:~/Downloads$ tcpdump port 13562 >>> > tcpdump: no suitable device found >>> >>> You are probably missing root privileges. Try sudo if you have it on the >>> machine in question, otherwise ask your sysadmin. >>> >>> Lars Nilsson >>> >> >> >> >> -- >> *Abdul Navaz* >> *Masters in Network Communications* >> *University of Houston* >> *Houston, TX - 77204-4020* >> *Ph - 281-685-0388 <281-685-0388>* >> *[email protected]* <[email protected]> >> >> > -- *Abdul Navaz* *Masters in Network Communications* *University of Houston* *Houston, TX - 77204-4020* *Ph - 281-685-0388* *[email protected]* <[email protected]>
