Min, Interesting. Could you file a jira and attach your producer code and dstat output there?
Thanks, Jun On Mon, Jul 9, 2012 at 11:53 PM, Min <mini...@gmail.com> wrote: > Hello, > > I tested again and I could reproduce the situation. > > I run 0.7.0 with the following shell command > ./kafka-0.7.0-incubating-src/bin/kafka-server-start.sh > config/server.properties > then 0.7.1 > ./kafka-0.7.1-incubating/bin/kafka-server-start.sh config/server.properties > > configuration values in the server.properties are all default but the > ZK address. > > My producer and broker were running on different machines in same rack. > They are not VMs but physical machines. Also I don't feel some other > process interfere my test. > > > I'm attaching the dstat result and my producer code. > Producer is so simple that read files in a given directory by line and > send every 1000 lines to the broker. > > Thanks > Min > > ./kafka-0.7.0-incubating-src/bin/kafka-server-start.sh > config/server.properties > (Producer) > ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 23 2 72 2 0 1| 88M 896k| 98k 50M| 0 0 |2855 2627 > 20 1 77 1 0 1| 81M 1144k| 79k 48M| 0 0 |2858 2423 > 21 1 77 0 0 1| 85M 0 | 77k 47M| 0 0 |2655 2222 > 6 0 93 0 0 0| 24M 240k| 38k 15M| 0 0 |1664 1232 > 18 1 80 0 1 1| 71M 0 | 72k 42M| 0 0 |2657 2152 > 21 1 69 7 0 1| 75M 1704k| 92k 48M| 0 0 |2959 2872 > 20 2 52 25 0 1| 90M 512k| 82k 48M| 0 0 |2884 2940 > 20 1 78 0 0 0| 85M 0 | 77k 47M| 0 0 |2800 2459 > 22 2 75 0 1 1| 88M 320k| 81k 51M| 0 0 |2840 2590 > 24 1 74 0 1 0| 90M 0 | 88k 52M| 0 0 |2819 2526 > 23 1 71 2 0 1| 87M 1408k| 96k 51M| 0 0 |2924 2661 > 23 2 73 1 1 1| 89M 744k| 82k 51M| 0 0 |2851 2485 > 25 1 72 0 1 1| 94M 0 | 89k 55M| 0 0 |2941 2545 > 22 1 76 0 0 1| 88M 240k| 81k 50M| 0 0 |2804 2581 > 23 1 75 0 0 1| 93M 0 | 83k 53M| 0 0 |2868 2530 > 23 1 69 5 0 1| 87M 1400k| 100k 53M| 0 0 |2959 2668 > > (Broker 0.7.0) > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 4 3 86 4 0 2|2056k 197M| 51M 68k| 0 0 | 11k 1911 > 5 3 86 4 0 2|2048k 209M| 53M 72k| 0 0 | 11k 2097 > 5 3 85 4 0 2|2056k 207M| 52M 77k| 0 0 | 11k 2084 > 4 4 86 4 0 2|2048k 208M| 53M 71k| 0 0 | 11k 2101 > 5 3 86 4 0 2|2064k 201M| 51M 68k| 0 0 | 11k 2055 > 4 4 86 4 0 2|2048k 207M| 53M 71k| 0 0 | 11k 2007 > 5 4 86 4 0 2|2048k 209M| 53M 71k| 0 0 | 11k 2018 > 5 3 85 4 0 2|2056k 207M| 52M 76k| 0 0 | 11k 1997 > 4 4 85 5 0 2|2056k 197M| 50M 67k| 0 0 | 11k 1907 > 5 4 86 4 0 2|2056k 205M| 52M 71k| 0 0 | 11k 2053 > 5 3 86 4 0 2|2048k 205M| 52M 70k| 0 0 | 11k 2000 > > ./kafka-0.7.1-incubating/bin/kafka-server-start.sh config/server.properties > > (Producer) > ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 9 1 89 1 0 0| 41M 416k| 51k 27M| 0 0 |2205 1618 > 9 1 88 2 0 0| 49M 0 | 47k 27M| 0 0 |2212 1952 > 8 1 89 1 0 0| 44M 0 | 58k 26M| 0 0 |2201 1713 > 10 1 87 2 0 0| 47M 256k| 57k 26M| 0 0 |2199 1992 > 9 1 84 5 0 1| 46M 2488k| 56k 27M| 0 0 |2472 1924 > 9 1 89 1 0 0| 42M 384k| 53k 26M| 0 0 |2187 1730 > 9 1 89 1 0 0| 48M 0 | 44k 27M| 0 0 |2189 1691 > 10 1 88 1 0 0| 45M 0 | 58k 27M| 0 0 |2205 1884 > 9 1 89 1 0 0| 48M 224k| 58k 27M| 0 0 |2225 1715 > 9 1 76 12 0 1| 44M 2496k| 57k 26M| 0 0 |2425 2133 > 9 1 88 2 0 0| 45M 584k| 51k 26M| 0 0 |2194 1685 > 9 1 89 1 0 0| 46M 0 | 43k 25M| 0 0 |2166 1727 > 9 1 90 1 0 0| 47M 0 | 54k 25M| 0 0 |2183 1629 > > (Broker - 0.7.1) > ----total-cpu-usage---- -dsk/total- -net/total- ---paging-- ---system-- > usr sys idl wai hiq siq| read writ| recv send| in out | int csw > 9 2 86 2 0 1|2048k 103M| 26M 36k| 0 0 |6376 1849 > 8 2 86 2 0 1|2056k 104M| 26M 37k| 0 0 |6431 1945 > 9 2 86 2 0 1|2048k 101M| 26M 42k| 0 0 |6431 1884 > 9 2 87 2 0 1|2048k 104M| 26M 35k| 0 0 |6300 1561 > 9 2 86 2 0 1|2048k 104M| 27M 37k| 0 0 |6542 1543 > 9 2 86 2 0 1|2056k 104M| 26M 37k| 0 0 |6487 1638 > 9 2 87 2 0 1|2048k 101M| 26M 36k| 0 0 |6402 1721 > 9 2 86 2 0 1|2048k 104M| 26M 43k| 0 0 |6436 1762 > 9 2 86 2 0 1|2048k 106M| 27M 37k| 0 0 |6527 1799 > 9 2 86 3 0 1|2056k 101M| 26M 36k| 0 0 |6378 1521 > 9 2 86 2 0 1|2048k 104M| 27M 37k| 0 0 |6521 1568 > 9 2 87 2 0 1|2048k 104M| 26M 37k| 0 0 |6452 1708 > > > > > 2012/7/9 Jay Kreps <jay.kr...@gmail.com>: > > Not that is known to me. Is this something you can reproduce? Are the > > settings the same on the two brokers? Can you check iostat between the > two > > and see if the change is due to disk activity or cpu? > > > > -Jay > > > > On Sun, Jul 8, 2012 at 12:03 AM, Min <mini...@gmail.com> wrote: > > > >> Hello, > >> > >> Recently I've upgrade to 0.7.1. But 0.7.1 didn't show performance as > >> 0.7.0. > >> > >> With 0.7.0 I can push 45MBps to the broker. But I only can push 27MBps > >> with 0.7.1. > >> > >> Is there any a fetch to affect the performance? > >> > >> Thanks > >> >
