[
https://issues.apache.org/jira/browse/KAFKA-4971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15955219#comment-15955219
]
Dasol Kim commented on KAFKA-4971:
----------------------------------
I was experimenting with the kafka producer installed on the SSD by sending
messages to one broker installed on another ssd server in throughput / sec. In
the same way, the experiment was conducted using the kafka of the HDD installed
server. According to the known facts, SSD experiments should show more
throughput per second.
test settings
OS : CentOs 7, Memory : 1G, Disk : 8G
it's my test result and command
VM - SSD(producer 1, broker 1, partition 1)
./bin/kafka-producer-perf-test.sh --num-records 1000000 --record-size 2000
--topic test --throughput 100000 --producer-props bootstrap.servers=SN02:9092
9169 records sent, 1833.8 records/sec (3.50 MB/sec), 2393.7 ms avg latency,
3621.0 max latency.
17840 records sent, 3566.6 records/sec (6.80 MB/sec), 4412.3 ms avg latency,
5282.0 max latency.
23568 records sent, 4704.2 records/sec (8.97 MB/sec), 4391.3 ms avg latency,
5132.0 max latency.
32872 records sent, 6574.4 records/sec (12.54 MB/sec), 2612.5 ms avg latency,
3055.0 max latency.
39352 records sent, 7870.4 records/sec (15.01 MB/sec), 2001.6 ms avg latency,
2644.0 max latency.
41168 records sent, 8228.7 records/sec (15.69 MB/sec), 2003.0 ms avg latency,
2585.0 max latency.
40568 records sent, 8105.5 records/sec (15.46 MB/sec), 2113.5 ms avg latency,
2374.0 max latency.
65528 records sent, 13105.6 records/sec (25.00 MB/sec), 1351.3 ms avg latency,
1778.0 max latency.
108096 records sent, 21619.2 records/sec (41.24 MB/sec), 780.2 ms avg latency,
1026.0 max latency.
79992 records sent, 15988.8 records/sec (30.50 MB/sec), 855.3 ms avg latency,
2238.0 max latency.
31152 records sent, 6230.4 records/sec (11.88 MB/sec), 2651.9 ms avg latency,
3180.0 max latency.
39520 records sent, 7899.3 records/sec (15.07 MB/sec), 1820.1 ms avg latency,
2536.0 max latency.
52824 records sent, 10564.8 records/sec (20.15 MB/sec), 1942.5 ms avg latency,
3243.0 max latency.
68912 records sent, 13760.4 records/sec (26.25 MB/sec), 1122.5 ms avg latency,
1678.0 max latency.
93024 records sent, 18601.1 records/sec (35.48 MB/sec), 954.2 ms avg latency,
1624.0 max latency.
80200 records sent, 16040.0 records/sec (30.59 MB/sec), 981.6 ms avg latency,
1417.0 max latency.
101720 records sent, 20344.0 records/sec (38.80 MB/sec), 829.8 ms avg latency,
1201.0 max latency.
73272 records sent, 14654.4 records/sec (27.95 MB/sec), 1076.5 ms avg latency,
1487.0 max latency.
1000000 records sent, 11094.223238 records/sec (21.16 MB/sec), 1444.94 ms avg
latency, 5282.00 ms max latency, 1181 ms 50th, 3116 ms 95th, 4794 ms 99th, 5158
ms 99.9th.
VM - HDD(producer 1, broker 1, partition 1)
./bin/kafka-producer-perf-test.sh --num-records 1000000 --record-size 2000
--topic test --throughput 100000 --producer-props bootstrap.servers=SN03:9092
11145 records sent, 2228.6 records/sec (4.25 MB/sec), 2209.9 ms avg latency,
3442.0 max latency.
19592 records sent, 3912.9 records/sec (7.46 MB/sec), 4165.9 ms avg latency,
4661.0 max latency.
18472 records sent, 3694.4 records/sec (7.05 MB/sec), 4416.4 ms avg latency,
4598.0 max latency.
33312 records sent, 6662.4 records/sec (12.71 MB/sec), 2862.6 ms avg latency,
4366.0 max latency.
49000 records sent, 9782.4 records/sec (18.66 MB/sec), 1923.6 ms avg latency,
2673.0 max latency.
41856 records sent, 8357.8 records/sec (15.94 MB/sec), 1760.9 ms avg latency,
2241.0 max latency.
48032 records sent, 9602.6 records/sec (18.32 MB/sec), 1863.5 ms avg latency,
2283.0 max latency.
78032 records sent, 15606.4 records/sec (29.77 MB/sec), 1096.7 ms avg latency,
1364.0 max latency.
93440 records sent, 18688.0 records/sec (35.64 MB/sec), 833.6 ms avg latency,
1299.0 max latency.
72184 records sent, 14436.8 records/sec (27.54 MB/sec), 1185.9 ms avg latency,
1421.0 max latency.
80352 records sent, 16070.4 records/sec (30.65 MB/sec), 955.9 ms avg latency,
1896.0 max latency.
64200 records sent, 12840.0 records/sec (24.49 MB/sec), 1319.0 ms avg latency,
1652.0 max latency.
86400 records sent, 17280.0 records/sec (32.96 MB/sec), 972.3 ms avg latency,
1292.0 max latency.
74472 records sent, 14894.4 records/sec (28.41 MB/sec), 1073.2 ms avg latency,
1224.0 max latency.
75912 records sent, 15182.4 records/sec (28.96 MB/sec), 961.6 ms avg latency,
1901.0 max latency.
30088 records sent, 6017.6 records/sec (11.48 MB/sec), 2746.6 ms avg latency,
3482.0 max latency.
70368 records sent, 14073.6 records/sec (26.84 MB/sec), 1257.5 ms avg latency,
2300.0 max latency.
1000000 records sent, 11269.129347 records/sec (21.49 MB/sec), 1423.68 ms avg
latency, 4661.00 ms max latency, 1179 ms 50th, 3467 ms 95th, 4460 ms 99th, 4597
ms 99.9th.
Comparing the two experimental results, we can see that the results of the two
experiments show little difference. I do not know what this is about.
please let me know what happened.
> Why is there no difference between kafka benchmark tests on SSD and HDD?
> -------------------------------------------------------------------------
>
> Key: KAFKA-4971
> URL: https://issues.apache.org/jira/browse/KAFKA-4971
> Project: Kafka
> Issue Type: Test
> Affects Versions: 0.10.0.0
> Environment: Oracle VM VirtualBox
> OS : CentOs 7
> Memory : 1G
> Disk : 8GB
> Reporter: Dasol Kim
>
> I installed OS and kafka in the two SSD and two HDDs to perform the kafka
> benchmark test based on the disc difference. As expected, the SSD should show
> faster results, but according to my experimental results, there is no big
> difference between SSD and HDD. why? Ohter settings have been set to default.
> *test settings
> zookeeper node : 1, producer node : 2, broker node : 2(SSD 1, HDD 1)
> test scenario : Two producers send messages to the broker and compare the
> throughtput per second of kafka installed on SSD and kafka on HDD
> command : ./bin/kafka-producer-perf-test.sh --num-records 1000000
> --record-size 2000 --topic test --throughput 100000 --producer-props
> bootstrap.servers=SN02:9092
>
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
