Are you running this on SSD? 2941 seems like a lot, given that for
each write you need to fsync to disk, which takes at least 8ms.
On 17 October 2014 17:42, Ivan Kelly <i...@ivankelly.net> wrote:
> Are you running this on SSD? 2941 seems like a lot, given that for
> each write you need to fsync to disk, which takes at least 8ms.
>
> On 17 October 2014 17:28, sky_cord <sky_c...@163.com> wrote:
>> Thanks for your help.
>> After adding the two config items into bk_server.conf :
>>
>> journalFlushWhenQueueEmpty=true
>> journalMaxGroupWaitMSec=0
>>
>> The write speed rise up to 800-900 entries per second in one ledger, length
>> of each entry is 50 bytes.
>>
>> But it's still lower than I expected（The sync write speed is 2941 entries/s
>> based onbookkeepr-4.2.2）.
>>
>> Could you describe your test result? And Is there any way to improve write
>> performance ?
>>
>> Thank you.
>>
>>
>>
>>
>> At 2014-10-17 20:13:07, "Ivan Kelly" <iv...@apache.org> wrote:
>>>Ah, I know what this is. Twitter added some changes that makes write
>>>throughput much higher when there's a high write rate. In this case
>>>the latency for a single writes goes go up though (as it's doing more
>>>buffering on the journal). Try setting:
>>>
>>>journalFlushWhenQueueEmpty=true
>>>journalMaxGroupWaitMSec=0
>>>
>>>in bk_server.conf
>>>
>>>
>>>Also, the journal and ledger directories are on the same disk. This
>>>will have a detrimental affect once the write buffers start flushing.
>>>We always recommend that you put the journal on its own independent
>>>disk.
>>>
>>>The reads are fast because it doesn't need to hit disk, as there's not
>>>much data that has been written.
>>>
>>>-Ivan
>>>
>>>On 17 October 2014 13:32, sky_c...@163.com <sky_c...@163.com> wrote:
>>>> The write speed of bookkeper that version less than 4.30 is bigger than
>>>> 1200
>>>> entries per second when i use the same test code
>>>>
>>>> ________________________________
>>>> sky_c...@163.com
>>>>
>>>>
>>>> From: Jiannan Wang
>>>> Date: 2014-10-17 18:31
>>>> To: sky_c...@163.com
>>>> CC: bookkeeper-user@zookeeper.apache.org
>>>> Subject: Re: hi, why write speed of bookkeeper-4.3.0 is very slow?
>>>> Could you share more information:
>>>> - What's the number of write_num?
>>>> - What's the read throughput under your test?
>>>> - Does the 5 bookie servers run in same machine?
>>>> - Could you check whether StringUtil.div works well?
>>>> - Could you also try async write API and tell us the output?
>>>>
>>>> Regards,
>>>> Jiannan
>>>>
>>>> From: "sky_c...@163.com" <sky_c...@163.com>
>>>> Reply-To: "bookkeeper-user@zookeeper.apache.org"
>>>> <bookkeeper-user@zookeeper.apache.org>
>>>> Date: Friday, October 17, 2014 at 3:38 PM
>>>> To: bookkeeper-user <bookkeeper-user@zookeeper.apache.org>
>>>> Subject: hi, why write speed of bookkeeper-4.3.0 is very slow?
>>>>
>>>> hi,
>>>> I have tried bookkeeper-4.3.0 today. But its write speed is 4
>>>> entries
>>>> per sec in my environment.
>>>> There are 5 bookkeeper-server-4.3.0 in my env.
>>>> Is there anything wrong? Test codes as follows:
>>>> public void syncWriteLedger(long write_num) throws InterruptedException,
>>>> BKException {
>>>> byte[] pwd = "foobar".getBytes();
>>>> LedgerHandle lh = client.createLedger(3, 2,
>>>> DigestType.MAC, pwd);
>>>>
>>>> long ledgerId = lh.getId();
>>>> System.out.println("ledgerId=" + ledgerId);
>>>> System.out
>>>> .println("data_len="
>>>> + ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" + write_num)
>>>> .getBytes().length);
>>>> byte[] data = ("Hello World!afdafdafdfxtyb457u8n9,7.,pbesgcerfqx" +
>>>> write_num).getBytes();
>>>> long st = System.currentTimeMillis();
>>>> for (int i = 0; i < write_num; i++) {
>>>> lh.addEntry(data);
>>>> }
>>>> long ut = System.currentTimeMillis() - st;
>>>> lh.close();
>>>> double uts = StringUtil.div(ut, 1000, 3);
>>>> double wspeed = StringUtil.div(write_num, uts, 2);
>>>> System.out.println("ut=" + ut + " ms, write_num=" + write_num
>>>> + ", write speed=" + wspeed + "/s");
>>>>
>>>>
>>>> LedgerHandle lh2 = client.openLedger(ledgerId, BookKeeper.DigestType.MAC,
>>>> pwd);
>>>> long lastEntry = lh2.getLastAddConfirmed();
>>>> st = System.currentTimeMillis();
>>>> Enumeration<LedgerEntry> entries = lh2.readEntries(0, lastEntry);
>>>> while (entries.hasMoreElements()) {
>>>> byte[] bytes = entries.nextElement().getEntry();
>>>> System.out.println(new String(bytes));
>>>> }
>>>> ut = System.currentTimeMillis() - st;
>>>> lh2.close();
>>>> uts = StringUtil.div(ut, 1000, 3);
>>>> double rspeed = StringUtil.div(write_num, uts, 2);
>>>> System.out.println("ut=" + ut + " ms, read_num=" + write_num
>>>> + ", read speed=" + rspeed + "/s");
>>>>
>>>> }
>>>> ________________________________
>>>> sky_c...@163.com
>>
>>
>>
