On Apr 30 2015, Maxim Kostrikin <[email protected]> wrote:
> среда, 29 апреля 2015 г., 2:55:05 UTC+6 пользователь Nikolaus Rath написал:
>> On Apr 28 2015, Maxim Kostrikin <[email protected] <javascript:>> wrote:
>> > I am faced with performance issue. My s3ql mount attached to s3 backend
>> > with options: --cachesize 20000000 --max-cache-entries 1000000 --threads
>> > 16
>> > --allow-other --backend-options no-ssl,tcp-timeout=30 --nfs
>> > I copy into s3ql mount many small files 100 - 1000 bytes. With time copy
>> > speed slowdown almost stall.
>>
>> Please be more precise. How did you measure the speed? How fast was it
>> initially, how fast at the end?
>>
> I was transfering files with tar c|dd|ssh s3qluser@s3ql_host tar xC
> /s3ql_mount kill -USR1 $dd_pid shows the speed. It starts from 5M/sec
> and falls to 5k/s or even freeze.
Well, did it fall to 5 k/s or did it freeze?
> Now I tar-pipe into temp directory and from the temp directory move into
> /s3ql_mount
>
>> I see almost 16 connections with s3 servers, but tcpdump shows small
>> rate
>> > of uploading PUTs.
>>
>> What do you mean with small? What do you expect it to be, and what did
>> you find instead?
>>
> 200 bytes - 1k per file.
> The dirty size was big (1-2GiB).
No, I was talking about the small rate of PUTs that you saw in tcpdump.
If we assume about 100 ms network latency, then with 200 byte files
you'd be able to transfer at most 200 bytes / 100 ms = 2 kB/s for each
active thread. So your 5 kb/s isn't completely the wrong order of
magnitude.
I would expect that the initial fast period is just when you file up
your cache. Am I right that, if you set the cache to something small
(say 2 * <number of threads> entries), you're getting a constant
transfer rate from the very beginning?
In tcpdump, how many PUTs do you see per second? Is that number constant?
>>> Ok, I remounted with --threads 1024 ( I bet it was a bad idea), but
>>> connections with s3 was 24 max and lots of cross locks between
>>> threads.
>>
>> How did you determine that?
>
> strace -fp $mount.s3ql_pid showed many FUTEX messages.
I think that doesn't mean that there's lock contention, just that locks
are being used.
Best,
-Nikolaus
--
GPG encrypted emails preferred. Key id: 0xD113FCAC3C4E599F
Fingerprint: ED31 791B 2C5C 1613 AF38 8B8A D113 FCAC 3C4E 599F
»Time flies like an arrow, fruit flies like a Banana.«
--
You received this message because you are subscribed to the Google Groups
"s3ql" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
For more options, visit https://groups.google.com/d/optout.
