For now I have changed mount.s3ql options to 256 threads. And combined small files where possible to bigger one. 256 threads works just great. Simultaneous connection jumps up-to 30, but dirty arbitrary small 10M or less. I guess there are just nothing to push faster.
rest of reply inline below: среда, 29 апреля 2015 г., 2:55:05 UTC+6 пользователь Nikolaus Rath написал: > > On Apr 28 2015, Maxim Kostrikin <[email protected] <javascript:>> wrote: > > Hello, > > I am faced with performance issue. My s3ql mount attached to s3 > backend > > with options: --cachesize 20000000 --max-cache-entries 1000000 --threads > 16 > > --allow-other --backend-options no-ssl,tcp-timeout=30 --nfs > > I copy into s3ql mount many small files 100 - 1000 bytes. With time > copy > > speed slowdown almost stall. > > Please be more precise. How did you measure the speed? How fast was it > initially, how fast at the end? > > I was transfering files with tar c|dd|ssh s3qluser@s3ql_host tar xC /s3ql_mount kill -USR1 $dd_pid shows the speed. It starts from 5M/sec and falls to 5k/s or even freeze Now I tar-pipe into temp directory and from the temp directory move into /s3ql_mount > I see almost 16 connections with s3 servers, but tcpdump shows small > rate > > of uploading PUTs. > > What do you mean with small? What do you expect it to be, and what did > you find instead? > 200 bytes - 1k per file. The dirty size was big (1-2GiB). > > Generally every file needs one network request, which means tens to > hundreds of milliseconds of network latency per file. > It is clear. The CPU was not loaded much ( top shows lots of idle ), and netstat shows small amount of connections to s3 And tcpdump shows 1 -2 trunsaction ( if I filter all connection at once - tcpdump -AAAAni eth0 net 54.0.0.0.0/8 ) > > Ok, I remounted with --threads 1024 ( I bet it was a bad idea), but > > connections with s3 was 24 max and lots of cross locks between > > threads. > > How did you determine that? strace -fp $mount.s3ql_pid showed many FUTEX messages. > > > > 1. How to increase parallels request to s3? Or increase upload > > performance? > > In theory more threads should give you more simulateous connection - > unless you're limited by something else. > We have no limits as far as I know. > > > 2. What limits could be reasonable for --threads parameter? Why? > SQLite > > bottleneck? > > Difficult to say without further information. > > I hope there are enough infomation. -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
