Hi Andrew, Any update on this?
-- Thanks -- Kishore On 29/05/17, 1:06 PM, "Battula Kishore" <batt...@adobe.com.INVALID> wrote: >Hi Andrew, > >Thanks andrew for the quick response. Here is the GitHub repo and instructions >on how to run the tests >https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkishore25kumar%2Fs3proxy-async-test-setup&data=02%7C01%7C%7C8969ed65c6a749aebfa608d4a66583cc%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636316402308126519&sdata=0YESeCSlmChUOicpe2m9SYC1WaB7m%2B7hIB6o%2BTOp1Jo%3D&reserved=0. > The READE.md also has the repo details of s3proxy as well as jclouds >implementation. Hope this helps? Let me know if you need anything else? > >In the mean time you review these results if you can let me know the design >review process I can be prepared for that. > >-- Thanks >-- Kishore > > > > > > > > > >On 26/05/17, 12:40 PM, "Andrew Gaul" <g...@apache.org> wrote: > >>Kishore, these are promising results! I reformatted the most important >>rows which show a 2x improvement in throughput and latency: >> >>10 10,000 Async Http Lib 209 282 48 >>10 10,000 OutputStream 392 542 25 >> >>Can you share the implementation and include instructions on how to >>replicate these tests? >> >>On Wed, May 24, 2017 at 05:50:52AM +0000, Battula Kishore wrote: >>> Hi, >>> >>> This is Kishore who is working on async poc using mail >>> id(kishore25ku...@gmail.com<mailto:kishore25ku...@gmail.com>). I work at >>> adobe and we wanted to implement async support for jclouds library and >>> contribute it back. >>> >>> From the last discussion I was asked to get the performance numbers for the >>> two approaches. >>> Approach 1: Using Http Async Library >>> Approach 2: Using Outputstream >>> >>> Test setup: >>> >>> 1. Both the s3 proxy server and test runner are running in same >>> Docker container in azure west-us region. >>> >>> 2. Azure storage account is also residing in same west-us region. >>> >>> 3. A bucket is prepopulated with 100,000 files, each file of 1 MB >>> size before test start. >>> >>> 4. The test runner sends unique requests to s3proxy to download files. >>> >>> Virtual Machine spec: CPU - 8 cores, Memory - 28 GB (Standard_D4 Azure >>> machine) >>> >>> S3proxy is running with 1 jetty worker thread in all the scenarios. The >>> payload size used is 1 MB file. Here are the performance numbers. >>> Test Runner Threads >>> >>> Iteration Per thread >>> >>> Approach >>> >>> Avg response time (ms) >>> >>> 99%tile time (ms) >>> >>> Throughput >>> (Requests / sec) >>> >>> 1 >>> >>> 10,000 >>> >>> Async Http Lib >>> >>> 45 >>> >>> 87 >>> >>> 22 >>> >>> 5 >>> >>> 10,000 >>> >>> Async Http Lib >>> >>> 107 >>> >>> 159 >>> >>> 47 >>> >>> 10 >>> >>> 10,000 >>> >>> Async Http Lib >>> >>> 209 >>> >>> 282 >>> >>> 48 >>> >>> 1 >>> >>> 10,000 >>> >>> OutputStream >>> >>> 41 >>> >>> 85 >>> >>> 24 >>> >>> 5 >>> >>> 10,000 >>> >>> OutputStream >>> >>> 190 >>> >>> 283 >>> >>> 26 >>> >>> 10 >>> >>> 10,000 >>> >>> OutputStream >>> >>> 392 >>> >>> 542 >>> >>> 25 >>> >>> >>> Summary: Under load Http Async Library approach is providing more >>> throughput compared to Output stream approach. >>> >>> >>> Both the approaches improve performance. The output stream approach can be >>> used along with Http Async library approach which is giving around (3-5 ms) >>> improvement in latency. >>> >>> Each approach is independent development. At this point I am keen to take >>> up Http Async Library development. >>> >>> >>> -- Thanks >>> -- Kishore >>> >> >>-- >>Andrew Gaul >>https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgaul.org%2F&data=02%7C01%7C%7C8690ee3e00bc4e3da0e608d4a406565e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636313794511467457&sdata=XQ%2FshVjdqC3KiVEuyH6%2FJvmDN5DHBmS0kIBx98V89KY%3D&reserved=0
