Hi Kishore
Andrew G. is travelling for most of June, as far as I understand, so
will likely be a bit slower to respond. Thanks for your patience!
ap
On 2017-06-05 23:42, Battula Kishore wrote:
Hi Andrew,
Any update on this?
-- Thanks
-- Kishore
On 29/05/17, 1:06 PM, "Battula Kishore" <batt...@adobe.com.INVALID>
wrote:
Hi Andrew,
Thanks andrew for the quick response. Here is the GitHub repo and
instructions on how to run the tests
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fkishore25kumar%2Fs3proxy-async-test-setup&data=02%7C01%7C%7C8969ed65c6a749aebfa608d4a66583cc%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636316402308126519&sdata=0YESeCSlmChUOicpe2m9SYC1WaB7m%2B7hIB6o%2BTOp1Jo%3D&reserved=0.
The READE.md also has the repo details of s3proxy as well as jclouds
implementation. Hope this helps? Let me know if you need anything
else?
In the mean time you review these results if you can let me know the
design review process I can be prepared for that.
-- Thanks
-- Kishore
On 26/05/17, 12:40 PM, "Andrew Gaul" <g...@apache.org> wrote:
Kishore, these are promising results! I reformatted the most
important
rows which show a 2x improvement in throughput and latency:
10 10,000 Async Http Lib 209 282 48
10 10,000 OutputStream 392 542 25
Can you share the implementation and include instructions on how to
replicate these tests?
On Wed, May 24, 2017 at 05:50:52AM +0000, Battula Kishore wrote:
Hi,
This is Kishore who is working on async poc using mail
id(kishore25ku...@gmail.com<mailto:kishore25ku...@gmail.com>). I
work at adobe and we wanted to implement async support for jclouds
library and contribute it back.
From the last discussion I was asked to get the performance numbers
for the two approaches.
Approach 1: Using Http Async Library
Approach 2: Using Outputstream
Test setup:
1. Both the s3 proxy server and test runner are running in
same Docker container in azure west-us region.
2. Azure storage account is also residing in same west-us
region.
3. A bucket is prepopulated with 100,000 files, each file of 1
MB size before test start.
4. The test runner sends unique requests to s3proxy to
download files.
Virtual Machine spec: CPU - 8 cores, Memory - 28 GB (Standard_D4
Azure machine)
S3proxy is running with 1 jetty worker thread in all the scenarios.
The payload size used is 1 MB file. Here are the performance
numbers.
Test Runner Threads
Iteration Per thread
Approach
Avg response time (ms)
99%tile time (ms)
Throughput
(Requests / sec)
1
10,000
Async Http Lib
45
87
22
5
10,000
Async Http Lib
107
159
47
10
10,000
Async Http Lib
209
282
48
1
10,000
OutputStream
41
85
24
5
10,000
OutputStream
190
283
26
10
10,000
OutputStream
392
542
25
Summary: Under load Http Async Library approach is providing more
throughput compared to Output stream approach.
Both the approaches improve performance. The output stream approach
can be used along with Http Async library approach which is giving
around (3-5 ms) improvement in latency.
Each approach is independent development. At this point I am keen to
take up Http Async Library development.
-- Thanks
-- Kishore
--
Andrew Gaul
https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fgaul.org%2F&data=02%7C01%7C%7C8690ee3e00bc4e3da0e608d4a406565e%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C636313794511467457&sdata=XQ%2FshVjdqC3KiVEuyH6%2FJvmDN5DHBmS0kIBx98V89KY%3D&reserved=0
