Re: [ovirt-users] ovirt sdk and pipelining
> Le 16 juin 2017 à 10:13, Juan Hernández a écrit : > > On 06/16/2017 09:52 AM, Fabrice Bacchella wrote: >> I just read the blog entry about performance increate in for the python sdk >> (https://www.ovirt.org/blog/2017/05/higher-performance-for-python-sdk/). >> >> I'm quite sceptical about pipelining. > In our tests pipe-lining dramatically increases the performance in large > scale environments with high latency. In our tests with 4000 virtual > machines 1 disks and 150ms of latency retrieving the complete > inventory is reduced from approx 1 hour to approx 2 minutes. > Bench are the ultimate judge. So if it works in many different use case for , that's nice and intersting. > Note that the usage of the HTTP protocol in this scenario is very > different from the typical usage when a browser retrieves a web page. Indeed, all the literature is about interactive usage. A very different use case. > >> It also talks about multiple connection, but don't use pycurl.CurlShare(). I >> thing this might be very helpfull, as it allows to share cookies, see >> https://curl.haxx.se/libcurl/c/CURLOPT_SHARE.html. >> > > The SDK uses the curl "multi" mechanism, which automatically shares the > DNS cache. This: https://curl.haxx.se/libcurl/c/CURLOPT_DNS_USE_GLOBAL_CACHE.html ? WARNING: this option is considered obsolete. Stop using it. Switch over to using the share interface instead! See CURLOPT_SHARE and curl_share_init. > In addition version 4 of the SDK does not use cookies. So > this shouldn't be relevant. From some of my own code: self._share.setopt(pycurl.SH_SHARE, pycurl.LOCK_DATA_COOKIE) self._share.setopt(pycurl.SH_SHARE, pycurl.LOCK_DATA_DNS) self._share.setopt(pycurl.SH_SHARE, pycurl.LOCK_DATA_SSL_SESSION) And users apaches settings can use cookies for custom usages. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt sdk and pipelining
El 2017-06-16 08:52, Fabrice Bacchella escribió: I just read the blog entry about performance increate in for the python sdk (https://www.ovirt.org/blog/2017/05/higher-performance-for-python-sdk/). I'm quite sceptical about pipelining. I disagree. Even without reading the post you mention, we already noticed that since this version everything is working much faster than with prior versions. We have a lot of stuff implemented with Python-SDK but on one of them the effect is quite notorious: A script checks VMs' permissions and takes decisions based on them. Without pipelining this script took about 5 minutes to execute, with pipelining it doesn't take more than 15 seconds on a ~1000VMs infrastructure. Regards, Nicolás A few explanation about that can be found at: https://devcentral.f5.com/articles/http-pipelining-a-security-risk-without-real-performance-benefits https://stackoverflow.com/questions/14810890/what-are-the-disadvantages-of-using-http-pipelining It also talks about multiple connection, but don't use pycurl.CurlShare(). I thing this might be very helpfull, as it allows to share cookies, see https://curl.haxx.se/libcurl/c/CURLOPT_SHARE.html. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Re: [ovirt-users] ovirt sdk and pipelining
On 06/16/2017 09:52 AM, Fabrice Bacchella wrote: > I just read the blog entry about performance increate in for the python sdk > (https://www.ovirt.org/blog/2017/05/higher-performance-for-python-sdk/). > > I'm quite sceptical about pipelining. > > A few explanation about that can be found at: > https://devcentral.f5.com/articles/http-pipelining-a-security-risk-without-real-performance-benefits > https://stackoverflow.com/questions/14810890/what-are-the-disadvantages-of-using-http-pipelining > Did you test it? Can you share the results? In our tests pipe-lining dramatically increases the performance in large scale environments with high latency. In our tests with 4000 virtual machines 1 disks and 150ms of latency retrieving the complete inventory is reduced from approx 1 hour to approx 2 minutes. Note that the usage of the HTTP protocol in this scenario is very different from the typical usage when a browser retrieves a web page. > It also talks about multiple connection, but don't use pycurl.CurlShare(). I > thing this might be very helpfull, as it allows to share cookies, see > https://curl.haxx.se/libcurl/c/CURLOPT_SHARE.html. > The SDK uses the curl "multi" mechanism, which automatically shares the DNS cache. In addition version 4 of the SDK does not use cookies. So this shouldn't be relevant. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
[ovirt-users] ovirt sdk and pipelining
I just read the blog entry about performance increate in for the python sdk (https://www.ovirt.org/blog/2017/05/higher-performance-for-python-sdk/). I'm quite sceptical about pipelining. A few explanation about that can be found at: https://devcentral.f5.com/articles/http-pipelining-a-security-risk-without-real-performance-benefits https://stackoverflow.com/questions/14810890/what-are-the-disadvantages-of-using-http-pipelining It also talks about multiple connection, but don't use pycurl.CurlShare(). I thing this might be very helpfull, as it allows to share cookies, see https://curl.haxx.se/libcurl/c/CURLOPT_SHARE.html. ___ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users