Thank you all for the comments. The client that we expect to call this API with thousands of network-ids is nova-scheduler.
Since this call is happening in the middle of scheduling, we don't want to spend time in paginating or sending multiple requests. I have tens of thousands of networks and subnets in my test cluster right now and with that scale, the extension takes more than 2 seconds to return. With multiple calls, scheduler will become very slow. I agree that sending payload with GET is not recommended and most libraries just drop the payload for such cases. On Wed, Jan 20, 2016 at 2:27 PM, Salvatore Orlando <[email protected]> wrote: > I tend to agree with Doug and Ryan's stance. If you need to pass 1000s of > network-id on a single request you're probably not doing things right on > the client side. > As Ryan suggested you can try and split the request in multiple requests > with acceptable URI lenght and send them in parallel; this will add some > overhead, but should work flawlessly. > > Once tags will be implemented you will be able to leverage those to > simplify your queries. > > Regarding GET requests with plenty of parameters, this discussion came up > on the mailing list a while ago [1]. A good proposal was made in that > thread but never formalised as an API-wg guideline; you consider submitting > a patch to the API-wg too. > Note however that Neutron won't be able to support it out of the box > considering its WSGI framework completely ignores request bodies on GET > requests. > > Salvatore > > [1] > http://lists.openstack.org/pipermail/openstack-dev/2015-November/078243.html > > On 20 January 2016 at 12:33, Ryan Brown <[email protected]> wrote: > >> So having a URI too long error is, in this case, likely an indication >> that you're requesting too many things at once. >> >> You could: >> 1. Request 100 at a time in parallel >> 2. Find a query that would give you all those networks & page through the >> reply >> 3. Page through all the user's networks and filter client-side >> >> How is the user supposed to be assembling this giant UUID list? I'd think >> it would be easier for them to specify a query (e.g. "get usage data for >> all my production subnets" or something). >> >> >> On 01/19/2016 06:59 PM, Shraddha Pandhe wrote: >> >>> Hi folks, >>> >>> >>> I am writing a Neutron extension which needs to take 1000s of >>> network-ids as argument for filtering. The CURL call is as follows: >>> >>> curl -i -X GET >>> 'http://hostname:port >>> /neutron/v2.0/extension_name.json?net-id=fffecbd1-0f6d-4f02-aee7-ca62094830f5&net-id=fffeee07-4f94-4cff-bf8e-a2aa7be59e2e' >>> -H "User-Agent: python-neutronclient" -H "Accept: application/json" -H >>> "X-Auth-Token: cccccccccccccccccccccccccccccccc" >>> >>> >>> The list of net-ids can go up to 1000s. The problem is, with such large >>> url, I get the "Request URI too long" error. I don't want to update this >>> limit as proxies can have their own limits. >>> >>> What options do I have to send 1000s of network IDs? >>> >>> 1. -d '{}' is not a recommended option for GET call and wsgi Controller >>> drops the data part when routing the request. >>> >>> 2. Use POST instead of GET? I will need to write the get_<resource> >>> logic inside create_resource logic for this to work. Its a hack, but >>> complies with HTTP standard. >>> >>> >>> >>> >>> __________________________________________________________________________ >>> OpenStack Development Mailing List (not for usage questions) >>> Unsubscribe: >>> [email protected]?subject:unsubscribe >>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >>> >>> >> -- >> Ryan Brown / Senior Software Engineer, Openstack / Red Hat, Inc. >> >> >> __________________________________________________________________________ >> OpenStack Development Mailing List (not for usage questions) >> Unsubscribe: >> [email protected]?subject:unsubscribe >> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev >> > > > __________________________________________________________________________ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: [email protected]?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > >
__________________________________________________________________________ OpenStack Development Mailing List (not for usage questions) Unsubscribe: [email protected]?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
