Jira (PDB-1251) Paging inconsistent across (parts of) an endpoint

[Daniele Sluijters (JIRA)]

Title: Message Title

Daniele Sluijters created an issue   PuppetDB / PDB-1251 Paging inconsistent across (parts of) an endpoint Issue Type: Bug Assignee: Unassigned Created: 2015/02/24 5:06 AM Priority: Normal Reporter: Daniele Sluijters While working on an update to pypuppetdb I wanted to implement paging of the results in a generic way, essentially leveraging a generator and not having the user care about what's happening behind the scenes. Most, if not all, endpoints support paging, but some subparts of endpoints don't. This is extremely confusing and makes implementing an API client that much more difficult since it's difficult to reason upfront about what the result of a query will be. As an example: curl -X GET -vvvv https://puppetdb:443/v4/nodes --data-urlencode 'include-total=true' --data-urlencode 'limit=2' --data-urlencode 'offset=0' * Hostname was NOT found in DNS cache * Trying 194.103.36.46... * Connected to puppetdb (194.103.36.46) port 443 (#0) * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 * Server certificate: * * Server certificate: DigiCert SHA2 Secure Server CA * Server certificate: DigiCert Global Root CA > GET /v4/nodes HTTP/1.1 > User-Agent: curl/7.37.1 > Host: puppetdb > Accept: */* > Content-Length: 35 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 35 out of 35 bytes < HTTP/1.1 200 OK < Date: Tue, 24 Feb 2015 13:03:25 GMT * Server Jetty(9.1.z-SNAPSHOT) is not blacklisted < Server: Jetty(9.1.z-SNAPSHOT) < Strict-Transport-Security: max-age=15768000 < Access-Control-Allow-Origin: * < Access-Control-Expose-Headers: X-Records < X-Records: 6392 < Content-Type: application/json; charset=utf-8 < Content-Length: 681 < Via: 1.1 lon3-puppetdb-a3.lon3 < Connection: close < [ { "catalog-environment" : "production", "catalog-timestamp" : "2015-02-24T12:46:56.170Z", "certname" : "aasia.ash", "deactivated" : null, "facts-environment" : "production", "facts-timestamp" : "2015-02-24T12:46:12.269Z", "report-environment" : "production", "report-timestamp" : "2015-02-24T12:48:07.653Z" }, { "catalog-environment" : "production", "catalog-timestamp" : "2015-02-24T12:44:04.724Z", "certname" : "abrianna.ash", "deactivated" : null, "facts-environment" : "production", "facts-timestamp" : "2015-02-24T12:43:33.103Z", "report-environment" : "production", "report-timestamp" : "2015-02-24T12:45:06.586Z" * Closing connection 0 } ]⏎ Now, lets try and fetch a single node: curl -X GET -vvvv https://puppetdb:443/v4/nodes/aasia.ash --data-urlencode 'include-total=true' --data-urlencode 'limit=2' --data-urlencode 'offset=0' * Hostname was NOT found in DNS cache * Trying 194.103.36.46... * Connected to puppetdb (194.103.36.46) port 443 (#0) * TLS 1.2 connection using TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 * Server certificate: * * Server certificate: DigiCert SHA2 Secure Server CA * Server certificate: DigiCert Global Root CA > GET /v4/nodes/aasia.ash HTTP/1.1 > User-Agent: curl/7.37.1 > Host: puppetdb > Accept: */* > Content-Length: 35 > Content-Type: application/x-www-form-urlencoded > * upload completely sent off: 35 out of 35 bytes < HTTP/1.1 400 Bad Request < Date: Tue, 24 Feb 2015 13:04:33 GMT * Server Jetty(9.1.z-SNAPSHOT) is not blacklisted < Server: Jetty(9.1.z-SNAPSHOT) < Strict-Transport-Security: max-age=15768000 < Access-Control-Allow-Origin: * < Access-Control-Expose-Headers: X-Records < Content-Length: 43 < Via: 1.1 lon3-puppetdb-a3.lon3 < Connection: close < * Closing connection 0 Unsupported query parameter 'include-total'⏎ I understand that it's not particularly useful to have an X-Records when you know this will only return one item but it should just accept include-total instead of blowing up on it. The same goes for the other parameters that did work in the first query. Some 'subparts' of endpoints don't support paging but that's not indicated in the documentation which makes this behaviour confusing and inconsistent. It's also really not harmful to support these options on queries that might return a single item, an offset=0 and limit=200 should be perfectly valid even if you just return 1 item. Add Comment   This message was sent by Atlassian JIRA (v6.3.10#6340-sha1:7ea293a) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at http://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.