Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/9923 )
Change subject: WIP KUDU-16 pt 3: add per-scan-token limits ...................................................................... Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/9923/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/9923/1//COMMIT_MSG@19 PS1, Line 19: sure this is actually a useful API or the one that we want to expose. yea, Im not sure this API provides much value beyond the caller just specifying the limit when they hydrate the token before they call Open() on the scan, which has the same net effect. In general I feel like the APIs we provide for scan tokens should be things that offer some optimization opportunity at that "planning" stage. For example, it's beneficial to express predicates because it may reduce the number of generated tokens based on partition pruning, but for something like limit, especially given the odd semantics that you described, it's less obvious. A further reason it might be better for the user to handle the limits is that, at least in the case of Impala, a single backend is responsible for processing several tokens (in parallel, even), and so they'll still need to do their own limit tracking there to early-stop scanning after the limit is exhausted. Maybe worth chatting with the Impala folks to see whether they think they would prefer to use an API like this to attach the limit to the scan tokens or whether or would be more useful for them to manage the limit themselves? -- To view, visit http://gerrit.cloudera.org:8080/9923 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: kudu Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I655f07f10a9a99e9766402729361de97e929136a Gerrit-Change-Number: 9923 Gerrit-PatchSet: 1 Gerrit-Owner: Andrew Wong <[email protected]> Gerrit-Reviewer: Andrew Wong <[email protected]> Gerrit-Reviewer: Dan Burkert <[email protected]> Gerrit-Reviewer: Kudu Jenkins Gerrit-Reviewer: Todd Lipcon <[email protected]> Gerrit-Comment-Date: Wed, 04 Apr 2018 19:23:10 +0000 Gerrit-HasComments: Yes
