paul-rogers commented on issue #1971: DRILL-7572: JSON structure parser URL: https://github.com/apache/drill/pull/1971#issuecomment-587300916 @vvysotskyi, thanks for pointing out the question; I missed it when reading the code comments. Looked at `CountingJsonReader`. Looks like creates a series or rows, one per input row, with just a bit field set to 1. This reader could do exactly the same by projecting none of the columns and instead writing that bit = 1 value for the start of each top-level object. The non-projected columns will "free-wheel" over the incoming JSON. A better solution is to actually return the count. Maybe we need another option on the format plugin, `supportsCountPushDown()` so that we return the per-file row count rather grind through the effort of making trivial rows. EVF has support for this idea with its notion of "project none" which occurs when the scan asks for now rows as in a `COUNT(*)`.
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
