Oleg Zinoviev created DRILL-6318: ------------------------------------ Summary: Push down limit into flatten is incorrect Key: DRILL-6318 URL: https://issues.apache.org/jira/browse/DRILL-6318 Project: Apache Drill Issue Type: Bug Environment: Ubuntu Mate 18.04
Apache Drill 1.14.0-SNAPSHOT Reporter: Oleg Zinoviev Fix For: 1.14.0, 1.13.0 Attachments: sample.json Sample data: [^sample.json] Result of {code:sql} select flatten(data) as d from dfs.root.`sample.json` {code} is ||d|| |Honored Boy Scout| |Yawning Wolf| |Closed Queen| |Innocent Volunteer| |Junior Wing| |Lame Mantis| |Old Master| |Numb Pawn| |Hollow Guardian| |Twin Hurricane| |Helpless Avalange| Let's try to get first 3 rows: {code:sql} select flatten(data) as d from dfs.root.`sample.json` limit 3 {code} Result has only 2 rows: ||d|| |Honored Boy Scout| |Yawning Wolf| *Reason:* Limit was pushed down below flatten and only 3 top rows from json was selected. In this 3 rows only 2nd has items in "data" field. Let's try to get 3 rows from the middle: {code:sql} select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5 {code} Result is empty. *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8 row from json was selected. This 3 rows contains only 3 items in "data" field. After flatten limit and offset applies second time and reject all select items. Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java -- This message was sent by Atlassian JIRA (v7.6.3#76005)