[
https://issues.apache.org/jira/browse/DRILL-6318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432804#comment-16432804
]
ASF GitHub Bot commented on DRILL-6318:
---------------------------------------
Github user gparai commented on a diff in the pull request:
https://github.com/apache/drill/pull/1204#discussion_r180539160
--- Diff:
exec/java-exec/src/main/java/org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
---
@@ -89,14 +92,17 @@ public void onMatch(RelOptRuleCall call) {
RelNode child = projectRel.getInput();
final RelNode limitUnderProject =
limitRel.copy(limitRel.getTraitSet(), ImmutableList.of(child));
final RelNode newProject = projectRel.copy(projectRel.getTraitSet(),
ImmutableList.of(limitUnderProject));
- if (DrillRelOptUtil.isProjectOutputRowcountUnknown(projectRel)) {
- //Preserve limit above the project since Flatten can produce more
rows. Also mark it so we do not fire the rule again.
- final RelNode limitAboveProject = new
DrillLimitRel(limitRel.getCluster(), limitRel.getTraitSet(), newProject,
- limitRel.getOffset(), limitRel.getFetch(), true);
- call.transformTo(limitAboveProject);
- } else {
- call.transformTo(newProject);
- }
+ call.transformTo(newProject);
+ // DRILL-6318:
--- End diff --
Cleanup commented out code.
> Push down limit past flatten is incorrect
> -----------------------------------------
>
> Key: DRILL-6318
> URL: https://issues.apache.org/jira/browse/DRILL-6318
> Project: Apache Drill
> Issue Type: Bug
> Affects Versions: 1.13.0, 1.14.0
> Environment: Ubuntu Mate 18.04
> Apache Drill 1.14.0-SNAPSHOT
> Reporter: Oleg Zinoviev
> Priority: Major
> Attachments: sample.json
>
>
> Sample data: [^sample.json]
> Result of
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json`
> {code}
> is
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> |Closed Queen|
> |Innocent Volunteer|
> |Junior Wing|
> |Lame Mantis|
> |Old Master|
> |Numb Pawn|
> |Hollow Guardian|
> |Twin Hurricane|
> |Helpless Avalange|
> Let's try to get first 3 rows:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3
> {code}
> Result has only 2 rows:
> ||d||
> |Honored Boy Scout|
> |Yawning Wolf|
> *Reason:* Limit was pushed down below flatten and only 3 top rows from json
> was selected. In this 3 rows only 2nd has items in "data" field.
> Let's try to get 3 rows from the middle:
> {code:sql}
> select flatten(data) as d from dfs.root.`sample.json` limit 3 offset 5
> {code}
> Result is empty.
> *Reason:* Limit and offset was pushed down below flatten and only 6, 7 and 8
> row from json was selected. This 3 rows contains only 3 items in "data"
> field. After flatten limit and offset applies second time and reject all
> select items.
> Error in org/apache/drill/exec/planner/logical/DrillPushLimitToScanRule.java
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
