[
https://issues.apache.org/jira/browse/FLINK-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17322782#comment-17322782
]
Flink Jira Bot commented on FLINK-22038:
----------------------------------------
This issue is assigned but has not received an update in 7 days so it has been
labeled "stale-assigned". If you are still working on the issue, please give an
update and remove the label. If you are no longer working on the issue, please
unassign so someone else may work on it. In 7 days the issue will be
automatically unassigned.
> Update TopN node to be without rowNumber if rowNumber field is projected out
> after TopN
> ---------------------------------------------------------------------------------------
>
> Key: FLINK-22038
> URL: https://issues.apache.org/jira/browse/FLINK-22038
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Planner
> Reporter: Andy
> Assignee: Andy
> Priority: Major
> Labels: stale-assigned
> Attachments: image-2021-03-30-16-03-09-876.png
>
>
> As describe in article
> [sql_queries|https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/sql/queries.html#no-ranking-output-optimization],
> an optimization way to improve performance of TopN is omitting rownum field
> in the outer SELECT clause of the Top-N query.
> However, some queries generated unexpected plan, even though we have followed
> the instructions in the documentation.
> {code:java}
> @Test
> def testRowNumberFiltered(): Unit = {
> util.addDataStream[(String, Long, Long, Long)](
> "T", 'uri, 'reqcount, 'start_time, 'bucket_id)
> val sql =
> """
> |SELECT
> | uri,
> | reqcount,
> | start_time
> |FROM
> | (
> | SELECT
> | uri,
> | reqcount,
> | rownum_2,
> | start_time
> | FROM
> | (
> | SELECT
> | uri,
> | reqcount,
> | start_time,
> | ROW_NUMBER() OVER (
> | PARTITION BY start_time
> | ORDER BY
> | reqcount DESC
> | ) AS rownum_2
> | FROM
> | (
> | SELECT
> | uri,
> | reqcount,
> | start_time,
> | ROW_NUMBER() OVER (
> | PARTITION BY start_time, bucket_id
> | ORDER BY
> | reqcount DESC
> | ) AS rownum_1
> | FROM T
> | )
> | WHERE
> | rownum_1 <= 100000
> | )
> | WHERE
> | rownum_2 <= 100000
> |)
> |""".stripMargin
> util.verifyExecPlan(sql)
> }
> {code}
> For example, we expect both outer and inner TopN could use without rowNumber
> optimization in the above queries, however inner TopN is not as we expected.
> The logical plan and physical plan as following, we could find even though
> the rowNumber field is projected out after inner topN, inner topN still with
> rowNumber.
>
> !image-2021-03-30-16-03-09-876.png!
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
