[
https://issues.apache.org/jira/browse/FLINK-16528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17189042#comment-17189042
]
Jark Wu commented on FLINK-16528:
---------------------------------
Hi [~danny0405], they do not conflict. It is valid to submit a query {{SELECT *
FROM kafka LIMIT 10;}} which will only read 10 records and stop, because any
records after 10 records will be dropped. This is just an optimization that the
Kafka don't need to poll new records after 10. We don't introduce any new
syntax or feature.
> Support Limit push down for Kafka streaming sources
> ---------------------------------------------------
>
> Key: FLINK-16528
> URL: https://issues.apache.org/jira/browse/FLINK-16528
> Project: Flink
> Issue Type: New Feature
> Components: Table SQL / Ecosystem, Table SQL / Planner
> Reporter: Jark Wu
> Assignee: Jark Wu
> Priority: Major
> Labels: usability
> Fix For: 1.12.0
>
>
> Currently, a limit query {{SELECT * FROM kafka LIMIT 10}} will be translated
> into TopN operator and will scan the full data in the source. However,
> {{LIMIT}} is a very useful feature in SQL CLI to explore data in the source.
> It doesn't make sense it never stop.
> We can support such case in streaming mode (ignore the text format):
> {code}
> flink > SELECT * FROM kafka LIMIT 10;
> kafka_key | user_name | lang | created_at
> --------------------+-----------------+------+-------------------------
> 494227746231685121 | burncaniff | en | 2014-07-29 14:07:31.000
> 494227746214535169 | gu8tn | ja | 2014-07-29 14:07:31.000
> 494227746219126785 | pequitamedicen | es | 2014-07-29 14:07:31.000
> 494227746201931777 | josnyS | ht | 2014-07-29 14:07:31.000
> 494227746219110401 | Cafe510 | en | 2014-07-29 14:07:31.000
> 494227746210332673 | Da_JuanAnd_Only | en | 2014-07-29 14:07:31.000
> 494227746193956865 | Smile_Kidrauhl6 | pt | 2014-07-29 14:07:31.000
> 494227750426017793 | CashforeverCD | en | 2014-07-29 14:07:32.000
> 494227750396653569 | FilmArsivimiz | tr | 2014-07-29 14:07:32.000
> 494227750388256769 | jmolas | es | 2014-07-29 14:07:32.000
> (10 rows)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
