[
https://issues.apache.org/jira/browse/FLINK-25262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
hehuiyuan updated FLINK-25262:
------------------------------
Description:
Send data to lookup table by hash , which can improve cache hit rate in
lookup table, futher improve processing performance and reduce the size of
cache.
Shoulder we consider to introducing it?
!image-2021-12-12-15-18-08-574.png|width=419,height=193!
I have a simple test. The parallelism is 10 and the kafka source has 100
million records and the hbase lookuptable has 100 thousands records. It need
100 minutes for forward and 5 minutes for hash.
was:
Send data to lookup table by hash , which can improve cache hit rate, futher
improve processing performance and reduce the size of cache.
Shoulder we consider to introducing it?
!image-2021-12-12-15-18-08-574.png|width=419,height=193!
I have a simple test. The parallelism is 10 and the kafka source has 100
million records and the hbase lookuptable has 100 thousands records. It need
100 minutes for forward and 5 minutes for hash.
> Support to send data to lookup table for KeyGroupStreamPartitioner way for
> SQL
> -------------------------------------------------------------------------------
>
> Key: FLINK-25262
> URL: https://issues.apache.org/jira/browse/FLINK-25262
> Project: Flink
> Issue Type: Improvement
> Reporter: hehuiyuan
> Priority: Minor
> Attachments: image-2021-12-12-15-15-48-540.png,
> image-2021-12-12-15-18-08-574.png
>
>
> Send data to lookup table by hash , which can improve cache hit rate in
> lookup table, futher improve processing performance and reduce the size of
> cache.
>
> Shoulder we consider to introducing it?
>
>
> !image-2021-12-12-15-18-08-574.png|width=419,height=193!
>
> I have a simple test. The parallelism is 10 and the kafka source has 100
> million records and the hbase lookuptable has 100 thousands records. It need
> 100 minutes for forward and 5 minutes for hash.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
