[
https://issues.apache.org/jira/browse/FLINK-25409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17463553#comment-17463553
]
Yuan Zhu edited comment on FLINK-25409 at 12/22/21, 3:06 AM:
-------------------------------------------------------------
In 1.14.2, neglecting some private LookupFunctions outside of connectors:
||Cache Type||Legacy||Not Legacy||
|Guava cache|JdbcLookupFunction|JdbcRowDataLookupFunction|
|Java map|CsvLookupFunction| |
|Java map| |FileSystemLookupFunction|
|Guava cache | |HBaseRowDataLookupFunction|
They all have cache inside. We can extract a common abstract class to init
cache metric if enabled and some useful metrics. The implementation logic can
be left by child class.
If some LookupSource without cache in connectors external repo won't to use
metric indeed, they can just implement TableFunction like before or implement
the newly abstract LookupFunction without init metric implementation.
[~jingzhang] WDYT?
was (Author: straw):
In 1.14.2, neglecting some private LookupFunctions outside of connectors:
||Cache Type||Legacy||Not Legacy||
|Guava cache|JdbcLookupFunction|JdbcRowDataLookupFunction|
|Java map|CsvLookupFunction| |
|Java map| |FileSystemLookupFunction|
|Guava cache | |HBaseRowDataLookupFunction|
They all have cache inside. We can extract a common abstract class to init
cache metric if enabled and some useful metrics. The implementation logic can
be left by child class.
If some LookupSource without cache in connectors external repo won't to use
metric indeed, they can just implement TableFunction like before or implement
the newly abstract LookupFunction without init metric implementation.
> Add cache metric to LookupFunction
> ----------------------------------
>
> Key: FLINK-25409
> URL: https://issues.apache.org/jira/browse/FLINK-25409
> Project: Flink
> Issue Type: Improvement
> Components: Table SQL / Ecosystem
> Reporter: Yuan Zhu
> Priority: Major
>
> Since we encounter performance problem when lookup join in production env
> frequently, adding metrics to monitor Lookup function cache is very helpful
> to troubleshoot.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
