Myracle commented on PR #27577: URL: https://github.com/apache/flink/pull/27577#issuecomment-6620863944
> thank you for the contribution > > the main problem that I see here: in case of several invocations like `REGEXP_SPLIT(...), REGEXP_SPLIT(...)` even for the same pattern it will compile pattern each time Thanks for pointing this out. Strictly speaking the pattern was not compiled on every call:`getRegexpPattern` goes through the shared thread-local`REGEXP_PATTERN_CACHE` (max 64 entries per thread). But it still did a cache lookup for every record, and the cache can evict entries when many patterns are in use. I've changed`RegexpSplitFunction` so that a literal regex is read from the`CallContext` during specialization and compiled once in`open()` . Each call site, e.g.`REGEXP_SPLIT(a, ','), REGEXP_SPLIT(b, ',')` , now gets its own pre-compiled`Pattern` with no per-record lookup. Non-literal regex arguments still use the cache. I also added a test case for a valid non-literal regex. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
