ryanmeowy commented on PR #13145: URL: https://github.com/apache/seatunnel/pull/13145#issuecomment-6656858407
Thanks for the careful review, @DanielLeens — you were right on all three points, and Issue 1 was a real regression that I had introduced. **Issue 1 (blocker) — fixed in ef6705e.** I verified your claim first: JSqlParser 4.9 does parse `RLIKE`, `REGEXP` and `ILIKE` into `LikeExpression`, with only `getLikeKeyWord()` differing: ``` name RLIKE 'A.*' -> LikeExpression | keyword=RLIKE name REGEXP 'A.*' -> LikeExpression | keyword=REGEXP name ILIKE 'a%ce' -> LikeExpression | keyword=ILIKE ``` And the behaviour really changed for the worse. On the parent commit vs. the previous head of this PR, for row `(1, "Alice")`: | predicate | parent commit | previous head of this PR | |---|---|---| | `name RLIKE 'A.*'` | **true** | **false** | | `name REGEXP 'A.*'` | **true** | **false** | | `name NOT RLIKE 'A.*'` | **false** | **true** | So my change had taken `RLIKE` from working to silently dropping every row — exactly the failure class this PR sets out to fix, moved to a different keyword. Sorry about that; I had checked the `LikeExpression` dispatch but not the keyword. Fix: a single `matchLikeExpression` helper that both `likeExpr` and `notLikeExpr` now use, dispatching on the keyword. `RLIKE` / `REGEXP` are compiled as regular expressions exactly as before (no LIKE rewriting), and only `LIKE` / `ILIKE` go through `likeMatches`. I went with "keep the regex behaviour" rather than throwing, because these give correct answers today and throwing would break working jobs. Pinned by `testRlikeAndRegexpKeepRegexSemantics`, which fails without the fix: ``` ZetaSQLFilterTest.testRlikeAndRegexpKeepRegexSemantics:182 expected: <true> but was: <false> ``` **Issue 2 (non-blocking) — you are right, my description was wrong.** Verified: `"a\nb" LIKE 'a%b'` was false on the parent commit and is true now; same for `'a_b'`. The old behaviour was a `Pattern.DOTALL` artifact rather than a decision, and the new one is the SQL standard, so I kept the new behaviour and fixed the description instead: the "user-facing change" section now says `%` and `_` also match line breaks, with its own row in the before/after table, plus `testLikeWildcardsMatchAcrossNewlines` to pin it. **Issue 3 (optional) — agreed, leaving it out.** `ESCAPE` stays as a pre-existing gap. Throwing on `getEscape() != null` is the right end state, but it turns a silently-wrong answer into a hard failure for jobs that run today, so I would rather do it as its own change with its own release note. It is listed under the known gaps in the description. **Nit — fixed.** `java.time.Duration` is imported now instead of fully qualified. `ZetaSQLFilterTest` is at 13 tests, all green; `ZetaSQLFilterTest + DateTimeFunctionsTest + SQLDateTimeFunctionsTest` is 49 passed; `spotless:check` passes on JDK 17. One note on CI: the fork `Build` run you linked did eventually run — `unit-test` passed on all four JDK/OS combinations and `Code style`/`License header` passed. The only red job was `all-connectors-it-5`, and it failed in `TiDBCDCIT.startUp` with `CommunicationsException: Communications link failure` / `EOFException` while starting the Testcontainers TiDB, i.e. the test never got to run its logic and it is unrelated to this change. Requesting a re-run of that job would confirm it. The head is now ef6705e, so a fresh run is needed anyway. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
