divyanshus2404 commented on PR #71711: URL: https://github.com/apache/airflow/pull/71711#issuecomment-5412482859
Simplified the implementation in ca82cacc. The tokeniser turned out to be unnecessary: the quadratic behaviour came from the leading `\S*?` retrying at every offset in the string, so anchoring the match at a token boundary with `(?<!\S)` is enough to make it linear. The value is then matched with explicit single-quoted / double-quoted / bare alternatives instead of the backreference-driven lookahead, with a quote closing the value only when whitespace or end-of-string follows it — which is what preserves masking of values that themselves contain quotes. That brings the change down to 21 lines and keeps it a single pattern, which should be easier to review and maintain than the hand-written scan. Timings on a single long token with no sensitive keyword (the worst case for the old pattern): | Input size | Original | This PR | |-----------|----------|---------| | 10,000 chars | 0.8s | <1ms | | 50,000 chars | 18.7s | 1.1ms | Checked against the previous regex over 200k generated command lines — including tabs, repeated spaces, newlines, and embedded quotes — with no output differences. Spark provider suite: 271 passed, 2 skipped. --- Drafted-by: Claude Code (no human review before posting) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
