divyanshus2404 commented on PR #71711: URL: https://github.com/apache/airflow/pull/71711#issuecomment-5412601016
Thanks for the review — tests added, and the implementation is simpler than the version you first looked at. **Tests** `test_masks_passwords` now covers every form the pattern handles: `key=value`, dotted and uppercase keys, space-separated values, values quoted across multiple tokens, values separated by tabs or repeated spaces, and a sensitive flag with no following value. `test_masks_passwords_is_linear_on_large_input` guards the backtracking this PR removes. **Implementation** The tokeniser I originally wrote turned out to be unnecessary. The quadratic behaviour came from the leading `\S*?` retrying at every offset in the string, so anchoring the match at a token boundary with `(?<!\S)` is enough to make the scan linear. The value is matched with explicit single-quoted / double-quoted / bare alternatives instead of the backreference-driven lookahead; a quote closes the value only when whitespace or end-of-string follows, which preserves masking of values that contain quotes. That keeps this a single pattern at 21 changed lines rather than a hand-written scan. **Verification** Timings on a single long token with no sensitive keyword — the worst case for the old pattern: | Input size | Original | This PR | |-----------|----------|---------| | 10,000 chars | 0.8s | <1ms | | 50,000 chars | 18.7s | 1.1ms | Checked against the previous regex over 200k generated command lines — tabs, repeated spaces, newlines, embedded and unterminated quotes — with no output differences, so masking behaviour is unchanged. Every pre-existing `test_masks_passwords` case passes untouched. Spark provider suite: 271 passed, 2 skipped. Happy to adjust if you'd prefer a different shape for any of this. --- Drafted-by: Claude Code; reviewed by @divyanshus2404 before posting -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
