SEZ9 commented on PR #12661: URL: https://github.com/apache/seatunnel/pull/12661#issuecomment-6207611123
Thanks @hutiefang76, a1218f7be looks good on the points from the last round. - Option name / LocalFile note: the switch to `datetime_format` and the explicit statement that CSV/Text source TIMESTAMP parsing auto-detects the pattern (and that the sink option only controls output) is what I was after. - Refresh path: skipping the re-parse when the re-match yields the same formatter and wrapping the second failure in `CommonError.formatDateTimeError(field, fieldName)` with the `DateTimeParseException` as cause resolves the raw-exception concern. Good that the regression cases cover the post-refresh failure. - Incompatible-changes entry and the shared `matchTimestampFormatter` helpers (no cache side effect in either reader, caller does the `put`, CSV TIMESTAMP_TZ fallback reusing the helper) are also fine now. Two small items still open from the earlier scope: 1. The behaviour note is currently only on the LocalFile source pages (EN/ZH). The other file-source docs that list the same `datetime_format` option go through the same `TextReadStrategy`/`CsvReadStrategy` path, so they should either get the same one-line note or a pointer to it. Could you add that, or tell me if you'd prefer a follow-up PR? 2. The refresh is exception-driven and single-slot, so a column that alternates between two supported layouts will throw and re-match on every row. I'm fine not optimising this here, but please add a short code comment next to the cache replacement calling out the trade-off so nobody later mistakes it for a bug. Since container E2E wasn't run locally, I'll wait for the CI run on this head before merging. Once the two doc/comment items above are in, I think this is ready. <!-- streview-comment:1800 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
