imbajin opened a new pull request, #858:
URL: https://github.com/apache/hugegraph-toolchain/pull/858
## Purpose of the PR
Fix #706. Takes over #710 by @bajisn-666 with a focused fix for the
reproduced concurrency failure.
Parallel file readers share one `FileSource`, including its `SkippedLine`
rule. Reusing that rule's mutable `Matcher` across readers can let a blank or
comment line through and fail CSV parsing. Each match now has independent state.
## Main Changes
- Cache the immutable compiled `Pattern` and create a `Matcher` for each
line; invalidate the cache when the regex changes.
- Exercise split readers with four threads, checking every expected row and
completed file progress, including comments, blank lines, empty files, suffix
filtering, and a serialized custom rule.
- Restore four parser threads in the three local tests previously limited to
one, and in the two HDFS cases from #710.
## Verifying these changes
- [x] Need tests and can be verified as follows:
- The mixed-line concurrent-reader regression fails on the previous
implementation and passes with this fix.
- Client and Loader unit suites pass on JDK 17.
- Server-backed local functional tests are still being verified; this PR
remains a draft until that check completes.
- HDFS integration has not been run locally.
## Does this PR potentially affect the following parts?
- [x] Nope
- [ ] Dependencies (add/update license info)
- [ ] Modify configurations
- [ ] The public API
- [ ] Other affects
## Documentation Status
- [ ] `Doc - TODO`
- [ ] `Doc - Done`
- [x] `Doc - No Need`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
