Davis-Zhang-Onehouse opened a new pull request, #13626:
URL: https://github.com/apache/hudi/pull/13626
### Change Logs
the test is flaky because it use a non thread safe ArrayList and use it in
the lambda function of HoodieListData.map, which underlying implementation is
parallelStream based.
You might have already noticed that anything captured in lambda function has
to be "effective final", which serves the purpose of safely publish the
variable across all threads that consume the lambda function. Yet if we want to
write to the list and later read its content back after the operation, we need
a thread safe variable. Otherwise, we only protect the safety of the reference
to the list variable, but not the reference to the elements that the list
holds, this is a classic escape pattern.
Hit such issues in Azure CI.
```
[ERROR] Tests run: 26, Failures: 0, Errors: 1, Skipped: 0, Time elapsed:
0.045 s <<< FAILURE! - in org.apache.hudi.common.data.TestHoodieListDataPairData
[ERROR] testReduceByKeyWithCloseableInput Time elapsed: 0.012 s <<< ERROR!
java.lang.NullPointerException
at
org.apache.hudi.common.data.TestHoodieListDataPairData.lambda$testReduceByKeyWithCloseableInput$0(TestHoodieListDataPairData.java:168)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at
org.apache.hudi.common.data.TestHoodieListDataPairData.testReduceByKeyWithCloseableInput(TestHoodieListDataPairData.java:168)
at java.lang.reflect.Method.invoke(Method.java:498)
at java.util.ArrayList.forEach(ArrayList.java:1259)
at java.util.ArrayList.forEach(ArrayList.java:1259)
```
Fixed by use a thread safe concurrent linked list instead.
### Impact
stablize the CI
### Risk level (write none, low medium or high below)
none
### Documentation Update
none
### Contributor's checklist
- [ ] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [ ] Change Logs and Impact were stated clearly
- [ ] Adequate tests were added if applicable
- [ ] CI passed
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
