[
https://issues.apache.org/jira/browse/BEAM-5813?focusedWorklogId=165930&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-165930
]
ASF GitHub Bot logged work on BEAM-5813:
----------------------------------------
Author: ASF GitHub Bot
Created on: 14/Nov/18 15:47
Start Date: 14/Nov/18 15:47
Worklog Time Spent: 10m
Work Description: echauchot opened a new pull request #7038: [BEAM-5813]
wait for the refresh of ES indices in all test utils that do index writes.
URL: https://github.com/apache/beam/pull/7038
Testsplit is flaky at least on ES6. From time to time, the expected number
of splits is higher that the actual number of sources after the split. But both
expected and actual number of splits are determined using
`Math.ceil(indexSize/desiredBundleSize)`. My guess is that the difference is
due to `indexSize` not being the same between the time when expected value is
calculated and the time when it is actually estimated in split. This can have 2
reasons:
1- index size in ES has a part of random (says Elastic dev team)
2- refresh of the index was not finished when split is done.
=> This PR addresses point 2 because there is nothing we can do about point
1: test util requests to ES are issued **synchronously** and they specify
`refresh=true` which causes a refresh to be done but I'm not sure that this
refresh is synchronous. Instead use `refresh=wait` so that the request waits
for the index refresh before returning.
R: @timrobertson100
CC: @kennknowles
**Please** add a meaningful description for your change here
------------------------
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
------------------------------------------------------------------------------------------------
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
[](https://builds.apache.org/job/beam_PostCommit_Java_PVR_Flink/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
</br> [](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
| --- | --- | ---
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 165930)
Time Spent: 10m
Remaining Estimate: 0h
> ElasticSearchIOTest.testSplit is flaky
> --------------------------------------
>
> Key: BEAM-5813
> URL: https://issues.apache.org/jira/browse/BEAM-5813
> Project: Beam
> Issue Type: Bug
> Components: io-java-elasticsearch
> Reporter: Kenneth Knowles
> Assignee: Etienne Chauchot
> Priority: Critical
> Labels: flake
> Time Spent: 10m
> Remaining Estimate: 0h
>
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1724/]
> [https://scans.gradle.com/s/xpsswtkdf62so/tests/y3up3nrnbr2ns-opjrlgkblx2ku]
> {code:java}
> Wrong number of splits expected:<25> but was:<22>
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
