sasmith-14 opened a new issue, #884: URL: https://github.com/apache/wayang/issues/884
**`JavaReservoirSampleOperator` returns the entire input instead of a sample whenever the dataset size is unknown.** ### The bug `evaluate()` takes a "return all" shortcut at [`JavaReservoirSampleOperator.java:84`](https://github.com/apache/wayang/blob/713faa7d8bf4726536f6b0920ebb73053160274c/wayang-platforms/wayang-java/src/main/java/org/apache/wayang/java/operators/JavaReservoirSampleOperator.java#L84): ```java if (sampleSize >= datasetSize) { // return all ``` `datasetSize` defaults to `SampleOperator.UNKNOWN_DATASET_SIZE` (`0`). So when the size isn't known, `sampleSize >= 0` is always true and the operator emits **every element**. The sibling `JavaRandomSampleOperator` already guards this with `isDataSetSizeKnown()` — the reservoir operator doesn't. ### Reproduce Sample 5 from 10 elements, without setting the dataset size (the normal streaming case): ```java JavaReservoirSampleOperator<Integer> op = new JavaReservoirSampleOperator<>( it -> 5, DataSetType.createDefaultUnchecked(Integer.class), it -> 42L); // no setDatasetSize(...) -> size is UNKNOWN // input: 1..10 ``` | | Result | | -------- | ------------------------ | | Expected | 5 elements | | Actual | 10 elements (all of them) | The existing `JavaReservoirSampleOperatorTest` always calls `setDatasetSize(10)`, so it never hits this path. ### Environment `wayang-platforms/wayang-java` · `main` (`713faa7d`) and `1.1.1`. ### Fix A one-line guard (`isDataSetSizeKnown() && …`) plus a regression test — the streaming reservoir algorithm already handles inputs smaller than the sample size. I have it ready and can open a PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
