robertwb commented on code in PR #27461:
URL: https://github.com/apache/beam/pull/27461#discussion_r1283830743
##########
sdks/java/fn-execution/src/main/java/org/apache/beam/sdk/fn/stream/DataStreams.java:
##########
@@ -185,15 +186,19 @@ public DataStreamDecoder(Coder<T> coder,
PrefetchableIterator<ByteString> inputS
* Skips any remaining bytes in the current {@link ByteString} moving to
the next {@link
* ByteString} in the underlying {@link ByteString} {@link Iterator
iterator} and decoding
* elements till at the next boundary.
+ *
+ * @param weightEstimatorFn Pass in what is the function to be used for
the weight.
*/
- public List<T> decodeFromChunkBoundaryToChunkBoundary() {
+ public WeightedList<T> decodeFromChunkBoundaryToChunkBoundary(
+ Function<T, Long> weightEstimatorFn) {
inbound.currentStream = inputByteStrings.next().newInput();
Review Comment:
Currently we're only keeping the total weight anyhow. (Keeping per-element
weights seems to be a large expense, especially as the goal with these weights
is to try to manage memory.)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
