acrites commented on code in PR #40639:
URL: https://github.com/apache/beam/pull/40639#discussion_r4327819573
##########
runners/google-cloud-dataflow-java/worker/src/main/java/org/apache/beam/runners/dataflow/worker/windmill/work/processing/ComputationWorkExecutorFactory.java:
##########
@@ -166,23 +168,27 @@ private static void trackAutoscalingBytesRead(
MapTask mapTask,
Nodes.ParallelInstructionNode readNode,
Coder<?> readCoder,
- ReadOperation readOperation,
+ Operation operation,
MapTaskExecutor mapTaskExecutor,
String counterName) {
- NameContext nameContext =
- NameContext.create(
- mapTask.getStageName(),
- readNode.getParallelInstruction().getOriginalName(),
- readNode.getParallelInstruction().getSystemName(),
- readNode.getParallelInstruction().getName());
- readOperation.receivers[0].addOutputCounter(
- counterName,
- new OutputObjectAndByteCounter(
- new
IntrinsicMapTaskExecutorFactory.ElementByteSizeObservableCoder<>(readCoder),
- mapTaskExecutor.getOutputCounters(),
- nameContext)
- .setSamplingPeriod(100)
- .countBytes(counterName));
+ if (operation.receivers != null
+ && operation.receivers.length > 0
+ && operation.receivers[0] != null) {
+ NameContext nameContext =
+ NameContext.create(
+ mapTask.getStageName(),
+ readNode.getParallelInstruction().getOriginalName(),
+ readNode.getParallelInstruction().getSystemName(),
+ readNode.getParallelInstruction().getName());
+ operation.receivers[0].addOutputCounter(
+ counterName,
+ new OutputObjectAndByteCounter(
+ new
IntrinsicMapTaskExecutorFactory.ElementByteSizeObservableCoder<>(readCoder),
+ mapTaskExecutor.getOutputCounters(),
+ nameContext)
+ .setSamplingPeriod(100)
Review Comment:
I'm happy to change it if we can agree on a better time. This is just what
we currently do for custom source reporting already.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
