Micah Wylde created BEAM-5455:
---------------------------------
Summary: Don't info log for every bundle in the python sdk
Key: BEAM-5455
URL: https://issues.apache.org/jira/browse/BEAM-5455
Project: Beam
Issue Type: Improvement
Components: sdk-py-harness
Reporter: Micah Wylde
Assignee: Robert Bradshaw
Currently the python sdk logs a bunch of messages at INFO level for every
bundle. For the portable flink runner, which uses a bundle size of 1 in
streaming mode, this produces huge log spew:
{code:java}
[grpc-default-executor-1] INFO sdk_worker.run - Got work 4
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DataOutputOperation >
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DoOperation window output_tags=['out']>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DoOperation pair_with_one output_tags=['out']>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DoOperation timestamp output_tags=['out']>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DoOperation decode output_tags=['out']>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - start
<DataInputOperation receivers=[ConsumerSet[.out0,
coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DataInputOperation receivers=[ConsumerSet[.out0,
coder=WindowedValueCoder[BytesCoder], len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DoOperation decode output_tags=['out'], receivers=[ConsumerSet[decode.out0,
coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DoOperation timestamp output_tags=['out'],
receivers=[ConsumerSet[timestamp.out0,
coder=WindowedValueCoder[FastPrimitivesCoder], len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DoOperation pair_with_one output_tags=['out'],
receivers=[ConsumerSet[pair_with_one.out0,
coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]],
len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DoOperation window output_tags=['out'], receivers=[ConsumerSet[window.out0,
coder=WindowedValueCoder[TupleCoder[FastPrimitivesCoder, FastPrimitivesCoder]],
len(consumers)=1]]>
[grpc-default-executor-1] INFO bundle_processor.process_bundle - finish
<DataOutputOperation >
{code}
These log messages should probably be at debug instead.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
