kennknowles opened a new issue, #19128:
URL: https://github.com/apache/beam/issues/19128
Hello, my name is Eugene. This is my first bug I am submitting to the Apache
Beam project!
I'm using Python's SDK on Google Cloud Platform to prototype with Beam
locally inside Datalab and then submit a job to Dataflow. In GCP, I'm using
Dataflow and use the logging as basically a print statement to check the state
of objects. For example, after a GroupByKey, if I run Beam locally, the grouped
elements are in a list. In Dataflow, the group elements sometimes will be a
generator. Hence, logging is very important to me.
When submitting a job to Dataflow using Beam 2.6.0, the logging works
correctly. When I click on a box on the Dataflow job's user interface, I can
see a bunch of logs. However, when submitting a job to Dataflow using Beam
2.7.0 or 2.8.0, the logs do not appear at all.
I believe this logging problem is specific to Dataflow/GCP and might not
apply to the other runners. The reason is that I look in the StackDriver for
the job submitted for Dataflow with Beam versions 2.6.0, 2.7.0, and 2.8.0–all
the logs exist in StackDriver. Hence, the logging was correctly accounted for.
However, in the Dataflow user interface, the logs do not appear for Beam 2.7.0
or 2.8.0 when you click on the box for a transformation.
It appears that this problem is GCP specific. I'm creating this bug here
since I don't know where else to raise this issue. Sorry if this is not the
appropriate place to post the bug. If there is a more appropriate place, please
tell me and I'll raise the bug there.
If there are any questions, I can provide screenshots or anything you like.
Thank you,
Eugene
Imported from Jira
[BEAM-6107](https://issues.apache.org/jira/browse/BEAM-6107). Original Jira may
contain additional context.
Reported by: eugeneh101.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
