[
https://issues.apache.org/jira/browse/BEAM-12395?focusedWorklogId=621088&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-621088
]
ASF GitHub Bot logged work on BEAM-12395:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jul/21 17:20
Start Date: 09/Jul/21 17:20
Worklog Time Spent: 10m
Work Description: y1chi commented on pull request #14877:
URL: https://github.com/apache/beam/pull/14877#issuecomment-877338303
> Could you describe the benefit of adding Cloud Profiler? Does this enable
automatic instrumentation of Beam pipelines running on GCP? My team would
actually benefit from having that feature :)
>
> FWIW, adding `google-cloud-profiler` makes using apache-beam within Docker
more difficult.
>
> `google-cloud-profiler` doesn't provide wheels, so upgrading `apache-beam`
from 2.30.0 to 2.31.0 requires adding a compiler toolchain to the Docker build.
To avoid bloating the final Docker image, that means setting up multi-stage
builds in the Dockerfile. Multi-stage builds to build a Python library is a
little obscure. The Cloud Profiler docs does have [good documentation for
it](https://cloud.google.com/profiler/docs/profiling-python#running_with_linux_alpine),
but it's a new step all Beam Docker users will need to deal with, and it's a
bit annoying to jump through that hoop for a transitive dependency.
>
> It would have been nice to have this change flagged in the release notes
(https://github.com/apache/beam/releases/tag/v2.31.0).
>
> I guess the _ideal_ solution here is for google-cloud-profiler to
distribute wheels... I filed
[GoogleCloudPlatform/cloud-profiler-python#98](https://github.com/GoogleCloudPlatform/cloud-profiler-python/issues/98)
Yes, this will enable automatic instrumentation of beam pipeline on GCP when
the pipeline options has '--experiments=enable_google_cloud_profiler', I'll add
that to release note. The package should have been pre-installed if running
GCP. Didn't notice that this will introduce complexity to docker user, thanks
for creating the issue for google-cloud-profiler to distribute the wheel. We
can remove the dependency from base requirement and maybe have user to provide
that dependency through requirements if they indeed need this functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 621088)
Time Spent: 2h (was: 1h 50m)
> Support google cloud profiler in python sdk
> -------------------------------------------
>
> Key: BEAM-12395
> URL: https://issues.apache.org/jira/browse/BEAM-12395
> Project: Beam
> Issue Type: New Feature
> Components: sdk-py-harness
> Reporter: Yichi Zhang
> Assignee: Yichi Zhang
> Priority: P2
> Fix For: 2.31.0
>
> Time Spent: 2h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
