[
https://issues.apache.org/jira/browse/BEAM-13384?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sergei Lebedev updated BEAM-13384:
----------------------------------
Description:
{{apache_beam/__init__.py}} does not re-export the {{metrics}} subpackage and
thus forces users to rely on import time side-effects. Specifically, it seems
fairly common to create metrics as
{code:java}
import apache_beam as beam
# ...
c = beam.metrics.Metrics.counter("ns", "counter"){code}
This works at runtime, because {{apache_beam}} imports the {{metrics}}
subpackage indirectly through one of its dependencies and get the "metrics"
attribute as a side-effect of that import (see [last
paragraph|https://docs.python.org/3/reference/import.html#submodules] in "The
import system" for an explanation of how that works).
Examples from GitHub
*
[tensorflow/transform|https://github.com/tensorflow/transform/blob/e0331bda765b7cc34347b38da74270dd3c01939b/tensorflow_transform/beam/impl.py#L282]
*
[tensorflow/tfx|https://github.com/tensorflow/tfx/blob/578c40cc23d5dd661826f5ed0e16db58479db1b4/tfx/components/transform/executor.py#L600]
*
[google-research/tapas|https://github.com/google-research/tapas/blob/f3d9f068e6eedb252883049b582516a1294ff951/tapas/utils/pretrain_utils.py#L65]
*
[google/tensorflow-recorder|https://github.com/google/tensorflow-recorder/blob/231b2c5422593d4eb7fc8502d5023425d08bedd6/tfrecorder/beam_pipeline.py#L141]
*
[kubeflow/examples|https://github.com/kubeflow/examples/blob/fcd2ef38027c1c0a91af2370c2beedb4c24298a8/code_search/src/code_search/dataflow/do_fns/prediction_do_fn.py#L112]
I think we should re-export metrics in {{apache_beam/__init__.py,}} similarly
to coders and io. Apart from legalizing a common way of defining metrics, this
would also make {{apache_beam}} more tooling friendly, since type checkers and
IDEs would be able to understand what {{apache_beam.metrics}} refers to without
any Beam-specific logic/hacks.
was:
apache_beam/__init__.py does not re-export the metrics subpackage and thus
forces users to rely on import time side-effects. Specifically, it seems fairly
common to create metrics as
{code:java}
import apache_beam as beam
# ...
c = beam.metrics.Metrics.counter("ns", "counter"){code}
This works at runtime, because apache_beam imports the metrics subpackage
indirectly through one of its dependencies and get the "metrics" attribute as a
side-effect of that import (see [last
paragraph|https://docs.python.org/3/reference/import.html#submodules] in "The
import system" for an explanation of how that works).
Examples from GitHub
*
[tensorflow/transform|https://github.com/tensorflow/transform/blob/e0331bda765b7cc34347b38da74270dd3c01939b/tensorflow_transform/beam/impl.py#L282]
*
[tensorflow/tfx|https://github.com/tensorflow/tfx/blob/578c40cc23d5dd661826f5ed0e16db58479db1b4/tfx/components/transform/executor.py#L600]
*
[google-research/tapas|https://github.com/google-research/tapas/blob/f3d9f068e6eedb252883049b582516a1294ff951/tapas/utils/pretrain_utils.py#L65]
*
[google/tensorflow-recorder|https://github.com/google/tensorflow-recorder/blob/231b2c5422593d4eb7fc8502d5023425d08bedd6/tfrecorder/beam_pipeline.py#L141]
*
[kubeflow/examples|https://github.com/kubeflow/examples/blob/fcd2ef38027c1c0a91af2370c2beedb4c24298a8/code_search/src/code_search/dataflow/do_fns/prediction_do_fn.py#L112]
I think we should re-export metrics in apache_beam/__init__.py, similarly to
coders and io.
> apache_beam does not explicitly re-export metrics
> -------------------------------------------------
>
> Key: BEAM-13384
> URL: https://issues.apache.org/jira/browse/BEAM-13384
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Sergei Lebedev
> Priority: P2
>
> {{apache_beam/__init__.py}} does not re-export the {{metrics}} subpackage and
> thus forces users to rely on import time side-effects. Specifically, it seems
> fairly common to create metrics as
> {code:java}
> import apache_beam as beam
> # ...
> c = beam.metrics.Metrics.counter("ns", "counter"){code}
> This works at runtime, because {{apache_beam}} imports the {{metrics}}
> subpackage indirectly through one of its dependencies and get the "metrics"
> attribute as a side-effect of that import (see [last
> paragraph|https://docs.python.org/3/reference/import.html#submodules] in "The
> import system" for an explanation of how that works).
> Examples from GitHub
> *
> [tensorflow/transform|https://github.com/tensorflow/transform/blob/e0331bda765b7cc34347b38da74270dd3c01939b/tensorflow_transform/beam/impl.py#L282]
> *
> [tensorflow/tfx|https://github.com/tensorflow/tfx/blob/578c40cc23d5dd661826f5ed0e16db58479db1b4/tfx/components/transform/executor.py#L600]
> *
> [google-research/tapas|https://github.com/google-research/tapas/blob/f3d9f068e6eedb252883049b582516a1294ff951/tapas/utils/pretrain_utils.py#L65]
> *
> [google/tensorflow-recorder|https://github.com/google/tensorflow-recorder/blob/231b2c5422593d4eb7fc8502d5023425d08bedd6/tfrecorder/beam_pipeline.py#L141]
> *
> [kubeflow/examples|https://github.com/kubeflow/examples/blob/fcd2ef38027c1c0a91af2370c2beedb4c24298a8/code_search/src/code_search/dataflow/do_fns/prediction_do_fn.py#L112]
> I think we should re-export metrics in {{apache_beam/__init__.py,}} similarly
> to coders and io. Apart from legalizing a common way of defining metrics,
> this would also make {{apache_beam}} more tooling friendly, since type
> checkers and IDEs would be able to understand what {{apache_beam.metrics}}
> refers to without any Beam-specific logic/hacks.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
