[
https://issues.apache.org/jira/browse/BEAM-6389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matthew Willson updated BEAM-6389:
----------------------------------
Priority: Major (was: Minor)
Description:
This fails under Python 2.7 but works fine under Python 3 – looks like an easy
fix.
I am reading from a beam.io.utils.CountingSource:
{{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
This fails under python 2.7 with:
Python exception: Can't pickle <class
'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup
apache_beam.metrics.metric.DelegatingCounter failed
Looks like this is because DelegatingCounter is actually an inner class defined
inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve
it under its qualified name
apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix
would be to move it to a top-level class in that module.
This also applies to other sources which use Metrics.counter.
was:
This fails under Python 2.7 but works fine under Python 3 – looks like an easy
fix.
I am reading from a beam.io.utils.CountingSource:
{{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
This fails under python 2.7 with:
Python exception: Can't pickle <class
'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup
apache_beam.metrics.metric.DelegatingCounter failed
Looks like this is because DelegatingCounter is actually an inner class defined
inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can resolve
it under its qualified name
apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A fix
would be to move it to a top-level class in that module.
Summary: Sources using Metrics.counter fail with 'Can't pickle ...
DelegatingCounter' (was: Read from python CountingSource fails with 'Can't
pickle ... DelegatingCounter')
> Sources using Metrics.counter fail with 'Can't pickle ... DelegatingCounter'
> ----------------------------------------------------------------------------
>
> Key: BEAM-6389
> URL: https://issues.apache.org/jira/browse/BEAM-6389
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Affects Versions: 2.9.0
> Reporter: Matthew Willson
> Assignee: Ahmet Altay
> Priority: Major
>
> This fails under Python 2.7 but works fine under Python 3 – looks like an
> easy fix.
> I am reading from a beam.io.utils.CountingSource:
> {{beam.io.Read(beam_io_utils.CountingSource(NUM_ROWS))}}
> This fails under python 2.7 with:
> Python exception: Can't pickle <class
> 'apache_beam.metrics.metric.DelegatingCounter'>: attribute lookup
> apache_beam.metrics.metric.DelegatingCounter failed
> Looks like this is because DelegatingCounter is actually an inner class
> defined inside apache_beam.metrics.metric.Metric. Under Python 3 pickling can
> resolve it under its qualified name
> apache_beam.metrics.metric.Metric.DelegatingCounter, but not in Python 2. A
> fix would be to move it to a top-level class in that module.
> This also applies to other sources which use Metrics.counter.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
