[
https://issues.apache.org/jira/browse/BEAM-4036?focusedWorklogId=89714&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-89714
]
ASF GitHub Bot logged work on BEAM-4036:
----------------------------------------
Author: ASF GitHub Bot
Created on: 10/Apr/18 23:43
Start Date: 10/Apr/18 23:43
Worklog Time Spent: 10m
Work Description: aaltay closed pull request #5072: [BEAM-4036] Fix
pickling for "recursive" classes.
URL: https://github.com/apache/beam/pull/5072
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/internal/module_test.py
b/sdks/python/apache_beam/internal/module_test.py
index c7bb320253a..be34ac27e25 100644
--- a/sdks/python/apache_beam/internal/module_test.py
+++ b/sdks/python/apache_beam/internal/module_test.py
@@ -61,3 +61,15 @@ def get(self):
XYZ_OBJECT = Xyz()
+
+
+class RecursiveClass(object):
+ """A class that contains a reference to itself."""
+
+ SELF_TYPE = None
+
+ def __init__(self, datum):
+ self.datum = 'RecursiveClass:%s' % datum
+
+
+RecursiveClass.SELF_TYPE = RecursiveClass
diff --git a/sdks/python/apache_beam/internal/pickler.py
b/sdks/python/apache_beam/internal/pickler.py
index 102cf23877d..3e4fe05974b 100644
--- a/sdks/python/apache_beam/internal/pickler.py
+++ b/sdks/python/apache_beam/internal/pickler.py
@@ -46,9 +46,14 @@ def _is_nested_class(cls):
def _find_containing_class(nested_class):
- """Finds containing class of a nestec class passed as argument."""
+ """Finds containing class of a nested class passed as argument."""
+
+ seen = set()
def _find_containing_class_inner(outer):
+ if outer in seen:
+ return None
+ seen.add(outer)
for k, v in outer.__dict__.items():
if v is nested_class:
return outer, k
diff --git a/sdks/python/apache_beam/internal/pickler_test.py
b/sdks/python/apache_beam/internal/pickler_test.py
index 05062d282da..e75dac38462 100644
--- a/sdks/python/apache_beam/internal/pickler_test.py
+++ b/sdks/python/apache_beam/internal/pickler_test.py
@@ -80,6 +80,10 @@ def test_generators(self):
with self.assertRaises(TypeError):
dumps((_ for _ in range(10)))
+ def test_recursive_class(self):
+ self.assertEquals('RecursiveClass:abc',
+ loads(dumps(module_test.RecursiveClass('abc').datum)))
+
if __name__ == '__main__':
unittest.main()
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
-------------------
Worklog Id: (was: 89714)
Time Spent: 1h 10m (was: 1h)
> Pickler enters infinite recursion with self-referential classes
> ---------------------------------------------------------------
>
> Key: BEAM-4036
> URL: https://issues.apache.org/jira/browse/BEAM-4036
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chuan Yu Foo
> Assignee: Ahmet Altay
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> The pickler recurses infinitely and dies with maximum recursion limit
> exceeded when a module contains a self-referential class (or any class which
> is part of a cycle).
> Here's a minimal example:
> {code}
> class RecursiveClass(object):
> SELF_TYPE = None
> def __init__(self, datum)
> self.datum = 'RecursiveClass:%s' % datum
> RecursiveClass.SELF_TYPE = RecursiveClass
> {code}
> If this is in a module, then the pickler will enter the infinite recursion
> when trying to pickle any nested class in that module.
>
> An actual example is with typing.Type, which is part of a cycle typing.Type
> -> type -> object -> typing.Type. If a module contains an attribute that
> refers to typing.Type, such as a type alias, it will trigger this bug.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
