[jira] [Closed] (BEAM-3731) Enable tests to run in Python 3
[ https://issues.apache.org/jira/browse/BEAM-3731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3731. -- Resolution: Duplicate Fix Version/s: Not applicable > Enable tests to run in Python 3 > --- > > Key: BEAM-3731 > URL: https://issues.apache.org/jira/browse/BEAM-3731 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: Not applicable > > Time Spent: 4h 20m > Remaining Estimate: 0h > > Currently the Python3 tests fail to run. This makes it difficult to continue > with Python 3 compatibility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3745) isort and pylint conflicting rules
Luke Zhu created BEAM-3745: -- Summary: isort and pylint conflicting rules Key: BEAM-3745 URL: https://issues.apache.org/jira/browse/BEAM-3745 Project: Beam Issue Type: Improvement Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay isort places "import ..." statements before "from ... import ..." statements. This sometimes triggers the ungrouped-imports rule in pylint. Since isort cannot be configured, the easiest way to resolve this would be to disable the pylint rule. It would also be good to remove the respective inline pylint comments as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-3724) Make the coders package compatible with Python 3
[ https://issues.apache.org/jira/browse/BEAM-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu resolved BEAM-3724. Resolution: Duplicate Fix Version/s: Not applicable > Make the coders package compatible with Python 3 > > > Key: BEAM-3724 > URL: https://issues.apache.org/jira/browse/BEAM-3724 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: Not applicable > > > The coders package is affect a lot by the fact that Strings are unicode in > Python 3. > > The planned approach is to > * Prefix bytestrings with 'b' where appropriate > * Replace uses of 'str' with 'bytes' where appropriate > * Use python-modernize to solve syntax and import errors > The goal of this subtask is not to make the coders package completely > compatible with Python 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3731) Enable tests to run in Python 3
Luke Zhu created BEAM-3731: -- Summary: Enable tests to run in Python 3 Key: BEAM-3731 URL: https://issues.apache.org/jira/browse/BEAM-3731 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Luke Zhu Currently the Python3 tests fail to run. This makes it difficult to continue with Python 3 compatibility. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3730) typehints.TypeVariable issues with __hash__
Luke Zhu created BEAM-3730: -- Summary: typehints.TypeVariable issues with __hash__ Key: BEAM-3730 URL: https://issues.apache.org/jira/browse/BEAM-3730 Project: Beam Issue Type: Bug Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay This class currently does not define a __hash__ function. This defies the assumption made by sets and dicts which contain TypeConstraint objects. In Python 3, classes which define ___eq___ also need to explicitly define ___hash___ to be hashable. However, this causes _combiners_test.CombineTest.test_to_list_and_to_dict_ to throw an error in slow_stream.py (adding NoneType and long). There are multiple TypeVariable instances with name 'K' or 'V' throughout the codebase. The equality of these instances may possibly cause issues. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-2304) State declared with one class cannot be accessed as a superclass (applies to BagState|CombiningState <: GroupingState)
[
https://issues.apache.org/jira/browse/BEAM-2304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luke Zhu resolved BEAM-2304.
Resolution: Fixed
Fix Version/s: 2.3.0
> State declared with one class cannot be accessed as a superclass (applies to
> BagState|CombiningState <: GroupingState)
> --
>
> Key: BEAM-2304
> URL: https://issues.apache.org/jira/browse/BEAM-2304
> Project: Beam
> Issue Type: Bug
> Components: sdk-java-core
>Reporter: Reuven Lax
>Assignee: Luke Zhu
>Priority: Major
> Labels: starter
> Fix For: 2.3.0
>
>
> The following code:
> {code}
> @StateId("foo")
> private final StateSpec>
> state =
> StateSpecs.combining(Sum.ofIntegers());
> @ProcessElement
> public void processElement(ProcessContext c,
>@StateId("foo") GroupingState Integer> state) {
> }
> {code}
> Fails with:
> {code}
> parameter of type GroupingState at index 1: reference to
> StateId exists with different type CombiningState
> {code}
> However since GroupingState is the base class, ideally this
> should work - and would make the API easier to use if it did.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (BEAM-3724) Make the coders package compatible with Python 3
[ https://issues.apache.org/jira/browse/BEAM-3724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu reassigned BEAM-3724: -- Assignee: Luke Zhu (was: Ahmet Altay) > Make the coders package compatible with Python 3 > > > Key: BEAM-3724 > URL: https://issues.apache.org/jira/browse/BEAM-3724 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > > The coders package is affect a lot by the fact that Strings are unicode in > Python 3. > > The planned approach is to > * Prefix bytestrings with 'b' where appropriate > * Replace uses of 'str' with 'bytes' where appropriate > * Use python-modernize to solve syntax and import errors > The goal of this subtask is not to make the coders package completely > compatible with Python 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3724) Make the coders package compatible with Python 3
Luke Zhu created BEAM-3724: -- Summary: Make the coders package compatible with Python 3 Key: BEAM-3724 URL: https://issues.apache.org/jira/browse/BEAM-3724 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay The coders package is affect a lot by the fact that Strings are unicode in Python 3. The planned approach is to * Prefix bytestrings with 'b' where appropriate * Replace uses of 'str' with 'bytes' where appropriate * Use python-modernize to solve syntax and import errors The goal of this subtask is not to make the coders package completely compatible with Python 3. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3723) Duration, Timestamp, and Window in utils/ are incompatible with Python 3
Luke Zhu created BEAM-3723: -- Summary: Duration, Timestamp, and Window in utils/ are incompatible with Python 3 Key: BEAM-3723 URL: https://issues.apache.org/jira/browse/BEAM-3723 Project: Beam Issue Type: Bug Components: sdk-py-core Reporter: Luke Zhu They use __cmp__ instead of rich comparison operators __eq__, __lt__, etc. This can be fixed by updating the Cython version (0.27.1 supports rich comparions), or possibly by configuring cythonize. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-3552) Support Python 3 in the smaller modules
[ https://issues.apache.org/jira/browse/BEAM-3552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3552. -- Resolution: Fixed Fix Version/s: 2.4.0 > Support Python 3 in the smaller modules > --- > > Key: BEAM-3552 > URL: https://issues.apache.org/jira/browse/BEAM-3552 > Project: Beam > Issue Type: Improvement > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: 2.4.0 > > Time Spent: 50m > Remaining Estimate: 0h > > Since most problems with the Python 2 to 3 conversion will occur in larger > modules like transforms, io, and coders, converting their dependencies first > should speed up the process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-3555) Support Python 3 in the typehints module
[ https://issues.apache.org/jira/browse/BEAM-3555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3555. -- Resolution: Duplicate Fix Version/s: 2.4.0 > Support Python 3 in the typehints module > > > Key: BEAM-3555 > URL: https://issues.apache.org/jira/browse/BEAM-3555 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: 2.4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-3554) Support python 3 in internal
[ https://issues.apache.org/jira/browse/BEAM-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3554. -- Resolution: Duplicate Fix Version/s: 2.4.0 > Support python 3 in internal > > > Key: BEAM-3554 > URL: https://issues.apache.org/jira/browse/BEAM-3554 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: 2.4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-3553) Support Python 3 in metrics
[ https://issues.apache.org/jira/browse/BEAM-3553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3553. -- Resolution: Fixed Fix Version/s: 2.4.0 > Support Python 3 in metrics > --- > > Key: BEAM-3553 > URL: https://issues.apache.org/jira/browse/BEAM-3553 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: 2.4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-3556) Support Python 3 in the utils module
[ https://issues.apache.org/jira/browse/BEAM-3556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu closed BEAM-3556. -- Resolution: Duplicate Fix Version/s: 2.4.0 > Support Python 3 in the utils module > > > Key: BEAM-3556 > URL: https://issues.apache.org/jira/browse/BEAM-3556 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > Fix For: 2.4.0 > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3555) Support Python 3 in the typehints module
Luke Zhu created BEAM-3555: -- Summary: Support Python 3 in the typehints module Key: BEAM-3555 URL: https://issues.apache.org/jira/browse/BEAM-3555 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Luke Zhu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3556) Support Python 3 in the utils module
Luke Zhu created BEAM-3556: -- Summary: Support Python 3 in the utils module Key: BEAM-3556 URL: https://issues.apache.org/jira/browse/BEAM-3556 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Luke Zhu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-3554) Support python 3 in internal
[ https://issues.apache.org/jira/browse/BEAM-3554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luke Zhu reassigned BEAM-3554: -- Assignee: Luke Zhu (was: Ahmet Altay) > Support python 3 in internal > > > Key: BEAM-3554 > URL: https://issues.apache.org/jira/browse/BEAM-3554 > Project: Beam > Issue Type: Sub-task > Components: sdk-py-core >Reporter: Luke Zhu >Assignee: Luke Zhu >Priority: Major > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3553) Support Python 3 in metrics
Luke Zhu created BEAM-3553: -- Summary: Support Python 3 in metrics Key: BEAM-3553 URL: https://issues.apache.org/jira/browse/BEAM-3553 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Luke Zhu -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3554) Support python 3 in internal
Luke Zhu created BEAM-3554: -- Summary: Support python 3 in internal Key: BEAM-3554 URL: https://issues.apache.org/jira/browse/BEAM-3554 Project: Beam Issue Type: Sub-task Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3552) Support Python 3 in the smaller modules
Luke Zhu created BEAM-3552: -- Summary: Support Python 3 in the smaller modules Key: BEAM-3552 URL: https://issues.apache.org/jira/browse/BEAM-3552 Project: Beam Issue Type: Improvement Components: sdk-py-core Reporter: Luke Zhu Assignee: Luke Zhu Since most problems with the Python 2 to 3 conversion will occur in larger modules like transforms, io, and coders, converting their dependencies first should speed up the process. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-3381) Handle trivial inference with default arguments
Luke Zhu created BEAM-3381: -- Summary: Handle trivial inference with default arguments Key: BEAM-3381 URL: https://issues.apache.org/jira/browse/BEAM-3381 Project: Beam Issue Type: Bug Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay Priority: Minor make_function in opcodes.py was updated to make for-comprehensions compatible with Python 3. Currently, functions that have default arguments are unsupported. Default arguments will be left on the stack, affecting how future instructions are processed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (BEAM-3380) Make type inference compatible with Python 3.6
Luke Zhu created BEAM-3380: -- Summary: Make type inference compatible with Python 3.6 Key: BEAM-3380 URL: https://issues.apache.org/jira/browse/BEAM-3380 Project: Beam Issue Type: Task Components: sdk-py-core Reporter: Luke Zhu Assignee: Ahmet Altay Priority: Minor Starting in Python 3.6, instructions always use two bytes. As the Python SDKs implementation of trivial type inference relies on these disassembled instructions, typehints/trivial_inference.py should be updated to be compatible with this change. This can be done by comparing https://github.com/python/cpython/blob/3.6/Lib/dis.py to https://github.com/python/cpython/blob/3.5/Lib/dis.py -- This message was sent by Atlassian JIRA (v6.4.14#64029)
