[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=154046=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154046
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Oct/18 22:00
Start Date: 12/Oct/18 22:00
Worklog Time Spent: 10m 
  Work Description: swegner commented on issue #6679: [BEAM-1251] Add a 
link to Python 3 Conversion Quick Start Guide to the list of ongoing efforts on 
Beam site.
URL: https://github.com/apache/beam/pull/6679#issuecomment-429474553
 
 
   Yup! 
   
   If you pop open the "All checks have passed" bar, you'll see a link to the 
"Website_Stage_GCS" job 
[results](https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/79/),
 which contains a link to your staged changes for review: 
http://apache-beam-website-pull-requests.storage.googleapis.com/6679/index.html
   
   (I'm brainstorming a way to make that link more prominent on GitHub; let me 
know if you have ideas)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 154046)
Time Spent: 22h 50m  (was: 22h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 22h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=154015=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154015
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Oct/18 20:37
Start Date: 12/Oct/18 20:37
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #6679: [BEAM-1251] Add a 
link to Python 3 Conversion Quick Start Guide to the list of ongoing efforts on 
Beam site.
URL: https://github.com/apache/beam/pull/6679#issuecomment-429455025
 
 
   Hey @swegner, am I using a correct way to change the Beam-site? Thanks. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 154015)
Time Spent: 22h 40m  (was: 22.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 22h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=154014=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154014
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Oct/18 20:34
Start Date: 12/Oct/18 20:34
Worklog Time Spent: 10m 
  Work Description: tvalentyn opened a new pull request #6679: [BEAM-1251] 
Add a link to Python 3 Conversion Quick Start Guide to the list of ongoing 
efforts on Beam site.
URL: https://github.com/apache/beam/pull/6679
 
 
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 154014)
Time Spent: 22.5h  (was: 22h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=154013=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154013
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Oct/18 20:32
Start Date: 12/Oct/18 20:32
Worklog Time Spent: 10m 
  Work Description: tvalentyn opened a new pull request #6678: [BEAM-1251] 
Add a link to Python 3 Conversion Quick Start Guide to the list of ongoing 
efforts on Beam site.
URL: https://github.com/apache/beam/pull/6678
 
 
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 154013)
Time Spent: 22h 20m  (was: 22h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=153574=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153574
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 11/Oct/18 16:47
Start Date: 11/Oct/18 16:47
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #6650: [BEAM-1251] Make it 
possible to unskip Py3 tests by setting an environment variable.
URL: https://github.com/apache/beam/pull/6650#issuecomment-429030972
 
 
   Thanks for a quick review, @manuzhang.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 153574)
Time Spent: 22h 10m  (was: 22h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 22h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=153408=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153408
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 11/Oct/18 05:40
Start Date: 11/Oct/18 05:40
Worklog Time Spent: 10m 
  Work Description: manuzhang closed pull request #6650: [BEAM-1251] Make 
it possible to unskip Py3 tests by setting an environment variable.
URL: https://github.com/apache/beam/pull/6650
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/pipeline_test.py 
b/sdks/python/apache_beam/pipeline_test.py
index 6397820e6e9..41f749d2f79 100644
--- a/sdks/python/apache_beam/pipeline_test.py
+++ b/sdks/python/apache_beam/pipeline_test.py
@@ -21,6 +21,7 @@
 
 import copy
 import logging
+import os
 import platform
 import sys
 import unittest
@@ -391,8 +392,9 @@ def process(self, element):
 assert_that(pcoll, equal_to([11, 12]))
 pipeline.run()
 
-  @unittest.skipIf(sys.version_info[0] == 3, 'This test still needs to be '
- 'fixed on Python 3')
+  @unittest.skipIf(sys.version_info[0] == 3 and
+   os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
+   'This test still needs to be fixed on Python 3.')
   def test_side_input_no_tag(self):
 class TestDoFn(DoFn):
   def process(self, element, prefix, suffix):
@@ -408,8 +410,9 @@ def process(self, element, prefix, suffix):
 assert_that(result, equal_to(['zyx-%s-xyz' % x for x in words_list]))
 pipeline.run()
 
-  @unittest.skipIf(sys.version_info[0] == 3, 'This test still needs to be '
- 'fixed on Python 3')
+  @unittest.skipIf(sys.version_info[0] == 3 and
+   os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
+   'This test still needs to be fixed on Python 3.')
   def test_side_input_tagged(self):
 class TestDoFn(DoFn):
   def process(self, element, prefix, suffix=DoFn.SideInputParam):
diff --git 
a/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py 
b/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py
index b21135c8717..767e06e527f 100644
--- a/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py
+++ b/sdks/python/apache_beam/runners/interactive/interactive_runner_test.py
@@ -24,6 +24,7 @@
 from __future__ import division
 from __future__ import print_function
 
+import os
 import sys
 import unittest
 
@@ -43,8 +44,9 @@ def printer(elem):
 
 class InteractiveRunnerTest(unittest.TestCase):
 
-  @unittest.skipIf(sys.version_info[0] == 3, 'This test still needs to be '
- 'fixed on Python 3')
+  @unittest.skipIf(sys.version_info[0] == 3 and
+   os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
+   'This test still needs to be fixed on Python 3.')
   def test_basic(self):
 p = beam.Pipeline(
 runner=interactive_runner.InteractiveRunner(
@@ -60,8 +62,9 @@ def test_basic(self):
 _ = pc0 | 'Print3' >> beam.Map(print_with_message('Run3'))
 p.run().wait_until_finish()
 
-  @unittest.skipIf(sys.version_info[0] == 3, 'This test still needs to be '
- 'fixed on Python 3')
+  @unittest.skipIf(sys.version_info[0] == 3 and
+   os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
+   'This test still needs to be fixed on Python 3.')
   def test_wordcount(self):
 
 class WordExtractingDoFn(beam.DoFn):
diff --git 
a/sdks/python/apache_beam/runners/interactive/pipeline_analyzer_test.py 
b/sdks/python/apache_beam/runners/interactive/pipeline_analyzer_test.py
index f2d82cd3288..caefbe04ba6 100644
--- a/sdks/python/apache_beam/runners/interactive/pipeline_analyzer_test.py
+++ b/sdks/python/apache_beam/runners/interactive/pipeline_analyzer_test.py
@@ -24,6 +24,7 @@
 from __future__ import division
 from __future__ import print_function
 
+import os
 import sys
 import unittest
 
@@ -87,8 +88,9 @@ def assertTransformEqual(self, pipeline_proto1, transform_id1,
 self.assertSetEqual(set(transform_proto1.outputs),
 set(transform_proto2.outputs))
 
-  @unittest.skipIf(sys.version_info[0] == 3, 'This test still needs to be '
- 'fixed on Python 3')
+  @unittest.skipIf(sys.version_info[0] == 3 and
+   os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
+   'This test still needs to be fixed on Python 3.')
   def test_basic(self):
 p = 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=153407=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153407
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 11/Oct/18 05:40
Start Date: 11/Oct/18 05:40
Worklog Time Spent: 10m 
  Work Description: manuzhang commented on issue #6650: [BEAM-1251] Make it 
possible to unskip Py3 tests by setting an environment variable.
URL: https://github.com/apache/beam/pull/6650#issuecomment-428827423
 
 
   LGTM


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 153407)
Time Spent: 21h 50m  (was: 21h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 21h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=153401=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153401
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 11/Oct/18 04:06
Start Date: 11/Oct/18 04:06
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #6650: [BEAM-1251] Make it 
possible to unskip Py3 tests by setting an environment variable.
URL: https://github.com/apache/beam/pull/6650#issuecomment-428813596
 
 
   cc: @manuzhang @Juta @Fematich @splovyt  


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 153401)
Time Spent: 21h 40m  (was: 21.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 21h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=153373=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153373
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 11/Oct/18 02:47
Start Date: 11/Oct/18 02:47
Worklog Time Spent: 10m 
  Work Description: tvalentyn opened a new pull request #6650: [BEAM-1251] 
Make it possible to unskip Py3 tests by setting an environment variable.
URL: https://github.com/apache/beam/pull/6650
 
 
   Make it possible to run tests that are skipped in Python 3 by setting an 
environment variable RUN_SKIPPED_PY3_TESTS=1. This will make it a little easier 
to unskip the tests for developer testing and to run entire test suite to see 
how many tests are still failing in Python 3.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 153373)
Time Spent: 21.5h  (was: 21h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147747=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147747
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 19:55
Start Date: 25/Sep/18 19:55
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #6489: [BEAM-1251] Upgrade 
pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/examples/complete/distribopt.py 
b/sdks/python/apache_beam/examples/complete/distribopt.py
index f79489516e9..a16a40b6137 100644
--- a/sdks/python/apache_beam/examples/complete/distribopt.py
+++ b/sdks/python/apache_beam/examples/complete/distribopt.py
@@ -211,12 +211,12 @@ def process(self, element, quantities):
 
   # Create (crop, quantity) lists for each greenhouse
   greenhouses = defaultdict(list)
-  for crop, greenhouse in mapping.iteritems():
+  for crop, greenhouse in mapping.items():
 quantity = quantities[crop]
 greenhouses[greenhouse].append((crop, quantity))
 
   # Create input for OptimizeProductParameters
-  for greenhouse, crops in greenhouses.iteritems():
+  for greenhouse, crops in greenhouses.items():
 key = (mapping_identifier, greenhouse)
 yield (key, crops)
 
diff --git a/sdks/python/apache_beam/io/vcfio.py 
b/sdks/python/apache_beam/io/vcfio.py
index d96c8f79d65..abe33fc0dc4 100644
--- a/sdks/python/apache_beam/io/vcfio.py
+++ b/sdks/python/apache_beam/io/vcfio.py
@@ -321,6 +321,7 @@ def _create_generator(self):
 def __iter__(self):
   return self
 
+# pylint: disable=next-method-defined
 def next(self):
   return self.__next__()
 
diff --git a/sdks/python/apache_beam/options/pipeline_options.py 
b/sdks/python/apache_beam/options/pipeline_options.py
index c1bb2ed89e9..7da91a668da 100644
--- a/sdks/python/apache_beam/options/pipeline_options.py
+++ b/sdks/python/apache_beam/options/pipeline_options.py
@@ -240,7 +240,7 @@ def _visible_option_list(self):
   for option in dir(self._visible_options) if option[0] != '_')
 
   def __dir__(self):
-return sorted(dir(type(self)) + list(self.__dict__.keys()) +
+return sorted(dir(type(self)) + list(self.__dict__) +
   self._visible_option_list())
 
   def __getattr__(self, name):
diff --git a/sdks/python/apache_beam/runners/direct/evaluation_context.py 
b/sdks/python/apache_beam/runners/direct/evaluation_context.py
index 01d0631c5ee..24b05b61527 100644
--- a/sdks/python/apache_beam/runners/direct/evaluation_context.py
+++ b/sdks/python/apache_beam/runners/direct/evaluation_context.py
@@ -82,7 +82,7 @@ def __init__(self, side_inputs):
 
   def __repr__(self):
 views_string = (', '.join(str(elm) for elm in self._views.values())
-if self._views.values() else '[]')
+if self._views else '[]')
 return '_SideInputsContainer(_views=%s)' % views_string
 
   def get_value_or_block_until_ready(self, side_input, task, block_until):
diff --git a/sdks/python/apache_beam/runners/interactive/cache_manager.py 
b/sdks/python/apache_beam/runners/interactive/cache_manager.py
index a9dd03d6455..d0c0b75716c 100644
--- a/sdks/python/apache_beam/runners/interactive/cache_manager.py
+++ b/sdks/python/apache_beam/runners/interactive/cache_manager.py
@@ -191,6 +191,7 @@ def expand(self, pcoll):
 
 class SafeFastPrimitivesCoder(coders.Coder):
   """This class add an quote/unquote step to escape special characters."""
+  # pylint: disable=deprecated-urllib-function
 
   def encode(self, value):
 return urllib.quote(coders.coders.FastPrimitivesCoder().encode(value))
diff --git a/sdks/python/apache_beam/runners/interactive/pipeline_analyzer.py 
b/sdks/python/apache_beam/runners/interactive/pipeline_analyzer.py
index 1f597810270..b0cf1342c46 100644
--- a/sdks/python/apache_beam/runners/interactive/pipeline_analyzer.py
+++ b/sdks/python/apache_beam/runners/interactive/pipeline_analyzer.py
@@ -116,7 +116,7 @@ def _analyze_pipeline(self):
 sample=True)
 
 required_transforms['_root'] = beam_runner_api_pb2.PTransform(
-subtransforms=top_level_required_transforms.keys())
+subtransforms=list(top_level_required_transforms.keys()))
 
 referenced_pcollection_ids = self._referenced_pcollection_ids(
 required_transforms)
diff --git a/sdks/python/apache_beam/runners/portability/fn_api_runner.py 
b/sdks/python/apache_beam/runners/portability/fn_api_runner.py
index 2fc92e887bf..1ea7b42be6a 100644
--- 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147730=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147730
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 19:06
Start Date: 25/Sep/18 19:06
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders commented on a change in pull request 
#6489: [BEAM-1251] Upgrade pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489#discussion_r220317601
 
 

 ##
 File path: sdks/python/apache_beam/runners/direct/evaluation_context.py
 ##
 @@ -81,6 +81,7 @@ def __init__(self, side_inputs):
   self._transform_to_side_inputs[side.pvalue.producer].append(side)
 
   def __repr__(self):
+# pylint: disable=dict-values-not-iterating
 
 Review comment:
   You're right. I've removed the `values()` call.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 147730)
Time Spent: 21h 10m  (was: 21h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 21h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147705=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147705
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 18:19
Start Date: 25/Sep/18 18:19
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on a change in pull request #6489: 
[BEAM-1251] Upgrade pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489#discussion_r220301977
 
 

 ##
 File path: sdks/python/apache_beam/runners/direct/evaluation_context.py
 ##
 @@ -81,6 +81,7 @@ def __init__(self, side_inputs):
   self._transform_to_side_inputs[side.pvalue.producer].append(side)
 
   def __repr__(self):
+# pylint: disable=dict-values-not-iterating
 
 Review comment:
   Would s/`if self._views.values()`/`if self._views`/ remove the lint warning?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 147705)
Time Spent: 21h  (was: 20h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 21h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147665=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147665
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 16:53
Start Date: 25/Sep/18 16:53
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders commented on a change in pull request 
#6489: [BEAM-1251] Upgrade pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489#discussion_r220272307
 
 

 ##
 File path: sdks/python/apache_beam/typehints/trivial_inference.py
 ##
 @@ -55,6 +55,7 @@ def instance_to_type(o):
   if o is None:
 return type(None)
   elif t not in typehints.DISALLOWED_PRIMITIVE_TYPES:
+# pylint: disable=deprecated-types-field
 
 Review comment:
   This is needed for old-style Python 2 classes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 147665)
Time Spent: 20h 50m  (was: 20h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147664=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147664
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 16:53
Start Date: 25/Sep/18 16:53
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders commented on a change in pull request 
#6489: [BEAM-1251] Upgrade pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489#discussion_r220272181
 
 

 ##
 File path: sdks/python/apache_beam/runners/interactive/cache_manager.py
 ##
 @@ -191,6 +191,7 @@ def expand(self, pcoll):
 
 class SafeFastPrimitivesCoder(coders.Coder):
   """This class add an quote/unquote step to escape special characters."""
+  # pylint: disable=deprecated-urllib-function
 
 Review comment:
   This code has already been changed in #6451 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 147664)
Time Spent: 20h 40m  (was: 20.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=147663=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-147663
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 25/Sep/18 16:52
Start Date: 25/Sep/18 16:52
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders opened a new pull request #6489: 
[BEAM-1251] Upgrade pylint version for py27-lint3
URL: https://github.com/apache/beam/pull/6489
 
 
   @tvalentyn @Fematich @aaltay
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 147663)
Time Spent: 20.5h  (was: 20h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=145764=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-145764
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 19/Sep/18 18:45
Start Date: 19/Sep/18 18:45
Worklog Time Spent: 10m 
  Work Description: swegner commented on a change in pull request #5842: 
[BEAM-1251] Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#discussion_r218921060
 
 

 ##
 File path: website/.jenkins/append_index_html_to_internal_links.py
 ##
 @@ -29,12 +29,18 @@
   'sudo apt-get install python-beautifulsoup4'.
 
 """
+from __future__ import print_function
 
 Review comment:
   The website source code is currently being migrated from 
https://github.com/apache/beam-site, but is not yet ready. Website changes in 
apache/beam will be overwritten on next merge. Please contribute changes at 
apache/beam-site according to the [website contribution 
guide](https://beam.apache.org/contribute/#contributing-to-the-website). You 
can track migration progress via 
[[BEAM-4493]](https://issues.apache.org/jira/browse/BEAM-4493).
   
   Do you know if this change was also made in the apache/beam-site repository? 
If not, it will need to be migrated.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 145764)
Time Spent: 20h 20m  (was: 20h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=136068=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136068
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 20/Aug/18 02:36
Start Date: 20/Aug/18 02:36
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414181430
 
 
   The first three files contained _print_ statements in live code.  These 
files are in __.test-infra/jenkins__ which flake8 does not currently test.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 136068)
Time Spent: 20h 10m  (was: 20h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=136060=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136060
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 20/Aug/18 01:06
Start Date: 20/Aug/18 01:06
Worklog Time Spent: 10m 
  Work Description: tvalentyn edited a comment on issue #6166: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414170979
 
 
   `utils` module is included py3 lint checks.
   
https://github.com/apache/beam/blob/f31b789d591f7b8f6f96899a79fb6edf45ebcd34/sdks/python/tox.ini#L110
   I think the tools did not catch it because the occurrences are in the 
comment.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 136060)
Time Spent: 20h  (was: 19h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 20h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=136059=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136059
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 20/Aug/18 01:06
Start Date: 20/Aug/18 01:06
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414170979
 
 
   `utils` method is included py3 lint checks.
   
https://github.com/apache/beam/blob/f31b789d591f7b8f6f96899a79fb6edf45ebcd34/sdks/python/tox.ini#L110
   I think the tools did not catch it because the occurrences are in the 
comment.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 136059)
Time Spent: 19h 50m  (was: 19h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 19h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=136011=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136011
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 19/Aug/18 00:15
Start Date: 19/Aug/18 00:15
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414094137
 
 
   @holdenk This is merged now but I am wondering how the testing did not catch 
this and how we can prevent similar regression in the future.  
https://github.com/apache/beam/blob/master/sdks/python/scripts/run_mini_py3lint.sh#L51
 should be able to spot these as syntax errors but perhaps it is only look at a 
portion of the codebase instead of looking at all Python files in the repo.  On 
Travis, it takes 20 sec to do this flake8 test on the entire repo.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 136011)
Time Spent: 19h 40m  (was: 19.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 19h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=136006=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-136006
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Aug/18 21:49
Start Date: 18/Aug/18 21:49
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #6166: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/6166
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git 
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
 
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
index b89907581f5..b4281cc5dd3 100644
--- 
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
+++ 
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
@@ -19,6 +19,7 @@
 #   This script performs testing of scenarios from 
verify_performance_test_results.py
 #
 
+from __future__ import print_function
 import unittest, mock
 from mock import patch, mock_open
 from datetime import datetime
@@ -44,7 +45,7 @@ class DependencyCheckReportGeneratorTest(unittest.TestCase):
   """Tests for `dependency_check_report_generator.py`."""
 
   def setUp(self):
-print "\n\nTest : " + self._testMethodName
+print("\n\nTest : " + self._testMethodName)
 
 
   @patch('dependency_check.bigquery_client_utils.BigQueryClientUtils')
@@ -134,4 +135,4 @@ def test_invalid_dep_input(self, *args):
 
 if __name__ == '__main__':
   unittest.main()
-  
\ No newline at end of file
+  
diff --git a/.test-infra/jenkins/jira_utils/jira_manager.py 
b/.test-infra/jenkins/jira_utils/jira_manager.py
index cdc4450d106..70daf0cd44c 100644
--- a/.test-infra/jenkins/jira_utils/jira_manager.py
+++ b/.test-infra/jenkins/jira_utils/jira_manager.py
@@ -1,3 +1,4 @@
+from __future__ import print_function
 #
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
@@ -66,7 +67,7 @@ def run(self, dep_name, dep_latest_version, sdk_type, 
group_id=None):
 Created a parent issue for {1}""".format(summary, group_id))
   try:
 parent_issue = self._create_issue(group_id, None)
-print parent_issue.key
+print(parent_issue.key)
   except:
 logging.error("""Failed creating a parent issue for {0}.
   Stop handling the JIRA issue for {1}, {2}""".format(group_id, 
dep_name, dep_latest_version))
diff --git a/.test-infra/jenkins/jira_utils/jira_manager_test.py 
b/.test-infra/jenkins/jira_utils/jira_manager_test.py
index 15677d08bb3..dcf11a69e5c 100644
--- a/.test-infra/jenkins/jira_utils/jira_manager_test.py
+++ b/.test-infra/jenkins/jira_utils/jira_manager_test.py
@@ -1,3 +1,4 @@
+from __future__ import print_function
 #
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
@@ -42,7 +43,7 @@ class JiraManagerTest(unittest.TestCase):
   """Tests for `jira_manager.py`."""
 
   def setUp(self):
-print "\n\nTest : " + self._testMethodName
+print("\n\nTest : " + self._testMethodName)
 
 
   def test_find_owners_with_single_owner(self, *args):
@@ -240,4 +241,3 @@ def _get_expected_description(self, dep_name, 
dep_latest_version, owners):
 
 if __name__ == '__main__':
   unittest.main()
-
diff --git a/sdks/python/apache_beam/io/tfrecordio_test.py 
b/sdks/python/apache_beam/io/tfrecordio_test.py
index ded8d794159..dfc660d2ec0 100644
--- a/sdks/python/apache_beam/io/tfrecordio_test.py
+++ b/sdks/python/apache_beam/io/tfrecordio_test.py
@@ -62,7 +62,7 @@
 # >>> writer.close()
 # >>> with open('/tmp/python_foo.tfrecord', 'rb') as f:
 # ...   data =  base64.b64encode(f.read())
-# ...   print data
+# ...   print(data)
 FOO_RECORD_BASE64 = 'AwCwmUkOZm9vYYq+/g=='
 
 # Same as above but containing two records ['foo', 'bar']
diff --git a/sdks/python/apache_beam/utils/annotations.py 
b/sdks/python/apache_beam/utils/annotations.py
index 7ec7150d174..29121329fc2 100644
--- a/sdks/python/apache_beam/utils/annotations.py
+++ b/sdks/python/apache_beam/utils/annotations.py
@@ -41,7 +41,7 @@
 same function 'multiply'.::
 
   def multiply(arg1, arg2):
-print arg1, '*', arg2, '=',
+print(arg1, '*', arg2, '=', end=' ')
 return arg1*arg2
 
 # This annotation marks 'old_multiply' as deprecated since 'v.1' and suggests
@@ -52,7 +52,7 @@ def old_multiply(arg1, arg2):
 result = 0
 for i in xrange(arg1):
 result += arg2
-print arg1, '*', arg2, 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=135997=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-135997
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Aug/18 16:25
Start Date: 18/Aug/18 16:25
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414069793
 
 
   @pabloem Your review please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 135997)
Time Spent: 19h 20m  (was: 19h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 19h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=135944=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-135944
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Aug/18 00:50
Start Date: 18/Aug/18 00:50
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-414020142
 
 
   @charlesccychen @holdenk Your reviews please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 135944)
Time Spent: 19h 10m  (was: 19h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 19h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=132376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-132376
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Aug/18 13:43
Start Date: 08/Aug/18 13:43
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #6166: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-411409828
 
 
   Thanks @superbobry, those instances were in comments, not in live code but I 
have converted them for completeness.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 132376)
Time Spent: 19h  (was: 18h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 19h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=132371=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-132371
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Aug/18 13:13
Start Date: 08/Aug/18 13:13
Worklog Time Spent: 10m 
  Work Description: superbobry commented on issue #6166: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/6166#issuecomment-411400392
 
 
   There seem to be two more occurences 
   
   ```
   sdks/python/apache_beam/io/tfrecordio_test.py
   65:# ...   print data
   
   sdks/python/apache_beam/utils/annotations.py
   44:print arg1, '*', arg2, '=',
   55:print arg1, '*', arg2, '(the old way)=',
   63:print arg1, '*', arg2, '(the experimental way)=',
   69:  print multiply(5, 6)
   70:  print old_multiply(5,6)
   71:  print exp_multiply(5,6)
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 132371)
Time Spent: 18h 50m  (was: 18h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 18h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=132014=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-132014
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 07/Aug/18 19:01
Start Date: 07/Aug/18 19:01
Worklog Time Spent: 10m 
  Work Description: cclauss opened a new pull request #6166: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/6166
 
 
   __print()__ is a function in Python 3  # just like #4784, #5531, and #5890
   
   @charlesccychen @holdenk @superbobry 
   
   **Please** add a meaningful description for your change here
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 132014)
Time Spent: 18h 40m  (was: 18.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Major
>  Time Spent: 18h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=122482=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122482
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Jul/18 19:46
Start Date: 12/Jul/18 19:46
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #5869: [BEAM-1251] Replace 
NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/coders/coder_impl.py 
b/sdks/python/apache_beam/coders/coder_impl.py
index 4a65c69f4e0..6fd9b169ed6 100644
--- a/sdks/python/apache_beam/coders/coder_impl.py
+++ b/sdks/python/apache_beam/coders/coder_impl.py
@@ -37,6 +37,9 @@
 from builtins import chr
 from builtins import object
 
+from past.builtins import long
+from past.builtins import unicode
+
 from apache_beam.coders import observable
 from apache_beam.utils import windowed_value
 from apache_beam.utils.timestamp import MAX_TIMESTAMP
@@ -61,13 +64,6 @@
   from .slow_stream import get_varint_size
 # pylint: enable=wrong-import-order, wrong-import-position, ungrouped-imports
 
-try:# Python 2
-  long  # pylint: disable=long-builtin
-  unicode   # pylint: disable=unicode-builtin
-except NameError:   # Python 3
-  long = int
-  unicode = str
-
 
 class CoderImpl(object):
   """For internal use only; no backwards-compatibility guarantees."""
diff --git a/sdks/python/apache_beam/coders/coders.py 
b/sdks/python/apache_beam/coders/coders.py
index 70237880cd2..cf4b9b5d520 100644
--- a/sdks/python/apache_beam/coders/coders.py
+++ b/sdks/python/apache_beam/coders/coders.py
@@ -24,8 +24,8 @@
 import base64
 from builtins import object
 
-import google.protobuf
-from google.protobuf import wrappers_pb2
+import google.protobuf.wrappers_pb2
+from future.moves import pickle
 
 from apache_beam.coders import coder_impl
 from apache_beam.portability import common_urns
@@ -33,12 +33,6 @@
 from apache_beam.portability.api import beam_runner_api_pb2
 from apache_beam.utils import proto_utils
 
-# This is for py2/3 compatibility. cPickle was renamed pickle in python 3.
-try:
-  import cPickle as pickle # Python 2
-except ImportError:
-  import pickle # Python 3
-
 # pylint: disable=wrong-import-order, wrong-import-position, ungrouped-imports
 try:
   from .stream import get_varint_size
@@ -277,7 +271,7 @@ def from_runner_api(cls, coder_proto, context):
   def to_runner_api_parameter(self, context):
 return (
 python_urns.PICKLED_CODER,
-wrappers_pb2.BytesValue(value=serialize_coder(self)),
+google.protobuf.wrappers_pb2.BytesValue(value=serialize_coder(self)),
 ())
 
   @staticmethod
diff --git a/sdks/python/apache_beam/coders/typecoders.py 
b/sdks/python/apache_beam/coders/typecoders.py
index e4efa2c7ffd..7afb01585d4 100644
--- a/sdks/python/apache_beam/coders/typecoders.py
+++ b/sdks/python/apache_beam/coders/typecoders.py
@@ -67,14 +67,11 @@ def MakeXyzs(v):
 
 from builtins import object
 
+from past.builtins import unicode
+
 from apache_beam.coders import coders
 from apache_beam.typehints import typehints
 
-try:
-  unicode   # pylint: disable=unicode-builtin
-except NameError:
-  unicode = str
-
 __all__ = ['registry']
 
 
diff --git a/sdks/python/apache_beam/examples/cookbook/datastore_wordcount.py 
b/sdks/python/apache_beam/examples/cookbook/datastore_wordcount.py
index d09a46a14d5..b8c4213643e 100644
--- a/sdks/python/apache_beam/examples/cookbook/datastore_wordcount.py
+++ b/sdks/python/apache_beam/examples/cookbook/datastore_wordcount.py
@@ -73,6 +73,7 @@
 from google.cloud.proto.datastore.v1 import query_pb2
 from googledatastore import helper as datastore_helper
 from googledatastore import PropertyFilter
+from past.builtins import unicode
 
 import apache_beam as beam
 from apache_beam.io import ReadFromText
@@ -83,11 +84,6 @@
 from apache_beam.options.pipeline_options import PipelineOptions
 from apache_beam.options.pipeline_options import SetupOptions
 
-try:
-  unicode   # pylint: disable=unicode-builtin
-except NameError:
-  unicode = str
-
 
 class WordExtractingDoFn(beam.DoFn):
   """Parse each line of input text into words."""
diff --git a/sdks/python/apache_beam/examples/snippets/snippets.py 
b/sdks/python/apache_beam/examples/snippets/snippets.py
index 161a67b037f..d854431e56a 100644
--- a/sdks/python/apache_beam/examples/snippets/snippets.py
+++ b/sdks/python/apache_beam/examples/snippets/snippets.py
@@ -36,6 +36,8 @@
 from builtins import object
 from builtins import range
 
+from past.builtins 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=122272=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122272
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Jul/18 11:06
Start Date: 12/Jul/18 11:06
Worklog Time Spent: 10m 
  Work Description: superbobry commented on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-404476038
 
 
   Rebased on top of master.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 122272)
Time Spent: 18h 20m  (was: 18h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 18h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=122266=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122266
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Jul/18 10:16
Start Date: 12/Jul/18 10:16
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders edited a comment on issue #5869: 
[BEAM-1251] Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-404463843
 
 
   LGTM, thank you.
   We discussed this change in the io PR 
[here](https://github.com/apache/beam/pull/5715#discussion_r199076687). I 
originally proposed to use the NameError-driven approach because of problems 
with typechecks when importing from `builtins`. However, it seems that imports 
from `past.builtins` import the native types and don't cause these problems.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 122266)
Time Spent: 18h 10m  (was: 18h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 18h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=122265=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122265
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Jul/18 10:15
Start Date: 12/Jul/18 10:15
Worklog Time Spent: 10m 
  Work Description: RobbeSneyders commented on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-404463843
 
 
   LGTM.
   We discussed this change in the io PR 
[here](https://github.com/apache/beam/pull/5715#discussion_r199076687). I 
originally proposed to use the NameError-driven approach because of problems 
with typechecks when importing from `builtins`. However, it seems that imports 
from `past.builtins` import the native types and don't cause these problems.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 122265)
Time Spent: 18h  (was: 17h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 18h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=122120=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-122120
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 12/Jul/18 01:01
Start Date: 12/Jul/18 01:01
Worklog Time Spent: 10m 
  Work Description: tvalentyn commented on issue #5869: [BEAM-1251] Replace 
NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-40435
 
 
   Thanks, @superbobry , this LGTM. I also ran this past coders microbenchmark 
in https://github.com/apache/beam/pull/5565 and did not see any performance 
impact.
   @RobbeSneyders, do you have any concerns?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 122120)
Time Spent: 17h 50m  (was: 17h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121644=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121644
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 21:59
Start Date: 10/Jul/18 21:59
Worklog Time Spent: 10m 
  Work Description: superbobry commented on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-403979524
 
 
   @aaltay thanks for the feedback. I've added a relevant comment on the 
migration proposal doc.
   
   @cclauss done.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121644)
Time Spent: 17h 40m  (was: 17.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121228=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121228
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 05:43
Start Date: 10/Jul/18 05:43
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5869: [BEAM-1251] Replace 
NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-403707090
 
 
   Please resolve conflict.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121228)
Time Spent: 17.5h  (was: 17h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121205=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121205
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 04:31
Start Date: 10/Jul/18 04:31
Worklog Time Spent: 10m 
  Work Description: aaltay edited a comment on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-403695936
 
 
   Thank you @superbobry.
   
   Could you coordinate your changes with rest of the people working on python 
3 changes on the mailing list. There was a recent reviewed proposal for python 
3 conversion and some of the bits you are removing were added as part of it. 
You can find the proposal on the Beam web site: 
https://beam.apache.org/contribute/#python-3-support
   
   @tvalentyn could also help with coordination.
   
   (I noticed that you are already started coordinating on the mailing list. 
That is great, please share major planned changes, such as the use of `past` in 
the mailing list.)


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121205)
Time Spent: 17h 20m  (was: 17h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121204=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121204
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 04:23
Start Date: 10/Jul/18 04:23
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #5869: [BEAM-1251] Replace 
NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-403695936
 
 
   Thank you @superbobry.
   
   Could you coordinate your changes with rest of the people working on python 
3 changes on the mailing list. There was a recent reviewed proposal for python 
3 conversion and some of the bits you are removing were added as part of it. 
You can find the proposal on the Beam web site: 
https://beam.apache.org/contribute/#python-3-support
   
   @tvalentyn could also help with coordination.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121204)
Time Spent: 17h 10m  (was: 17h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121203=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121203
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 04:19
Start Date: 10/Jul/18 04:19
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403695410
 
 
   @angoenka @cclauss The postcommit issue was fixed and merged in 
https://github.com/apache/beam/pull/5908.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121203)
Time Spent: 17h  (was: 16h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 17h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121189
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 03:55
Start Date: 10/Jul/18 03:55
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #5911: [BEAM-1251] 
Upgrade snappy and use a memoryview
URL: https://github.com/apache/beam/pull/5911
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/io/avroio.py 
b/sdks/python/apache_beam/io/avroio.py
index 9b86b58982b..f90dc3c6833 100644
--- a/sdks/python/apache_beam/io/avroio.py
+++ b/sdks/python/apache_beam/io/avroio.py
@@ -341,8 +341,8 @@ def _decompress_bytes(data, codec):
 
   # Compressed data includes a 4-byte CRC32 checksum which we verify.
   # We take care to avoid extra copies of data while slicing large objects
-  # by use of a buffer.
-  result = snappy.decompress(buffer(data)[:-4])
+  # by use of a memoryview.
+  result = snappy.decompress(memoryview(data)[:-4])
   avroio.BinaryDecoder(io.BytesIO(data[-4:])).check_crc32(result)
   return result
 else:
diff --git a/sdks/python/apache_beam/io/tfrecordio.py 
b/sdks/python/apache_beam/io/tfrecordio.py
index 989247a96ee..2ef7c5b4c72 100644
--- a/sdks/python/apache_beam/io/tfrecordio.py
+++ b/sdks/python/apache_beam/io/tfrecordio.py
@@ -43,7 +43,7 @@ def _default_crc32c_fn(value):
   if not _default_crc32c_fn.fn:
 try:
   import snappy  # pylint: disable=import-error
-  _default_crc32c_fn.fn = snappy._crc32c  # pylint: 
disable=protected-access
+  _default_crc32c_fn.fn = snappy._snappy._crc32c  # pylint: 
disable=protected-access
 except ImportError:
   logging.warning('Couldn\'t find python-snappy so the implementation of '
   '_TFRecordUtil._masked_crc32c is not as fast as it could 
'
diff --git a/sdks/python/apache_beam/runners/dataflow/internal/names.py 
b/sdks/python/apache_beam/runners/dataflow/internal/names.py
index c31e43f78a5..fb4643fe0a1 100644
--- a/sdks/python/apache_beam/runners/dataflow/internal/names.py
+++ b/sdks/python/apache_beam/runners/dataflow/internal/names.py
@@ -42,7 +42,7 @@
 
 # Update this version to the next version whenever there is a change that will
 # require changes to legacy Dataflow worker execution environment.
-BEAM_CONTAINER_VERSION = 'beam-master-20180619'
+BEAM_CONTAINER_VERSION = 'beam-master-20180709'
 # Update this version to the next version whenever there is a change that
 # requires changes to SDK harness container or SDK harness launcher.
 BEAM_FNAPI_CONTAINER_VERSION = 'beam-master-20180619'
diff --git a/sdks/python/container/Dockerfile b/sdks/python/container/Dockerfile
index 90348c6e231..afb6b43f938 100644
--- a/sdks/python/container/Dockerfile
+++ b/sdks/python/container/Dockerfile
@@ -70,7 +70,7 @@ RUN \
 # Optional packages
 pip install "cython == 0.28.1" && \
 pip install "guppy == 0.1.10" && \
-pip install "python-snappy == 0.5.1" && \
+pip install "python-snappy == 0.5.3" && \
 # These are additional packages likely to be used by customers.
 pip install "numpy == 1.13.3" --no-binary=:all: && \
 pip install "pandas == 0.18.1" && \


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121189)
Time Spent: 16h 50m  (was: 16h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121176=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121176
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 02:57
Start Date: 10/Jul/18 02:57
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5911: [BEAM-1251] 
Upgrade snappy and use a memoryview
URL: https://github.com/apache/beam/pull/5911#issuecomment-403683836
 
 
   R: @aaltay 
   CC: @cclauss 
   
   All tests pass with this change.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121176)
Time Spent: 16h 40m  (was: 16.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121165=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121165
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 01:52
Start Date: 10/Jul/18 01:52
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5911: [BEAM-1251] 
Upgrade snappy and use a memoryview
URL: https://github.com/apache/beam/pull/5911#issuecomment-403673794
 
 
   Run Python PostCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121165)
Time Spent: 16.5h  (was: 16h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121164=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121164
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 01:52
Start Date: 10/Jul/18 01:52
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5911: [BEAM-1251] 
Upgrade snappy and use a memoryview
URL: https://github.com/apache/beam/pull/5911#issuecomment-403673739
 
 
   Run Python Dataflow ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121164)
Time Spent: 16h 20m  (was: 16h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121154=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121154
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 00:52
Start Date: 10/Jul/18 00:52
Worklog Time Spent: 10m 
  Work Description: charlesccychen opened a new pull request #5911: 
[BEAM-1251] Upgrade snappy and use a memoryview
URL: https://github.com/apache/beam/pull/5911
 
 
   This change undoes the rollback of #5887 (done in #5908) and incorporates 
appropriate fixes.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121154)
Time Spent: 16h 10m  (was: 16h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121152=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121152
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 00:45
Start Date: 10/Jul/18 00:45
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #5908: [BEAM-1251] 
Revert #5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/io/avroio.py 
b/sdks/python/apache_beam/io/avroio.py
index f90dc3c6833..9b86b58982b 100644
--- a/sdks/python/apache_beam/io/avroio.py
+++ b/sdks/python/apache_beam/io/avroio.py
@@ -341,8 +341,8 @@ def _decompress_bytes(data, codec):
 
   # Compressed data includes a 4-byte CRC32 checksum which we verify.
   # We take care to avoid extra copies of data while slicing large objects
-  # by use of a memoryview.
-  result = snappy.decompress(memoryview(data)[:-4])
+  # by use of a buffer.
+  result = snappy.decompress(buffer(data)[:-4])
   avroio.BinaryDecoder(io.BytesIO(data[-4:])).check_crc32(result)
   return result
 else:
diff --git a/sdks/python/container/Dockerfile b/sdks/python/container/Dockerfile
index afb6b43f938..90348c6e231 100644
--- a/sdks/python/container/Dockerfile
+++ b/sdks/python/container/Dockerfile
@@ -70,7 +70,7 @@ RUN \
 # Optional packages
 pip install "cython == 0.28.1" && \
 pip install "guppy == 0.1.10" && \
-pip install "python-snappy == 0.5.3" && \
+pip install "python-snappy == 0.5.1" && \
 # These are additional packages likely to be used by customers.
 pip install "numpy == 1.13.3" --no-binary=:all: && \
 pip install "pandas == 0.18.1" && \


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121152)
Time Spent: 16h  (was: 15h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 16h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121151=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121151
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 00:45
Start Date: 10/Jul/18 00:45
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5908: [BEAM-1251] 
Revert #5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908#issuecomment-403663314
 
 
   Tests pass; merging.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121151)
Time Spent: 15h 50m  (was: 15h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121129=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121129
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 00:13
Start Date: 10/Jul/18 00:13
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #5908: [BEAM-1251] Revert 
#5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908#issuecomment-403658724
 
 
   Run Python Dataflow ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121129)
Time Spent: 15h 40m  (was: 15.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121125=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121125
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 10/Jul/18 00:03
Start Date: 10/Jul/18 00:03
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403656957
 
 
   @angoenka Yes


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121125)
Time Spent: 15.5h  (was: 15h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121119=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121119
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:56
Start Date: 09/Jul/18 23:56
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5908: [BEAM-1251] 
Revert #5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908#issuecomment-403655681
 
 
   R: @angoenka 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121119)
Time Spent: 15h 10m  (was: 15h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121120=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121120
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:56
Start Date: 09/Jul/18 23:56
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5908: [BEAM-1251] 
Revert #5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908#issuecomment-403655697
 
 
   Run Python PostCommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121120)
Time Spent: 15h 20m  (was: 15h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121118=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121118
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:55
Start Date: 09/Jul/18 23:55
Worklog Time Spent: 10m 
  Work Description: charlesccychen opened a new pull request #5908: 
[BEAM-1251] Revert #5887 to unbreak Python PostCommit
URL: https://github.com/apache/beam/pull/5908
 
 
   This change reverts #5887 to fix issues with the Python PostCommit.  I will 
take care of undoing this rollback after an appropriate fix.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121118)
Time Spent: 15h  (was: 14h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 15h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121112=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121112
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:48
Start Date: 09/Jul/18 23:48
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #5887: [BEAM-1251] Upgrade 
from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403654529
 
 
   Is this the only commit that needs to be reverted?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121112)
Time Spent: 14h 50m  (was: 14h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 14h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=12=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-12
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:47
Start Date: 09/Jul/18 23:47
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403654392
 
 
   We should roll back until we figure out a proper fix.  Unfortunately, we 
can't just add a python-snappy pin to the setup file, since it doesn't compile 
on all platforms.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 12)
Time Spent: 14h 40m  (was: 14.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 14h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=121108=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-121108
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 23:43
Start Date: 09/Jul/18 23:43
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #5887: [BEAM-1251] Upgrade 
from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403653679
 
 
   The build seems to be broken after this. Can you please take a look. 
   https://scans.gradle.com/s/ek5enzlgtrm3c/console-log#L3941
   https://builds.apache.org/job/beam_PostCommit_Python_Verify/5531/consoleFull
   
   Thannks


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 121108)
Time Spent: 14.5h  (was: 14h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 14.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120948=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120948
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 18:17
Start Date: 09/Jul/18 18:17
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403572381
 
 
   Root cause is the following.  Looks like we need to add "python-snappy == 
0.5.3" into the setup.py file.
   
   ```
   Encountered exception in PrefetchingSourceSetIterable reader thread: 
Traceback (most recent call last):
 File 
"/usr/local/lib/python2.7/dist-packages/apache_beam/runners/worker/sideinputs.py",
 line 137, in _reader_thread
   for value in reader:
 File 
"/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativefileio.py", line 
198, in __iter__
   for record in self.read_next_block():
 File 
"/usr/local/lib/python2.7/dist-packages/dataflow_worker/nativeavroio.py", line 
94, in read_next_block
   self._sync_marker)
 File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/avroio.py", 
line 261, in read_block_from_file
   return _AvroBlock(block_bytes, num_records, codec, schema, offset, size)
 File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/avroio.py", 
line 313, in __init__
   self._decompressed_block_bytes = self._decompress_bytes(block_bytes, 
codec)
 File "/usr/local/lib/python2.7/dist-packages/apache_beam/io/avroio.py", 
line 345, in _decompress_bytes
   result = snappy.decompress(memoryview(data)[:-4])
 File "/usr/local/lib/python2.7/dist-packages/snappy.py", line 91, in 
uncompress
   return _uncompress(data)
   TypeError: argument 1 must be string or read-only buffer, not memoryview
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120948)
Time Spent: 14h 20m  (was: 14h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 14h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120674=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120674
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 08:50
Start Date: 09/Jul/18 08:50
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403407139
 
 
   Heads up that this change caused issues in postcommits.  Will investigate 
more in the morning.
   
   ```
   ==
   ERROR: test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT)
   --
   Traceback (most recent call last):
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_it_test.py",
 line 68, in test_wordcount_it
   wordcount.run(test_pipeline.get_full_options_as_args(**extra_opts))
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount.py",
 line 118, in run
   result = p.run()
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
 line 395, in run
   self.to_runner_api(), self.runner, self._options).run(False)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
 line 408, in run
   return self.runner.run_pipeline(self)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 61, in run_pipeline
   self.result.wait_until_finish(duration=wait_duration)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 1135, in wait_until_finish
   (self.state, getattr(self._runner, 'last_error_msg', None)), self)
   DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
   Traceback (most recent call last):
 File 
"/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
642, in do_work
   work_executor.execute()
 File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", 
line 156, in execute
   op.start()
 File "dataflow_worker/shuffle_operations.py", line 49, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   def start(self):
 File "dataflow_worker/shuffle_operations.py", line 50, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.scoped_start_state:
 File "dataflow_worker/shuffle_operations.py", line 65, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.scoped_process_state:
 File "dataflow_worker/shuffle_operations.py", line 66, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.shuffle_source.reader() as reader:
 File "dataflow_worker/shuffle_operations.py", line 70, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   self.output(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 180, in 
apache_beam.runners.worker.operations.Operation.output
   cython.cast(Receiver, 
self.receivers[output_index]).receive(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 90, in 
apache_beam.runners.worker.operations.ConsumerSet.receive
   cython.cast(Operation, consumer).process(windowed_value)
 File "dataflow_worker/shuffle_operations.py", line 229, in 
dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.process
   with self.scoped_process_state:
 File "dataflow_worker/shuffle_operations.py", line 236, in 
dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.process
   self.output(wvalue.with_value((k, wvalue.value)))
 File "apache_beam/runners/worker/operations.py", line 180, in 
apache_beam.runners.worker.operations.Operation.output
   cython.cast(Receiver, 
self.receivers[output_index]).receive(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 90, in 
apache_beam.runners.worker.operations.ConsumerSet.receive
   cython.cast(Operation, consumer).process(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 404, in 
apache_beam.runners.worker.operations.DoOperation.process
   with self.scoped_process_state:
 File "apache_beam/runners/worker/operations.py", line 405, in 
apache_beam.runners.worker.operations.DoOperation.process
   self.dofn_receiver.receive(o)
 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120675=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120675
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 08:50
Start Date: 09/Jul/18 08:50
Worklog Time Spent: 10m 
  Work Description: charlesccychen edited a comment on issue #5887: 
[BEAM-1251] Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887#issuecomment-403407139
 
 
   Heads up that this change caused issues in postcommits.  Will investigate 
more in the morning.
   
   ```
   ==
   ERROR: test_wordcount_it (apache_beam.examples.wordcount_it_test.WordCountIT)
   --
   Traceback (most recent call last):
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount_it_test.py",
 line 68, in test_wordcount_it
   wordcount.run(test_pipeline.get_full_options_as_args(**extra_opts))
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/examples/wordcount.py",
 line 118, in run
   result = p.run()
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
 line 395, in run
   self.to_runner_api(), self.runner, self._options).run(False)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/pipeline.py",
 line 408, in run
   return self.runner.run_pipeline(self)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 61, in run_pipeline
   self.result.wait_until_finish(duration=wait_duration)
 File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_Verify/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 1135, in wait_until_finish
   (self.state, getattr(self._runner, 'last_error_msg', None)), self)
   DataflowRuntimeException: Dataflow pipeline failed. State: FAILED, Error:
   Traceback (most recent call last):
 File 
"/usr/local/lib/python2.7/dist-packages/dataflow_worker/batchworker.py", line 
642, in do_work
   work_executor.execute()
 File "/usr/local/lib/python2.7/dist-packages/dataflow_worker/executor.py", 
line 156, in execute
   op.start()
 File "dataflow_worker/shuffle_operations.py", line 49, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   def start(self):
 File "dataflow_worker/shuffle_operations.py", line 50, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.scoped_start_state:
 File "dataflow_worker/shuffle_operations.py", line 65, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.scoped_process_state:
 File "dataflow_worker/shuffle_operations.py", line 66, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   with self.shuffle_source.reader() as reader:
 File "dataflow_worker/shuffle_operations.py", line 70, in 
dataflow_worker.shuffle_operations.GroupedShuffleReadOperation.start
   self.output(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 180, in 
apache_beam.runners.worker.operations.Operation.output
   cython.cast(Receiver, 
self.receivers[output_index]).receive(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 90, in 
apache_beam.runners.worker.operations.ConsumerSet.receive
   cython.cast(Operation, consumer).process(windowed_value)
 File "dataflow_worker/shuffle_operations.py", line 229, in 
dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.process
   with self.scoped_process_state:
 File "dataflow_worker/shuffle_operations.py", line 236, in 
dataflow_worker.shuffle_operations.BatchGroupAlsoByWindowsOperation.process
   self.output(wvalue.with_value((k, wvalue.value)))
 File "apache_beam/runners/worker/operations.py", line 180, in 
apache_beam.runners.worker.operations.Operation.output
   cython.cast(Receiver, 
self.receivers[output_index]).receive(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 90, in 
apache_beam.runners.worker.operations.ConsumerSet.receive
   cython.cast(Operation, consumer).process(windowed_value)
 File "apache_beam/runners/worker/operations.py", line 404, in 
apache_beam.runners.worker.operations.DoOperation.process
   with self.scoped_process_state:
 File "apache_beam/runners/worker/operations.py", line 405, in 
apache_beam.runners.worker.operations.DoOperation.process
   

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120565
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 05:05
Start Date: 09/Jul/18 05:05
Worklog Time Spent: 10m 
  Work Description: cclauss closed pull request #5890: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5890
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/dependency_check/bigquery_client_utils.py 
b/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
index 08571b7dce3..823ae89ea4e 100644
--- a/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
+++ b/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
@@ -15,6 +15,8 @@
 # limitations under the License.
 #
 
+from __future__ import print_function
+
 import datetime
 import logging
 from google.cloud import bigquery
@@ -101,7 +103,7 @@ def insert_dep_to_table(self, dep, version, release_date, 
is_currently_used=Fals
 try:
   query_job = self.bigquery_client.query(query)
   if not query_job.done():
-print query_job.result()
+print(query_job.result())
 except:
   raise
 
@@ -123,7 +125,7 @@ def delete_dep_from_table(self, dep, version):
 try:
   query_job = self.bigquery_client.query(query)
   if not query_job.done():
-print query_job.result()
+print(query_job.result())
 except:
   raise
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120565)
Time Spent: 13h 50m  (was: 13h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 13h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120564=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120564
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Jul/18 05:05
Start Date: 09/Jul/18 05:05
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5890: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403360900
 
 
   Fixed in #5842 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120564)
Time Spent: 13h 40m  (was: 13.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 13h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120519=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120519
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Jul/18 21:32
Start Date: 08/Jul/18 21:32
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #5842: [BEAM-1251] 
Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/dependency_check/bigquery_client_utils.py 
b/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
index 08571b7dce3..f7cd0fe24b9 100644
--- a/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
+++ b/.test-infra/jenkins/dependency_check/bigquery_client_utils.py
@@ -1,3 +1,4 @@
+from __future__ import print_function
 #
 # Licensed to the Apache Software Foundation (ASF) under one or more
 # contributor license agreements.  See the NOTICE file distributed with
@@ -101,7 +102,7 @@ def insert_dep_to_table(self, dep, version, release_date, 
is_currently_used=Fals
 try:
   query_job = self.bigquery_client.query(query)
   if not query_job.done():
-print query_job.result()
+print(query_job.result())
 except:
   raise
 
@@ -123,7 +124,7 @@ def delete_dep_from_table(self, dep, version):
 try:
   query_job = self.bigquery_client.query(query)
   if not query_job.done():
-print query_job.result()
+print(query_job.result())
 except:
   raise
 
diff --git 
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py 
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
index 1703375e695..981d9eb4b05 100644
--- a/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
+++ b/.test-infra/jenkins/dependency_check/dependency_check_report_generator.py
@@ -56,7 +56,7 @@ def extract_results(file_path):
   see_oudated_deps = True
 raw_report.close()
 return outdated_deps
-  except Exception, e:
+  except Exception as e:
 raise
 
 
@@ -266,7 +266,7 @@ def generate_report(file_path, sdk_type, project_id, 
dataset_id, table_id):
 for dep in high_priority_deps:
   report.write("%s" % dep)
 report.write("\n")
-  except Exception, e:
+  except Exception as e:
 report.write(' {0} '.format(str(e)))
 
   report.close()
diff --git 
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
 
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
index 3a48f2a7230..770527a04c6 100644
--- 
a/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
+++ 
b/.test-infra/jenkins/dependency_check/dependency_check_report_generator_test.py
@@ -19,6 +19,7 @@
 #   This script performs testing of scenarios from 
verify_performance_test_results.py
 #
 
+from __future__ import print_function
 import unittest, mock
 from mock import patch
 from datetime import datetime
@@ -39,7 +40,7 @@ class DependencyCheckReportGeneratorTest(unittest.TestCase):
   """Tests for `dependency_check_report_generator.py`."""
 
   def setUp(self):
-print "Test name:", self._testMethodName
+print("Test name:", self._testMethodName)
 
 
   @patch('google.cloud.bigquery.Client')
diff --git a/sdks/python/apache_beam/runners/worker/sdk_worker.py 
b/sdks/python/apache_beam/runners/worker/sdk_worker.py
index b8fa422536b..0ebcea5bacc 100644
--- a/sdks/python/apache_beam/runners/worker/sdk_worker.py
+++ b/sdks/python/apache_beam/runners/worker/sdk_worker.py
@@ -176,7 +176,7 @@ def _request_process_bundle_progress(self, request):
 def task():
   instruction_reference = getattr(
   request, request.WhichOneof('request')).instruction_reference
-  if self._instruction_id_vs_worker.has_key(instruction_reference):
+  if instruction_reference in self._instruction_id_vs_worker:
 self._execute(
 lambda: self._instruction_id_vs_worker[
 instruction_reference
diff --git a/sdks/python/apache_beam/runners/worker/sdk_worker_main.py 
b/sdks/python/apache_beam/runners/worker/sdk_worker_main.py
index cbd28568343..3e2353b2592 100644
--- a/sdks/python/apache_beam/runners/worker/sdk_worker_main.py
+++ b/sdks/python/apache_beam/runners/worker/sdk_worker_main.py
@@ -158,10 +158,10 @@ def _get_worker_count(pipeline_options):
 an int containing the worker_threads to use. Default is 1
   """
   pipeline_options = pipeline_options.get(
-  'options') if pipeline_options.has_key('options') else {}
+  'options') if 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120518=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120518
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Jul/18 21:32
Start Date: 08/Jul/18 21:32
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5842: [BEAM-1251] 
Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-403318870
 
 
   Thanks @cclauss!


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120518)
Time Spent: 13h 20m  (was: 13h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 13h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120516=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120516
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Jul/18 21:31
Start Date: 08/Jul/18 21:31
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-403318724
 
 
   ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120516)
Time Spent: 13h  (was: 12h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 13h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120517=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120517
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Jul/18 21:31
Start Date: 08/Jul/18 21:31
Worklog Time Spent: 10m 
  Work Description: cclauss removed a comment on issue #5842: [BEAM-1251] 
Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-403318724
 
 
   ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120517)
Time Spent: 13h 10m  (was: 13h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 13h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=120509=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-120509
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 08/Jul/18 21:07
Start Date: 08/Jul/18 21:07
Worklog Time Spent: 10m 
  Work Description: charlesccychen closed pull request #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/io/avroio.py 
b/sdks/python/apache_beam/io/avroio.py
index 9b86b58982b..f90dc3c6833 100644
--- a/sdks/python/apache_beam/io/avroio.py
+++ b/sdks/python/apache_beam/io/avroio.py
@@ -341,8 +341,8 @@ def _decompress_bytes(data, codec):
 
   # Compressed data includes a 4-byte CRC32 checksum which we verify.
   # We take care to avoid extra copies of data while slicing large objects
-  # by use of a buffer.
-  result = snappy.decompress(buffer(data)[:-4])
+  # by use of a memoryview.
+  result = snappy.decompress(memoryview(data)[:-4])
   avroio.BinaryDecoder(io.BytesIO(data[-4:])).check_crc32(result)
   return result
 else:
diff --git a/sdks/python/container/Dockerfile b/sdks/python/container/Dockerfile
index 90348c6e231..afb6b43f938 100644
--- a/sdks/python/container/Dockerfile
+++ b/sdks/python/container/Dockerfile
@@ -70,7 +70,7 @@ RUN \
 # Optional packages
 pip install "cython == 0.28.1" && \
 pip install "guppy == 0.1.10" && \
-pip install "python-snappy == 0.5.1" && \
+pip install "python-snappy == 0.5.3" && \
 # These are additional packages likely to be used by customers.
 pip install "numpy == 1.13.3" --no-binary=:all: && \
 pip install "pandas == 0.18.1" && \


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 120509)
Time Spent: 12h 50m  (was: 12h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=12=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-12
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 06/Jul/18 19:52
Start Date: 06/Jul/18 19:52
Worklog Time Spent: 10m 
  Work Description: pabloem commented on issue #5890: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403130759
 
 
   This LGTM. I'll merge in a couple hours.
   
   On Fri, Jul 6, 2018, 9:59 AM cclauss  wrote:
   
   > Jenkins retest this please
   >
   > —
   > You are receiving this because you were mentioned.
   > Reply to this email directly, view it on GitHub
   > , or mute
   > the thread
   > 

   > .
   >
   -- 
   Got feedback? go/pabloem-feedback
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 12)
Time Spent: 12h 40m  (was: 12.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119922=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119922
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 06/Jul/18 16:59
Start Date: 06/Jul/18 16:59
Worklog Time Spent: 10m 
  Work Description: cclauss removed a comment on issue #5890: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403090780
 
 
   @jenkins retest this please
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 119922)
Time Spent: 12h 20m  (was: 12h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119923=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119923
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 06/Jul/18 16:59
Start Date: 06/Jul/18 16:59
Worklog Time Spent: 10m 
  Work Description: cclauss removed a comment on issue #5890: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403090902
 
 
   Jenkins retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 119923)
Time Spent: 12.5h  (was: 12h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119920=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119920
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 06/Jul/18 16:58
Start Date: 06/Jul/18 16:58
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5890: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403090780
 
 
   @jenkins retest this please
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 119920)
Time Spent: 12h  (was: 11h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119921=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119921
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 06/Jul/18 16:58
Start Date: 06/Jul/18 16:58
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5890: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5890#issuecomment-403090902
 
 
   Jenkins retest this please


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 119921)
Time Spent: 12h 10m  (was: 12h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 12h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119565=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119565
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 05/Jul/18 20:23
Start Date: 05/Jul/18 20:23
Worklog Time Spent: 10m 
  Work Description: cclauss opened a new pull request #5890: [BEAM-1251] 
print() is a function in Python 3
URL: https://github.com/apache/beam/pull/5890
 
 
   **Please** add a meaningful description for your change here
   print() is a function in Python 3.
   
   @pabloem Your review please.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 119565)
Time Spent: 11h 50m  (was: 11h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 11h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=119054=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-119054
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 04/Jul/18 11:37
Start Date: 04/Jul/18 11:37
Worklog Time Spent: 10m 
  Work Description: cclauss opened a new pull request #5887: [BEAM-1251] 
Upgrade from buffer to memoryview (again)
URL: https://github.com/apache/beam/pull/5887
 
 
   This PR is a second shot at #4820 based on [the 
work](https://github.com/andrix/python-snappy/pull/72#) that @pitrou has done 
to allow python-snappy to support Python memoryviews.  This has in turn has 
enabled @martindurant to [release python-snappy 0.5.2 to 
PyPI](https://pypi.org/project/python-snappy/).
   
   This PR recommends the use of that release to upgrade from buffer to 
memoryview in avroio.py because buffer was removed in Python 3 in favor of 
memoryview which is supported in a  Python versions that Beam supports.
   
   Reviews please from @aaltay @holdenk @superbobry 
   
   flake8 testing of https://github.com/apache/beam on Python 3.6.3
   
   $ __flake8 . --count --select=E901,E999,F821,F822,F823 --show-source 
--statistics__
   ```
   ./sdks/python/apache_beam/io/avroio.py:343:34: F821 undefined name 'buffer'
 result = snappy.decompress(buffer(data)[:-4])
^
   1F821 undefined name 'buffer'
   1
   ```
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118971=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118971
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 04/Jul/18 06:57
Start Date: 04/Jul/18 06:57
Worklog Time Spent: 10m 
  Work Description: superbobry edited a comment on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-402380346
 
 
   Will do. I can fix `buffer` if you are short on time, just let me know.
   
   @cclauss also, can we merged futurization PRs first?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118971)
Time Spent: 11.5h  (was: 11h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 11.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118970=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118970
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 04/Jul/18 06:47
Start Date: 04/Jul/18 06:47
Worklog Time Spent: 10m 
  Work Description: superbobry commented on issue #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-402380346
 
 
   Will do. I can fix `buffer` if you are short on time, just let me know.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118970)
Time Spent: 11h 20m  (was: 11h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 11h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118947=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118947
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 04/Jul/18 04:48
Start Date: 04/Jul/18 04:48
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5869: [BEAM-1251] Replace 
NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869#issuecomment-402361993
 
 
   This is great stuff...  Can you just try:
   * __flake8 . --count --show-source --statistics 
--select=E901,E999,F821,F822,F823__ and try to whack out as many of those 
lingering issues as you can?
   
   We still have
   4 E999 SyntaxError: invalid syntax
   11F821 undefined name 'buffer'
   
   Leave that 'buffer' one to me...  I have nice progress on that one but all 
other are fair game.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118947)
Time Spent: 11h 10m  (was: 11h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 11h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118942=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118942
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 04/Jul/18 04:25
Start Date: 04/Jul/18 04:25
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-402359256
 
 
   Rebased


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118942)
Time Spent: 11h  (was: 10h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 11h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118897=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118897
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 03/Jul/18 23:25
Start Date: 03/Jul/18 23:25
Worklog Time Spent: 10m 
  Work Description: pabloem commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-402319441
 
 
   LGTM. Can you rebase the change?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118897)
Time Spent: 10h 50m  (was: 10h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118592=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118592
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 03/Jul/18 06:09
Start Date: 03/Jul/18 06:09
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-402024007
 
 
   @superbobry @holdenk Could I please get your review on these changes?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118592)
Time Spent: 10h 40m  (was: 10.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118467
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 02/Jul/18 22:30
Start Date: 02/Jul/18 22:30
Worklog Time Spent: 10m 
  Work Description: superbobry opened a new pull request #5869: [BEAM-1251] 
Replace NameError-driven dispatch with ``past``
URL: https://github.com/apache/beam/pull/5869
 
 
   The only leftover is the ``ToStringCoder`` which uses ``NameError`` to 
detect Python2/3. The logic there looks fishy (it encodes text data on 2.X but 
not on 3.X), so I'd rather address it in a follow up PR.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118467)
Time Spent: 10.5h  (was: 10h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118393=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118393
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 02/Jul/18 21:03
Start Date: 02/Jul/18 21:03
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-401935532
 
 
   @charlesccychen It is not so much that these PRs are conflicting.  It is 
more like they are trying to make many of the same changes.  We have been 
[trying for a 
while](https://github.com/apache/beam/pulls?q=is%3Apr+author%3Acclauss+is%3Aclosed)
 to make this repo Python 3 compatible by running futurize so let's not slow 
things down by waiting.  Full speed ahead on all efforts.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118393)
Time Spent: 10h 20m  (was: 10h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Assignee: Robbe
>Priority: Trivial
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118375
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 02/Jul/18 20:32
Start Date: 02/Jul/18 20:32
Worklog Time Spent: 10m 
  Work Description: charlesccychen commented on issue #5842: [BEAM-1251] 
Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-401925611
 
 
   Note that this change conflicts with 
https://github.com/apache/beam/pull/5373, which fixes some of the same issues.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118375)
Time Spent: 10h 10m  (was: 10h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118329=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118329
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 02/Jul/18 18:13
Start Date: 02/Jul/18 18:13
Worklog Time Spent: 10m 
  Work Description: pabloem commented on issue #5842: [BEAM-1251] Modernize 
Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842#issuecomment-401889265
 
 
   It seems that Lint and Docs failed. You can run them via `tox -e py27-lint`, 
`tox -e py3-lint` and `tox -e docs` from within `sdks/python/`


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118329)
Time Spent: 10h  (was: 9h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 10h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=118194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-118194
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 02/Jul/18 14:47
Start Date: 02/Jul/18 14:47
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-401830608
 
 
   A fix has been checked into https://github.com/andrix/python-snappy/pull/72


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 118194)
Time Spent: 9h 50m  (was: 9h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 9h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=117787=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-117787
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 30/Jun/18 07:40
Start Date: 30/Jun/18 07:40
Worklog Time Spent: 10m 
  Work Description: cclauss opened a new pull request #5842: [BEAM-1251] 
Modernize Python 2 code to get ready for Python 3
URL: https://github.com/apache/beam/pull/5842
 
 
   Signed-off-by: cclauss 
   
   **Please** add a meaningful description for your change here
   
   Fix Python 3 syntax errors and undefined names.
   
   @aaltay I am confused why it is necessary to keep making these PRs.  I 
though that we put automated testing in place on Python 3 that would use flake8 
to flag syntax errors and undefined names so that these anachronisms would not 
be checked into the codebase.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [x] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [x] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | --- | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 117787)
Time Spent: 9h 40m  (was: 9.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
> 

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=112092=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-112092
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 14/Jun/18 22:46
Start Date: 14/Jun/18 22:46
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #5531: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5531
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/.test-infra/jenkins/verify_performance_test_results_test.py 
b/.test-infra/jenkins/verify_performance_test_results_test.py
index 22f6f3b52fe..f8d7d803962 100644
--- a/.test-infra/jenkins/verify_performance_test_results_test.py
+++ b/.test-infra/jenkins/verify_performance_test_results_test.py
@@ -19,6 +19,8 @@
 #   This script performs testing of scenarios from 
verify_performance_test_results.py
 #
 
+from __future__ import print_function
+
 import unittest, mock
 from mock import patch
 from verify_performance_test_results import create_report
@@ -27,7 +29,7 @@ class VerifyAnalysisScript(unittest.TestCase):
 """Tests for `verify_performance_test_results.py`."""
 
 def setUp(self):
-print "Test name:", self._testMethodName
+print("Test name:", self._testMethodName)
 
 @patch('verify_performance_test_results.count_queries', return_value=0)
 def test_create_daily_report_when_no_data_was_uploaded(self, *args):
@@ -86,4 +88,4 @@ def 
test_create_daily_report_when_average_time_does_not_change(self, *args):
 #TODO: Add more testing scenarios, when single performance tests will be 
finished.
 
 if __name__ == '__main__':
-unittest.main()
\ No newline at end of file
+unittest.main()


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 112092)
Time Spent: 9h 20m  (was: 9h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 9h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=112093=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-112093
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 14/Jun/18 22:46
Start Date: 14/Jun/18 22:46
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #5531: [BEAM-1251] print() is 
a function in Python 3
URL: https://github.com/apache/beam/pull/5531#issuecomment-397460823
 
 
   Merged. Thank you @cclauss for the reminder.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 112093)
Time Spent: 9.5h  (was: 9h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 9.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=112046=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-112046
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 14/Jun/18 21:03
Start Date: 14/Jun/18 21:03
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #5531: [BEAM-1251] print() 
is a function in Python 3
URL: https://github.com/apache/beam/pull/5531#issuecomment-397437592
 
 
   @aaltay Can you please review this PR?  It would be a shame if the __stale__ 
bot closed it. 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 112046)
Time Spent: 9h 10m  (was: 9h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 9h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=109614=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-109614
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 07/Jun/18 04:01
Start Date: 07/Jun/18 04:01
Worklog Time Spent: 10m 
  Work Description: stale[bot] commented on issue #4820: [BEAM-1251] 
Upgrade from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-395285658
 
 
   This pull request has been marked as stale due to 60 days of inactivity. It 
will be closed in 1 week if no further activity occurs. If you think that’s 
incorrect or this pull request requires a review, please simply write any 
comment. If closed, you can revive the PR at any time and @mention a reviewer 
or discuss it on the d...@beam.apache.org list. Thank you for your 
contributions.
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 109614)
Time Spent: 9h  (was: 8h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 9h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=82774=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-82774
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 21/Mar/18 15:16
Start Date: 21/Mar/18 15:16
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374974712
 
 
   https://github.com/andrix/python-snappy/issues/65#issuecomment-374938521


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 82774)
Time Spent: 8h 50m  (was: 8h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=82458=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-82458
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 20/Mar/18 20:15
Start Date: 20/Mar/18 20:15
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374741285
 
 
   Yes we are depending on the python-snappy 
[pypi](https://pypi.python.org/pypi/python-snappy). Dataflow has 0.5.1 
installed, not the latest 0.5.2. But I do not think there is a change related 
to this. I tested with the latest available version for this PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 82458)
Time Spent: 8h 40m  (was: 8.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=82166=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-82166
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 20/Mar/18 05:45
Start Date: 20/Mar/18 05:45
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374483374
 
 
   Are we using the current [python-snappy 
0.52](https://github.com/andrix/python-snappy/releases)?  Perhaps @martindurant 
has some ideas for us.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 82166)
Time Spent: 8.5h  (was: 8h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=82096=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-82096
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 19/Mar/18 23:44
Start Date: 19/Mar/18 23:44
Worklog Time Spent: 10m 
  Work Description: aaltay commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374423323
 
 
   No, the changed version also does not work. This 
`six.binary_type(memoryview(data)[:-4])` results in the literal string of the 
form `` and fails with `snappy.UncompressError: Error 
while decompressing: invalid input`
   
   Besides binary_type is just str, even if it worked as expected in this case 
it would have created a copy of data, which beats the purpose.
   
   The real solution here would be to upgrade snappy to accept memoryview as an 
argument. If we cannot do that, we can remove the optimization and settle for 
`snappy.decompress(data[:-4])`. Or perhaps better we can conditionally keep the 
`buffer` for python2 only.
   
   CC'ing a few people who might have an idea of the impact of copying data 
here:
   cc: @chamikaramj @katsiapis
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 82096)
Time Spent: 8h 20m  (was: 8h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=82033=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-82033
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 19/Mar/18 20:46
Start Date: 19/Mar/18 20:46
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374369470
 
 
   @aaltay Can you please retry with this update?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 82033)
Time Spent: 8h 10m  (was: 8h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81694=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81694
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Mar/18 21:30
Start Date: 18/Mar/18 21:30
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374046915
 
 
   Thanks for catching this.  I did not have an effective way to test.  Reading 
through:
   * https://docs.python.org/2.7/library/stdtypes.html#memoryview
   * https://docs.python.org/3/library/stdtypes.html#memoryview
   I get the sense that the next thing to try would be to wrap it with 
__six.binary_type()__ as in __six.binary_type(memoryview(data)[:-4])__
   
   __memoryview__ exists in all versions of Python that Beam supports so once 
we find a __memoryview__-based solution that works, we should be able to drop 
__buffer__ altogether.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81694)
Time Spent: 8h  (was: 7h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 8h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81692=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81692
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Mar/18 21:07
Start Date: 18/Mar/18 21:07
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374046915
 
 
   Thanks for catching this.  I did not have an effective way to test.  Reading 
through:
   * https://docs.python.org/2.7/library/stdtypes.html#memoryview
   * https://docs.python.org/3/library/stdtypes.html#memoryview
   I get the sense that the next thing to try would be to wrap it with 
__bytes()__ as in __bytes(memoryview(data)[:-4])__
   
   __memoryview__ exists in all versions of Python that Beam supports so once 
we find a __memoryview__-based solution that works, we should be able to drop 
__buffer__ altogether.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81692)
Time Spent: 7h 50m  (was: 7h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81690=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81690
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Mar/18 21:03
Start Date: 18/Mar/18 21:03
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-374046915
 
 
   Thanks for catching this.  I did not have an effective way to test.  Reading 
through:
   * https://docs.python.org/2.7/library/stdtypes.html#memoryview
   * https://docs.python.org/3/library/stdtypes.html#memoryview
   I get the sense that the next thing to try would be: __buffer(data)[:-4]__ 
—> __bytes(buffer(data)[:-4])__
   
   __memoryview__ exists in all versions of Python that Beam supports so once 
we find a __memoryview__-based solution that works, we should be able to drop 
__buffer__ altogether.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81690)
Time Spent: 7h 40m  (was: 7.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81681=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81681
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 18/Mar/18 18:54
Start Date: 18/Mar/18 18:54
Worklog Time Spent: 10m 
  Work Description: aaltay commented on a change in pull request #4820: 
[BEAM-1251] Upgrade from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#discussion_r175299521
 
 

 ##
 File path: sdks/python/apache_beam/io/avroio.py
 ##
 @@ -309,8 +309,8 @@ def _decompress_bytes(data, codec):
 
   # Compressed data includes a 4-byte CRC32 checksum which we verify.
   # We take care to avoid extra copies of data while slicing large objects
-  # by use of a buffer.
-  result = snappy.decompress(buffer(data)[:-4])
 
 Review comment:
   Have you tested this change? When I ran it, it fails with: `TypeError: 
argument 1 must be string or read-only buffer, not memoryview`.
   
   This is because, a slice of a `buffer` will return the raw data, but in case 
of `memoryview` a slice will return a `memoryview` object for that sub section.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81681)
Time Spent: 7.5h  (was: 7h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81536=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81536
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 17/Mar/18 09:11
Start Date: 17/Mar/18 09:11
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-373905823
 
 
   This is the only PR that I have that touches this code.  I mentioned this 
issue in #4798 but I did not propose a fix in that PR.
   
   @aaltay Your review please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81536)
Time Spent: 7h 20m  (was: 7h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=81535=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-81535
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 17/Mar/18 09:09
Start Date: 17/Mar/18 09:09
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-373905823
 
 
   This is the only PR that I have that touches this code.  I mentioned this 
issue in #4798 but I did not propose a fix in that PR.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 81535)
Time Spent: 7h 10m  (was: 7h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7h 10m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=80467=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80467
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 14/Mar/18 19:00
Start Date: 14/Mar/18 19:00
Worklog Time Spent: 10m 
  Work Description: holdenk commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-373138165
 
 
   LGTM but from my memory I think I saw a similar PR, was that also yours? (Or 
am I just imagining things).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80467)
Time Spent: 7h  (was: 6h 50m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 7h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=80048=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80048
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 13/Mar/18 20:26
Start Date: 13/Mar/18 20:26
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #4834: [BEAM-1251] Fix 
basestring for Python 3 - again
URL: https://github.com/apache/beam/pull/4834
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/io/filesystems.py 
b/sdks/python/apache_beam/io/filesystems.py
index 642c1cdd2a6..17d8d37a061 100644
--- a/sdks/python/apache_beam/io/filesystems.py
+++ b/sdks/python/apache_beam/io/filesystems.py
@@ -19,6 +19,8 @@
 
 import re
 
+from six import string_types
+
 from apache_beam.io.filesystem import BeamIOError
 from apache_beam.io.filesystem import CompressionTypes
 from apache_beam.io.filesystem import FileSystem
@@ -261,7 +263,7 @@ def delete(paths):
 Raises:
   ``BeamIOError`` if any of the delete operations fail
 """
-if isinstance(paths, basestring):
+if isinstance(paths, string_types):
   raise BeamIOError('Delete passed string argument instead of list: %s' %
 paths)
 if len(paths) == 0:


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80048)
Time Spent: 6h 50m  (was: 6h 40m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 6h 50m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=80026=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80026
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 13/Mar/18 19:10
Start Date: 13/Mar/18 19:10
Worklog Time Spent: 10m 
  Work Description: cclauss commented on issue #4820: [BEAM-1251] Upgrade 
from buffer to memoryview for Python 3
URL: https://github.com/apache/beam/pull/4820#issuecomment-372784095
 
 
   @holdenk Your review please?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80026)
Time Spent: 6h 40m  (was: 6.5h)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 6h 40m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=80013=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-80013
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 13/Mar/18 18:49
Start Date: 13/Mar/18 18:49
Worklog Time Spent: 10m 
  Work Description: holdenk commented on issue #4834: [BEAM-1251] Fix 
basestring for Python 3 - again
URL: https://github.com/apache/beam/pull/4834#issuecomment-372777647
 
 
   LGTM, cc @aaltay @robertwb 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 80013)
Time Spent: 6.5h  (was: 6h 20m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 6.5h
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-1251) Python 3 Support

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-1251?focusedWorklogId=79018=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-79018
 ]

ASF GitHub Bot logged work on BEAM-1251:


Author: ASF GitHub Bot
Created on: 09/Mar/18 20:56
Start Date: 09/Mar/18 20:56
Worklog Time Spent: 10m 
  Work Description: aaltay closed pull request #4835: [BEAM-1251] Change 
unicode --> six.text_type for Python 3 - again
URL: https://github.com/apache/beam/pull/4835
 
 
   

This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:

As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):

diff --git a/sdks/python/apache_beam/io/gcp/pubsub.py 
b/sdks/python/apache_beam/io/gcp/pubsub.py
index c6d93d03af9..d5afee95580 100644
--- a/sdks/python/apache_beam/io/gcp/pubsub.py
+++ b/sdks/python/apache_beam/io/gcp/pubsub.py
@@ -26,6 +26,8 @@
 
 import re
 
+from six import text_type
+
 from apache_beam import coders
 from apache_beam.io.iobase import Read
 from apache_beam.io.iobase import Write
@@ -184,7 +186,7 @@ def expand(self, pcoll):
  | _ReadFromPubSub(self.topic, self.subscription, self.id_label,
with_attributes=False)
  | 'DecodeString' >> Map(lambda b: b.decode('utf-8')))
-p.element_type = unicode
+p.element_type = text_type
 return p
 
 


 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 79018)
Time Spent: 6h 20m  (was: 6h 10m)

> Python 3 Support
> 
>
> Key: BEAM-1251
> URL: https://issues.apache.org/jira/browse/BEAM-1251
> Project: Beam
>  Issue Type: Improvement
>  Components: sdk-py-core
>Reporter: Eyad Sibai
>Priority: Trivial
>  Time Spent: 6h 20m
>  Remaining Estimate: 0h
>
> I have been trying to use google datalab with python3. As I see there are 
> several packages that does not support python3 yet which google datalab 
> depends on. This is one of them.
> https://github.com/GoogleCloudPlatform/DataflowPythonSDK/issues/6



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)