[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941147#comment-16941147 ] Valentyn Tymofieiev commented on BEAM-8324: --- I also checked that 2.15.0 release was not affected

[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941146#comment-16941146 ] Valentyn Tymofieiev commented on BEAM-8324: --- I think we should restore the upper bound for dill,

[jira] [Comment Edited] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941140#comment-16941140 ] Valentyn Tymofieiev edited comment on BEAM-8324 at 9/30/19 5:01 PM: I

[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941140#comment-16941140 ] Valentyn Tymofieiev commented on BEAM-8324: --- I think the cause of the error is that we pickle

[jira] [Commented] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941128#comment-16941128 ] Valentyn Tymofieiev commented on BEAM-5878: --- I will also investigate and take care of required

[jira] [Updated] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-5878: -- Fix Version/s: 2.16.0 > Support DoFns with Keyword-only arguments in Python 3. >

[jira] [Commented] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16941126#comment-16941126 ] Valentyn Tymofieiev commented on BEAM-5878: --- Thanks a lot for looking into this, please add the

[jira] [Updated] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-29 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8324: -- Fix Version/s: 2.16.0 > Pre/Postcommit Dataflow IT tests fail: _create_function() takes

[jira] [Commented] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-29 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16940566#comment-16940566 ] Valentyn Tymofieiev commented on BEAM-8324: --- This may be triggered by a dill release that

[jira] [Created] (BEAM-8324) Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given

2019-09-29 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8324: - Summary: Pre/Postcommit Dataflow IT tests fail: _create_function() takes from 2 to 6 positional arguments but 7 were given Key: BEAM-8324 URL:

[jira] [Resolved] (BEAM-6522) Dill fails to pickle avro.RecordSchema classes on Python 3.

2019-09-19 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev resolved BEAM-6522. --- Fix Version/s: 2.16.0 Resolution: Not A Problem > Dill fails to pickle

[jira] [Commented] (BEAM-6522) Dill fails to pickle avro.RecordSchema classes on Python 3.

2019-09-19 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933753#comment-16933753 ] Valentyn Tymofieiev commented on BEAM-6522: --- Beam 2.16.0 and newer releases will pick up new

[jira] [Commented] (BEAM-6522) Dill fails to pickle avro.RecordSchema classes on Python 3.

2019-09-19 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16933752#comment-16933752 ] Valentyn Tymofieiev commented on BEAM-6522: --- This issue is fixed in head version of Dill. >

[jira] [Updated] (BEAM-8287) Update documentation for Python 3 support after Beam 2.16.0.

2019-09-19 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8287: -- Status: Open (was: Triage Needed) > Update documentation for Python 3 support after

[jira] [Created] (BEAM-8287) Update documentation for Python 3 support after Beam 2.16.0.

2019-09-19 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8287: - Summary: Update documentation for Python 3 support after Beam 2.16.0. Key: BEAM-8287 URL: https://issues.apache.org/jira/browse/BEAM-8287 Project: Beam

[jira] [Created] (BEAM-8268) Strengthen retry strategy in GCS Java IO

2019-09-17 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8268: - Summary: Strengthen retry strategy in GCS Java IO Key: BEAM-8268 URL: https://issues.apache.org/jira/browse/BEAM-8268 Project: Beam Issue Type:

[jira] [Assigned] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-8215: - Assignee: Mark Liu (was: Chamikara Jayalath) > Wordcount 1GB Python PKB

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930886#comment-16930886 ] Valentyn Tymofieiev commented on BEAM-8215: --- For some reason, I reproduced the error above on

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930869#comment-16930869 ] Valentyn Tymofieiev commented on BEAM-8215: --- Seems like a recent change in GCP? Can be

[jira] [Assigned] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-8215: - Assignee: Chamikara Jayalath (was: Mark Liu) > Wordcount 1GB Python PKB

[jira] [Commented] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930814#comment-16930814 ] Valentyn Tymofieiev commented on BEAM-8198: --- Last command fails most of the time with GCS-IO

[jira] [Updated] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8215: -- Summary: Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error,

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930772#comment-16930772 ] Valentyn Tymofieiev commented on BEAM-8215: --- [~chamikara] [~udim] Have you seen these errors

[jira] [Comment Edited] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930766#comment-16930766 ] Valentyn Tymofieiev edited comment on BEAM-8215 at 9/16/19 6:41 PM: I

[jira] [Comment Edited] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930766#comment-16930766 ] Valentyn Tymofieiev edited comment on BEAM-8215 at 9/16/19 6:41 PM: I

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-16 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16930766#comment-16930766 ] Valentyn Tymofieiev commented on BEAM-8215: --- I am running these benchmarks locally, without

[jira] [Commented] (BEAM-8229) Python postcommit 2.7 tests are failing since we try to run a test with Python 3.6+ syntax on Python 2.

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929378#comment-16929378 ] Valentyn Tymofieiev commented on BEAM-8229: --- As Udi said, there aren't any Py3-only integration

[jira] [Updated] (BEAM-8229) Python postcommit 2.7 tests are failing since we try to run a test with Python 3.6+ syntax on Python 2.

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8229: -- Summary: Python postcommit 2.7 tests are failing since we try to run a test with Python

[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929356#comment-16929356 ] Valentyn Tymofieiev commented on BEAM-8229: --- Let's continue the conversation here. [~chadrik]

[jira] [Closed] (BEAM-8230) Python pre-commits need to include python 3.x tests

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8230?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-8230. - Fix Version/s: Not applicable Resolution: Fixed > Python pre-commits need to

[jira] [Comment Edited] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929339#comment-16929339 ] Valentyn Tymofieiev edited comment on BEAM-8229 at 9/13/19 4:29 PM:

[jira] [Comment Edited] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929339#comment-16929339 ] Valentyn Tymofieiev edited comment on BEAM-8229 at 9/13/19 4:28 PM:

[jira] [Commented] (BEAM-8229) Python postcommit 3.7 tests are failing with syntax error

2019-09-13 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16929339#comment-16929339 ] Valentyn Tymofieiev commented on BEAM-8229: --- The rootcause of this failure is that we run a

[jira] [Created] (BEAM-8225) Skip portable runner tests when Docker is not installed.

2019-09-12 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8225: - Summary: Skip portable runner tests when Docker is not installed. Key: BEAM-8225 URL: https://issues.apache.org/jira/browse/BEAM-8225 Project: Beam

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-12 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928929#comment-16928929 ] Valentyn Tymofieiev commented on BEAM-8215: --- To clarify, these performance tests run as

[jira] [Commented] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928090#comment-16928090 ] Valentyn Tymofieiev commented on BEAM-5878: --- Per conversation offline with Mark, removing the

[jira] [Updated] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-5878: -- Fix Version/s: (was: 2.16.0) > Support DoFns with Keyword-only arguments in Python

[jira] [Commented] (BEAM-8216) GCS IO fails with uninformative 'Broken pipe' errors while attempting to write to a GCS bucket without proper permissions.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928073#comment-16928073 ] Valentyn Tymofieiev commented on BEAM-8216: --- Over to [~chamikara] to triage. > GCS IO fails

[jira] [Created] (BEAM-8216) GCS IO fails with uninformative 'Broken pipe' errors while attempting to write to a GCS bucket without proper permissions.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8216: - Summary: GCS IO fails with uninformative 'Broken pipe' errors while attempting to write to a GCS bucket without proper permissions. Key: BEAM-8216 URL:

[jira] [Commented] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928063#comment-16928063 ] Valentyn Tymofieiev commented on BEAM-8198: --- It's a bit hard working with PKB, due to issues

[jira] [Updated] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail without uninformative error

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8215: -- Summary: Wordcount 1GB Python PKB benchmarks sometimes fail without uninformative error

[jira] [Updated] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8215: -- Summary: Wordcount 1GB Python PKB benchmarks sometimes fail with uninformative error

[jira] [Commented] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928053#comment-16928053 ] Valentyn Tymofieiev commented on BEAM-5878: --- Per comment above, we expect 2.16.0 to be the fix

[jira] [Updated] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-5878: -- Fix Version/s: (was: 2.17.0) 2.16.0 > Support DoFns with

[jira] [Commented] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail without an informative error

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928049#comment-16928049 ] Valentyn Tymofieiev commented on BEAM-8215: --- cc: [~manishab] > Wordcount 1GB Python PKB

[jira] [Created] (BEAM-8215) Wordcount 1GB Python PKB benchmarks sometimes fail without an informative error

2019-09-11 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8215: - Summary: Wordcount 1GB Python PKB benchmarks sometimes fail without an informative error Key: BEAM-8215 URL: https://issues.apache.org/jira/browse/BEAM-8215

[jira] [Commented] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928020#comment-16928020 ] Valentyn Tymofieiev commented on BEAM-8198: --- Looking at Jenkins jobs for Wordcount 1 GB

[jira] [Comment Edited] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16928020#comment-16928020 ] Valentyn Tymofieiev edited comment on BEAM-8198 at 9/11/19 9:37 PM:

[jira] [Assigned] (BEAM-8188) pyarrow imports should be guarded with try/except

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-8188: - Assignee: Alexey Strokach > pyarrow imports should be guarded with try/except >

[jira] [Closed] (BEAM-8188) pyarrow imports should be guarded with try/except

2019-09-11 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-8188. - Fix Version/s: 2.16.0 Resolution: Fixed > pyarrow imports should be guarded with

[jira] [Commented] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926887#comment-16926887 ] Valentyn Tymofieiev commented on BEAM-8198: --- [~thw] reported a bug in streaming stack, I think

[jira] [Updated] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8198: -- Description: context:

[jira] [Created] (BEAM-8200) Add performance benchmarks for Python streaming

2019-09-10 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8200: - Summary: Add performance benchmarks for Python streaming Key: BEAM-8200 URL: https://issues.apache.org/jira/browse/BEAM-8200 Project: Beam Issue

[jira] [Created] (BEAM-8199) Improve tooling to enable alerts on performance regression based on data collected from Beam benchmarks

2019-09-10 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8199: - Summary: Improve tooling to enable alerts on performance regression based on data collected from Beam benchmarks Key: BEAM-8199 URL:

[jira] [Updated] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8198: -- Description: context:

[jira] [Updated] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8198: -- Status: Open (was: Triage Needed) > Investigate possible performance regression of

[jira] [Created] (BEAM-8198) Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8198: - Summary: Investigate possible performance regression of Wordcount 1GB batch benchmark on Py3. Key: BEAM-8198 URL: https://issues.apache.org/jira/browse/BEAM-8198

[jira] [Commented] (BEAM-7223) Add ValidatesRunner test suite for Flink on Python 3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926820#comment-16926820 ] Valentyn Tymofieiev commented on BEAM-7223: --- No, this is not done yet. I moved the fix version

[jira] [Updated] (BEAM-7223) Add ValidatesRunner test suite for Flink on Python 3.

2019-09-10 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-7223: -- Fix Version/s: (was: 2.16.0) 2.17.0 > Add ValidatesRunner test

[jira] [Closed] (BEAM-8181) Beam fails to install on Windows machine with 32bit interpreters due to pyarrow requirements.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-8181. - Fix Version/s: 2.16.0 Resolution: Fixed > Beam fails to install on Windows machine

[jira] [Commented] (BEAM-8188) pyarrow imports should be guarded with try/except

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926211#comment-16926211 ] Valentyn Tymofieiev commented on BEAM-8188: --- Sample import guard:

[jira] [Created] (BEAM-8188) pyarrow imports should be guarded with try/except

2019-09-09 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8188: - Summary: pyarrow imports should be guarded with try/except Key: BEAM-8188 URL: https://issues.apache.org/jira/browse/BEAM-8188 Project: Beam Issue

[jira] [Commented] (BEAM-8181) Beam fails to install on Windows machine with 32bit interpreters due to pyarrow requirements.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926149#comment-16926149 ] Valentyn Tymofieiev commented on BEAM-8181: --- Arrow codepath that triggers the error:

[jira] [Created] (BEAM-8181) Beam fails to install on Windows machine with 32bit interpreters due to pyarrow requirements.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8181: - Summary: Beam fails to install on Windows machine with 32bit interpreters due to pyarrow requirements. Key: BEAM-8181 URL:

[jira] [Assigned] (BEAM-8181) Beam fails to install on Windows machine with 32bit interpreters due to pyarrow requirements.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8181?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-8181: - Assignee: Valentyn Tymofieiev > Beam fails to install on Windows machine with

[jira] [Commented] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926107#comment-16926107 ] Valentyn Tymofieiev commented on BEAM-5878: --- This should be fixed once new version of dill is

[jira] [Reopened] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-09 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reopened BEAM-5878: --- > Support DoFns with Keyword-only arguments in Python 3. >

[jira] [Updated] (BEAM-8171) Python container build fails

2019-09-06 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8171?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8171: -- Labels: currently-failing (was: ) > Python container build fails >

[jira] [Resolved] (BEAM-7839) Distinguish unknown and unrecognized states in Dataflow runner.

2019-09-05 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev resolved BEAM-7839. --- Fix Version/s: 2.16.0 Assignee: Valentyn Tymofieiev Resolution: Fixed

[jira] [Comment Edited] (BEAM-8123) Support CloudPickle as pickler for Apache Beam.

2019-09-04 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922843#comment-16922843 ] Valentyn Tymofieiev edited comment on BEAM-8123 at 9/4/19 8:37 PM: ---

[jira] [Commented] (BEAM-8123) Support CloudPickle as pickler for Apache Beam.

2019-09-04 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8123?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16922843#comment-16922843 ] Valentyn Tymofieiev commented on BEAM-8123: --- Python 3 postcommit suites seem to successfully

[jira] [Created] (BEAM-8152) Provide a way to better control minor+patch versions of Python 3.x interpreters used to run Beam tests locally and on Jenkins.

2019-09-04 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8152: - Summary: Provide a way to better control minor+patch versions of Python 3.x interpreters used to run Beam tests locally and on Jenkins. Key: BEAM-8152 URL:

[jira] [Closed] (BEAM-8142) --profile_memory flag is py2 only

2019-09-03 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-8142. - Fix Version/s: Not applicable Resolution: Duplicate > --profile_memory flag is py2

[jira] [Commented] (BEAM-8142) --profile_memory flag is py2 only

2019-09-03 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16921779#comment-16921779 ] Valentyn Tymofieiev commented on BEAM-8142: --- Right, I think we can track this in BEAM-7815. >

[jira] [Resolved] (BEAM-5878) Support DoFns with Keyword-only arguments in Python 3.

2019-09-03 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-5878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev resolved BEAM-5878. --- Resolution: Fixed > Support DoFns with Keyword-only arguments in Python 3. >

[jira] [Commented] (BEAM-7993) portable python precommit is flaky

2019-08-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919973#comment-16919973 ] Valentyn Tymofieiev commented on BEAM-7993: --- If the error is reoccurring, we could try to SSH

[jira] [Commented] (BEAM-6158) Using --save_main_session fails on Python 3 when main module has invocations of superclass method using 'super' .

2019-08-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16919945#comment-16919945 ] Valentyn Tymofieiev commented on BEAM-6158: --- The error is happens when main pipeline module has

[jira] [Updated] (BEAM-6158) Using --save_main_session fails on Python 3 when main module has invocations of superclass method using 'super' .

2019-08-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-6158?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-6158: -- Summary: Using --save_main_session fails on Python 3 when main module has invocations

[jira] [Updated] (BEAM-8123) Support CloudPickle as pickler for Apache Beam.

2019-08-30 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8123: -- Status: Open (was: Triage Needed) > Support CloudPickle as pickler for Apache Beam. >

[jira] [Created] (BEAM-8123) Support CloudPickle as pickler for Apache Beam.

2019-08-30 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8123: - Summary: Support CloudPickle as pickler for Apache Beam. Key: BEAM-8123 URL: https://issues.apache.org/jira/browse/BEAM-8123 Project: Beam Issue

[jira] [Commented] (BEAM-7463) BigQueryQueryToTableIT is flaky on Direct runner in PostCommit suites: incorrect checksum

2019-08-27 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16917294#comment-16917294 ] Valentyn Tymofieiev commented on BEAM-7463: --- Another instance: {noformat} Expected: (Test

[jira] [Commented] (BEAM-8046) Unable to read from bigquery and publish to pubsub using dataflow runner (python SDK)

2019-08-27 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16916888#comment-16916888 ] Valentyn Tymofieiev commented on BEAM-8046: --- AFAIK the limitations described here are correct as

[jira] [Updated] (BEAM-8093) py36-gcp is missing from the current tox.ini

2019-08-26 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-8093?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-8093: -- Status: Open (was: Triage Needed) > py36-gcp is missing from the current tox.ini >

[jira] [Created] (BEAM-8093) py36-gcp is missing from the current tox.ini

2019-08-26 Thread Valentyn Tymofieiev (Jira)
Valentyn Tymofieiev created BEAM-8093: - Summary: py36-gcp is missing from the current tox.ini Key: BEAM-8093 URL: https://issues.apache.org/jira/browse/BEAM-8093 Project: Beam Issue

[jira] [Commented] (BEAM-7713) Migrate to "typing" module typing types in Beam typehints (on Py2 and Py3).

2019-08-26 Thread Valentyn Tymofieiev (Jira)
[ https://issues.apache.org/jira/browse/BEAM-7713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16915991#comment-16915991 ] Valentyn Tymofieiev commented on BEAM-7713: --- Hey [~udim], could you please check what's the

[jira] [Commented] (BEAM-7575) Add Wordcount-on-Flink to Python 3.6 and 3.7 test suite

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907168#comment-16907168 ] Valentyn Tymofieiev commented on BEAM-7575: --- This may be a good suite to exercise portable

[jira] [Commented] (BEAM-7455) Improve Avro IO integration test coverage on Python 3.

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7455?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907167#comment-16907167 ] Valentyn Tymofieiev commented on BEAM-7455: --- There is a PR in flight, taking this over as an

[jira] [Updated] (BEAM-7575) Add Wordcount-on-Flink to Python 3.6 and 3.7 test suite

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-7575: -- Priority: Minor (was: Major) > Add Wordcount-on-Flink to Python 3.6 and 3.7 test suite

[jira] [Assigned] (BEAM-7575) Add Wordcount-on-Flink to Python 3.6 and 3.7 test suite

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-7575: - Assignee: (was: Frederik Bode) > Add Wordcount-on-Flink to Python 3.6 and

[jira] [Commented] (BEAM-7223) Add ValidatesRunner test suite for Flink on Python 3.

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907166#comment-16907166 ] Valentyn Tymofieiev commented on BEAM-7223: --- There is a PR in flight, taking this over as an

[jira] [Assigned] (BEAM-7455) Improve Avro IO integration test coverage on Python 3.

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-7455: - Assignee: Valentyn Tymofieiev (was: Frederik Bode) > Improve Avro IO

[jira] [Updated] (BEAM-5627) Redesign test_split_at_fraction_exhaustive tests for Python 3

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev updated BEAM-5627: -- Summary: Redesign test_split_at_fraction_exhaustive tests for Python 3 (was:

[jira] [Assigned] (BEAM-7223) Add ValidatesRunner test suite for Flink on Python 3.

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-7223: - Assignee: Valentyn Tymofieiev (was: Frederik Bode) > Add ValidatesRunner test

[jira] [Assigned] (BEAM-5627) Investigate why test_split_at_fraction_exhaustive consistently fails to split after 101 attempts on Python 3

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-5627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-5627: - Assignee: (was: Frederik Bode) > Investigate why

[jira] [Assigned] (BEAM-7474) Add SDK harness containers for Py 3.6, Py 3.7

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev reassigned BEAM-7474: - Assignee: Hannah Jiang (was: Frederik Bode) > Add SDK harness containers for Py

[jira] [Commented] (BEAM-7474) Add SDK harness containers for Py 3.6, Py 3.7

2019-08-14 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16907162#comment-16907162 ] Valentyn Tymofieiev commented on BEAM-7474: --- It makes sense to consolidate these tasks,

[jira] [Comment Edited] (BEAM-1251) Python 3 Support

2019-08-08 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-1251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16890504#comment-16890504 ] Valentyn Tymofieiev edited comment on BEAM-1251 at 8/8/19 8:04 PM: ---

[jira] [Closed] (BEAM-2103) Document Python 3 support in Beam starting from 2.14.0

2019-08-08 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-2103?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-2103. - Resolution: Fixed Fix Version/s: Not applicable > Document Python 3 support in

[jira] [Commented] (BEAM-7849) UserScore example fails on Python 3.5 as of 2.13.0 and 2.14.0 with Dataflow runner

2019-08-07 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16902513#comment-16902513 ] Valentyn Tymofieiev commented on BEAM-7849: --- Yes, closing. > UserScore example fails on Python

[jira] [Closed] (BEAM-7849) UserScore example fails on Python 3.5 as of 2.13.0 and 2.14.0 with Dataflow runner

2019-08-07 Thread Valentyn Tymofieiev (JIRA)
[ https://issues.apache.org/jira/browse/BEAM-7849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Valentyn Tymofieiev closed BEAM-7849. - Resolution: Duplicate > UserScore example fails on Python 3.5 as of 2.13.0 and 2.14.0

<    1   2   3   4   5   6   7   8   9   10   >