[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=154082=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154082
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 13/Oct/18 04:25
Start Date: 13/Oct/18 04:25
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r224949182
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@tvalentynI came across this python2->python3 doc from python.org, LINK:
python-2-3 Difference .
Interesting read. Section "Use feature detection instead of version
detection" talks about replying on sys.version_info. figured maybe we can weigh
in the point this doc brought up before we apply the change to every place.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 154082)
Time Spent: 5h 40m (was: 5.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 5h 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=154083=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154083
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 13/Oct/18 04:28
Start Date: 13/Oct/18 04:28
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r224949182
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@tvalentynI came across this python2->python3 doc from python.org, LINK:
python-2-3 Difference .
Interesting read. Section "Use feature detection instead of version
detection" talks about replying on sys.version_info. Though the point does
makes our use here into a negative.
just FYI.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 154083)
Time Spent: 5h 50m (was: 5h 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 5h 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=154081=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-154081
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 13/Oct/18 04:24
Start Date: 13/Oct/18 04:24
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r224949182
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
I came across this python2->python3 doc from python.org.
python-2-3 Difference
Interesting read. Section "Use feature detection instead of version
detection" talks about replying on sys.version_info, figured maybe we can weigh
in the point this doc brought up before we apply the change to every place.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 154081)
Time Spent: 5.5h (was: 5h 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 5.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153402=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153402
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 04:10
Start Date: 11/Oct/18 04:10
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6628: [BEAM-5626] Unskip
hadoopfilesystem_test which is already passing in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428814157
Thanks for review & merge, @manuzhang!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153402)
Time Spent: 5h 20m (was: 5h 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 5h 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153376
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 03:23
Start Date: 11/Oct/18 03:23
Worklog Time Spent: 10m
Work Description: manuzhang closed pull request #6628: [BEAM-5626] Unskip
hadoopfilesystem_test which is already passing in Python 3.
URL: https://github.com/apache/beam/pull/6628
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/io/hadoopfilesystem_test.py
b/sdks/python/apache_beam/io/hadoopfilesystem_test.py
index 3dd94b7ed8c..ae2b810f5fc 100644
--- a/sdks/python/apache_beam/io/hadoopfilesystem_test.py
+++ b/sdks/python/apache_beam/io/hadoopfilesystem_test.py
@@ -322,10 +322,6 @@ def test_create_success(self):
expected_file = FakeFile(url, 'wb')
self.assertEqual(self._fake_hdfs.files[url], expected_file)
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_create_write_read_compressed(self):
url = self.fs.join(self.tmpdir, 'new_file.gz')
@@ -363,10 +359,6 @@ def _cmpfiles(self, url1, url2):
data2 = f2.read()
return data1 == data2
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_copy_file(self):
url1 = self.fs.join(self.tmpdir, 'new_file1')
url2 = self.fs.join(self.tmpdir, 'new_file2')
@@ -377,10 +369,6 @@ def test_copy_file(self):
self.assertTrue(self._cmpfiles(url1, url2))
self.assertTrue(self._cmpfiles(url1, url3))
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_copy_file_overwrite_error(self):
url1 = self.fs.join(self.tmpdir, 'new_file1')
url2 = self.fs.join(self.tmpdir, 'new_file2')
@@ -392,10 +380,6 @@ def test_copy_file_overwrite_error(self):
BeamIOError, r'already exists.*%s' % posixpath.basename(url2)):
self.fs.copy([url1], [url2])
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_copy_file_error(self):
url1 = self.fs.join(self.tmpdir, 'new_file1')
url2 = self.fs.join(self.tmpdir, 'new_file2')
@@ -409,10 +393,6 @@ def test_copy_file_error(self):
self.fs.copy([url1, url3], [url2, url4])
self.assertTrue(self._cmpfiles(url3, url4))
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_copy_directory(self):
url_t1 = self.fs.join(self.tmpdir, 't1')
url_t1_inner = self.fs.join(self.tmpdir, 't1/inner')
@@ -430,10 +410,6 @@ def test_copy_directory(self):
self.fs.copy([url_t1], [url_t2])
self.assertTrue(self._cmpfiles(url1, url2))
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_copy_directory_overwrite_error(self):
url_t1 = self.fs.join(self.tmpdir, 't1')
url_t1_inner = self.fs.join(self.tmpdir, 't1/inner')
@@ -458,10 +434,6 @@ def test_copy_directory_overwrite_error(self):
with self.assertRaisesRegexp(BeamIOError, r'already exists'):
self.fs.copy([url_t1], [url_t2])
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still needs to be fixed on Python 3'
- 'TODO: BEAM-5627')
def test_rename_file(self):
url1 = self.fs.join(self.tmpdir, 'f1')
url2 = self.fs.join(self.tmpdir, 'f2')
@@ -490,10 +462,6 @@ def test_rename_file_error(self):
self.assertFalse(self.fs.exists(url3))
self.assertTrue(self.fs.exists(url4))
- @unittest.skipIf(sys.version_info[0] == 3 and
- os.environ.get('RUN_SKIPPED_PY3_TESTS') != '1',
- 'This test still
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153375
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 03:22
Start Date: 11/Oct/18 03:22
Worklog Time Spent: 10m
Work Description: manuzhang commented on issue #6628: [BEAM-5626] Unskip
hadoopfilesystem_test which is already passing in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428807062
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153375)
Time Spent: 5h (was: 4h 50m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153367
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 02:24
Start Date: 11/Oct/18 02:24
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6628: [BEAM-5626] Run
more tests in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428796399
It makes more sense to split commits into their own PRs.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153367)
Time Spent: 4h 50m (was: 4h 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153365=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153365
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 02:14
Start Date: 11/Oct/18 02:14
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6628: [BEAM-5626] Run
more tests in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428794469
BEAM-5626 is solved, that's exactly why I would like to add
hadoopfilesystem_test to the test suite.
This PR does not skip other tests. I'll update the description.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153365)
Time Spent: 4h 40m (was: 4.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 4h 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153364=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153364
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 02:09
Start Date: 11/Oct/18 02:09
Worklog Time Spent: 10m
Work Description: manuzhang commented on issue #6628: [BEAM-5626] Run
more tests in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428793648
Isn't [BEAM-5626](https://issues.apache.org/jira/browse/BEAM-5626) already
solved ? It seems this PR also skips some tests. Could you please give a more
meaningful description ?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153364)
Time Spent: 4.5h (was: 4h 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153339=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153339
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 11/Oct/18 00:24
Start Date: 11/Oct/18 00:24
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6628: [BEAM-5626] Unskip
the tests already passing in Python 3.
URL: https://github.com/apache/beam/pull/6628#issuecomment-428776175
retest this please
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153339)
Time Spent: 4h 20m (was: 4h 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Fix For: 2.8.0
>
> Time Spent: 4h 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=153008=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-153008
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 10/Oct/18 01:24
Start Date: 10/Oct/18 01:24
Worklog Time Spent: 10m
Work Description: tvalentyn opened a new pull request #6628: [BEAM-5626]
Unskip the tests already passing in Python 3.
URL: https://github.com/apache/beam/pull/6628
Tests were fixed in https://github.com/apache/beam/pull/6587.
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
| --- | --- | ---
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 153008)
Time Spent: 4h 10m (was: 4h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
>
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152439=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152439
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 22:11
Start Date: 08/Oct/18 22:11
Worklog Time Spent: 10m
Work Description: charlesccychen closed pull request #6587: [BEAM-5626]
Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/sdks/python/apache_beam/io/hadoopfilesystem_test.py
b/sdks/python/apache_beam/io/hadoopfilesystem_test.py
index a943a12bb4d..8421c43e629 100644
--- a/sdks/python/apache_beam/io/hadoopfilesystem_test.py
+++ b/sdks/python/apache_beam/io/hadoopfilesystem_test.py
@@ -22,6 +22,7 @@
import io
import logging
import posixpath
+import sys
import unittest
from builtins import object
@@ -153,7 +154,7 @@ def delete(self, path, recursive=True):
_ = self.status(path)
-for filepath in self.files.keys(): # pylint:
disable=consider-iterating-dictionary
+for filepath in list(self.files):
if filepath.startswith(path):
del self.files[filepath]
@@ -197,6 +198,12 @@ def checksum(self, path):
class HadoopFileSystemTest(unittest.TestCase):
+ @classmethod
+ def setUpClass(cls):
+# Method has been renamed in Python 3
+if sys.version_info[0] < 3:
+ cls.assertCountEqual = cls.assertItemsEqual
+
def setUp(self):
self._fake_hdfs = FakeHdfs()
hdfs.hdfs.InsecureClient = (
@@ -258,7 +265,7 @@ def test_match_file(self):
returned_files = [f.path
for match_result in result
for f in match_result.metadata_list]
-self.assertItemsEqual(expected_files, returned_files)
+self.assertCountEqual(expected_files, returned_files)
def test_match_file_with_limits(self):
expected_files = [self.fs.join(self.tmpdir, filename)
@@ -296,7 +303,7 @@ def test_match_directory(self):
# structure, so listing without a '/' will return no results.
result = self.fs.match([self.tmpdir + '/'])[0]
files = [f.path for f in result.metadata_list]
-self.assertItemsEqual(files, expected_files)
+self.assertCountEqual(files, expected_files)
def test_match_directory_trailing_slash(self):
expected_files = [self.fs.join(self.tmpdir, filename)
@@ -304,7 +311,7 @@ def test_match_directory_trailing_slash(self):
result = self.fs.match([self.tmpdir + '/'])[0]
files = [f.path for f in result.metadata_list]
-self.assertItemsEqual(files, expected_files)
+self.assertCountEqual(files, expected_files)
def test_create_success(self):
url = self.fs.join(self.tmpdir, 'new_file')
@@ -322,7 +329,7 @@ def test_create_write_read_compressed(self):
path = self.fs._parse_url(url)
expected_file = FakeFile(path, 'wb')
self.assertEqual(self._fake_hdfs.files[path], expected_file)
-data = 'abc' * 10
+data = b'abc' * 10
handle.write(data)
# Compressed data != original data
self.assertNotEquals(data, self._fake_hdfs.files[path].getvalue())
@@ -336,7 +343,7 @@ def test_create_write_read_compressed(self):
def test_open(self):
url = self.fs.join(self.tmpdir, 'old_file1')
handle = self.fs.open(url)
-expected_data = ''
+expected_data = b''
data = handle.read()
self.assertEqual(data, expected_data)
@@ -356,7 +363,7 @@ def test_copy_file(self):
url2 = self.fs.join(self.tmpdir, 'new_file2')
url3 = self.fs.join(self.tmpdir, 'new_file3')
with self.fs.create(url1) as f1:
- f1.write('Hello')
+ f1.write(b'Hello')
self.fs.copy([url1, url1], [url2, url3])
self.assertTrue(self._cmpfiles(url1, url2))
self.assertTrue(self._cmpfiles(url1, url3))
@@ -365,9 +372,9 @@ def test_copy_file_overwrite_error(self):
url1 = self.fs.join(self.tmpdir, 'new_file1')
url2 = self.fs.join(self.tmpdir, 'new_file2')
with self.fs.create(url1) as f1:
- f1.write('Hello')
+ f1.write(b'Hello')
with self.fs.create(url2) as f2:
- f2.write('nope')
+ f2.write(b'nope')
with self.assertRaisesRegexp(
BeamIOError, r'already exists.*%s' % posixpath.basename(url2)):
self.fs.copy([url1], [url2])
@@ -378,7 +385,7 @@ def test_copy_file_error(self):
url3 = self.fs.join(self.tmpdir, 'new_file3')
url4 = self.fs.join(self.tmpdir, 'new_file4')
with self.fs.create(url3) as f:
- f.write('Hello')
+ f.write(b'Hello')
with self.assertRaisesRegexp(
BeamIOError, r'^Copy operation failed .*%s.*%s.* not found' % (
url1,
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152355=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152355
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:40
Start Date: 08/Oct/18 18:40
Worklog Time Spent: 10m
Work Description: charlesccychen commented on issue #6587: [BEAM-5626]
Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427938745
Run Python PostCommit
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152355)
Time Spent: 3h 50m (was: 3h 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3h 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152354=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152354
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:40
Start Date: 08/Oct/18 18:40
Worklog Time Spent: 10m
Work Description: charlesccychen commented on issue #6587: [BEAM-5626]
Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427938663
Thank you! This LGTM.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152354)
Time Spent: 3h 40m (was: 3.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3h 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152353=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152353
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:39
Start Date: 08/Oct/18 18:39
Worklog Time Spent: 10m
Work Description: HuangLED commented on issue #6587: [BEAM-5626] Fix
hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427938439
@aaltay for merge.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152353)
Time Spent: 3.5h (was: 3h 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152352=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152352
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:35
Start Date: 08/Oct/18 18:35
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6587: [BEAM-5626] Fix
hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427937192
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152352)
Time Spent: 3h 20m (was: 3h 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3h 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152341=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152341
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:11
Start Date: 08/Oct/18 18:11
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223453352
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
Updated by moving the validation into SetUpClass, and relying on
version_info.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152341)
Time Spent: 3h (was: 2h 50m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152342=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152342
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 18:11
Start Date: 08/Oct/18 18:11
Worklog Time Spent: 10m
Work Description: HuangLED commented on issue #6587: [BEAM-5626] Fix
hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427929566
PTAL.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152342)
Time Spent: 3h 10m (was: 3h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152286=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152286
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 15:11
Start Date: 08/Oct/18 15:11
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223399804
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@HuangLED I think from code-readability standpoint, using
`sys.version_info[0]` is a more direct way, since `sys.version_info` is a
standard library function whose purpose is to return the version of
interpreter. A reader of the code, who is not aware of `assertItemsEqual`
deprecation will probably guess that this method is not available on Python
3.x, while `try/except` require a clarifying comment in order for a reader not
to wonder why a method sometimes may not be available. Note that
https://github.com/apache/beam/pull/6599/files is changing this as well, so
consider removing these lies from the PR to avoid a merge conflict.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152286)
Time Spent: 2h 50m (was: 2h 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=152264=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152264
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 08/Oct/18 14:53
Start Date: 08/Oct/18 14:53
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223393229
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -153,7 +153,7 @@ def delete(self, path, recursive=True):
_ = self.status(path)
-for filepath in self.files.keys(): # pylint:
disable=consider-iterating-dictionary
+for filepath in list(self.files):
Review comment:
We prefer to use idiomatic Python 3, instead of a back-port code, see the
conversion proposal in: https://s.apache.org/beam-python-3. When there are
concerns with Python 3 code being memory-inefficient we use
future.utils.iterkeys() of branch implementation based on version.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 152264)
Time Spent: 2h 40m (was: 2.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151998=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151998
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 06/Oct/18 18:30
Start Date: 06/Oct/18 18:30
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223157210
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@aaltay, sure. just to clarify. Did you mean move this to top of this
Setup() function?
@tvalentyn I might be wrong, but I think relying on sys.version_info[0] is
more indirect way of verifying things, thus less preferred to me. And in test
file a bit code redundancy for self-containing makes sense to me. just my 2
cents.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151998)
Time Spent: 2.5h (was: 2h 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151989=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151989
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 06/Oct/18 16:39
Start Date: 06/Oct/18 16:39
Worklog Time Spent: 10m
Work Description: rakeshcusat commented on a change in pull request
#6587: [BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223187329
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -153,7 +153,7 @@ def delete(self, path, recursive=True):
_ = self.status(path)
-for filepath in self.files.keys(): # pylint:
disable=consider-iterating-dictionary
+for filepath in list(self.files):
Review comment:
This could be also efficient in terms of memory.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151989)
Time Spent: 2h 20m (was: 2h 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2h 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151988=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151988
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 06/Oct/18 16:29
Start Date: 06/Oct/18 16:29
Worklog Time Spent: 10m
Work Description: rakeshcusat commented on a change in pull request
#6587: [BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223187094
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -153,7 +153,7 @@ def delete(self, path, recursive=True):
_ = self.status(path)
-for filepath in self.files.keys(): # pylint:
disable=consider-iterating-dictionary
+for filepath in list(self.files):
Review comment:
Any specific reason for not using `six` library for porting? I see our
dependencies are using six.
In case we are using six this could be something like this:
```
import six
..
..
for filepath in six.iterkeys(self.files):
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151988)
Time Spent: 2h 10m (was: 2h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2h 10m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151912=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151912
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 22:58
Start Date: 05/Oct/18 22:58
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223157210
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@aaltay, sure. just to clarify. Did you mean move this to top of this
Setup() function?
@tvalentyn I might be wrong, but I think relying on sys.version_info[0] is
more indirect way of verifying things, thus less preferred to me. And in test
file a bit code redundancy is not a bad thing. just my 2 cents.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151912)
Time Spent: 2h (was: 1h 50m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 2h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151903=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151903
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 22:44
Start Date: 05/Oct/18 22:44
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223155466
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
@aaltay , see related discussion here:
https://issues.apache.org/jira/browse/BEAM-5620.
@HuangLED
We could reduce the number of lines by checking `if sys.version_info[0] <
3:`, but I'm OK either way since this PR already adds value.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151903)
Time Spent: 1h 50m (was: 1h 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151895=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151895
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 22:23
Start Date: 05/Oct/18 22:23
Worklog Time Spent: 10m
Work Description: aaltay commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223152551
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,11 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+try:# Python 2
Review comment:
Could you move this to the top of the file?
Should we have a single place to put this usage and similar usage? (I think
we are copying this pattern across files.)
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151895)
Time Spent: 1h 40m (was: 1.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151853=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151853
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 20:42
Start Date: 05/Oct/18 20:42
Worklog Time Spent: 10m
Work Description: tvalentyn commented on issue #6587: [BEAM-5626] Fix
hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427492945
Last postcommit status is red, but precommit tests on this PR are green.
LGTM
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151853)
Time Spent: 1.5h (was: 1h 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151841=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151841
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 20:12
Start Date: 05/Oct/18 20:12
Worklog Time Spent: 10m
Work Description: HuangLED commented on issue #6587: [BEAM-5626] Fix
hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#issuecomment-427485349
Fixed.
Not sure why the PR shows build failing, but in fact on my local repository
testPython2 and testPython3 both succeeded.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151841)
Time Spent: 1h 20m (was: 1h 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1h 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151833=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151833
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 20:02
Start Date: 05/Oct/18 20:02
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223124692
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
sg.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151833)
Time Spent: 1h 10m (was: 1h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151782=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151782
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 18:39
Start Date: 05/Oct/18 18:39
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223103877
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
Pinged that thread, let's repeat
https://github.com/apache/beam/blob/c34c367f5da6f9bef8a46471195470923a201af9/sdks/python/apache_beam/coders/coders_test_common.py#L61
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151782)
Time Spent: 1h (was: 50m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 1h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151781=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151781
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 18:36
Start Date: 05/Oct/18 18:36
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223103104
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
We don't want to use six.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151781)
Time Spent: 50m (was: 40m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 50m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151764=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151764
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 18:13
Start Date: 05/Oct/18 18:13
Worklog Time Spent: 10m
Work Description: HuangLED commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223096454
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
sg if we decide to use six.assertCountEqual. I am however confused by the
comments in that thread, which on the other hand says "we've removed the
dependency on six module".
Do we have a consensus which route to take?
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151764)
Time Spent: 40m (was: 0.5h)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151757=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151757
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 18:02
Start Date: 05/Oct/18 18:02
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223093358
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
I think we should switch to assertCountEqual, instead of adding own
implementation. See also: https://issues.apache.org/jira/browse/BEAM-5620
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151757)
Time Spent: 0.5h (was: 20m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151755=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151755
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 18:02
Start Date: 05/Oct/18 18:02
Worklog Time Spent: 10m
Work Description: tvalentyn commented on a change in pull request #6587:
[BEAM-5626] Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587#discussion_r223093358
##
File path: sdks/python/apache_beam/io/hadoopfilesystem_test.py
##
@@ -214,6 +214,13 @@ def setUp(self):
url = self.fs.join(self.tmpdir, filename)
self.fs.create(url).close()
+ def assertItemsEqual(self, l1, l2):
Review comment:
I think we should switch to assertCountEqual, see also:
https://issues.apache.org/jira/browse/BEAM-5620
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151755)
Time Spent: 20m (was: 10m)
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
> Key: BEAM-5626
> URL: https://issues.apache.org/jira/browse/BEAM-5626
> Project: Beam
> Issue Type: Sub-task
> Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Ruoyun Huang
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> ERROR: test_delete_dir
> (apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest)
> --
> Traceback (most recent call last):
> File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem_test.py",
> line 506, in test_delete_dir
> self.fs.delete([url_t1])
>File
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/io/hadoopfilesystem.py",
> line 370, in delete
> raise BeamIOError("Delete operation failed", exceptions)
> apache_beam.io.filesystem.BeamIOError: Delete operation failed with
> exceptions {'hdfs://test_dir/new_dir1': RuntimeError('dictionary changed size
> during iteration', )}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work logged] (BEAM-5626) Several IO tests fail in Python 3 with RuntimeError('dictionary changed size during iteration',)}
[
https://issues.apache.org/jira/browse/BEAM-5626?focusedWorklogId=151753=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-151753
]
ASF GitHub Bot logged work on BEAM-5626:
Author: ASF GitHub Bot
Created on: 05/Oct/18 17:52
Start Date: 05/Oct/18 17:52
Worklog Time Spent: 10m
Work Description: HuangLED opened a new pull request #6587: [BEAM-5626]
Fix hadoop filesystem test for py3.
URL: https://github.com/apache/beam/pull/6587
Hadoop filesystem test for python3.
After this PR, following test passes:
python ./setup.py test -s
apache_beam.io.hadoopfilesystem_test.HadoopFileSystemTest
Follow this checklist to help us incorporate your contribution quickly and
easily:
- [ X] Format the pull request title like `[BEAM-XXX] Fixes bug in
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA
issue, if applicable. This will automatically link the pull request to the
issue.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
It will help us expedite review of your Pull Request if you tag someone
(e.g. `@username`) to look at it.
Post-Commit Tests Status (on master branch)
Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
--- | --- | --- | --- | --- | --- | --- | ---
Go | [](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
| --- | --- | --- | --- | --- | ---
Java | [](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
Python | [](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
| --- | [](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
[](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
| [](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
| --- | --- | ---
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 151753)
Time Spent: 10m
Remaining Estimate: 0h
> Several IO tests fail in Python 3 with RuntimeError('dictionary changed size
> during iteration',)}
> -
>
>
