Krisztian Szucs created ARROW-4053:
--------------------------------------
Summary: [Python/Integration] HDFS Tests failing with I/O
operation on closed file
Key: ARROW-4053
URL: https://issues.apache.org/jira/browse/ARROW-4053
Project: Apache Arrow
Issue Type: Bug
Components: Integration, Python
Reporter: Krisztian Szucs
To reporoduce run `docker-compose run hdfs-integration`
{code:python}
self = <pyarrow.tests.test_hdfs.TestLibHdfs testMethod=test_cat>
def test_cat(self):
path = pjoin(self.tmp_path, 'cat-test')
data = b'foobarbaz'
with self.hdfs.open(path, 'wb') as f:
f.write(data)
> contents = self.hdfs.cat(path)
opt/conda/lib/python3.6/site-packages/pyarrow/tests/test_hdfs.py:99:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
opt/conda/lib/python3.6/site-packages/pyarrow/filesystem.py:42: in cat
return f.read()
pyarrow/io.pxi:298: in pyarrow.lib.NativeFile.read
???
pyarrow/io.pxi:180: in pyarrow.lib.NativeFile.size
???
pyarrow/io.pxi:142: in pyarrow.lib.NativeFile.get_random_access_file
???
pyarrow/io.pxi:159: in pyarrow.lib.NativeFile._assert_readable
???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> ???
E ValueError: I/O operation on closed file
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
