[
https://issues.apache.org/jira/browse/ARROW-2306?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Uwe L. Korn resolved ARROW-2306.
--------------------------------
Resolution: Fixed
Issue resolved by pull request 1742
[https://github.com/apache/arrow/pull/1742]
> [Python] HDFS test failures
> ---------------------------
>
> Key: ARROW-2306
> URL: https://issues.apache.org/jira/browse/ARROW-2306
> Project: Apache Arrow
> Issue Type: Bug
> Components: Python
> Reporter: Wes McKinney
> Assignee: Wes McKinney
> Priority: Blocker
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> These weren't caught because we aren't running the HDFS tests in Travis CI
> {code}
> pyarrow/tests/test_hdfs.py::TestLibHdfs::test_write_to_dataset_no_partitions
> FAILED
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> self = <pyarrow.tests.test_hdfs.TestLibHdfs
> testMethod=test_write_to_dataset_no_partitions>
> @test_parquet.parquet
> def test_write_to_dataset_no_partitions(self):
> tmpdir = pjoin(self.tmp_path, 'write-no_partitions-' + guid())
> self.hdfs.mkdir(tmpdir)
> test_parquet._test_write_to_dataset_no_partitions(
> > tmpdir, filesystem=self.hdfs)
> pyarrow/tests/test_hdfs.py:367:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> pyarrow/tests/test_parquet.py:1475: in _test_write_to_dataset_no_partitions
> filesystem=filesystem)
> pyarrow/parquet.py:1059: in write_to_dataset
> _mkdir_if_not_exists(fs, root_path)
> pyarrow/parquet.py:1006: in _mkdir_if_not_exists
> if fs._isfilestore() and not fs.exists(path):
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> self = <pyarrow.hdfs.HadoopFileSystem object at 0x7f09e87a4c48>
> def _isfilestore(self):
> """
> Returns True if this FileSystem is a unix-style file store with
> directories.
> """
> > raise NotImplementedError
> E NotImplementedError
> pyarrow/filesystem.py:143: NotImplementedError
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > /home/wesm/code/arrow/python/pyarrow/filesystem.py(143)_isfilestore()
> -> raise NotImplementedError
> (Pdb) c
> pyarrow/tests/test_hdfs.py::TestLibHdfs::test_write_to_dataset_with_partitions
> FAILED
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> self = <pyarrow.tests.test_hdfs.TestLibHdfs
> testMethod=test_write_to_dataset_with_partitions>
> @test_parquet.parquet
> def test_write_to_dataset_with_partitions(self):
> tmpdir = pjoin(self.tmp_path, 'write-partitions-' + guid())
> self.hdfs.mkdir(tmpdir)
> test_parquet._test_write_to_dataset_with_partitions(
> > tmpdir, filesystem=self.hdfs)
> pyarrow/tests/test_hdfs.py:360:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> pyarrow/tests/test_parquet.py:1433: in _test_write_to_dataset_with_partitions
> filesystem=filesystem)
> pyarrow/parquet.py:1059: in write_to_dataset
> _mkdir_if_not_exists(fs, root_path)
> pyarrow/parquet.py:1006: in _mkdir_if_not_exists
> if fs._isfilestore() and not fs.exists(path):
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> self = <pyarrow.hdfs.HadoopFileSystem object at 0x7f09e87a4c48>
> def _isfilestore(self):
> """
> Returns True if this FileSystem is a unix-style file store with
> directories.
> """
> > raise NotImplementedError
> E NotImplementedError
> pyarrow/filesystem.py:143: NotImplementedError
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> entering PDB
> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> > /home/wesm/code/arrow/python/pyarrow/filesystem.py(143)_isfilestore()
> -> raise NotImplementedError
> (Pdb) c
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
