[
https://issues.apache.org/jira/browse/PARQUET-154?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14258391#comment-14258391
]
Ryan Blue commented on PARQUET-154:
-----------------------------------
bq. If the path is not Qualified and we pass path and file system separately in
a write call then there should not be any issue
I don't think the problem isn't the cases where this works, like the one you
point out here. The problem is that we can't guarantee the FS and the Path
always match and I think it is better to have the caller construct a Path
correctly. I think that will lead to fewer lurking bugs (where they do not
match) because the caller has to think about whether or not they do and
hopefully test the logic.
> parquet Constructors not taking file System object
> --------------------------------------------------
>
> Key: PARQUET-154
> URL: https://issues.apache.org/jira/browse/PARQUET-154
> Project: Parquet
> Issue Type: Bug
> Components: parquet-mr
> Affects Versions: parquet-mr_1.6.0
> Reporter: Manish Agarwal
>
> I am trying to create a file in parquet file format and in RC file format .
>
> No Parquet constructor accepts fileSystem object as an argument. This means
> that i will have to append the uri from the file system in front of the
> filepath object everytime i need to create a new file .
> In RC format the file system object is allowed to be passed in the
> constructor .
> The advantage of passing the file System object into the constructor is that
> i can specify my yarn instance file system pointer to be used while
> creating the file and its quite straight forward . For example RC file
> constructors uses the file System which we have passed .
> In parquet i see everywhere file System object being derived out of the
> Parameters or created a fresh .
> Is there some reason we avoided using the fileSystem object in constructor ?
> If we allow a file System object constructor as well , I would not have to
> worry about modifying my file name to contain the uri part .
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
