Jay Sen commented on GOBBLIN-900:

Looks like GOBBLIN-889 was filed for the same but solves bit differently.
my fix was following one line change
{{this.fileContext = FileContext.getFileContext(this.fs.getUri());}}

{{this.fs}} is already above this code line which is basically the FileSystem 
object so we can directly get the uri from there.  

> FsDataWriter dont use writer config for FileSystem
> --------------------------------------------------
>                 Key: GOBBLIN-900
>                 URL: https://issues.apache.org/jira/browse/GOBBLIN-900
>             Project: Apache Gobblin
>          Issue Type: Bug
>          Components: gobblin-core
>    Affects Versions: 0.14.0
>            Reporter: Jay Sen
>            Assignee: Abhishek Tiwari
>            Priority: Major
>             Fix For: 0.15.0
> FsDataWriter.Java:107
> {{this.fileContext = FileContext.getFileContext(conf);}} 
> FileContext is used while moving data from staging to final output path. 
> currently it uses default hadoop Configuration which can be picked up by 
> local hadoop config in classpath, which is not right. the Writer fiulesystem 
> should be always driven by the writer config mentioned as per job or platform 
> config.

This message was sent by Atlassian Jira

Reply via email to