[jira] [Commented] (TEZ-1348) Setup configs required for local mode automatically, instead of relying on changes to tez-site
[
https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16813715#comment-16813715
]
TezQA commented on TEZ-1348:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
0s{color} | {color:blue} Docker mode activated. {color} |
| {color:red}-1{color} | {color:red} patch {color} | {color:red} 0m 5s{color}
| {color:red} TEZ-1348 does not apply to master. Rebase required? Wrong Branch?
See https://cwiki.apache.org/confluence/display/TEZ/How+to+Contribute+to+Tez
for help. {color} |
\\
\\
|| Subsystem || Report/Notes ||
| JIRA Issue | TEZ-1348 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12965358/tez-1348.patch |
| Console output |
https://builds.apache.org/job/PreCommit-TEZ-Build/137/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Setup configs required for local mode automatically, instead of relying on
> changes to tez-site
> --
>
> Key: TEZ-1348
> URL: https://issues.apache.org/jira/browse/TEZ-1348
> Project: Apache Tez
> Issue Type: Sub-task
>Reporter: Siddharth Seth
>Assignee: Todd Lipcon
>Priority: Critical
> Attachments: tez-1348.patch
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in
> tez-site would work for local mode.
> Currently the requirement is that tez-site.xml must have fs.defaultFS set to
> file:///.
> While that works, it doesn't allow for seamless execution in either
> local-mode or on a cluster.
> The main issue here is that when Inputs / Outputs are configured - they use a
> version of configuration which reads tez-site, and do not use the
> configuration from the client itself (which is correct behaviour).
> Not sure what a good way to fix this is
> 1) It may be possible to override this value each time an instance of
> Configuration/TezConfiguration is created. One possible way would be to
> statically add a default resource to Configuration the moment a local client
> is created.
> 2) Provide information in the contexts on whether this is local or not. This
> is fairly ugly, and would get in the way of running mixed mode tasks.
> Anyone have other suggestions ?
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (TEZ-1348) Setup configs required for local mode automatically, instead of relying on changes to tez-site
[ https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated TEZ-1348: - Target Version/s: 0.10.0 (was: 0.8.6) > Setup configs required for local mode automatically, instead of relying on > changes to tez-site > -- > > Key: TEZ-1348 > URL: https://issues.apache.org/jira/browse/TEZ-1348 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Todd Lipcon >Priority: Critical > Attachments: tez-1348.patch > > Time Spent: 20m > Remaining Estimate: 0h > > In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in > tez-site would work for local mode. > Currently the requirement is that tez-site.xml must have fs.defaultFS set to > file:///. > While that works, it doesn't allow for seamless execution in either > local-mode or on a cluster. > The main issue here is that when Inputs / Outputs are configured - they use a > version of configuration which reads tez-site, and do not use the > configuration from the client itself (which is correct behaviour). > Not sure what a good way to fix this is > 1) It may be possible to override this value each time an instance of > Configuration/TezConfiguration is created. One possible way would be to > statically add a default resource to Configuration the moment a local client > is created. > 2) Provide information in the contexts on whether this is local or not. This > is fairly ugly, and would get in the way of running mixed mode tasks. > Anyone have other suggestions ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (TEZ-1348) Setup configs required for local mode automatically, instead of relying on changes to tez-site
[ https://issues.apache.org/jira/browse/TEZ-1348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Todd Lipcon updated TEZ-1348: - Attachment: tez-1348.patch > Setup configs required for local mode automatically, instead of relying on > changes to tez-site > -- > > Key: TEZ-1348 > URL: https://issues.apache.org/jira/browse/TEZ-1348 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Siddharth Seth >Assignee: Todd Lipcon >Priority: Critical > Attachments: tez-1348.patch > > Time Spent: 20m > Remaining Estimate: 0h > > In TEZ-717, I incorrect thought setting fs.defaultFS programmatically in > tez-site would work for local mode. > Currently the requirement is that tez-site.xml must have fs.defaultFS set to > file:///. > While that works, it doesn't allow for seamless execution in either > local-mode or on a cluster. > The main issue here is that when Inputs / Outputs are configured - they use a > version of configuration which reads tez-site, and do not use the > configuration from the client itself (which is correct behaviour). > Not sure what a good way to fix this is > 1) It may be possible to override this value each time an instance of > Configuration/TezConfiguration is created. One possible way would be to > statically add a default resource to Configuration the moment a local client > is created. > 2) Provide information in the contexts on whether this is local or not. This > is fairly ugly, and would get in the way of running mixed mode tasks. > Anyone have other suggestions ? -- This message was sent by Atlassian JIRA (v7.6.3#76005)
