[jira] [Updated] (TEZ-4014) Allow DAGAppMaster to read configuration from plaintext
[ https://issues.apache.org/jira/browse/TEZ-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated TEZ-4014: -- Description: When launching DAGAppMaster independently, clients won't have to chance localize the configuration for DAGAppMaster in the protobuf format. In these cases it is more convenient for a platform to provide an AM configuration as plaintext. This Jira adds util methods to load tez-site.xml and service_plugins_descriptor.json from the classpath. Some changes are made to the plugin descriptor classes to make them compatible with jackson. These util methods are unused in this Jira (execpt by tests). In another JIRA, they will get plugged into DAGAppMaster. was: When launching DAGAppMaster independently, clients won't have to chance localize the configuration for DAGAppMaster in the protobuf format. In these cases it is more convenient for a platform to provide an AM configuration as plaintext. The change here will be to look for the protobuf conf first, and then fall back to: # Looking for a tez-site.xml text file in the classpath # Looking for a service_plugins_descriptor in JSON format. > Allow DAGAppMaster to read configuration from plaintext > --- > > Key: TEZ-4014 > URL: https://issues.apache.org/jira/browse/TEZ-4014 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: TEZ-4014.1.patch, TEZ-4014.2.patch, TEZ-4014.3.patch, > TEZ-4014.4.patch > > > When launching DAGAppMaster independently, clients won't have to chance > localize the configuration for DAGAppMaster in the protobuf format. > In these cases it is more convenient for a platform to provide an AM > configuration as plaintext. > This Jira adds util methods to load tez-site.xml and > service_plugins_descriptor.json from the classpath. > Some changes are made to the plugin descriptor classes to make them > compatible with jackson. > These util methods are unused in this Jira (execpt by tests). In another > JIRA, they will get plugged into DAGAppMaster. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive
[
https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737487#comment-16737487
]
TezQA commented on TEZ-4004:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 7m
57s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:red}-1{color} | {color:red} test4tests {color} | {color:red} 0m
0s{color} | {color:red} The patch doesn't appear to include any new or modified
tests. Please justify why no new tests are needed for this patch. Also please
list what manual steps were performed to verify this patch. {color} |
|| || || || {color:brown} branch-0.9 Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
2s{color} | {color:green} branch-0.9 passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} branch-0.9 passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
40s{color} | {color:green} branch-0.9 passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} unit {color} | {color:red} 56m 26s{color}
| {color:red} root in the patch failed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
26s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 79m 9s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| Failed junit tests | tez.tests.TestExtServicesWithLocalMode |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/tez:ccf8de2 |
| JIRA Issue | TEZ-4004 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12946218/TEZ-4004.001-branch-0.9.patch
|
| Optional Tests | dupname asflicense javac javadoc unit xml compile |
| uname | Linux 8337b70dbc43 4.4.0-139-generic #165-Ubuntu SMP Wed Oct 24
10:58:50 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | branch-0.9 / aa00ce2 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| unit |
https://builds.apache.org/job/PreCommit-TEZ-Build/76/artifact/out/patch-unit-root.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-TEZ-Build/76/testReport/ |
| Max. process+thread count | 1531 (vs. ulimit of 1) |
| modules | C: . U: . |
| Console output | https://builds.apache.org/job/PreCommit-TEZ-Build/76/console
|
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Update jetty9 to align with Hadoop and Hive
> ---
>
> Key: TEZ-4004
> URL: https://issues.apache.org/jira/browse/TEZ-4004
> Project: Apache Tez
> Issue Type: Bug
>Reporter: Jonathan Eagles
>Assignee: Jonathan Eagles
>Priority: Major
> Fix For: 0.10.0, 0.10.1
>
> Attachments: TEZ-4004.001-branch-0.9.patch, TEZ-4004.001.patch
>
>
> https://abi-laboratory.pro/index.php?view=timeline=java=jetty
> https://issues.apache.org/jira/browse/HADOOP-15815
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (TEZ-4014) Allow DAGAppMaster to read configuration from plaintext
[
https://issues.apache.org/jira/browse/TEZ-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737449#comment-16737449
]
TezQA commented on TEZ-4014:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
13s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 3 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 5m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
14s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
17s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 27s{color} | {color:orange} tez-api: The patch generated 39 new + 492
unchanged - 11 fixed = 531 total (was 503) {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 19s{color} | {color:orange} tez-common: The patch generated 8 new + 110
unchanged - 0 fixed = 118 total (was 110) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 1 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
50s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 1m
48s{color} | {color:green} tez-api in the patch passed. {color} |
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 0m
23s{color} | {color:green} tez-common in the patch passed. {color} |
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
24s{color} | {color:red} The patch generated 1 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 24s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/tez:d4a62de |
| JIRA Issue | TEZ-4014 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12954209/TEZ-4014.4.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
checkstyle compile xml |
| uname | Linux 9ac10db3ef0c 4.4.0-138-generic #164-Ubuntu SMP Tue Oct 2
17:16:02 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | master / a031817 |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.0.1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-TEZ-Build/77/artifact/out/diff-checkstyle-tez-api.txt
|
| checkstyle |
https://builds.apache.org/job/PreCommit-TEZ-Build/77/artifact/out/diff-checkstyle-tez-common.txt
|
| whitespace |
https://builds.apache.org/job/PreCommit-TEZ-Build/77/artifact/out/whitespace-eol.txt
|
| Test Results |
[jira] [Updated] (TEZ-4014) Allow DAGAppMaster to read configuration from plaintext
[ https://issues.apache.org/jira/browse/TEZ-4014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter updated TEZ-4014: -- Attachment: TEZ-4014.4.patch > Allow DAGAppMaster to read configuration from plaintext > --- > > Key: TEZ-4014 > URL: https://issues.apache.org/jira/browse/TEZ-4014 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: TEZ-4014.1.patch, TEZ-4014.2.patch, TEZ-4014.3.patch, > TEZ-4014.4.patch > > > When launching DAGAppMaster independently, clients won't have to chance > localize the configuration for DAGAppMaster in the protobuf format. > In these cases it is more convenient for a platform to provide an AM > configuration as plaintext. > The change here will be to look for the protobuf conf first, and then fall > back to: > # Looking for a tez-site.xml text file in the classpath > # Looking for a service_plugins_descriptor in JSON format. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
