[jira] [Commented] (TEZ-4045) Task should be accessible from TaskAttempt
[
https://issues.apache.org/jira/browse/TEZ-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16797492#comment-16797492
]
TezQA commented on TEZ-4045:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:blue}0{color} | {color:blue} reexec {color} | {color:blue} 0m
17s{color} | {color:blue} Docker mode activated. {color} |
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
| {color:green}+1{color} | {color:green} test4tests {color} | {color:green} 0m
0s{color} | {color:green} The patch appears to include 4 new or modified test
files. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
18s{color} | {color:green} the patch passed {color} |
| {color:orange}-0{color} | {color:orange} checkstyle {color} | {color:orange}
0m 23s{color} | {color:orange} tez-dag: The patch generated 1 new + 444
unchanged - 0 fixed = 445 total (was 444) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} unit {color} | {color:green} 3m
32s{color} | {color:green} tez-dag in the patch passed. {color} |
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
7s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 8m 59s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Docker | Client=17.05.0-ce Server=17.05.0-ce Image:yetus/tez:d4a62de |
| JIRA Issue | TEZ-4045 |
| JIRA Patch URL |
https://issues.apache.org/jira/secure/attachment/12963170/TEZ-4045.001.patch |
| Optional Tests | dupname asflicense javac javadoc unit findbugs
checkstyle compile |
| uname | Linux 9bd598b25eea 4.4.0-138-generic #164~14.04.1-Ubuntu SMP Fri Oct
5 08:56:16 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /testptch/patchprocess/precommit/personality/provided.sh |
| git revision | master / 6bbb41f |
| maven | version: Apache Maven 3.3.9 |
| Default Java | 1.8.0_191 |
| findbugs | v3.0.1 |
| checkstyle |
https://builds.apache.org/job/PreCommit-TEZ-Build/130/artifact/out/diff-checkstyle-tez-dag.txt
|
| Test Results |
https://builds.apache.org/job/PreCommit-TEZ-Build/130/testReport/ |
| Max. process+thread count | 220 (vs. ulimit of 1) |
| modules | C: tez-dag U: tez-dag |
| Console output |
https://builds.apache.org/job/PreCommit-TEZ-Build/130/console |
| Powered by | Apache Yetus 0.8.0 http://yetus.apache.org |
This message was automatically generated.
> Task should be accessible from TaskAttempt
> --
>
> Key: TEZ-4045
> URL: https://issues.apache.org/jira/browse/TEZ-4045
> Project: Apache Tez
> Issue Type: Improvement
>Reporter: Yingda Chen
>Assignee: Ying Han
>Priority: Major
> Attachments: TEZ-4045.001.patch
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently In the DAG component hierarchy of DAG->Vertex->Task->TaskAttempt,
> we usually allows the entity lower in the hierarchy to access its upstream
> entity, for example, we have Task.getVertex(), and Vertex.getDAG().
>
> However, TaskAttempt today is missing an
[jira] [Updated] (TEZ-4045) Task should be accessible from TaskAttempt
[ https://issues.apache.org/jira/browse/TEZ-4045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jonathan Eagles updated TEZ-4045: - Attachment: TEZ-4045.001.patch > Task should be accessible from TaskAttempt > -- > > Key: TEZ-4045 > URL: https://issues.apache.org/jira/browse/TEZ-4045 > Project: Apache Tez > Issue Type: Improvement >Reporter: Yingda Chen >Assignee: Ying Han >Priority: Major > Attachments: TEZ-4045.001.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently In the DAG component hierarchy of DAG->Vertex->Task->TaskAttempt, > we usually allows the entity lower in the hierarchy to access its upstream > entity, for example, we have Task.getVertex(), and Vertex.getDAG(). > > However, TaskAttempt today is missing an interface to easily retreat the Task > it belongs to. This can be tricky (it is still doable today, but quite messy > and inefficient) when TaskAttempt is trying to access properties/interfaces > defined by Task. See, for example, the discussion in > [https://github.com/apache/tez/pull/37.] > > This Jira proposes to add an TaskAttempt.getTask() method, and refactor codes > where this new method can help clarify the logic. Ideally, A TaskAttempt > should get Task object passing in as a constructing parameter. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-4055) Align slf4j version to be same as Hadoop3
[
https://issues.apache.org/jira/browse/TEZ-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16796849#comment-16796849
]
Syed Shameerur Rahman commented on TEZ-4055:
[~jeagles] , The issue of missing logs is not seen in case of hivecli(not HS2)
, To reproduce the issue
# Start HS2
# Connect to HS2 with beeline
Now we can see the issue of missing logs , Logs are not printed in beeline
console as well as ${sys:test.tmp.dir}/log/hive.log. The possible reason would
be while building tez it creates slf4j-log4j12-1.7.10.jar and
slf4j-api-1.7.10.jar where as hadoop has slf4j-log4j12-1.7.25.jar and
slf4j-api-1.7.25.jar which clearly conflicts.
Now when we connect to HS2 with beeline SLF4J gives such a warning *SLF4J:
Class path contains multiple SLF4J bindings.* According to the information in
[https://www.slf4j.org/codes.html#multiple_bindings] Even when multiple
bindings are present, SLF4J will pick one logging framework/implementation and
bind with it. The way SLF4J picks a binding is determined by the JVM and for
all practical purposes should be considered random. I think this is where the
issue arises.
Now coming to the solution i think rather than aligning the slf4j version to be
same as hadoop as a long time solution we can exclude the creation of slf4j
jars in tez with *provided* because any how JVM loads the slf4j
JAR from hadoop.
Any Suggestions ?
> Align slf4j version to be same as Hadoop3
> -
>
> Key: TEZ-4055
> URL: https://issues.apache.org/jira/browse/TEZ-4055
> Project: Apache Tez
> Issue Type: Bug
>Affects Versions: 0.9.0, 0.9.1
>Reporter: Syed Shameerur Rahman
>Priority: Major
> Fix For: 0.10.0
>
> Attachments: TEZ-4055.patch
>
>
> Currently tez is using slf4j version 1.7.10 where as hadoop3 is using slf4j
> version 1.7.25 , due to conflicting version of slf4j version no logs are
> getting written in ${sys:test.tmp.dir}/log/hive.log. The statements
> LOG.info("") , LOG.error("") . are directed to write logs into
> ${sys:test.tmp.dir}/log/hive.log.
> When we connect to HS2 with beeline the slf4j jars are being searched
> SLF4J: Class path contains multiple SLF4J bindings.
> SLF4J: Found binding in [jar:]
> SLF4J: Found binding in [jar:]
> SLF4J: See [http://www.slf4j.org/codes.html#multiple_bindings] for an
> explanation.
> SLF4J: Actual binding is of type
> [org.apache.logging.slf4j.Log4jLoggerFactory]
> cc [~hitesh] [~kshukla]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
