[jira] [Commented] (TEZ-4006) Make FrameworkClient Pluggable
[ https://issues.apache.org/jira/browse/TEZ-4006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16668069#comment-16668069 ] Eric Wohlstadter commented on TEZ-4006: --- [~gopalv] [~sseth] [~jeagles] [~jlowe] Could someone help to review this one? It is self contained and should be generally useful even outside the context of this umbrella jira. > Make FrameworkClient Pluggable > -- > > Key: TEZ-4006 > URL: https://issues.apache.org/jira/browse/TEZ-4006 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Eric Wohlstadter >Assignee: Eric Wohlstadter >Priority: Major > Attachments: TEZ-4006.1.patch > > > Currently {{FrameworkClient.createFrameworkClient}} is hardcoded to use > either LocalClient or YarnClient. > Change it to allow specifying a FrameworkClient impl. class from > configuration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (TEZ-4014) Allow DAGAppMaster to read configuration from tez-site.xml plaintext
Eric Wohlstadter created TEZ-4014: - Summary: Allow DAGAppMaster to read configuration from tez-site.xml plaintext Key: TEZ-4014 URL: https://issues.apache.org/jira/browse/TEZ-4014 Project: Apache Tez Issue Type: Sub-task Reporter: Eric Wohlstadter Assignee: Eric Wohlstadter When launching DAGAppMaster independently, clients won't have to chance localize the configuration for DAGAppMaster in the protobuf format. In these cases it is more convenient for a platform to provide an AM configuration as a plaintext tez-site.xml. The change here will be to look for the protobuf conf first, and then fall back to looking for a tez-site.xml text file in the classpath. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667578#comment-16667578 ] Sergey Shelukhin commented on TEZ-3957: --- CPU milliseconds is CPU time, this reports wall clock time. No idea about MR. > Report TASK_DURATION_MILLIS as a Counter for completed tasks > > > Key: TEZ-3957 > URL: https://issues.apache.org/jira/browse/TEZ-3957 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Sergey Shelukhin >Priority: Major > Attachments: TEZ-3957.01.patch, TEZ-3957.patch > > > timeTaken is already being reported by {{TaskAttemptFinishedEvent}}, but not > as a Counter. > Combined with TEZ-3911, this provides min(timeTaken), max(timeTaken), > avg(timeTaken). > The value will be: {{finishTime - launchTime}} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks
[ https://issues.apache.org/jira/browse/TEZ-3957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667570#comment-16667570 ] Jonathan Eagles commented on TEZ-3957: -- What does this new counter offer above TaskCounter.CPU_MILLISECONDS? Also, does MapReduce provide this counter? > Report TASK_DURATION_MILLIS as a Counter for completed tasks > > > Key: TEZ-3957 > URL: https://issues.apache.org/jira/browse/TEZ-3957 > Project: Apache Tez > Issue Type: Improvement >Reporter: Eric Wohlstadter >Assignee: Sergey Shelukhin >Priority: Major > Attachments: TEZ-3957.01.patch, TEZ-3957.patch > > > timeTaken is already being reported by {{TaskAttemptFinishedEvent}}, but not > as a Counter. > Combined with TEZ-3911, this provides min(timeTaken), max(timeTaken), > avg(timeTaken). > The value will be: {{finishTime - launchTime}} > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667449#comment-16667449 ] Jaume M commented on TEZ-3995: -- >From the asflicense warning in https://issues.apache.org/jira/browse/TEZ-4004 >looks like I missed some files although I'm not sure why they are not being >shown for this patch. In particular those are: {code} Lines that start with ? in the ASF License report indicate files that do not have an Apache license header: !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_10_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_17_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540595265269_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_18_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594087945_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_12_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_11_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_2_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540595174650_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_5_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_15_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_25_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_19_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594107077_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_29_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_26_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_21_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540595136842_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_13_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594097691_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540593742325_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594107344_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594097912_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540593744799_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_27_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_24_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_28_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_20_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540593745083_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594097395_0001_1_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_9_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_31_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_6_priority.dot !? /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build/sourcedir/tez-tests/dag_1540594019017_0001_3_priority.dot !? /home/jenkins/jenkins-slave/works
[jira] [Reopened] (TEZ-3995) Fix dot files produced by tests to prevent ASF license warnings in yetus
[ https://issues.apache.org/jira/browse/TEZ-3995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M reopened TEZ-3995: -- > Fix dot files produced by tests to prevent ASF license warnings in yetus > > > Key: TEZ-3995 > URL: https://issues.apache.org/jira/browse/TEZ-3995 > Project: Apache Tez > Issue Type: Sub-task >Reporter: Jonathan Eagles >Assignee: Jaume M >Priority: Major > Fix For: 0.9.2, 0.10.0 > > Attachments: TEZ-3995.1-branch-0.9.patch, TEZ-3995.1.patch, > TEZ-3995.2.patch > > > From > https://builds.apache.org/job/PreCommit-TEZ-Build-Yetus/10/artifact/out/patch-asflicense-problems.txt > {code} > Lines that start with ? in the ASF License report indicate files that do > not have an Apache license header: !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name1/current/seen_txid > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name1/current/fsimage_000.md5 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name1/current/fsimage_000 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name1/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name2/current/seen_txid > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name2/current/fsimage_000.md5 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name2/current/fsimage_000 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/name2/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data4/current/BP-1822420483-67.195.81.145-1538067940282/scanner.cursor > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data4/current/BP-1822420483-67.195.81.145-1538067940282/current/dfsUsed > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data4/current/BP-1822420483-67.195.81.145-1538067940282/current/finalized/subdir0/subdir0/blk_1073741827 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data4/current/BP-1822420483-67.195.81.145-1538067940282/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data4/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data9/current/BP-1822420483-67.195.81.145-1538067940282/scanner.cursor > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data9/current/BP-1822420483-67.195.81.145-1538067940282/current/dfsUsed > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data9/current/BP-1822420483-67.195.81.145-1538067940282/current/finalized/subdir0/subdir0/blk_1073741826 > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data9/current/BP-1822420483-67.195.81.145-1538067940282/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data9/current/VERSION > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data10/current/BP-1822420483-67.195.81.145-1538067940282/scanner.cursor > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data10/current/BP-1822420483-67.195.81.145-1538067940282/current/dfsUsed > !? > /home/jenkins/jenkins-slave/workspace/PreCommit-TEZ-Build-Yetus/sourcedir/tez-plugins/tez-aux-services/build/test/data/dfs/data/data10/curre
[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667443#comment-16667443 ] Jaume M commented on TEZ-4004: -- [~jeagles] looks like that. I'll reopen https://issues.apache.org/jira/browse/TEZ-3995. > Update jetty9 to align with Hadoop and Hive > --- > > Key: TEZ-4004 > URL: https://issues.apache.org/jira/browse/TEZ-4004 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles >Priority: Major > Attachments: TEZ-4004.001.patch > > > https://abi-laboratory.pro/index.php?view=timeline&lang=java&l=jetty > https://issues.apache.org/jira/browse/HADOOP-15815 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Issue Comment Deleted] (TEZ-4004) Update jetty9 to align with Hadoop and Hive
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated TEZ-4004: - Comment: was deleted (was: [~jeagles] looks like that. I'll reopen https://issues.apache.org/jira/browse/TEZ-3995.) > Update jetty9 to align with Hadoop and Hive > --- > > Key: TEZ-4004 > URL: https://issues.apache.org/jira/browse/TEZ-4004 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles >Priority: Major > Attachments: TEZ-4004.001.patch > > > https://abi-laboratory.pro/index.php?view=timeline&lang=java&l=jetty > https://issues.apache.org/jira/browse/HADOOP-15815 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667442#comment-16667442 ] Jaume M commented on TEZ-4004: -- [~jeagles] looks like that. I'll reopen https://issues.apache.org/jira/browse/TEZ-3995. > Update jetty9 to align with Hadoop and Hive > --- > > Key: TEZ-4004 > URL: https://issues.apache.org/jira/browse/TEZ-4004 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles >Priority: Major > Attachments: TEZ-4004.001.patch > > > https://abi-laboratory.pro/index.php?view=timeline&lang=java&l=jetty > https://issues.apache.org/jira/browse/HADOOP-15815 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (TEZ-4004) Update jetty9 to align with Hadoop and Hive
[ https://issues.apache.org/jira/browse/TEZ-4004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16667419#comment-16667419 ] Jonathan Eagles commented on TEZ-4004: -- As HADOOP-15815, has agreed to 9.3.24, this patch is ready to be go in after review. [~kshukla], the asflicense warnings are a remnant of the dot file changes unrelated to this jira. [~jmarhuen], can you see the asflicense changes need to be another addendum to address the dot files in tez-tests and tez-ext-service-tests? > Update jetty9 to align with Hadoop and Hive > --- > > Key: TEZ-4004 > URL: https://issues.apache.org/jira/browse/TEZ-4004 > Project: Apache Tez > Issue Type: Bug >Reporter: Jonathan Eagles >Assignee: Jonathan Eagles >Priority: Major > Attachments: TEZ-4004.001.patch > > > https://abi-laboratory.pro/index.php?view=timeline&lang=java&l=jetty > https://issues.apache.org/jira/browse/HADOOP-15815 -- This message was sent by Atlassian JIRA (v7.6.3#76005)