Re: [VOTE] Release Apache Tez-0.9.1 RC2

I've run this on all the 1TB TPC-H and TPC-DS queries from hive-testbench without issues. Didn't check performance against previous Tez version, but I didn't see any errors. +1 (non-binding) --Eric On Wed, Jan 3, 2018 at 6:29 PM, Siddharth Seth wrote: > +1 (binding). Verified signatures, check

Fwd: Switched PreCommit-Admin over to Apache Yetus

I think we may be experiencing this problem on Tez JIRA (failure of PreCommit-Admin hook). Has anyone tried submitting a patch in the last few days? My patch submissions aren't triggering test results ... If anyone knows how to fix this, that would be awesome. Otherwise I can reach out to people

Re: Switched PreCommit-Admin over to Apache Yetus

Hi Allen, Thanks for providing the update. Is it possible for you to check whether Tez has been affected? https://issues.apache.org/jira/projects/TEZ/ For the last few days my patches aren't triggering any submission hook. Here is an example: https://issues.apache.org/jira/browse/TEZ-3888 In ex

Re: Build failed in Jenkins: Tez-Build #1801

ob/Tez-Build/1801/display/ > redirect?page=changes> > > Changes: > > [gopalv] TEZ-3888: Update Jetty to org.eclipse.jetty 9.x (Eric Wohlstadter, > > -- > [...truncated 337.20 KB...] > [INFO] Updating project files from SCM: skipped

Recent tickets for Maven dependency changes

Hi all, There are several tickets for Maven dependency changes that we need to resolve; primarily motivated by Hadoop 3 changes. I linked them to this umbrella ticket to make it easier for people to track: https://issues.apache.org/jira/browse/TEZ-3903 Here is my understanding: 1. Jetty upgrade

Re: Tez statistics model

Hi Xuan, Jonathan, We have some use-cases that could really benefit from "solidify and enhance the Tez Statistics API so that applications (pig/hive/scope/etc) can provide their own custom implementations" I'd be happy to contribute in some way. I think it would be a good idea to leverage standa

[DISCUSS] Aligning master with Hadoop 3 and create separate 0.9.x line

Hi all, I’d like to propose that we move towards aligning the Tez master branch with support for Hadoop 3+ only. A separate branch and distribution (e.g. on Maven Central) would be created to maintain the 0.9.x line with support for Hadoop 2.7+. This will help ensure that Tez can continue to mo

[VOTE] Move master to Hadoop 3+ and create separate 0.9.x line

Please vote (binding or unbinding) on the following proposal. The vote will be open until 3pm (Pacific) April 13th. Proposal: Move master to support minimum Hadoop 3+ (0.10.x line) and create separate branch for Hadoop 2 (0.9.x line) Details: - Tez master branch would support only Hadoop

Re: [VOTE] Move master to Hadoop 3+ and create separate 0.9.x line

gt; >> Regards, > >> Kuhu > >> > >> On Mon, Apr 9, 2018 at 9:56 AM, Jason Lowe > wrote: > >> > >> > +1 > >> > > >> > Jason > >> > > >> > On Fri, Apr 6, 2018 at 4:45 PM, Eric Wohlstadt

Re: [VOTE] Move master to Hadoop 3+ and create separate 0.9.x line

t 5:33 PM, Gopal Vijayaraghavan > > > wrote: > > > > > +1 > > > > > > Cheers, > > > Gopal > > > > > > > > > On 4/12/18, 5:22 PM, "Eric Wohlstadter" wrote: > > > > > > Just a friendly remind

Re: [DISCUSS] Master branch 0.10.0 or 1.0.0?

Using 0.10.0 for the next version sounds good to me. That would get us to a baseline for different branches on Hadoop3 and Hadoop2. Once that is squared away, we would have more time to discuss what people think should be included in a 1.0. Maybe something like? Hadoop2: 0.9.1 -> 0.9.2 -> 0.9.3

Fwd: Announce: Hive-MR3 0.2

Sharing this for anyone not on the hive mailing lists... Interesting use-case of Tez. -- Forwarded message -- From: Sungwoo Park Date: Thu, May 24, 2018 at 9:16 AM Subject: Announce: Hive-MR3 0.2 To: u...@hive.apache.org Hello Hive users, I am pleased to announce the release o

Re: [DISCUSS] Request for 0.10 release manager

Hi Jonathan, I would like to volunteer as the release manager for 0.10. --Eric On Thu, Jun 14, 2018 at 2:40 PM, Jonathan Eagles wrote: > The Apache Tez Community is of a new Release Manager for the 0.10 release > line. > > This release line is aimed at hadoop3+ compatibility and a formal relea

0.10.next release version

In preparation for 0.10.0 release planning, can someone help to add a 0.10.next Release Version to the JIRA? Thanks!

[DISCUSS] Tez 0.10.0 release

Hi all, Kicking off the discussion for the release plan of Tez 0.10.0. Note that there is a significant overlap with the upcoming 0.9.2 release. It may be that all (or almost all) tickets planned for 0.9.2 are included in 0.10.0, so I include those also in the links below. Some of the tickets tha

[VOTE] Tez 0.10.0 release deadline

Hi all, I'd like to propose a deadline of July 27th for work on tickets to be included in the 0.10.0 release. Please +1 or -1 this proposal by July 20th. My understanding is that this proposal requires a Lazy Majority.

Re: [VOTE] Tez 0.10.0 release deadline

anks, > > Gunther. > > > > On Fri, Jul 20, 2018 at 8:33 AM Jonathan Eagles > wrote: > > > > > +1. Seems like a good plan > > > > > > On Wed, Jul 18, 2018, 2:57 PM Eric Wohlstadter > > wrote: > > > > > > > Hi all, > &g

Re: [VOTE] Tez 0.10.0 release deadline

I've created a ticket to add my key to KEYS file for release management. Can someone please review? https://issues.apache.org/jira/browse/TEZ-3977 Thanks! On Mon, Jul 23, 2018 at 3:37 PM, Eric Wohlstadter wrote: > Thanks to everyone for voting. > > The vote passes with (4)

Re: [VOTE] Tez 0.10.0 release deadline

Also, it looks like I need permission to create new git tags for release candidates. Can someone with admin give me permission to create new tags? On Tue, Aug 7, 2018 at 12:20 PM, Eric Wohlstadter wrote: > I've created a ticket to add my key to KEYS file for release management. > &g

Re: [VOTE] Tez 0.10.0 release deadline

Hi all, I am still stuck on the issue of branch/tag creation for making a 0.10.0 release candidate. As the assigned release-manager, can I get permissions for branch administration? On Thu, Aug 9, 2018 at 10:51 AM, Eric Wohlstadter wrote: > Also, it looks like I need permission to create

Re: [VOTE] Tez 0.10.0 release deadline

you don't have permissions I have created > branch-0.10.0. Please create a JIRA to update master to point version to > 0.10.1-SNAPSHOT and branch-0.10.0 to 0.10.0. Setting the versions is a step > in the how to release section of the wiki. > > On Wed, Aug 22, 2018 at 1:29 PM, Er

Re: [DISCUSS] Slack for Apache Tez

I like the idea for general questions and discussions. Agree that reviews shouldn't take place on Slack. On Tue, Oct 2, 2018 at 7:13 AM Eric Badger wrote: > What would the purpose of the slack be? Would it be for hashing out big > details about Tez, discussing issues that people are seeing, and

Re: [DISCUSS] Slack for Apache Tez

stions/discuss releases and features. Agree that reviews should be > JIRA > > based only. > > > > Regards, > > Kuhu > > > > On Tue, Oct 2, 2018 at 2:17 PM Eric Wohlstadter > wrote: > > > > > I like the idea for general questions and discussions.

Encapsulate "modes" through java.util.ServiceLoader?

Hi all, While working on TEZ-3991 I'm starting to see feature variability that creates a lot of tangled/scattered code. Features like session-mode and local-mode scatter themselves using a lot of hard to manage "if" conditions. I think it was ok wi

[jira] [Created] (TEZ-3874) NPE in TezClientUtils when "yarn.resourcemanager.zk-address" is present in Configuration

Eric Wohlstadter created TEZ-3874: - Summary: NPE in TezClientUtils when "yarn.resourcemanager.zk-address" is present in Configuration Key: TEZ-3874 URL: https://issues.apache.org/jira/brows

[jira] [Created] (TEZ-3875) Add "reconnect" API to TezClient for HSI HA

Eric Wohlstadter created TEZ-3875: - Summary: Add "reconnect" API to TezClient for HSI HA Key: TEZ-3875 URL: https://issues.apache.org/jira/browse/TEZ-3875 Project: Apache Tez Issue

[jira] [Created] (TEZ-3888) Update Jetty to org.eclipse.jetty 9.x

Eric Wohlstadter created TEZ-3888: - Summary: Update Jetty to org.eclipse.jetty 9.x Key: TEZ-3888 URL: https://issues.apache.org/jira/browse/TEZ-3888 Project: Apache Tez Issue Type

[jira] [Created] (TEZ-3889) flaky test: TestHttpConnection.testAsyncHttpConnectionInterrupt

Eric Wohlstadter created TEZ-3889: - Summary: flaky test: TestHttpConnection.testAsyncHttpConnectionInterrupt Key: TEZ-3889 URL: https://issues.apache.org/jira/browse/TEZ-3889 Project: Apache Tez

[jira] [Created] (TEZ-3890) Jenkins test patch scripts don't post results to Tez JIRA

Eric Wohlstadter created TEZ-3890: - Summary: Jenkins test patch scripts don't post results to Tez JIRA Key: TEZ-3890 URL: https://issues.apache.org/jira/browse/TEZ-3890 Project: Apach

[jira] [Created] (TEZ-3891) Migrate patch submisssion scripts and hooks to Yetus 0.7.0

Eric Wohlstadter created TEZ-3891: - Summary: Migrate patch submisssion scripts and hooks to Yetus 0.7.0 Key: TEZ-3891 URL: https://issues.apache.org/jira/browse/TEZ-3891 Project: Apache Tez

[jira] [Created] (TEZ-3892) Tez-AM Daemon-Mode

Eric Wohlstadter created TEZ-3892: - Summary: Tez-AM Daemon-Mode Key: TEZ-3892 URL: https://issues.apache.org/jira/browse/TEZ-3892 Project: Apache Tez Issue Type: New Feature

[jira] [Created] (TEZ-3901) Add hadoop3 profile for upgrade to Jersey 1.19

Eric Wohlstadter created TEZ-3901: - Summary: Add hadoop3 profile for upgrade to Jersey 1.19 Key: TEZ-3901 URL: https://issues.apache.org/jira/browse/TEZ-3901 Project: Apache Tez Issue Type

[jira] [Created] (TEZ-3902) Upgrade to netty-3.10.5.Final.jar

Eric Wohlstadter created TEZ-3902: - Summary: Upgrade to netty-3.10.5.Final.jar Key: TEZ-3902 URL: https://issues.apache.org/jira/browse/TEZ-3902 Project: Apache Tez Issue Type: Improvement

[jira] [Created] (TEZ-3903) [Umbrella] 0.9.2 maven deps. changes

Eric Wohlstadter created TEZ-3903: - Summary: [Umbrella] 0.9.2 maven deps. changes Key: TEZ-3903 URL: https://issues.apache.org/jira/browse/TEZ-3903 Project: Apache Tez Issue Type

[jira] [Created] (TEZ-3905) Change BUILDING.TXT to minimum JDK 1.8

Eric Wohlstadter created TEZ-3905: - Summary: Change BUILDING.TXT to minimum JDK 1.8 Key: TEZ-3905 URL: https://issues.apache.org/jira/browse/TEZ-3905 Project: Apache Tez Issue Type: Task

[jira] [Created] (TEZ-3911) Optional mix/max/avg aggr. task counters reported to HistoryLoggingService at final counter aggr.

Eric Wohlstadter created TEZ-3911: - Summary: Optional mix/max/avg aggr. task counters reported to HistoryLoggingService at final counter aggr. Key: TEZ-3911 URL: https://issues.apache.org/jira/browse/TEZ-3911

[jira] [Resolved] (TEZ-2686) TestFaultTolerance fails frequently

[ https://issues.apache.org/jira/browse/TEZ-2686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-2686. --- Resolution: Cannot Reproduce > TestFaultTolerance fails frequen

[jira] [Created] (TEZ-3916) Add hadoop-azure-datalake jar to azure profile

Eric Wohlstadter created TEZ-3916: - Summary: Add hadoop-azure-datalake jar to azure profile Key: TEZ-3916 URL: https://issues.apache.org/jira/browse/TEZ-3916 Project: Apache Tez Issue Type

[jira] [Created] (TEZ-3923) Move master to Hadoop 3+ and create separate 0.9.x line

Eric Wohlstadter created TEZ-3923: - Summary: Move master to Hadoop 3+ and create separate 0.9.x line Key: TEZ-3923 URL: https://issues.apache.org/jira/browse/TEZ-3923 Project: Apache Tez

[jira] [Created] (TEZ-3929) Upgrade Jersey to 1.19

Eric Wohlstadter created TEZ-3929: - Summary: Upgrade Jersey to 1.19 Key: TEZ-3929 URL: https://issues.apache.org/jira/browse/TEZ-3929 Project: Apache Tez Issue Type: Task

[jira] [Created] (TEZ-3944) TestTaskScheduler

Eric Wohlstadter created TEZ-3944: - Summary: TestTaskScheduler Key: TEZ-3944 URL: https://issues.apache.org/jira/browse/TEZ-3944 Project: Apache Tez Issue Type: Bug Reporter

[jira] [Created] (TEZ-3945) TestHistoryParser fails with Hadoop3 build

Eric Wohlstadter created TEZ-3945: - Summary: TestHistoryParser fails with Hadoop3 build Key: TEZ-3945 URL: https://issues.apache.org/jira/browse/TEZ-3945 Project: Apache Tez Issue Type: Bug

[jira] [Created] (TEZ-3946) NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3)

Eric Wohlstadter created TEZ-3946: - Summary: NoClassDefFoundError, org.apache.hadoop.mapred.ShuffleHandler. (Multiple tests with Hadoop3) Key: TEZ-3946 URL: https://issues.apache.org/jira/browse/TEZ-3946

[jira] [Created] (TEZ-3947) TestATSHistoryWithACLs fails with Hadoop3 and Jersey 1.19

Eric Wohlstadter created TEZ-3947: - Summary: TestATSHistoryWithACLs fails with Hadoop3 and Jersey 1.19 Key: TEZ-3947 URL: https://issues.apache.org/jira/browse/TEZ-3947 Project: Apache Tez

[jira] [Resolved] (TEZ-3945) TestHistoryParser fails with Hadoop3 build

[ https://issues.apache.org/jira/browse/TEZ-3945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-3945. --- Resolution: Duplicate Resolving as Duplicate of TEZ-3946 > TestHistoryParser fails w

[jira] [Created] (TEZ-3957) Report TASK_DURATION_MILLIS as a Counter for completed tasks

Eric Wohlstadter created TEZ-3957: - Summary: Report TASK_DURATION_MILLIS as a Counter for completed tasks Key: TEZ-3957 URL: https://issues.apache.org/jira/browse/TEZ-3957 Project: Apache Tez

[jira] [Created] (TEZ-3959) HTTP 502 for bower install

Eric Wohlstadter created TEZ-3959: - Summary: HTTP 502 for bower install Key: TEZ-3959 URL: https://issues.apache.org/jira/browse/TEZ-3959 Project: Apache Tez Issue Type: Bug

[jira] [Resolved] (TEZ-3884) Hadoop3-beta1 fixes for Tez tests

[ https://issues.apache.org/jira/browse/TEZ-3884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-3884. --- Resolution: Duplicate > Hadoop3-beta1 fixes for Tez te

[jira] [Resolved] (TEZ-3903) [Umbrella] Hadoop 3 dependency changes

[ https://issues.apache.org/jira/browse/TEZ-3903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-3903. --- Resolution: Fixed Target Version/s: 0.10.0 (was: 0.9.2) Resolving as all child

[jira] [Resolved] (TEZ-3875) Add "reconnect" API to TezClient for HSI HA

[ https://issues.apache.org/jira/browse/TEZ-3875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-3875. --- Resolution: Duplicate Duplicates TEZ-3892 > Add "reconnect" API to TezClie

[jira] [Resolved] (TEZ-3800) Format Hive queries in Tez UI

[ https://issues.apache.org/jira/browse/TEZ-3800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-3800. --- Resolution: Won't Fix > Format Hive queries i

[jira] [Created] (TEZ-3977) Add Eric Wohlstadter's public key to KEYS

Eric Wohlstadter created TEZ-3977: - Summary: Add Eric Wohlstadter's public key to KEYS Key: TEZ-3977 URL: https://issues.apache.org/jira/browse/TEZ-3977 Project: Apache Tez Issue Type:

[jira] [Created] (TEZ-3986) Change branch-0.10.0 artifact versions, add CHANGES.txt (branch-0.10.0 only)

Eric Wohlstadter created TEZ-3986: - Summary: Change branch-0.10.0 artifact versions, add CHANGES.txt (branch-0.10.0 only) Key: TEZ-3986 URL: https://issues.apache.org/jira/browse/TEZ-3986 Project

[jira] [Created] (TEZ-3988) Update snapshot version in master to 0.10.1-SNAPSHOT

Eric Wohlstadter created TEZ-3988: - Summary: Update snapshot version in master to 0.10.1-SNAPSHOT Key: TEZ-3988 URL: https://issues.apache.org/jira/browse/TEZ-3988 Project: Apache Tez Issue

[jira] [Created] (TEZ-4006) Make FrameworkClient Pluggable

Eric Wohlstadter created TEZ-4006: - Summary: Make FrameworkClient Pluggable Key: TEZ-4006 URL: https://issues.apache.org/jira/browse/TEZ-4006 Project: Apache Tez Issue Type: Sub-task

[jira] [Created] (TEZ-4007) Zookeeper based FrameworkClient

Eric Wohlstadter created TEZ-4007: - Summary: Zookeeper based FrameworkClient Key: TEZ-4007 URL: https://issues.apache.org/jira/browse/TEZ-4007 Project: Apache Tez Issue Type: Sub-task

[jira] [Created] (TEZ-4008) Pluggable AM pool registry

Eric Wohlstadter created TEZ-4008: - Summary: Pluggable AM pool registry Key: TEZ-4008 URL: https://issues.apache.org/jira/browse/TEZ-4008 Project: Apache Tez Issue Type: Sub-task

[jira] [Created] (TEZ-4009) Zookeeper based AM Registry

Eric Wohlstadter created TEZ-4009: - Summary: Zookeeper based AM Registry Key: TEZ-4009 URL: https://issues.apache.org/jira/browse/TEZ-4009 Project: Apache Tez Issue Type: Sub-task

[jira] [Created] (TEZ-4014) Allow DAGAppMaster to read configuration from tez-site.xml plaintext

Eric Wohlstadter created TEZ-4014: - Summary: Allow DAGAppMaster to read configuration from tez-site.xml plaintext Key: TEZ-4014 URL: https://issues.apache.org/jira/browse/TEZ-4014 Project: Apache Tez

[jira] [Created] (TEZ-4016) Make TEZ_AM_LOCAL_RESOURCES_PB optional for unmanaged AM

Eric Wohlstadter created TEZ-4016: - Summary: Make TEZ_AM_LOCAL_RESOURCES_PB optional for unmanaged AM Key: TEZ-4016 URL: https://issues.apache.org/jira/browse/TEZ-4016 Project: Apache Tez

[jira] [Created] (TEZ-4024) DAGAppMaster should access resources in UGI context on startup

Eric Wohlstadter created TEZ-4024: - Summary: DAGAppMaster should access resources in UGI context on startup Key: TEZ-4024 URL: https://issues.apache.org/jira/browse/TEZ-4024 Project: Apache Tez

[jira] [Resolved] (TEZ-4009) Zookeeper based AM Registry

[ https://issues.apache.org/jira/browse/TEZ-4009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Wohlstadter resolved TEZ-4009. --- Resolution: Duplicate Subsumed by TEZ-4007 > Zookeeper based AM Regis