Re: Question on Event

2014-06-27 Thread Hitesh Shah
Hi Jeff, Events are a control-flow mechanism used by Tez. They are a way for user-land code to communicate with either the framework or other parts of user-land code. In addition to this, events are also used by the framework to propagate information to other parts of the framework. All events

Re: [VOTE] Release Apache Tez-0.4.1-incubating RC0

2014-06-30 Thread Hitesh Shah
+1. Check sigs. Built locally, deployed on a single node cluster and ran simple jobs. thanks — HItesh On Jun 27, 2014, at 1:30 PM, Siddharth Seth wrote: > I have created a tez-0.4.1-incubating release candidate (rc0). > This is bug fix release created from branch-0.4, and does not change APIs

[VOTE] Graduate Apache Tez from Apache Incubator to TLP

2014-06-30 Thread Hitesh Shah
TLP [ ] +0 Don't care. [ ] -1 Don't graduate Apache Tez as a TLP because… The vote will remain open for 72 hours. thanks — Hitesh Shah ( on behalf of Tez PPMC ) [1] http://mail-archives.apache.org/mod_mbox/incubator-tez-dev/201406.mbox/%3ccfc4fef8.16337a%25chris.a.mattm...@jpl.nasa

Re: [VOTE] Graduate Apache Tez from Apache Incubator to TLP

2014-07-01 Thread Hitesh Shah
ez.incubator.apache.org" > Date: Tuesday, July 1, 2014 5:13 AM > To: "gene...@incubator.apache.org" > Cc: "dev@tez.incubator.apache.org" > Subject: Re: [VOTE] Graduate Apache Tez from Apache Incubator to TLP > >> On 1 July 2014 05:16, Hitesh Shah

Re: [VOTE] Graduate Apache Tez from Apache Incubator to TLP

2014-07-02 Thread Hitesh Shah
Thanks Justin. I updated the project status.xml for the ICLAs requirement. Created jiras for the version and team list issues in the website. Will look into addressing them soon. thanks — Hitesh On Jul 1, 2014, at 9:35 PM, Justin Mclean wrote: > Hi, > > +1 (binding) > >> - we generate t

Re: [VOTE] Graduate Apache Tez from Apache Incubator to TLP

2014-07-02 Thread Hitesh Shah
as the initial members of the Apache Tez Project: * Alan Gates * Arun C. Murthy * Ashutosh Chauhan * Bill Graham * Bikas Saha * Chris Douglas * Chris Mattmann * Daryn Sharp * Devaraj Das * Gopal Vijayaraghavan * Gunther Hagleitner * Hitesh Shah * Jitendra Pandey * Jason Lowe * Jak

[RESULT][VOTE] Graduate Apache Tez from Apache Incubator to TLP

2014-07-04 Thread Hitesh Shah
— Hitesh On Jun 30, 2014, at 9:16 PM, Hitesh Shah wrote: > Hello folks > > Tez entered incubation in February, 2013. Since then, we have made progress > towards graduation[1]. The Tez community recently voted positively towards > graduation[2] with 27 +1s. > Of the 27, there

Re: Breaking change in TezClient

2014-07-04 Thread Hitesh Shah
Hi Rohini For now, I think fixing the build to use fail-at-end would be the best option. I don’t have build configure permissions so if you could make the change that would be great. Also, would you mind filing a jira that we should revert the build to deploy-at-end later once the tests are fix

Re: 1st hit for Apache Tez

2014-07-16 Thread Hitesh Shah
Hi Chris, I guess the problem with the google rankings is that the Hortonworks website has more content and links related to Tez as compared to tez.incubator.apache.org. The main thing, IMHO, from the community point of view will likely be to start adding a lot more docs/information to the Tez

Re: 1st hit for Apache Tez

2014-07-17 Thread Hitesh Shah
Hi Sally, To answer your question, the initial codebase that was used to seed the Incubator podling came from Hortonworks ( http://wiki.apache.org/incubator/TezProposal ). thanks — Hitesh On Jul 17, 2014, at 12:26 AM, Sally Khudairi wrote: > Thanks, Bertrand. > > @Tez team: to clarify, d

Graduation-related steps from Incubator

2014-07-17 Thread Hitesh Shah
Hi folks As Chris mentioned earlier, the Apache Board voted to approve our graduation to a TLP. The details on the graduation process ( if anyone is interested ) are listed here: http://incubator.apache.org/guides/graduation.html As part of the migration, the git repo, website, mailing lists

Re: 1st hit for Apache Tez

2014-07-17 Thread Hitesh Shah
Going back to the concerns raised about the content at http://hortonworks.com/hadoop/tez/, a few of us who work at Hortonworks reached out to the webmaster to make some changes to this page. To summarize the changes done: - The initial description now clearly states that Tez is developed

New git repo setup and available to use

2014-07-19 Thread Hitesh Shah
Hi folks, The new repo should be up and running. incubator-tez.git should no longer be accessible. - https://git-wip-us.apache.org/repos/asf?p=tez.git - https://svn.apache.org/repos/asf/tez/site thanks — Hitesh

Re: New git repo setup and available to use

2014-07-19 Thread Hitesh Shah
gt; > https://git-wip-us.apache.org/repos/asf/tez.git > > Thanks, > - Tsuyoshi > > On Sun, Jul 20, 2014 at 9:25 AM, Tsuyoshi OZAWA > wrote: >> Thank you for sharing, Hitesh! >> >> - Tsuyoshi >> >> On Sun, Jul 20, 2014 at 9:15 AM, Hitesh Shah

Re: Breaking change in TezClient

2014-07-22 Thread Hitesh Shah
Try the tez branch in hive: https://github.com/apache/hive/tree/tez — Hitesh On Jul 22, 2014, at 7:49 PM, Makoto Yui wrote: > Bikas, > > Yes, tez-0.4.1 worked on hive. > > Is there any hive branch that the up-to-date tez API is used? > > Thanks, > Makoto > > 2014-07-23 2:18 GMT+09:00 Bikas

[TEST] Test email

2014-07-22 Thread Hitesh Shah
Testing new mailing list.

TEZ-1: DAG Application Master

2013-04-18 Thread Hitesh Shah
Hello A few of us at Hortonworks have been doing some initial work on a DAG ApplicationMaster. We recently committed to the TEZ svn in the TEZ-1 branch ( http://svn.apache.org/viewvc/incubator/tez/branches/TEZ-1/ ). Going forwards, we will be working directly off this branch. There is a basic

Re: TEZ-1: DAG Application Master

2013-04-18 Thread Hitesh Shah
gt; Great stuff - I'll start digging in. > Incidentally, will there be a git mirror of this branch? > > -mike. > > -Original Message- > From: Hitesh Shah [mailto:hit...@hortonworks.com] > Sent: Thursday, April 18, 2013 5:06 PM > To: dev@tez.incubator.apache.org &g

Re: TEZ-1: DAG Application Master

2013-04-19 Thread Hitesh Shah
Subject: RE: TEZ-1: DAG Application Master > >> Hitesh, >> >> Great stuff - I'll start digging in. >> Incidentally, will there be a git mirror of this branch? >> >> -mike. >> >> -Original Message- >> From: Hitesh S

MIgrating to GIT

2013-05-02 Thread Hitesh Shah
Folks, I filed a request sometime back ( INFRA-6171 ) to migrate from svn to git. David ( from infra ) has currently migrated the code to git and marked svn as read-only. I did the basic sanity checks to ensure that the last set of commits are in the write-enabled git. Could you someone else a

Re: We need a website

2013-05-08 Thread Hitesh Shah
Something like http://incubator.apache.org/projects/tez.html? -- Hitesh On May 8, 2013, at 11:21 AM, Alan Gates wrote: > In reviewing Tez's monthly report to the Incubator I noticed that despite > being two months old Tez still lacks a website (incubator.apache.org/tez). > This makes it muc

Re: Subscription

2013-05-22 Thread Hitesh Shah
You can use dev-subscr...@tez.incubator.apache.org to subscribe to the list. -- Hitesh On May 21, 2013, at 2:55 AM, Jung-Yup Lee wrote: > i would like to Subscribe this mailing list > thanks

Re: compilation error in Tez master

2013-05-28 Thread Hitesh Shah
Hi Julien Most of the active development is happening on branch TEZ-1. We plan to replace the master branch with TEZ-1 at some point soon. For now, the master branch should be compatible with the 2.0.3/4 releases of hadoop. As of this morning, the TEZ-1 branch was compiling against hadoop bui

Builds up and running for Tez

2013-06-07 Thread Hitesh Shah
Thanks to Giri, we have https://builds.apache.org/job/Tez-Build/8/console -- Hitesh

Re: Builds up and running for Tez

2013-06-10 Thread Hitesh Shah
, 2013, at 2:36 PM, Hitesh Shah wrote: > Thanks to Giri, we have https://builds.apache.org/job/Tez-Build/8/console > > -- Hitesh

git.apache read-only mirror

2013-06-11 Thread Hitesh Shah
For folks who prefer to use git://git.apache.org/tez.git as a read-only mirror, it now points to the correct source ( https://issues.apache.org/jira/browse/INFRA-6226 ). thanks -- Hitesh

Re: ask for receiving tez developer mailing list

2013-07-10 Thread Hitesh Shah
Hi Lianhui, Please send a mail to dev-subscr...@tez.incubator.apache.org thanks -- Hitesh On Jul 10, 2013, at 5:55 PM, Lianhui Wang wrote: > hi > i want to ask for receiving tez developer mailing list > > -- > thanks > > 王联辉(Lianhui Wang) > blog; http://blog.csdn.net/lance_123 > 兴趣方向:数据库,

Re: Tez Jars/Lib Exception

2013-07-22 Thread Hitesh Shah
This is the location of the tez jars in HDFS. The jars need to launch the AM on the cluster are required to be in a central location which in this case is HDFS. Refer to INSTALL.txt at the top of the source tree for more details. -- Hitesh On Jul 22, 2013, at 9:26 PM, Achal Soni wrote: > Wha

Re: Tez Jars/Lib Exception

2013-07-22 Thread Hitesh Shah
I am not sure what you mean by the jar containing the submission. All the tez jars ( from the distribution tarball ) are added as local resources for the AM as well as all vertices by default. -- Hitesh On Jul 22, 2013, at 10:59 PM, Achal Soni wrote: > Ah I see. And is it necessary to add the

Re: Tez Jars/Lib Exception

2013-07-23 Thread Hitesh Shah
Status.getModificationTime()); > >commonLocalResources.put("dag_job.jar", dagJarLocalRsrc); > > > This causes issues in Eclipse because there is no jar per se. > > > On Mon, Jul 22, 2013 at 11:25 PM, Hitesh Shah wrote: > >> I am not sure what you mean

Re: Tez Jars/Lib Exception

2013-07-23 Thread Hitesh Shah
educer classes? > > > On Tue, Jul 23, 2013 at 1:22 PM, Hitesh Shah wrote: > >> Hi Achal, >> >> I am not sure if that use-case is doable at the moment. It seems like what >> you are asking for is the ability to create a jar on the fly if a jar is >> not

Re: Running Tez on Cluster

2013-07-24 Thread Hitesh Shah
Siddharth Seth and Gunther H. have run quite a few tests on a 20 node cluster running both basic mapreduce jobs like sort/wordcount as well as some hive queries. Are you asking if tez works on a larger cluster? There were a few bugs observed and fixed as a result. You may likely find more edge-c

Re: Application Lifetime

2013-07-25 Thread Hitesh Shah
For now, until tez is extended to support multiple dags, you will need a new application id per dag. -- Hitesh On Jul 25, 2013, at 1:16 PM, Achal Soni wrote: > For a higher level application like Pig, what is the best practice for > Application ID's and such? > > For the time being, Pig will c

Re: Distributed Cache in Tez

2013-07-26 Thread Hitesh Shah
Hi Achal We want to force folks to use local resources as it makes the users more aware of how to use the cache. Pushing local files to distributed cache for each job does not bring any performance improvement. All it does is ensure that the local files are now available on the remote node i

Re: Distributed Cache in Tez

2013-07-26 Thread Hitesh Shah
to LocalResources. For example, Pig needs > to distribute jars to the nodes. These need to be added to the classpath. > Does DC do that which the LR way won't? > > Thanks, > Achal > > > On Fri, Jul 26, 2013 at 2:11 PM, Hitesh Shah wrote: > >> Hi Achal >

Re: Distributed Cache in Tez

2013-07-26 Thread Hitesh Shah
jars is an interesting point. Without a >> YARN/LocalResource-NM/Tez support to help edit the classpath (and even >> notify users that they need to do something to make that archive available >> via the classpath) its hard for users to do the right thing. >> >> Bikas &g

Re: Incubator PMC/Board report for Aug 2013 ([ppmc])

2013-08-07 Thread Hitesh Shah
Hi I went ahead and updated the August report for Tez with some basic updates since the last report in June. Feel free to edit or add more comments as needed. thanks -- Hitesh On Aug 7, 2013, at 2:39 PM, Marvin wrote: > > > Dear podling, > > This email was sent by an automated system on

Setting up the tez website - request for mentor help.

2013-08-08 Thread Hitesh Shah
Hi I recently filed https://issues.apache.org/jira/browse/INFRA-6641 which ends up pointing me to https://infra.apache.org/officers/webreq. The auth requirement for that request form seems to be ASF members/officers only. Could a mentor who has the necessary privileges file a request for setti

Re: Setting up the tez website - request for mentor help.

2013-08-09 Thread Hitesh Shah
u want? > > Alan. > > On Aug 8, 2013, at 2:34 PM, Hitesh Shah wrote: > >> Hi >> >> I recently filed https://issues.apache.org/jira/browse/INFRA-6641 which ends >> up pointing me to https://infra.apache.org/officers/webreq. The auth >> requirement for that

Re: tez programming

2013-08-13 Thread Hitesh Shah
Hi Zhangwei, Most of the basic documentation/design doc/talks are available at http://tez.incubator.apache.org/ For example code, look at the tez-mapreduce-examples module. In particular, the MRRSleepJob: https://git-wip-us.apache.org/repos/asf?p=incubator-tez.git;a=blob;f=tez-mapreduce-examp

Re: tez programming

2013-08-13 Thread Hitesh Shah
Manu Zhang wrote: > Hi, could you share some examples of Hive over Tez and Pig over Tez ? > Thanks > > > > On Wed, Aug 14, 2013 at 2:31 AM, Hitesh Shah wrote: > >> Hi Zhangwei, >> >> Most of the basic documentation/design doc/talks are available at &g

Heads up related to protobuf dependency upgrade to 2.5.0

2013-08-14 Thread Hitesh Shah
Hi folks Hadoop recently moved to using protobug-2.5.0. Tez depends on yarn and also uses protobufs, therefore, we are also planning to move to using 2.5.0 Folks having problems compiling against latest hadoop trunk can use the patch attached to https://issues.apache.org/jira/browse/TEZ-363. R

TEZ-398: Changes to the underlying Task/Input/Output/Processor engine layer

2013-08-29 Thread Hitesh Shah
Hi folks, Siddharth, Bikas and I have been working on looking at how to re-factor/clean up the tez-engine layer ( i.e the Input/Output/Processor and Task constructs ) to allow non-MR processors to be built and allow easy introduction of new inputs and outputs. I posted an initial design/overv

Re: Merge branch TEZ-398 to master

2013-09-24 Thread Hitesh Shah
@Vinod, All of the re-factor was targeted on the engine apis. These APIs are for developers to write implementations of Inputs/Outputs/Processors. At this point in time, almost all implementations of Inputs/Outputs/Processors (I/O/P), all the ones that are commonly used have been ported, have b

Re: Merge branch TEZ-398 to master

2013-09-24 Thread Hitesh Shah
Filed https://issues.apache.org/jira/browse/TEZ-492 Working on this - should have a patch shortly. -- Hitesh On Sep 24, 2013, at 2:12 PM, Bikas Saha wrote: > rename engine.api to runtime.api and engine to > runtime.library

Re: Setting Java options on the DagAM

2013-10-14 Thread Hitesh Shah
Hi Mark, Code snippet from TezClient#createApplicationSubmissionContext(): vargs.add(amConfig.getAMConf().get(TezConfiguration.TEZ_AM_JAVA_OPTS, TezConfiguration.DEFAULT_TEZ_AM_JAVA_OPTS)); It uses the TezConfiguration from AMConfiguration to extract the java opts for the AM. For lo

Re: Incubator PMC/Board report for Nov 2013 ([ppmc])

2013-11-06 Thread Hitesh Shah
Already updated. I believe Arun and Owen (mentors ) also already signed off on the report. thanks -- HItesh On Nov 6, 2013, at 1:16 PM, Alan Gates wrote: > Anybody working on putting this together? It's due today. > > Alan. > > On Nov 6, 2013, at 8:25 AM, Marvin wrote: > >> >> >> Dear pod

Re: iss...@tez.incubator.apache.org ?

2013-11-06 Thread Hitesh Shah
Thanks for the heads up Koji. Should have a fixed docs set pushed soon. -- Hitesh On Nov 6, 2013, at 2:17 PM, Koji Noguchi wrote: > Hi. > > I've been wondering why I wasn't getting any jira updates for Tez even though > I subscribed to dev@tez.incubator.apache.org a while back. > Then today,

Re: Newbie in the dev list

2013-11-13 Thread Hitesh Shah
Thanks Austin. Looking forward to your contributions. If you are familiar with contributing to Hadoop, the process is pretty much the same in Tez. Take a look at http://wiki.apache.org/hadoop/HowToContribute on general contribution guidelines. For new folks, there are some jiras that have marke

[VOTE] Release Apache Tez-0.2.0-incubating RC0

2013-11-25 Thread Hitesh Shah
Hello folks, I have created a tez-0.2.0-incubating release candidate (rc0). This is the first release for Tez. Therefore, please do not be gentle. GIT source tag: https://git-wip-us.apache.org/repos/asf/incubator-tez/repo?p=incubator-tez.git;a=log;h=refs/tags/release-0.2.0-rc0 Staging site: h

Re: [VOTE] Release Apache Tez-0.2.0-incubating RC0

2013-11-28 Thread Hitesh Shah
t;> Tsuyoshi >>> >>> On Thu, Nov 28, 2013 at 10:50 AM, Mark Wagner >>> wrote: >>>> +1 (non-binding): Built from source tag, deployed, and ran some Pig >>>> scripts w/ groups and joins. >>>> >>>> -Mark >>>> >

[ANNOUNCE] Apache Tez 0.2.0-incubating Released

2013-12-02 Thread Hitesh Shah
release possible. thanks Hitesh Shah on behalf of The Apache Tez Team

Fwd: Release Verification Checklist

2013-12-03 Thread Hitesh Shah
Useful information for folks when verifying releases. (The checklist is still getting firmed up) thanks -- Hitesh Begin forwarded message: > From: Marvin Humphrey > Date: November 27, 2013 12:45:51 PM PST > To: "gene...@incubator.apache.org" > Subject: Release Verification Checklist > Reply-T

Re: 0.2.1 version

2013-12-05 Thread Hitesh Shah
Hi Tsuyoshi, Affects version should be 0.2.0, target version would be 0.2.1 as the fix is targeted for that release. -- Hitesh On Dec 5, 2013, at 11:52 AM, Tsuyoshi OZAWA wrote: > Hi, > > I tried to change the affect version of TEZ-653 because tez-0.2.0 has > already released. In previous

Re: 0.2.1 version

2013-12-05 Thread Hitesh Shah
I have added a 0.2.1 version to Tez JIRA for all patches that should be targeted to it. There have been quite a few patches which have gone into master so we may need to do some vetting to see which patches go into 0.2.1. thanks -- Hitesh On Dec 5, 2013, at 11:52 AM, Tsuyoshi OZAWA wrote: > Hi

Fwd: How to setup tez cluster for TPC-DS benchmark

2013-12-05 Thread Hitesh Shah
Fwding to dev@ list as there are probably some folks who do not monitor the user@ lists at the moment. -- Hitesh Begin forwarded message: > From: Tsuyoshi OZAWA > Date: December 5, 2013 11:33:43 AM PST > To: u...@tez.incubator.apache.org > Subject: How to setup tez cluster for TPC-DS benchmark

Re: Tez compatibility with MR

2014-01-10 Thread Hitesh Shah
Hi Jonathan Most of the points below summarize the views of the current Tez devs: We have tried to get a lot of MR aspects to work on Tez but not to full completion as most of the current contributors moved on to focus on other aspects of Tez. With respect to MR, some features we have not got

MR compatibility - related to TEZ-722

2014-01-15 Thread Hitesh Shah
Hi folks, Jonathan raised the issue in TEZ-722 which has implications with respect to what version of Hadoop that is Tez compatible with. A recent change in hadoop branch-2 ( 2.4.0 ) implies Tez master branch no longer compiles against hadoop-2.4.0-SNAPSHOT. There seem to be multiple options

Re: Minutes of the meeting with Pig team

2014-01-23 Thread Hitesh Shah
At the moment, we are looking at integrating Tez with YARN's application history server mostly for data-related integration. YARN will be providing webservices that provides access to this data and therefore allow anyone to build a UI. However, there are no concrete plans/timelines around the UI

Re: How to decide number of partitions in a Map Output

2014-01-23 Thread Hitesh Shah
Yes. It looks at the size of outputs generated by the tasks in the previous vertex and determines the no. of tasks to run based on the configured amount of input data size per task. -- Hitesh On Jan 23, 2014, at 7:59 PM, Manu Zhang wrote: > So that's what *ShuffleVertexManager.determineParalle

Fwd: Ten Years of Google Summer of Code - GSoC 2014 is coming

2014-01-28 Thread Hitesh Shah
Begin forwarded message: > From: Ulrich Stärk > Date: January 28, 2014 2:05:47 PM PST > To: p...@apache.org > Subject: Ten Years of Google Summer of Code - GSoC 2014 is coming > Reply-To: priv...@hadoop.apache.org > > Hello PMCs, > > Google Summer of Code [1] is celebrating it’s tenth year an

[DISCUSS] Targeting a 0.3.0-incubating release

2014-02-14 Thread Hitesh Shah
Hello folks, Given that our last release was in early December, 2013, it seems like a good time to target a new release. I believe most of the critical security related fixes have gone in and the few that remain in TEZ-669 could be targeted for a future release. I have marked a few jiras that

Re: [DISCUSS] Targeting a 0.3.0-incubating release

2014-02-18 Thread Hitesh Shah
se developers be recommended to upgrade > from 0.2 to 0.3? > We should clarify the polity against the application developers > though I know Tez is in alpha stage. What do you think? > > Thanks, Tsuyoshi > > > On Fri, Feb 14, 2014 at 2:53 PM, Hitesh Shah wrote: >>

Re: [DISCUSS] Targeting a 0.3.0-incubating release

2014-02-21 Thread Hitesh Shah
+1 >>> >>> Thanks, >>> Gunther. >>> >>> >>> On Thu, Feb 20, 2014 at 4:58 PM, Bikas Saha wrote: >>> >>>> +1. All outstanding blockers are resolved now. >>>> >>>> -Orig

[VOTE] Release Apache Tez-0.3.0-incubating RC0

2014-02-23 Thread Hitesh Shah
Hello folks, I have created a tez-0.3.0-incubating release candidate (rc0). This is the second release for Tez. GIT source tag: https://git-wip-us.apache.org/repos/asf/incubator-tez/repo?p=incubator-tez.git;a=log;h=refs/tags/release-0.3.0-incubating-rc0 Staging site: https://dist.apache.org/r

Release version 0.4.0 created

2014-02-24 Thread Hitesh Shah
Hi folks All committers should now use version 0.4.0 when marking a jira as Fixed for all patches committed to master. thanks -- Hitesh

Re: [VOTE] Release Apache Tez-0.3.0-incubating RC0

2014-02-25 Thread Hitesh Shah
Chen, Could you file bugs for the exception errors that you are seeing? thanks -- Hitesh On Feb 24, 2014, at 2:33 PM, Chen He wrote: > +0 non-binding > > download the source code, compile and deploy on Fedora 17; > successfully run orderedwordcount based on hadoop-2.2; > > turn on DEBUG of

Re: [VOTE] Release Apache Tez-0.3.0-incubating RC0

2014-02-25 Thread Hitesh Shah
lease is pushed. >> - Built from source >> - Deployed to a single-node YARN cluster >> - Ran some sample Tez jobs >> >> Jason >> >> >> On 02/23/2014 01:31 PM, Hitesh Shah wrote: >> >>> Hello folks, >>> >>> I have c

Fwd: [CANCEL] [VOTE] Release Apache Tez-0.3.0-incubating RC0

2014-02-25 Thread Hitesh Shah
Begin forwarded message: > From: Hitesh Shah > Date: February 25, 2014 2:43:10 PM PST > To: gene...@incubator.apache.org > Subject: [CANCEL] [VOTE] Release Apache Tez-0.3.0-incubating RC0 > > The vote is being cancelled due to the Pig team finding an issue when testing >

Re: [VOTE] Release Apache Tez-0.3.0-incubating RC0

2014-02-25 Thread Hitesh Shah
>>> >>> On Tue, Feb 25, 2014 at 12:22 PM, Jason Lowe wrote: >>> >>>> +1 >>>> >>>> - Verified signature >>>> - MD5 digest matched but tez-0.3.0-incubating.tar.gz.sha1 did not match >>>> gpg --print-md sha1 tez-0.3.0-incu

[VOTE] Release Apache Tez-0.3.0-incubating RC1

2014-02-25 Thread Hitesh Shah
Hello folks, I have created a tez-0.3.0-incubating release candidate (rc1). This is the second release for Tez. The previous RC was cancelled due to a bug discovered when the RC was tested against downstream projects. GIT source tag: https://git-wip-us.apache.org/repos/asf/incubator-tez/repo?

Re: [VOTE] Release Apache Tez-0.3.0-incubating RC1

2014-02-28 Thread Hitesh Shah
gt;> http://hortonworks.com/ >> >> >> On Fri, Feb 28, 2014 at 8:05 AM, Jason Lowe wrote: >>> +1 (non-binding) >>> >>> - Verified signatures >>> - Built from source, installed on single-node cluster >>> - Ran some sample Tez job

Re: [VOTE] Release Apache Tez-0.3.0-incubating RC1

2014-02-28 Thread Hitesh Shah
ortonworks Inc. >>> http://hortonworks.com/ >>> >>> >>> On Fri, Feb 28, 2014 at 8:05 AM, Jason Lowe wrote: >>>> +1 (non-binding) >>>> >>>> - Verified signatures >>>> - Built from source, installed on single-

[RESULT][VOTE] Release Apache Tez-0.3.0-incubating RC1

2014-02-28 Thread Hitesh Shah
Hello all, Closing the vote since 72 hours have passed. With the required 3 IPMC votes and no -1s, this vote passes. +1s (binding - IPMC ) Arun C Murthy Alan Gates Mahadev Konar +1s (non-binding) Hitesh Shah Chen He Daniel Dai Tsuyoshi Ozawa Rohini Palaniswamy Jonathan Eagles Siddharth Seth

[ANNOUNCE] Apache Tez 0.3.0-incubating Released

2014-03-01 Thread Hitesh Shah
release possible. thanks Hitesh Shah on behalf of The Apache Tez Team

Re: AMConfiguration Necessity

2014-03-03 Thread Hitesh Shah
Hello Chen, If you are referring to org.apache.tez.client. AMConfiguration, it is not really a Configuration object. If you look more into it, it effectively governs the construction of the AM's container as well as how the application is submitted to YARN i.e. queue info for YARN and local re

Re: Cannot submit Tez application

2014-05-11 Thread Hitesh Shah
BCC??ing dev@tez. Re-directing to hive list. @Azury Yu, would you mind providing the query details to the hive folks. Seems like an issue in the DAG plan generation in Hive. thanks ?? Hitesh On May 10, 2014, at 8:26 PM, AzuryYu wrote: > Hi, > > I built Tez-0.5 against hadoop-2.4.0, and I'v

Re: Cannot submit Tez application

2014-05-11 Thread Hitesh Shah
ge Tez cannot find hdfs-site. > > Ps. All fine if I disable Tez. > > > Sent from my iPhone5s > >> On 2014??5??11??, at 23:01, "Hitesh Shah" wrote: >> >> BCC??ing dev@tez. >> >> Re-directing to hive list. >> >> @Azury Yu,

Re: Cannot submit Tez application

2014-05-11 Thread Hitesh Shah
@Azury Yu, a question - is the hdfs-site and core-site that is configured on the cluster different that the one in use on the client? Also, is yarn.application.classpath set up such that the cluster??s hdfs-site and core-site are not in that class path? At the moment, it seems like there is a

Re: Tez HIve problems.

2014-05-13 Thread Hitesh Shah
@Archit, if the hive shell was shut down, can you obtain all the tez logs using: $yarn logs -applicationId Or, for a running query, go to the RM UI, find the application id and use the logs link to view/download the logs. thanks — Hitesh On May 12, 2014, at 11:57 PM, Archit Thakur wrote:

Re: Cannot submit Tez application

2014-05-14 Thread Hitesh Shah
Is there a ??test-cluster?? configured as the host for your namenode in any of the config files? It seems like it is being picked up from some loaded config file and then being used to set up the AM. thanks ?? HItesh On May 11, 2014, at 9:20 AM, AzuryYu wrote: > Hi Hitesh, > > Thanks for y

Re: 0.5 incubating release

2014-05-16 Thread Hitesh Shah
+1 to get the usability changes to users quickly. I would also like to get the YARN timeline server integration and staging directory cleanup as part of the 0.5.0 release and will update jiras accordingly ( assuming no one has any concerns ). The timeline server change will move the hadoop d

Re: 0.5 incubating release

2014-05-17 Thread Hitesh Shah
Thanks for the heads-up, Tsuyoshi. The issue was seen earlier and handled in TEZ-939 ( the fix is present in the 0.4.0 release ) thanks — Hitesh On May 17, 2014, at 1:54 AM, Tsuyoshi OZAWA wrote: > +1(non-binding) for having effort to stabilize APIs quickly. > >> The timeline server change wi

Issues relating to assigning of Jiras

2014-05-19 Thread Hitesh Shah
https://issues.apache.org/jira/browse/INFRA-7675 ended up causing a change for all project’s roles. Filed https://issues.apache.org/jira/browse/INFRA-7766 for now. Does anyone believe only committers should be allowed to assign jiras? thanks — Hitesh

hadoop dependency change

2014-05-21 Thread Hitesh Shah
Hi folks Just wanted to warn everyone that given that TEZ-1066 has gone in ( integration with YARN timeline ), the hadoop dependency has been bumped to 2.4.0. For folks who want to use either 2.2.0 or 2.3.0, please change the top level pom.xml to modify the hadoop.version and then run the buil

Re: About DAGAppMaster working directory

2014-05-30 Thread Hitesh Shah
Hi Chen, The path to the plan file on HDFS would be the staging directory. As for within the context of the AM, it will be available in the working directory of the AM container. YARN downloads local resources and copies them ( it actually uses symlinks ) into the container’s working dir. The abso

Re: Debugging Tez release

2014-06-03 Thread Hitesh Shah
Hello Anton, The HDP release of Tez is not an exact match of the Apache release. You can look at http://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.1.1/bk_releasenotes_hdp_2.1/content/ch_relnotes-hdp-2.1.1-tez.html for the full list of additional patches. Feel free to send any questions regd. Te

Re: meet a problem when using "pig -x tez_local"

2014-06-05 Thread Hitesh Shah
@Chen, this might be a bug in the tez local execution code. Can you confirm that ObjectRegistryFactory::getObjectRegistry returns a non-null value? thanks — Hitesh On Wed, Jun 4, 2014 at 4:21 PM, Chen He wrote: > I am using pig -x tez_local to test tez local mode and met following > problems (

Re: meet a problem when using "pig -x tez_local"

2014-06-05 Thread Hitesh Shah
gt; > > On Thu, Jun 5, 2014 at 2:59 PM, Chen He wrote: > > > Hi Hitesh > > > > Yes, objectRegistry returns NPE. I saw it is a inject static variable. > > > > Regards! > > > > Chen > > > > > > On Thu, Jun 5, 2014 at 9:15 AM, Hi

Re: Tez configuration initialization ignoring JobConfigurable

2014-06-06 Thread Hitesh Shah
Hi Subroto Could you provide some more context on what you are trying to do? Are you trying to run MR-on-Tez? or a native Tez job? If you could provide us with some code showing what you are trying to do, we can help further. There are probably some bugs in the MR compatibility that we may have no

Re: Tez configuration initialization ignoring JobConfigurable

2014-06-06 Thread Hitesh Shah
e base. ** I patched the Tez-common so that TezUtils.createConfFromUserPayload returns a JobConf instead on Configuration which solves the problem(may not be a good solution). On Fri, Jun 6, 2014 at 6:57 PM, Hitesh Shah wrote: Hi Subroto Could you provide some more context on what you are tryi

[ANNOUNCE] New Apache Tez Committers

2014-06-09 Thread Hitesh Shah
Hello folks I am pleased to announce that the Apache Tez PPMC voted in the following folks as committers for the Apache Tez project. - Tassapol Athiapinya - Mohammad Kamrul Islam - Rohini Palaniswamy - Rajesh Balamohan We appreciate all the work that you folks

Re: Deadlock in DAGAppMaster during shutdown.

2014-06-10 Thread Hitesh Shah
Hi Subroto, Could you file a jira for this with the output of jstack for the AM process and the AM logs? thanks — Hitesh On Jun 10, 2014, at 3:26 PM, Subroto Sanyal wrote: > Hi, > > I have build the Tez jars from the git repository today; still, I see the > DAGAppMaster running even after

Re: Tez DAG question

2014-06-16 Thread Hitesh Shah
Hi Jeff A few questions to clarify your use-case: - You seem to have clarified that you need each stage to write to HDFS but just wanted to confirm in any case whether this is a strict requirement or something just done for faster recovery in case of failures? - Would I be correct in sayi

Re: [DISCUSS] Graduation

2014-06-17 Thread Hitesh Shah
Hi Chris, Thanks for the vote of confidence on the Tez community. Will wait for other folks to chime in the next couple of days and get started on the listed steps towards graduation. thanks — Hitesh On Jun 16, 2014, at 8:07 PM, Mattmann, Chris A (3980) wrote: > Hey Guys, > > So at first I

Re: Tez graduation [Was: Request for mentor assessment]

2014-06-23 Thread Hitesh Shah
Hi folks, To be very clear, the project's source of truth has always been its status file: https://svn.apache.org/repos/asf/incubator/public/trunk/content/projects/tez.xml For the tez website, http://tez.incubator.apache.org/team-list.html, the onus was for the committers to test out their co

Re: Created branch branch-0.4.1-incubating

2014-06-23 Thread Hitesh Shah
Hey Gopal Seems like a relevant bug fix to put into 0.4.1. If it does not break compatibility, just go ahead and merge it into the 0.4.1 branch and change fix version to 0.4.1 when committed. — Hitesh On Jun 23, 2014, at 4:06 PM, Gopal V wrote: > On 6/23/14, 11:24 AM, Siddharth Seth wrote

Re: [VOTE] Graduate Apache Tez from the Incubator

2014-06-24 Thread Hitesh Shah
ject: > > * Alan Gates > * Arun C. Murthy > * Ashutosh Chauhan > * Bill Graham > * Bikas Saha > * Chris Douglas > * Chris Mattmann > * Daryn Sharp > * Devaraj Das > * Gopal Vijayaraghavan > * Gunther Hagleitner > * Hitesh Shah > * Jit