Re: Plans of moving towards JDK7 in trunk

[Vinod Kumar Vavilapalli]

Hey all,

This one started as an innocuous thread of enabling JDK7 on trunk and now it 
seems like (haven't still finished reading the entire thing, and I started a 
while ago) it has become a full blown proposal on 2.x, 3.x and 4.x releases. 
Some of us haven't been tracking this (at least me and a few others who 
indicated offline as such) assuming this is only about letting Jenkins run 
JDK7, but it has the potential to impact all future work.

I propose we fork this thread into a new one which clarifies the topic clearly 
for others to follow too.

Thanks,
+Vinod

On Jun 23, 2014, at 1:53 PM, sanjay Radia  wrote:

> 
> On Jun 21, 2014, at 8:01 AM, Andrew Wang  wrote:
> 
>> This is why I'd like to keep my original proposal on the table: keep going
>> with branch-2 in the near term, while working towards a JDK8-based Hadoop 3
>> by April next year. It doesn't need to be a big bang release either. I'd be
>> delighted if we could rolling upgrade from one to the other. I just didn't
>> want to rule out the inclusion of some very compelling feature outright.
>> Trust me though, I'd be the first person to ask about compatibility if such
>> a feature does come up.
> 
> 
> Given your above statement  on compatibility (such as rolling upgrades),  it 
> should be fine for the JDK8-based-Hadoop-release to not be 3.0 and instead 
> merely be 2.x? Or do you have any incompatible changes to Hadoop protocol or 
> APIs in mind during the same time period?
> 
> sanjay
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: hadoop-2.5 - June end?

[Vinod Kumar Vavilapalli]

I started reviewing YARN-1492 and synced up offline with Sangjin and Chris who 
are leading this. I proposed that we should get the reviews and commits going 
as they go and then mark the feature alpha, beta or stable depending on how far 
we are ahead with the feature and how it aligns with the release in progress.

+Vinod

On Jun 23, 2014, at 2:30 PM, Sangjin Lee  wrote:

> It would be great if we can consider the shared cache (YARN-1492) as part
> of 2.5. It obviously will depend on the reviews from folks, but the feature
> itself should be reasonably solid.
> 
> Thanks,
> Sangjin
> 
> 
> On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy  wrote:
> 
>> Thanks Karthik!
>> 
>> I've updated https://wiki.apache.org/hadoop/Roadmap with features which
>> are very close to completion. Let's see if that makes sense and if we get
>> any further feedback.
>> 
>> Arun
>> 
>> On Jun 23, 2014, at 2:09 PM, Karthik Kambatla  wrote:
>> 
>>> I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
>> might
>>> need some help from someone more familiar with HDFS.
>>> 
>>> 
>>> On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy 
>> wrote:
>>> 
 Folks,
 
 I'd appreciate some help here. Due to family reasons (all good ones),
 I'll be away for a couple of weeks. Can someone else pick up the RM
>> duties
 for hadoop-2.5? Maybe Andrew since he's expressed interest in the past?
>> I
 will pick up the thread again for hadoop-2.6, but I don't want to block
 hadoop-2.5 due to my non-availability.
 
 thanks,
 Arun
 
 On Jun 9, 2014, at 9:39 AM, Arun C Murthy  wrote:
 
> Folks,
> 
> As you can see from the Roadmap wiki, it looks like several items are
 still a bit away from being ready.
> 
> I think rather than wait for them, it will be useful to create an
 intermediate release (2.5) this month - I think ATS security is pretty
 close, so we can ship that. I'm thinking of creating hadoop-2.5 by end
>> of
 the month, with a branch a couple of weeks prior.
> 
> Thoughts?
> 
> thanks,
> Arun
> 
 
 
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or
>> entity to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the
>> reader
 of this message is not the intended recipient, you are hereby notified
>> that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
>> immediately
 and delete it from your system. Thank You.
 
>> 
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/hdp/
>> 
>> 
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: hadoop-2.5 - June end?

[Vinod Kumar Vavilapalli]

Tx Jian, updated the wiki.

+Vinod

On Jun 23, 2014, at 3:13 PM, Jian He  wrote:

> Just found the roadmap includes a wrong jira link for "RM Webservices
> YARN-1731", it should be YARN-1695
> .
> 
> Jian
> 
> 
> On Mon, Jun 23, 2014 at 2:51 PM, Karthik Kambatla 
> wrote:
> 
>> While YARN-1492 would greatly benefit performance, I think it might be a
>> little too ambitious at this point to include it in 2.5 without delaying
>> it.
>> 
>> As I mentioned before and from my offline conversation with Arun, we might
>> want to release 2.6 end of July and that might be a better vehicle for
>> YARN-1492.
>> 
>> 
>> On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:
>> 
>>> It would be great if we can consider the shared cache (YARN-1492) as part
>>> of 2.5. It obviously will depend on the reviews from folks, but the
>> feature
>>> itself should be reasonably solid.
>>> 
>>> Thanks,
>>> Sangjin
>>> 
>>> 
>>> On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
>>> wrote:
>>> 
 Thanks Karthik!
 
 I've updated https://wiki.apache.org/hadoop/Roadmap with features
>> which
 are very close to completion. Let's see if that makes sense and if we
>> get
 any further feedback.
 
 Arun
 
 On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 
>>> wrote:
 
> I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
 might
> need some help from someone more familiar with HDFS.
> 
> 
> On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy >> 
 wrote:
> 
>> Folks,
>> 
>> I'd appreciate some help here. Due to family reasons (all good
>> ones),
>> I'll be away for a couple of weeks. Can someone else pick up the RM
 duties
>> for hadoop-2.5? Maybe Andrew since he's expressed interest in the
>>> past?
 I
>> will pick up the thread again for hadoop-2.6, but I don't want to
>>> block
>> hadoop-2.5 due to my non-availability.
>> 
>> thanks,
>> Arun
>> 
>> On Jun 9, 2014, at 9:39 AM, Arun C Murthy 
>>> wrote:
>> 
>>> Folks,
>>> 
>>> As you can see from the Roadmap wiki, it looks like several items
>> are
>> still a bit away from being ready.
>>> 
>>> I think rather than wait for them, it will be useful to create an
>> intermediate release (2.5) this month - I think ATS security is
>> pretty
>> close, so we can ship that. I'm thinking of creating hadoop-2.5 by
>> end
 of
>> the month, with a branch a couple of weeks prior.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>> 
>> 
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or
 entity to
>> which it is addressed and may contain information that is
>>> confidential,
>> privileged and exempt from disclosure under applicable law. If the
 reader
>> of this message is not the intended recipient, you are hereby
>> notified
 that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender
 immediately
>> and delete it from your system. Thank You.
>> 
 
 --
 Arun C. Murthy
 Hortonworks Inc.
 http://hortonworks.com/hdp/
 
 
 
 --
 CONFIDENTIALITY NOTICE
 NOTICE: This message is intended for the use of the individual or
>> entity
>>> to
 which it is addressed and may contain information that is confidential,
 privileged and exempt from disclosure under applicable law. If the
>> reader
 of this message is not the intended recipient, you are hereby notified
>>> that
 any printing, copying, dissemination, distribution, disclosure or
 forwarding of this communication is strictly prohibited. If you have
 received this communication in error, please contact the sender
>>> immediately
 and delete it from your system. Thank You.
 
>>> 
>> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended r

Re: [VOTE] Release Apache Hadoop 0.23.11

[Zhijie Shen]

+1 (non-binding)

built it from source and ram some MR examples successfully.


On Mon, Jun 23, 2014 at 11:59 PM, Mit Desai 
wrote:

> +1 (non-binding)
>
> Tested on: Fedora17
> -Successful build from src
> -Verified Signature
> -Deployed source to my single node cluster and ran couple of sample MR jobs
>
>
> -Mit Desai
>
>
>
>
>
> On 6/19/14, 10:14 AM, "Thomas Graves" 
> wrote:
>
> >Hey Everyone,
> >
> >There have been various bug fixes that have went into
> >branch-0.23 since the 0.23.10 release.  We think its time to do a 0.23.11.
> >
> >This is also the last planned release off of branch-0.23 we plan on doing.
> >
> >The RC is available at:
> >http://people.apache.org/~tgraves/hadoop-0.23.11-candidate-0/
> >
> >
> >The RC Tag in svn is here:
> >http://svn.apache.org/viewvc/hadoop/common/tags/release-0.23.11-rc0/
> >
> >The maven artifacts are available via repository.apache.org.
> >
> >Please try the release and vote; the vote will run for the usual 7 days
> >til June 26th.
> >
> >I am +1 (binding).
> >
> >thanks,
> >Tom Graves
> >
> >
> >
> >
>
>


-- 
Zhijie Shen
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: hadoop-2.5 - June end?

[Akira AJISAKA]

Extended attributes (HDFS-2006) is already merged to branch-2,
so I'll update the Roadmap wiki to change it from 2.6 to 2.5.
Please correct me if I am wrong.

Thanks,
Akira

(2014/06/23 16:28), Tsuyoshi OZAWA wrote:

IMHO, I'd like to +1 to include YARN-1492 and work-preserving restart
in 2.6. I believe they are good improvement, but they are
work-in-progress and it takes a bit time to be done.

Thanks,
- Tsuyoshi

On Tue, Jun 24, 2014 at 7:13 AM, Jian He  wrote:

Just found the roadmap includes a wrong jira link for "RM Webservices
YARN-1731", it should be YARN-1695
.

Jian


On Mon, Jun 23, 2014 at 2:51 PM, Karthik Kambatla 
wrote:


While YARN-1492 would greatly benefit performance, I think it might be a
little too ambitious at this point to include it in 2.5 without delaying
it.

As I mentioned before and from my offline conversation with Arun, we might
want to release 2.6 end of July and that might be a better vehicle for
YARN-1492.


On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:


It would be great if we can consider the shared cache (YARN-1492) as part
of 2.5. It obviously will depend on the reviews from folks, but the

feature

itself should be reasonably solid.

Thanks,
Sangjin


On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
wrote:


Thanks Karthik!

I've updated https://wiki.apache.org/hadoop/Roadmap with features

which

are very close to completion. Let's see if that makes sense and if we

get

any further feedback.

Arun

On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 

wrote:



I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I

might

need some help from someone more familiar with HDFS.


On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy 


wrote:



Folks,

I'd appreciate some help here. Due to family reasons (all good

ones),

I'll be away for a couple of weeks. Can someone else pick up the RM

duties

for hadoop-2.5? Maybe Andrew since he's expressed interest in the

past?

I

will pick up the thread again for hadoop-2.6, but I don't want to

block

hadoop-2.5 due to my non-availability.

thanks,
Arun

On Jun 9, 2014, at 9:39 AM, Arun C Murthy 

wrote:



Folks,

As you can see from the Roadmap wiki, it looks like several items

are

still a bit away from being ready.


I think rather than wait for them, it will be useful to create an

intermediate release (2.5) this month - I think ATS security is

pretty

close, so we can ship that. I'm thinking of creating hadoop-2.5 by

end

of

the month, with a branch a couple of weeks prior.


Thoughts?

thanks,
Arun





--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or

entity to

which it is addressed and may contain information that is

confidential,

privileged and exempt from disclosure under applicable law. If the

reader

of this message is not the intended recipient, you are hereby

notified

that

any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender

immediately

and delete it from your system. Thank You.



--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/hdp/



--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or

entity

to

which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the

reader

of this message is not the intended recipient, you are hereby notified

that

any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender

immediately

and delete it from your system. Thank You.







--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.

Where is hadoop result?

[EdwardKing]

I use hadoop 2.2.0 under Redhat Linux,I run hadoop mapreduce examples like 
follows:

[yarn@localhost sbin]$ hadoop jar 
/opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.2.0.jar
 pi 
-Dmapreduce.clientfactory.class.name=org.apache.hadoop.mapred.YarnClientFactory 
-libjars 
/opt/yarn/hadoop-2.2.0/share/hadoop/mapreduce/hadoop-mapreduce-client-jobclient-2.2.0.jar
 16 1
Number of Maps  = 16
Samples per Map = 1
Wrote input for Map #0
Wrote input for Map #1
Wrote input for Map #2
Wrote input for Map #3
Wrote input for Map #4
Wrote input for Map #5
Wrote input for Map #6
Wrote input for Map #7
Wrote input for Map #8
Wrote input for Map #9
Wrote input for Map #10
Wrote input for Map #11
Wrote input for Map #12
Wrote input for Map #13
Wrote input for Map #14
Wrote input for Map #15
Starting Job
14/06/22 19:57:58 INFO client.RMProxy: Connecting to ResourceManager at 
/0.0.0.0:8032
14/06/22 19:57:59 INFO input.FileInputFormat: Total input paths to process : 16
14/06/22 19:57:59 INFO mapreduce.JobSubmitter: number of splits:16
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.classpath.files is 
deprecated. Instead, use mapreduce.job.classpath.files
14/06/22 19:57:59 INFO Configuration.deprecation: user.name is deprecated. 
Instead, use mapreduce.job.user.name
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.jar is deprecated. 
Instead, use mapreduce.job.jar
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files.filesizes 
is deprecated. Instead, use mapreduce.job.cache.files.filesizes
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files is 
deprecated. Instead, use mapreduce.job.cache.files
14/06/22 19:57:59 INFO Configuration.deprecation: 
mapred.map.tasks.speculative.execution is deprecated. Instead, use 
mapreduce.map.speculative
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.reduce.tasks is 
deprecated. Instead, use mapreduce.job.reduces
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.value.class is 
deprecated. Instead, use mapreduce.job.output.value.class
14/06/22 19:57:59 INFO Configuration.deprecation: 
mapred.reduce.tasks.speculative.execution is deprecated. Instead, use 
mapreduce.reduce.speculative
14/06/22 19:57:59 INFO Configuration.deprecation: 
mapred.used.genericoptionsparser is deprecated. Instead, use 
mapreduce.client.genericoptionsparser.used
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.map.class is 
deprecated. Instead, use mapreduce.job.map.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.job.name is 
deprecated. Instead, use mapreduce.job.name
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.reduce.class is 
deprecated. Instead, use mapreduce.job.reduce.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.inputformat.class 
is deprecated. Instead, use mapreduce.job.inputformat.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.input.dir is 
deprecated. Instead, use mapreduce.input.fileinputformat.inputdir
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.dir is 
deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir
14/06/22 19:57:59 INFO Configuration.deprecation: mapreduce.outputformat.class 
is deprecated. Instead, use mapreduce.job.outputformat.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.map.tasks is 
deprecated. Instead, use mapreduce.job.maps
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.cache.files.timestamps 
is deprecated. Instead, use mapreduce.job.cache.files.timestamps
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.output.key.class is 
deprecated. Instead, use mapreduce.job.output.key.class
14/06/22 19:57:59 INFO Configuration.deprecation: mapred.working.dir is 
deprecated. Instead, use mapreduce.job.working.dir
14/06/22 19:58:00 INFO mapreduce.JobSubmitter: Submitting tokens for job: 
job_1403492007236_0001
14/06/22 19:58:01 INFO impl.YarnClientImpl: Submitted application 
application_1403492007236_0001 to ResourceManager at /0.0.0.0:8032
14/06/22 19:58:01 INFO mapreduce.Job: The url to track the job: 
http://localhost:8088/proxy/application_1403492007236_0001/
14/06/22 19:58:01 INFO mapreduce.Job: Running job: job_1403492007236_0001
14/06/22 19:58:22 INFO mapreduce.Job: Job job_1403492007236_0001 running in 
uber mode : false
14/06/22 19:58:22 INFO mapreduce.Job:  map 0% reduce 0%
14/06/22 20:01:45 INFO mapreduce.Job:  map 6% reduce 0%
14/06/22 20:01:52 INFO mapreduce.Job:  map 13% reduce 0%
14/06/22 20:01:53 INFO mapreduce.Job:  map 19% reduce 0%
14/06/22 20:01:56 INFO mapreduce.Job:  map 38% reduce 0%
14/06/22 20:02:36 INFO mapreduce.Job:  map 50% reduce 15%
14/06/22 20:02:37 INFO mapreduce.Job:  map 69% reduce 15%
14/06/22 20:02:39 INFO mapreduce.Job:  map 69% reduce 23%
14/06/22 20:03:05 INFO mapreduce.Job:  map 94% reduce 23%
14/06/22 20:03:06 INFO mapreduce.Job:  map 100% reduce 31%
14/06/22 20:03:08 INFO mapreduce.Job:  map 100% reduce 

Re: hadoop-2.5 - June end?

[Tsuyoshi OZAWA]

IMHO, I'd like to +1 to include YARN-1492 and work-preserving restart
in 2.6. I believe they are good improvement, but they are
work-in-progress and it takes a bit time to be done.

Thanks,
- Tsuyoshi

On Tue, Jun 24, 2014 at 7:13 AM, Jian He  wrote:
> Just found the roadmap includes a wrong jira link for "RM Webservices
> YARN-1731", it should be YARN-1695
> .
>
> Jian
>
>
> On Mon, Jun 23, 2014 at 2:51 PM, Karthik Kambatla 
> wrote:
>
>> While YARN-1492 would greatly benefit performance, I think it might be a
>> little too ambitious at this point to include it in 2.5 without delaying
>> it.
>>
>> As I mentioned before and from my offline conversation with Arun, we might
>> want to release 2.6 end of July and that might be a better vehicle for
>> YARN-1492.
>>
>>
>> On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:
>>
>> > It would be great if we can consider the shared cache (YARN-1492) as part
>> > of 2.5. It obviously will depend on the reviews from folks, but the
>> feature
>> > itself should be reasonably solid.
>> >
>> > Thanks,
>> > Sangjin
>> >
>> >
>> > On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
>> > wrote:
>> >
>> > > Thanks Karthik!
>> > >
>> > > I've updated https://wiki.apache.org/hadoop/Roadmap with features
>> which
>> > > are very close to completion. Let's see if that makes sense and if we
>> get
>> > > any further feedback.
>> > >
>> > > Arun
>> > >
>> > > On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 
>> > wrote:
>> > >
>> > > > I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
>> > > might
>> > > > need some help from someone more familiar with HDFS.
>> > > >
>> > > >
>> > > > On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy > >
>> > > wrote:
>> > > >
>> > > >> Folks,
>> > > >>
>> > > >> I'd appreciate some help here. Due to family reasons (all good
>> ones),
>> > > >> I'll be away for a couple of weeks. Can someone else pick up the RM
>> > > duties
>> > > >> for hadoop-2.5? Maybe Andrew since he's expressed interest in the
>> > past?
>> > > I
>> > > >> will pick up the thread again for hadoop-2.6, but I don't want to
>> > block
>> > > >> hadoop-2.5 due to my non-availability.
>> > > >>
>> > > >> thanks,
>> > > >> Arun
>> > > >>
>> > > >> On Jun 9, 2014, at 9:39 AM, Arun C Murthy 
>> > wrote:
>> > > >>
>> > > >>> Folks,
>> > > >>>
>> > > >>> As you can see from the Roadmap wiki, it looks like several items
>> are
>> > > >> still a bit away from being ready.
>> > > >>>
>> > > >>> I think rather than wait for them, it will be useful to create an
>> > > >> intermediate release (2.5) this month - I think ATS security is
>> pretty
>> > > >> close, so we can ship that. I'm thinking of creating hadoop-2.5 by
>> end
>> > > of
>> > > >> the month, with a branch a couple of weeks prior.
>> > > >>>
>> > > >>> Thoughts?
>> > > >>>
>> > > >>> thanks,
>> > > >>> Arun
>> > > >>>
>> > > >>
>> > > >>
>> > > >>
>> > > >> --
>> > > >> CONFIDENTIALITY NOTICE
>> > > >> NOTICE: This message is intended for the use of the individual or
>> > > entity to
>> > > >> which it is addressed and may contain information that is
>> > confidential,
>> > > >> privileged and exempt from disclosure under applicable law. If the
>> > > reader
>> > > >> of this message is not the intended recipient, you are hereby
>> notified
>> > > that
>> > > >> any printing, copying, dissemination, distribution, disclosure or
>> > > >> forwarding of this communication is strictly prohibited. If you have
>> > > >> received this communication in error, please contact the sender
>> > > immediately
>> > > >> and delete it from your system. Thank You.
>> > > >>
>> > >
>> > > --
>> > > Arun C. Murthy
>> > > Hortonworks Inc.
>> > > http://hortonworks.com/hdp/
>> > >
>> > >
>> > >
>> > > --
>> > > CONFIDENTIALITY NOTICE
>> > > NOTICE: This message is intended for the use of the individual or
>> entity
>> > to
>> > > which it is addressed and may contain information that is confidential,
>> > > privileged and exempt from disclosure under applicable law. If the
>> reader
>> > > of this message is not the intended recipient, you are hereby notified
>> > that
>> > > any printing, copying, dissemination, distribution, disclosure or
>> > > forwarding of this communication is strictly prohibited. If you have
>> > > received this communication in error, please contact the sender
>> > immediately
>> > > and delete it from your system. Thank You.
>> > >
>> >
>>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contac

Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7

[Arun Murthy]

Thanks everyone. I'll start a vote tmrw if there are no objections.

Arun


On Mon, Jun 23, 2014 at 12:06 PM, Jitendra Pandey 
wrote:

> +1, sounds good!
>
>
> On Mon, Jun 23, 2014 at 12:02 PM, Andrew Wang 
> wrote:
>
> > +1 here as well, let's do a vote thread (for 7 days, maybe for the last
> > time!)
> >
> >
> > On Mon, Jun 23, 2014 at 11:46 AM, Vinod Kumar Vavilapalli <
> > vino...@apache.org> wrote:
> >
> > > This seems reasonable, +1.
> > >
> > > In case there is inactivity, we already extend dates even today, we can
> > do
> > > the same going forward.
> > >
> > > I don't see major disagreements, time to start a vote?
> > >
> > > Thanks
> > > +Vinod
> > >
> > > On Jun 20, 2014, at 10:54 PM, Arun C Murthy 
> wrote:
> > >
> > > > Folks,
> > > >
> > > > I'd like to propose we change our by-laws to reduce our voting
> periods
> > > on new releases from 7 days to 5.
> > > >
> > > > Currently, it just takes too long to turn around releases;
> particularly
> > > if we have critical security fixes etc.
> > > >
> > > > Thoughts?
> > > >
> > > > thanks,
> > > > Arun
> > > >
> > > >
> > > > --
> > > > CONFIDENTIALITY NOTICE
> > > > NOTICE: This message is intended for the use of the individual or
> > entity
> > > to
> > > > which it is addressed and may contain information that is
> confidential,
> > > > privileged and exempt from disclosure under applicable law. If the
> > reader
> > > > of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > > any printing, copying, dissemination, distribution, disclosure or
> > > > forwarding of this communication is strictly prohibited. If you have
> > > > received this communication in error, please contact the sender
> > > immediately
> > > > and delete it from your system. Thank You.
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>
>
>
> --
> 
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>



-- 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: hadoop-2.5 - June end?

[Jian He]

Just found the roadmap includes a wrong jira link for "RM Webservices
YARN-1731", it should be YARN-1695
.

Jian


On Mon, Jun 23, 2014 at 2:51 PM, Karthik Kambatla 
wrote:

> While YARN-1492 would greatly benefit performance, I think it might be a
> little too ambitious at this point to include it in 2.5 without delaying
> it.
>
> As I mentioned before and from my offline conversation with Arun, we might
> want to release 2.6 end of July and that might be a better vehicle for
> YARN-1492.
>
>
> On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:
>
> > It would be great if we can consider the shared cache (YARN-1492) as part
> > of 2.5. It obviously will depend on the reviews from folks, but the
> feature
> > itself should be reasonably solid.
> >
> > Thanks,
> > Sangjin
> >
> >
> > On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
> > wrote:
> >
> > > Thanks Karthik!
> > >
> > > I've updated https://wiki.apache.org/hadoop/Roadmap with features
> which
> > > are very close to completion. Let's see if that makes sense and if we
> get
> > > any further feedback.
> > >
> > > Arun
> > >
> > > On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 
> > wrote:
> > >
> > > > I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
> > > might
> > > > need some help from someone more familiar with HDFS.
> > > >
> > > >
> > > > On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy  >
> > > wrote:
> > > >
> > > >> Folks,
> > > >>
> > > >> I'd appreciate some help here. Due to family reasons (all good
> ones),
> > > >> I'll be away for a couple of weeks. Can someone else pick up the RM
> > > duties
> > > >> for hadoop-2.5? Maybe Andrew since he's expressed interest in the
> > past?
> > > I
> > > >> will pick up the thread again for hadoop-2.6, but I don't want to
> > block
> > > >> hadoop-2.5 due to my non-availability.
> > > >>
> > > >> thanks,
> > > >> Arun
> > > >>
> > > >> On Jun 9, 2014, at 9:39 AM, Arun C Murthy 
> > wrote:
> > > >>
> > > >>> Folks,
> > > >>>
> > > >>> As you can see from the Roadmap wiki, it looks like several items
> are
> > > >> still a bit away from being ready.
> > > >>>
> > > >>> I think rather than wait for them, it will be useful to create an
> > > >> intermediate release (2.5) this month - I think ATS security is
> pretty
> > > >> close, so we can ship that. I'm thinking of creating hadoop-2.5 by
> end
> > > of
> > > >> the month, with a branch a couple of weeks prior.
> > > >>>
> > > >>> Thoughts?
> > > >>>
> > > >>> thanks,
> > > >>> Arun
> > > >>>
> > > >>
> > > >>
> > > >>
> > > >> --
> > > >> CONFIDENTIALITY NOTICE
> > > >> NOTICE: This message is intended for the use of the individual or
> > > entity to
> > > >> which it is addressed and may contain information that is
> > confidential,
> > > >> privileged and exempt from disclosure under applicable law. If the
> > > reader
> > > >> of this message is not the intended recipient, you are hereby
> notified
> > > that
> > > >> any printing, copying, dissemination, distribution, disclosure or
> > > >> forwarding of this communication is strictly prohibited. If you have
> > > >> received this communication in error, please contact the sender
> > > immediately
> > > >> and delete it from your system. Thank You.
> > > >>
> > >
> > > --
> > > Arun C. Murthy
> > > Hortonworks Inc.
> > > http://hortonworks.com/hdp/
> > >
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> > >
> >
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: hadoop-2.5 - June end?

[Karthik Kambatla]

While YARN-1492 would greatly benefit performance, I think it might be a
little too ambitious at this point to include it in 2.5 without delaying
it.

As I mentioned before and from my offline conversation with Arun, we might
want to release 2.6 end of July and that might be a better vehicle for
YARN-1492.


On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:

> It would be great if we can consider the shared cache (YARN-1492) as part
> of 2.5. It obviously will depend on the reviews from folks, but the feature
> itself should be reasonably solid.
>
> Thanks,
> Sangjin
>
>
> On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
> wrote:
>
> > Thanks Karthik!
> >
> > I've updated https://wiki.apache.org/hadoop/Roadmap with features which
> > are very close to completion. Let's see if that makes sense and if we get
> > any further feedback.
> >
> > Arun
> >
> > On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 
> wrote:
> >
> > > I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
> > might
> > > need some help from someone more familiar with HDFS.
> > >
> > >
> > > On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy 
> > wrote:
> > >
> > >> Folks,
> > >>
> > >> I'd appreciate some help here. Due to family reasons (all good ones),
> > >> I'll be away for a couple of weeks. Can someone else pick up the RM
> > duties
> > >> for hadoop-2.5? Maybe Andrew since he's expressed interest in the
> past?
> > I
> > >> will pick up the thread again for hadoop-2.6, but I don't want to
> block
> > >> hadoop-2.5 due to my non-availability.
> > >>
> > >> thanks,
> > >> Arun
> > >>
> > >> On Jun 9, 2014, at 9:39 AM, Arun C Murthy 
> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> As you can see from the Roadmap wiki, it looks like several items are
> > >> still a bit away from being ready.
> > >>>
> > >>> I think rather than wait for them, it will be useful to create an
> > >> intermediate release (2.5) this month - I think ATS security is pretty
> > >> close, so we can ship that. I'm thinking of creating hadoop-2.5 by end
> > of
> > >> the month, with a branch a couple of weeks prior.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>> thanks,
> > >>> Arun
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> CONFIDENTIALITY NOTICE
> > >> NOTICE: This message is intended for the use of the individual or
> > entity to
> > >> which it is addressed and may contain information that is
> confidential,
> > >> privileged and exempt from disclosure under applicable law. If the
> > reader
> > >> of this message is not the intended recipient, you are hereby notified
> > that
> > >> any printing, copying, dissemination, distribution, disclosure or
> > >> forwarding of this communication is strictly prohibited. If you have
> > >> received this communication in error, please contact the sender
> > immediately
> > >> and delete it from your system. Thank You.
> > >>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/hdp/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Re: hadoop-2.5 - June end?

[Andrew Wang]

Thanks for volunteering Karthik, I can help out with anything on the HDFS
side if you need it.

Best,
Andrew


On Mon, Jun 23, 2014 at 2:30 PM, Sangjin Lee  wrote:

> It would be great if we can consider the shared cache (YARN-1492) as part
> of 2.5. It obviously will depend on the reviews from folks, but the feature
> itself should be reasonably solid.
>
> Thanks,
> Sangjin
>
>
> On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy 
> wrote:
>
> > Thanks Karthik!
> >
> > I've updated https://wiki.apache.org/hadoop/Roadmap with features which
> > are very close to completion. Let's see if that makes sense and if we get
> > any further feedback.
> >
> > Arun
> >
> > On Jun 23, 2014, at 2:09 PM, Karthik Kambatla 
> wrote:
> >
> > > I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
> > might
> > > need some help from someone more familiar with HDFS.
> > >
> > >
> > > On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy 
> > wrote:
> > >
> > >> Folks,
> > >>
> > >> I'd appreciate some help here. Due to family reasons (all good ones),
> > >> I'll be away for a couple of weeks. Can someone else pick up the RM
> > duties
> > >> for hadoop-2.5? Maybe Andrew since he's expressed interest in the
> past?
> > I
> > >> will pick up the thread again for hadoop-2.6, but I don't want to
> block
> > >> hadoop-2.5 due to my non-availability.
> > >>
> > >> thanks,
> > >> Arun
> > >>
> > >> On Jun 9, 2014, at 9:39 AM, Arun C Murthy 
> wrote:
> > >>
> > >>> Folks,
> > >>>
> > >>> As you can see from the Roadmap wiki, it looks like several items are
> > >> still a bit away from being ready.
> > >>>
> > >>> I think rather than wait for them, it will be useful to create an
> > >> intermediate release (2.5) this month - I think ATS security is pretty
> > >> close, so we can ship that. I'm thinking of creating hadoop-2.5 by end
> > of
> > >> the month, with a branch a couple of weeks prior.
> > >>>
> > >>> Thoughts?
> > >>>
> > >>> thanks,
> > >>> Arun
> > >>>
> > >>
> > >>
> > >>
> > >> --
> > >> CONFIDENTIALITY NOTICE
> > >> NOTICE: This message is intended for the use of the individual or
> > entity to
> > >> which it is addressed and may contain information that is
> confidential,
> > >> privileged and exempt from disclosure under applicable law. If the
> > reader
> > >> of this message is not the intended recipient, you are hereby notified
> > that
> > >> any printing, copying, dissemination, distribution, disclosure or
> > >> forwarding of this communication is strictly prohibited. If you have
> > >> received this communication in error, please contact the sender
> > immediately
> > >> and delete it from your system. Thank You.
> > >>
> >
> > --
> > Arun C. Murthy
> > Hortonworks Inc.
> > http://hortonworks.com/hdp/
> >
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>

Re: hadoop-2.5 - June end?

[Sangjin Lee]

It would be great if we can consider the shared cache (YARN-1492) as part
of 2.5. It obviously will depend on the reviews from folks, but the feature
itself should be reasonably solid.

Thanks,
Sangjin


On Mon, Jun 23, 2014 at 2:24 PM, Arun C Murthy  wrote:

> Thanks Karthik!
>
> I've updated https://wiki.apache.org/hadoop/Roadmap with features which
> are very close to completion. Let's see if that makes sense and if we get
> any further feedback.
>
> Arun
>
> On Jun 23, 2014, at 2:09 PM, Karthik Kambatla  wrote:
>
> > I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I
> might
> > need some help from someone more familiar with HDFS.
> >
> >
> > On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy 
> wrote:
> >
> >> Folks,
> >>
> >> I'd appreciate some help here. Due to family reasons (all good ones),
> >> I'll be away for a couple of weeks. Can someone else pick up the RM
> duties
> >> for hadoop-2.5? Maybe Andrew since he's expressed interest in the past?
> I
> >> will pick up the thread again for hadoop-2.6, but I don't want to block
> >> hadoop-2.5 due to my non-availability.
> >>
> >> thanks,
> >> Arun
> >>
> >> On Jun 9, 2014, at 9:39 AM, Arun C Murthy  wrote:
> >>
> >>> Folks,
> >>>
> >>> As you can see from the Roadmap wiki, it looks like several items are
> >> still a bit away from being ready.
> >>>
> >>> I think rather than wait for them, it will be useful to create an
> >> intermediate release (2.5) this month - I think ATS security is pretty
> >> close, so we can ship that. I'm thinking of creating hadoop-2.5 by end
> of
> >> the month, with a branch a couple of weeks prior.
> >>>
> >>> Thoughts?
> >>>
> >>> thanks,
> >>> Arun
> >>>
> >>
> >>
> >>
> >> --
> >> CONFIDENTIALITY NOTICE
> >> NOTICE: This message is intended for the use of the individual or
> entity to
> >> which it is addressed and may contain information that is confidential,
> >> privileged and exempt from disclosure under applicable law. If the
> reader
> >> of this message is not the intended recipient, you are hereby notified
> that
> >> any printing, copying, dissemination, distribution, disclosure or
> >> forwarding of this communication is strictly prohibited. If you have
> >> received this communication in error, please contact the sender
> immediately
> >> and delete it from your system. Thank You.
> >>
>
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/hdp/
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: hadoop-2.5 - June end?

[Arun C Murthy]

Thanks Karthik!

I've updated https://wiki.apache.org/hadoop/Roadmap with features which are 
very close to completion. Let's see if that makes sense and if we get any 
further feedback.

Arun

On Jun 23, 2014, at 2:09 PM, Karthik Kambatla  wrote:

> I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I might
> need some help from someone more familiar with HDFS.
> 
> 
> On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy  wrote:
> 
>> Folks,
>> 
>> I'd appreciate some help here. Due to family reasons (all good ones),
>> I'll be away for a couple of weeks. Can someone else pick up the RM duties
>> for hadoop-2.5? Maybe Andrew since he's expressed interest in the past? I
>> will pick up the thread again for hadoop-2.6, but I don't want to block
>> hadoop-2.5 due to my non-availability.
>> 
>> thanks,
>> Arun
>> 
>> On Jun 9, 2014, at 9:39 AM, Arun C Murthy  wrote:
>> 
>>> Folks,
>>> 
>>> As you can see from the Roadmap wiki, it looks like several items are
>> still a bit away from being ready.
>>> 
>>> I think rather than wait for them, it will be useful to create an
>> intermediate release (2.5) this month - I think ATS security is pretty
>> close, so we can ship that. I'm thinking of creating hadoop-2.5 by end of
>> the month, with a branch a couple of weeks prior.
>>> 
>>> Thoughts?
>>> 
>>> thanks,
>>> Arun
>>> 
>> 
>> 
>> 
>> --
>> CONFIDENTIALITY NOTICE
>> NOTICE: This message is intended for the use of the individual or entity to
>> which it is addressed and may contain information that is confidential,
>> privileged and exempt from disclosure under applicable law. If the reader
>> of this message is not the intended recipient, you are hereby notified that
>> any printing, copying, dissemination, distribution, disclosure or
>> forwarding of this communication is strictly prohibited. If you have
>> received this communication in error, please contact the sender immediately
>> and delete it from your system. Thank You.
>> 

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/hdp/



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: hadoop-2.5 - June end?

[Karthik Kambatla]

I can pick up the RM duties for 2.5. If I run into any HDFS doubts, I might
need some help from someone more familiar with HDFS.


On Mon, Jun 23, 2014 at 12:07 PM, Arun C Murthy  wrote:

> Folks,
>
>  I'd appreciate some help here. Due to family reasons (all good ones),
> I'll be away for a couple of weeks. Can someone else pick up the RM duties
> for hadoop-2.5? Maybe Andrew since he's expressed interest in the past? I
> will pick up the thread again for hadoop-2.6, but I don't want to block
> hadoop-2.5 due to my non-availability.
>
> thanks,
> Arun
>
> On Jun 9, 2014, at 9:39 AM, Arun C Murthy  wrote:
>
> > Folks,
> >
> > As you can see from the Roadmap wiki, it looks like several items are
> still a bit away from being ready.
> >
> > I think rather than wait for them, it will be useful to create an
> intermediate release (2.5) this month - I think ATS security is pretty
> close, so we can ship that. I'm thinking of creating hadoop-2.5 by end of
> the month, with a branch a couple of weeks prior.
> >
> > Thoughts?
> >
> > thanks,
> > Arun
> >
>
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Plans of moving towards JDK7 in trunk

[Sandy Ryza]

Andrew, correct me if I'm misunderstanding, but the incompatible change
that would require a major version bump is dropping support for JDK6.


On Mon, Jun 23, 2014 at 1:53 PM, sanjay Radia 
wrote:

>
> On Jun 21, 2014, at 8:01 AM, Andrew Wang  wrote:
>
> > This is why I'd like to keep my original proposal on the table: keep
> going
> > with branch-2 in the near term, while working towards a JDK8-based
> Hadoop 3
> > by April next year. It doesn't need to be a big bang release either. I'd
> be
> > delighted if we could rolling upgrade from one to the other. I just
> didn't
> > want to rule out the inclusion of some very compelling feature outright.
> > Trust me though, I'd be the first person to ask about compatibility if
> such
> > a feature does come up.
>
>
> Given your above statement  on compatibility (such as rolling upgrades),
>  it should be fine for the JDK8-based-Hadoop-release to not be 3.0 and
> instead merely be 2.x? Or do you have any incompatible changes to Hadoop
> protocol or APIs in mind during the same time period?
>
> sanjay
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: Plans of moving towards JDK7 in trunk

[sanjay Radia]

On Jun 21, 2014, at 8:01 AM, Andrew Wang  wrote:

> This is why I'd like to keep my original proposal on the table: keep going
> with branch-2 in the near term, while working towards a JDK8-based Hadoop 3
> by April next year. It doesn't need to be a big bang release either. I'd be
> delighted if we could rolling upgrade from one to the other. I just didn't
> want to rule out the inclusion of some very compelling feature outright.
> Trust me though, I'd be the first person to ask about compatibility if such
> a feature does come up.


Given your above statement  on compatibility (such as rolling upgrades),  it 
should be fine for the JDK8-based-Hadoop-release to not be 3.0 and instead 
merely be 2.x? Or do you have any incompatible changes to Hadoop protocol or 
APIs in mind during the same time period?

sanjay
-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

[jira] [Created] (MAPREDUCE-5939) StartTime showing up as the epoch time in JHS UI after upgrade

[Kihwal Lee (JIRA)]

Kihwal Lee created MAPREDUCE-5939:
-

 Summary: StartTime showing up as the epoch time in JHS UI after 
upgrade
 Key: MAPREDUCE-5939
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5939
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 2.5.0
Reporter: Kihwal Lee


After upgrading from 0.23.x to 2.5, the start time of old apps are showing up 
as the epoch time.  It looks like 2.5 expects start time to be encoded at the 
end of the jhist file name (-[timestamp].jhist). It should have been 
made backward compatible.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Re: hadoop-2.5 - June end?

[Arun C Murthy]

Folks,

 I'd appreciate some help here. Due to family reasons (all good ones), I'll be 
away for a couple of weeks. Can someone else pick up the RM duties for 
hadoop-2.5? Maybe Andrew since he's expressed interest in the past? I will pick 
up the thread again for hadoop-2.6, but I don't want to block hadoop-2.5 due to 
my non-availability.

thanks,
Arun

On Jun 9, 2014, at 9:39 AM, Arun C Murthy  wrote:

> Folks,
> 
> As you can see from the Roadmap wiki, it looks like several items are still a 
> bit away from being ready.
> 
> I think rather than wait for them, it will be useful to create an 
> intermediate release (2.5) this month - I think ATS security is pretty close, 
> so we can ship that. I'm thinking of creating hadoop-2.5 by end of the month, 
> with a branch a couple of weeks prior.
> 
> Thoughts?
> 
> thanks,
> Arun
> 



-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7

[Jitendra Pandey]

+1, sounds good!


On Mon, Jun 23, 2014 at 12:02 PM, Andrew Wang 
wrote:

> +1 here as well, let's do a vote thread (for 7 days, maybe for the last
> time!)
>
>
> On Mon, Jun 23, 2014 at 11:46 AM, Vinod Kumar Vavilapalli <
> vino...@apache.org> wrote:
>
> > This seems reasonable, +1.
> >
> > In case there is inactivity, we already extend dates even today, we can
> do
> > the same going forward.
> >
> > I don't see major disagreements, time to start a vote?
> >
> > Thanks
> > +Vinod
> >
> > On Jun 20, 2014, at 10:54 PM, Arun C Murthy  wrote:
> >
> > > Folks,
> > >
> > > I'd like to propose we change our by-laws to reduce our voting periods
> > on new releases from 7 days to 5.
> > >
> > > Currently, it just takes too long to turn around releases; particularly
> > if we have critical security fixes etc.
> > >
> > > Thoughts?
> > >
> > > thanks,
> > > Arun
> > >
> > >
> > > --
> > > CONFIDENTIALITY NOTICE
> > > NOTICE: This message is intended for the use of the individual or
> entity
> > to
> > > which it is addressed and may contain information that is confidential,
> > > privileged and exempt from disclosure under applicable law. If the
> reader
> > > of this message is not the intended recipient, you are hereby notified
> > that
> > > any printing, copying, dissemination, distribution, disclosure or
> > > forwarding of this communication is strictly prohibited. If you have
> > > received this communication in error, please contact the sender
> > immediately
> > > and delete it from your system. Thank You.
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
> >
>



-- 


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7

[Andrew Wang]

+1 here as well, let's do a vote thread (for 7 days, maybe for the last
time!)


On Mon, Jun 23, 2014 at 11:46 AM, Vinod Kumar Vavilapalli <
vino...@apache.org> wrote:

> This seems reasonable, +1.
>
> In case there is inactivity, we already extend dates even today, we can do
> the same going forward.
>
> I don't see major disagreements, time to start a vote?
>
> Thanks
> +Vinod
>
> On Jun 20, 2014, at 10:54 PM, Arun C Murthy  wrote:
>
> > Folks,
> >
> > I'd like to propose we change our by-laws to reduce our voting periods
> on new releases from 7 days to 5.
> >
> > Currently, it just takes too long to turn around releases; particularly
> if we have critical security fixes etc.
> >
> > Thoughts?
> >
> > thanks,
> > Arun
> >
> >
> > --
> > CONFIDENTIALITY NOTICE
> > NOTICE: This message is intended for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential,
> > privileged and exempt from disclosure under applicable law. If the reader
> > of this message is not the intended recipient, you are hereby notified
> that
> > any printing, copying, dissemination, distribution, disclosure or
> > forwarding of this communication is strictly prohibited. If you have
> > received this communication in error, please contact the sender
> immediately
> > and delete it from your system. Thank You.
>
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7

[Vinod Kumar Vavilapalli]

This seems reasonable, +1.

In case there is inactivity, we already extend dates even today, we can do the 
same going forward.

I don't see major disagreements, time to start a vote?

Thanks
+Vinod

On Jun 20, 2014, at 10:54 PM, Arun C Murthy  wrote:

> Folks,
> 
> I'd like to propose we change our by-laws to reduce our voting periods on new 
> releases from 7 days to 5.
> 
> Currently, it just takes too long to turn around releases; particularly if we 
> have critical security fixes etc.
> 
> Thoughts?
> 
> thanks,
> Arun
> 
> 
> -- 
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to 
> which it is addressed and may contain information that is confidential, 
> privileged and exempt from disclosure under applicable law. If the reader 
> of this message is not the intended recipient, you are hereby notified that 
> any printing, copying, dissemination, distribution, disclosure or 
> forwarding of this communication is strictly prohibited. If you have 
> received this communication in error, please contact the sender immediately 
> and delete it from your system. Thank You.


-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.


signature.asc
Description: Message signed with OpenPGP using GPGMail

Re: [VOTE] Release Apache Hadoop 2.4.1

[Steve Loughran]

someone's filed a JIRA on loops in Hedged Read, with tests ...

https://issues.apache.org/jira/browse/HDFS-6591


On 23 June 2014 08:58, Mit Desai  wrote:

> +1 (non-binding)
>
> Tested on: Fedora17
> -Successful build from src (including native)
> -Verified Signature
> -Deployed source to my single node cluster and ran couple of sample MR jobs
>
>
> - M!T
>
>
>
>
>
> On 6/21/14, 1:51 AM, "Arun C Murthy"  wrote:
>
> >Folks,
> >
> >I've created another release candidate (rc1) for hadoop-2.4.1 based on
> >the feedback that I would like to push out.
> >
> >The RC is available at:
> >http://people.apache.org/~acmurthy/hadoop-2.4.1-rc1
> >The RC tag in svn is here:
> >https://svn.apache.org/repos/asf/hadoop/common/tags/release-2.4.1-rc1
> >
> >The maven artifacts are available via repository.apache.org.
> >
> >Please try the release and vote; the vote will run for the usual 7 days.
> >
> >thanks,
> >Arun
> >
> >
> >
> >--
> >Arun C. Murthy
> >Hortonworks Inc.
> >http://hortonworks.com/hdp/
> >
> >
> >
> >--
> >CONFIDENTIALITY NOTICE
> >NOTICE: This message is intended for the use of the individual or entity
> >to
> >which it is addressed and may contain information that is confidential,
> >privileged and exempt from disclosure under applicable law. If the reader
> >of this message is not the intended recipient, you are hereby notified
> >that
> >any printing, copying, dissemination, distribution, disclosure or
> >forwarding of this communication is strictly prohibited. If you have
> >received this communication in error, please contact the sender
> >immediately
> >and delete it from your system. Thank You.
>
>

-- 
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to 
which it is addressed and may contain information that is confidential, 
privileged and exempt from disclosure under applicable law. If the reader 
of this message is not the intended recipient, you are hereby notified that 
any printing, copying, dissemination, distribution, disclosure or 
forwarding of this communication is strictly prohibited. If you have 
received this communication in error, please contact the sender immediately 
and delete it from your system. Thank You.

Hadoop-Mapreduce-trunk - Build # 1810 - Still Failing

[Apache Jenkins Server]

See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1810/

###
## LAST 60 LINES OF THE CONSOLE 
###
[...truncated 36203 lines...]
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.523 sec - in 
org.apache.hadoop.mapreduce.lib.partition.TestBinaryPartitioner
Running org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedPartitioner
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.456 sec - in 
org.apache.hadoop.mapreduce.lib.partition.TestMRKeyFieldBasedPartitioner
Running org.apache.hadoop.mapreduce.TestChild
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 82.836 sec - in 
org.apache.hadoop.mapreduce.TestChild
Running org.apache.hadoop.mapreduce.filecache.TestURIFragments
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.065 sec - in 
org.apache.hadoop.mapreduce.filecache.TestURIFragments
Running org.apache.hadoop.mapreduce.TestMapReduce
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.055 sec - in 
org.apache.hadoop.mapreduce.TestMapReduce
Running org.apache.hadoop.mapreduce.TestNewCombinerGrouping
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.598 sec - in 
org.apache.hadoop.mapreduce.TestNewCombinerGrouping

Results :

Tests run: 494, Failures: 0, Errors: 0, Skipped: 11

[INFO] 
[INFO] Reactor Summary:
[INFO] 
[INFO] hadoop-mapreduce-client ... SUCCESS [2.483s]
[INFO] hadoop-mapreduce-client-core .. SUCCESS [54.953s]
[INFO] hadoop-mapreduce-client-common  SUCCESS [26.838s]
[INFO] hadoop-mapreduce-client-shuffle ... SUCCESS [4.314s]
[INFO] hadoop-mapreduce-client-app ... SUCCESS [7:07.962s]
[INFO] hadoop-mapreduce-client-hs  SUCCESS [4:43.125s]
[INFO] hadoop-mapreduce-client-jobclient . FAILURE 
[1:38:14.465s]
[INFO] hadoop-mapreduce-client-hs-plugins  SKIPPED
[INFO] Apache Hadoop MapReduce Examples .. SKIPPED
[INFO] hadoop-mapreduce .. SKIPPED
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
[INFO] Total time: 1:51:35.190s
[INFO] Finished at: Mon Jun 23 15:18:22 UTC 2014
[INFO] Final Memory: 28M/99M
[INFO] 
[ERROR] Failed to execute goal 
org.apache.maven.plugins:maven-surefire-plugin:2.16:test (default-test) on 
project hadoop-mapreduce-client-jobclient: There was a timeout or other error 
in the fork -> [Help 1]
[ERROR] 
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e 
switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR] 
[ERROR] For more information about the errors and possible solutions, please 
read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR] 
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR]   mvn  -rf :hadoop-mapreduce-client-jobclient
Build step 'Execute shell' marked build as failure
[FINDBUGS] Skipping publisher since build result is FAILURE
Archiving artifacts
Sending artifact delta relative to Hadoop-Mapreduce-trunk #1765
Archived 2 artifacts
Archive block size is 32768
Received 0 blocks and 16954851 bytes
Compression is 0.0%
Took 9.8 sec
Updating HDFS-6580
Updating HDFS-6507
Email was triggered for: Failure
Sending email for trigger: Failure



###
## FAILED TESTS (if any) 
##
No tests ran.

Re: [DISCUSS] Change by-laws on release votes: 5 days instead of 7

[Thomas Graves]

I¹d be fine with 5 days.  Any shorter and you could have issues with folks
being just out for a weekend and miss the entire vote unless we explicitly
tried to stay away from ending the vote on the weekend.


Tom



On 6/21/14, 12:54 AM, "Arun C Murthy"  wrote:

>Folks,
>
> I'd like to propose we change our by-laws to reduce our voting periods
>on new releases from 7 days to 5.
>
> Currently, it just takes too long to turn around releases; particularly
>if we have critical security fixes etc.
>
> Thoughts?
>
>thanks,
>Arun
>
>
>-- 
>CONFIDENTIALITY NOTICE
>NOTICE: This message is intended for the use of the individual or entity
>to 
>which it is addressed and may contain information that is confidential,
>privileged and exempt from disclosure under applicable law. If the reader
>of this message is not the intended recipient, you are hereby notified
>that 
>any printing, copying, dissemination, distribution, disclosure or
>forwarding of this communication is strictly prohibited. If you have
>received this communication in error, please contact the sender
>immediately 
>and delete it from your system. Thank You.

[jira] [Created] (MAPREDUCE-5938) Shuffle port in nodemanager is binding to all IPs

[Ashutosh Jindal (JIRA)]

Ashutosh Jindal created MAPREDUCE-5938:
--

 Summary: Shuffle port in nodemanager is binding to all IPs 
 Key: MAPREDUCE-5938
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5938
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Reporter: Ashutosh Jindal
Assignee: Ashutosh Jindal


nodemanager port mapreduce.shuffle.port is listening to all ip



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Where is the map input transfered to the 'map worker'/container?

[Christian Grote]

Hey,

I'm looking for the place where the actual map input is transfered to 
the 'map worker'/container (in case it's assigned to a host that doesn't 
have the data already).


Something similar to the copyFromHost(..) method in Fetcher.java 
(org.apache.hadoop.mapreduce.task.reduce), where the map output is 
transfered.



Best Regards,
Christian Grote