[jira] [Created] (MAPREDUCE-4990) Construct debug strings conditionally in ShuffleHandler.Shuffle#sendMapOutput()

2013-02-07 Thread Karthik Kambatla (JIRA) 
Karthik Kambatla created MAPREDUCE-4990:
---

 Summary: Construct debug strings conditionally in 
ShuffleHandler.Shuffle#sendMapOutput()
 Key: MAPREDUCE-4990
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4990
 Project: Hadoop Map/Reduce
  Issue Type: Bug
Affects Versions: 0.23.5, 2.0.3-alpha
Reporter: Karthik Kambatla
Assignee: Karthik Kambatla
Priority: Trivial


In {{#sendMapOutput()}}, the debug statements are not wrapped in the customary 
{{#isDebugEnabled()}} condition. Given this piece of code is critical for 
performance, it would be nice to fix it.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: Heads up - merge branch-trunk-win to trunk

2013-02-07 Thread Eli Collins 
Thanks for the update Suresh.  Has any testing been done on the branch on
Linux aside from running the unit tests?

Thanks,
Eli


On Thu, Feb 7, 2013 at 5:42 PM, Suresh Srinivas wrote:

> The support for Hadoop on Windows was proposed in
> HADOOP-8079 almost
> a year ago. The goal was to make Hadoop natively integrated, full-featured,
> and performance and scalability tuned on Windows Server or Windows Azure.
> We are happy to announce that a lot of progress has been made in this
> regard.
>
> Initial work started in a feature branch, branch-1-win, based on branch-1.
> The details related to the work done in the branch can be seen in
> CHANGES.txt<
> http://svn.apache.org/viewvc/hadoop/common/branches/branch-1-win/CHANGES.branch-1-win.txt?view=markup
> >.
> This work has been ported to a branch, branch-trunk-win, based on trunk.
> Merge patch for this is available on
> HADOOP-8562
> .
>
> Highlights of the work done so far:
> 1. Necessary changes in Hadoop to run natively on Windows. These changes
> handle differences in platforms related to path names, process/task
> management etc.
> 2. Addition of winutils tools for managing file permissions and ownership,
> user group mapping, hardlinks, symbolic links, chmod, disk utilization, and
> process/task management.
> 3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh,
> start and stop scripts.
> 4. Addition of block placement policy implemnation to support cloud
> enviroment, more specifically Azure.
>
> We are very close to wrapping up the work in branch-trunk-win and getting
> ready for a merge. Currently the merge patch is passing close to 100% of
> unit tests on Linux. Soon I will call for a vote to merge this branch into
> trunk.
>
> Next steps:
> 1. Call for vote to merge branch-trunk-win to trunk, when the work
> completes and precommit build is clean.
> 2. Start a discussion on adding Jenkins precommit builds on windows and how
> to integrate that with the existing commit process.
>
> Let me know if you have any questions.
>
> Regards,
> Suresh
>

Heads up - merge branch-trunk-win to trunk

2013-02-07 Thread Suresh Srinivas 
The support for Hadoop on Windows was proposed in
HADOOP-8079 almost
a year ago. The goal was to make Hadoop natively integrated, full-featured,
and performance and scalability tuned on Windows Server or Windows Azure.
We are happy to announce that a lot of progress has been made in this
regard.

Initial work started in a feature branch, branch-1-win, based on branch-1.
The details related to the work done in the branch can be seen in
CHANGES.txt.
This work has been ported to a branch, branch-trunk-win, based on trunk.
Merge patch for this is available on
HADOOP-8562
.

Highlights of the work done so far:
1. Necessary changes in Hadoop to run natively on Windows. These changes
handle differences in platforms related to path names, process/task
management etc.
2. Addition of winutils tools for managing file permissions and ownership,
user group mapping, hardlinks, symbolic links, chmod, disk utilization, and
process/task management.
3. Added cmd scripts equivalent to existing shell scripts hadoop-daemon.sh,
start and stop scripts.
4. Addition of block placement policy implemnation to support cloud
enviroment, more specifically Azure.

We are very close to wrapping up the work in branch-trunk-win and getting
ready for a merge. Currently the merge patch is passing close to 100% of
unit tests on Linux. Soon I will call for a vote to merge this branch into
trunk.

Next steps:
1. Call for vote to merge branch-trunk-win to trunk, when the work
completes and precommit build is clean.
2. Start a discussion on adding Jenkins precommit builds on windows and how
to integrate that with the existing commit process.

Let me know if you have any questions.

Regards,
Suresh

[jira] [Created] (MAPREDUCE-4989) JSONify DataTables input data for Attempts page

2013-02-07 Thread Ravi Prakash (JIRA) 
Ravi Prakash created MAPREDUCE-4989:
---

 Summary: JSONify DataTables input data for Attempts page
 Key: MAPREDUCE-4989
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4989
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: jobhistoryserver, mr-am
Affects Versions: 0.23.6
Reporter: Ravi Prakash
Assignee: Ravi Prakash


Use deferred rendering for the attempts page as was done in MAPREDUCE-4720. I'm 
sorry I didn't realize earlier that this table could be huge too. Thanks to 
[~jlowe] for pointing it out.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Hitesh Shah 
+1 (non-binding) 

Downloaded ( verified checksums ) and built from source, deployed and 
successfully ran both MR and distributed shell examples. 

-- Hitesh

On Feb 6, 2013, at 7:59 PM, Arun C Murthy wrote:

> Folks,
> 
> I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I would 
> like to release.
> 
> This release contains several major enhancements such as QJM for HDFS HA, 
> multi-resource scheduling for YARN, YARN ResourceManager restart etc.  
> Also YARN has achieved significant stability at scale (more details from Y! 
> folks here: http://s.apache.org/VYO).
> 
> The RC is available at: 
> http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> The RC tag in svn is here: 
> http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> 
> The maven artifacts are available via repository.apache.org.
> 
> Please try the release and vote; the vote will run for the usual 7 days.
> 
> thanks,
> Arun
> 
> 
> 
> --
> Arun C. Murthy
> Hortonworks Inc.
> http://hortonworks.com/
> 
>

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Chris Nauroth 
+1 non-binding

I downloaded hadoop-2.0.3-alpha.tar.gz and verified the checksum and
signature.  I deployed to a set of Ubuntu VMs: NN, RM, 2*DN/NM, and 2NN.  I
tested a few HDFS operations and MapReduce jobs.  I verified that the 2NN
could complete a checkpoint.

Thank you,
--Chris


On Thu, Feb 7, 2013 at 1:37 PM, Thomas Graves  wrote:

> +1 (binding). I downloaded it, verified checksums, built from source,
> installed (both binary and one I built) and ran some basic jobs.
>
> Tom
>
>
>
> On 2/6/13 9:59 PM, "Arun C Murthy"  wrote:
>
> >Folks,
> >
> >I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I
> >would like to release.
> >
> >This release contains several major enhancements such as QJM for HDFS HA,
> >multi-resource scheduling for YARN, YARN ResourceManager restart etc.
> >Also YARN has achieved significant stability at scale (more details from
> >Y! folks here: http://s.apache.org/VYO).
> >
> >The RC is available at:
> >http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
> >The RC tag in svn is here:
> >http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
> >
> >The maven artifacts are available via repository.apache.org.
> >
> >Please try the release and vote; the vote will run for the usual 7 days.
> >
> >thanks,
> >Arun
> >
> >
> >
> >--
> >Arun C. Murthy
> >Hortonworks Inc.
> >http://hortonworks.com/
> >
> >
>
>

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Thomas Graves 
+1 (binding). I downloaded it, verified checksums, built from source,
installed (both binary and one I built) and ran some basic jobs.

Tom



On 2/6/13 9:59 PM, "Arun C Murthy"  wrote:

>Folks,
>
>I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I
>would like to release.
>
>This release contains several major enhancements such as QJM for HDFS HA,
>multi-resource scheduling for YARN, YARN ResourceManager restart etc.
>Also YARN has achieved significant stability at scale (more details from
>Y! folks here: http://s.apache.org/VYO).
>
>The RC is available at:
>http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
>The RC tag in svn is here:
>http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
>
>The maven artifacts are available via repository.apache.org.
>
>Please try the release and vote; the vote will run for the usual 7 days.
>
>thanks,
>Arun
>
>
>
>--
>Arun C. Murthy
>Hortonworks Inc.
>http://hortonworks.com/
>
>

[jira] [Created] (MAPREDUCE-4988) ClusterWithCapacityScheduler and related testcases needs to be ported to JUnit4.

2013-02-07 Thread Amir Sanjar (JIRA) 
Amir Sanjar created MAPREDUCE-4988:
--

 Summary: ClusterWithCapacityScheduler and related testcases needs 
to be ported to JUnit4.
 Key: MAPREDUCE-4988
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4988
 Project: Hadoop Map/Reduce
  Issue Type: Bug
  Components: test
Affects Versions: 1.1.1, 1.1.2
 Environment: Linux on PPC and x86
Reporter: Amir Sanjar
 Fix For: 1.1.2


TestJobTrackerRestartWithCS, TestCapacitySchedulerServlet, and 
TestCapacitySchedulerWithJobTracker testcases potentially could fail when they 
are build with ant 1.8.4.
Solution:
port above testcases and ClusterWithCapacityScheduler class from Junit3 to 
Junit4.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Re: [VOTE] Release hadoop-2.0.3-alpha

2013-02-07 Thread Robert Evans 
I downloaded the binary package and ran a few example jobs on a 3 node
cluster.  Everything seems to be working OK on it, I did see

WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable

For every shell command, but just like with 0.23.6 I don't think it is a
blocker.

+1 (Binding)

--Bobby

On 2/6/13 9:59 PM, "Arun C Murthy"  wrote:

>Folks,
>
>I've created a release candidate (rc0) for hadoop-2.0.3-alpha that I
>would like to release.
>
>This release contains several major enhancements such as QJM for HDFS HA,
>multi-resource scheduling for YARN, YARN ResourceManager restart etc.
>Also YARN has achieved significant stability at scale (more details from
>Y! folks here: http://s.apache.org/VYO).
>
>The RC is available at:
>http://people.apache.org/~acmurthy/hadoop-2.0.3-alpha-rc0/
>The RC tag in svn is here:
>http://svn.apache.org/viewvc/hadoop/common/tags/release-2.0.3-alpha-rc0/
>
>The maven artifacts are available via repository.apache.org.
>
>Please try the release and vote; the vote will run for the usual 7 days.
>
>thanks,
>Arun
>
>
>
>--
>Arun C. Murthy
>Hortonworks Inc.
>http://hortonworks.com/
>
>