Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Wangda Tan]

Thanks Junping,

+1 (binding).

Deployed a single node cluster and ran some simple jobs.

- Wangda


On Mon, Oct 23, 2017 at 4:05 PM, Chris Douglas  wrote:

> +1 (binding)
>
> Looked through the src distribution. Checksum, signatures match, ran
> some of the unit tests. Also checked the site docs; thanks for
> updating the Docker container security docs.
>
> Thanks, Junping. -C
>
> On Thu, Oct 19, 2017 at 5:42 PM, Junping Du  wrote:
> > Hi folks,
> >  I've created our new release candidate (RC1) for Apache Hadoop
> 2.8.2.
> >
> >  Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line
> and will be the latest stable/production release for Apache Hadoop - it
> includes 315 new fixed issues since 2.8.1 and 69 fixes are marked as
> blocker/critical issues.
> >
> >   More information about the 2.8.2 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
> >
> >   New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.2-RC1 du/hadoop-2.8.2-RC0>
> >
> >   The RC tag in git is: release-2.8.2-RC1, and the latest commit id
> is: 66c47f2a01ad9637879e95f80c41f798373828fb
> >
> >   The maven artifacts are available via repository.apache.org<
> http://repository.apache.org/> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1064 repository.apache.org/content/repositories/orgapachehadoop-1062>
> >
> >   Please try the release and vote; the vote will run for the usual 5
> days, ending on 10/24/2017 6pm PST time.
> >
> > Thanks,
> >
> > Junping
> >
>
> -
> To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Chris Douglas]

+1 (binding)

Looked through the src distribution. Checksum, signatures match, ran
some of the unit tests. Also checked the site docs; thanks for
updating the Docker container security docs.

Thanks, Junping. -C

On Thu, Oct 19, 2017 at 5:42 PM, Junping Du  wrote:
> Hi folks,
>  I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.
>
>  Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line and 
> will be the latest stable/production release for Apache Hadoop - it includes 
> 315 new fixed issues since 2.8.1 and 69 fixes are marked as blocker/critical 
> issues.
>
>   More information about the 2.8.2 release plan can be found here: 
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>   New RC is available at: 
> http://home.apache.org/~junping_du/hadoop-2.8.2-RC1
>
>   The RC tag in git is: release-2.8.2-RC1, and the latest commit id is: 
> 66c47f2a01ad9637879e95f80c41f798373828fb
>
>   The maven artifacts are available via 
> repository.apache.org at: 
> https://repository.apache.org/content/repositories/orgapachehadoop-1064
>
>   Please try the release and vote; the vote will run for the usual 5 
> days, ending on 10/24/2017 6pm PST time.
>
> Thanks,
>
> Junping
>

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6989) Uploader tool for Distributed Cache Deploy

[Miklos Szegedi (JIRA)]

Miklos Szegedi created MAPREDUCE-6989:
-

 Summary: Uploader tool for Distributed Cache Deploy
 Key: MAPREDUCE-6989
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6989
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
Reporter: Miklos Szegedi
Assignee: Miklos Szegedi


The proposal is to create a tool that collects all available jars in the Hadoop 
classpath and adds them to a single tarball file. It then uploads the resulting 
archive to an HDFS directory. This saves the cluster administrator from having 
to set this up manually for Distributed Cache Deploy.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Junping Du]

Thanks Jason and all for trying the release RC and vote!

A kindly reminder: we will close the vote on tomorrow night (PST time), so 
please do the verification and vote before that time if you plan to. Thanks!?


Thanks,


Junping



From: Jason Lowe 
Sent: Monday, October 23, 2017 2:49 PM
To: Junping Du
Cc: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

My apologies, false alarm on the CHANGES.md and RELEASENOTES.md.  I was in the 
process of reviewing the release and was interrupted, and when I resumed I 
thought I had already downloaded the CHANGES and RELEASENOTES, but in fact they 
were the old versions from a prior review of 2.8.0.  I reviewed both of them 
for 2.8.2 (for real this time!) and they look correct.  Again my apologies for 
the confusion.

Jason

On Mon, Oct 23, 2017 at 3:26 PM, Jason Lowe 
> wrote:
+1 (binding)

- Verified signatures and digests
- Performed a native build from source
- Deployed to a single-node cluster
- Ran some sample jobs

The CHANGES.md and RELEASENOTES.md both refer to release 2.8.0 instead of 
2.8.2, and I do not see the list of JIRAs in CHANGES.md that have been 
committed since 2.8.1.  Since we're voting on the source bits rather than the 
change log I kept my vote as a +1 as I do see the 2.8.2 changes in the source 
code.

Jason


On Thu, Oct 19, 2017 at 7:42 PM, Junping Du 
> wrote:
Hi folks,
 I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.

 Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line and 
will be the latest stable/production release for Apache Hadoop - it includes 
315 new fixed issues since 2.8.1 and 69 fixes are marked as blocker/critical 
issues.

  More information about the 2.8.2 release plan can be found here: 
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

  New RC is available at: 
http://home.apache.org/~junping_du/hadoop-2.8.2-RC1

  The RC tag in git is: release-2.8.2-RC1, and the latest commit id is: 
66c47f2a01ad9637879e95f80c41f798373828fb

  The maven artifacts are available via 
repository.apache.org
 at: 
https://repository.apache.org/content/repositories/orgapachehadoop-1064

  Please try the release and vote; the vote will run for the usual 5 days, 
ending on 10/24/2017 6pm PST time.

Thanks,

Junping

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Jason Lowe]

My apologies, false alarm on the CHANGES.md and RELEASENOTES.md.  I was in
the process of reviewing the release and was interrupted, and when I
resumed I thought I had already downloaded the CHANGES and RELEASENOTES,
but in fact they were the old versions from a prior review of 2.8.0.  I
reviewed both of them for 2.8.2 (for real this time!) and they look
correct.  Again my apologies for the confusion.

Jason

On Mon, Oct 23, 2017 at 3:26 PM, Jason Lowe  wrote:

> +1 (binding)
>
> - Verified signatures and digests
> - Performed a native build from source
> - Deployed to a single-node cluster
> - Ran some sample jobs
>
> The CHANGES.md and RELEASENOTES.md both refer to release 2.8.0 instead of
> 2.8.2, and I do not see the list of JIRAs in CHANGES.md that have been
> committed since 2.8.1.  Since we're voting on the source bits rather than
> the change log I kept my vote as a +1 as I do see the 2.8.2 changes in the
> source code.
>
> Jason
>
>
> On Thu, Oct 19, 2017 at 7:42 PM, Junping Du  wrote:
>
>> Hi folks,
>>  I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.
>>
>>  Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line
>> and will be the latest stable/production release for Apache Hadoop - it
>> includes 315 new fixed issues since 2.8.1 and 69 fixes are marked as
>> blocker/critical issues.
>>
>>   More information about the 2.8.2 release plan can be found here:
>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>>
>>   New RC is available at: http://home.apache.org/~junpin
>> g_du/hadoop-2.8.2-RC1> >
>>
>>   The RC tag in git is: release-2.8.2-RC1, and the latest commit id
>> is: 66c47f2a01ad9637879e95f80c41f798373828fb
>>
>>   The maven artifacts are available via repository.apache.org<
>> http://repository.apache.org/> at: https://repository.apache.org/
>> content/repositories/orgapachehadoop-1064> .apache.org/content/repositories/orgapachehadoop-1062>
>>
>>   Please try the release and vote; the vote will run for the usual 5
>> days, ending on 10/24/2017 6pm PST time.
>>
>> Thanks,
>>
>> Junping
>>
>>
>

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Jason Lowe]

+1 (binding)

- Verified signatures and digests
- Performed a native build from source
- Deployed to a single-node cluster
- Ran some sample jobs

The CHANGES.md and RELEASENOTES.md both refer to release 2.8.0 instead of
2.8.2, and I do not see the list of JIRAs in CHANGES.md that have been
committed since 2.8.1.  Since we're voting on the source bits rather than
the change log I kept my vote as a +1 as I do see the 2.8.2 changes in the
source code.

Jason


On Thu, Oct 19, 2017 at 7:42 PM, Junping Du  wrote:

> Hi folks,
>  I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.
>
>  Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line
> and will be the latest stable/production release for Apache Hadoop - it
> includes 315 new fixed issues since 2.8.1 and 69 fixes are marked as
> blocker/critical issues.
>
>   More information about the 2.8.2 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>   New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.2-RC1 du/hadoop-2.8.2-RC0>
>
>   The RC tag in git is: release-2.8.2-RC1, and the latest commit id
> is: 66c47f2a01ad9637879e95f80c41f798373828fb
>
>   The maven artifacts are available via repository.apache.org repository.apache.org/> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1064 repository.apache.org/content/repositories/orgapachehadoop-1062>
>
>   Please try the release and vote; the vote will run for the usual 5
> days, ending on 10/24/2017 6pm PST time.
>
> Thanks,
>
> Junping
>
>

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Allen Wittenauer]

With no other information or access to go on, my current hunch is that one of 
the HDFS unit tests is ballooning in memory size.  The easiest way to kill a 
Linux machine is to eat all of the RAM, thanks to overcommit and that’s what 
this “feels” like.

Someone should verify if 2.8.2 has the same issues before a release goes out …


> On Oct 23, 2017, at 12:38 PM, Subramaniam V K  wrote:
> 
> Hi Allen,
> 
> I had set up the build (or intended to) in anticipation 2.9 release. Thanks 
> for fixing the configuration!
> 
> We did face HDFS tests timeouts in branch-2 when run together but 
> individually the tests pass:
> https://issues.apache.org/jira/browse/HDFS-12620
> 
> Folks in HDFS, can you please take a look at HDFS tests in branch-2 as we are 
> not able to get even a single Yetus run to complete due to multiple test 
> failures/timeout.
> 
> Thanks,
> Subru
> 
> On Mon, Oct 23, 2017 at 11:26 AM, Vrushali C  wrote:
> Hi Allen,
> 
> I have filed https://issues.apache.org/jira/browse/YARN-7380 for the
> timeline service findbugs warnings.
> 
> thanks
> Vrushali
> 
> 
> On Mon, Oct 23, 2017 at 11:14 AM, Allen Wittenauer  > wrote:
> 
> >
> > I’m really confused why this causes the Yahoo! QA boxes to go catatonic
> > (!?!) during the run.  As in, never come back online, probably in a kernel
> > panic. It’s pretty consistently in hadoop-hdfs, so something is going wrong
> > there… is branch-2 hdfs behaving badly?  Someone needs to run the
> > hadoop-hdfs unit tests to see what is going on.
> >
> > It’s probably worth noting that findbugs says there is a problem in the
> > timeline server hbase code.Someone should probably verify + fix that
> > issue.
> >
> >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
> 


-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[John Zhuge]

 +1 (binding)

   - Verified checksums and signatures of all tarballs
   - Built source with native, Java 1.8.0_131-b11 on Mac OS X 10.12.6
   - Verified cloud connectors:
  - All S3A integration tests
  - All ADL live unit tests
   - Deployed both binary and built source to a pseudo cluster, passed the
   following sanity tests in insecure, SSL, and SSL+Kerberos mode:
  - HDFS basic and ACL
  - DistCp basic
  - MapReduce wordcount (skipped in SSL+Kerberos mode)
  - KMS and HttpFS basic
  - Balancer start/stop


On Mon, Oct 23, 2017 at 9:03 AM, Ajay Kumar 
wrote:

> Thanks, Junping Du!
>
> +1 (non-binding)
>
> - Built from source
> - Ran hdfs commands
> - Ran pi and sample MR test.
> - Verified the UI's
>
> Thanks,
> Ajay Kumar
>
> On 10/23/17, 8:14 AM, "Shane Kumpf"  wrote:
>
> Thanks, Junping!
>
> +1 (non-binding)
>
> - Verified checksums and signatures
> - Deployed a single node cluster on CentOS 7.2 using the binary tgz,
> source
> tgz, and git tag
> - Ran hdfs commands
> - Ran pi and distributed shell using the default and docker runtimes
> - Verified Docker docs
> - Verified docker runtime can be disabled
> - Verified the UI's
>
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>


-- 
John

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Subramaniam V K]

Hi Allen,

I had set up the build (or intended to) in anticipation 2.9 release. Thanks
for fixing the configuration!

We did face HDFS tests timeouts in branch-2 when run together but
individually the tests pass:
https://issues.apache.org/jira/browse/HDFS-12620

Folks in HDFS, can you please take a look at HDFS tests in branch-2 as we
are not able to get even a single Yetus run to complete due to multiple
test failures/timeout.

Thanks,
Subru

On Mon, Oct 23, 2017 at 11:26 AM, Vrushali C 
wrote:

> Hi Allen,
>
> I have filed https://issues.apache.org/jira/browse/YARN-7380 for the
> timeline service findbugs warnings.
>
> thanks
> Vrushali
>
>
> On Mon, Oct 23, 2017 at 11:14 AM, Allen Wittenauer <
> a...@effectivemachines.com
> > wrote:
>
> >
> > I’m really confused why this causes the Yahoo! QA boxes to go catatonic
> > (!?!) during the run.  As in, never come back online, probably in a
> kernel
> > panic. It’s pretty consistently in hadoop-hdfs, so something is going
> wrong
> > there… is branch-2 hdfs behaving badly?  Someone needs to run the
> > hadoop-hdfs unit tests to see what is going on.
> >
> > It’s probably worth noting that findbugs says there is a problem in the
> > timeline server hbase code.Someone should probably verify + fix that
> > issue.
> >
> >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
>

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Vrushali C]

Hi Allen,

I have filed https://issues.apache.org/jira/browse/YARN-7380 for the
timeline service findbugs warnings.

thanks
Vrushali


On Mon, Oct 23, 2017 at 11:14 AM, Allen Wittenauer  wrote:

>
> I’m really confused why this causes the Yahoo! QA boxes to go catatonic
> (!?!) during the run.  As in, never come back online, probably in a kernel
> panic. It’s pretty consistently in hadoop-hdfs, so something is going wrong
> there… is branch-2 hdfs behaving badly?  Someone needs to run the
> hadoop-hdfs unit tests to see what is going on.
>
> It’s probably worth noting that findbugs says there is a problem in the
> timeline server hbase code.Someone should probably verify + fix that
> issue.
>
>
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>

Re: Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Allen Wittenauer]

I’m really confused why this causes the Yahoo! QA boxes to go catatonic (!?!) 
during the run.  As in, never come back online, probably in a kernel panic. 
It’s pretty consistently in hadoop-hdfs, so something is going wrong there… is 
branch-2 hdfs behaving badly?  Someone needs to run the hadoop-hdfs unit tests 
to see what is going on.

It’s probably worth noting that findbugs says there is a problem in the 
timeline server hbase code.Someone should probably verify + fix that issue.



-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: 答复: [DISCUSSION] Merging HDFS-7240 Object Store (Ozone) to trunk

[Jitendra Pandey]

I have filed https://issues.apache.org/jira/browse/HDFS-12697 to ensure ozone 
stays disabled in a secure environment.
Since ozone is disabled by default and will not come with security on, it will 
not expose any new attack surface in a Hadoop deployment.
Ozone security effort will need a detailed design and discussion on a community 
jira. Hopefully, that effort will start soon after the merge.

Thanks
jitendra

On 10/20/17, 2:40 PM, "larry mccay"  wrote:

All -

I broke this list of questions out into a separate DISCUSS thread where we
can iterate over how a security audit process at merge time might look and
whether it is even something that we want to take on.

I will try and continue discussion on that thread and drive that to some
conclusion before bringing it into any particular merge discussion.

thanks,

--larry

On Fri, Oct 20, 2017 at 12:37 PM, larry mccay  wrote:

> I previously sent this same email from my work email and it doesn't seem
> to have gone through - resending from apache account (apologizing up from
> for the length)
>
> For such sizable merges in Hadoop, I would like to start doing security
> audits in order to have an initial idea of the attack surface, the
> protections available for known threats, what sort of configuration is
> being used to launch processes, etc.
>
> I dug into the architecture documents while in the middle of this list -
> nice docs!
> I do intend to try and make a generic check list like this for such
> security audits in the future so a lot of this is from that but I tried to
> also direct specific questions from those docs as well.
>
> 1. UIs
> I see there are at least two UIs - Storage Container Manager and Key Space
> Manager. There are a number of typical vulnerabilities that we find in UIs
>
> 1.1. What sort of validation is being done on any accepted user input?
> (pointers to code would be appreciated)
> 1.2. What explicit protections have been built in for (pointers to code
> would be appreciated):
>   1.2.1. cross site scripting
>   1.2.2. cross site request forgery
>   1.2.3. click jacking (X-Frame-Options)
> 1.3. What sort of authentication is required for access to the UIs?
> 1.4. What authorization is available for determining who can access what
> capabilities of the UIs for either viewing, modifying data or affecting
> object stores and related processes?
> 1.5. Are the UIs built with proxying in mind by leveraging X-Forwarded
> headers?
> 1.6. Is there any input that will ultimately be persisted in configuration
> for executing shell commands or processes?
> 1.7. Do the UIs support the trusted proxy pattern with doas impersonation?
> 1.8. Is there TLS/SSL support?
>
> 2. REST APIs
>
> 2.1. Do the REST APIs support the trusted proxy pattern with doas
> impersonation capabilities?
> 2.2. What explicit protections have been built in for:
>   2.2.1. cross site scripting (XSS)
>   2.2.2. cross site request forgery (CSRF)
>   2.2.3. XML External Entity (XXE)
> 2.3. What is being used for authentication - Hadoop Auth Module?
> 2.4. Are there separate processes for the HTTP resources (UIs and REST
> endpoints) or are the part of existing HDFS processes?
> 2.5. Is there TLS/SSL support?
> 2.6. Are there new CLI commands and/or clients for access the REST APIs?
> 2.7. Bucket Level API allows for setting of ACLs on a bucket - what
> authorization is required here - is there a restrictive ACL set on 
creation?
> 2.8. Bucket Level API allows for deleting a bucket - I assume this is
> dependent on ACLs based access control?
> 2.9. Bucket Level API to list bucket returns up to 1000 keys - is there
> paging available?
> 2.10. Storage Level APIs indicate “Signed with User Authorization” what
> does this refer to exactly?
> 2.11. Object Level APIs indicate that there is no ACL support and only
> bucket owners can read and write - but there are ACL APIs on the Bucket
> Level are they meaningless for now?
> 2.12. How does a REST client know which Ozone Handler to connect to or am
> I missing some well known NN type endpoint in the architecture doc
> somewhere?
>
> 3. Encryption
>
> 3.1. Is there any support for encryption of persisted data?
> 3.2. If so, is KMS and the hadoop key command used for key management?
>
> 4. Configuration
>
> 4.1. Are there any passwords or secrets being added to configuration?
> 4.2. If so, are they accessed via Configuration.getPassword() to allow for
> provisioning in credential providers?
> 4.3. Are there any settings that are used to launch docker containers or
> shell out any commands, etc?
>
> 5. HA
>
> 

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/8/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/

No changes




-1 overall


The following subsystems voted -1:
asflicense unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.yarn.server.nodemanager.scheduler.TestDistributedScheduler 
   
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesSchedulerActivities 
   hadoop.yarn.server.resourcemanager.rmcontainer.TestRMContainerImpl 
   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesCapacitySched 
   hadoop.yarn.server.resourcemanager.TestRMHA 
   hadoop.yarn.server.resourcemanager.TestApplicationMasterService 
   hadoop.yarn.server.resourcemanager.TestRMHAForNodeLabels 
   hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerLazyPreemption
 
   hadoop.yarn.server.resourcemanager.TestApplicationMasterLauncher 
   
hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesAppsModification 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation 
   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacitySchedulerDynamicBehavior
 
   hadoop.yarn.server.resourcemanager.scheduler.fifo.TestFifoScheduler 
   hadoop.yarn.server.resourcemanager.TestRMHATimelineCollectors 
   hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestNodeLabelContainerAllocation
 
   
hadoop.yarn.server.resourcemanager.scheduler.capacity.TestCapacityScheduler 
   hadoop.yarn.server.resourcemanager.TestReservationSystemWithRMHA 
   hadoop.yarn.server.resourcemanager.webapp.TestRMWebServicesApps 
   hadoop.yarn.server.resourcemanager.scheduler.fair.TestFSLeafQueue 
   hadoop.yarn.sls.nodemanager.TestNMSimulator 

Timed out junit tests :

   
org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.conf.TestZKConfigurationStore
 
   
org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore 
   org.apache.hadoop.yarn.server.resourcemanager.TestLeaderElectorService 
   org.apache.hadoop.mapred.pipes.TestPipeApplication 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-compile-javac-root.txt
  [284K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-checkstyle-root.txt
  [17M]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-patch-pylint.txt
  [20K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-patch-shellcheck.txt
  [20K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/whitespace-eol.txt
  [8.5M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/whitespace-tabs.txt
  [292K]

   javadoc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/diff-javadoc-javadoc-root.txt
  [760K]

   unit:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-nodemanager.txt
  [40K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager.txt
  [728K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/patch-unit-hadoop-mapreduce-project_hadoop-mapreduce-client_hadoop-mapreduce-client-jobclient.txt
  [84K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/patch-unit-hadoop-tools_hadoop-sls.txt
  [8.0K]

   asflicense:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/566/artifact/out/patch-asflicense-problems.txt
  [4.0K]

Powered by Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Shane Kumpf]

Thanks, Junping!

+1 (non-binding)

- Verified checksums and signatures
- Deployed a single node cluster on CentOS 7.2 using the binary tgz, source
tgz, and git tag
- Ran hdfs commands
- Ran pi and distributed shell using the default and docker runtimes
- Verified Docker docs
- Verified docker runtime can be disabled
- Verified the UI's

FYI, the link to the RC1 bits in the vote email references RC0.

Thanks,
-Shane

On Thu, Oct 19, 2017 at 6:42 PM, Junping Du  wrote:

> Hi folks,
>  I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.
>
>  Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line
> and will be the latest stable/production release for Apache Hadoop - it
> includes 315 new fixed issues since 2.8.1 and 69 fixes are marked as
> blocker/critical issues.
>
>   More information about the 2.8.2 release plan can be found here:
> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release
>
>   New RC is available at: http://home.apache.org/~
> junping_du/hadoop-2.8.2-RC1 du/hadoop-2.8.2-RC0>
>
>   The RC tag in git is: release-2.8.2-RC1, and the latest commit id
> is: 66c47f2a01ad9637879e95f80c41f798373828fb
>
>   The maven artifacts are available via repository.apache.org repository.apache.org/> at: https://repository.apache.org/
> content/repositories/orgapachehadoop-1064 repository.apache.org/content/repositories/orgapachehadoop-1062>
>
>   Please try the release and vote; the vote will run for the usual 5
> days, ending on 10/24/2017 6pm PST time.
>
> Thanks,
>
> Junping
>
>

RE: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

[Brahma Reddy Battula]

Hi Junping,

Thanks for driving the 2.8.2 release.

+1 (non-binding)

--Built from source and installed 3 Node HA Cluster
--Ran basic hdfs/webhdfs commands
--Ran pi,wordcount jobs
--Verified the UI's




--Brahma Reddy Battula

-Original Message-
From: Junping Du [mailto:j...@hortonworks.com] 
Sent: 20 October 2017 06:13
To: common-...@hadoop.apache.org; hdfs-...@hadoop.apache.org; 
mapreduce-dev@hadoop.apache.org; yarn-...@hadoop.apache.org
Subject: [VOTE] Release Apache Hadoop 2.8.2 (RC1)

Hi folks,
 I've created our new release candidate (RC1) for Apache Hadoop 2.8.2.

 Apache Hadoop 2.8.2 is the first stable release of Hadoop 2.8 line and 
will be the latest stable/production release for Apache Hadoop - it includes 
315 new fixed issues since 2.8.1 and 69 fixes are marked as blocker/critical 
issues.

  More information about the 2.8.2 release plan can be found here: 
https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+2.8+Release

  New RC is available at: 
http://home.apache.org/~junping_du/hadoop-2.8.2-RC1

  The RC tag in git is: release-2.8.2-RC1, and the latest commit id is: 
66c47f2a01ad9637879e95f80c41f798373828fb

  The maven artifacts are available via 
repository.apache.org at: 
https://repository.apache.org/content/repositories/orgapachehadoop-1064

  Please try the release and vote; the vote will run for the usual 5 days, 
ending on 10/24/2017 6pm PST time.

Thanks,

Junping


-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/7/

No changes




-1 overall


The following subsystems voted -1:
docker


Powered by Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/6/

No changes




-1 overall


The following subsystems voted -1:
docker


Powered by Apache Yetus 0.6.0-SNAPSHOT   http://yetus.apache.org

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/5/

No changes


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org

[jira] [Created] (MAPREDUCE-6988) Let JHS support different file systems for intermediate_done and done

[Johan Gustavsson (JIRA)]

Johan Gustavsson created MAPREDUCE-6988:
---

 Summary: Let JHS support different file systems for 
intermediate_done and done
 Key: MAPREDUCE-6988
 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6988
 Project: Hadoop Map/Reduce
  Issue Type: Improvement
  Components: jobhistoryserver
Reporter: Johan Gustavsson
Priority: Minor


Currently JHS uses filecontext to move files from intermediate_done to done 
folder. Since filecontext limits the use to 1 filesystem it makes it harder to 
use s3 as a storage for jhist files. By moving this to filesystem interface we 
can set hdfs for intermediate storage and s3 as long term storage therefore 
reducing the number of puts to s3 and removing the need for all M/R containers 
to carry a s3 sdk.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org