Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

[Andrew Wang]

Hi Arpit,

I agree the timing is not great here, but extending it to meaningfully
avoid the holidays would mean extending it an extra week (e.g. to the
29th). We've been coordinating with ASF PR for that Tuesday, so I'd really,
really like to get the RC out before then.

In terms of downstream testing, we've done extensive integration testing
with downstreams via the alphas and betas, and we have continuous
integration running at Cloudera against branch-3.0. Because of this, I have
more confidence in our integration for 3.0.0 than most Hadoop releases.

Is it meaningful to extend to say, the 21st, which provides for a full week
of voting?

Best,
Andrew

On Fri, Nov 17, 2017 at 1:27 PM, Arpit Agarwal 
wrote:

> Hi Andrew,
>
> Thank you for your hard work in getting us to this step. This is our first
> major GA release in many years.
>
> I feel a 5-day vote window ending over the weekend before thanksgiving may
> not provide sufficient time to evaluate this RC especially for downstream
> components.
>
> Would you please consider extending the voting deadline until a few days
> after the thanksgiving holiday? It would be a courtesy to our broader
> community and I see no harm in giving everyone a few days to evaluate it
> more thoroughly.
>
> On a lighter note, your deadline is also 4 minutes short of the required 5
> days. :)
>
> Regards,
> Arpit
>
>
>
> On 11/14/17, 1:34 PM, "Andrew Wang"  wrote:
>
> Hi folks,
>
> Thanks as always to the many, many contributors who helped with this
> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
> available here:
>
> http://people.apache.org/~wang/3.0.0-RC0/
>
> This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.
>
> 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
> additions include the merge of YARN resource types, API-based
> configuration
> of the CapacityScheduler, and HDFS router-based federation.
>
> I've done my traditional testing with a pseudo cluster and a Pi job.
> My +1
> to start.
>
> Best,
> Andrew
>
>
>

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/44/

[Nov 16, 2017 5:12:52 PM] (sunilg) YARN-7469. Capacity Scheduler Intra-queue 
preemption: User can starve if
[Nov 16, 2017 7:12:01 PM] (rkanter) HADOOP-14982. Clients using 
FailoverOnNetworkExceptionRetry can go into
[Nov 17, 2017 6:35:35 AM] (vvasudev) YARN-7430. Enable user re-mapping for 
Docker containers by default.


[Error replacing 'FILE' - Workspace is not accessible]

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

[Eric Payne]

Thanks for the all of the work it took to finally get htere.
+1 (binding)
Built from source and stood up a pseudo cluster with 4 NMs
Tested the following:
o Cross queue preemption
o Restarting the RM preserves work
o User limits are honored during In-queue preemption 

o Priorities are honored during In-queue preemption
o Users with different weights are assigned resources proportional to their 
weights.
o User weights are refreshable, and in-queue preemption works to honor the 
post-refresh weights 

Thanks,
-Eric Payne

  From: Andrew Wang 
 To: "common-dev@hadoop.apache.org" ; 
"yarn-...@hadoop.apache.org" ; 
"mapreduce-...@hadoop.apache.org" ; 
"hdfs-...@hadoop.apache.org"  
 Sent: Tuesday, November 14, 2017 3:34 PM
 Subject: [VOTE] Release Apache Hadoop 3.0.0 RC0
   
Hi folks,

Thanks as always to the many, many contributors who helped with this
release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
available here:

http://people.apache.org/~wang/3.0.0-RC0/

This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.

3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
additions include the merge of YARN resource types, API-based configuration
of the CapacityScheduler, and HDFS router-based federation.

I've done my traditional testing with a pseudo cluster and a Pi job. My +1
to start.

Best,
Andrew


   

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

[Subru Krishnan]

Wrapping up the vote with my +1.

Deployed RC3 on a federated YARN cluster with 6 sub-clusters:
- ran multiple sample jobs
- enabled opportunistic containers and submitted more samples
- configured HDFS federation and reran jobs.


With 13 binding +1s and 7 non-binding +1s and no +/-1s, pleased to announce
the vote is passed successfully.

Thanks to the many of you who contributed to the release and made this
possible and to everyone in this thread who took the time/effort to
validate and vote!

We’ll push the release bits and send out an announcement for 2.9.0 soon.

Cheers,
Subru

On Fri, Nov 17, 2017 at 12:41 PM Eric Payne 
wrote:

> Thanks Arun and Subru for the hard work on this release.
>
> +1 (binding)
>
> Built from source and stood up a pseudo cluster with 4 NMs
>
> Tested the following:
>
> o User limits are honored during In-queue preemption
>
> o Priorities are honored during In-queue preemption
>
> o Can kill applications from the command line
>
> o Users with different weights are assigned resources proportional to
> their weights.
>
> Thanks,
> -Eric Payne
>
>
> --
> *From:* Arun Suresh 
> *To:* yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; Hadoop
> Common ; Hdfs-dev <
> hdfs-...@hadoop.apache.org>
> *Cc:* Subramaniam Krishnan 
> *Sent:* Monday, November 13, 2017 6:10 PM
>
> *Subject:* [VOTE] Release Apache Hadoop 2.9.0 (RC3)
>
> Hi Folks,
>
> Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be the
> starting release for Apache Hadoop 2.9.x line - it includes 30 New Features
> with 500+ subtasks, 407 Improvements, 790 Bug fixes new fixed issues since
> 2.8.2.
>
> More information about the 2.9.0 release plan can be found here:
> *
> https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Version2.9
> <
> https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Version2.9
> >*
>
> New RC is available at: *
> https://home.apache.org/~asuresh/hadoop-2.9.0-RC3/
> *
>
> The RC tag in git is: release-2.9.0-RC3, and the latest commit id is:
> 756ebc8394e473ac25feac05fa493f6d612e6c50.
>
> The maven artifacts are available via repository.apache.org at:
> <
> https://www.google.com/url?q=https%3A%2F%2Frepository.apache.org%2Fcontent%2Frepositories%2Forgapachehadoop-1066=D=1=AFQjCNFcern4uingMV_sEreko_zeLlgdlg
> >*https://repository.apache.org/content/repositories/orgapachehadoop-1068/
>  >*
>
> We are carrying over the votes from the previous RC given that the delta is
> the license fix.
>
> Given the above - we are also going to stick with the original deadline for
> the vote : ending on Friday 17th November 2017 2pm PT time.
>
> Thanks,
> -Arun/Subru
>
>
>

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

[Arpit Agarwal]

Hi Andrew,

Thank you for your hard work in getting us to this step. This is our first 
major GA release in many years.

I feel a 5-day vote window ending over the weekend before thanksgiving may not 
provide sufficient time to evaluate this RC especially for downstream 
components.

Would you please consider extending the voting deadline until a few days after 
the thanksgiving holiday? It would be a courtesy to our broader community and I 
see no harm in giving everyone a few days to evaluate it more thoroughly.

On a lighter note, your deadline is also 4 minutes short of the required 5 
days. :)

Regards,
Arpit



On 11/14/17, 1:34 PM, "Andrew Wang"  wrote:

Hi folks,

Thanks as always to the many, many contributors who helped with this
release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
available here:

http://people.apache.org/~wang/3.0.0-RC0/

This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.

3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
additions include the merge of YARN resource types, API-based configuration
of the CapacityScheduler, and HDFS router-based federation.

I've done my traditional testing with a pseudo cluster and a Pi job. My +1
to start.

Best,
Andrew

[jira] [Created] (HADOOP-15051) FSDataOutputStream returned by LocalFileSystem#createNonRecursive doesn

[Ted Yu (JIRA)]

Ted Yu created HADOOP-15051:
---

 Summary: FSDataOutputStream returned by 
LocalFileSystem#createNonRecursive doesn
 Key: HADOOP-15051
 URL: https://issues.apache.org/jira/browse/HADOOP-15051
 Project: Hadoop Common
  Issue Type: Bug
Affects Versions: 3.0.0-beta1
Reporter: Ted Yu






--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

[Eric Payne]

Thanks Arun and Subru for the hard work on this release.

+1 (binding)

Built from source and stood up a pseudo cluster with 4 NMs
Tested the following:
o User limits are honored during In-queue preemption 

o Priorities are honored during In-queue preemption
o Can kill applications from the command line
o Users with different weights are assigned resources proportional to their 
weights.
Thanks,-Eric Payne


  From: Arun Suresh 
 To: yarn-...@hadoop.apache.org; mapreduce-...@hadoop.apache.org; Hadoop Common 
; Hdfs-dev  
Cc: Subramaniam Krishnan 
 Sent: Monday, November 13, 2017 6:10 PM
 Subject: [VOTE] Release Apache Hadoop 2.9.0 (RC3)
   
Hi Folks,

Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be the
starting release for Apache Hadoop 2.9.x line - it includes 30 New Features
with 500+ subtasks, 407 Improvements, 790 Bug fixes new fixed issues since
2.8.2.

More information about the 2.9.0 release plan can be found here:
*https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#Roadmap-Version2.9
*

New RC is available at: *https://home.apache.org/~asuresh/hadoop-2.9.0-RC3/
*

The RC tag in git is: release-2.9.0-RC3, and the latest commit id is:
756ebc8394e473ac25feac05fa493f6d612e6c50.

The maven artifacts are available via repository.apache.org at:
*https://repository.apache.org/content/repositories/orgapachehadoop-1068/
*

We are carrying over the votes from the previous RC given that the delta is
the license fix.

Given the above - we are also going to stick with the original deadline for
the vote : ending on Friday 17th November 2017 2pm PT time.

Thanks,
-Arun/Subru


   

Re: [VOTE] Release Apache Hadoop 3.0.0 RC0

[Andrew Wang]

Thanks for the spot, normally create-release spits those out. I uploaded
asc and mds for the release artifacts.

Best,
Andrew

On Thu, Nov 16, 2017 at 11:33 PM, Akira Ajisaka  wrote:

> Hi Andrew,
>
> Signatures are missing. Would you upload them?
>
> Thanks,
> Akira
>
>
> On 2017/11/15 6:34, Andrew Wang wrote:
>
>> Hi folks,
>>
>> Thanks as always to the many, many contributors who helped with this
>> release. I've created RC0 for Apache Hadoop 3.0.0. The artifacts are
>> available here:
>>
>> http://people.apache.org/~wang/3.0.0-RC0/
>>
>> This vote will run 5 days, ending on Nov 19th at 1:30pm Pacific.
>>
>> 3.0.0 GA contains 291 fixed JIRA issues since 3.0.0-beta1. Notable
>> additions include the merge of YARN resource types, API-based
>> configuration
>> of the CapacityScheduler, and HDFS router-based federation.
>>
>> I've done my traditional testing with a pseudo cluster and a Pi job. My +1
>> to start.
>>
>> Best,
>> Andrew
>>
>>

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

[Jason Lowe]

Thanks for putting this release together!

+1 (binding)

- Verified signatures and digests
- Successfully built from source including native
- Deployed to single-node cluster and ran some test jobs

Jason


On Mon, Nov 13, 2017 at 6:10 PM, Arun Suresh  wrote:

> Hi Folks,
>
> Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be the
> starting release for Apache Hadoop 2.9.x line - it includes 30 New Features
> with 500+ subtasks, 407 Improvements, 790 Bug fixes new fixed issues since
> 2.8.2.
>
> More information about the 2.9.0 release plan can be found here:
> *https://cwiki.apache.org/confluence/display/HADOOP/
> Roadmap#Roadmap-Version2.9
>  Roadmap#Roadmap-Version2.9>*
>
> New RC is available at: *https://home.apache.org/~
> asuresh/hadoop-2.9.0-RC3/
> *
>
> The RC tag in git is: release-2.9.0-RC3, and the latest commit id is:
> 756ebc8394e473ac25feac05fa493f6d612e6c50.
>
> The maven artifacts are available via repository.apache.org at:
>  apache.org%2Fcontent%2Frepositories%2Forgapachehadoop-1066=D&
> sntz=1=AFQjCNFcern4uingMV_sEreko_zeLlgdlg>*https://
> repository.apache.org/content/repositories/orgapachehadoop-1068/
>  >*
>
> We are carrying over the votes from the previous RC given that the delta is
> the license fix.
>
> Given the above - we are also going to stick with the original deadline for
> the vote : ending on Friday 17th November 2017 2pm PT time.
>
> Thanks,
> -Arun/Subru
>

[jira] [Created] (HADOOP-15050) swift:// doesn't support createNonRecursive. hence the new FS createFile(path) builder

[Steve Loughran (JIRA)]

Steve Loughran created HADOOP-15050:
---

 Summary: swift:// doesn't support createNonRecursive. hence the 
new FS createFile(path) builder
 Key: HADOOP-15050
 URL: https://issues.apache.org/jira/browse/HADOOP-15050
 Project: Hadoop Common
  Issue Type: Bug
  Components: fs/swift
Affects Versions: 2.9.0
Reporter: Steve Loughran


swift throws an exception in createNonRecursive(), so fails to work with the 
new {{FileSystem.createFile(path)}} which uses that mode by default (because 
its the right thing to do :)



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [DISCUSS] Apache Hadoop 2.7.5 Release Plan

[Wei-Chiu Chuang]

Hi Konstantin,
Thanks for initiating the release effort.

I am marking HDFS-12641  as
a blocker for Hadoop 2.7.5 because during our internal testing for CDH we
found a regression in HDFS-11445 that was fixed by HDFS-11755 (technically
not a real regression since HDFS-11755 was committed before HDFS-11445).
The regression results in bogus corrupt block reports. It is not clear to
me if the same behavior is in Apache Hadoop, but since the later
(HDFS-11755) is currently Hadoop 2.8.x and above, I would want to be more
cautious about it.

On Thu, Nov 16, 2017 at 5:20 PM, Konstantin Shvachko 
wrote:

> Hi developers,
>
> We have accumulated about 30 commits on branch-2.7. Those are mostly
> valuable bug fixes, minor optimizations and test corrections. I would like
> to propose to make a quick maintenance release 2.7.5.
>
> If there are no objections I'll start preparations.
>
> Thanks,
> --Konstantin
>



-- 
A very happy Clouderan

Re: [VOTE] Release Apache Hadoop 2.9.0 (RC3)

[Arun Suresh]

+1 (binding)

* Built from source
* Set up a 4 node cluster with RM HA enabled
* Performed some basic HDFS commands
* Ran a bunch of pi & sleep jobs - with and without opportunistic containers
* Performed some basic RM failover testing.

Cheers
-Arun

On Thu, Nov 16, 2017 at 11:12 PM, Akira Ajisaka  wrote:

> +1
>
> * Downloaded source tarball and verified checksum and signature
> * Compiled the source with OpenJDK 1.8.0_151 and CentOS 7.4
> * Deployed a pseudo cluster and ran some simple MR jobs
>
> I noticed ISA-L build options are documented in BUILDING.txt
> but the options do not exist in 2.x releases.
> Filed HADOOP-15045 to fix this issue.
> I think this issue doesn't block the release.
>
> Thanks and regards,
> Akira
>
> On 2017/11/14 9:10, Arun Suresh wrote:
>
>> Hi Folks,
>>
>> Apache Hadoop 2.9.0 is the first release of Hadoop 2.9 line and will be
>> the
>> starting release for Apache Hadoop 2.9.x line - it includes 30 New
>> Features
>> with 500+ subtasks, 407 Improvements, 790 Bug fixes new fixed issues since
>> 2.8.2.
>>
>> More information about the 2.9.0 release plan can be found here:
>> *https://cwiki.apache.org/confluence/display/HADOOP/Roadmap#
>> Roadmap-Version2.9
>> > Roadmap-Version2.9>*
>>
>> New RC is available at: *https://home.apache.org/~asur
>> esh/hadoop-2.9.0-RC3/
>> *
>>
>> The RC tag in git is: release-2.9.0-RC3, and the latest commit id is:
>> 756ebc8394e473ac25feac05fa493f6d612e6c50.
>>
>> The maven artifacts are available via repository.apache.org at:
>> > e.org%2Fcontent%2Frepositories%2Forgapachehadoop-1066=D
>> ntz=1=AFQjCNFcern4uingMV_sEreko_zeLlgdlg>*https://reposi
>> tory.apache.org/content/repositories/orgapachehadoop-1068/
>> > >*
>>
>> We are carrying over the votes from the previous RC given that the delta
>> is
>> the license fix.
>>
>> Given the above - we are also going to stick with the original deadline
>> for
>> the vote : ending on Friday 17th November 2017 2pm PT time.
>>
>> Thanks,
>> -Arun/Subru
>>
>>

[jira] [Resolved] (HADOOP-9327) Out of date code examples

[Andras Bokor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Bokor resolved HADOOP-9327.
--
Resolution: Invalid

> Out of date code examples
> -
>
> Key: HADOOP-9327
> URL: https://issues.apache.org/jira/browse/HADOOP-9327
> Project: Hadoop Common
>  Issue Type: Bug
>Affects Versions: 2.0.3-alpha
>Reporter: Hao Zhong
>
> 1. This page contains code examples that use JobConfigurationParser
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/tools/rumen/package-summary.html
> "JobConfigurationParser jcp = 
>   new JobConfigurationParser(interestedProperties);"
> JobConfigurationParser is deleted in 2.0.3
> 2. This page contains code examples that use ContextFactory
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/metrics/package-summary.html
> " ContextFactory factory = ContextFactory.getFactory();
> ... examine and/or modify factory attributes ...
> MetricsContext context = factory.getContext("myContext");"
> ContextFactory is deleted in 2.0.3
> 3. This page contains code examples that use LoggedNetworkTopology
> http://hadoop.apache.org/docs/current/api/org/apache/hadoop/tools/rumen/package-summary.html
> " do.init("topology.json", conf);
> 
>   // get the job summary using TopologyBuilder
>   LoggedNetworkTopology topology = topologyBuilder.build();"
> LoggedNetworkTopology is deleted in 2.0.3
> Please revise the documentation to reflect the code.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-7085) Documentation uses urls with "current"

[Andras Bokor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-7085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Bokor resolved HADOOP-7085.
--
Resolution: Won't Fix

This is not an issue in 2.x+ documentations.

> Documentation uses urls with "current"
> --
>
> Key: HADOOP-7085
> URL: https://issues.apache.org/jira/browse/HADOOP-7085
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: documentation
>Reporter: Eli Collins
>Priority: Minor
>  Labels: newbie
>
> The mapred and hdfs default links on the following page are broken. Looking 
> into the cause, the links always point to the current docs so old generated 
> docs have the wrong pointers (they should point to docs/ instead of 
> docs/current). This is because cluster_setup.xml uses site.xml which defines 
> references to the docs using http://hadoop.apache.org/core/docs/current.
> http://hadoop.apache.org/common/docs/r0.20.2/cluster_setup.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-9386) MapFile.fix does not allow custom index intervals

[Andras Bokor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-9386?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andras Bokor resolved HADOOP-9386.
--
Resolution: Duplicate

> MapFile.fix does not allow custom index intervals
> -
>
> Key: HADOOP-9386
> URL: https://issues.apache.org/jira/browse/HADOOP-9386
> Project: Hadoop Common
>  Issue Type: Improvement
>  Components: io
>Affects Versions: 1.0.4
>Reporter: Robert Dyer
>Assignee: Andras Bokor
>Priority: Minor
>
> MapFile.Writer provides a setIndexInterval() method for customizing how 
> frequently an index is written.  By default it is set to 128.
> The method MapFile.fix() provides the ability to regenerate just the index 
> file from the existing data file.  This method hard codes the index interval 
> to 128, thus making it impossible to re-generate an index with custom 
> intervals.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Resolved] (HADOOP-15048) ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run

[Steve Loughran (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Steve Loughran resolved HADOOP-15048.
-
Resolution: Won't Fix

> ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in 
> parallel run
> 
>
> Key: HADOOP-15048
> URL: https://issues.apache.org/jira/browse/HADOOP-15048
> Project: Hadoop Common
>  Issue Type: Sub-task
>  Components: fs/azure
>Affects Versions: 3.1.0
>Reporter: Steve Loughran
>Priority: Minor
>
> ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock failed with 
> assert(irstEndTime < secondStartTime)
> Fairly uniformative stack trace on an assert failure, test run 
> {{-Dparallel-tests -DtestsThreadCount=10  -Dscale}}
> Irrespective of root cause, exception needs changing to provide enough 
> information; include raised exceptions in thrown failures, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15048) ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock assert failure in parallel run

[Steve Loughran (JIRA)]

Steve Loughran created HADOOP-15048:
---

 Summary: ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock 
assert failure in parallel run
 Key: HADOOP-15048
 URL: https://issues.apache.org/jira/browse/HADOOP-15048
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/azure
Affects Versions: 3.1.0
Reporter: Steve Loughran
Priority: Minor


ITestNativeAzureFileSystemLive.testLeaseAsDistributedLock failed with 
assert(irstEndTime < secondStartTime)


Fairly uniformative stack trace on an assert failure, test run 
{{-Dparallel-tests -DtestsThreadCount=10  -Dscale}}

Irrespective of root cause, exception needs changing to provide enough 
information; include raised exceptions in thrown failures, etc.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15047) Python is required for -Preleasedoc but not documented in branch-2.8

[Akira Ajisaka (JIRA)]

Akira Ajisaka created HADOOP-15047:
--

 Summary: Python is required for -Preleasedoc but not documented in 
branch-2.8
 Key: HADOOP-15047
 URL: https://issues.apache.org/jira/browse/HADOOP-15047
 Project: Hadoop Common
  Issue Type: Bug
  Components: build, documentation
Reporter: Akira Ajisaka


Python is required for -Preleasedoc but not documented in branch-2.8.

* In trunk and branch-3.0, it was documented by HADOOP-10854.
* In branch-2 and branch-2.9, it was documented by YARN-4849.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15046) Document Apache Hadoop does not support Java 9 in BUILDING.txt

[Akira Ajisaka (JIRA)]

Akira Ajisaka created HADOOP-15046:
--

 Summary: Document Apache Hadoop does not support Java 9 in 
BUILDING.txt
 Key: HADOOP-15046
 URL: https://issues.apache.org/jira/browse/HADOOP-15046
 Project: Hadoop Common
  Issue Type: Bug
  Components: documentation
Reporter: Akira Ajisaka


Now the java version is documented as "JDK 1.8+" or "JDK 1.7+", we should 
update this to "JDK 1.8" or "JDK 1.7 or 1.8" to exclude Java 9.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/43/

[Nov 15, 2017 6:33:48 PM] (jlowe) YARN-7361. Improve the docker container 
runtime documentation.
[Nov 16, 2017 12:45:06 AM] (xiao) HADOOP-15023. ValueQueue should also validate 
(int) (lowWatermark *




-1 overall


The following subsystems voted -1:
asflicense unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Unreaped Processes :

   hadoop-hdfs:17 
   bkjournal:7 
   hadoop-mapreduce-client-jobclient:9 
   hadoop-archive-logs:1 
   hadoop-archives:1 
   hadoop-distcp:5 
   hadoop-extras:1 
   hadoop-sls:1 
   hadoop-yarn-applications-distributedshell:1 
   hadoop-yarn-client:5 
   hadoop-yarn-server-tests:2 
   hadoop-yarn-server-timelineservice:1 

Failed junit tests :

   hadoop.net.TestDNS 
   hadoop.hdfs.TestAppendSnapshotTruncate 
   hadoop.hdfs.TestDFSAddressConfig 
   hadoop.mapred.TestSpecialCharactersInOutputPath 
   hadoop.tools.TestIntegration 
   hadoop.tools.TestDistCpViewFs 
   hadoop.tools.TestDistCpSystem 
   hadoop.resourceestimator.solver.impl.TestLpSolver 
   hadoop.resourceestimator.service.TestResourceEstimatorService 
   hadoop.yarn.sls.nodemanager.TestNMSimulator 
   
hadoop.yarn.applications.distributedshell.TestDistributedShellWithNodeLabels 

Timed out junit tests :

   org.apache.hadoop.hdfs.TestReservedRawPaths 
   org.apache.hadoop.hdfs.TestAclsEndToEnd 
   org.apache.hadoop.hdfs.TestFileCreation 
   org.apache.hadoop.hdfs.TestDatanodeDeath 
   org.apache.hadoop.hdfs.TestDFSClientRetries 
   org.apache.hadoop.hdfs.TestFileAppend2 
   org.apache.hadoop.hdfs.TestFileCorruption 
   org.apache.hadoop.hdfs.TestFileCreationDelete 
   org.apache.hadoop.hdfs.TestSeekBug 
   org.apache.hadoop.hdfs.TestRestartDFS 
   org.apache.hadoop.hdfs.TestDFSClientSocketSize 
   org.apache.hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead 
   org.apache.hadoop.hdfs.TestDFSRollback 
   org.apache.hadoop.hdfs.TestDFSClientExcludedNodes 
   org.apache.hadoop.hdfs.TestAbandonBlock 
   org.apache.hadoop.contrib.bkjournal.TestBootstrapStandbyWithBKJM 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperJournalManager 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperAsHASharedDir 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperEditLogStreams 
   org.apache.hadoop.contrib.bkjournal.TestBookKeeperSpeculativeRead 
   org.apache.hadoop.contrib.bkjournal.TestCurrentInprogress 
   org.apache.hadoop.mapred.TestClusterMapReduceTestCase 
   org.apache.hadoop.mapred.TestMRIntermediateDataEncryption 
   org.apache.hadoop.mapred.TestJobSysDirWithDFS 
   org.apache.hadoop.mapred.TestMRTimelineEventHandling 
   org.apache.hadoop.mapred.join.TestDatamerge 
   org.apache.hadoop.mapred.TestMiniMRWithDFSWithDistinctUsers 
   org.apache.hadoop.mapred.TestMiniMRClientCluster 
   org.apache.hadoop.mapred.TestReduceFetchFromPartialMem 
   org.apache.hadoop.mapred.TestMROpportunisticMaps 
   org.apache.hadoop.tools.TestHadoopArchiveLogsRunner 
   org.apache.hadoop.tools.TestHadoopArchives 
   org.apache.hadoop.tools.TestDistCpWithAcls 
   org.apache.hadoop.tools.TestDistCpSync 
   org.apache.hadoop.tools.TestDistCpWithXAttrs 
   org.apache.hadoop.tools.TestDistCpSyncReverseFromTarget 
   org.apache.hadoop.tools.TestDistCpSyncReverseFromSource 
   org.apache.hadoop.tools.TestCopyFiles 
   org.apache.hadoop.yarn.sls.TestSLSRunner 
   
org.apache.hadoop.yarn.applications.distributedshell.TestDistributedShell 
   org.apache.hadoop.yarn.client.api.impl.TestAMRMProxy 
   org.apache.hadoop.yarn.client.cli.TestYarnCLI 
   org.apache.hadoop.yarn.client.TestApplicationClientProtocolOnHA 
   org.apache.hadoop.yarn.client.api.impl.TestAMRMClient 
   org.apache.hadoop.yarn.client.api.impl.TestYarnClient 
   org.apache.hadoop.yarn.server.TestMiniYarnCluster 
   org.apache.hadoop.yarn.server.TestContainerManagerSecurity 
   
org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServices
 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/43/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/43/artifact/out/diff-compile-javac-root.txt
  [324K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/43/artifact/out/diff-checkstyle-root.txt
  [16M]

   pylint: