[jira] [Reopened] (HADOOP-15922) DelegationTokenAuthenticationFilter get wrong doAsUser since it does not decode URL

[Eric Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HADOOP-15922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Yang reopened HADOOP-15922:


> DelegationTokenAuthenticationFilter get wrong doAsUser since it does not 
> decode URL
> ---
>
> Key: HADOOP-15922
> URL: https://issues.apache.org/jira/browse/HADOOP-15922
> Project: Hadoop Common
>  Issue Type: Bug
>  Components: common, kms
>Reporter: He Xiaoqiao
>Assignee: He Xiaoqiao
>Priority: Major
> Fix For: 3.3.0, 3.1.2, 3.2.1
>
> Attachments: HADOOP-15922.001.patch, HADOOP-15922.002.patch, 
> HADOOP-15922.003.patch, HADOOP-15922.004.patch, HADOOP-15922.005.patch
>
>
> DelegationTokenAuthenticationFilter get wrong doAsUser when proxy user from 
> client is complete kerberos name (e.g., user/hostn...@realm.com, actually it 
> is acceptable), because DelegationTokenAuthenticationFilter does not decode 
> DOAS parameter in URL which is encoded by {{URLEncoder}} at client.
> e.g. KMS as example:
> a. KMSClientProvider creates connection to KMS Server using 
> DelegationTokenAuthenticatedURL#openConnection.
> b. If KMSClientProvider is a doAsUser, KMSClientProvider will put {{doas}} 
> with url encoded user as one parameter of http request. 
> {code:java}
> // proxyuser
> if (doAs != null) {
>   extraParams.put(DO_AS, URLEncoder.encode(doAs, "UTF-8"));
> }
> {code}
> c. when KMS server receives the request, it does not decode the proxy user.
> As result, KMS Server will get the wrong proxy user if this proxy user is 
> complete Kerberos Name or it includes some special character. Some other 
> authentication and authorization exception will throws next to it.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

[Sunil G]

Thanks @Eric Payne 
Due to another issue, we have to spin an RC1. In this case, could we revert
the patch which caused this problem.

Cancelling this RC0. Thanks to everyone who voted.
I will spin RC1 as soon as possible.

- Sunil


On Fri, Nov 30, 2018 at 4:14 AM Eric Payne 
wrote:

> The problem is not with preemption. The yarn-site.xml that I use for my
> pseudo-cluster includes a second xml:
> xi:include href=".../yarn-scheduler.xml"
>
> The property for yarn.resourcemanager.scheduler.monitor.enable = true is
> in this yarn-scheduler.xml.
>
> This value IS READ when then RM starts.
>
> However, when the refreshQueues command is run, this value IS NOT READ.
>
> So, it looks like xml include files are not read on refresh. This will
> affect any property. I just happened to notice it on the preemption
> properties.
>
> I would like input from all of you to determine if this is a blocker for
> release. I'm on the fence.
>
> Thanks,
> -Eric
>
>
>
>
>
>
> On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne <
> erichadoo...@yahoo.com.INVALID> wrote:
>
>
>
>
>
> Sunil,
>
> So, the basic symptoms are that if preemption is enabled on any queue, the
> preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all
> of the preemption-specific properties are set back to the default values.
>
> This was introduced in branch-3.1, so it is NOT new behavior for release
> 3.2.0. I am still tracking down the cause. I will open a JIRA once I do
> further investigation if there is not one already.
>
> This will be a problem for installations which use preemption and which
> use the refreshQueues feature.
>
> Thanks,
> -Eric
>
>
> On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne <
> eric.payne1...@yahoo.com> wrote:
>
>
>
>
>
> Sunil, thanks for all of the hard work on this release.
>
> I have discovered that queue refresh doesn't work in some cases. For
> example, when I change
> yarn.scheduler.capacity.root.default.disable_preemption, it doesn't take
> effect unless I restart the RM.
>
> I am still investigating, but I thought I should bring this up asap.
>
> Thanks,
> -Eric
>
>
>
>
> On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G 
> wrote:
>
>
>
>
>
> Hi folks,
>
>
>
> Thanks to all contributors who helped in this release [1]. I have created
>
> first release candidate (RC0) for Apache Hadoop 3.2.0.
>
>
> Artifacts for this RC are available here:
>
> http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
>
>
>
> RC tag in git is release-3.2.0-RC0.
>
>
>
> The maven artifacts are available via repository.apache.org at
>
> https://repository.apache.org/content/repositories/orgapachehadoop-1174/
>
>
> This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
>
>
>
> 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> additions
>
> are the highlights of this release.
>
> 1. Node Attributes Support in YARN
>
> 2. Hadoop Submarine project for running Deep Learning workloads on YARN
>
> 3. Support service upgrade via YARN Service API and CLI
>
> 4. HDFS Storage Policy Satisfier
>
> 5. Support Windows Azure Storage - Blob file system in Hadoop
>
> 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
>
> 7. Improvements in Router-based HDFS federation
>
>
>
> Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
>
> I have done few testing with my pseudo cluster. My +1 to start.
>
>
>
> Regards,
>
> Sunil
>
>
>
> [1]
>
>
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
>
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> ORDER BY fixVersion ASC
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

[Eric Payne]

The problem is not with preemption. The yarn-site.xml that I use for my 
pseudo-cluster includes a second xml:
xi:include href=".../yarn-scheduler.xml"

The property for yarn.resourcemanager.scheduler.monitor.enable = true is in 
this yarn-scheduler.xml.

This value IS READ when then RM starts.

However, when the refreshQueues command is run, this value IS NOT READ.

So, it looks like xml include files are not read on refresh. This will affect 
any property. I just happened to notice it on the preemption properties.

I would like input from all of you to determine if this is a blocker for 
release. I'm on the fence.

Thanks,
-Eric






On Wednesday, November 28, 2018, 4:58:50 PM CST, Eric Payne 
 wrote: 





Sunil,

So, the basic symptoms are that if preemption is enabled on any queue, the 
preemption is disabled after a 'yarn rm -refreshQueues'. In addition, all of 
the preemption-specific properties are set back to the default values.

This was introduced in branch-3.1, so it is NOT new behavior for release 3.2.0. 
I am still tracking down the cause. I will open a JIRA once I do further 
investigation if there is not one already.

This will be a problem for installations which use preemption and which use the 
refreshQueues feature.

Thanks,
-Eric


On Wednesday, November 28, 2018, 11:47:06 AM CST, Eric Payne 
 wrote: 





Sunil, thanks for all of the hard work on this release.

I have discovered that queue refresh doesn't work in some cases. For example, 
when I change yarn.scheduler.capacity.root.default.disable_preemption, it 
doesn't take effect unless I restart the RM.

I am still investigating, but I thought I should bring this up asap.

Thanks,
-Eric




On Friday, November 23, 2018, 6:07:04 AM CST, Sunil G  
wrote: 





Hi folks,



Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/



RC tag in git is release-3.2.0-RC0.



The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/


This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.



3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]

https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org


-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15961) S3A committers: make sure there's regular progress() calls

[Steve Loughran (JIRA)]

Steve Loughran created HADOOP-15961:
---

 Summary: S3A committers: make sure there's regular progress() calls
 Key: HADOOP-15961
 URL: https://issues.apache.org/jira/browse/HADOOP-15961
 Project: Hadoop Common
  Issue Type: Sub-task
  Components: fs/s3
Reporter: Steve Loughran


MAPREDUCE-7164 highlights how inside job/task commit more context.progress() 
callbacks are needed, just for HDFS.

the S3A committers should be reviewed similarly.

At a glance:

StagingCommitter.commitTaskInternal() is at risk if a task write upload enough 
data to the localfs that the upload takes longer than the timeout.

it should call progress it every single file commits, or better: modify 
{{uploadFileToPendingCommit}} to take a Progressable for progress callbacks 
after every part upload.





--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

[jira] [Created] (HADOOP-15960) Update guava to 27.0-jre in hadoop-common

[Gabor Bota (JIRA)]

Gabor Bota created HADOOP-15960:
---

 Summary: Update guava to 27.0-jre in hadoop-common
 Key: HADOOP-15960
 URL: https://issues.apache.org/jira/browse/HADOOP-15960
 Project: Hadoop Common
  Issue Type: Bug
  Components: common, security
Affects Versions: 3.1.0
Reporter: Gabor Bota
Assignee: Gabor Bota


com.google.guava:guava should be upgraded to 27.0-jre due to new CVE's found 
CVE-2018-10237



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/

[Nov 27, 2018 11:57:24 PM] (aengineer) HDDS-846. Exports ozone metrics to 
prometheus. Contributed by Elek,
[Nov 28, 2018 8:42:14 AM] (sumasai) YARN-9034. ApplicationCLI should have 
option to take clusterId.
[Nov 28, 2018 8:46:53 AM] (sumasai) YARN-9044. LogsCLI should contact ATSv2 for 
-am option. Contributed by
[Nov 28, 2018 8:57:42 AM] (aajisaka) MAPREDUCE-6190. If a task stucks before 
its first heartbeat, it never
[Nov 28, 2018 2:46:11 PM] (jlowe) YARN-8812. Containers fail during creating a 
symlink which started with
[Nov 28, 2018 5:45:09 PM] (mackrorysd) HADOOP-15370. S3A log message on rm 
s3a://bucket/ not intuitive.
[Nov 28, 2018 5:45:09 PM] (mackrorysd) HADOOP-15798. LocalMetadataStore put() 
does not retain isDeleted in
[Nov 28, 2018 5:45:09 PM] (mackrorysd) HADOOP-15947. Fix 
ITestDynamoDBMetadataStore test error issues.
[Nov 28, 2018 6:53:12 PM] (hanishakoneru) HDDS-284. Checksum for ChunksData.
[Nov 28, 2018 7:25:47 PM] (gifuma) HDFS-14108. Performance improvement in 
BlockManager Data Structures.
[Nov 28, 2018 7:33:22 PM] (gifuma) HDFS-14102. Performance improvement in 
BlockPlacementPolicyDefault.
[Nov 28, 2018 8:54:59 PM] (jlowe) MAPREDUCE-7164. FileOutputCommitter does not 
report progress while
[Nov 28, 2018 9:36:21 PM] (wangda) YARN-9030. Log aggregation changes to handle 
filesystems which do not
[Nov 28, 2018 10:09:52 PM] (wangda) YARN-8882. [YARN-8851] Add a shared device 
mapping manager (scheduler)
[Nov 28, 2018 10:31:31 PM] (wangda) YARN-9061. Improve the GPU/FPGA module log 
message of
[Nov 28, 2018 10:36:30 PM] (wangda) YARN-8974. Improve the assertion message in 
TestGPUResourceHandler.
[Nov 28, 2018 10:39:06 PM] (wangda) YARN-8975. [Submarine] Use predefined 
Charset object
[Nov 28, 2018 11:03:06 PM] (wangda) YARN-8989. [YARN-8851] Move 
DockerCommandPlugin volume related APIs'
[Nov 29, 2018 1:34:14 AM] (wwei) YARN-9067. YARN Resource Manager is running 
OOM because of leak of




-1 overall


The following subsystems voted -1:
asflicense findbugs hadolint pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.hdfs.web.TestWebHdfsTimeouts 
   hadoop.hdfs.web.TestWebHDFS 
  

   cc:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-compile-cc-root.txt
  [4.0K]

   javac:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-compile-javac-root.txt
  [336K]

   checkstyle:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-checkstyle-root.txt
  [17M]

   hadolint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-patch-hadolint.txt
  [4.0K]

   pathlen:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/pathlen.txt
  [12K]

   pylint:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-patch-pylint.txt
  [40K]

   shellcheck:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-patch-shellcheck.txt
  [68K]

   shelldocs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/diff-patch-shelldocs.txt
  [12K]

   whitespace:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/whitespace-eol.txt
  [9.3M]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/whitespace-tabs.txt
  [1.1M]

   findbugs:

   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-hdds_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-hdds_framework.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-hdds_tools.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-ozone_client.txt
  [8.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-ozone_common.txt
  [4.0K]
   
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/972/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt
  [8.0K]
   

[jira] [Created] (HADOOP-15959) revert HADOOP-12751

[Steve Loughran (JIRA)]

Steve Loughran created HADOOP-15959:
---

 Summary: revert HADOOP-12751
 Key: HADOOP-15959
 URL: https://issues.apache.org/jira/browse/HADOOP-15959
 Project: Hadoop Common
  Issue Type: Improvement
  Components: security
Affects Versions: 2.8.5, 2.7.7, 3.0.3, 2.9.2, 3.1.1, 3.2.0
Reporter: Steve Loughran
Assignee: Steve Loughran


HADOOP-12751 doesn't quite work right. Revert.

(this patch is so jenkins can do the test runs)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: Apache Hadoop 3.1.2 release plan

[Steve Loughran]

sorry, catching up on this

1 Got the HADOOP-15932
2. There's something I want to revert; I'' do that ASAP


On 28 Nov 2018, at 23:30, Wangda Tan 
mailto:wheele...@gmail.com>> wrote:

Hi Steve,

Is there any other JIRAs need to be released 3.1.2? I want to get 3.1.2 release 
starts earlier if possible.

Thanks,
Wangda

On Wed, Nov 21, 2018 at 5:31 PM Wangda Tan 
mailto:wheele...@gmail.com>> wrote:
Hi Steve,

It gonna be really good if you could get the patch committed during US 
Thanksgiving holiday so I can start releasing starting next week.

Thanks,
Wangda

On Mon, Nov 19, 2018 at 10:21 AM Wangda Tan 
mailto:wheele...@gmail.com>> wrote:
Hi Steve,

It's helpful if you could share any timeline about when this will be done?

Thanks,
Wangda

On Mon, Nov 19, 2018 at 6:46 AM Steve Loughran 
mailto:ste...@hortonworks.com>> wrote:
I think I need to a quick rollback of something in S3A, for 3.0+; turns out 
that copyFromLocalFile() is also used to upload directories, and the 
high-performance one doesn't.

https://issues.apache.org/jira/browse/HADOOP-15932

The patch doesn't delete the new code, just takes it out of the execution path



> On 16 Nov 2018, at 18:14, Wangda Tan 
> mailto:wheele...@gmail.com>> wrote:
>
> Just cleaned up all tickets and updated tickets which have target version
> == 3.1.2 to 3.1.3.
>
> I will roll an RC next Monday, if you have any tickets need to be released
> in 3.1.2, please let me know.
>
> Thanks,
> Wangda
>
> On Wed, Oct 24, 2018 at 7:30 PM Vinod Kumar Vavilapalli 
> mailto:vino...@apache.org>>
> wrote:
>
>> 231 fixed JIRAs is already quite a bunch!
>>
>> I only see 7 JIRAs marked with Affects Version 3.1.2 and only one of them
>> as blocker.
>>
>> Why not just release now as soon as there are no blockers?
>>
>> Thanks
>> +Vinod
>>
>>> On Oct 24, 2018, at 4:36 PM, Wangda Tan 
>>> mailto:wheele...@gmail.com>> wrote:
>>>
>>> Hi, All
>>>
>>> We have released Apache Hadoop 3.1.1 on Aug 8, 2018. To further
>>> improve the quality of the release, I plan to release 3.1.2
>>> by Nov. The focus of 3.1.2 will be fixing blockers / critical bugs
>>> and other enhancements. So far there are 231 JIRAs [1] have fix
>>> version marked to 3.1.2
>>>
>>> I plan to cut branch-3.1 on Nov 15 and vote for RC on the same day.
>>>
>>> Please feel free to share your insights.
>>>
>>> Thanks,
>>> Wangda Tan
>>>
>>> [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop
>> Map/Reduce")
>>> AND fixVersion = 3.1.2
>>
>>

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

[Takanobu Asanuma]

Thanks for driving the release, Sunil!
 
+1(non-binding)
   - verified checksums
   - succeeded in building the package (CentOS7, Java 8u181)
   - started NN HA clusters
   - verified Web UI (NN, Router)
   - verified some operations of Erasure Coding
   - verified some operations of Router-based Federation
 
Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G" wrote:

Hi folks,



Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/



RC tag in git is release-3.2.0-RC0.



The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/


This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.



3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]


https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

[Zsolt Venczel]

Thanks Sunil for preparing this release!

+1 (non-binding)

- Built from source at tag 3.2.0-rc0 (Ubuntu 16.04, JDK1.8.0_191)
- Installed on a 4-node virtual cluster
- Executed teragen/terasort/teravalidate with and without EC (using
XOR-2-1-1024k policy)
- Created two snapshots and performed snapshot diff
- Checked web UIs (NN, DN, RM)
- Run unit tests for hadoop-hdfs (found one failure case of
https://issues.apache.org/jira/browse/HDFS-12116 that is not a regression)

Best regards,
Zsolt

On Thu, Nov 29, 2018 at 6:15 AM Wilfred Spiegelenburg
 wrote:

> +1 (non binding)
>
> - build from source on MacOSX 10.14.1, 1.8.0u181
> - successful native build on Ubuntu 16.04.3
> - confirmed the checksum and signature
> - deployed a single node cluster  (jdk 1.8u191 / centos 7.5)
>
> Wilfred
>
> > On 23 Nov 2018, at 23:06, Sunil G  wrote:
> >
> > Hi folks,
> >
> >
> >
> > Thanks to all contributors who helped in this release [1]. I have created
> >
> > first release candidate (RC0) for Apache Hadoop 3.2.0.
> >
> >
> > Artifacts for this RC are available here:
> >
> > http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/
> >
> >
> >
> > RC tag in git is release-3.2.0-RC0.
> >
> >
> >
> > The maven artifacts are available via repository.apache.org at
> >
> > https://repository.apache.org/content/repositories/orgapachehadoop-1174/
> >
> >
> > This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.
> >
> >
> >
> > 3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
> > additions
> >
> > are the highlights of this release.
> >
> > 1. Node Attributes Support in YARN
> >
> > 2. Hadoop Submarine project for running Deep Learning workloads on YARN
> >
> > 3. Support service upgrade via YARN Service API and CLI
> >
> > 4. HDFS Storage Policy Satisfier
> >
> > 5. Support Windows Azure Storage - Blob file system in Hadoop
> >
> > 6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a
> >
> > 7. Improvements in Router-based HDFS federation
> >
> >
> >
> > Thanks to Wangda, Vinod, Marton for helping me in preparing the release.
> >
> > I have done few testing with my pseudo cluster. My +1 to start.
> >
> >
> >
> > Regards,
> >
> > Sunil
> >
> >
> >
> > [1]
> >
> >
> https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E
> >
> > [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
> > AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
> > ORDER BY fixVersion ASC
>
>
> Wilfred Spiegelenburg | Software Engineer
> cloudera.com 
>
>
>
>
>
>
>
>