Re: [VOTE] Release Apache Hadoop 3.4.0 (RC3)

2024-03-14 Thread Takanobu Asanuma 
+1 (binding).

Thanks for the great work, Shilun Fan.

* Verified signatures and checksums
* Successfully built from source with native code
* Deployed a distributed cluster (on K8s)
* Successfully ran some Erasure Coding operations with ISA-L codec
* Successfully ran some HDFS RBF operations

Regards,
- Takanobu Asanuma

2024年3月14日(木) 15:19 Xiaoqiao He :

> Thanks Ayush for highlighting this information. Absolutely true, we should
> count RM's vote when explicit +1 here.
>
> Best Regards,
> - He Xiaoqiao
>
> On Thu, Mar 14, 2024 at 3:55 AM Ayush Saxena  wrote:
>
> > >  Counter should be with yourself vote, where the current summary
> > is 5 +1 binding and 1 +1 non-binding. Let's re-count when deadline.
> >
> > Just on the process: The release manager needs to "explicitly" vote like
> > any other before counting their own vote, there has been a lot of
> > discussions around that at multiple places & the official apache doc has
> > been updated as well [1], the last paragraph reads:
> >
> > "Note that there is no implicit +1 from the release manager, or from
> > anyone in any ASF vote. Only explicit votes are valid. The release
> manager
> > is encouraged to vote on releases, like any reviewer would do."
> >
> > So, do put an explicit +1, before you count yourself. Good Luck!!!
> >
> > -Ayush
> >
> > [1] https://www.apache.org/foundation/voting.html#ReleaseVotes
> >
> > On Tue, 12 Mar 2024 at 17:27, Steve Loughran  >
> > wrote:
> >
> >> followup: overnight work happy too.
> >>
> >> one interesting pain point is that on a raspberry pi 64 os checknative
> >> complains that libcrypto is missing
> >>
> >> > bin/hadoop checknative
> >>
> >> 2024-03-12 11:50:24,359 INFO bzip2.Bzip2Factory: Successfully loaded &
> >> initialized native-bzip2 library system-native
> >> 2024-03-12 11:50:24,363 INFO zlib.ZlibFactory: Successfully loaded &
> >> initialized native-zlib library
> >> 2024-03-12 11:50:24,370 WARN erasurecode.ErasureCodeNative: ISA-L
> support
> >> is not available in your platform... using builtin-java codec where
> >> applicable
> >> 2024-03-12 11:50:24,429 INFO nativeio.NativeIO: The native code was
> built
> >> without PMDK support.
> >> 2024-03-12 11:50:24,431 WARN crypto.OpensslCipher: Failed to load
> OpenSSL
> >> Cipher.
> >> java.lang.UnsatisfiedLinkError: Cannot load libcrypto.so (libcrypto.so:
> >> cannot open shared object file: No such file or directory)!
> >> at org.apache.hadoop.crypto.OpensslCipher.initIDs(Native Method)
> >> at
> >> org.apache.hadoop.crypto.OpensslCipher.(OpensslCipher.java:90)
> >> at
> >>
> >>
> org.apache.hadoop.util.NativeLibraryChecker.main(NativeLibraryChecker.java:111)
> >> Native library checking:
> >> hadoop:  true
> >>
> >>
> /home/stevel/Projects/hadoop-release-support/target/arm-untar/hadoop-3.4.0/lib/native/libhadoop.so.1.0.0
> >> zlib:true /lib/aarch64-linux-gnu/libz.so.1
> >> zstd  :  true /lib/aarch64-linux-gnu/libzstd.so.1
> >> bzip2:   true /lib/aarch64-linux-gnu/libbz2.so.1
> >> openssl: false Cannot load libcrypto.so (libcrypto.so: cannot open
> shared
> >> object file: No such file or directory)!
> >> ISA-L:   false libhadoop was built without ISA-L support
> >> PMDK:false The native code was built without PMDK support.
> >>
> >> which happens because its not in /lib/aarch64-linux-gnu but instead in
> >> /usr/lib/aarch64-linux-gnu/l
> >> ls -l /usr/lib/aarch64-linux-gnu/libcrypto*
> >> -rw-r--r-- 1 root root 2739952 Sep 19 13:09
> >> /usr/lib/aarch64-linux-gnu/libcrypto.so.1.1
> >> -rw-r--r-- 1 root root 4466856 Oct 27 13:40
> >> /usr/lib/aarch64-linux-gnu/libcrypto.so.3
> >>
> >> Anyone got any insights on how I should set up this (debian-based) OS
> >> here?
> >> I know it's only a small box but with arm64 VMs becoming available in
> >> cloud
> >> infras, it'd be good to know if they are similar.
> >>
> >> Note: checknative itself is happy; but checknative -a will fail because
> of
> >> this -though it's an OS setup issue, nothing related to the hadoop
> >> binaries.
> >>
> >> steve
> >>
> >> On Tue, 12 Mar 2024 at 02:26, Xiaoqiao He 
> wrote:
> >>
> >> > Hi Shilun, Counter should be with yourself vote, where the current
> >> summary
> >>

Re: [VOTE] Release Apache Hadoop Thirdparty 1.2.0 (RC1)

2024-02-06 Thread Takanobu Asanuma 
+1 (binding).

* Verified signatures and checksums
* Reviewed the documents
* Successfully built from source with `mvn clean install`
* Successfully compiled Hadoop trunk and branch-3.4 with `mvn clean install
-DskipTests` using the thirdparty 1.2.0-RC1
* There are not any diffs between tag and src

Regards,
- Takanobu Asanuma

2024年2月6日(火) 11:03 Xiaoqiao He :

> cc @PJ Fanning  @Ayush Saxena  
> @Steve
> Loughran  @Takanobu Asanuma  @Shuyan
> Zhang  @inigo...@apache.org 
>
> On Mon, Feb 5, 2024 at 11:30 AM Xiaoqiao He  wrote:
>
>> +1(binding).
>>
>> I checked the following items:
>> - [X] Download links are valid.
>> - [X] Checksums and PGP signatures are valid.
>> - [X] LICENSE and NOTICE files are correct for the repository.
>> - [X] Source code artifacts have correct names matching the current
>> release.
>> - [X] All files have license headers if necessary.
>> - [X] Building is OK using `mvn clean install` on JDK_1.8.0_202.
>> - [X] Built Hadoop trunk successfully with updated thirdparty (include
>> update protobuf shaded path).
>> - [X] No difference between tag and release src tar.
>>
>> Good Luck!
>>
>> Best Regards,
>> - He Xiaoqiao
>>
>>
>> On Sun, Feb 4, 2024 at 10:29 PM slfan1989  wrote:
>>
>>> Hi folks,
>>>
>>> Xiaoqiao He and I have put together a release candidate (RC1) for Hadoop
>>> Thirdparty 1.2.0.
>>>
>>> The RC is available at:
>>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.2.0-RC1
>>>
>>> The RC tag is
>>>
>>> https://github.com/apache/hadoop-thirdparty/releases/tag/release-1.2.0-RC1
>>>
>>> The maven artifacts are staged at
>>> https://repository.apache.org/content/repositories/orgapachehadoop-1401
>>>
>>> Comparing to 1.1.1, there are three additional fixes:
>>>
>>> HADOOP-18197. Upgrade Protobuf-Java to 3.21.12
>>> https://github.com/apache/hadoop-thirdparty/pull/26
>>>
>>> HADOOP-18921. Upgrade to avro 1.11.3
>>> https://github.com/apache/hadoop-thirdparty/pull/24
>>>
>>> HADOOP-18843. Guava version 32.0.1 bump to fix CVE-2023-2976 (#23)
>>> https://github.com/apache/hadoop-thirdparty/pull/23
>>>
>>> You can find my public key at :
>>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>>
>>> Best Regards,
>>> Shilun Fan.
>>>
>>>

Re: [VOTE] Release Apache Hadoop Thirdparty 1.2.0 RC0

2024-02-01 Thread Takanobu Asanuma 
It also looks good to me, except for the diff.

* Verified signatures and hashes
* Reviewed the documents
* Successfully built from source with `mvn clean install`
* Successfully compiled Hadoop trunk and branch-3.4 using the Hadoop
thirdparty 1.2.0

Anyway, since hadoop-thirdparty-1.1.1 has some high vulnerabilities,
hadoop-thirdparty-1.2.0 would be required for Hadoop-3.4.0.

Thanks,
- Takanobu

2024年2月2日(金) 4:45 slfan1989 :

> Thank you for helping to review Hadoop-Thirdparty-1.2.0-RC0 and providing
> feedback!
>
> I followed the "how to release" documentation and tried to package it using
> create-release and Dockerfile, but I couldn't successfully package it
> directly. Some modifications are required before compilation. I should
> submit a pull request to fix this issue before
> Hadoop-Thirdparty-1.2.0-RC0 compile.
>
> This is an area that needs improvement. We should ensure that the code of
> src is consistent with the tag.
>
> On Fri, Feb 2, 2024 at 2:25 AM Ayush Saxena  wrote:
>
> >
> > There is some diff b/w the git tag & the src tar, the Dockerfile & the
> > create-release are different, Why?
> >
> > Files hadoop-thirdparty/dev-support/bin/create-release and
> > hadoop-thirdparty-1.2.0-src/dev-support/bin/create-release differ
> >
> > Files hadoop-thirdparty/dev-support/docker/Dockerfile and
> > hadoop-thirdparty-1.2.0-src/dev-support/docker/Dockerfile differ
> >
> >
> > ayushsaxena@ayushsaxena hadoop-thirdparty-1.2.0-RC0 % diff
> > hadoop-thirdparty/dev-support/bin/create-release
> > hadoop-thirdparty-1.2.0-src/dev-support/bin/create-release
> >
> > 444,446c444,446
> >
> > < echo "RUN groupadd --non-unique -g ${group_id} ${user_name}"
> >
> > < echo "RUN useradd -g ${group_id} -u ${user_id} -m ${user_name}"
> >
> > < echo "RUN chown -R ${user_name} /home/${user_name}"
> >
> > ---
> >
> > > echo "RUN groupadd --non-unique -g ${group_id} ${user_name}; exit
> > 0;"
> >
> > > echo "RUN useradd -g ${group_id} -u ${user_id} -m ${user_name};
> > exit 0;"
> >
> > > echo "RUN chown -R ${user_name} /home/${user_name}; exit 0;"
> >
> > ayushsaxena@ayushsaxena hadoop-thirdparty-1.2.0-RC0 % diff
> > hadoop-thirdparty/dev-support/docker/Dockerfile
> > hadoop-thirdparty-1.2.0-src/dev-support/docker/Dockerfile
> >
> > 103a104,105
> >
> > > RUN rm -f /etc/maven/settings.xml && ln -s /home/root/.m2/settings.xml
> > /etc/maven/settings.xml
> >
> > >
> >
> > 126a129,130
> >
> > > RUN pip2 install setuptools-scm==5.0.2
> >
> > > RUN pip2 install lazy-object-proxy==1.5.0
> >
> > 159d162
> >
> > <
> >
> >
> >
> >
> > Other things look Ok,
> > * Built from source
> > * Verified Checksums
> > * Verified Signatures
> > * Validated files have ASF header
> >
> > Not sure if having diff b/w the git tag & src tar is ok, this doesn't
> look
> > like core code change though, can anybody check & confirm?
> >
> > -Ayush
> >
> >
> > On Thu, 1 Feb 2024 at 13:39, Xiaoqiao He  wrote:
> >
> >> Gentle ping. @Ayush Saxena  @Steve Loughran
> >>  @inigo...@apache.org 
> >> @Masatake
> >> Iwasaki  and some other folks.
> >>
> >> On Wed, Jan 31, 2024 at 10:17 AM slfan1989 
> wrote:
> >>
> >> > Thank you for the review and vote! Looking forward to other forks
> >> helping
> >> > with voting and verification.
> >> >
> >> > Best Regards,
> >> > Shilun Fan.
> >> >
> >> > On Tue, Jan 30, 2024 at 6:20 PM Xiaoqiao He 
> >> wrote:
> >> >
> >> > > Thanks Shilun for driving it and making it happen.
> >> > >
> >> > > +1(binding).
> >> > >
> >> > > [x] Checksums and PGP signatures are valid.
> >> > > [x] LICENSE files exist.
> >> > > [x] NOTICE is included.
> >> > > [x] Rat check is ok. `mvn clean apache-rat:check`
> >> > > [x] Built from source works well: `mvn clean install`
> >> > > [x] Built Hadoop trunk with updated thirdparty successfully (include
> >> > update
> >> > > protobuf shaded path).
> >> > >
> >> > > BTW, hadoop-thirdparty-1.2.0 will be included in release-3.4.0, hope
> >> we
> >> > > could finish this vote before 2024/02/06(UTC) if there are no
> >> concerns.
> >> > > Thanks all.
> >> > >
> >> > > Best Regards,
> >> > > - He Xiaoqiao
> >> > >
> >> > >
> >> > >
> >> > > On Mon, Jan 29, 2024 at 10:42 PM slfan1989 
> >> wrote:
> >> > >
> >> > > > Hi folks,
> >> > > >
> >> > > > Xiaoqiao He and I have put together a release candidate (RC0) for
> >> > Hadoop
> >> > > > Thirdparty 1.2.0.
> >> > > >
> >> > > > The RC is available at:
> >> > > >
> >> > >
> >> >
> >>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-thirdparty-1.2.0-RC0
> >> > > >
> >> > > > The RC tag is
> >> > > >
> >> > >
> >> >
> >>
> https://github.com/apache/hadoop-thirdparty/releases/tag/release-1.2.0-RC0
> >> > > >
> >> > > > The maven artifacts are staged at
> >> > > >
> >> >
> https://repository.apache.org/content/repositories/orgapachehadoop-1398
> >> > > >
> >> > > > Comparing to 1.1.1, there are three additional fixes:
> >> > > >
> >> > > > HADOOP-18197. Upgrade Protobuf-Java to 3.21.12
> >> > > >

Re: [INFO] Hadoop-3.4 Release Update

2024-01-20 Thread Takanobu Asanuma 
Hi Shilun,

Thank you for leading the 3.4.0 release.

> 2. In JIRA issues where other release versions have been released, remove
> the 3.4.0 entry from the fix version.

As I mentioned in my comment on HADOOP-18045, I think 3.4.0 should still be
included in the Fix versions,
even if it was already released in 3.3.x. If I understand correctly, this
is usually our practice.

Thanks,
- Takanobu

2024年1月21日(日) 13:56 slfan1989 :

> Hi All,
>
> I am currently preparing to release hadoop-3.4.0-RC1 and need to perform
> some operations on the JIRA issues related to HADOOP, HDFS, YARN, and
> MAPREDUCE. Specifically, the tasks include:
>
> 1. For JIRA issues with target version set to 3.4.0 and marked as
> non-blocker, update the target version to 3.5.0.
>
> 2. In JIRA issues where other release versions have been released, remove
> the 3.4.0 entry from the fix version.
>
> 3. For JIRA issues with target version set to 3.4.0, fix version set to
> 3.4.0, and status set to
> RESOLVED, if Affects Version/s, Component/s, and Hadoop Flags are not set,
> complete the information.
>
> Best Regards,
> - Shilun Fan.
>
> On Fri, Jan 19, 2024 at 3:23 PM Xiaoqiao He  wrote:
>
> > Hi All,
> >
> > Branch trunk has been set to 3.5.0-SNAPSHOT and branch-3.4 is created[1].
> >
> > Please set the proper fix version while committing jira, 3.5.0 should
> > be priority. If there is any blocker/critical please also backport to
> > branch-3.4 or sync to me or RM Shilun Fan.
> >
> > Shilun will start 3.4.0 RC1 voting in the next few days.
> >
> > Thanks.
> >
> > Best Regards,
> > - He Xiaoqiao
> >
> > [1] https://github.com/apache/hadoop/commits/branch-3.4
> >
>

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Takanobu Asanuma 
I'm now -1 as I found the wrong information on the top page (index.md).

> 1. HDFS-13522, HDFS-16767 & Related Jiras: Allow Observer Reads in HDFS
Router Based Federation.

The fix version of HDFS-13522 and HDFS-16767 also included 3.3.5 before,
though it is actually not in branch-3.3. I corrected the fix version and
created HDFS-16889 to backport them to branch-3.3 about a month ago.
Unfortunately, it won't be fixed soon. I should have let you know at that
time, sorry.  Supporting Observer NameNode in RBF is a prominent feature.
So I think we have to delete the description from the top page not to
confuse Hadoop users.

- Takanobu

2023年2月23日(木) 17:17 Takanobu Asanuma :

> Thanks for driving the release, Steve and Mukund.
>
> I found that there were some jiras with wrong fix versions.
>
> The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
> - HDFS-16845
> - HADOOP-18345
>
> The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
> (and it is not in release-3.3.4) :
> - HADOOP-17276
> - HDFS-13293
> - HDFS-15630
> - HDFS-16266
> - HADOOP-18003
> - HDFS-16310
> - HADOOP-18014
>
> I corrected all the wrong fix versions just now. I'm not sure we should
> revote it since it only affects the changelog.
>
> - Takanobu
>
> 2023年2月21日(火) 22:43 Steve Loughran :
>
>> Apache Hadoop 3.3.5
>>
>> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>>
>> What we would like is for anyone who can to verify the tarballs,
>> especially
>> anyone who can try the arm64 binaries as we want to include them too.
>>
>> The RC is available at:
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>>
>> The git tag is release-3.3.5-RC1, commit 274f91a3259
>>
>> The maven artifacts are staged at
>> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>>
>> You can find my public key at:
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>>
>> Change log
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>>
>> Release notes
>>
>> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>>
>> This is off branch-3.3 and is the first big release since 3.3.2.
>>
>> Key changes include
>>
>> * Big update of dependencies to try and keep those reports of
>>   transitive CVEs under control -both genuine and false positives.
>> * HDFS RBF enhancements
>> * Critical fix to ABFS input stream prefetching for correct reading.
>> * Vectored IO API for all FSDataInputStream implementations, with
>>   high-performance versions for file:// and s3a:// filesystems.
>>   file:// through java native io
>>   s3a:// parallel GET requests.
>> * This release includes Arm64 binaries. Please can anyone with
>>   compatible systems validate these.
>>
>> Note, because the arm64 binaries are built separately on a different
>> platform and JVM, their jar files may not match those of the x86
>> release -and therefore the maven artifacts. I don't think this is
>> an issue (the ASF actually releases source tarballs, the binaries are
>> there for help only, though with the maven repo that's a bit blurred).
>>
>> The only way to be consistent would actually untar the x86.tar.gz,
>> overwrite its binaries with the arm stuff, retar, sign and push out
>> for the vote. Even automating that would be risky.
>>
>> Please try the release and vote. The vote will run for 5 days.
>>
>> Steve and Mukund
>>
>

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-02-23 Thread Takanobu Asanuma 
Thanks for driving the release, Steve and Mukund.

I found that there were some jiras with wrong fix versions.

The fix versions included 3.3.5, but actually, it isn't in 3.3.5-RC1:
- HDFS-16845
- HADOOP-18345

The fix versions didn't include 3.3.5, but actually, it is in 3.3.5-RC1
(and it is not in release-3.3.4) :
- HADOOP-17276
- HDFS-13293
- HDFS-15630
- HDFS-16266
- HADOOP-18003
- HDFS-16310
- HADOOP-18014

I corrected all the wrong fix versions just now. I'm not sure we should
revote it since it only affects the changelog.

- Takanobu

2023年2月21日(火) 22:43 Steve Loughran :

> Apache Hadoop 3.3.5
>
> Mukund and I have put together a release candidate (RC1) for Hadoop 3.3.5.
>
> What we would like is for anyone who can to verify the tarballs, especially
> anyone who can try the arm64 binaries as we want to include them too.
>
> The RC is available at:
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/
>
> The git tag is release-3.3.5-RC1, commit 274f91a3259
>
> The maven artifacts are staged at
> https://repository.apache.org/content/repositories/orgapachehadoop-1368/
>
> You can find my public key at:
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
> Change log
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/CHANGELOG.md
>
> Release notes
>
> https://dist.apache.org/repos/dist/dev/hadoop/hadoop-3.3.5-RC1/RELEASENOTES.md
>
> This is off branch-3.3 and is the first big release since 3.3.2.
>
> Key changes include
>
> * Big update of dependencies to try and keep those reports of
>   transitive CVEs under control -both genuine and false positives.
> * HDFS RBF enhancements
> * Critical fix to ABFS input stream prefetching for correct reading.
> * Vectored IO API for all FSDataInputStream implementations, with
>   high-performance versions for file:// and s3a:// filesystems.
>   file:// through java native io
>   s3a:// parallel GET requests.
> * This release includes Arm64 binaries. Please can anyone with
>   compatible systems validate these.
>
> Note, because the arm64 binaries are built separately on a different
> platform and JVM, their jar files may not match those of the x86
> release -and therefore the maven artifacts. I don't think this is
> an issue (the ASF actually releases source tarballs, the binaries are
> there for help only, though with the maven repo that's a bit blurred).
>
> The only way to be consistent would actually untar the x86.tar.gz,
> overwrite its binaries with the arm stuff, retar, sign and push out
> for the vote. Even automating that would be risky.
>
> Please try the release and vote. The vote will run for 5 days.
>
> Steve and Mukund
>

Re: [DISCUSS] Checkin Hadoop code formatter

2021-09-12 Thread Takanobu Asanuma 
Thanks for the initiative, Viraj.
I agree to add the code formatter.

- Takanobu

2021年9月12日(日) 5:01 Ayush Saxena :

> Thanx Viraj for initiating, Makes sense to me to include a formmater
> inline with our checkstyle rules in the code, would make life simpler for
> all devs.
>
> -Ayush
>
> > On 12-Sep-2021, at 12:28 AM, Viraj Jasani  wrote:
> >
> > + common-...@hadoop.apache.org
> >
> > -- Forwarded message -
> > From: Viraj Jasani 
> > Date: Tue, Sep 7, 2021 at 6:18 PM
> > Subject: Checkin Hadoop code formatter
> > To: common-...@hadoop.apache.org 
> >
> >
> > It seems some recent new devs are not familiar with the common code
> > formatter that we use for our codebase.
> > While we already have Wiki page [1] for new contributors and it mentions:
> > "Code must be formatted according to Sun's conventions
> > <
> http://www.oracle.com/technetwork/java/javase/documentation/codeconvtoc-136057.html
> >"
> > but this Oracle's code conventions page is not being actively maintained
> > (no update has been received after 1999) and hence, I believe we should
> > check-in and maintain code formatter xmls for supported IDEs in our
> > codebase only (under dev-support) for all devs to be able to import it in
> > the respective IDE.
> > Keeping this in mind, I have created this PR 3387
> > . If you could please take a
> > look and if the PR receives sufficient +1s, we might want to update our
> > Wiki page to directly refer to our own codebase for code formatters that
> we
> > maintain. Thoughts?
> >
> >
> > 1.
> >
> https://cwiki.apache.org/confluence/display/HADOOP/How+To+Contribute#HowToContribute-MakingChanges
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-13 Thread Takanobu Asanuma 
+1.
 - Verified hashes
 - Confirmed native build on CentOS7
 - Started kerberized cluster (using docker)
 - Checked NN/RBF Web UI
 - Ran basic Erasure Coding shell commands

Thanks for the great work, Wei-Chiu.

- Takanobu

2021年6月13日(日) 3:25 Vinayakumar B :

> +1 (Binding)
>
> 1. Built from Tag.
> 2. Successful Native Build on Ubuntu 20.04
> 3. Verified Checksums
> 4. Deployed the docker cluster with 3 nodes
> 5. Ran sample MR Jobs
>
> -Vinay
>
>
> On Sat, Jun 12, 2021 at 6:40 PM Ayush Saxena  wrote:
>
> > +1,
> > Built from Source.
> > Successful Native Build on Ubuntu 20.04
> > Verified Checksums
> > Ran basic hdfs shell commands.
> > Ran simple MR jobs.
> > Browsed NN,DN,RM and NM UI.
> >
> > Thanx Wei-Chiu for driving the release.
> >
> > -Ayush
> >
> >
> > > On 12-Jun-2021, at 1:45 AM, epa...@apache.org wrote:
> > >
> > > +1 (binding)
> > > Eric
> > >
> > >
> > > On Tuesday, June 1, 2021, 5:29:49 AM CDT, Wei-Chiu Chuang <
> > weic...@apache.org> wrote:
> > >
> > > Hi community,
> > >
> > > This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All
> > blocker
> > > issues have been resolved [1] again.
> > >
> > > There are 2 additional issues resolved for RC3:
> > > * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after
> > > HADOOP-16878
> > > * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the
> > source
> > > and destination are the same
> > >
> > > There are 4 issues resolved for RC2:
> > > * HADOOP-17666. Update LICENSE for 3.3.1
> > > * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053)
> > > * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055)
> > > * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064)
> > >
> > > The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two
> extra
> > > fixes compared to hadoop-thirdparty 1.1.0:
> > > * HADOOP-17707. Remove jaeger document from site index.
> > > * HADOOP-17730. Add back error_prone
> > >
> > > *RC tag is release-3.3.1-RC3
> > > https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3
> > >
> > > *The RC3 artifacts are at*:
> > > https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/
> > > ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/
> > >
> > > *The maven artifacts are hosted here:*
> > >
> https://repository.apache.org/content/repositories/orgapachehadoop-1320/
> > >
> > > *My public key is available here:*
> > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> > >
> > >
> > > Things I've verified:
> > > * all blocker issues targeting 3.3.1 have been resolved.
> > > * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible.
> > > * LICENSE and NOTICE files checked
> > > * RELEASENOTES and CHANGELOG
> > > * rat check passed.
> > > * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests.
> > > * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests.
> > > * Extra: built 50 other open source projects on top of Hadoop 3.3.1
> RC2.
> > > Had to patch some of them due to commons-lang migration (Hadoop 3.2.0)
> > and
> > > dependency divergence. Issues are being identified but so far nothing
> > > blocker for Hadoop itself.
> > >
> > > Please try the release and vote. The vote will run for 5 days.
> > >
> > > My +1 to start,
> > >
> > > [1] https://issues.apache.org/jira/issues/?filter=12350491
> > > [2]
> > >
> >
> https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
>

[jira] [Resolved] (YARN-10805) Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project

2021-06-09 Thread Takanobu Asanuma (Jira) 


 [ 
https://issues.apache.org/jira/browse/YARN-10805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takanobu Asanuma resolved YARN-10805.
-
Fix Version/s: 3.4.0
   Resolution: Fixed

> Replace Guava Lists usage by Hadoop's own Lists in hadoop-yarn-project
> --
>
> Key: YARN-10805
> URL: https://issues.apache.org/jira/browse/YARN-10805
> Project: Hadoop YARN
>  Issue Type: Task
>Reporter: Viraj Jasani
>Assignee: Viraj Jasani
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.4.0
>
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Takanobu Asanuma 
+1

- Takanobu

2021年6月4日(金) 3:47 Ayush Saxena :

> +1
>
> -Ayush
>
> > On 03-Jun-2021, at 11:26 PM, hemanth boyina 
> wrote:
> >
> > +1
> >
> > Thanks
> > HemanthBoyina
> >
> >> On Thu, 3 Jun 2021, 20:33 Wanqiang Ji,  wrote:
> >>
> >> +1 (non-binding)
> >>
> >> Wanqiang Ji
> >>
> >>> On Thu, Jun 3, 2021 at 10:47 PM Sangjin Lee  wrote:
> >>>
> >>> +1
> >>>
> >>> On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
> >>> wrote:
> >>>
>  +1
> 
> > On Jun 3, 2021, at 1:14 AM, Akira Ajisaka 
> >> wrote:
> >
> > Dear Hadoop developers,
> >
> > Given the feedback from the discussion thread [1], I'd like to start
> > an official vote
> > thread for the community to vote and start the 3.1 EOL process.
> >
> > What this entails:
> >
> > (1) an official announcement that no further regular Hadoop 3.1.x
>  releases
> > will be made after 3.1.4.
> > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> >
> > This vote will run for 7 days and conclude by June 10th, 16:00 JST
> >> [2].
> >
> > Committers are eligible to cast binding votes. Non-committers are
>  welcomed
> > to cast non-binding votes.
> >
> > Here is my vote, +1
> >
> > [1] https://s.apache.org/w9ilb
> > [2]
> 
> >>>
> >>
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> >
> > Regards,
> > Akira
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> 
> 
> 
>  -
>  To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>  For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
> 
> >>>
> >>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [DISCUSS] Change project style guidelines to allow line length 100

2021-05-20 Thread Takanobu Asanuma 
+1 for 100 characters. Thanks.

2021年5月21日(金) 8:38 Chao Sun :

> +1. I think this will be very helpful.
>
> On Thu, May 20, 2021 at 3:10 PM Sangjin Lee  wrote:
>
> > +1 (binding). It's long overdue IMHO.
> >
> > On Thu, May 20, 2021 at 2:11 PM Gergely Pollak
> >  wrote:
> >
> > > I really like this initiative, thank you!
> > > +1 for line length increase to 100 characters.
> > >
> > > Regards,
> > >   Gergely Pollak
> > >
> > > On Thu, May 20, 2021 at 8:30 PM Vivek Ratnavel <
> vivekratna...@apache.org
> > >
> > > wrote:
> > >
> > > > +1 (non-binding) to increase line length to 100 characters. This will
> > > > definitely help clear most of the checkstyle violations.
> > > >
> > > > Thank you Sean for starting this thread!
> > > >
> > > > On Thu, May 20, 2021 at 7:17 AM Sean Busbey
>  > >
> > > > wrote:
> > > >
> > > > > Hi Bhavik!
> > > > >
> > > > > What concerns do you have about back porting patches to earlier
> > release
> > > > > branches?
> > > > >
> > > > > If we change our style guidelines then presumably we can do that
> for
> > > all
> > > > > branches, so a backport from e.g. trunk to branch-3.3 won’t fail a
> > > style
> > > > > check on the destination branch unless something changed in the
> > > > backporting.
> > > > >
> > > > > If you are referring to patches for clearing up line length
> > violations,
> > > > my
> > > > > usual preference is to aim for my changes to be on all active
> release
> > > > > lines. So at least in the case of the patches coming from me or
> being
> > > > > committed by me, there’d be effort to make sure all branches end up
> > as
> > > > easy
> > > > > to backport to as they were prior to the clean up.
> > > > >
> > > > >
> > > > >
> > > > > > On May 20, 2021, at 2:27 AM, Bhavik Patel <
> > bhavikpatel...@gmail.com>
> > > > > wrote:
> > > > > >
> > > > > > I am just worried about the backporting of the Jira to child
> > branch!!
> > > > How
> > > > > > we are planning to handle this?
> > > > > >
> > > > > > On Thu, May 20, 2021, 11:09 AM Qi Zhu <821684...@qq.com  > > > > 821684...@qq.com>> wrote:
> > > > > >
> > > > > >> +1 100 is reasonable.
> > > > > >>
> > > > > >>
> > > > > >>
> > > > > >> ---Original---
> > > > > >> From: "Xiaoqiao He" > > > hexiaoq...@apache.org
> > > > > >
> > > > > >> Date: Thu, May 20, 2021 13:35 PM
> > > > > >> To: "Masatake Iwasaki" > > > > iwasak...@oss.nttdata.co.jp>;
> > > > > >> Cc: "Akira Ajisaka" > aajis...@apache.org
> > > > >;"Hadoop
> > > > > Common"<
> > > > > >> common-...@hadoop.apache.org  common-...@hadoop.apache.org
> > > > > >;"Hdfs-dev" > > > > hdfs-...@hadoop.apache.org>
> > > > > >> ;"yarn-dev" > > > > yarn-dev@hadoop.apache.org>;"mapreduce-dev"<
> > > > > >> mapreduce-...@hadoop.apache.org  > > > mapreduce-...@hadoop.apache.org
> > > > > >;
> > > > > >> Subject: Re: [DISCUSS] Change project style guidelines to allow
> > line
> > > > > >> length 100
> > > > > >>
> > > > > >>
> > > > > >> +1 for <= 100 chars long per line length.
> > > > > >>
> > > > > >> On Thu, May 20, 2021 at 10:28 AM Masatake Iwasaki <
> > > > > >> iwasak...@oss.nttdata.co.jp wrote:
> > > > > >>
> > > > > >>  I'm +1 too.
> > > > > >>  I feel 80 characters limit tends to degrade readability by
> > > > > introducing
> > > > > >>  useless line breaks.
> > > > > >> 
> > > > > >>  
> > > > > >> 
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > <
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > >
> > > > > >> 
> > > > > >> <
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > <
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > >>
> > > > > >> ;
> > > > > >>  I have no inconvenience on 100 characters for using Emacs
> and
> > > > > >> side-by-side
> > > > > >>  diff even on 13-inch MBP.
> > > > > >> 
> > > > > >>  Masatake Iwasaki
> > > > > >> 
> > > > > >>  On 2021/05/20 11:00, Akira Ajisaka wrote:
> > > > > >>   I'm +1 to allow <= 100 chars.
> > > > > >>  
> > > > > >>   FYI: There were some discussions long before:
> > > > > >>   -
> > > > > >> 
> > > > > >>
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > <
> > > > >
> > > >
> > >
> >
> https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1dd4e%401406489966%40%3Ccommon-dev.hadoop.apache.org%3E
> > > > > >
> > > > > >> 
> > > > > >> <
> > > > >
> > > >
> > >
> >
>

[jira] [Created] (YARN-10407) Add phantomjsdriver.log to gitignore

2020-08-25 Thread Takanobu Asanuma (Jira) 
Takanobu Asanuma created YARN-10407:
---

 Summary: Add phantomjsdriver.log to gitignore
 Key: YARN-10407
 URL: https://issues.apache.org/jira/browse/YARN-10407
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


After testing hadoop-yarn-applications-catalog-webapp, it genrates 
phantomjsdriver.log.

{noformat}
$ mvn test --projects org.apache.hadoop:hadoop-yarn-applications-catalog-webapp
{noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0

2020-07-13 Thread Takanobu Asanuma 
+1(non-binding)
   - verified checksums
   - succeeded in building the package with OpenJDK 8
   - started HDFS cluster with kerberos with OpenJDK 11
   - verified Web UIs (NN, DN, Router)
   - Ran some operations of Router-based Federation with Security
   - Ran some operations of Erasure Coding

Thanks,
Takanobu


From: Bilwa S T 
Sent: Monday, July 13, 2020 2:23
To: Surendra Singh Lilhore; hemanth boyina
Cc: Brahma Reddy Battula; mapreduce-dev; Hdfs-dev; Hadoop Common; yarn-dev
Subject: RE: [VOTE] Release Apache Hadoop 3.3.0 - RC0

+1(non-binding)

1. Deployed 3 node cluster
2. Browsed through Web UI (RM, NM)
3. Executed Jobs (pi, wordcount, TeraGen, TeraSort)
4. Verified basic yarn commands

Thanks,
Bilwa

-Original Message-
From: Surendra Singh Lilhore [mailto:surendralilh...@gmail.com]
Sent: 12 July 2020 18:32
To: hemanth boyina 
Cc: Iñigo Goiri ; Vinayakumar B ; 
Brahma Reddy Battula ; mapreduce-dev 
; Hdfs-dev ; 
Hadoop Common ; yarn-dev 

Subject: Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0

+1(binding)

Deployed HDFS and Yarn Cluster
> Verified basic shell commands
> Ran some jobs
> Verified UI

-Surendra

On Sat, Jul 11, 2020 at 9:41 PM hemanth boyina 
wrote:

> +1(non-binding)
> Deployed Cluster with Namenodes and Router *)verified shell commands
> *)Executed various jobs *)Browsed UI's
>
>
> Thanks,
> HemanthBoyina
>
>
> On Sat, 11 Jul 2020, 00:05 Iñigo Goiri,  wrote:
>
> > +1 (Binding)
> >
> > Deployed a cluster on Azure VMs with:
> > * 3 VMs with HDFS Namenodes and Routers
> > * 2 VMs with YARN Resource Managers
> > * 5 VMs with HDFS Datanodes and Node Managers
> >
> > Tests:
> > * Executed Tergagen+Terasort+Teravalidate.
> > * Executed wordcount.
> > * Browsed through the Web UI.
> >
> >
> >
> > On Fri, Jul 10, 2020 at 1:06 AM Vinayakumar B
> > 
> > wrote:
> >
> > > +1 (Binding)
> > >
> > > -Verified all checksums and Signatures.
> > > -Verified site, Release notes and Change logs
> > >   + May be changelog and release notes could be grouped based on
> > > the project at second level for better look (this needs to be
> > > supported
> from
> > > yetus)
> > > -Tested in x86 local 3-node docker cluster.
> > >   + Built from source with OpenJdk 8 and Ubuntu 18.04
> > >   + Deployed 3 node docker cluster
> > >   + Ran various Jobs (wordcount, Terasort, Pi, etc)
> > >
> > > No Issues reported.
> > >
> > > -Vinay
> > >
> > > On Fri, Jul 10, 2020 at 1:19 PM Sheng Liu 
> > wrote:
> > >
> > > > +1 (non-binding)
> > > >
> > > > - checkout the "3.3.0-aarch64-RC0" binaries packages
> > > >
> > > > - started a clusters with 3 nodes VMs of Ubuntu 18.04
> > > > ARM/aarch64, openjdk-11-jdk
> > > >
> > > > - checked some web UIs (NN, DN, RM, NM)
> > > >
> > > > - Executed a wordcount, TeraGen, TeraSort and TeraValidate
> > > >
> > > > - Executed a TestDFSIO job
> > > >
> > > > - Executed a Pi job
> > > >
> > > > BR,
> > > > Liusheng
> > > >
> > > > Zhenyu Zheng  于2020年7月10日周五 下午3:45写道：
> > > >
> > > > > +1 (non-binding)
> > > > >
> > > > > - Verified all hashes and checksums
> > > > > - Tested on ARM platform for the following actions:
> > > > >   + Built from source on Ubuntu 18.04, OpenJDK 8
> > > > >   + Deployed a pseudo cluster
> > > > >   + Ran some example jobs(grep, wordcount, pi)
> > > > >   + Ran teragen/terasort/teravalidate
> > > > >   + Ran TestDFSIO job
> > > > >
> > > > > BR,
> > > > >
> > > > > Zhenyu
> > > > >
> > > > > On Fri, Jul 10, 2020 at 2:40 PM Akira Ajisaka
> > > > >  >
> > > > wrote:
> > > > >
> > > > > > +1 (binding)
> > > > > >
> > > > > > - Verified checksums and signatures.
> > > > > > - Built from the source with CentOS 7 and OpenJDK 8.
> > > > > > - Successfully upgraded HDFS to 3.3.0-RC0 in our development
> > cluster
> > > > > (with
> > > > > > RBF, security, and OpenJDK 11) for end-users. No issues reported.
> > > > > > - The document looks good.
> > > > > > - Deployed pseudo cluster and ran some MapReduce jobs.
> > > > > >
> > > > > > Thanks,
> > > > > > Akira
> > > > > >
> > > > > >
> > > > > > On Tue, Jul 7, 2020 at 7:27 AM Brahma Reddy Battula <
> > > bra...@apache.org
> > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > Hi folks,
> > > > > > >
> > > > > > > This is the first release candidate for the first release
> > > > > > > of
> > Apache
> > > > > > > Hadoop 3.3.0
> > > > > > > line.
> > > > > > >
> > > > > > > It contains *1644[1]* fixed jira issues since 3.2.1 which
> > include a
> > > > lot
> > > > > > of
> > > > > > > features and improvements(read the full set of release notes).
> > > > > > >
> > > > > > > Below feature additions are the highlights of the release.
> > > > > > >
> > > > > > > - ARM Support
> > > > > > > - Enhancements and new features on S3a,S3Guard,ABFS
> > > > > > > - Java 11 Runtime support and TLS 1.3.
> > > > > > > - Support Tencent Cloud COS File System.
> > > > > > > - Added security to HDFS Router.
> > > > > > > - Support non-volatile storage class memory(SCM) in HDFS
> > > > > > >

Re: [ANNOUNCE] New Apache Hadoop Committer - Siyao Meng

2020-03-23 Thread Takanobu Asanuma 
Congrats, Siyao!

Regards,
- Takanobu


From: Ayush Saxena 
Sent: Sunday, March 22, 2020 14:55
To: Zhankun Tang
Cc: Xiaoyu Yao; Hadoop Common; Hdfs-dev; mapreduce-dev; 
ozone-...@hadoop.apache.org; yarn-dev
Subject: Re: [ANNOUNCE] New Apache Hadoop Committer - Siyao Meng

Congratulations Siyao!!!

-Ayush

> On 21-Mar-2020, at 8:13 AM, Zhankun Tang  wrote:
>
> Welcome onboard. Siyao!
>
> BR,
> Zhankun
>
> Xiaoyu Yao 于2020年3月21日 周六上午1:24写道：
>
>> It's my pleasure to announce that Siyao Meng has been elected as committer
>> on the Apache Hadoop project recognizing his continued contributions to the
>> project.
>>
>> Please join me in congratulating him.
>>
>> Congratulations & Welcome aboard Siyao!
>>
>> Xiaoyu Yao
>> (On behalf of the Hadoop PMC)
>>

Re: [DISCUSS] Use of AssertJ for testing

2019-03-31 Thread Takanobu Asanuma 
Thanks Akira. +1 for your opinion. I will review YARN-8943 after this 
discussion has finished.

- Takanobu

on 2019/04/01 14:07, "Sree V" wrote:

My opinion:
- Upgrade to JUnit5.- Use an Assertion Framework. Preferably, AssertJ as 
JUnit5 user guide full of it.

Thank you./Sree 

On Sunday, March 31, 2019, 9:37:52 PM PDT, Akira Ajisaka 
 wrote:  
 
 Hi folks,

Now I'm going to upgrade the JUnit version from 4 to 5 for Java 11 support.
I wanted to start with the small module, so I uploaded a patch to upgrade
the API in hadoop-yarn-api module at first (YARN-8943), and in this JIRA,
Szilard Nemeth suggested using AssertJ with JUnit 5. (Thanks Szilard
for the suggestion!)

I think the JUnit upgrade and the use of AssertJ are separated, but
related tasks.
Therefore, I'd like to decide:
- Use AssertJ or not
- If we are going to use AssertJ, when to use AssertJ (before
upgrading JUnit or after?)

My opinion is:
- JUnit migration is required for Java 11, so upgrading JUnit as soon
as possible.
- After the migration, we may use AssertJ for existing tests.
- We may use AssertJ for new tests. (not must)

Any thoughts?

Thanks,
Akira

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Propose to start new Hadoop sub project "submarine"

2019-02-04 Thread Takanobu Asanuma 
+1 (non-binding). Thanks, Wangda!

- Takanobu

on 2019/02/02 7:24, "Wangda Tan" wrote:

Hi all,

According to positive feedbacks from the thread [1]

This is vote thread to start a new subproject named "hadoop-submarine"
which follows the release process already established for ozone.

The vote runs for usual 7 days, which ends at Feb 8th 5 PM PDT.

Thanks,
Wangda Tan

[1]

https://lists.apache.org/thread.html/f864461eb188bd12859d51b0098ec38942c4429aae7e4d001a633d96@%3Cyarn-dev.hadoop.apache.org%3E

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC1

2019-01-15 Thread Takanobu Asanuma 
Thanks for driving the release, Sunil.

+1(non-binding)
 - verified checksums
 - built package from source code
 - started RBF(HDFS) cluster
 - verified Web UI (NN, RM, Router)
 - verified some operations of Router-based Federation
 - verified some operations of Erasure Coding

Regards,
- Takanobu

on 2019/01/08 20:42, "Sunil G" wrote:

Hi folks,


Thanks to all of you who helped in this release [1] and for helping to vote
for RC0. I have created second release candidate (RC1) for Apache Hadoop
3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC1/


RC tag in git is release-3.2.0-RC1.



The maven artifacts are available via repository.apache.org at
https://repository.apache.org/content/repositories/orgapachehadoop-1178/


This vote will run 7 days (5 weekdays), ending on 14th Jan at 11:59 pm PST.



3.2.0 contains 1092 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]


https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 3.2.0 - RC0

2018-11-29 Thread Takanobu Asanuma 
Thanks for driving the release, Sunil!
 
+1(non-binding)
   - verified checksums
   - succeeded in building the package (CentOS7, Java 8u181)
   - started NN HA clusters
   - verified Web UI (NN, Router)
   - verified some operations of Erasure Coding
   - verified some operations of Router-based Federation
 
Regards,
-Takanobu

on 2018/11/23 21:07, "Sunil G" wrote:

Hi folks,



Thanks to all contributors who helped in this release [1]. I have created

first release candidate (RC0) for Apache Hadoop 3.2.0.


Artifacts for this RC are available here:

http://home.apache.org/~sunilg/hadoop-3.2.0-RC0/



RC tag in git is release-3.2.0-RC0.



The maven artifacts are available via repository.apache.org at

https://repository.apache.org/content/repositories/orgapachehadoop-1174/


This vote will run 7 days (5 weekdays), ending on Nov 30 at 11:59 pm PST.



3.2.0 contains 1079 [2] fixed JIRA issues since 3.1.0. Below feature
additions

are the highlights of this release.

1. Node Attributes Support in YARN

2. Hadoop Submarine project for running Deep Learning workloads on YARN

3. Support service upgrade via YARN Service API and CLI

4. HDFS Storage Policy Satisfier

5. Support Windows Azure Storage - Blob file system in Hadoop

6. Phase 3 improvements for S3Guard and Phase 5 improvements S3a

7. Improvements in Router-based HDFS federation



Thanks to Wangda, Vinod, Marton for helping me in preparing the release.

I have done few testing with my pseudo cluster. My +1 to start.



Regards,

Sunil



[1]


https://lists.apache.org/thread.html/68c1745dcb65602aecce6f7e6b7f0af3d974b1bf0048e7823e58b06f@%3Cyarn-dev.hadoop.apache.org%3E

[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0, 3.0.0, 3.0.0-beta1) AND status = Resolved
ORDER BY fixVersion ASC

Re: [VOTE] Release Apache Hadoop 2.9.2 (RC0)

2018-11-14 Thread Takanobu Asanuma 
Thanks for driving the release, Akira!
 
+1 (non-binding)
   - verified checksums
   - succeeded in building the package
   - started hadoop cluster with 1 master and 5 slaves
   - ran TeraGen/TeraSort
   - verified Web UI (NN, RM, JobHistory, Timeline)
   - verified some operations of Router-based Federation

Thanks,
-Takanobu

on 2018/11/14 10:02, "Akira Ajisaka" wrote:

Hi folks,

I have put together a release candidate (RC0) for Hadoop 2.9.2. It
includes 204 bug fixes and improvements since 2.9.1. [1]

The RC is available at http://home.apache.org/~aajisaka/hadoop-2.9.2-RC0/
Git signed tag is release-2.9.2-RC0 and the checksum is
826afbeae31ca687bc2f8471dc841b66ed2c6704
The maven artifacts are staged at
https://repository.apache.org/content/repositories/orgapachehadoop-1166/

You can find my public key at:
https://dist.apache.org/repos/dist/release/hadoop/common/KEYS

Please try the release and vote. The vote will run for 5 days.

[1] https://s.apache.org/2.9.2-fixed-jiras

Thanks,
Akira

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8941) Findbugs warnings in hadoop-yarn-csi

2018-10-24 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8941:
--

 Summary: Findbugs warnings in hadoop-yarn-csi
 Key: YARN-8941
 URL: https://issues.apache.org/jira/browse/YARN-8941
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Takanobu Asanuma






--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8939) Javadoc build fails in hadoop-yarn-csi

2018-10-23 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8939:
--

 Summary: Javadoc build fails in hadoop-yarn-csi
 Key: YARN-8939
 URL: https://issues.apache.org/jira/browse/YARN-8939
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


{noformat}
$ mvn javadoc:javadoc --projects hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi
...
[ERROR] 
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-csi/src/main/java/org/apache/hadoop/yarn/csi/client/CsiGrpcClient.java:92:
 error: exception not thrown: java.lang.InterruptedException
[ERROR]* @throws InterruptedException
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8919) Some tests fail due to NoClassDefFoundError for OperatorCreationException

2018-10-19 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8919:
--

 Summary: Some tests fail due to NoClassDefFoundError for 
OperatorCreationException
 Key: YARN-8919
 URL: https://issues.apache.org/jira/browse/YARN-8919
 Project: Hadoop YARN
  Issue Type: Bug
Reporter: Takanobu Asanuma


These tests still fail due to {{NoClassDefFoundError: 
org/bouncycastle/operator/OperatorCreationException}}. 

{noformat}
hadoop.streaming.TestFileArgs
hadoop.streaming.TestMultipleCachefiles
hadoop.streaming.TestStreamingBadRecords
hadoop.streaming.TestSymLink
hadoop.streaming.TestMultipleArchiveFiles
hadoop.mapred.gridmix.TestGridmixSubmission
hadoop.mapred.gridmix.TestDistCacheEmulation
hadoop.mapred.gridmix.TestLoadJob
hadoop.mapred.gridmix.TestSleepJob
hadoop.tools.TestDistCh
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8866) Fix a parsing error for crossdomain.xml

2018-10-10 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8866:
--

 Summary: Fix a parsing error for crossdomain.xml
 Key: YARN-8866
 URL: https://issues.apache.org/jira/browse/YARN-8866
 Project: Hadoop YARN
  Issue Type: Bug
  Components: build, yarn-ui-v2
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


[QBT|https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/] reports a 
parsing error for crossdomain.xml in hadoop-yarn-ui.
{noformat}
Parsing Error(s): 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml
 
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Resolved] (YARN-8420) Multithreaded correctness Warning occurs in AllocationFileLoaderService

2018-06-12 Thread Takanobu Asanuma (JIRA) 


 [ 
https://issues.apache.org/jira/browse/YARN-8420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Takanobu Asanuma resolved YARN-8420.

Resolution: Duplicate

> Multithreaded correctness Warning occurs in AllocationFileLoaderService
> ---
>
> Key: YARN-8420
> URL: https://issues.apache.org/jira/browse/YARN-8420
> Project: Hadoop YARN
>  Issue Type: Bug
>  Components: fairscheduler
>        Reporter: Takanobu Asanuma
>Priority: Critical
>
> This is reported by findbugs. See: 
> [https://builds.apache.org/job/PreCommit-YARN-Build/21007/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html]
> {noformat}
> Inconsistent synchronization of 
> org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadListener;
>  locked 75% of time
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8420) Multithreaded correctness Warning occurs in AllocationFileLoaderService

2018-06-12 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8420:
--

 Summary: Multithreaded correctness Warning occurs in 
AllocationFileLoaderService
 Key: YARN-8420
 URL: https://issues.apache.org/jira/browse/YARN-8420
 Project: Hadoop YARN
  Issue Type: Bug
  Components: fairscheduler
Reporter: Takanobu Asanuma


This is reported by findbugs. See: 
[https://builds.apache.org/job/PreCommit-YARN-Build/21007/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html]

{noformat}
Inconsistent synchronization of 
org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadListener;
 locked 75% of time
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8408) Upgrade common-langs version to 3.7 in HDDS and Ozone

2018-06-08 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8408:
--

 Summary: Upgrade common-langs version to 3.7 in HDDS and Ozone
 Key: YARN-8408
 URL: https://issues.apache.org/jira/browse/YARN-8408
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


commons-lang 2.6 is widely used. Let's upgrade to 3.6.

This jira is separated from HADOOP-10783.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8389) Improve the description of machine-list property in Federation docs

2018-06-04 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8389:
--

 Summary: Improve the description of machine-list property in 
Federation docs
 Key: YARN-8389
 URL: https://issues.apache.org/jira/browse/YARN-8389
 Project: Hadoop YARN
  Issue Type: Improvement
  Components: documentation, federation
Affects Versions: 3.2.0, 3.1.1
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


The current example and the description seem to be a bit ambiguous.

http://hadoop.apache.org/docs/r3.1.0/hadoop-yarn/hadoop-yarn-site/Federation.html#Optional:



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8377) Javadoc build failed in hadoop-yarn-server-nodemanager

2018-05-29 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8377:
--

 Summary: Javadoc build failed in hadoop-yarn-server-nodemanager
 Key: YARN-8377
 URL: https://issues.apache.org/jira/browse/YARN-8377
 Project: Hadoop YARN
  Issue Type: Bug
  Components: build, docs
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


This is the same cause as YARN-8369.

{code}
[ERROR] 
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/container/SlidingWindowRetryPolicy.java:88:
 error: bad use of '>'
[ERROR]* When failuresValidityInterval is > 0, it also removes time entries 
from
[ERROR]   ^
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8369) Javadoc build failed due to "bad use of '>'"

2018-05-25 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8369:
--

 Summary: Javadoc build failed due to "bad use of '>'"
 Key: YARN-8369
 URL: https://issues.apache.org/jira/browse/YARN-8369
 Project: Hadoop YARN
  Issue Type: Bug
  Components: docs
    Reporter: Takanobu Asanuma
    Assignee: Takanobu Asanuma


{noformat}
$ mvn javadoc:javadoc --projects 
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common
...
[ERROR] 
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceCalculator.java:263:
 error: bad use of '>'
[ERROR]* included) has a >0 value.
[ERROR]  ^
[ERROR] 
/hadoop/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/util/resource/ResourceCalculator.java:266:
 error: bad use of '>'
[ERROR]* @return returns true if any resource is >0
[ERROR]  ^
{noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8363) Upgrade common-langs version to 3.7 in hadoop-yarn-project

2018-05-25 Thread Takanobu Asanuma (JIRA) 
Takanobu Asanuma created YARN-8363:
--

 Summary: Upgrade common-langs version to 3.7 in hadoop-yarn-project
 Key: YARN-8363
 URL: https://issues.apache.org/jira/browse/YARN-8363
 Project: Hadoop YARN
  Issue Type: Improvement
Reporter: Takanobu Asanuma
Assignee: Takanobu Asanuma


commons-lang 2.6 is widely used. Let's upgrade to 3.6.

This jira is separated from HADOOP-10783.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org