Re: [VOTE] Release Apache Hadoop 3.4.0 (RC3)

2024-03-08 Thread Masatake Iwasaki
WARN crypto.OpensslCipher: Failed to load OpenSSL Cipher. java.lang.UnsatisfiedLinkError: Cannot find AES-CTR/SM4-CTR support, is your version of Openssl new enough? at org.apache.hadoop.crypto.OpensslCipher.initIDs(Native Method) Masatake Iwasaki On 2024/03/05 7:07, slfan1989 wrote

Re: [DISCUSS] Support/Fate of HBase v1 in Hadoop

2024-03-08 Thread Masatake Iwasaki
+1 on supporting only HBase v2. Masatake Iwasaki On 2024/03/06 13:18, slfan1989 wrote: +1 Option2 I also agree with the idea of upgrading hbase 2.2 to 2.5. Shilun Fan. Hbase v1 is EOL for a while now, so option 2 probably makes sense. While you are at it you should probably update

Re: [INFO] Hadoop-3.4 Release Update

2024-01-21 Thread Masatake Iwasaki
> As I mentioned in my comment on HADOOP-18045, I think 3.4.0 should still be > included in the Fix versions, > even if it was already released in 3.3.x. If I understand correctly, this > is usually our practice. +1. We can not assume 3.4.x contains all fixes of 3.3.y since we are maintaining

Re: [VOTE] Release Apache Hadoop 3.4.0 RC0

2024-01-10 Thread Masatake Iwasaki
. In addition, the description about new features of previous release should be removed from the index.md.vm. [1] https://github.com/apache/hadoop/blob/branch-3.4.0/hadoop-project/src/site/markdown/index.md.vm Masatake Iwasaki On 2024/01/11 14:15, slfan1989 wrote: Hello all, We plan to release

Re: [VOTE] Hadoop 3.2.x EOL

2023-12-06 Thread Masatake Iwasaki
+1 On Wed, Dec 6, 2023 at 1:10 PM Xiaoqiao He wrote: > > Dear Hadoop devs, > > Given the feedback from the discussion thread [1], I'd like to start > an official thread for the community to vote on release line 3.2 EOL. > > It will include, > a. An official announcement informs no further

Re: [DISCUSS] Make some release lines EOL

2023-12-05 Thread Masatake Iwasaki
+1 on marking 3.2 EOL. For branch-2.10, just keeping cherry-picking critical our CVEs without release sounds good. If someone really want, I can try to put 2.10.3 up. dev-support/bin/create-release looks still working with some trivial fixes. On Tue, Dec 5, 2023 at 7:45 PM Steve Loughran wrote:

Re: [VOTE] Release Apache Hadoop 3.3.6 RC1

2023-06-20 Thread Masatake Iwasaki
and read files via httpfs. + built RPM packages by Bigtop (modified to use ZooKeepr 3.6) on Rocky Linux 8 (x86_64). + built HBase and Hive against Hadoop 3.3.6. + ran smoke-tests of hdfs, yarn, mapreduce, hbase and hive. + skimmed the contents of site documentation. Thanks, Masatake Iwasaki

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

2023-06-19 Thread Masatake Iwasaki
[1] https://github.com/apache/hadoop/pull/3241/files#diff-b273546d6f060e617553eaa49da69039d2c655a77d42022779c2281d0f6cd08eR135 [2] https://github.com/apache/hadoop/pull/3241#issuecomment-889185103 On Mon, 19 Jun 2023 at 06:44, Masatake Iwasaki wrote: I got compilation error against ZooKeeper 3.5 due

Re: [VOTE] Release Apache Hadoop 3.3.6 RC0

2023-06-18 Thread Masatake Iwasaki
code is the cause. Thanks, Masatake Iwasaki On 2023/06/18 4:57, Wei-Chiu Chuang wrote: I was going to do another RC in case something comes up. But it looks like the only thing that needs to be fixed is the Changelog. 1. HADOOP-18596 <https://issues.apache.org/jira/browse/HADOOP-18

Re: [VOTE] Release Apache Hadoop 3.3.5 (RC3)

2023-03-19 Thread Masatake Iwasaki
and skimmed the contents. + Javadocs are contained. Thanks, Masatake Iwasaki On 2023/03/16 4:47, Steve Loughran wrote: Apache Hadoop 3.3.5 Mukund and I have put together a release candidate (RC3) for Hadoop 3.3.5. What we would like is for anyone who can to verify the tarballs, especially anyone who

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-01-02 Thread Masatake Iwasaki
elp 1] ... On 2023/01/02 22:02, Masatake Iwasaki wrote: Thanks for your great effort for the new release, Steve and Mukund. +1 while it would be nice if we can address missed Javadocs. + verified the signature and checksum. + built from source tarball on Rocky Linux 8 and OpenJDK 8 with nat

Re: [VOTE] Release Apache Hadoop 3.3.5

2023-01-02 Thread Masatake Iwasaki
3.1.3 against 3.3.5 failed due to dependency change. # while building HBase 2.4.13 and Hive 3.1.3 against Hadoop 3.3.4 worked. + skimmed the site contents. - Javadocs are not contained (under r3.3.5/api). # The issue can be reproduced even if I built site docs from the source. Masatake

[jira] [Resolved] (HDFS-16625) Unit tests aren't checking for PMDK availability

2022-08-23 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-16625. - Hadoop Flags: Reviewed Resolution: Fixed > Unit tests aren't checking for P

Re: [VOTE] Release Apache Hadoop 3.3.4

2022-07-31 Thread Masatake Iwasaki
://github.com/apache/bigtop/pull/964 * HBsae needed to fix relocation rules due to dependency on okhttp added by HDFS-16453. * built Spark 3.3.0 using the staging repository. Thanks, Masatake Iwasaki On 2022/07/30 3:47, Steve Loughran wrote: I have put together a release candidate (RC1) for Hadoop

[ANNOUNCE] Apache Hadoop 3.2.4 release

2022-07-21 Thread Masatake Iwasaki
nd testing reviewing the final artifacts. Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-21 Thread Masatake Iwasaki
The vote passed with the following result. Binding PMC members: +1 Chris Nauroth +1 Ayush Saxena +1 Masatake Iwasaki Non binding votes: +1 Ashutosh Gupta I'm going to publish the artifacts, site docs, maven artifacts, then announce. Thank you to all who helped to push this release out

Re: [VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-20 Thread Masatake Iwasaki
Hi developers, I'm still waiting for your vote. I'm considering the intermittent test failures mentioned by Chris are not blocker. Please file a JIRA and let me know if you find a blocker issue. I will appreciate your help for the release process. Regards, Masatake Iwasaki On 2022/07/20 14

Re: [VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-19 Thread Masatake Iwasaki
ails. YARN 8867 did not modified other code under hadoop-yarn-services. If it is the case, TestServiceAM can be tagged as flaky in branch-3.2. On 2022/07/20 14:21, Masatake Iwasaki wrote: Thanks for testing the RC0, Chris. The following are new test failures for me on 3.2.4: * TestAMRMProxy

Re: [VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-19 Thread Masatake Iwasaki
ailures. I don't know if it's a quirk to my environment, though I'm using the start-build-env.sh Docker container, so any build dependencies should be consistent. I'd be comfortable moving ahead if others are seeing these tests pass. Chris Nauroth On Thu, Jul 14, 2022 at 7:57 AM Masatake Iwasaki w

Re: [VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-14 Thread Masatake Iwasaki
/content/repositories/orgapachehadoop-1354 true true Thanks, Masatake Iwasaki On 2022/07/13 1:14, Masatake Iwasaki wrote: Hi all, Here's Hadoop 3.2.4 release candidate #0: The RC is available at:   https://home.apache.org/~iwasakims/hadoop

[VOTE] Release Apache Hadoop 3.2.4 - RC0

2022-07-12 Thread Masatake Iwasaki
/repositories/orgapachehadoop-1354 You can find my public key at: https://downloads.apache.org/hadoop/common/KEYS Please evaluate the RC and vote. The vote will be open for (at least) 5 days. Thanks, Masatake Iwasaki

Re: [DISCUSS] Hadoop 3.2.4 release

2022-07-11 Thread Masatake Iwasaki
Hi committers, I created branch-3.2.4. Please set the value of "Fix versions/s" field to 3.2.5 on JIRA issue if the fix is committed to branch-3.2 and not cherry-picked to branch-3.2.4. Thanks, Masatake Iwasaki On 2022/07/11 13:21, Akira Ajisaka wrote: Thank you Masatake for

Re: [DISCUSS] Hadoop 3.2.4 release

2022-07-10 Thread Masatake Iwasaki
I'm going to cut branch-3.2.4 today. If we need more time for 3.3.4 to fix the issue around HADOOP-18033, 3.2.4 can be released first. Thanks, Masatake Iwasaki On 2022/05/10 0:02, Masatake Iwasaki wrote: Hi team, Shaded client artifacts (hadoop-client-api and hadoop-client-runtime) of Hadoop

[ANNOUNCE] Apache Hadoop 2.10.2 release

2022-05-31 Thread Masatake Iwasaki
e building and testing reviewing the final artifacts. Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 2.10.2 - RC0

2022-05-30 Thread Masatake Iwasaki
The vote passed with the following result. Binding PMC members: +1 Chris Nauroth +1 Ayush Saxena +1 Steve Loughran +1 Masatake Iwasaki Non binding votes: +1 Viraj Jasani +1 Ashutosh Gupta I'm going to publish the artifacts, site docs, maven artifacts, then announce. Thank you to all who

Re: [VOTE] Release Apache Hadoop 2.10.2 - RC0

2022-05-29 Thread Masatake Iwasaki
Thanks for the help, Ayush. I committed HADOOP-16663/HADOOP-16664 and cherry-picked HADOOP-16985 to branch-2.10 (and branch-3.2). If I need to cut RC1, I will try cherry-picking them to branch-2.10.2 Masatake Iwasaki On 2022/05/28 5:23, Ayush Saxena wrote: The checksum stuff was addressed

Re: [VOTE] Release Apache Hadoop 2.10.2 - RC0

2022-05-27 Thread Masatake Iwasaki
2.3.6 and Spark 2.4.5 against Hadoop 2.10.2 RC0 on CentOS 7 (x86_64) by using Bigtop branch-1.5 and ran smoke-tests. https://github.com/apache/bigtop/pull/906 Masatake Iwasaki On 2022/05/25 11:40, Masatake Iwasaki wrote: Hi all, Here's Hadoop 2.10.2 release candidate #0: The RC is available

[VOTE] Release Apache Hadoop 2.10.2 - RC0

2022-05-24 Thread Masatake Iwasaki
/repositories/orgapachehadoop-1350 You can find my public key at: https://downloads.apache.org/hadoop/common/KEYS Please evaluate the RC and vote. The vote will be open for (at least) 5 days. Thanks, Masatake Iwasaki

Re: [DISCUSS] Hadoop 2.10.2 release

2022-05-16 Thread Masatake Iwasaki
Hi committers, I created branch-2.10.2 and bumped the version of branch-2.10 to 2.10.3-SNAPSHOT. Please set the "Fix Version/s:" of JIRA to 2.10.3 if you commit the fix to branch-2.10 without cherry-picking to branch-2.10.2. Regards, Masatake Iwasaki On 2022/05/16 23:16, Masata

Re: [DISCUSS] Hadoop 2.10.2 release

2022-05-16 Thread Masatake Iwasaki
Thanks, Akira and Xiaoqiao for positive feedback. I'm going to cut branch-2.10.2 shortly. Masatake Iwasaki On 2022/05/12 11:38, Xiaoqiao He wrote: Thanks Masatake and Akira for bringing this release up. +1 for 2.10.2 release. Sorry for the late response. Regards, - He Xiaoqiao On Mon, Mar

Re: [VOTE] Release Apache Hadoop 3.3.3 (RC1)

2022-05-15 Thread Masatake Iwasaki
enabled 3-nodes docker cluster and ran sample MR jobs. * built site documentation by `mvn site site:stage -Preleasedocs` and skimmed the contents. * built Spark 3.2.1 against 3.3.3 RC1 using the staging repository (after `rm -rf ~/.m2/repository/org/apache/hadoop`). Thanks, Masatake Iwasaki On 2022

[DISCUSS] Hadoop 3.2.4 release

2022-05-09 Thread Masatake Iwasaki
%20AND%20fixVersion%20%3D%203.2.4 Thanks, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-09 Thread Masatake Iwasaki
Step #10 in "Creating the release candidate (X.Y.Z-RC)" section of the Wiki still instructs to run `mvn deploy` with `-DskipShade`. https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease#HowToRelease-Creatingthereleasecandidate(X.Y.Z-RC%3CN%3E) On 2022/05/09 23:18, Masata

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-09 Thread Masatake Iwasaki
Wiki still instructs to run `mvn deploy` with `-DskipShade`. 2 sets of artifact are deployed after creating RC based on the instruction. The latest one contains empty shaded jars. hadoop-client-api and hadoop-client-runtime of already released 3.2.3 looks having same issue... Masatake Iwasaki On

Re: [VOTE] Release Apache Hadoop 3.3.3

2022-05-06 Thread Masatake Iwasaki
enabled 3-nodes docker cluster and ran sample MR jobs. * launched pseudo distributed cluster and `spark-shell --master yarn` with spark-3.2.1-bin-without-hadoop and ran some tutorial code. * built site documentation by `mvn site site:stage -Preleasedocs` and skimmed the contents. Thanks, Masatake

[ANNOUNCE] Apache Hadoop 3.2.3 release

2022-03-28 Thread Masatake Iwasaki
/3.2.3/CHANGELOG.3.2.3.html Best Regards, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

[RESULT][VOTE] Release Apache Hadoop 3.2.3 - RC1

2022-03-27 Thread Masatake Iwasaki
With 3 binding +1s and 1 non-binding +1, the vote passes. I'm going to publish the release and the announcement. Thanks, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC1

2022-03-27 Thread Masatake Iwasaki
3.1.2 and Spark 3.1.2 against Hadoop 3.2.3 RC1 on Rocky Linux 8 (x86_64) by using Bigtop and ran smoke-tests. https://issues.apache.org/jira/browse/BIGTOP-3606 Masatake Iwasaki On 2022/03/20 14:32, Masatake Iwasaki wrote: Hi all, Here's Hadoop 3.2.3 release candidate #1: The RC is available

[VOTE] Release Apache Hadoop 3.2.3 - RC1

2022-03-19 Thread Masatake Iwasaki
/repositories/orgapachehadoop-1342 You can find my public key at: https://downloads.apache.org/hadoop/common/KEYS Please evaluate the RC and vote. The vote will be open for (at least) 5 days. Thanks, Masatake Iwasaki

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

2022-03-18 Thread Masatake Iwasaki
, Masatake Iwasaki On 2022/03/18 10:56, Masatake Iwasaki wrote: I found that INFO level log message is emitted on every CLI invocation. $ bin/hdfs dfs -mkdir /tmp 2022-03-17 23:38:55,141 INFO Configuration.deprecation: No unit for dfs.client.datanode-restart.timeout(30) assuming SECONDS

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

2022-03-17 Thread Masatake Iwasaki
-default.xml by DFS-12920, There seems to be no quick fix though it is noisy. HDFS-14630 is the fix for this. I will backport this to branch-3.2.3 if I need to cut RC1. On 2022/03/18 0:54, Masatake Iwasaki wrote: +1 from myself. * built the source tarball on CentOS 7 (x86_64) and OpenJDK 8 by `mvn install

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

2022-03-17 Thread Masatake Iwasaki
che/hadoop/pull/4073>, few tests are flaky but they are passing locally. On Mon, Mar 14, 2022 at 12:45 PM Masatake Iwasaki mailto:iwasak...@oss.nttdata.co.jp>> wrote: Hi all, Here's Hadoop 3.2.3 release candidate #0: The RC is available at: https://home.apache.org/~iwas

Re: [VOTE] Release Apache Hadoop 3.2.3 - RC0

2022-03-17 Thread Masatake Iwasaki
Ejbyfeldt wrote: Hi, There is no aarch64 artifact in the release candidate. Is this something that is intended? Best, Emil Ejbyfeldt On 14/03/2022 08:14, Masatake Iwasaki wrote: Hi all, Here's Hadoop 3.2.3 release candidate #0: The RC is available at: https://home.apache.org/~iwasakims

[VOTE] Release Apache Hadoop 3.2.3 - RC0

2022-03-14 Thread Masatake Iwasaki
/repositories/orgapachehadoop-1339 You can find my public key at: https://downloads.apache.org/hadoop/common/KEYS Please evaluate the RC and vote. Thanks, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr

Re: Hadoop-3.2.3 Release Update

2022-03-13 Thread Masatake Iwasaki
in Proxy User documentation. (#2780) Signed-off-by: Akira Ajisaka (cherry picked from commit c8d327a4f1a7f15d6be35051414199d1d3fdc5ef) I will create RC0 shortly after updating index.md.vm (which is the top page of site documentation). Thanks, Masatake Iwasaki On 2022/03/11 22:17

Re: Hadoop-3.2.3 Release Update

2022-03-10 Thread Masatake Iwasaki
if we need. https://issues.apache.org/jira/browse/HADOOP-18088 Thanks, Masatake Iwasaki On 2022/03/10 12:23, Xiaoqiao He wrote: Thanks Masatake for reaching out. Not sure if Brahma has bandwidth to push this release forward. If there is no more bandwidth, I think we should involve other volunteers

[DISCUSS] Hadoop 2.10.2 release

2022-03-04 Thread Masatake Iwasaki
/browse/HADOOP-18088 Thanks, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: Hadoop-3.2.3 Release Update

2022-03-03 Thread Masatake Iwasaki
Hi Brahma, I would like to use 3.2.3 for next minor release of Bigtop. https://issues.apache.org/jira/browse/BIGTOP-3605 Let me know if there is any blocker I can help. Thanks, Masatake Iwasaki On 2022/01/12 4:41, Brahma Reddy Battula wrote: Sorry, I was looking for some CVE’s fixes.will

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC5

2022-02-25 Thread Masatake Iwasaki
enabled 3-nodes docker cluster and ran sample MR jobs. * launched pseudo distributed cluster and `spark-shell --master yarn` with spark-3.2.1-bin-without-hadoop and ran some tutorial code. * built site documentation by `mvn site site:stage` and skimmed the contents. Thanks, Masatake Iwasaki On 2022/02

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC3

2022-01-28 Thread Masatake Iwasaki
should be discussed separately. Thanks and regards, Akira On Fri, Jan 28, 2022 at 1:39 PM Masatake Iwasaki mailto:iwasak...@oss.nttdata.co.jp>> wrote: Thanks for putting this up, Chao Sun. I got following error on building the RC3 source tarball. It is reproducible even in the contai

Re: [VOTE] Release Apache Hadoop 3.3.2 - RC3

2022-01-27 Thread Masatake Iwasaki
this command. [INFO] error Found incompatible module [INFO] ---- ``` Masatake Iwasaki On 2022/01/27 4:16, Chao Sun wrote: Hi all, I've put together Hadoop 3.3.2 RC3 below: The RC is available at: http://people.apache.org/~sun

[jira] [Created] (HDFS-16354) Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc

2021-11-24 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-16354: --- Summary: Add description of GETSNAPSHOTDIFFLISTING to WebHDFS doc Key: HDFS-16354 URL: https://issues.apache.org/jira/browse/HDFS-16354 Project: Hadoop HDFS

Re: [DISCUSS] Add remote port information to HDFS audit log

2021-10-11 Thread Masatake Iwasaki
it is optional feature disabled by default. https://github.com/apache/hadoop/blob/rel/release-3.3.1/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java#L8480-L8498 If we need to add fields, making it optional might be an option. Masatake Iwasaki

[jira] [Created] (HDFS-16249) Avoid calling getSnapshotDiffReportListing multiple times if not supported

2021-10-01 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-16249: --- Summary: Avoid calling getSnapshotDiffReportListing multiple times if not supported Key: HDFS-16249 URL: https://issues.apache.org/jira/browse/HDFS-16249

[jira] [Created] (HDFS-16248) Add builder to SnapshotDiffReportListing

2021-10-01 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-16248: --- Summary: Add builder to SnapshotDiffReportListing Key: HDFS-16248 URL: https://issues.apache.org/jira/browse/HDFS-16248 Project: Hadoop HDFS Issue

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

2021-09-28 Thread Masatake Iwasaki
s Java 10 or upper. https://issues.apache.org/jira/browse/HADOOP-15304 Thanks, Akira On Tue, Sep 28, 2021 at 10:27 AM Masatake Iwasaki < iwasak...@oss.nttdata.co.jp> wrote: In particular, there has been an outstanding problem with doclet support for filtering javadocs by annotation since JD

[jira] [Created] (HDFS-16240) Replace unshaded guava in HttpFSServerWebServer

2021-09-27 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-16240: --- Summary: Replace unshaded guava in HttpFSServerWebServer Key: HDFS-16240 URL: https://issues.apache.org/jira/browse/HDFS-16240 Project: Hadoop HDFS

Re: [DISCUSS] Migrate to Yetus Interface classification annotations

2021-09-27 Thread Masatake Iwasaki
In particular, there has been an outstanding problem with doclet support for filtering javadocs by annotation since JDK9 came out. Could you give me a pointer to relevant Yetus JIRA or ML thread? On 2021/09/28 1:17, Sean Busbey wrote: I think consolidating on a common library and tooling for

Re: Hadoop-3.2.3 Release Update

2021-08-16 Thread Masatake Iwasaki
the exclusion even for zooekeeper-3.4.13. I filed HADOOP-17849 for this. On 2021/08/16 12:02, Masatake Iwasaki wrote: Thanks for bringing this up, Akira. Let me explain some background. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs

Re: Hadoop-3.2.3 Release Update

2021-08-15 Thread Masatake Iwasaki
Thanks for bringing this up, Akira. Let me explain some background. - https://github.com/apache/bigtop/blob/master/bigtop-packages/src/common/hadoop/patch2-exclude-spotbugs-annotations.diff This is for building hadoop-3.2.2 against zookeeper-3.4.14. we do not see the issue usually since

Re: [DISCUSS] Tips for improving productivity, workflow in the Hadoop project?

2021-07-14 Thread Masatake Iwasaki
- recently, JIRA became some sort of a "number generator" with insufficient description/details as the developers and the reviewers spending more time discussing in the PR. JIRA issues contain useful information in the fields. We are leveraging them in development and release process. *

[jira] [Created] (HDFS-16091) WebHDFS should support getSnapshotDiffReportListing

2021-06-26 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-16091: --- Summary: WebHDFS should support getSnapshotDiffReportListing Key: HDFS-16091 URL: https://issues.apache.org/jira/browse/HDFS-16091 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-10 Thread Masatake Iwasaki
and ran TestMiniLlapCliDrivera (fixed by HDFS-15790). Masatake Iwasaki On 2021/06/01 19:29, Wei-Chiu Chuang wrote: Hi community, This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker issues have been resolved [1] again. There are 2 additional issues resolved for RC3

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-07 Thread Masatake Iwasaki
issue. [1] https://issues.apache.org/jira/browse/HDFS-15098?focusedCommentId=17112893=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-17112893 Masatake Iwasaki On 2021/06/07 18:04, Masatake Iwasaki wrote: I found that the top page of site documentation (index.html

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-07 Thread Masatake Iwasaki
/) if you need to cut RC4. Thanks, Masatake Iwasaki On 2021/06/01 19:29, Wei-Chiu Chuang wrote: Hi community, This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker issues have been resolved [1] again. There are 2 additional issues resolved for RC3: * Revert "MAPREDUCE

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Masatake Iwasaki
+1 Masatake Iwasaki On 2021/06/03 15:14, Akira Ajisaka wrote: Dear Hadoop developers, Given the feedback from the discussion thread [1], I'd like to start an official vote thread for the community to vote and start the 3.1 EOL process. What this entails: (1) an official announcement

Re: Do I need a +1 to merge a backport PR?

2021-06-01 Thread Masatake Iwasaki
I think we usually merge backport without explicit +1 if the conflict is trivial, same as done on JIRA-only workflow (on which we attached patch like HADOOP-xxx-branch-3.3.001.patch for precommit check then commit it without explicit +1). If committers want review since the change is not

Re: [DISCUSS] Change project style guidelines to allow line length 100

2021-05-19 Thread Masatake Iwasaki
for using Emacs and side-by-side diff even on 13-inch MBP. Masatake Iwasaki On 2021/05/20 11:00, Akira Ajisaka wrote: I'm +1 to allow <= 100 chars. FYI: There were some discussions long before: - https://lists.apache.org/thread.html/7813c2f8a49b1d1e7655dad180f2d915a280b2f4d562cfe981e1d

[jira] [Resolved] (HDFS-15802) Address build failure of hadoop-hdfs-native-client on RHEL/CentOS 8

2021-01-31 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-15802. - Resolution: Duplicate > Address build failure of hadoop-hdfs-native-client on R

[jira] [Created] (HDFS-15802) Address build failure of hadoop-hdfs-native-client on RHEL/CentOS 8

2021-01-29 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-15802: --- Summary: Address build failure of hadoop-hdfs-native-client on RHEL/CentOS 8 Key: HDFS-15802 URL: https://issues.apache.org/jira/browse/HDFS-15802 Project

Re: [DISCUSS] HADOOP-17464. Create hadoop-compression module

2021-01-18 Thread Masatake Iwasaki
-compression a compile scope dependency of hadoop-common first then removing it on the next major release (4.0.0)? Thanks, Masatake Iwasaki On 2021/01/12 22:58, Steve Loughran wrote: Following on from discussions in a previous PR, Liang-Chi Hsieh has created a PR to add a new hadoop-compression module

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC5

2021-01-07 Thread Masatake Iwasaki
by `mvn site` and skimmed the contents including top-level index.html. * checked hadoop-3.2.2-site.tar.gz * verified signature and checksums. * skimmed the contents including top-level index.html. Thanks, Masatake Iwasaki On 2021/01/03 23:39, Xiaoqiao He wrote: Hi folks, The release

[jira] [Created] (HDFS-15743) Fix -Pdist build failure of hadoop-hdfs-native-client

2020-12-21 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-15743: --- Summary: Fix -Pdist build failure of hadoop-hdfs-native-client Key: HDFS-15743 URL: https://issues.apache.org/jira/browse/HDFS-15743 Project: Hadoop HDFS

Re: [VOTE] Release Apache Hadoop 3.2.2 - RC4

2020-12-16 Thread Masatake Iwasaki
w features and big improvements. The instruction might be a bit unclear and tend to be missed by release manager. # I missed this on 2.10.1 release.. Thanks, Masatake Iwasaki On 2020/12/10 2:00, Xiaoqiao He wrote: Hi folks, The release candidate (RC4) for Hadoop-3.2.2 is available now. There are

[jira] [Resolved] (HDFS-15670) Testcase TestBalancer#testBalancerWithPinnedBlocks always fails

2020-12-02 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15670?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-15670. - Resolution: Cannot Reproduce I'm closing this as not reproducible. I guess it should

[jira] [Created] (HDFS-15702) Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission

2020-11-30 Thread Masatake Iwasaki (Jira)
Masatake Iwasaki created HDFS-15702: --- Summary: Fix intermittent falilure of TestDecommission#testAllocAndIBRWhileDecommission Key: HDFS-15702 URL: https://issues.apache.org/jira/browse/HDFS-15702

[jira] [Resolved] (HDFS-15676) TestRouterRpcMultiDestination#testNamenodeMetrics fails on trunk

2020-11-30 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15676?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-15676. - Resolution: Duplicate I'm closing this as fixed by HDFS-15677. I will reopen

[jira] [Resolved] (HDFS-15296) TestBPOfferService#testMissBlocksWhenReregister is flaky

2020-11-23 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-15296. - Resolution: Duplicate I'm closing this as duplicate of HDFS-15654. > TestBPOfferServ

Re: Precommit job fails to write comments in the GitHub pull requests

2020-09-29 Thread Masatake Iwasaki
Yetus when JDK11 Javadoc issue is fixed. Regards, Masatake Iwasaki On 2020/09/29 2:55, Akira Ajisaka wrote: Apache Hadoop developers, After YETUS-994, the jenkins job updates the commit status instead of writing a comment to the pull request. It requires an OAuth token with write access

Re: [VOTE] Moving Ozone to a separated Apache project

2020-09-28 Thread Masatake Iwasaki
+1 Thanks, Masatake Iwasaki On 2020/09/25 14:59, Elek, Marton wrote: Hi all, Thank you for all the feedback and requests, As we discussed in the previous thread(s) [1], Ozone is proposed to be a separated Apache Top Level Project (TLP) The proposal with all the details, motivation

Re: [ANNOUNCE] Apache Hadoop 2.10.1 release

2020-09-22 Thread Masatake Iwasaki
and CHANGES have valid contents. Thanks, Masatake Iwasaki On 2020/09/23 5:17, Masatake Iwasaki wrote: Hi all, I am happy to announce that the Apache Hadoop 2.10.1 has been released. Apache Hadoop 2.10.1 is the second release in the Apache Hadoop 2.10 line. You can download the release from

[ANNOUNCE] Apache Hadoop 2.10.1 release

2020-09-22 Thread Masatake Iwasaki
, and changelog, are on the 2.10.1 announcement page [2]. [1] https://hadoop.apache.org/releases.html [2] https://hadoop.apache.org/release/2.10.1.html Thanks, Masatake Iwasaki - To unsubscribe, e-mail: hdfs-dev-unsubscr

Re: [VOTE] Release Apache Hadoop 2.10.1 (RC0)

2020-09-21 Thread Masatake Iwasaki
The vote passed with 6 binging +1's and 2 non-binding +1's. Thanks all for pushing this. I'm going to publish the release. binding +1's: - Eric Payne - Sunil Govindan - Wei-Chiu Chuang - Vinayakumar B - Mingliang Liu - Masatake Iwasaki non-binding +1's: - Jim Brennan - Eric Badger Bests

Re: [VOTE] Release Apache Hadoop 2.10.1 (RC0)

2020-09-20 Thread Masatake Iwasaki
will appreciate additional votes. Thanks, Masatake Iwasaki [1] https://lists.apache.org/thread.html/r16a7f36315a0673c7d522c41065e7ef9c9ee15c76ffcb5db80931002%40%3Ccommon-dev.hadoop.apache.org%3E [2] https://lists.apache.org/thread.html/e392b902273ee0c14ba34d72c44630e05f54cb3976109af510592ea2

Re: [VOTE] Release Apache Hadoop 2.10.1 (RC0)

2020-09-18 Thread Masatake Iwasaki
Iwasaki On 2020/09/15 2:59, Masatake Iwasaki wrote: Hi folks, This is the first release candidate for the second release of Apache Hadoop 2.10. It contains 218 fixes/improvements since 2.10.0 [1]. The RC0 artifacts are at: http://home.apache.org/~iwasakims/hadoop-2.10.1-RC0/ RC tag is release

[VOTE] Release Apache Hadoop 2.10.1 (RC0)

2020-09-14 Thread Masatake Iwasaki
. [1] https://issues.apache.org/jira/issues/?jql=project%20in%20(HDFS%2C%20YARN%2C%20HADOOP%2C%20MAPREDUCE)%20AND%20resolution%20%3D%20Fixed%20AND%20fixVersion%20%3D%202.10.1 Thanks, Masatake Iwasaki - To unsubscribe, e-mail

Re: [DISCUSS] Hadoop 2.10.1 release

2020-09-12 Thread Masatake Iwasaki
Only HADOOP-16918 has release-blocker label. HADOOP-16918 was resolved. I'm going to cut branch-2.10.1 and RC0 tomorrow. Thanks, Masatake Iwasaki On 2020/09/08 23:14, Masatake Iwasaki wrote: https://issues.apache.org/jira/issues/?jql=project%20in%20(HADOOP%2C%20HDFS%2C%20YARN%2C%20MAPREDUCE

Re: [DISCUSS] Ozone TLP proposal

2020-09-08 Thread Masatake Iwasaki
this approach. (for Submarine at that time) Since I'm willing to contribute, I added my name to the wiki. Thanks, Masatake Iwasaki On 2020/09/07 21:04, Elek, Marton wrote: Hi, The Hadoop community earlier decided to move out Ozone sub-project to a separated Apache Top Level Project (TLP

Re: [DISCUSS] Hadoop 2.10.1 release

2020-09-08 Thread Masatake Iwasaki
or applicable patches. Only HADOOP-16918 has release-blocker label. I filed HADOOP-17249 and HADOOP-17251 as sub tasks. If those can not be merged in a few days, I would like to put them off until next release and cut a release candidate. Thanks, Masatake Iwasaki On 2020/09/03 21:35, Masatake

Re: [DISCUSS] Hadoop 2.10.1 release

2020-09-03 Thread Masatake Iwasaki
/jira/browse/YARN-10177 https://issues.apache.org/jira/browse/HDFS-15422 https://issues.apache.org/jira/browse/HDFS-14277 https://issues.apache.org/jira/browse/HDFS-12548 I'm going to look into them and update 'Target Version/s:' of non-critical issues. Masatake Iwasaki On 2020/09/03 4:25, Jim

Re: [DISCUSS] Hadoop 2.10.1 release

2020-09-01 Thread Masatake Iwasaki
Thanks, Mingliang Liu. I volunteer to take the RM role then. I will appreciate advice from who have the experience. Masatake Iwasaki On 2020/09/01 10:38, Mingliang Liu wrote: I can see how I can help, but I can not take the RM role this time. Thanks, On Mon, Aug 31, 2020 at 12:15 PM Wei

Re: [VOTE] End of Life Hadoop 2.9

2020-09-01 Thread Masatake Iwasaki
+1 Thanks, Masatake Iwasaki On 2020/09/01 4:09, Wei-Chiu Chuang wrote: Dear fellow Hadoop developers, Given the overwhelming feedback from the discussion thread https://s.apache.org/hadoop2.9eold, I'd like to start an official vote thread for the community to vote and start the 2.9 EOL

Re: [DISCUSS] fate of branch-2.9

2020-08-27 Thread Masatake Iwasaki
+1 on EOL of branch-2.9. Since there are a lot of backported fixes in branch-2.10 after 2.10.0 (2019 Oct 29), it would be nice to release 2.10.1. I'm happy to help release work. Masatake Iwasaki On 2020/08/28 3:25, Mingliang Liu wrote: are there any Hadoop branch-2 releases planned, ever

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC4)

2020-07-23 Thread Masatake Iwasaki
-distributed cluster with Kerberos and SSL enabled, ran basic EZ operation, ran example MR jobs. * followed the reproduction step reported in  HDFS-15313 to see if the fix works. Thanks, Masatake Iwasaki On 2020/07/21 21:50, Gabor Bota wrote: Hi folks, I have put together a release candidate (RC4

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC3)

2020-07-14 Thread Masatake Iwasaki
nks, Masatake Iwasaki On 2020/07/13 19:36, Gabor Bota wrote: Hi folks, I have put together a release candidate (RC3) for Hadoop 3.1.4. * The RC includes in addition to the previous ones: * fix of YARN-10347. Fix double locking in CapacityScheduler#reinitialize in branch-3.1 (ht

Re: [VOTE] Release Apache Hadoop 3.3.0 - RC0

2020-07-13 Thread Masatake Iwasaki
cluster on CentOS 7 and CentOS 8. * ran example jobs on 3 nodes docker cluster with NN-HA and RM-HA enblaed on CentOS 7. Masatake Iwasaki On 2020/07/07 7:27, Brahma Reddy Battula wrote: Hi folks, This is the first release candidate for the first release of Apache Hadoop 3.3.0 line. It contains

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC2)

2020-07-09 Thread Masatake Iwasaki
Hi Gabor Bota, I committed the fix of YARN-10347 to branch-3.1. I think this should be blocker for 3.1.4. Could you cherry-pick it to branch-3.1.4 and cut a new RC? Thanks, Masatake Iwasaki On 2020/07/08 23:31, Masatake Iwasaki wrote: Thanks Steve and Prabhu for the information. The cause

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC2)

2020-07-08 Thread Masatake Iwasaki
Thanks Steve and Prabhu for the information. The cause turned out to be locking in CapacityScheduler#reinitialize. I think the method is called after transitioning to active stat if RM-HA is enabled. I filed YARN-10347 and created PR. Masatake Iwasaki On 2020/07/08 16:33, Prabhu Joseph

Re: [VOTE] Release Apache Hadoop 3.1.4 (RC2)

2020-07-05 Thread Masatake Iwasaki
(Server.java:943)     at java.security.AccessController.doPrivileged(Native Method)     at javax.security.auth.Subject.doAs(Subject.java:422)     at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)     at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2943)

[jira] [Resolved] (HDFS-15298) Fix the findbugs warnings introduced in HDFS-15217

2020-04-27 Thread Masatake Iwasaki (Jira)
[ https://issues.apache.org/jira/browse/HDFS-15298?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Masatake Iwasaki resolved HDFS-15298. - Fix Version/s: 3.4.0 Resolution: Fixed Committed to trunk. Thanks, [~brfrn169

  1   2   >