Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Chao Sun 
Hi Wei-Chiu,

It seems the Maven staging repository is still pointing at RC0:
https://repository.apache.org/content/repositories/staging/org/apache/hadoop/hadoop-client/3.3.1/,
and it's probably because you need to drop the old RCs in Apache Nexus
server. Could you do that? Thanks.

Chao

On Thu, Jun 3, 2021 at 12:52 PM Steve Loughran 
wrote:

> Extend for a bit from the last RC, as it takes time to qualify.
>
> I'm busy testing, doing
>
> - packaging 
> - CLI working with abfs and s3, both fs and cloudstore library calls
> - building downstream projects (so validating maven artifacts). cloudstore
> and spark there
> - building downstream of downstream projects, i.e. my spark cloud
> IO/committer test module. Moving to spark 3 cost me the afternoon, not
> through any incompatible changes there but because the upgraded scalatest
> "moved" their foundational FunTest class to a different package and name.
> Not happy with Team Scalatest there.
> - reviewing the docs in the -aws and azure modules to see they link
> together OK.
>
> So far so good.
>
> One troublespot (which isn't any reason to hold up the release), is that
> the table in the directory_markers markdown file doesn't render right.
>  Created https://issues.apache.org/jira/browse/HADOOP-17746.
>
> This is *not a blocker*
>
> I can prepare a fix and we can have it in so that if any other changes come
> in the page will look OK.
>
>
>
>
> On Thu, 3 Jun 2021 at 17:30, Wei-Chiu Chuang  wrote:
>
> > Hello,
> > do we want to extend the release vote? I understand a big release like
> this
> > takes time to validate.
> >
> > I am aware a number of people are testing it: Attila tested Ozone on
> Hadoop
> > 3.3.1 RC3, Stack is testing HBase, Chao tested Spark.
> > I also learned that anecdotally Spark on S3 on Hadoop 3.3 is faster by
> 20%
> > over Hadoop 3.2 library.
> >
>
> ooh. That'll be from Mukund's listing improvements translating into query
> planning speedups,
>
> Nice
>
> If someone benchmarking this stuff were to enable directory marker
> retention
> fs.s3a.directory.marker.retention=keep , I'd be interested to know how much
> speedup that
> delivers on versioned and unversioned buckets.
>
> Unversioned: reduces risk of IO throttling on writes
> Versioned: that and should stop subsequent LIST operations from getting
> slowed down from all the tombstones
>
>
> >
> > Looks like we may need some more time to test. How about extending it by
> a
> > week?
> >
>
>
> That would be good. This week included some holidays for people in the
> US/UK which is why I'm a bit behind on my testing.
>
>
> >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread 孙立晟 
+1


Akira Ajisaka  于2021年6月3日周四 下午2:15写道：

> Dear Hadoop developers,
>
> Given the feedback from the discussion thread [1], I'd like to start
> an official vote
> thread for the community to vote and start the 3.1 EOL process.
>
> What this entails:
>
> (1) an official announcement that no further regular Hadoop 3.1.x releases
> will be made after 3.1.4.
> (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
>
> This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
>
> Committers are eligible to cast binding votes. Non-committers are welcomed
> to cast non-binding votes.
>
> Here is my vote, +1
>
> [1] https://s.apache.org/w9ilb
> [2]
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
>
> Regards,
> Akira
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Qi Zhu 
+1
Qi Zhu

---Original---
From: "Takanobu Asanuma"https://s.apache.org/w9ilb
  [2]
 
 
 
 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=4iso=20210610T16p1=248
 
  Regards,
  Akira
 
  
-
  To unsubscribe, e-mail: 
common-dev-unsubscr...@hadoop.apache.org
  For additional commands, e-mail: 
common-dev-h...@hadoop.apache.org
 
 
 
 
  
-
  To unsubscribe, e-mail: 
common-dev-unsubscr...@hadoop.apache.org
  For additional commands, e-mail: 
common-dev-h...@hadoop.apache.org
 
 
 
 

 -
 To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
 For additional commands, e-mail: common-dev-h...@hadoop.apache.org

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Takanobu Asanuma 
+1

- Takanobu

2021年6月4日(金) 3:47 Ayush Saxena :

> +1
>
> -Ayush
>
> > On 03-Jun-2021, at 11:26 PM, hemanth boyina 
> wrote:
> >
> > +1
> >
> > Thanks
> > HemanthBoyina
> >
> >> On Thu, 3 Jun 2021, 20:33 Wanqiang Ji,  wrote:
> >>
> >> +1 (non-binding)
> >>
> >> Wanqiang Ji
> >>
> >>> On Thu, Jun 3, 2021 at 10:47 PM Sangjin Lee  wrote:
> >>>
> >>> +1
> >>>
> >>> On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
> >>> wrote:
> >>>
>  +1
> 
> > On Jun 3, 2021, at 1:14 AM, Akira Ajisaka 
> >> wrote:
> >
> > Dear Hadoop developers,
> >
> > Given the feedback from the discussion thread [1], I'd like to start
> > an official vote
> > thread for the community to vote and start the 3.1 EOL process.
> >
> > What this entails:
> >
> > (1) an official announcement that no further regular Hadoop 3.1.x
>  releases
> > will be made after 3.1.4.
> > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> >
> > This vote will run for 7 days and conclude by June 10th, 16:00 JST
> >> [2].
> >
> > Committers are eligible to cast binding votes. Non-committers are
>  welcomed
> > to cast non-binding votes.
> >
> > Here is my vote, +1
> >
> > [1] https://s.apache.org/w9ilb
> > [2]
> 
> >>>
> >>
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> >
> > Regards,
> > Akira
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> 
> 
> 
>  -
>  To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
>  For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
> 
> >>>
> >>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

如何从邮件列表中退出？

2021-06-03 Thread xiannenglin 
请问如何从邮件列表中退出？

Apache Hadoop qbt Report: trunk+JDK11 on Linux/x86_64

2021-06-03 Thread Apache Jenkins Server 
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java11-linux-x86_64/179/

[Jun 1, 2021 7:26:47 AM] (noreply) HDFS-15998. Fix NullPointException In 
listOpenFiles (#3036)
[Jun 1, 2021 1:57:22 PM] (Gergely Pollák) YARN-10797. Logging parameter issues 
in scheduler package. Contributed by Szilard Nemeth
[Jun 1, 2021 2:01:39 PM] (Szilard Nemeth) YARN-10787. Queue submit ACL check is 
wrong when CS queue is ambiguous. Contributed by Gergely Pollak




-1 overall


The following subsystems voted -1:
blanks mvnsite pathlen spotbugs unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

spotbugs :

   module:hadoop-hdfs-project/hadoop-hdfs 
   Redundant nullcheck of oldLock, which is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory)
 Redundant null check at DataStorage.java:is known to be non-null in 
org.apache.hadoop.hdfs.server.datanode.DataStorage.isPreUpgradableLayout(Storage$StorageDirectory)
 Redundant null check at DataStorage.java:[line 694] 
   
org.apache.hadoop.hdfs.server.namenode.top.window.RollingWindowManager$UserCounts
 doesn't override java.util.ArrayList.equals(Object) At 
RollingWindowManager.java:At RollingWindowManager.java:[line 1] 

spotbugs :

   module:hadoop-yarn-project/hadoop-yarn 
   Redundant nullcheck of it, which is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:[line 343] 
   Redundant nullcheck of it, which is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:[line 356] 
   Boxed value is unboxed and then immediately reboxed in 
org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
 byte[], byte[], KeyConverter, ValueConverter, boolean) At 
ColumnRWHelper.java:then immediately reboxed in 
org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result,
 byte[], byte[], KeyConverter, ValueConverter, boolean) At 
ColumnRWHelper.java:[line 333] 

spotbugs :

   module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server 
   Redundant nullcheck of it, which is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:is known to be non-null in 
org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.recoverTrackerResources(LocalResourcesTracker,
 NMStateStoreService$LocalResourceTrackerState) Redundant null check at 
ResourceLocalizationService.java:[line 343] 
   Redundant nullcheck of it, which is known to be non-null in

Apache Hadoop qbt Report: branch-2.10+JDK7 on Linux/x86_64

2021-06-03 Thread Apache Jenkins Server 
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86_64/319/

No changes




-1 overall


The following subsystems voted -1:
asflicense hadolint mvnsite pathlen unit


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

Failed junit tests :

   hadoop.util.TestDiskCheckerWithDiskIo 
   hadoop.fs.TestFileUtil 
   hadoop.crypto.key.kms.server.TestKMS 
   hadoop.contrib.bkjournal.TestBookKeeperHACheckpoints 
   hadoop.tracing.TestTraceAdmin 
   hadoop.hdfs.TestGetBlocks 
   hadoop.hdfs.TestDataTransferKeepalive 
   hadoop.fs.viewfs.TestViewFsWithXAttrs 
   hadoop.hdfs.TestSetrepIncreasing 
   hadoop.hdfs.client.impl.TestBlockReaderLocal 
   hadoop.fs.contract.hdfs.TestHDFSContractDelete 
   hadoop.cli.TestAclCLI 
   hadoop.fs.permission.TestStickyBit 
   hadoop.hdfs.server.blockmanagement.TestBlockReportRateLimiting 
   hadoop.hdfs.TestSnapshotCommands 
   hadoop.hdfs.TestDFSShellGenericOptions 
   hadoop.fs.contract.hdfs.TestHDFSContractSetTimes 
   hadoop.hdfs.server.datanode.TestDataNodeReconfiguration 
   hadoop.hdfs.TestRollingUpgradeDowngrade 
   hadoop.hdfs.server.datanode.TestFsDatasetCache 
   hadoop.hdfs.server.blockmanagement.TestHeartbeatHandling 
   hadoop.hdfs.TestLocalDFS 
   hadoop.hdfs.TestHFlush 
   hadoop.hdfs.TestFileAppend4 
   hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation 
   hadoop.hdfs.TestFSOutputSummer 
   hadoop.hdfs.TestModTime 
   hadoop.hdfs.TestSafeMode 
   hadoop.hdfs.server.balancer.TestBalancerWithEncryptedTransfer 
   hadoop.hdfs.server.balancer.TestBalancer 
   hadoop.hdfs.TestFileCorruption 
   hadoop.hdfs.TestParallelShortCircuitReadNoChecksum 
   hadoop.hdfs.TestDataStream 
   hadoop.fs.viewfs.TestViewFileSystemWithTruncate 
   hadoop.hdfs.TestAclsEndToEnd 
   hadoop.fs.TestSymlinkHdfsFileContext 
   
hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForContentSummary 
   
hadoop.hdfs.server.blockmanagement.TestReplicationPolicyWithUpgradeDomain 
   hadoop.hdfs.TestFileConcurrentReader 
   hadoop.hdfs.TestDataTransferProtocol 
   hadoop.hdfs.client.impl.TestClientBlockVerification 
   hadoop.hdfs.TestReplication 
   hadoop.hdfs.TestMaintenanceState 
   hadoop.hdfs.TestFetchImage 
   hadoop.hdfs.tools.TestGetGroups 
   hadoop.fs.viewfs.TestViewFsAtHdfsRoot 
   hadoop.fs.TestSymlinkHdfsFileSystem 
   hadoop.hdfs.TestDFSOutputStream 
   hadoop.hdfs.TestDatanodeStartupFixesLegacyStorageIDs 
   hadoop.hdfs.server.namenode.ha.TestHAAppend 
   hadoop.hdfs.TestRestartDFS 
   hadoop.hdfs.TestEncryptionZonesWithHA 
   hadoop.cli.TestCryptoAdminCLI 
   hadoop.fs.contract.hdfs.TestHDFSContractRename 
   hadoop.hdfs.tools.TestDFSAdminWithHA 
   hadoop.fs.TestWebHdfsFileContextMainOperations 
   hadoop.hdfs.TestSetTimes 
   hadoop.hdfs.TestWriteRead 
   hadoop.fs.TestFcHdfsSetUMask 
   hadoop.fs.viewfs.TestViewFileSystemWithAcls 
   hadoop.hdfs.crypto.TestHdfsCryptoStreams 
   hadoop.hdfs.TestParallelRead 
   hadoop.hdfs.TestRollingUpgrade 
   hadoop.hdfs.TestRenameWhileOpen 
   hadoop.hdfs.TestHDFSTrash 
   hadoop.hdfs.shortcircuit.TestShortCircuitLocalRead 
   hadoop.hdfs.TestMissingBlocksAlert 
   hadoop.hdfs.tools.TestDFSZKFailoverController 
   hadoop.fs.viewfs.TestViewFileSystemWithXAttrs 
   hadoop.hdfs.TestBlockTokenWrappingQOP 
   hadoop.hdfs.TestDatanodeLayoutUpgrade 
   hadoop.hdfs.TestExtendedAcls 
   hadoop.hdfs.server.balancer.TestBalancerWithNodeGroup 
   hadoop.hdfs.TestDFSClientRetries 
   hadoop.hdfs.TestBlockStoragePolicy 
   hadoop.hdfs.server.datanode.TestBlockRecovery 
   hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys 
   hadoop.hdfs.tools.offlineImageViewer.TestOfflineImageViewerForAcl 
   hadoop.hdfs.TestReplaceDatanodeFailureReplication 
   hadoop.hdfs.TestMultipleNNPortQOP 
   hadoop.fs.TestEnhancedByteBufferAccess 
   hadoop.fs.loadGenerator.TestLoadGenerator 
   hadoop.hdfs.TestHdfsAdmin 
   hadoop.cli.TestCacheAdminCLI 
   hadoop.hdfs.TestByteBufferPread 
   hadoop.hdfs.TestReservedRawPaths 
   hadoop.hdfs.qjournal.TestSecureNNWithQJM 
   hadoop.hdfs.security.token.block.TestBlockToken 
   hadoop.hdfs.TestSetrepDecreasing 
   hadoop.hdfs.tools.TestDebugAdmin 
   hadoop.hdfs.server.datanode.TestDataNodeVolumeMetrics 
   hadoop.hdfs.TestConnCache 
   hadoop.fs.contract.hdfs.TestHDFSContractMkdir 
   hadoop.hdfs.TestEncryptionZonesWithKMS

Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64

2021-06-03 Thread Apache Jenkins Server 
For more details, see 
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/

No changes




-1 overall


The following subsystems voted -1:
blanks pathlen unit xml


The following subsystems voted -1 but
were configured to be filtered/ignored:
cc checkstyle javac javadoc pylint shellcheck


The following subsystems are considered long running:
(runtime bigger than 1h  0m  0s)
unit


Specific tests:

XML :

   Parsing Error(s): 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-excerpt.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-output-missing-tags2.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/test/resources/nvidia-smi-sample-output.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/fair-scheduler-invalid.xml
 
   
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/test/resources/yarn-site-with-invalid-allocation-file-ref.xml
 

Failed junit tests :

   hadoop.hdfs.server.datanode.fsdataset.impl.TestFsDatasetImpl 
   hadoop.hdfs.TestViewDistributedFileSystemContract 
   
hadoop.yarn.server.timelineservice.storage.common.TestHBaseTimelineStorageUtils 
   hadoop.yarn.csi.client.TestCsiClient 
   hadoop.tools.dynamometer.workloadgenerator.TestWorkloadGenerator 
   hadoop.tools.dynamometer.TestDynamometerInfra 
   hadoop.tools.dynamometer.blockgenerator.TestBlockGen 
   hadoop.tools.dynamometer.TestDynamometerInfra 
   hadoop.tools.dynamometer.blockgenerator.TestBlockGen 
   hadoop.tools.dynamometer.workloadgenerator.TestWorkloadGenerator 
  

   cc:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-compile-cc-root.txt
 [96K]

   javac:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-compile-javac-root.txt
 [380K]

   blanks:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/blanks-eol.txt
 [13M]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/blanks-tabs.txt
 [2.0M]

   checkstyle:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-checkstyle-root.txt
 [16M]

   pathlen:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-pathlen.txt
 [16K]

   pylint:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-pylint.txt
 [20K]

   shellcheck:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-shellcheck.txt
 [28K]

   xml:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/xml.txt
 [24K]

   javadoc:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/results-javadoc-javadoc-root.txt
 [408K]

   unit:

  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
 [536K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-client.txt
 [24K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-csi.txt
 [20K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-tools_hadoop-dynamometer_hadoop-dynamometer-workload.txt
 [32K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-tools_hadoop-dynamometer_hadoop-dynamometer-infra.txt
 [12K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-tools_hadoop-dynamometer_hadoop-dynamometer-blockgen.txt
 [28K]
  
https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/527/artifact/out/patch-unit-hadoop-tools_hadoop-dynamometer.txt
 [68K]

Powered by Apache Yetus 0.14.0-SNAPSHOT   https://yetus.apache.org

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Steve Loughran 
Extend for a bit from the last RC, as it takes time to qualify.

I'm busy testing, doing

- packaging 
- CLI working with abfs and s3, both fs and cloudstore library calls
- building downstream projects (so validating maven artifacts). cloudstore
and spark there
- building downstream of downstream projects, i.e. my spark cloud
IO/committer test module. Moving to spark 3 cost me the afternoon, not
through any incompatible changes there but because the upgraded scalatest
"moved" their foundational FunTest class to a different package and name.
Not happy with Team Scalatest there.
- reviewing the docs in the -aws and azure modules to see they link
together OK.

So far so good.

One troublespot (which isn't any reason to hold up the release), is that
the table in the directory_markers markdown file doesn't render right.
 Created https://issues.apache.org/jira/browse/HADOOP-17746.

This is *not a blocker*

I can prepare a fix and we can have it in so that if any other changes come
in the page will look OK.




On Thu, 3 Jun 2021 at 17:30, Wei-Chiu Chuang  wrote:

> Hello,
> do we want to extend the release vote? I understand a big release like this
> takes time to validate.
>
> I am aware a number of people are testing it: Attila tested Ozone on Hadoop
> 3.3.1 RC3, Stack is testing HBase, Chao tested Spark.
> I also learned that anecdotally Spark on S3 on Hadoop 3.3 is faster by 20%
> over Hadoop 3.2 library.
>

ooh. That'll be from Mukund's listing improvements translating into query
planning speedups,

Nice

If someone benchmarking this stuff were to enable directory marker retention
fs.s3a.directory.marker.retention=keep , I'd be interested to know how much
speedup that
delivers on versioned and unversioned buckets.

Unversioned: reduces risk of IO throttling on writes
Versioned: that and should stop subsequent LIST operations from getting
slowed down from all the tombstones


>
> Looks like we may need some more time to test. How about extending it by a
> week?
>


That would be good. This week included some holidays for people in the
US/UK which is why I'm a bit behind on my testing.


>
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Ayush Saxena 
+1

-Ayush

> On 03-Jun-2021, at 11:26 PM, hemanth boyina  
> wrote:
> 
> +1
> 
> Thanks
> HemanthBoyina
> 
>> On Thu, 3 Jun 2021, 20:33 Wanqiang Ji,  wrote:
>> 
>> +1 (non-binding)
>> 
>> Wanqiang Ji
>> 
>>> On Thu, Jun 3, 2021 at 10:47 PM Sangjin Lee  wrote:
>>> 
>>> +1
>>> 
>>> On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
>>> wrote:
>>> 
 +1
 
> On Jun 3, 2021, at 1:14 AM, Akira Ajisaka 
>> wrote:
> 
> Dear Hadoop developers,
> 
> Given the feedback from the discussion thread [1], I'd like to start
> an official vote
> thread for the community to vote and start the 3.1 EOL process.
> 
> What this entails:
> 
> (1) an official announcement that no further regular Hadoop 3.1.x
 releases
> will be made after 3.1.4.
> (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> 
> This vote will run for 7 days and conclude by June 10th, 16:00 JST
>> [2].
> 
> Committers are eligible to cast binding votes. Non-committers are
 welcomed
> to cast non-binding votes.
> 
> Here is my vote, +1
> 
> [1] https://s.apache.org/w9ilb
> [2]
 
>>> 
>> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> 
> Regards,
> Akira
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 
 
 
 
 -
 To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
 For additional commands, e-mail: common-dev-h...@hadoop.apache.org
 
 
>>> 
>> 

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread hemanth boyina 
+1

Thanks
HemanthBoyina

On Thu, 3 Jun 2021, 20:33 Wanqiang Ji,  wrote:

> +1 (non-binding)
>
> Wanqiang Ji
>
> On Thu, Jun 3, 2021 at 10:47 PM Sangjin Lee  wrote:
>
> > +1
> >
> > On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
> > wrote:
> >
> > > +1
> > >
> > > > On Jun 3, 2021, at 1:14 AM, Akira Ajisaka 
> wrote:
> > > >
> > > > Dear Hadoop developers,
> > > >
> > > > Given the feedback from the discussion thread [1], I'd like to start
> > > > an official vote
> > > > thread for the community to vote and start the 3.1 EOL process.
> > > >
> > > > What this entails:
> > > >
> > > > (1) an official announcement that no further regular Hadoop 3.1.x
> > > releases
> > > > will be made after 3.1.4.
> > > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > > >
> > > > This vote will run for 7 days and conclude by June 10th, 16:00 JST
> [2].
> > > >
> > > > Committers are eligible to cast binding votes. Non-committers are
> > > welcomed
> > > > to cast non-binding votes.
> > > >
> > > > Here is my vote, +1
> > > >
> > > > [1] https://s.apache.org/w9ilb
> > > > [2]
> > >
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> > > >
> > > > Regards,
> > > > Akira
> > > >
> > > > -
> > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > > >
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Wei-Chiu Chuang 
So I was thinking 5+7 days from Tuesday = next Sunday so that gives
everyone a whole week to validate.

On Fri, Jun 4, 2021 at 12:38 AM Sean Busbey  wrote:

> Sounds good to me. That would be until Thursday June 10th, right?
>
> As a side note it’s concerning that a double-dot maintenance release is a
> big release, but I get that it’s the current state of the project.
>
> > On Jun 3, 2021, at 11:30 AM, Wei-Chiu Chuang  wrote:
> >
> > Hello,
> > do we want to extend the release vote? I understand a big release like
> this
> > takes time to validate.
> >
> > I am aware a number of people are testing it: Attila tested Ozone on
> Hadoop
> > 3.3.1 RC3, Stack is testing HBase, Chao tested Spark.
> > I also learned that anecdotally Spark on S3 on Hadoop 3.3 is faster by
> 20%
> > over Hadoop 3.2 library.
> >
> > Looks like we may need some more time to test. How about extending it by
> a
> > week?
> >
> > On Tue, Jun 1, 2021 at 6:29 PM Wei-Chiu Chuang 
> wrote:
> >
> >> Hi community,
> >>
> >> This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All
> blocker
> >> issues have been resolved [1] again.
> >>
> >> There are 2 additional issues resolved for RC3:
> >> * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after
> >> HADOOP-16878
> >> * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the
> source
> >> and destination are the same
> >>
> >> There are 4 issues resolved for RC2:
> >> * HADOOP-17666. Update LICENSE for 3.3.1
> >> * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053)
> >> * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055)
> >> * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064)
> >>
> >> The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two extra
> >> fixes compared to hadoop-thirdparty 1.1.0:
> >> * HADOOP-17707. Remove jaeger document from site index.
> >> * HADOOP-17730. Add back error_prone
> >>
> >> *RC tag is release-3.3.1-RC3
> >> https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3
> >>
> >> *The RC3 artifacts are at*:
> >> https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/
> >> ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/
> >>
> >> *The maven artifacts are hosted here:*
> >>
> https://repository.apache.org/content/repositories/orgapachehadoop-1320/
> >>
> >> *My public key is available here:*
> >> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
> >>
> >>
> >> Things I've verified:
> >> * all blocker issues targeting 3.3.1 have been resolved.
> >> * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible.
> >> * LICENSE and NOTICE files checked
> >> * RELEASENOTES and CHANGELOG
> >> * rat check passed.
> >> * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests.
> >> * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests.
> >> * Extra: built 50 other open source projects on top of Hadoop 3.3.1 RC2.
> >> Had to patch some of them due to commons-lang migration (Hadoop 3.2.0)
> and
> >> dependency divergence. Issues are being identified but so far nothing
> >> blocker for Hadoop itself.
> >>
> >> Please try the release and vote. The vote will run for 5 days.
> >>
> >> My +1 to start,
> >>
> >> [1] https://issues.apache.org/jira/issues/?filter=12350491
> >> [2]
> >>
> https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3
> >>
> >>
> >>
>
>
>

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Sean Busbey 
Sounds good to me. That would be until Thursday June 10th, right?

As a side note it’s concerning that a double-dot maintenance release is a big 
release, but I get that it’s the current state of the project.

> On Jun 3, 2021, at 11:30 AM, Wei-Chiu Chuang  wrote:
> 
> Hello,
> do we want to extend the release vote? I understand a big release like this
> takes time to validate.
> 
> I am aware a number of people are testing it: Attila tested Ozone on Hadoop
> 3.3.1 RC3, Stack is testing HBase, Chao tested Spark.
> I also learned that anecdotally Spark on S3 on Hadoop 3.3 is faster by 20%
> over Hadoop 3.2 library.
> 
> Looks like we may need some more time to test. How about extending it by a
> week?
> 
> On Tue, Jun 1, 2021 at 6:29 PM Wei-Chiu Chuang  wrote:
> 
>> Hi community,
>> 
>> This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker
>> issues have been resolved [1] again.
>> 
>> There are 2 additional issues resolved for RC3:
>> * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after
>> HADOOP-16878
>> * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source
>> and destination are the same
>> 
>> There are 4 issues resolved for RC2:
>> * HADOOP-17666. Update LICENSE for 3.3.1
>> * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053)
>> * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055)
>> * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064)
>> 
>> The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two extra
>> fixes compared to hadoop-thirdparty 1.1.0:
>> * HADOOP-17707. Remove jaeger document from site index.
>> * HADOOP-17730. Add back error_prone
>> 
>> *RC tag is release-3.3.1-RC3
>> https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3
>> 
>> *The RC3 artifacts are at*:
>> https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/
>> ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/
>> 
>> *The maven artifacts are hosted here:*
>> https://repository.apache.org/content/repositories/orgapachehadoop-1320/
>> 
>> *My public key is available here:*
>> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>> 
>> 
>> Things I've verified:
>> * all blocker issues targeting 3.3.1 have been resolved.
>> * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible.
>> * LICENSE and NOTICE files checked
>> * RELEASENOTES and CHANGELOG
>> * rat check passed.
>> * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests.
>> * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests.
>> * Extra: built 50 other open source projects on top of Hadoop 3.3.1 RC2.
>> Had to patch some of them due to commons-lang migration (Hadoop 3.2.0) and
>> dependency divergence. Issues are being identified but so far nothing
>> blocker for Hadoop itself.
>> 
>> Please try the release and vote. The vote will run for 5 days.
>> 
>> My +1 to start,
>> 
>> [1] https://issues.apache.org/jira/issues/?filter=12350491
>> [2]
>> https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3
>> 
>> 
>> 



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Release Apache Hadoop 3.3.1 RC3

2021-06-03 Thread Wei-Chiu Chuang 
Hello,
do we want to extend the release vote? I understand a big release like this
takes time to validate.

I am aware a number of people are testing it: Attila tested Ozone on Hadoop
3.3.1 RC3, Stack is testing HBase, Chao tested Spark.
I also learned that anecdotally Spark on S3 on Hadoop 3.3 is faster by 20%
over Hadoop 3.2 library.

Looks like we may need some more time to test. How about extending it by a
week?

On Tue, Jun 1, 2021 at 6:29 PM Wei-Chiu Chuang  wrote:

> Hi community,
>
> This is the release candidate RC3 of Apache Hadoop 3.3.1 line. All blocker
> issues have been resolved [1] again.
>
> There are 2 additional issues resolved for RC3:
> * Revert "MAPREDUCE-7303. Fix TestJobResourceUploader failures after
> HADOOP-16878
> * Revert "HADOOP-16878. FileUtil.copy() to throw IOException if the source
> and destination are the same
>
> There are 4 issues resolved for RC2:
> * HADOOP-17666. Update LICENSE for 3.3.1
> * MAPREDUCE-7348. TestFrameworkUploader#testNativeIO fails. (#3053)
> * Revert "HADOOP-17563. Update Bouncy Castle to 1.68. (#2740)" (#3055)
> * HADOOP-17739. Use hadoop-thirdparty 1.1.1. (#3064)
>
> The Hadoop-thirdparty 1.1.1, as previously mentioned, contains two extra
> fixes compared to hadoop-thirdparty 1.1.0:
> * HADOOP-17707. Remove jaeger document from site index.
> * HADOOP-17730. Add back error_prone
>
> *RC tag is release-3.3.1-RC3
> https://github.com/apache/hadoop/releases/tag/release-3.3.1-RC3
>
> *The RC3 artifacts are at*:
> https://home.apache.org/~weichiu/hadoop-3.3.1-RC3/
> ARM artifacts: https://home.apache.org/~weichiu/hadoop-3.3.1-RC3-arm/
>
> *The maven artifacts are hosted here:*
> https://repository.apache.org/content/repositories/orgapachehadoop-1320/
>
> *My public key is available here:*
> https://dist.apache.org/repos/dist/release/hadoop/common/KEYS
>
>
> Things I've verified:
> * all blocker issues targeting 3.3.1 have been resolved.
> * stable/evolving API changes between 3.3.0 and 3.3.1 are compatible.
> * LICENSE and NOTICE files checked
> * RELEASENOTES and CHANGELOG
> * rat check passed.
> * Built HBase master branch on top of Hadoop 3.3.1 RC2, ran unit tests.
> * Built Ozone master on top fo Hadoop 3.3.1 RC2, ran unit tests.
> * Extra: built 50 other open source projects on top of Hadoop 3.3.1 RC2.
> Had to patch some of them due to commons-lang migration (Hadoop 3.2.0) and
> dependency divergence. Issues are being identified but so far nothing
> blocker for Hadoop itself.
>
> Please try the release and vote. The vote will run for 5 days.
>
> My +1 to start,
>
> [1] https://issues.apache.org/jira/issues/?filter=12350491
> [2]
> https://github.com/apache/hadoop/compare/release-3.3.1-RC1...release-3.3.1-RC3
>
>
>

[jira] [Created] (HDFS-16054) Replace Guava Lists usage by Hadoop's own Lists in hadoop-hdfs-project

2021-06-03 Thread Viraj Jasani (Jira) 
Viraj Jasani created HDFS-16054:
---

 Summary: Replace Guava Lists usage by Hadoop's own Lists in 
hadoop-hdfs-project
 Key: HDFS-16054
 URL: https://issues.apache.org/jira/browse/HDFS-16054
 Project: Hadoop HDFS
  Issue Type: Task
Reporter: Viraj Jasani
Assignee: Viraj Jasani






--
This message was sent by Atlassian Jira
(v8.3.4#803005)

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Wanqiang Ji 
+1 (non-binding)

Wanqiang Ji

On Thu, Jun 3, 2021 at 10:47 PM Sangjin Lee  wrote:

> +1
>
> On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
> wrote:
>
> > +1
> >
> > > On Jun 3, 2021, at 1:14 AM, Akira Ajisaka  wrote:
> > >
> > > Dear Hadoop developers,
> > >
> > > Given the feedback from the discussion thread [1], I'd like to start
> > > an official vote
> > > thread for the community to vote and start the 3.1 EOL process.
> > >
> > > What this entails:
> > >
> > > (1) an official announcement that no further regular Hadoop 3.1.x
> > releases
> > > will be made after 3.1.4.
> > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > >
> > > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> > >
> > > Committers are eligible to cast binding votes. Non-committers are
> > welcomed
> > > to cast non-binding votes.
> > >
> > > Here is my vote, +1
> > >
> > > [1] https://s.apache.org/w9ilb
> > > [2]
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> > >
> > > Regards,
> > > Akira
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> >
> >
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Sangjin Lee 
+1

On Thu, Jun 3, 2021 at 7:35 AM Sean Busbey 
wrote:

> +1
>
> > On Jun 3, 2021, at 1:14 AM, Akira Ajisaka  wrote:
> >
> > Dear Hadoop developers,
> >
> > Given the feedback from the discussion thread [1], I'd like to start
> > an official vote
> > thread for the community to vote and start the 3.1 EOL process.
> >
> > What this entails:
> >
> > (1) an official announcement that no further regular Hadoop 3.1.x
> releases
> > will be made after 3.1.4.
> > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> >
> > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> >
> > Committers are eligible to cast binding votes. Non-committers are
> welcomed
> > to cast non-binding votes.
> >
> > Here is my vote, +1
> >
> > [1] https://s.apache.org/w9ilb
> > [2]
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> >
> > Regards,
> > Akira
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
>
>
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Sean Busbey 
+1

> On Jun 3, 2021, at 1:14 AM, Akira Ajisaka  wrote:
> 
> Dear Hadoop developers,
> 
> Given the feedback from the discussion thread [1], I'd like to start
> an official vote
> thread for the community to vote and start the 3.1 EOL process.
> 
> What this entails:
> 
> (1) an official announcement that no further regular Hadoop 3.1.x releases
> will be made after 3.1.4.
> (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> 
> This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> 
> Committers are eligible to cast binding votes. Non-committers are welcomed
> to cast non-binding votes.
> 
> Here is my vote, +1
> 
> [1] https://s.apache.org/w9ilb
> [2] 
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> 
> Regards,
> Akira
> 
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> 



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [E] [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Jim Brennan 
+1


On Thu, Jun 3, 2021 at 1:14 AM Akira Ajisaka  wrote:

> Dear Hadoop developers,
>
> Given the feedback from the discussion thread [1], I'd like to start
> an official vote
> thread for the community to vote and start the 3.1 EOL process.
>
> What this entails:
>
> (1) an official announcement that no further regular Hadoop 3.1.x releases
> will be made after 3.1.4.
> (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
>
> This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
>
> Committers are eligible to cast binding votes. Non-committers are welcomed
> to cast non-binding votes.
>
> Here is my vote, +1
>
> [1]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__s.apache.org_w9ilb=DwIBaQ=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw=ZWrvhaLN6hDkRibeGSwUQolnityfAhfxPBcxL9gc7P4=cuVKhSKq_G3un1VWO70Ri9KiZAn-VubytwVGtnljFTI=
> [2]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.timeanddate.com_worldclock_fixedtime.html-3Fmsg-3D4-26iso-3D20210610T16-26p1-3D248=DwIBaQ=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY=7Imi06B91L3gbxmt5ChzH4cwlA2_f2tmXh3OXmV9MLw=ZWrvhaLN6hDkRibeGSwUQolnityfAhfxPBcxL9gc7P4=cM_wi2TYoRB0rofjBMmB-2Ln9s-LdbFNQtrBwhQfy5k=
>
> Regards,
> Akira
>
> -
> To unsubscribe, e-mail: mapreduce-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: mapreduce-dev-h...@hadoop.apache.org
>
>

Re: [E] Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Eric Badger 
+1

On Thu, Jun 3, 2021 at 6:14 AM Surendra Singh Lilhore <
surendralilh...@gmail.com> wrote:

> +1
>
> Thanks and Regards
> Surendra
>
>
> On Thu, Jun 3, 2021 at 4:31 PM tom lee  wrote:
>
> > +1
> >
> > Masatake Iwasaki  于2021年6月3日周四 下午5:43写道：
> >
> > > +1
> > >
> > > Masatake Iwasaki
> > >
> > > On 2021/06/03 15:14, Akira Ajisaka wrote:
> > > > Dear Hadoop developers,
> > > >
> > > > Given the feedback from the discussion thread [1], I'd like to start
> > > > an official vote
> > > > thread for the community to vote and start the 3.1 EOL process.
> > > >
> > > > What this entails:
> > > >
> > > > (1) an official announcement that no further regular Hadoop 3.1.x
> > > releases
> > > > will be made after 3.1.4.
> > > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > > >
> > > > This vote will run for 7 days and conclude by June 10th, 16:00 JST
> [2].
> > > >
> > > > Committers are eligible to cast binding votes. Non-committers are
> > > welcomed
> > > > to cast non-binding votes.
> > > >
> > > > Here is my vote, +1
> > > >
> > > > [1]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__s.apache.org_w9ilb=DwIFaQ=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY=KVdP1SUmHYb-tZP8tcigmw=7qPmaUPm6mGPWVMDM8Mw3T10AGE1J_pl-r1Asa9e238=rwL5wXoWZCvwORFw83YrWc7-vooDvzTqHkYdro380zg=
> > > > [2]
> > >
> >
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.timeanddate.com_worldclock_fixedtime.html-3Fmsg-3D4-26iso-3D20210610T16-26p1-3D248=DwIFaQ=sWW_bEwW_mLyN3Kx2v57Q8e-CRbmiT9yOhqES_g_wVY=KVdP1SUmHYb-tZP8tcigmw=7qPmaUPm6mGPWVMDM8Mw3T10AGE1J_pl-r1Asa9e238=Zg_od8AJUI5KResvSO-PiNJeb_l5JmtUT27Y-F_MhTQ=
> > > >
> > > > Regards,
> > > > Akira
> > > >
> > > > -
> > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> > >
> > >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Surendra Singh Lilhore 
+1

Thanks and Regards
Surendra


On Thu, Jun 3, 2021 at 4:31 PM tom lee  wrote:

> +1
>
> Masatake Iwasaki  于2021年6月3日周四 下午5:43写道：
>
> > +1
> >
> > Masatake Iwasaki
> >
> > On 2021/06/03 15:14, Akira Ajisaka wrote:
> > > Dear Hadoop developers,
> > >
> > > Given the feedback from the discussion thread [1], I'd like to start
> > > an official vote
> > > thread for the community to vote and start the 3.1 EOL process.
> > >
> > > What this entails:
> > >
> > > (1) an official announcement that no further regular Hadoop 3.1.x
> > releases
> > > will be made after 3.1.4.
> > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > >
> > > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> > >
> > > Committers are eligible to cast binding votes. Non-committers are
> > welcomed
> > > to cast non-binding votes.
> > >
> > > Here is my vote, +1
> > >
> > > [1] https://s.apache.org/w9ilb
> > > [2]
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> > >
> > > Regards,
> > > Akira
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
> >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread tom lee 
+1

Masatake Iwasaki  于2021年6月3日周四 下午5:43写道：

> +1
>
> Masatake Iwasaki
>
> On 2021/06/03 15:14, Akira Ajisaka wrote:
> > Dear Hadoop developers,
> >
> > Given the feedback from the discussion thread [1], I'd like to start
> > an official vote
> > thread for the community to vote and start the 3.1 EOL process.
> >
> > What this entails:
> >
> > (1) an official announcement that no further regular Hadoop 3.1.x
> releases
> > will be made after 3.1.4.
> > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> >
> > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> >
> > Committers are eligible to cast binding votes. Non-committers are
> welcomed
> > to cast non-binding votes.
> >
> > Here is my vote, +1
> >
> > [1] https://s.apache.org/w9ilb
> > [2]
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> >
> > Regards,
> > Akira
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
>
> -
> To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
>
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Masatake Iwasaki 

+1

Masatake Iwasaki

On 2021/06/03 15:14, Akira Ajisaka wrote:

Dear Hadoop developers,

Given the feedback from the discussion thread [1], I'd like to start
an official vote
thread for the community to vote and start the 3.1 EOL process.

What this entails:

(1) an official announcement that no further regular Hadoop 3.1.x releases
will be made after 3.1.4.
(2) resolve JIRAs that specifically target 3.1.5 as won't fix.

This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].

Committers are eligible to cast binding votes. Non-committers are welcomed
to cast non-binding votes.

Here is my vote, +1

[1] https://s.apache.org/w9ilb
[2] 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248

Regards,
Akira

-
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org



-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

?????? [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Qi Zhu 
+1




----
??: 
   "Xiaoqiao He"

https://s.apache.org/w9ilb
[2]
   
  
 
 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=4iso=20210610T16p1=248
   
Regards,
Akira
   

-
To unsubscribe, e-mail: 
common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: 
common-dev-h...@hadoop.apache.org

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Xiaoqiao He 
+1

On Thu, Jun 3, 2021 at 4:18 PM Bharat Viswanadham  wrote:

> +1
>
> Thanks,
> Bharat
>
>
> On Thu, Jun 3, 2021 at 1:00 PM Viraj Jasani  wrote:
>
> > +1 (non-binding)
> >
> > On Thu, 3 Jun 2021 at 12:21 PM, Wei-Chiu Chuang 
> > wrote:
> >
> > > +1
> > >
> > > On Thu, Jun 3, 2021 at 2:14 PM Akira Ajisaka 
> > wrote:
> > >
> > > > Dear Hadoop developers,
> > > >
> > > > Given the feedback from the discussion thread [1], I'd like to start
> > > > an official vote
> > > > thread for the community to vote and start the 3.1 EOL process.
> > > >
> > > > What this entails:
> > > >
> > > > (1) an official announcement that no further regular Hadoop 3.1.x
> > > releases
> > > > will be made after 3.1.4.
> > > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > > >
> > > > This vote will run for 7 days and conclude by June 10th, 16:00 JST
> [2].
> > > >
> > > > Committers are eligible to cast binding votes. Non-committers are
> > > welcomed
> > > > to cast non-binding votes.
> > > >
> > > > Here is my vote, +1
> > > >
> > > > [1] https://s.apache.org/w9ilb
> > > > [2]
> > > >
> > >
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> > > >
> > > > Regards,
> > > > Akira
> > > >
> > > > -
> > > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > > >
> > > >
> > >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Bharat Viswanadham 
+1

Thanks,
Bharat


On Thu, Jun 3, 2021 at 1:00 PM Viraj Jasani  wrote:

> +1 (non-binding)
>
> On Thu, 3 Jun 2021 at 12:21 PM, Wei-Chiu Chuang 
> wrote:
>
> > +1
> >
> > On Thu, Jun 3, 2021 at 2:14 PM Akira Ajisaka 
> wrote:
> >
> > > Dear Hadoop developers,
> > >
> > > Given the feedback from the discussion thread [1], I'd like to start
> > > an official vote
> > > thread for the community to vote and start the 3.1 EOL process.
> > >
> > > What this entails:
> > >
> > > (1) an official announcement that no further regular Hadoop 3.1.x
> > releases
> > > will be made after 3.1.4.
> > > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> > >
> > > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> > >
> > > Committers are eligible to cast binding votes. Non-committers are
> > welcomed
> > > to cast non-binding votes.
> > >
> > > Here is my vote, +1
> > >
> > > [1] https://s.apache.org/w9ilb
> > > [2]
> > >
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> > >
> > > Regards,
> > > Akira
> > >
> > > -
> > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> > >
> > >
> >
>

Discusstion and Review for reverting HDFS-9260 FoldedTreeSet

2021-06-03 Thread Hui Fei 
Hi Community,

HDFS-9260's performance issue has existed for years, and many
companies using hadoop 3.x encounter it. Maybe it's time to revert
FoldedTreeSet.

Now the PR from Haibin Huang is ready, please see
https://github.com/apache/hadoop/pull/3065

Does it make sense? it is a huge change, please help to review it,
Thanks.

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Viraj Jasani 
+1 (non-binding)

On Thu, 3 Jun 2021 at 12:21 PM, Wei-Chiu Chuang  wrote:

> +1
>
> On Thu, Jun 3, 2021 at 2:14 PM Akira Ajisaka  wrote:
>
> > Dear Hadoop developers,
> >
> > Given the feedback from the discussion thread [1], I'd like to start
> > an official vote
> > thread for the community to vote and start the 3.1 EOL process.
> >
> > What this entails:
> >
> > (1) an official announcement that no further regular Hadoop 3.1.x
> releases
> > will be made after 3.1.4.
> > (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
> >
> > This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
> >
> > Committers are eligible to cast binding votes. Non-committers are
> welcomed
> > to cast non-binding votes.
> >
> > Here is my vote, +1
> >
> > [1] https://s.apache.org/w9ilb
> > [2]
> >
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
> >
> > Regards,
> > Akira
> >
> > -
> > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> > For additional commands, e-mail: common-dev-h...@hadoop.apache.org
> >
> >
>

Re: [VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Wei-Chiu Chuang 
+1

On Thu, Jun 3, 2021 at 2:14 PM Akira Ajisaka  wrote:

> Dear Hadoop developers,
>
> Given the feedback from the discussion thread [1], I'd like to start
> an official vote
> thread for the community to vote and start the 3.1 EOL process.
>
> What this entails:
>
> (1) an official announcement that no further regular Hadoop 3.1.x releases
> will be made after 3.1.4.
> (2) resolve JIRAs that specifically target 3.1.5 as won't fix.
>
> This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].
>
> Committers are eligible to cast binding votes. Non-committers are welcomed
> to cast non-binding votes.
>
> Here is my vote, +1
>
> [1] https://s.apache.org/w9ilb
> [2]
> https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248
>
> Regards,
> Akira
>
> -
> To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
> For additional commands, e-mail: common-dev-h...@hadoop.apache.org
>
>

[VOTE] Hadoop 3.1.x EOL

2021-06-03 Thread Akira Ajisaka 
Dear Hadoop developers,

Given the feedback from the discussion thread [1], I'd like to start
an official vote
thread for the community to vote and start the 3.1 EOL process.

What this entails:

(1) an official announcement that no further regular Hadoop 3.1.x releases
will be made after 3.1.4.
(2) resolve JIRAs that specifically target 3.1.5 as won't fix.

This vote will run for 7 days and conclude by June 10th, 16:00 JST [2].

Committers are eligible to cast binding votes. Non-committers are welcomed
to cast non-binding votes.

Here is my vote, +1

[1] https://s.apache.org/w9ilb
[2] 
https://www.timeanddate.com/worldclock/fixedtime.html?msg=4=20210610T16=248

Regards,
Akira

-
To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org

Re: [DISCUSS] which release lines should we still consider actively maintained?

2021-06-03 Thread Akira Ajisaka 
Thank you for your comments. I'll create a vote thread to mark 3.1.x EOL.

-Akira

On Tue, May 25, 2021 at 12:46 AM Ayush Saxena  wrote:
>
> +1, to mark 3.1.x EOL.
> Apache Hive does depends on 3.1.0 as of now, but due to guave upgrade on 
> branch-3.1, the attempt to migrate to latest 3.1.x didn’t work for me atleast 
> couple of months back. So, mostly 3.3.1 would be the only option replacing 
> 3.1.0 there or at worst 3.3.2 in a couple of months.
>
>
> -Ayush
>
> > On 24-May-2021, at 8:43 PM, Arpit Agarwal  
> > wrote:
> >
> > +1 to EOL 3.1.x at least.
> >
> >
> >> On May 23, 2021, at 9:51 PM, Wei-Chiu Chuang 
> >>  wrote:
> >>
> >> Sean,
> >>
> >> For reasons I don't understand, I never received emails from your new
> >> address in the mailing list. Only Akira's response.
> >>
> >> I was just able to start a thread like this.
> >>
> >> I am +1 to EOL 3.1.5.
> >> Reason? Spark is already on Hadoop 3.2. Hive and Tez are actively working
> >> to support Hadoop 3.3. HBase supports Hadoop 3.3 already. They are the most
> >> common Hadoop applications so I think a 3.1 isn't that necessarily
> >> important.
> >>
> >> With Hadoop 3.3.1, we have a number of improvements to support a better
> >> HDFS upgrade experience, so upgrading from Hadoop 3.1 should be relatively
> >> easy. Application upgrade takes some effort though (commons-lang ->
> >> commons-lang3 migration for example)
> >> I've been maintaining the HDFS code in branch-3.1, so from a
> >> HDFS perspective the branch is always in a ready to release state.
> >>
> >> The Hadoop 3.1 line is more than 3 years old. Maintaining this branch is
> >> getting trickier. I am +100 to reduce the number of actively maintained
> >> release line. IMO, 2 Hadoop 3 lines + 1 Hadoop 2 line is a good idea.
> >>
> >>
> >>
> >> For Hadoop 3.3 line: If no one beats me, I plan to make a 3.3.2 in 2-3
> >> months. And another one in another 2-3 months.
> >> The Hadoop 3.3.1 has nearly 700 commits not in 3.3.0. It is very difficult
> >> to make/validate a maint release with such a big divergence in the code.
> >>
> >>
> >>> On Mon, May 24, 2021 at 12:06 PM Akira Ajisaka  >>> > wrote:
> >>>
> >>> Hi Sean,
> >>>
> >>> Thank you for starting the discussion.
> >>>
> >>> I think branch-2.10, branch-3.1, branch-3.2, branch-3.3, and trunk
> >>> (3.4.x) are actively maintained.
> >>>
> >>> The next releases will be:
> >>> - 3.4.0
> >>> - 3.3.1 (Thanks, Wei-Chiu!)
> >>> - 3.2.3
> >>> - 3.1.5
> >>> - 2.10.2
> >>>
>  Are there folks willing to go through being release managers to get more
> >>> of these release lines on a steady cadence?
> >>>
> >>> Now I'm interested in becoming a release manager of 3.1.5.
> >>>
>  If I were to take up maintenance release for one of them which should it
> >>> be?
> >>>
> >>> 3.2.3 or 2.10.2 seems to be a good choice.
> >>>
>  Should we declare to our downstream users that some of these lines
> >>> aren’t going to get more releases?
> >>>
> >>> Now I think we don't need to declare that. I believe 3.3.1, 3.2.3,
> >>> 3.1.5, and 2.10.2 will be released in the near future.
> >>> There are some earlier discussions of 3.1.x EoL, so 3.1.5 may be a
> >>> final release of the 3.1.x release line.
> >>>
>  Is there downstream facing documentation somewhere that I missed for
> >>> setting expectations about our release cadence and actively maintained
> >>> branches?
> >>>
> >>> As you commented, the confluence wiki pages for Hadoop releases were
> >>> out of date. Updated [1].
> >>>
>  Do we have a backlog of work written up that could make the release
> >>> process easier for our release managers?
> >>>
> >>> The release process is documented and maintained:
> >>> https://cwiki.apache.org/confluence/display/HADOOP2/HowToRelease
> >>> Also, there are some backlogs [1], [2].
> >>>
> >>> [1]:
> >>> https://cwiki.apache.org/confluence/display/HADOOP/Hadoop+Active+Release+Lines
> >>> [2]: https://cwiki.apache.org/confluence/display/HADOOP/Roadmap
> >>>
> >>> Thanks,
> >>> Akira
> >>>
> >>> On Fri, May 21, 2021 at 7:12 AM Sean Busbey 
> >>> wrote:
> 
> 
>  Hi folks!
> 
>  Which release lines do we as a community still consider actively
> >>> maintained?
> 
>  I found an earlier discussion[1] where we had consensus to consider
> >>> branches that don’t get maintenance releases on a regular basis 
> >>> end-of-life
> >>> for practical purposes. The result of that discussion was written up in 
> >>> our
> >>> wiki docs in the “EOL Release Branches” page, summarized here
> 
> > If no volunteer to do a maintenance release in a short to mid-term
> >>> (like 3 months to 1 or 1.5 year).
> 
>  Looking at release lines that are still on our download page[3]:
> 
>  * Hadoop 2.10.z - last release 8 months ago
>  * Hadoop 3.1.z - last release 9.5 months ago
>  * Hadoop 3.2.z - last release 4.5 months ago
>  * Hadoop 3.3.z - last release 10 months ago
> 
>  And