Apache Hadoop qbt Report: branch2.10+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch-2.10-java7-linux-x86/740/ No changes -1 overall The following subsystems voted -1: docker Powered by Apache Yetushttps://yetus.apache.org - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[VOTE] Release Apache Hadoop 3.3.0 - RC0
Hi folks, This is the first release candidate for the first release of Apache Hadoop 3.3.0 line. It contains *1644[1]* fixed jira issues since 3.2.1 which include a lot of features and improvements(read the full set of release notes). Below feature additions are the highlights of the release. - ARM Support - Enhancements and new features on S3a,S3Guard,ABFS - Java 11 Runtime support and TLS 1.3. - Support Tencent Cloud COS File System. - Added security to HDFS Router. - Support non-volatile storage class memory(SCM) in HDFS cache directives - Support Interactive Docker Shell for running Containers. - Scheduling of opportunistic containers - A pluggable device plugin framework to ease vendor plugin development *The RC0 artifacts are at*: http://home.apache.org/~brahma/Hadoop-3.3.0-RC0/ *First release to include ARM binary, Have a check.* *RC tag is *release-3.3.0-RC0. *The maven artifacts are hosted here:* https://repository.apache.org/content/repositories/orgapachehadoop-1271/ *My public key is available here:* https://dist.apache.org/repos/dist/release/hadoop/common/KEYS The vote will run for 5 weekdays, until Tuesday, July 13 at 3:50 AM IST. I have done a few testing with my pseudo cluster. My +1 to start. Regards, Brahma Reddy Battula 1. project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.3.0) AND fixVersion not in (3.2.0, 3.2.1, 3.1.3) AND status = Resolved ORDER BY fixVersion ASC
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86_64
For more details, see https://ci-hadoop.apache.org/job/hadoop-qbt-trunk-java8-linux-x86_64/195/ No changes [Error replacing 'FILE' - Workspace is not accessible] - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Add Guava common packages/classes to illegal imports
Hi folks, I believe we need prevent future commits from re-introducing Guava classes This can be achieved by two options: 1. We add rules on the go. Each time we add an illegal import to the class that has been replaced. This won't prevent developers from committing code that uses Guava lasses. The disadvantage, that it makes the task a moving target somehow. Another disadvantage in that approach is an increased conflict of code changes because each patch has to append to the illegal-imports rules. 2. Add rules to warn against Guava packages. This will prevent introducing Guava imports. Both options are not bullet-proof to prevent future usage of Guava through the code because the checkstyle illegal import does not cover API usage. Does anyone have concerns adding those rules to checkstyle configurations? And which option should we go for? -- Best Regards, *Ahmed Hussein, PhD*
[jira] [Resolved] (HADOOP-17081) MetricsSystem doesn't start the sink adapters on restart
[ https://issues.apache.org/jira/browse/HADOOP-17081?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved HADOOP-17081. - Fix Version/s: 3.3.1 3.2.2 Resolution: Fixed +1 -committed to branches 3.2, 3.3 and trunk. Thanks! > MetricsSystem doesn't start the sink adapters on restart > > > Key: HADOOP-17081 > URL: https://issues.apache.org/jira/browse/HADOOP-17081 > Project: Hadoop Common > Issue Type: Bug > Components: metrics > Environment: NA >Reporter: Madhusoodan >Assignee: Madhusoodan >Priority: Minor > Fix For: 3.2.2, 3.3.1 > > > In HBase we use dynamic metrics and when a metric is removed, we have to > refresh the JMX beans, since there is no API from Java to do it, a hack like > stopping the metrics system and restarting it was used (Read the comment on > the class > [https://github.com/mmpataki/hbase/blob/master/hbase-hadoop-compat/src/main/java/org/apache/hadoop/metrics2/impl/JmxCacheBuster.java]) > > It calls the below APIs in the same order > MetricsSystem.stop > MetricsSystem.start > > MetricsSystem.stop stops all the SinkAdapters, *but doesn't remove them from > the sink list* (allSinks is the variable). When the metrics system is started > again, *it is assumed that the SinkAdapters are restarted, but they are not* > due to the check done in the beginning of the function register. -- This message was sent by Atlassian Jira (v8.3.4#803005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: [VOTE] Release Apache Hadoop 3.1.4 (RC2)
hmm YARN-9341 went through all of the yarn lock code -it's in 3.3 but not in 3.1. And we do not want to attempt to backport 175KB of lock acquire/release code, do we? anyone in yarn-dev got any thoughts here? On Sun, 5 Jul 2020 at 22:14, Masatake Iwasaki wrote: > Thanks for putting this up, Gabor Bota. > > I'm testing the RC2 on 3 node docker cluster with NN-HA and RM-HA enabled. > ResourceManager reproducibly blocks on submitApplication while launching > example MR jobs. > Does anyone run into the same issue? > > The same configuration worked for 3.1.3. > I got no issue if RM-HA is disabled. > > > "IPC Server handler 1 on default port 8032" #167 daemon prio=5 os_prio=0 > tid=0x7fe91821ec50 nid=0x3b9 waiting on condition [0x7fe901bac000] > java.lang.Thread.State: WAITING (parking) > at sun.misc.Unsafe.park(Native Method) > - parking to wait for <0x85d37a40> (a > java.util.concurrent.locks.ReentrantReadWriteLock$NonfairSync) > at > java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) > at > > java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836) > at > > java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireShared(AbstractQueuedSynchronizer.java:967) > at > > java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireShared(AbstractQueuedSynchronizer.java:1283) > at > > java.util.concurrent.locks.ReentrantReadWriteLock$ReadLock.lock(ReentrantReadWriteLock.java:727) > at > > org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.checkAndGetApplicationPriority(CapacityScheduler.java:2521) > at > > org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.createAndPopulateNewRMApp(RMAppManager.java:417) > at > > org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.submitApplication(RMAppManager.java:342) > at > > org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.submitApplication(ClientRMService.java:678) > at > > org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.submitApplication(ApplicationClientProtocolPBServiceImpl.java:277) > at > > org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:563) > at > > org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:527) > at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1036) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:1015) > at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:943) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:422) > at > > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729) > at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2943) > > > Masatake Iwasaki > > On 2020/06/26 22:51, Gabor Bota wrote: > > Hi folks, > > > > I have put together a release candidate (RC2) for Hadoop 3.1.4. > > > > The RC is available at: > http://people.apache.org/~gabota/hadoop-3.1.4-RC2/ > > The RC tag in git is here: > > https://github.com/apache/hadoop/releases/tag/release-3.1.4-RC2 > > The maven artifacts are staged at > > https://repository.apache.org/content/repositories/orgapachehadoop-1269/ > > > > You can find my public key at: > > https://dist.apache.org/repos/dist/release/hadoop/common/KEYS > > and http://keys.gnupg.net/pks/lookup?op=get&search=0xB86249D83539B38C > > > > Please try the release and vote. The vote will run for 5 weekdays, > > until July 6. 2020. 23:00 CET. > > > > The release includes the revert of HDFS-14941, as it caused > > HDFS-15421. IBR leak causes standby NN to be stuck in safe mode. > > (https://issues.apache.org/jira/browse/HDFS-15421) > > The release includes HDFS-15323, as requested. > > (https://issues.apache.org/jira/browse/HDFS-15323) > > > > Thanks, > > Gabor > > > > - > > To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org > > For additional commands, e-mail: common-dev-h...@hadoop.apache.org > > >
