[jira] [Created] (HADOOP-16267) Performance gain if you use replace() instead of replaceAll() for replacing patterns that do not use a regex
bd2019us created HADOOP-16267: - Summary: Performance gain if you use replace() instead of replaceAll() for replacing patterns that do not use a regex Key: HADOOP-16267 URL: https://issues.apache.org/jira/browse/HADOOP-16267 Project: Hadoop Common Issue Type: Improvement Reporter: bd2019us Performance gain if you use replace() instead of replaceAll() for replacing patterns that do not use a regex. This happens because replace() does not need to compile the regex pattern like replaceAll() does. Affected files: * hadoop-common/hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/mapred/CopyCommitter.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/state/Graph.java * hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/java/org/apache/hadoop/yarn/state/Graph.java -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Re: VOTE: Hadoop Ozone 0.4.0-alpha RC1
+1 (non-binding) - build from source - run all the smoketest (from the fresh build based on the src package) - run all the smoketest from the binary package - signature files are checked - sha512 checksums are verified - ozone version shows the right commit information Thanks Ajay all the releasing work, Marton ps: used archlinux, java 8, docker-compose 1.23.2, docker 18.09.4-ce On 4/20/19 3:24 PM, Xiaoyu Yao wrote: > > +1 (binding) > > - Build from source > - Misc security tests with docker compose > - MR and Spark sample jobs with secure ozone cluster > > —Xiaoyu > >> On Apr 19, 2019, at 3:40 PM, Anu Engineer >> wrote: >> >> +1 (Binding) >> >> -- Verified the checksums. >> -- Built from sources. >> -- Sniff tested the functionality. >> >> --Anu >> >> >> On Mon, Apr 15, 2019 at 4:09 PM Ajay Kumar >> wrote: >> >>> Hi all, >>> >>> We have created the second release candidate (RC1) for Apache Hadoop Ozone >>> 0.4.0-alpha. >>> >>> This release contains security payload for Ozone. Below are some important >>> features in it: >>> >>> * Hadoop Delegation Tokens and Block Tokens supported for Ozone. >>> * Transparent Data Encryption (TDE) Support - Allows data blocks to be >>> encrypted-at-rest. >>> * Kerberos support for Ozone. >>> * Certificate Infrastructure for Ozone - Tokens use PKI instead of >>> shared secrets. >>> * Datanode to Datanode communication secured via mutual TLS. >>> * Ability secure ozone cluster that works with Yarn, Hive, and Spark. >>> * Skaffold support to deploy Ozone clusters on K8s. >>> * Support S3 Authentication Mechanisms like - S3 v4 Authentication >>> protocol. >>> * S3 Gateway supports Multipart upload. >>> * S3A file system is tested and supported. >>> * Support for Tracing and Profiling for all Ozone components. >>> * Audit Support - including Audit Parser tools. >>> * Apache Ranger Support in Ozone. >>> * Extensive failure testing for Ozone. >>> >>> The RC artifacts are available at >>> https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc1 >>> >>> The RC tag in git is ozone-0.4.0-alpha-RC1 (git hash >>> d673e16d14bb9377f27c9017e2ffc1bcb03eebfb) >>> >>> Please try out< >>> https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, >>> vote, or just give us feedback. >>> >>> The vote will run for 5 days, ending on April 20, 2019, 19:00 UTC. >>> >>> Thank you very much, >>> >>> Ajay >>> >>> >>> > > - > To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org > For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org > - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/ [Apr 19, 2019 4:41:06 PM] (inigoiri) YARN-9448. Fix Opportunistic Scheduling for node local allocations. [Apr 19, 2019 5:29:57 PM] (billie) YARN-9254. Add support for storing application catalog data to HDFS. -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager Null passed for non-null parameter of com.google.common.util.concurrent.SettableFuture.set(Object) in org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore, RMStateStoreEvent) At RMStateStore.java:of com.google.common.util.concurrent.SettableFuture.set(Object) in org.apache.hadoop.yarn.server.resourcemanager.recovery.RMStateStore$UpdateAppTransition.transition(RMStateStore, RMStateStoreEvent) At RMStateStore.java:[line 291] Null passed for non-null parameter of com.google.common.util.concurrent.SettableFuture.set(Object) in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updateApplicationPriority(Priority, ApplicationId, SettableFuture, UserGroupInformation) At CapacityScheduler.java:of com.google.common.util.concurrent.SettableFuture.set(Object) in org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.updateApplicationPriority(Priority, ApplicationId, SettableFuture, UserGroupInformation) At CapacityScheduler.java:[line 2650] FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-documentstore Unread field:TimelineEventSubDoc.java:[line 56] Unread field:TimelineMetricSubDoc.java:[line 44] Failed junit tests : hadoop.util.TestDiskChecker hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 hadoop.yarn.applications.distributedshell.TestDistributedShell hadoop.mapreduce.v2.app.TestRuntimeEstimators hadoop.hdds.scm.net.TestNodeSchemaManager hadoop.hdds.scm.net.TestNetworkTopologyImpl hadoop.ozone.TestOzoneConfigurationFields hadoop.ozone.client.rpc.TestBlockOutputStreamWithFailures hadoop.ozone.scm.node.TestSCMNodeMetrics cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-compile-javac-root.txt [336K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-patch-pylint.txt [84K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-patch-shelldocs.txt [44K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/whitespace-eol.txt [9.6M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/whitespace-tabs.txt [1.1M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-resourcemanager-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-documentstore-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/branch-findbugs-hadoop-submarine_hadoop-submarine-yarnservice-runtime.txt [4.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/diff-javadoc-javadoc-root.txt [752K] unit: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/1112/artifact/out/patch-unit-hadoop-common-project_hadoop-common.txt [184K]
Re: VOTE: Hadoop Ozone 0.4.0-alpha RC1
+1 (binding) - Build from source - Misc security tests with docker compose - MR and Spark sample jobs with secure ozone cluster —Xiaoyu > On Apr 19, 2019, at 3:40 PM, Anu Engineer > wrote: > > +1 (Binding) > > -- Verified the checksums. > -- Built from sources. > -- Sniff tested the functionality. > > --Anu > > > On Mon, Apr 15, 2019 at 4:09 PM Ajay Kumar > wrote: > >> Hi all, >> >> We have created the second release candidate (RC1) for Apache Hadoop Ozone >> 0.4.0-alpha. >> >> This release contains security payload for Ozone. Below are some important >> features in it: >> >> * Hadoop Delegation Tokens and Block Tokens supported for Ozone. >> * Transparent Data Encryption (TDE) Support - Allows data blocks to be >> encrypted-at-rest. >> * Kerberos support for Ozone. >> * Certificate Infrastructure for Ozone - Tokens use PKI instead of >> shared secrets. >> * Datanode to Datanode communication secured via mutual TLS. >> * Ability secure ozone cluster that works with Yarn, Hive, and Spark. >> * Skaffold support to deploy Ozone clusters on K8s. >> * Support S3 Authentication Mechanisms like - S3 v4 Authentication >> protocol. >> * S3 Gateway supports Multipart upload. >> * S3A file system is tested and supported. >> * Support for Tracing and Profiling for all Ozone components. >> * Audit Support - including Audit Parser tools. >> * Apache Ranger Support in Ozone. >> * Extensive failure testing for Ozone. >> >> The RC artifacts are available at >> https://home.apache.org/~ajay/ozone-0.4.0-alpha-rc1 >> >> The RC tag in git is ozone-0.4.0-alpha-RC1 (git hash >> d673e16d14bb9377f27c9017e2ffc1bcb03eebfb) >> >> Please try out< >> https://cwiki.apache.org/confluence/display/HADOOP/Running+via+Apache+Release>, >> vote, or just give us feedback. >> >> The vote will run for 5 days, ending on April 20, 2019, 19:00 UTC. >> >> Thank you very much, >> >> Ajay >> >> >>
Apache Hadoop qbt Report: branch2+JDK7 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/ No changes -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-build-tools/src/main/resources/checkstyle/checkstyle.xml hadoop-build-tools/src/main/resources/checkstyle/suppressions.xml hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/conf/empty-configuration.xml hadoop-tools/hadoop-azure/src/config/checkstyle.xml hadoop-tools/hadoop-resourceestimator/src/config/checkstyle.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/public/crossdomain.xml hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-common-project/hadoop-common Class org.apache.hadoop.fs.GlobalStorageStatistics defines non-transient non-serializable instance field map In GlobalStorageStatistics.java:instance field map In GlobalStorageStatistics.java FindBugs : module:hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-yarn-server-timelineservice-hbase-client Boxed value is unboxed and then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:then immediately reboxed in org.apache.hadoop.yarn.server.timelineservice.storage.common.ColumnRWHelper.readResultsWithTimestamps(Result, byte[], byte[], KeyConverter, ValueConverter, boolean) At ColumnRWHelper.java:[line 335] Failed junit tests : hadoop.ha.TestZKFailoverController hadoop.hdfs.server.namenode.TestNameNodeHttpServerXFrame hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.hdfs.TestRollingUpgrade hadoop.contrib.bkjournal.TestBookKeeperJournalManager hadoop.contrib.bkjournal.TestBookKeeperJournalManager hadoop.yarn.sls.TestSLSRunner hadoop.registry.secure.TestSecureLogins hadoop.yarn.server.timelineservice.security.TestTimelineAuthFilterForV2 cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-compile-cc-root-jdk1.7.0_95.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-compile-javac-root-jdk1.7.0_95.txt [328K] cc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-compile-cc-root-jdk1.8.0_191.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-compile-javac-root-jdk1.8.0_191.txt [308K] checkstyle: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-checkstyle-root.txt [16M] hadolint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-patch-shellcheck.txt [72K] shelldocs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-patch-shelldocs.txt [8.0K] whitespace: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/whitespace-eol.txt [12M] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/whitespace-tabs.txt [1.2M] xml: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/xml.txt [20K] findbugs: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/branch-findbugs-hadoop-common-project_hadoop-common-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/branch-findbugs-hadoop-yarn-project_hadoop-yarn_hadoop-yarn-server_hadoop-yarn-server-timelineservice-hbase_hadoop-yarn-server-timelineservice-hbase-client-warnings.html [8.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-branch2-java7-linux-x86/297/artifact/out/diff-javadoc-javadoc-root-jdk1.7.0_95.txt [16K]
