[jira] [Created] (HADOOP-15887) Add an option to avoid writing data locally in Distcp
Tao Jie created HADOOP-15887: Summary: Add an option to avoid writing data locally in Distcp Key: HADOOP-15887 URL: https://issues.apache.org/jira/browse/HADOOP-15887 Project: Hadoop Common Issue Type: Improvement Affects Versions: 3.0.0, 2.8.2 Reporter: Tao Jie Assignee: Tao Jie When copying large amount of data from one cluster to another via Distcp, and the Distcp jobs run in the target cluster, the datanode local usage would be imbalanced. Because the default placement policy chooses the local node to store the first replication. In https://issues.apache.org/jira/browse/HDFS-3702 we add a flag in DFSClient to avoid replicating to the local datanode. We can make use of this flag in Distcp. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
[jira] [Created] (HADOOP-15885) Add base64 (urlString) support to DTUtil
Íñigo Goiri created HADOOP-15885: Summary: Add base64 (urlString) support to DTUtil Key: HADOOP-15885 URL: https://issues.apache.org/jira/browse/HADOOP-15885 Project: Hadoop Common Issue Type: New Feature Reporter: Íñigo Goiri HADOOP-12563 added a utility to manage Delegation Token files. Currently, it supports Java and Protobuf formats. However, When interacting with WebHDFS, we use base64. In addition, when printing a token, we also print the base64 value. We should be able to import base64 tokens in the utility. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/ [Oct 29, 2018 12:04:59 AM] (weichiu) Revert "HADOOP-15864. Job submitter / executor fail when SBN domain name -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: XML : Parsing Error(s): hadoop-yarn-project/hadoop-yarn/hadoop-yarn-ui/src/main/webapp/public/crossdomain.xml FindBugs : module:hadoop-common-project/hadoop-registry Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOTCP(InetAddress, int) At RegistryDNS.java:[line 900] Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.addNIOUDP(InetAddress, int) At RegistryDNS.java:[line 926] Exceptional return value of java.util.concurrent.ExecutorService.submit(Callable) ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel, InetAddress, int) At RegistryDNS.java:ignored in org.apache.hadoop.registry.server.dns.RegistryDNS.serveNIOTCP(ServerSocketChannel, InetAddress, int) At RegistryDNS.java:[line 850] Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.hdfs.server.blockmanagement.TestBlockTokenWithDFSStriped hadoop.yarn.server.nodemanager.containermanager.TestContainerManager hadoop.yarn.server.resourcemanager.TestApplicationMasterServiceCapacity hadoop.yarn.server.resourcemanager.scheduler.capacity.TestContainerAllocation hadoop.yarn.client.api.impl.TestAMRMClient hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage hadoop.yarn.sls.appmaster.TestAMSimulator cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-compile-javac-root.txt [324K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-pylint.txt [40K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-shellcheck.txt [68K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-common-project_hadoop-registry-warnings.html [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_client.txt [24K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/941/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K]
[jira] [Reopened] (HADOOP-15339) Support additional key/value propereties in JMX bean registration
[ https://issues.apache.org/jira/browse/HADOOP-15339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Elek, Marton reopened HADOOP-15339: --- Since the commit we use this change from ozone/hdds and it worked well. This change is required to have a working ozone/hdds webui as the shared code path tags the common jmx beans with generic key/value tags. I reopen this issue and propose to backport it to branch-3.1 to make it easier to use hdds/ozone with older hadoop versions. # It's a small change # Backward compatible # Safe to use (no issue during the last 6 months) # No conflicts for cherry-pick. > Support additional key/value propereties in JMX bean registration > - > > Key: HADOOP-15339 > URL: https://issues.apache.org/jira/browse/HADOOP-15339 > Project: Hadoop Common > Issue Type: Improvement > Components: common >Reporter: Elek, Marton >Assignee: Elek, Marton >Priority: Major > Fix For: 3.2.0 > > Attachments: HADOOP-15339.001.patch, HADOOP-15339.002.patch, > HADOOP-15339.003.patch > > > org.apache.hadoop.metrics2.util.MBeans.register is a utility function to > register objects to the JMX registry with a given name prefix and name. > JMX supports any additional key value pairs which could be part the the > address of the jmx bean. For example: > _java.lang:type=MemoryManager,name=CodeCacheManager_ > Using this method we can query a group of mbeans, for example we can add the > same tag to similar mbeans from namenode and datanode. > This patch adds a small modification to support custom key value pairs and > also introduce a new unit test for MBeans utility which was missing until now. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
Apache Hadoop 2.9.2 release plan
Hi all, We have released Apache Hadoop 2.9.1 on May 5, 2018. To further improve the quality of the release, I'm planning to release 2.9.2. The focus of 2.9.2 will be fixing blocker/critical bugs and other enhancements. So far there are 189 JIRAs [1] have fixed in branch-2.9. There are no blocker/critical bugs targeted for 2.9.2 now [2], so I plan to cut branch-2.9.2 and create RC by the end of this week. If someone want to be a release manager for 2.9.2. You can do it instead of me, and I'll help you. Please feel free to share your thoughts. Thanks, Akira Ajisaka [1] https://s.apache.org/2.9.2-fixed-jiras [2] https://s.apache.org/2.9.2-targeted-jiras - To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: common-dev-h...@hadoop.apache.org
