[jira] [Created] (YARN-8620) [UI2] YARN Services UI new submission failures are not debuggable
Akhil PB created YARN-8620: -- Summary: [UI2] YARN Services UI new submission failures are not debuggable Key: YARN-8620 URL: https://issues.apache.org/jira/browse/YARN-8620 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Akhil PB Assignee: Akhil PB -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8619) Automate docker network configuration through YARN API
Eric Yang created YARN-8619:
---
Summary: Automate docker network configuration through YARN API
Key: YARN-8619
URL: https://issues.apache.org/jira/browse/YARN-8619
Project: Hadoop YARN
Issue Type: Sub-task
Components: yarn-native-services
Reporter: Eric Yang
Docker supports bridge, host, overlay, macvlan networking. It might be useful
to automate docker network setup through a set of YARN API to improve
management of docker networks. Each type of network driver requires different
type of parameters. For Hadoop use case, it seems more useful to focus on
macvlan networking for ease of use and configuration. It would be great
addition to support commands like:
{code}
yarn network create -d macvlan \
--subnet=172.16.86.0/24 \
--gateway=172.16.86.1 \
-o parent=eth0 \
my-macvlan-net
{code}
This changes docker configuration to hosts managed by YARN.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Apache Hadoop qbt Report: trunk+JDK8 on Windows/x64
For more details, see https://builds.apache.org/job/hadoop-trunk-win/546/ [Aug 1, 2018 3:51:40 PM] (billie) YARN-8403. Change the log level for fail to download resource from INFO [Aug 1, 2018 6:22:01 PM] (skumpf) YARN-8600. RegistryDNS hang when remote lookup does not reply. [Aug 1, 2018 7:32:01 PM] (arp) HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay [Aug 2, 2018 12:41:43 AM] (eyang) YARN-8610. Fixed initiate upgrade error message. [Aug 2, 2018 3:04:09 AM] (sunilg) YARN-8593. Add RM web service endpoint to get user information. [Aug 2, 2018 6:05:22 AM] (nanda) HDDS-310. VolumeSet shutdown hook fails on datanode restart. Contributed [Aug 2, 2018 7:11:06 AM] (sunilg) YARN-8594. [UI2] Display current logged in user. Contributed by Akhil [Aug 2, 2018 10:40:54 AM] (sunilg) YARN-8592. [UI2] rmip:port/ui2 endpoint shows a blank page in windows OS [Aug 2, 2018 12:04:17 PM] (msingh) HDDS-304. Process ContainerAction from datanode heartbeat in SCM. -1 overall The following subsystems voted -1: compile mvninstall pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc javac The following subsystems are considered long running: (runtime bigger than 1h 00m 00s) unit Specific tests: Failed junit tests : hadoop.crypto.key.kms.server.TestKMS hadoop.cli.TestAclCLI hadoop.cli.TestAclCLIWithPosixAclInheritance hadoop.cli.TestCacheAdminCLI hadoop.cli.TestCryptoAdminCLI hadoop.cli.TestDeleteCLI hadoop.cli.TestErasureCodingCLI hadoop.cli.TestHDFSCLI hadoop.cli.TestXAttrCLI hadoop.fs.contract.hdfs.TestHDFSContractAppend hadoop.fs.contract.hdfs.TestHDFSContractConcat hadoop.fs.contract.hdfs.TestHDFSContractCreate hadoop.fs.contract.hdfs.TestHDFSContractDelete hadoop.fs.contract.hdfs.TestHDFSContractGetFileStatus hadoop.fs.contract.hdfs.TestHDFSContractMkdir hadoop.fs.contract.hdfs.TestHDFSContractOpen hadoop.fs.contract.hdfs.TestHDFSContractPathHandle hadoop.fs.contract.hdfs.TestHDFSContractRename hadoop.fs.contract.hdfs.TestHDFSContractRootDirectory hadoop.fs.contract.hdfs.TestHDFSContractSeek hadoop.fs.contract.hdfs.TestHDFSContractSetTimes hadoop.fs.loadGenerator.TestLoadGenerator hadoop.fs.permission.TestStickyBit hadoop.fs.shell.TestHdfsTextCommand hadoop.fs.TestEnhancedByteBufferAccess hadoop.fs.TestFcHdfsCreateMkdir hadoop.fs.TestFcHdfsPermission hadoop.fs.TestFcHdfsSetUMask hadoop.fs.TestGlobPaths hadoop.fs.TestHDFSFileContextMainOperations hadoop.fs.TestHDFSMultipartUploader hadoop.fs.TestHdfsNativeCodeLoader hadoop.fs.TestResolveHdfsSymlink hadoop.fs.TestSWebHdfsFileContextMainOperations hadoop.fs.TestSymlinkHdfsDisable hadoop.fs.TestSymlinkHdfsFileContext hadoop.fs.TestSymlinkHdfsFileSystem hadoop.fs.TestUnbuffer hadoop.fs.TestUrlStreamHandler hadoop.fs.TestWebHdfsFileContextMainOperations hadoop.fs.viewfs.TestViewFileSystemAtHdfsRoot hadoop.fs.viewfs.TestViewFileSystemHdfs hadoop.fs.viewfs.TestViewFileSystemLinkFallback hadoop.fs.viewfs.TestViewFileSystemLinkMergeSlash hadoop.fs.viewfs.TestViewFileSystemWithAcls hadoop.fs.viewfs.TestViewFileSystemWithTruncate hadoop.fs.viewfs.TestViewFileSystemWithXAttrs hadoop.fs.viewfs.TestViewFsAtHdfsRoot hadoop.fs.viewfs.TestViewFsDefaultValue hadoop.fs.viewfs.TestViewFsFileStatusHdfs hadoop.fs.viewfs.TestViewFsHdfs hadoop.fs.viewfs.TestViewFsWithAcls hadoop.fs.viewfs.TestViewFsWithXAttrs hadoop.hdfs.client.impl.TestBlockReaderLocal hadoop.hdfs.client.impl.TestBlockReaderLocalLegacy hadoop.hdfs.client.impl.TestBlockReaderRemote hadoop.hdfs.client.impl.TestClientBlockVerification hadoop.hdfs.crypto.TestHdfsCryptoStreams hadoop.hdfs.protocol.datatransfer.sasl.TestSaslDataTransfer hadoop.hdfs.qjournal.client.TestEpochsAreUnique hadoop.hdfs.qjournal.client.TestQJMWithFaults hadoop.hdfs.qjournal.client.TestQuorumJournalManager hadoop.hdfs.qjournal.server.TestJournal hadoop.hdfs.qjournal.server.TestJournalNode hadoop.hdfs.qjournal.server.TestJournalNodeMXBean hadoop.hdfs.qjournal.server.TestJournalNodeRespectsBindHostKeys hadoop.hdfs.qjournal.server.TestJournalNodeSync hadoop.hdfs.qjournal.TestMiniJournalCluster hadoop.hdfs.qjournal.TestNNWithQJM hadoop.hdfs.qjournal.TestSecureNNWithQJM hadoop.hdfs.security.TestDelegationToken hadoop.hdfs.security.TestDelegationTokenForProxyUser hadoop.hdfs.security.token.block.TestBlockToken hadoop.hdfs.server.balancer.TestBalancer hadoop.hdfs.serve
[jira] [Created] (YARN-8618) Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail
Chandni Singh created YARN-8618: --- Summary: Yarn Service: When all the components of a service have restart policy NEVER then initiation of service upgrade should fail Key: YARN-8618 URL: https://issues.apache.org/jira/browse/YARN-8618 Project: Hadoop YARN Issue Type: Improvement Reporter: Chandni Singh Assignee: Chandni Singh -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Resolved] (YARN-8587) Delays are noticed to launch docker container
[
https://issues.apache.org/jira/browse/YARN-8587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eric Yang resolved YARN-8587.
-
Resolution: Not A Bug
> Delays are noticed to launch docker container
> -
>
> Key: YARN-8587
> URL: https://issues.apache.org/jira/browse/YARN-8587
> Project: Hadoop YARN
> Issue Type: Bug
>Affects Versions: 3.1.1
>Reporter: Yesha Vora
>Priority: Major
> Labels: Docker
>
> Launch dshell application. Wait for application to go in RUNNING state.
> {code:java}
> yarn jar /xx/hadoop-yarn-applications-distributedshell-*.jar -shell_command
> "sleep 300" -num_containers 1 -shell_env YARN_CONTAINER_RUNTIME_TYPE=docker
> -shell_env YARN_CONTAINER_RUNTIME_DOCKER_IMAGE=httpd:0.1 -shell_env
> YARN_CONTAINER_RUNTIME_DOCKER_DELAYED_REMOVAL=true -jar
> /usr/hdp/current/hadoop-yarn-client/hadoop-yarn-applications-distributedshell-xx.jar
> {code}
> Find out container allocation. Run docker inspect command for docker
> containers launched by app.
> Sometimes, the container is allocated to NM but docker PID is not up.
> {code:java}
> Command ssh -q -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null
> xxx "sudo su - -c \"docker ps -a | grep
> container_e02_1531189225093_0003_01_02\" root" failed after 0 retries
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8617) Aggregated Application Logs accumulates for long running jobs
Prabhu Joseph created YARN-8617: --- Summary: Aggregated Application Logs accumulates for long running jobs Key: YARN-8617 URL: https://issues.apache.org/jira/browse/YARN-8617 Project: Hadoop YARN Issue Type: New Feature Components: log-aggregation Affects Versions: 2.7.4 Reporter: Prabhu Joseph Currently AggregationDeletionService will delete older aggregated log files once when they are complete. This will cause logs to accumulate for Long Running Jobs like Llap, Spark Streaming. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[VOTE] Release Apache Hadoop 3.1.1 - RC0
Hi folks, I've created RC0 for Apache Hadoop 3.1.1. The artifacts are available here: http://people.apache.org/~wangda/hadoop-3.1.1-RC0/ The RC tag in git is release-3.1.1-RC0: https://github.com/apache/hadoop/commits/release-3.1.1-RC0 The maven artifacts are available via repository.apache.org at https://repository.apache.org/content/repositories/orgapachehadoop-1139/ You can find my public key at http://svn.apache.org/repos/asf/hadoop/common/dist/KEYS This vote will run 5 days from now. 3.1.1 contains 435 [1] fixed JIRA issues since 3.1.0. I have done testing with a pseudo cluster and distributed shell job. My +1 to start. Best, Wangda Tan [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.1.1) ORDER BY priority DESC
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/ [Aug 1, 2018 1:44:49 AM] (fabbri) HDFS-13322 fuse dfs - uid persists when switching between ticket caches. [Aug 1, 2018 3:03:30 AM] (sunilg) YARN-8397. Potential thread leak in ActivitiesManager. Contributed by [Aug 1, 2018 3:33:00 AM] (msingh) HDDS-226. Client should update block length in OM while committing the [Aug 1, 2018 5:34:53 AM] (wangda) YARN-8522. Application fails with InvalidResourceRequestException. (Zian [Aug 1, 2018 6:47:18 AM] (sunilg) YARN-8606. Opportunistic scheduling does not work post RM failover. [Aug 1, 2018 8:57:54 AM] (sunilg) YARN-8595. [UI2] Container diagnostic information is missing from [Aug 1, 2018 3:51:40 PM] (billie) YARN-8403. Change the log level for fail to download resource from INFO [Aug 1, 2018 6:22:01 PM] (skumpf) YARN-8600. RegistryDNS hang when remote lookup does not reply. [Aug 1, 2018 7:32:01 PM] (arp) HADOOP-15476. fix logging for split-dns multihome . Contributed by Ajay [Aug 2, 2018 12:41:43 AM] (eyang) YARN-8610. Fixed initiate upgrade error message. -1 overall The following subsystems voted -1: asflicense findbugs pathlen unit xml The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed CTEST tests : test_test_libhdfs_threaded_hdfs_static test_libhdfs_threaded_hdfspp_test_shim_static Failed junit tests : hadoop.security.TestRaceWhenRelogin hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.mapred.TestMRTimelineEventHandling cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-compile-javac-root.txt [332K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-checkstyle-root.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-patch-pylint.txt [24K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-patch-shellcheck.txt [20K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-patch-shelldocs.txt [16K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/whitespace-eol.txt [9.4M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/whitespace-tabs.txt [1.1M] xml: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/xml.txt [4.0K] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-hdds_client.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [52K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [12K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [56K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [16K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_client.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_common.txt [28K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_ozonefs.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/branch-findbugs-hadoop-ozone_tools.txt [4.0K] javadoc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/diff-javadoc-javadoc-root.txt [760K] CTEST: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/856/artifact/out/patch-hadoop-hdfs-project_hadoop-hdfs-native-client-ctest.txt [116K] unit: https://builds.apache.org/job/
[jira] [Created] (YARN-8616) System.currentTimeMillis() used in RMAppImpl, instead of getting value from systemClock
Szilard Nemeth created YARN-8616: Summary: System.currentTimeMillis() used in RMAppImpl, instead of getting value from systemClock Key: YARN-8616 URL: https://issues.apache.org/jira/browse/YARN-8616 Project: Hadoop YARN Issue Type: Improvement Reporter: Szilard Nemeth Assignee: Szilard Nemeth -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8615) [UI2] Resource Usage tab shows only memory related info. Drop Down box disappears to select vcores/gpu
Akhil PB created YARN-8615: -- Summary: [UI2] Resource Usage tab shows only memory related info. Drop Down box disappears to select vcores/gpu Key: YARN-8615 URL: https://issues.apache.org/jira/browse/YARN-8615 Project: Hadoop YARN Issue Type: Bug Components: yarn-ui-v2 Reporter: Akhil PB Assignee: Akhil PB -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-8614) Some typos in YarnConfiguration
Sen Zhao created YARN-8614: -- Summary: Some typos in YarnConfiguration Key: YARN-8614 URL: https://issues.apache.org/jira/browse/YARN-8614 Project: Hadoop YARN Issue Type: Bug Reporter: Sen Zhao Fix some typos in comments. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
