[jira] [Created] (YARN-9032) Support sh shell for interactive shell
Eric Yang created YARN-9032: --- Summary: Support sh shell for interactive shell Key: YARN-9032 URL: https://issues.apache.org/jira/browse/YARN-9032 Project: Hadoop YARN Issue Type: Sub-task Components: yarn Reporter: Eric Yang CLI can supply second parameter such as: yarn container -shell [container-id] sh This is supposed to launch sh instead of bash. This is not fully working yet in YARN-8778 patch and some organization of code clean up to support non-docker container in YARN-8962 making this change in YARN-8778 difficult. Hence, I file this as a separate sub-task to close the gap. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
Re: Apache Hadoop 3.1.2 release plan
Just cleaned up all tickets and updated tickets which have target version == 3.1.2 to 3.1.3. I will roll an RC next Monday, if you have any tickets need to be released in 3.1.2, please let me know. Thanks, Wangda On Wed, Oct 24, 2018 at 7:30 PM Vinod Kumar Vavilapalli wrote: > 231 fixed JIRAs is already quite a bunch! > > I only see 7 JIRAs marked with Affects Version 3.1.2 and only one of them > as blocker. > > Why not just release now as soon as there are no blockers? > > Thanks > +Vinod > > > On Oct 24, 2018, at 4:36 PM, Wangda Tan wrote: > > > > Hi, All > > > > We have released Apache Hadoop 3.1.1 on Aug 8, 2018. To further > > improve the quality of the release, I plan to release 3.1.2 > > by Nov. The focus of 3.1.2 will be fixing blockers / critical bugs > > and other enhancements. So far there are 231 JIRAs [1] have fix > > version marked to 3.1.2 > > > > I plan to cut branch-3.1 on Nov 15 and vote for RC on the same day. > > > > Please feel free to share your insights. > > > > Thanks, > > Wangda Tan > > > > [1] project in (YARN, "Hadoop HDFS", "Hadoop Common", "Hadoop > Map/Reduce") > > AND fixVersion = 3.1.2 > >
Apache Hadoop qbt Report: trunk+JDK8 on Linux/x86
For more details, see https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/ [Nov 15, 2018 7:30:24 AM] (aajisaka) HADOOP-15926. Document upgrading the section in NOTICE.txt when [Nov 15, 2018 10:08:48 AM] (elek) HDDS-832. Docs folder is missing from the Ozone distribution package. [Nov 15, 2018 11:22:16 AM] (elek) HDDS-658. Implement s3 bucket list backend call and use it from rest [Nov 15, 2018 12:59:56 PM] (elek) HDDS-528. add cli command to checkChill mode status and exit chill mode. [Nov 15, 2018 1:04:55 PM] (elek) HDDS-828. Fix deprecation log generated by getting value of the setting [Nov 15, 2018 1:23:24 PM] (elek) HDDS-827. TestStorageContainerManagerHttpServer should use dynamic port. [Nov 15, 2018 2:18:07 PM] (elek) HDDS-223. Create acceptance test for using datanode plugin. Contributed [Nov 15, 2018 5:25:25 PM] (inigoiri) YARN-8856. TestTimelineReaderWebServicesHBaseStorage tests failing with [Nov 15, 2018 5:29:14 PM] (inigoiri) HDFS-14054. TestLeaseRecovery2: [Nov 15, 2018 6:58:57 PM] (inigoiri) HDFS-14045. Use different metrics in DataNode to better measure latency [Nov 15, 2018 8:42:31 PM] (arp) HADOOP-15936. [JDK 11] MiniDFSClusterManager & MiniHadoopClusterManager [Nov 15, 2018 9:58:13 PM] (arp) HADOOP-12558. distcp documentation is woefully out of date. Contributed [Nov 15, 2018 10:21:42 PM] (bharat) HDDS-821. Handle empty x-amz-storage-class header in Ozone S3 gateway. [Nov 15, 2018 10:54:41 PM] (gifuma) HDDS-843. [JDK11] Fix Javadoc errors in hadoop-hdds-server-scm module. [Nov 15, 2018 10:59:31 PM] (gifuma) HDDS-842. [JDK11] Fix Javadoc errors in hadoop-hdds-common module. [Nov 16, 2018 1:36:09 AM] (aengineer) HDDS-825. Code cleanup based on messages from ErrorProne. Contributed by -1 overall The following subsystems voted -1: asflicense findbugs hadolint pathlen unit The following subsystems voted -1 but were configured to be filtered/ignored: cc checkstyle javac javadoc pylint shellcheck shelldocs whitespace The following subsystems are considered long running: (runtime bigger than 1h 0m 0s) unit Specific tests: Failed junit tests : hadoop.hdfs.web.TestWebHdfsTimeouts hadoop.mapreduce.jobhistory.TestEvents hadoop.yarn.sls.TestSLSRunner cc: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-compile-cc-root.txt [4.0K] javac: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-compile-javac-root.txt [324K] checkstyle: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-checkstyle-root.txt [17M] hadolint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-patch-hadolint.txt [4.0K] pathlen: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/pathlen.txt [12K] pylint: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-patch-pylint.txt [40K] shellcheck: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-patch-shellcheck.txt [68K] shelldocs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/diff-patch-shelldocs.txt [12K] whitespace: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/whitespace-eol.txt [9.3M] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/whitespace-tabs.txt [1.1M] findbugs: https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-hdds_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-hdds_container-service.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-hdds_framework.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-hdds_server-scm.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-hdds_tools.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-ozone_client.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-ozone_common.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-ozone_objectstore-service.txt [8.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoop-ozone_ozone-manager.txt [4.0K] https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-x86/959/artifact/out/branch-findbugs-hadoo
Re: [VOTE] Release Apache Hadoop Ozone 0.3.0-alpha (RC1)
+1 (non-binding). - Verified signatures - Verified checksums - Checked LICENSE/NOTICE files - Built from source - Ran smoke tests. Thanks Marton for putting up the release together. Thanks Shashi On 11/14/18, 10:44 PM, "Elek, Marton" wrote: Hi all, I've created the second release candidate (RC1) for Apache Hadoop Ozone 0.3.0-alpha including one more fix on top of the previous RC0 (HDDS-854) This is the second release of Apache Hadoop Ozone. Notable changes since the first release: * A new S3 compatible rest server is added. Ozone can be used from any S3 compatible tools (HDDS-434) * Ozone Hadoop file system URL prefix is renamed from o3:// to o3fs:// (HDDS-651) * Extensive testing and stability improvements of OzoneFs. * Spark, YARN and Hive support and stability improvements. * Improved Pipeline handling and recovery. * Separated/dedicated classpath definitions for all the Ozone components. (HDDS-447) The RC artifacts are available from: https://home.apache.org/~elek/ozone-0.3.0-alpha-rc1/ The RC tag in git is: ozone-0.3.0-alpha-RC1 (ebbf459e6a6) Please try it out, vote, or just give us feedback. The vote will run for 5 days, ending on November 19, 2018 18:00 UTC. Thank you very much, Marton PS: The easiest way to try it out is: 1. Download the binary artifact 2. Read the docs from ./docs/index.html 3. TLDR; cd compose/ozone && docker-compose up -d 4. open localhost:9874 or localhost:9876 The easiest way to try it out from the source: 1. mvn install -DskipTests -Pdist -Dmaven.javadoc.skip=true -Phdds -DskipShade -am -pl :hadoop-ozone-dist 2. cd hadoop-ozone/dist/target/ozone-0.3.0-alpha && docker-compose up -d The easiest way to test basic functionality (with acceptance tests): 1. mvn install -DskipTests -Pdist -Dmaven.javadoc.skip=true -Phdds -DskipShade -am -pl :hadoop-ozone-dist 2. cd hadoop-ozone/dist/target/ozone-0.3.0-alpha/smoketest 3. ./test.sh - To unsubscribe, e-mail: hdfs-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: hdfs-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-9031) YARN UI2 should make use of "yarn.log.server.web-service.url" to fetch container logs
Rohith Sharma K S created YARN-9031: --- Summary: YARN UI2 should make use of "yarn.log.server.web-service.url" to fetch container logs Key: YARN-9031 URL: https://issues.apache.org/jira/browse/YARN-9031 Project: Hadoop YARN Issue Type: Improvement Reporter: Rohith Sharma K S Post YARN-5742, ATSv2.0 introduces log servlet which could be used for log fetching. Many cluster deployment ATS1.5 may not exist which causes log fetching failure in UI2. # UI2 should be able to fetch logs from ATS2.0 log servlet. # Its better to take log server url based on the property "yarn.log.server.web-service.url" # Each ATSv2 log URL takes clusterid as query params. UI2 should send clusterid query param if required. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
[jira] [Created] (YARN-9030) Log aggregation changes to handle filesystems which do not support permissions
Suma Shivaprasad created YARN-9030:
--
Summary: Log aggregation changes to handle filesystems which do
not support permissions
Key: YARN-9030
URL: https://issues.apache.org/jira/browse/YARN-9030
Project: Hadoop YARN
Issue Type: Bug
Reporter: Suma Shivaprasad
Assignee: Suma Shivaprasad
Some cloud storages like ABFS do not support permissions in which case they
throw an UnsupportedOperationException. Log aggregation should hanlde these
case and not set permissions for log aggregation base dir/ sub dirs
{noformat}
2018-11-12 15:37:28,726 WARN logaggregation.LogAggregationService
(LogAggregationService.java:initApp(209)) - Application failed to init
aggregation
org.apache.hadoop.yarn.exceptions.YarnRuntimeException: Failed to check
permissions for dir [abfs://testc...@test.blob.core.windows.net/app-logs]
at
org.apache.hadoop.yarn.logaggregation.filecontroller.LogAggregationFileController.verifyAndCreateRemoteLogDir(LogAggregationFileController.java:277)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initAppAggregator(LogAggregationService.java:238)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.initApp(LogAggregationService.java:204)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:347)
at
org.apache.hadoop.yarn.server.nodemanager.containermanager.logaggregation.LogAggregationService.handle(LogAggregationService.java:69)
at
org.apache.hadoop.yarn.event.AsyncDispatcher.dispatch(AsyncDispatcher.java:197)
at
org.apache.hadoop.yarn.event.AsyncDispatcher$1.run(AsyncDispatcher.java:126)
at java.lang.Thread.run(Thread.java:748)
{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org
