Re: Can't pull the trunk code in windows- >File name too long

[Vrushali C]

Thanks for the update Brahma. 

> On Mar 14, 2018, at 7:12 PM, Brahma Reddy Battula 
>  wrote:
> 
> Hi,
> 
> Able to clone/pull after setting the following.
> 
> git config --system core.longpaths true
> 
> 
> --Brahma Reddy Battula
> 
> -Original Message-
> From: Vinayakumar B [mailto:vinayakum...@apache.org] 
> Sent: 13 March 2018 14:30
> To: Brahma Reddy Battula 
> Cc: Hadoop Common ; yarn-dev@hadoop.apache.org
> Subject: Re: Can't pull the trunk code in windows- >File name too long
> 
> this is happening  in Windows, Please somebody from yarn look into this.
> 
> -Vinay
> 
> On 13 Mar 2018 11:30 am, "Brahma Reddy Battula" < 
> brahmareddy.batt...@huawei.com> wrote:
> 
> Hi
> 
> Not able to pull/clone the trunk code. Does it possible make following 
> shorten.
> 
> 
> fatal: cannot create directory at 'hadoop-yarn-project/hadoop-
> yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-
> yarn-server-timelineservice-hbase-server/hadoop-yarn-
> server-timelineservice-hbase-server-1/src/main/java/org/
> apache/hadoop/yarn/server/timelineservice': Filename too long
> 
> 
> 
> --Brahma Reddy Battula
> ТÐÐ¥FòVç7V'67&–ÂRÖÖ–â–&âÖFWb×Vç7V'67&–†Fö÷æ6†Ræ÷Фf÷"FF—F–öæÂ6öÖÖæG2ÂRÖÖ–â–&âÖFWbÖ†VdžFö÷æ6†Ræ÷Ð
>  

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

RE: Can't pull the trunk code in windows- >File name too long

[Brahma Reddy Battula]

Hi,

Able to clone/pull after setting the following.

git config --system core.longpaths true


--Brahma Reddy Battula

-Original Message-
From: Vinayakumar B [mailto:vinayakum...@apache.org] 
Sent: 13 March 2018 14:30
To: Brahma Reddy Battula 
Cc: Hadoop Common ; yarn-dev@hadoop.apache.org
Subject: Re: Can't pull the trunk code in windows- >File name too long

this is happening  in Windows, Please somebody from yarn look into this.

-Vinay

On 13 Mar 2018 11:30 am, "Brahma Reddy Battula" < 
brahmareddy.batt...@huawei.com> wrote:

Hi

Not able to pull/clone the trunk code. Does it possible make following shorten.


fatal: cannot create directory at 'hadoop-yarn-project/hadoop-
yarn/hadoop-yarn-server/hadoop-yarn-server-timelineservice-hbase/hadoop-
yarn-server-timelineservice-hbase-server/hadoop-yarn-
server-timelineservice-hbase-server-1/src/main/java/org/
apache/hadoop/yarn/server/timelineservice': Filename too long



--Brahma Reddy Battula

Apache Hadoop qbt Report: trunk+JDK8 on Linux/ppc64le

[Apache Jenkins Server]

For more details, see 
https://builds.apache.org/job/hadoop-qbt-trunk-java8-linux-ppc/378/

[Oct 12, 2017 5:38:58 PM] (carlo curino) YARN-7317. Fix overallocation resulted 
from ceiling in
[Oct 12, 2017 5:59:20 PM] (ctrezzo) MAPREDUCE-5951. Add support for the YARN 
Shared Cache.
[Oct 13, 2017 9:00:25 AM] (aajisaka) Revert "HADOOP-13514. Upgrade maven 
surefire plugin to 2.19.1."
[Oct 13, 2017 9:47:05 AM] (rohithsharmaks) YARN-7180. Remove class 
ResourceType. Contributed by Sunil G.
[Oct 13, 2017 7:41:59 PM] (yufei) YARN-7270. Fix unsafe casting from long to 
int for class Resource and
[Oct 13, 2017 8:52:58 PM] (rkanter) HADOOP-14938. 
Configuration.updatingResource map should be initialized
[Oct 13, 2017 9:22:21 PM] (arp) HDFS-12553. Add nameServiceId to 
QJournalProtocol. Contributed by Bharat
[Oct 13, 2017 10:49:21 PM] (haibochen) YARN-7310. 
TestAMRMProxy#testAMRMProxyE2E fails with FairScheduler.
[Oct 14, 2017 12:43:21 AM] (manojpec) HADOOP-13055. Implement linkMergeSlash 
and linkFallback for
[Oct 14, 2017 5:41:58 PM] (wangda) Addendum fix for: YARN-7269. Tracking URL in 
the app state does not get
[Oct 16, 2017 2:28:22 AM] (xiao) HDFS-12659. Update 
TestDeadDatanode#testNonDFSUsedONDeadNodeReReg to
[Oct 16, 2017 3:01:47 PM] (stevel) HADOOP-14935. Azure: POSIX permissions are 
taking effect in access()
[Oct 16, 2017 4:51:10 PM] (xiao) HDFS-12603. Enable async edit logging by 
default. Contributed by Andrew
[Oct 16, 2017 4:55:22 PM] (xiao) HDFS-12637. Extend 
TestDistributedFileSystemWithECFile with a random EC
[Oct 16, 2017 5:34:06 PM] (xiao) HDFS-12642. Log block and datanode details in 
BlockRecoveryWorker.
[Oct 16, 2017 7:57:48 PM] (weichiu) HDFS-12485. expunge may fail to remove 
trash from encryption zone.
[Oct 16, 2017 8:12:15 PM] (arun suresh) YARN-7275. NM Statestore cleanup for 
Container updates. (Kartheek
[Oct 16, 2017 8:57:03 PM] (junping_du) YARN-7124. LogAggregationTFileController 
deletes/renames while file is
[Oct 16, 2017 9:19:31 PM] (xiao) HADOOP-14949. TestKMS#testACLs fails 
intermittently.
[Oct 16, 2017 10:00:38 PM] (nroberts) YARN-7333. container-executor fails to 
remove entries from a directory
[Oct 16, 2017 10:34:32 PM] (rkanter) YARN-7308. TestApplicationACLs fails with 
FairScheduler (rkanter)
[Oct 16, 2017 11:42:59 PM] (weichiu) HADOOP-14948. Document missing config key 
hadoop.treat.subject.external.
[Oct 17, 2017 12:42:41 AM] (manojpec) HDFS-12614. 
FSPermissionChecker#getINodeAttrs() throws NPE when
[Oct 17, 2017 2:44:30 AM] (lei) HDFS-12613. Native EC coder should implement 
release() as idempotent
[Oct 17, 2017 5:15:53 PM] (haibochen) YARN-7341. 
TestRouterWebServiceUtil#testMergeMetrics is flakey. (Robert
[Oct 17, 2017 7:38:06 PM] (subu) YARN-7311. Fix TestRMWebServicesReservation 
parametrization for fair
[Oct 17, 2017 10:52:09 PM] (lei) HDFS-12612. DFSStripedOutputStream.close will 
throw if called a second
[Oct 17, 2017 11:04:19 PM] (haibochen) YARN-6546. SLS is slow while loading 10k 
queues. (Yufei Gu via Haibo
[Oct 18, 2017 2:06:45 AM] (xiao) HADOOP-14944. Add JvmMetrics to KMS.
[Oct 18, 2017 2:18:39 AM] (aajisaka) MAPREDUCE-6972. Enable try-with-resources 
for RecordReader. Contributed
[Oct 18, 2017 10:06:30 PM] (junping_du) HADOOP-14958. Fix source-level 
compatibility after HADOOP-11252.
[Oct 19, 2017 6:51:24 AM] (zhz) HDFS-12502. nntop should support a category 
based on
[Oct 19, 2017 1:02:13 PM] (weichiu) HADOOP-14880. [KMS] Document missing 
KMS client side configs.
[Oct 19, 2017 1:17:59 PM] (weichiu) HDFS-12619. Do not catch and throw 
unchecked exceptions if IBRs fail to
[Oct 19, 2017 8:25:08 PM] (haibochen) HADOOP-14771. hadoop-client does not 
include hadoop-yarn-client. (Ajay
[Oct 19, 2017 9:44:42 PM] (wangda) YARN-7338. Support same origin policy for 
cross site scripting
[Oct 19, 2017 9:45:44 PM] (wangda) YARN-7345. GPU Isolation: Incorrect minor 
device numbers written to
[Oct 19, 2017 11:39:25 PM] (yufei) YARN-7294. 
TestSignalContainer#testSignalRequestDeliveryToNM fails
[Oct 19, 2017 11:45:18 PM] (cdouglas) HADOOP-14816. Update Dockerfile to use 
Xenial. Contributed by Allen
[Oct 19, 2017 11:51:47 PM] (yufei) YARN-7359. 
TestAppManager.testQueueSubmitWithNoPermission() should be
[Oct 20, 2017 1:08:45 AM] (inigoiri) HDFS-12620. Backporting HDFS-10467 to 
branch-2. Contributed by Inigo
[Oct 20, 2017 1:42:04 AM] (kai.zheng) HDFS-12448. Make sure user defined 
erasure coding policy ID will not
[Oct 20, 2017 4:58:40 AM] (wangda) YARN-7170. Improve bower dependencies for 
YARN UI v2. (Sunil G via
[Oct 20, 2017 8:32:20 AM] (yufei) YARN-4090. Make Collections.sort() more 
efficient by caching resource
[Oct 20, 2017 4:02:06 PM] (eyang) YARN-7353. Improved volume mount check for 
directories and unit test
[Oct 20, 2017 5:00:13 PM] (yufei) YARN-7261. Add debug message for better 
download latency monitoring.
[Oct 20, 2017 6:15:20 PM] (yufei) YARN-7355. TestDistributedShell should be 
scheduler agnostic.
[Oct 20, 2017 8:27:21 PM] (wang) HDFS-12497. 

Re: About reset branch-3.1 to trunk before release.

[Vinod Kumar Vavilapalli]

I see one new feature: https://issues.apache.org/jira/browse/YARN-7626: Allow 
regular expression matching in container-executor.cfg for devices and named 
docker volumes mount.

There are 21 sub-tasks. There are three feature-type JIRAs in those - 
https://issues.apache.org/jira/browse/YARN-7972, 
https://issues.apache.org/jira/browse/YARN-7891 and 
https://issues.apache.org/jira/browse/YARN-5015. These should be okay - not 
major disrupting features.

Everything else is either a bug-fix or an improvement so we should be good.

From the list, it doesn't look like resetting will destabilize 3.1, +1 for 
doing this.

Thanks
+Vinod

> On Mar 14, 2018, at 1:54 PM, Wangda Tan  wrote:
> 
> Hi mapreduce/yarn/common/hdfs-devs, 
> 
> As of now, we have all blockers done for 3.1.0 release [1]. The release is 
> running behind schedule due to a few security-related issues. Because of this 
> and since branch-3.1 is cut 5 weeks before on Feb 8, trunk 3.2 is already 
> diverging. There're 64 commits in trunk but not in branch-3.1. [2]
> 
> I took a quick scan of them, most of them are good fixes which we should 
> bring to 3.1.0 as well. And this can also reduce differences between 3.2.0 
> and 3.1.0 release for less maintenance burden in the future.
> 
> Unless anyone objects, we will reset branch-3.1 to trunk in 1-2 days and cut 
> RC after that.
> 
> Thoughts?
> 
> - Wangda
> 
> [1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker, 
> Critical) AND resolution = Unresolved AND "Target Version/s" = 3.1.0 ORDER BY 
> priority DESC
> [2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0) AND 
> fixVersion not in (3.1.0)


-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

About reset branch-3.1 to trunk before release.

[Wangda Tan]

Hi mapreduce/yarn/common/hdfs-devs,

As of now, we have all blockers done for 3.1.0 release [1]. The release is
running behind schedule due to a few security-related issues. Because of
this and since branch-3.1 is cut 5 weeks before on Feb 8, trunk 3.2 is
already diverging. There're 64 commits in trunk but not in branch-3.1. [2]

I took a quick scan of them, most of them are good fixes which we should
bring to 3.1.0 as well. And this can also reduce differences between 3.2.0
and 3.1.0 release for less maintenance burden in the future.

Unless anyone objects, we will reset branch-3.1 to trunk in 1-2 days and
cut RC after that.

Thoughts?

- Wangda

[1] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND priority in (Blocker,
Critical) AND resolution = Unresolved AND "Target Version/s" = 3.1.0 ORDER
BY priority DESC
[2] project in (YARN, HADOOP, MAPREDUCE, HDFS) AND fixVersion in (3.2.0)
AND fixVersion not in (3.1.0)

Re: [VOTE] Merging branch HDFS-7240 to trunk

[Owen O'Malley]

This discussion seems to have died down coming closer consensus without a
resolution.

I'd like to propose the following compromise:

* HDSL become a subproject of Hadoop.
* HDSL will release separately from Hadoop. Hadoop releases will not
contain HDSL and vice versa.
* HDSL will get its own jira instance so that the release tags stay
separate.
* On trunk (as opposed to release branches) HDSL will be a separate module
in Hadoop's source tree. This will enable the HDSL to work on their trunk
and the Hadoop trunk without making releases for every change.
* Hadoop's trunk will only build HDSL if a non-default profile is enabled.
* When Hadoop creates a release branch, the RM will delete the HDSL module
from the branch.
* HDSL will have their own Yetus checks and won't cause failures in the
Hadoop patch check.

I think this accomplishes most of the goals of encouraging HDSL development
while minimizing the potential for disruption of HDFS development.

Thoughts? Andrew, Jitendra, & Sanjay?

Thanks,
   Owen

[jira] [Created] (YARN-8030) Use the common sliding window retry policy for AM restart

[Chandni Singh (JIRA)]

Chandni Singh created YARN-8030:
---

 Summary: Use the common sliding window retry policy for AM restart
 Key: YARN-8030
 URL: https://issues.apache.org/jira/browse/YARN-8030
 Project: Hadoop YARN
  Issue Type: Sub-task
Reporter: Chandni Singh
Assignee: Chandni Singh


In YARN-5015, we added a SlidingWindowRetryPolicy for container restart. 
We need to refactor AM restart to use the common SlidingWindowRetryPolicy.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org

[jira] [Created] (YARN-8029) YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS should not use commas as separators

[Jim Brennan (JIRA)]

Jim Brennan created YARN-8029:
-

 Summary: YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS should not use 
commas as separators
 Key: YARN-8029
 URL: https://issues.apache.org/jira/browse/YARN-8029
 Project: Hadoop YARN
  Issue Type: Bug
  Components: yarn
Affects Versions: 3.0.0
Reporter: Jim Brennan


The following docker-related environment variables specify a comma-separated 
list of mounts:

YARN_CONTAINER_RUNTIME_DOCKER_LOCAL_RESOURCE_MOUNTS
YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS

This is a problem because hadoop -Dmapreduce.map.env and related options use  
comma as a delimiter.   So if I put more than one mount in 
YARN_CONTAINER_RUNTIME_DOCKER_MOUNTS the comma in the variable will be treated 
as a delimiter for the hadoop command line option and all but the first mount 
will be ignored.




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: yarn-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: yarn-dev-h...@hadoop.apache.org