[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191270#comment-17191270 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17191260#comment-17191260 ] Masatake Iwasaki commented on HADOOP-14176: --- updated the target version for preparing 2.10.1

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16952127#comment-16952127 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16678621#comment-16678621 ] Tak Lon (Stephen) Wu commented on HADOOP-14176: --- any update on this? I recently got into

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16186640#comment-16186640 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945985#comment-15945985 ] Allen Wittenauer commented on HADOOP-14176: --- Actually, the more I think about it: why,

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945439#comment-15945439 ] Allen Wittenauer commented on HADOOP-14176: --- It's not that simple. For example, you CANNOT

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15945410#comment-15945410 ] Fei Hui commented on HADOOP-14176: -- I think we should pick common parameters and set them default in

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15939630#comment-15939630 ] Fei Hui commented on HADOOP-14176: -- [~aw] What is the way you suggest on branch-2 for fix this issue?

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15936396#comment-15936396 ] Allen Wittenauer commented on HADOOP-14176: --- -1. This is an incompatible change. See also

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933950#comment-15933950 ] Fei Hui commented on HADOOP-14176: -- CC [~jrottinghuis] [~cnauroth] > distcp reports beyond physical

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15933552#comment-15933552 ] Ravi Prakash commented on HADOOP-14176: --- Hi Fei Hui! What I meant was that I'd lean towards

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931100#comment-15931100 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15931076#comment-15931076 ] Fei Hui commented on HADOOP-14176: -- In hadoop 3, {{mapreduce.job.heap.memory-mb.ratio}} is 0.8 by

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15929504#comment-15929504 ] Fei Hui commented on HADOOP-14176: -- [~raviprak] I think {code}mapreduce.map.memory.mb{code} should

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15928858#comment-15928858 ] Ravi Prakash commented on HADOOP-14176: --- Just fyi! [~yzhangal] is working on HADOOP-11794 and I

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927824#comment-15927824 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927768#comment-15927768 ] Fei Hui commented on HADOOP-14176: -- agree with [~jrottinghuis] [~cnauroth] * overrides

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15927201#comment-15927201 ] Chris Nauroth commented on HADOOP-14176: [~jrottinghuis], thanks for some great suggestions on

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15926230#comment-15926230 ] Fei Hui commented on HADOOP-14176: -- [~atm] [~cnauroth] [~umamaheswararao] could you please take a look?

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925845#comment-15925845 ] Steve Loughran commented on HADOOP-14176: - I'm watching this, but don't know enough about distcp

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15925506#comment-15925506 ] Fei Hui commented on HADOOP-14176: -- [~ste...@apache.org] could you please take a look ?thanks > distcp

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923934#comment-15923934 ] Hadoop QA commented on HADOOP-14176: | (x) *{color:red}-1 overall{color}* | \\ \\ || Vote ||

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923549#comment-15923549 ] Fei Hui commented on HADOOP-14176: -- [~jrottinghuis] Maybe should we add mapreduce.map.java.opts to

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923535#comment-15923535 ] Fei Hui commented on HADOOP-14176: -- [~jrottinghuis] There is only mapreduce.map.memory.mb, but no

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923246#comment-15923246 ] Joep Rottinghuis commented on HADOOP-14176: --- There is a patch in upstream (not quite sure

[jira] [Commented] (HADOOP-14176) distcp reports beyond physical memory limits on 2.X

[ https://issues.apache.org/jira/browse/HADOOP-14176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922942#comment-15922942 ] Joep Rottinghuis commented on HADOOP-14176: --- IIUC the limits in distcp-default are to indicate