[ 
https://issues.apache.org/jira/browse/YARN-160?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14106406#comment-14106406
 ] 

Junping Du commented on YARN-160:
---------------------------------

Thanks for addressing my comments, [~vvasudev]!
{code}
+  private static final Pattern PHYSICAL_ID_FORMAT =
+      Pattern.compile("^physical id[ \t]*:[ \t]*([0-9]*)");
+  private static final Pattern CORE_ID_FORMAT =
+      Pattern.compile("^core id[ \t]*:[ \t]*([0-9]*)");
{code}
Both physical id and core id are not guaranteed to have in /proc/cpuinfo 
(please see below for my local VM's info). We may use processor number instead 
in case these ids are 0 (like we did in Windows). Again, this weak my 
confidence that this automatic way of getting CPU/memory resources should 
happen by default (not sure if any cross-platform issues). May be a safer way 
here is to keep previous default behavior (with some static setting) with an 
extra config to enable this. We can wait this feature to be more stable later 
to change the default behavior. 

{noformat}
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 70
model name      : Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz
stepping        : 1
cpu MHz         : 2295.265
cache size      : 6144 KB
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov 
pat pse36 clflush dts mmx fxsr sse sse2 ss syscall nx pdpe1gb rdtscp lm 
constant_tsc up arch_perfmon pebs bts xtopology tsc_reliable nonstop_tsc 
aperfmperf unfair_spinlock pni pclmulqdq vmx ssse3 fma cx16 pcid sse4_1 sse4_2 
x2apic movbe popcnt aes xsave avx f16c rdrand hypervisor lahf_lm ida arat epb 
xsaveopt pln pts dts tpr_shadow vnmi ept vpid fsgsbase smep
bogomips        : 4590.53
clflush size    : 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management:
{noformat} 

> nodemanagers should obtain cpu/memory values from underlying OS
> ---------------------------------------------------------------
>
>                 Key: YARN-160
>                 URL: https://issues.apache.org/jira/browse/YARN-160
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: nodemanager
>    Affects Versions: 2.0.3-alpha
>            Reporter: Alejandro Abdelnur
>            Assignee: Varun Vasudev
>             Fix For: 2.6.0
>
>         Attachments: apache-yarn-160.0.patch, apache-yarn-160.1.patch
>
>
> As mentioned in YARN-2
> *NM memory and CPU configs*
> Currently these values are coming from the config of the NM, we should be 
> able to obtain those values from the OS (ie, in the case of Linux from 
> /proc/meminfo & /proc/cpuinfo). As this is highly OS dependent we should have 
> an interface that obtains this information. In addition implementations of 
> this interface should be able to specify a mem/cpu offset (amount of mem/cpu 
> not to be avail as YARN resource), this would allow to reserve mem/cpu for 
> the OS and other services outside of YARN containers.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Reply via email to