Github user chenghao-intel commented on the issue:
https://github.com/apache/spark/pull/15579
Oh, thank you @jerryshao , just noticed you gave inputs also. :)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your projec
Github user chenghao-intel commented on the issue:
https://github.com/apache/spark/pull/15579
@srowen
Besides `numactl`, some profiling tools like the `valgrind`, `strace`,
`vtune`, and also the system call hackings we probably needed before the
executor process launched.
Github user jerryshao commented on the issue:
https://github.com/apache/spark/pull/15579
@srowen, another use cases would be trace tools like `strace` which will
trace the system calls for process. One way of using `strace` is to add
`strace` before executing command.
@sheepd
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15579
**[Test build #3370 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3370/consoleFull)**
for PR 15579 at commit
[`a24aff9`](https://github.com/apache/spark/commit
Github user SparkQA commented on the issue:
https://github.com/apache/spark/pull/15579
**[Test build #3370 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3370/consoleFull)**
for PR 15579 at commit
[`a24aff9`](https://github.com/apache/spark/commit/
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15579
What's another use case? I can't think of one. We wouldn't do this with env
variables anyway. (PS you need to fix up the title/description in any event)
---
If your project is set up for it, you can
Github user sheepduke commented on the issue:
https://github.com/apache/spark/pull/15579
@srowen Hi, at first we wanted to add support for numactl only, but later
we thought that it is rather better to make it possible for adding the prefix
command. It is not only for NUMA, but also f
Github user sheepduke commented on the issue:
https://github.com/apache/spark/pull/15579
@mridulm Yes, I tested it in our cluster (5 nodes including 1 master). My
colleague tested with some benchmarks. It seems that NUMA helps a lot for those
applications that have very bad cache loca
Github user srowen commented on the issue:
https://github.com/apache/spark/pull/15579
I'm still not clear this is worth doing just for numactl which few or no
OSes will have installed by default
---
If your project is set up for it, you can reply to this email and have your
reply app
Github user mridulm commented on the issue:
https://github.com/apache/spark/pull/15579
Btw, curious if you have actually tested this in yarn - I have a feeling it
wont work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well
Github user sheepduke commented on the issue:
https://github.com/apache/spark/pull/15579
This is rather useful sometimes because you wan tto add some extra tuning
arguments like 'numactl'.
Otherwise it is not even possible to achieve that.
Yes it only works with YARN for n
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15579
Please also review
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark
Thanks a lot!
---
If your project is set up for it, you can reply to this email and have your
reply
Github user rxin commented on the issue:
https://github.com/apache/spark/pull/15579
This seems really weird to do (and also only works in YARN).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not ha
Github user AmplabJenkins commented on the issue:
https://github.com/apache/spark/pull/15579
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feat
14 matches
Mail list logo
