Hi Andrew,
I suggest that you narrow down your scope for performance testing using the
same setup and doing incremental changes keeping other systematics the same.
Spark itself can run on local, standalone, yarn client and yarn cluster
modes So really you need to target a particular setup of run
Yea, I'm looking for any personal experiences people have had with tools like
these.
> On Jul 8, 2016, at 8:57 PM, charles li wrote:
>
> Hi, Andrew, I've got lots of materials when asking google for "spark
> performance test"
>
> https://github.com/databricks/spark-perf
> https://spark-summi
Hi, Andrew, I've got lots of materials when asking google for "*spark
performance test*"
- https://github.com/databricks/spark-perf
-
https://spark-summit.org/2014/wp-content/uploads/2014/06/Testing-Spark-Best-Practices-Anupama-Shetty-Neil-Marshall.pdf
- http://people.cs.vt.edu/~butt
Hi group,
What solutions are people using to do performance testing and tuning of spark
applications? I have been doing a pretty manual technique where I lay out an
Excel sheet of various memory settings and caching parameters and then execute
each one by hand. It’s pretty tedious though, so I’
