a straightforward Spark RDD implementation of the same.
--
Christopher T. Nguyen
Co-founder CEO, Adatao
linkedin.com/in/ctnguyen
On Wed, Jan 29, 2014 at 8:10 PM, Chen Jin karen...@gmail.com wrote:
Hi All,
https://amplab.cs.berkeley.edu/benchmark/ has given a nice benchmark
report. I am trying
?
No, (a) gives u 8 workers with each core per worker (b) gives 1 worker
while each worker has eight cores.
Let me know, if any doubts.
Thanks and Regards,
Archit Thakur.
On Sun, Jan 26, 2014 at 5:58 AM, Chen Jin karen...@gmail.com wrote:
Hi all,
From spark document, we can set the number
Hi all,
From spark document, we can set the number of workers by
SPARK_WORKER_INSTANCES and the max number of cores that worker can
take by using SPARK_WORKER_CORES, if I have 5 8-core machine, which
one would perform better between
(a)
SPARK_WORKER_INSTANCES = 8
SPARK_WORKER_CORES = 1
and
Hi,
Currently my application jars and logs are stored in $SPARK_HOME/work,
I would like to change it to somewhere having more space. Could anyone
advise me on this? Changing the log dir is straightforward which just
to export SPARK_LOG_DIR, however, there is no environment variable for
Yeah, that's was the problem, I fixed it, thanks a lot.
On Sat, Jan 11, 2014 at 4:41 PM, Patrick Wendell pwend...@gmail.com wrote:
You may need to set the master to be one of the HVM types as well.
What type is the master in your cluster?
On Fri, Jan 10, 2014 at 2:15 PM, Chen Jin karen
Hi ,
I am trying to run spark with EC2 SSD instance i2.2xlarge,
But, when I try to launch EC2 cluster using spark-ec2 script with
--instance-type=i2.2x.large
I received the following error:
ResponseErrorsErrorCodeInvalidParameterCombination/CodeMessageVirtualization
type 'hvm' is required for
?
ResponseErrorsErrorCodeInvalidParameterCombination/CodeMessageNon-Windows
instances with a virtualization type of 'hvm' are currently not
supported for this instance
type./Message/Error/ErrorsRequestID60aa8aa0-acf3-4bc8-b2b4-120f0b7d1e36/RequestID/Response
On Fri, Jan 10, 2014 at 1:40 PM, Chen Jin karen...@gmail.com wrote:
Thanks, the patch
