t;> >> > at py4j.GatewayConnection.run(GatewayConnection.java:207)
>> >> > -
>> >> > And the test code is a follows:
>> >> > conf =
>> >> >
>> >> > SparkConf().setAppName('brodyliu_LR&#x
s a follows:
> >> > conf =
> >> >
> >> > SparkConf().setAppName('brodyliu_LR').setMaster('spark://
> 10.231.131.87:5051')
> >> > conf.set('spark.executor.memory', '4000m')
> >> > conf.set('spa
conf.set('spark.ui.port','8081')
>> > conf.set('spark.cores.max','150')
>> > #conf.set('spark.rdd.compress', 'True')
>> > conf.set('spark.default.parallelism', '300')
>> > #configur
> > To unsubscribe, e-mail: [hidden email]
> <http://user/SendEmail.jtp?type=node&node=18673&i=1>
> > For additional commands, e-mail: [hidden email]
> <http://user/SendEmail.jtp?type=node&node
allelism', '300')
> #configure the spark environment
> sc = SparkContext(conf=conf, batchSize=1)
>
> vec = np.random.rand(3500)
> a = sc.broadcast(vec)
>
>
>
>
>
>
> --
> View this message in context:
> http://apache-spark-user-
t(conf=conf, batchSize=1)
vec = np.random.rand(3500)
a = sc.broadcast(vec)
--
View this message in context:
http://apache-spark-user-list.1001560
