Re: Using Spark
Awesome, thanks On Sunday, June 22, 2014, Matei Zaharia matei.zaha...@gmail.com wrote: Alright, added you. On Jun 20, 2014, at 2:52 PM, Ricky Thomas ri...@truedash.io javascript:_e(%7B%7D,'cvml','ri...@truedash.io'); wrote: Hi, Would like to add ourselves to the user list if possible please? Company: truedash url: truedash.io Automatic pulling of all your data in to Spark for enterprise visualisation, predictive analytics and data exploration at a low cost. Currently in development with a few clients. Thanks
Re: Spark throws NoSuchFieldError when testing on cluster mode
Right problem solved in a most disgraceful manner. Just add a package relocation in maven shade config. The downside is that it is not compatible with my IDE (IntelliJ IDEA), will cause: Error:scala.reflect.internal.MissingRequirementError: object scala.runtime in compiler mirror not found.: object scala.runtime in compiler mirror not found. and all scala object inspection fail and marked as error. So I'm still looking for an alternative -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-throws-NoSuchFieldError-when-testing-on-cluster-mode-tp8064p8088.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
InputStreamsSuite test failed
Hello ,I am a new guy on scala spark, yestday i compile spark from 1.0.0 source code and run test,there is and testcase failed: For example run command in shell : sbt/sbt testOnly org.apache.spark.streaming.InputStreamsSuite the testcase: test(socket input stream) would failed , test result like : ===test result=== [info] InputStreamsSuite: [info] - socket input stream *** FAILED *** (20 seconds, 547 milliseconds) [info] 0 did not equal 5 (InputStreamsSuite.scala:96) [info] org.scalatest.exceptions.TestFailedException: [info] at org.scalatest.Assertions$class.newAssertionFailedException(Assertions.scala:318) [info] at org.apache.spark.streaming.InputStreamsSuite.newAssertionFailedException(InputStreamsSuite.scala:44) [info] at org.scalatest.Assertions$class.assert(Assertions.scala:401) [info] at org.apache.spark.streaming.InputStreamsSuite.assert(InputStreamsSuite.scala:44) [info] at org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply$mcV$sp(InputStreamsSuite.scala:96) [info] at org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply(InputStreamsSuite.scala:46) [info] at org.apache.spark.streaming.InputStreamsSuite$$anonfun$1.apply(InputStreamsSuite.scala:46) [info] at org.scalatest.FunSuite$$anon$1.apply(FunSuite.scala:1265) [info] at org.scalatest.Suite$class.withFixture(Suite.scala:1974) [info] at org.apache.spark.streaming.InputStreamsSuite.withFixture(InputStreamsSuite.scala:44) [info] at org.scalatest.FunSuite$class.invokeWithFixture$1(FunSuite.scala:1262) [info] at org.scalatest.FunSuite$$anonfun$runTest$1.apply(FunSuite.scala:1271) [info] at org.scalatest.FunSuite$$anonfun$runTest$1.apply(FunSuite.scala:1271) [info] at org.scalatest.SuperEngine.runTestImpl(Engine.scala:198) [info] at org.scalatest.FunSuite$class.runTest(FunSuite.scala:1271) [info] at org.apache.spark.streaming.InputStreamsSuite.org$scalatest$BeforeAndAfter$$super$runTest(InputStreamsSuite.scala:44) [info] at org.scalatest.BeforeAndAfter$class.runTest(BeforeAndAfter.scala:171) [info] at org.apache.spark.streaming.InputStreamsSuite.runTest(InputStreamsSuite.scala:44) [info] at org.scalatest.FunSuite$$anonfun$runTests$1.apply(FunSuite.scala:1304) [info] at org.scalatest.FunSuite$$anonfun$runTests$1.apply(FunSuite.scala:1304) [info] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:260) [info] at org.scalatest.SuperEngine$$anonfun$org$scalatest$SuperEngine$$runTestsInBranch$1.apply(Engine.scala:249) [info] at scala.collection.immutable.List.foreach(List.scala:318) [info] at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:249) [info] at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:326) [info] at org.scalatest.FunSuite$class.runTests(FunSuite.scala:1304) [info] at org.apache.spark.streaming.InputStreamsSuite.runTests(InputStreamsSuite.scala:44) [info] at org.scalatest.Suite$class.run(Suite.scala:2303) [info] at org.apache.spark.streaming.InputStreamsSuite.org$scalatest$FunSuite$$super$run(InputStreamsSuite.scala:44) [info] at org.scalatest.FunSuite$$anonfun$run$1.apply(FunSuite.scala:1310) [info] at org.scalatest.FunSuite$$anonfun$run$1.apply(FunSuite.scala:1310) [info] at org.scalatest.SuperEngine.runImpl(Engine.scala:362) [info] at org.scalatest.FunSuite$class.run(FunSuite.scala:1310) [info] at org.apache.spark.streaming.InputStreamsSuite.org$scalatest$BeforeAndAfter$$super$run(InputStreamsSuite.scala:44) [info] at org.scalatest.BeforeAndAfter$class.run(BeforeAndAfter.scala:208) [info] at org.apache.spark.streaming.InputStreamsSuite.run(InputStreamsSuite.scala:44) [info] at org.scalatest.tools.ScalaTestFramework$ScalaTestRunner.run(ScalaTestFramework.scala:214) [info] at sbt.RunnerWrapper$1.runRunner2(FrameworkWrapper.java:223) [info] at sbt.RunnerWrapper$1.execute(FrameworkWrapper.java:236) [info] at sbt.ForkMain$Run$2.call(ForkMain.java:294) [info] at sbt.ForkMain$Run$2.call(ForkMain.java:284) [info] at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) [info] at java.util.concurrent.FutureTask.run(FutureTask.java:138) [info] at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) [info] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) [info] at java.lang.Thread.run(Thread.java:662) I check the souce code, it seem's this assert would fail (line number is 96): assert(output.size === expectedOutput.size) so i print the output outputBuffer expectedOutput value ： // Verify whether data received was as expected 85 println() 86 println(output.size = + output.size) 87 88 output.foreach(x = println([ + x.mkString(,) + ])) 89 println(outputBuffer.size=+outputBuffer.size) 90
Shark vs Impala
Hi folks, I was looking at the benchmark provided by Cloudera at http://blog.cloudera.com/blog/2014/05/new-sql-choices-in-the-apache-hadoop-ecosystem-why-impala-continues-to-lead/ . Is it real that Shark cannot execute some query if you don't have enough memory? And is it true/reliable that Impala overcome so much Spark when executing complex queries? Best, Flavio
Re: Shark vs Impala
For the second question, I would say it is mainly because the projects have not the same aim. Impala does have a cost-based optimizer and predicate propagation capability which is natural because it is interpreting pseudo-SQL query. In the realm of relational database, it is often not a good idea to compete against the optimizer, it is of course also true for 'BigData'. Bertrand On Sun, Jun 22, 2014 at 1:32 PM, Flavio Pompermaier pomperma...@okkam.it wrote: Hi folks, I was looking at the benchmark provided by Cloudera at http://blog.cloudera.com/blog/2014/05/new-sql-choices-in-the-apache-hadoop-ecosystem-why-impala-continues-to-lead/ . Is it real that Shark cannot execute some query if you don't have enough memory? And is it true/reliable that Impala overcome so much Spark when executing complex queries? Best, Flavio
Re: Shark vs Impala
I've just benchmarked Spark and Impala. Same data (in s3), same query, same cluster. Impala has a long load time, since it cannot load directly from s3. I have to create a Hive table on s3, then insert from that to an Impala table. This takes a long time; Spark took about 600s for the query, Impala 250s, but Impala required 6k seconds to load data from s3. If you're going to go the long-initial-load-then-quick-queries route, go for Redshift. On equivalent hardware, that took about 4k seconds to load, but then queries are like 5s each.
Re: Shark vs Impala
600s for Spark vs 5s for Redshift...The numbers look much different from the amplab benchmark... https://amplab.cs.berkeley.edu/benchmark/ Is it like SSDs or something that's helping redshift or the whole data is in memory when you run the query ? Could you publish the query ? Also after spark-sql are we planning to add spark-sql runtimes in the amplab benchmark as well ? On Sun, Jun 22, 2014 at 9:13 AM, Toby Douglass t...@avocet.io wrote: I've just benchmarked Spark and Impala. Same data (in s3), same query, same cluster. Impala has a long load time, since it cannot load directly from s3. I have to create a Hive table on s3, then insert from that to an Impala table. This takes a long time; Spark took about 600s for the query, Impala 250s, but Impala required 6k seconds to load data from s3. If you're going to go the long-initial-load-then-quick-queries route, go for Redshift. On equivalent hardware, that took about 4k seconds to load, but then queries are like 5s each.
MLLib sample data format
Hello, I am looking into a couple of MLLib data files in https://github.com/apache/spark/tree/master/data/mllib. But I cannot find any explanation for these files? Does anyone know if they are documented? Thanks. Justin
Re: MLLib sample data format
Hi Shuo, Yes. I was reading the guide as well as the sample code. For example, in http://spark.apache.org/docs/latest/mllib-linear-methods.html#linear-support-vector-machine-svm, nowhere in the github repository I can find the file: sc.textFile( mllib/data/ridge-data/lpsa.data). Thanks. Justin On Sun, Jun 22, 2014 at 3:24 PM, Justin Yip yipjus...@gmail.com wrote: Hi Shuo, Yes. I was reading the guide as well as the sample code. For example, in http://spark.apache.org/docs/latest/mllib-linear-methods.html#linear-support-vector-machine-svm, now where in the github repository I can find the file: sc.textFile( mllib/data/ridge-data/lpsa.data). Thanks. Justin On Sun, Jun 22, 2014 at 2:40 PM, Shuo Xiang shuoxiang...@gmail.com wrote: Hi, you might find http://spark.apache.org/docs/latest/mllib-guide.html helpful. On Sun, Jun 22, 2014 at 2:35 PM, Justin Yip yipjus...@gmail.com wrote: Hello, I am looking into a couple of MLLib data files in https://github.com/apache/spark/tree/master/data/mllib. But I cannot find any explanation for these files? Does anyone know if they are documented? Thanks. Justin
Re: MLLib sample data format
Hi Shuo, Yes. I was reading the guide as well as the sample code. For example, in http://spark.apache.org/docs/latest/mllib-linear-methods.html#linear-support-vector-machine-svm, now where in the github repository I can find the file: sc.textFile( mllib/data/ridge-data/lpsa.data). Thanks. Justin On Sun, Jun 22, 2014 at 2:40 PM, Shuo Xiang shuoxiang...@gmail.com wrote: Hi, you might find http://spark.apache.org/docs/latest/mllib-guide.html helpful. On Sun, Jun 22, 2014 at 2:35 PM, Justin Yip yipjus...@gmail.com wrote: Hello, I am looking into a couple of MLLib data files in https://github.com/apache/spark/tree/master/data/mllib. But I cannot find any explanation for these files? Does anyone know if they are documented? Thanks. Justin
Re: MLLib sample data format
These files follow the libsvm format where each line is a record, the first column is a label, and then after that the fields are offset:value where offset is the offset into the feature vector, and value is the value of the input feature. This is a fairly efficient representation for sparse but can double (or more) storage requirements for dense data. - Evan On Jun 22, 2014, at 3:35 PM, Justin Yip yipjus...@gmail.com wrote: Hello, I am looking into a couple of MLLib data files in https://github.com/apache/spark/tree/master/data/mllib. But I cannot find any explanation for these files? Does anyone know if they are documented? Thanks. Justin
hi
Hi
Can someone help me with the following error that I faced while setting
up single node spark framework.
karthik@karthik-OptiPlex-9020:~/spark-1.0.0$ MASTER=spark://localhost:7077
sbin/spark-shell
bash: sbin/spark-shell: No such file or directory
karthik@karthik-OptiPlex-9020:~/spark-1.0.0$ MASTER=spark://localhost:7077
bin/spark-shell
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=128m; support was removed in 8.0
14/06/23 10:44:53 INFO spark.SecurityManager: Changing view acls to: karthik
14/06/23 10:44:53 INFO spark.SecurityManager: SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(karthik)
14/06/23 10:44:53 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:53 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:53 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:39588
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.0.0
/_/
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java
1.8.0_05)
Type in expressions to have them evaluated.
Type :help for more information.
14/06/23 10:44:55 INFO spark.SecurityManager: Changing view acls to: karthik
14/06/23 10:44:55 INFO spark.SecurityManager: SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(karthik)
14/06/23 10:44:55 INFO slf4j.Slf4jLogger: Slf4jLogger started
14/06/23 10:44:55 INFO Remoting: Starting remoting
14/06/23 10:44:55 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://spark@karthik-OptiPlex-9020:50294]
14/06/23 10:44:55 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://spark@karthik-OptiPlex-9020:50294]
14/06/23 10:44:55 INFO spark.SparkEnv: Registering MapOutputTracker
14/06/23 10:44:55 INFO spark.SparkEnv: Registering BlockManagerMaster
14/06/23 10:44:55 INFO storage.DiskBlockManager: Created local directory at
/tmp/spark-local-20140623104455-3297
14/06/23 10:44:55 INFO storage.MemoryStore: MemoryStore started with
capacity 294.6 MB.
14/06/23 10:44:55 INFO network.ConnectionManager: Bound socket to port
60264 with id = ConnectionManagerId(karthik-OptiPlex-9020,60264)
14/06/23 10:44:55 INFO storage.BlockManagerMaster: Trying to register
BlockManager
14/06/23 10:44:55 INFO storage.BlockManagerInfo: Registering block manager
karthik-OptiPlex-9020:60264 with 294.6 MB RAM
14/06/23 10:44:55 INFO storage.BlockManagerMaster: Registered BlockManager
14/06/23 10:44:55 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:55 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:55 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:38307
14/06/23 10:44:55 INFO broadcast.HttpBroadcast: Broadcast server started at
http://10.0.1.61:38307
14/06/23 10:44:55 INFO spark.HttpFileServer: HTTP File server directory is
/tmp/spark-082a44f6-e877-48cc-8ab7-1bcbcf8136b0
14/06/23 10:44:55 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:55 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:55 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:58745
14/06/23 10:44:56 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:56 INFO server.AbstractConnector: Started
SelectChannelConnector@0.0.0.0:4040
14/06/23 10:44:56 INFO ui.SparkUI: Started SparkUI at
http://karthik-OptiPlex-9020:4040
14/06/23 10:44:56 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/06/23 10:44:56 INFO client.AppClient$ClientActor: Connecting to master
spark://localhost:7077...
14/06/23 10:44:56 INFO repl.SparkILoop: Created spark context..
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
Spark context available as sc.
scala 14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not
connect to akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:45:16 INFO client.AppClient$ClientActor: Connecting to master
spark://localhost:7077...
14/06/23 10:45:16 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:45:16 WARN client.AppClient$ClientActor: Could
Persistent Local Node variables
*TL;DR:* I want to run a pre-processing step on the data from each partition
(such as parsing) and retain the parsed object on each node for future
processing calls to avoid repeated parsing.
/More detail:/
I have a server and two nodes in my cluster, and data partitioned using
hdfs.
I am trying to use spark to process the data and send back results.
The data is available as text, and I would like to first parse this text,
and then run future processing.
To do this, I call a simple:
JavaRDD.foreachPartition(IteratorString)(new
VoidFunctionIteratorlt;String(){
@Override
public void call(IteratorString i){
ParsedData p=new ParsedData(i);
}
});
I would like to retain this ParsedData object on each node for future
processing calls, so as to avoid parsing all over again. So in my next call,
I'd like to do something like this:
JavaRDD.foreachPartition(IteratorString)(new
VoidFunctionIteratorlt;String(){
@Override
public void call(IteratorString i){
//refer to previously created ParsedData object
p.process();
//accumulate some results
}
});
--
View this message in context:
http://apache-spark-user-list.1001560.n3.nabble.com/Persistent-Local-Node-variables-tp8104.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: MLLib sample data format
I see. That's good. Thanks. Justin On Sun, Jun 22, 2014 at 4:59 PM, Evan Sparks evan.spa...@gmail.com wrote: Oh, and the movie lens one is userid::movieid::rating - Evan On Jun 22, 2014, at 3:35 PM, Justin Yip yipjus...@gmail.com wrote: Hello, I am looking into a couple of MLLib data files in https://github.com/apache/spark/tree/master/data/mllib. But I cannot find any explanation for these files? Does anyone know if they are documented? Thanks. Justin
Re: Persistent Local Node variables
Will using mapPartitions and creating a new RDD of ParsedData objects avoid multiple parsing? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Persistent-Local-Node-variables-tp8104p8107.html Sent from the Apache Spark User List mailing list archive at Nabble.com.
Re: hi
Open your webUI in the browser and see the spark url in the top left corner
of the page and use it while starting your spark shell instead of
localhost:7077.
Thanks
Best Regards
On Mon, Jun 23, 2014 at 10:56 AM, rapelly kartheek kartheek.m...@gmail.com
wrote:
Hi
Can someone help me with the following error that I faced while setting
up single node spark framework.
karthik@karthik-OptiPlex-9020:~/spark-1.0.0$
MASTER=spark://localhost:7077 sbin/spark-shell
bash: sbin/spark-shell: No such file or directory
karthik@karthik-OptiPlex-9020:~/spark-1.0.0$
MASTER=spark://localhost:7077 bin/spark-shell
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option
MaxPermSize=128m; support was removed in 8.0
14/06/23 10:44:53 INFO spark.SecurityManager: Changing view acls to:
karthik
14/06/23 10:44:53 INFO spark.SecurityManager: SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(karthik)
14/06/23 10:44:53 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:53 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:53 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:39588
Welcome to
__
/ __/__ ___ _/ /__
_\ \/ _ \/ _ `/ __/ '_/
/___/ .__/\_,_/_/ /_/\_\ version 1.0.0
/_/
Using Scala version 2.10.4 (Java HotSpot(TM) 64-Bit Server VM, Java
1.8.0_05)
Type in expressions to have them evaluated.
Type :help for more information.
14/06/23 10:44:55 INFO spark.SecurityManager: Changing view acls to:
karthik
14/06/23 10:44:55 INFO spark.SecurityManager: SecurityManager:
authentication disabled; ui acls disabled; users with view permissions:
Set(karthik)
14/06/23 10:44:55 INFO slf4j.Slf4jLogger: Slf4jLogger started
14/06/23 10:44:55 INFO Remoting: Starting remoting
14/06/23 10:44:55 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://spark@karthik-OptiPlex-9020:50294]
14/06/23 10:44:55 INFO Remoting: Remoting now listens on addresses:
[akka.tcp://spark@karthik-OptiPlex-9020:50294]
14/06/23 10:44:55 INFO spark.SparkEnv: Registering MapOutputTracker
14/06/23 10:44:55 INFO spark.SparkEnv: Registering BlockManagerMaster
14/06/23 10:44:55 INFO storage.DiskBlockManager: Created local directory
at /tmp/spark-local-20140623104455-3297
14/06/23 10:44:55 INFO storage.MemoryStore: MemoryStore started with
capacity 294.6 MB.
14/06/23 10:44:55 INFO network.ConnectionManager: Bound socket to port
60264 with id = ConnectionManagerId(karthik-OptiPlex-9020,60264)
14/06/23 10:44:55 INFO storage.BlockManagerMaster: Trying to register
BlockManager
14/06/23 10:44:55 INFO storage.BlockManagerInfo: Registering block manager
karthik-OptiPlex-9020:60264 with 294.6 MB RAM
14/06/23 10:44:55 INFO storage.BlockManagerMaster: Registered BlockManager
14/06/23 10:44:55 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:55 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:55 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:38307
14/06/23 10:44:55 INFO broadcast.HttpBroadcast: Broadcast server started
at http://10.0.1.61:38307
14/06/23 10:44:55 INFO spark.HttpFileServer: HTTP File server directory is
/tmp/spark-082a44f6-e877-48cc-8ab7-1bcbcf8136b0
14/06/23 10:44:55 INFO spark.HttpServer: Starting HTTP Server
14/06/23 10:44:55 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:55 INFO server.AbstractConnector: Started
SocketConnector@0.0.0.0:58745
14/06/23 10:44:56 INFO server.Server: jetty-8.y.z-SNAPSHOT
14/06/23 10:44:56 INFO server.AbstractConnector: Started
SelectChannelConnector@0.0.0.0:4040
14/06/23 10:44:56 INFO ui.SparkUI: Started SparkUI at
http://karthik-OptiPlex-9020:4040
14/06/23 10:44:56 WARN util.NativeCodeLoader: Unable to load native-hadoop
library for your platform... using builtin-java classes where applicable
14/06/23 10:44:56 INFO client.AppClient$ClientActor: Connecting to master
spark://localhost:7077...
14/06/23 10:44:56 INFO repl.SparkILoop: Created spark context..
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
Spark context available as sc.
scala 14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not
connect to akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:44:56 WARN client.AppClient$ClientActor: Could not connect to
akka.tcp://sparkMaster@localhost:7077:
akka.remote.EndpointAssociationException: Association failed with
[akka.tcp://sparkMaster@localhost:7077]
14/06/23 10:45:16 INFO
