[ https://issues.apache.org/jira/browse/SPARK-8089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14571774#comment-14571774 ]
Malte Buecken commented on SPARK-8089: -------------------------------------- mea culpa. there was a cassandra connector dependency that was causing the data loss. I apologize > Operations on RDD cause empty collections > ----------------------------------------- > > Key: SPARK-8089 > URL: https://issues.apache.org/jira/browse/SPARK-8089 > Project: Spark > Issue Type: Bug > Components: Mesos > Affects Versions: 1.2.2, 1.3.1 > Environment: Debian 3.16.7 64bit, mesos 0.22, cassandra 2.1 > Reporter: Malte Buecken > > I honestly do not know what is happening here: > I use Apache Spark on Mesos and had to switch to httpbroadcast because > Torrentbroadcast fails with SPARK-5594 > Now I cannot use any operation on my RDDs without losing all of their data. > I tried both Spark 1.2.2 and Spark 1.3.1, both with the same effect > example: > {code:scala} > val data = sc.parallelize(Array(2,3,4,5,6,7,8,9)) > val newdata = data.repartition(2) > println(newdata.count()) // 0 > {code} > same with groupBy and other operations. I always end up with an empty RDD > Attached a log where I try to different operations, all ending up with empty > RDDs before finally failing when I try data.first(). There is a long > Thread.sleep when I wanted to look around the web UI > {code} > I0603 14:19:54.324846 18091 fetcher.cpp:214] Fetching URI > 'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz' > I0603 14:19:55.870537 18091 fetcher.cpp:99] Fetching URI > 'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz' using Hadoop Client > I0603 14:19:55.870594 18091 fetcher.cpp:109] Downloading resource from > 'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz' to > '/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3.tgz' > I0603 14:20:00.973871 18091 fetcher.cpp:78] Extracted resource > '/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3.tgz' > into > '/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e' > I0603 14:20:01.038312 18285 exec.cpp:132] Version: 0.22.0 > I0603 14:20:01.040046 18286 exec.cpp:206] Executor registered on slave > 20150530-111452-947384330-5050-2394-S3 > Using Spark's default log4j profile: > org/apache/spark/log4j-defaults.properties > 15/06/03 14:20:02 INFO CoarseGrainedExecutorBackend: Registered signal > handlers for [TERM, HUP, INT] > 15/06/03 14:20:03 WARN NativeCodeLoader: Unable to load native-hadoop library > for your platform... using builtin-java classes where applicable > 15/06/03 14:20:03 INFO SecurityManager: Changing view acls to: admin > 15/06/03 14:20:03 INFO SecurityManager: Changing modify acls to: admin > 15/06/03 14:20:03 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(admin); users > with modify permissions: Set(admin) > 15/06/03 14:20:03 INFO Slf4jLogger: Slf4jLogger started > 15/06/03 14:20:04 INFO Remoting: Starting remoting > 15/06/03 14:20:04 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://driverPropsFetcher@10.240.241.48:44036] > 15/06/03 14:20:04 INFO Utils: Successfully started service > 'driverPropsFetcher' on port 44036. > 15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Shutting > down remote daemon. > 15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Remote > daemon shut down; proceeding with flushing remote transports. > 15/06/03 14:20:04 INFO SecurityManager: Changing view acls to: admin > 15/06/03 14:20:04 INFO SecurityManager: Changing modify acls to: admin > 15/06/03 14:20:04 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(admin); users > with modify permissions: Set(admin) > 15/06/03 14:20:04 INFO Slf4jLogger: Slf4jLogger started > 15/06/03 14:20:04 INFO Remoting: Starting remoting > 15/06/03 14:20:04 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://sparkExecutor@10.240.241.48:40338] > 15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Remoting > shut down. > 15/06/03 14:20:04 INFO Utils: Successfully started service 'sparkExecutor' on > port 40338. > 15/06/03 14:20:04 INFO AkkaUtils: Connecting to MapOutputTracker: > akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448/user/MapOutputTracker > 15/06/03 14:20:04 INFO AkkaUtils: Connecting to BlockManagerMaster: > akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448/user/BlockManagerMaster > 15/06/03 14:20:04 INFO DiskBlockManager: Created local directory at > /mnt/spark/active/spark-36127f0b-e5c8-4f39-8137-8ab633b38df1/blockmgr-7d491649-bcb8-4a04-8395-acd7e880c549 > 15/06/03 14:20:04 INFO MemoryStore: MemoryStore started with capacity 15.5 GB > 15/06/03 14:20:05 INFO AkkaUtils: Connecting to OutputCommitCoordinator: > akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448/user/OutputCommitCoordinator > 15/06/03 14:20:05 INFO CoarseGrainedExecutorBackend: Connecting to driver: > akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448/user/CoarseGrainedScheduler > 15/06/03 14:20:05 INFO CoarseGrainedExecutorBackend: Successfully registered > with driver > 15/06/03 14:20:05 INFO Executor: Starting executor ID > 20150530-111452-947384330-5050-2394-S3 on host 10.240.241.48 > 15/06/03 14:20:05 INFO NettyBlockTransferService: Server created on 60337 > 15/06/03 14:20:05 INFO BlockManagerMaster: Trying to register BlockManager > 15/06/03 14:20:05 INFO BlockManagerMaster: Registered BlockManager > 15/06/03 14:20:05 INFO AkkaUtils: Connecting to HeartbeatReceiver: > akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448/user/HeartbeatReceiver > 15/06/03 14:20:14 INFO CoarseGrainedExecutorBackend: Got assigned task 0 > 15/06/03 14:20:14 INFO Executor: Running task 0.0 in stage 0.0 (TID 0) > 15/06/03 14:20:14 INFO Executor: Fetching > http://10.240.119.56:51874/jars/graphit_2.10-1.0.jar with timestamp > 1433366393559 > 15/06/03 14:20:14 INFO Utils: Fetching > http://10.240.119.56:51874/jars/graphit_2.10-1.0.jar to > /mnt/spark/active/spark-5ab6a324-e4a9-418a-8209-0924b76953e4/fetchFileTemp8154164797092940721.tmp > 15/06/03 14:20:14 INFO Utils: Copying > /mnt/spark/active/spark-5ab6a324-e4a9-418a-8209-0924b76953e4/21079050381433366393559_cache > to > /mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./graphit_2.10-1.0.jar > 15/06/03 14:20:14 INFO Executor: Adding > file:/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./graphit_2.10-1.0.jar > to class loader > 15/06/03 14:20:14 INFO Executor: Fetching file:/mnt/spark/spark_1.3.1.fat.jar > with timestamp 1433366393530 > 15/06/03 14:20:14 INFO Utils: Copying /mnt/spark/spark_1.3.1.fat.jar to > /mnt/spark/active/spark-b084b800-1ede-4a4c-a5d6-9860070ba736/-12338568801433366393530_cache > 15/06/03 14:20:14 INFO Utils: Copying > /mnt/spark/active/spark-b084b800-1ede-4a4c-a5d6-9860070ba736/-12338568801433366393530_cache > to > /mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./spark_1.3.1.fat.jar > 15/06/03 14:20:14 INFO Executor: Adding > file:/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./spark_1.3.1.fat.jar > to class loader > 15/06/03 14:20:14 INFO HttpBroadcast: Started reading broadcast variable 0 > 15/06/03 14:20:14 INFO MemoryStore: ensureFreeSpace(11032) called with > curMem=0, maxMem=16669841817 > 15/06/03 14:20:14 INFO MemoryStore: Block broadcast_0 stored as values in > memory (estimated size 10.8 KB, free 15.5 GB) > 15/06/03 14:20:14 INFO HttpBroadcast: Reading broadcast variable 0 took > 0.123571332 s > 15/06/03 14:20:15 INFO SparkContext: Running Spark version 1.3.1 > 15/06/03 14:20:15 WARN SparkConf: In Spark 1.0 and later spark.local.dir will > be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS > in mesos/standalone and LOCAL_DIRS in YARN). > 15/06/03 14:20:15 INFO SecurityManager: Changing view acls to: admin > 15/06/03 14:20:15 INFO SecurityManager: Changing modify acls to: admin > 15/06/03 14:20:15 INFO SecurityManager: SecurityManager: authentication > disabled; ui acls disabled; users with view permissions: Set(admin); users > with modify permissions: Set(admin) > 15/06/03 14:20:15 INFO Slf4jLogger: Slf4jLogger started > 15/06/03 14:20:15 INFO Remoting: Starting remoting > 15/06/03 14:20:15 INFO Remoting: Remoting started; listening on addresses > :[akka.tcp://sparkDriver@10.240.241.48:59187] > 15/06/03 14:20:15 INFO Utils: Successfully started service 'sparkDriver' on > port 59187. > 15/06/03 14:20:15 INFO SparkEnv: Registering MapOutputTracker > 15/06/03 14:20:15 INFO SparkEnv: Registering BlockManagerMaster > 15/06/03 14:20:15 INFO DiskBlockManager: Created local directory at > /mnt/spark/active/spark-563a0744-77e4-4589-8672-6ec20cb5aac2/blockmgr-972a80d1-dfef-4c3a-a679-58a5aae4d331 > 15/06/03 14:20:15 INFO MemoryStore: MemoryStore started with capacity 15.5 GB > 15/06/03 14:20:15 INFO HttpFileServer: HTTP File server directory is > /mnt/spark/active/spark-bb2e728c-88f2-4457-b88b-0b2f4f313e0b/httpd-547ca257-dad4-447e-9473-d090ba76f11f > 15/06/03 14:20:15 INFO HttpServer: Starting HTTP Server > 15/06/03 14:20:15 INFO Server: jetty-8.y.z-SNAPSHOT > 15/06/03 14:20:15 INFO AbstractConnector: Started > SocketConnector@0.0.0.0:43459 > 15/06/03 14:20:15 INFO Utils: Successfully started service 'HTTP file server' > on port 43459. > 15/06/03 14:20:15 INFO SparkEnv: Registering OutputCommitCoordinator > 15/06/03 14:20:15 INFO Server: jetty-8.y.z-SNAPSHOT > 15/06/03 14:20:15 INFO AbstractConnector: Started > SelectChannelConnector@0.0.0.0:4040 > 15/06/03 14:20:15 INFO Utils: Successfully started service 'SparkUI' on port > 4040. > 15/06/03 14:20:15 INFO SparkUI: Started SparkUI at http://10.240.241.48:4040 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@712: Client > environment:zookeeper.version=zookeeper C client 3.4.5 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@716: Client > environment:host.name=mesos-slave4 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@723: Client > environment:os.name=Linux > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@724: Client > environment:os.arch=3.16.0-0.bpo.4-amd64 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@725: Client > environment:os.version=#1 SMP Debian 3.16.7-ckt4-3~bpo70+1 (2015-02-12) > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@733: Client > environment:user.name=admin > I0603 14:20:15.705294 18463 sched.cpp:157] Version: 0.22.0 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@741: Client > environment:user.home=/home/admin > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@753: Client > environment:user.dir=/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3 > 2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@zookeeper_init@786: > Initiating client connection, host=10.240.119.56:2181 sessionTimeout=10000 > watcher=0x7fd4bca88200 sessionId=0 sessionPasswd=<null> context=0x26b7ad0 > flags=0 > 2015-06-03 14:20:15,707:18296(0x7fd4b2bf6700):ZOO_INFO@check_events@1703: > initiated connection to server [10.240.119.56:2181] > 2015-06-03 14:20:15,711:18296(0x7fd4b2bf6700):ZOO_INFO@check_events@1750: > session establishment complete on server [10.240.119.56:2181], > sessionId=0x14da609b63100d5, negotiated timeout=10000 > I0603 14:20:15.711565 18470 group.cpp:313] Group process > (group(1)@10.240.241.48:45432) connected to ZooKeeper > I0603 14:20:15.711585 18470 group.cpp:790] Syncing group operations: queue > size (joins, cancels, datas) = (0, 0, 0) > I0603 14:20:15.711596 18470 group.cpp:385] Trying to create path '/mesos' in > ZooKeeper > I0603 14:20:15.713131 18470 detector.cpp:138] Detected a new leader: > (id='180') > I0603 14:20:15.713296 18466 group.cpp:659] Trying to get > '/mesos/info_0000000180' in ZooKeeper > I0603 14:20:15.713991 18466 detector.cpp:452] A new leading master > (UPID=master@10.240.119.56:5050) is detected > I0603 14:20:15.714061 18466 sched.cpp:254] New master detected at > master@10.240.119.56:5050 > I0603 14:20:15.714205 18466 sched.cpp:264] No credentials provided. > Attempting to register without authentication > I0603 14:20:15.716712 18466 sched.cpp:448] Framework registered with > 20150530-111452-947384330-5050-2394-0200 > 15/06/03 14:20:15 INFO CoarseMesosSchedulerBackend: Registered as framework > ID 20150530-111452-947384330-5050-2394-0200 > 15/06/03 14:20:15 INFO NettyBlockTransferService: Server created on 34315 > 15/06/03 14:20:15 INFO BlockManagerMaster: Trying to register BlockManager > 15/06/03 14:20:15 INFO BlockManagerMasterActor: Registering block manager > 10.240.241.48:34315 with 15.5 GB RAM, BlockManagerId(<driver>, 10.240.241.48, > 34315) > 15/06/03 14:20:15 INFO BlockManagerMaster: Registered BlockManager > 15/06/03 14:20:15 INFO EventLoggingListener: Logging events to > file:///mnt/spark/sparkLogs/20150530-111452-947384330-5050-2394-0200 > 15/06/03 14:20:15 INFO CoarseMesosSchedulerBackend: SchedulerBackend is ready > for scheduling beginning after reached minRegisteredResourcesRatio: 0.0 > 15/06/03 14:20:16 INFO NettyUtil: Found Netty's native epoll transport in the > classpath, using it > 15/06/03 14:20:17 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1' > for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct > datacenter name with DCAwareRoundRobinPolicy constructor) > 15/06/03 14:20:17 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 > added > 15/06/03 14:20:17 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042 > added > 15/06/03 14:20:17 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 > added > 15/06/03 14:20:17 INFO CassandraConnector: Connected to Cassandra cluster: > clusterName > 15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1935 > bytes result sent to driver > 15/06/03 14:20:18 INFO CassandraConnector: Disconnected from Cassandra > cluster: clusterName > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 1 > 15/06/03 14:20:18 INFO Executor: Running task 0.0 in stage 1.0 (TID 1) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 2 > 15/06/03 14:20:18 INFO Executor: Running task 1.0 in stage 1.0 (TID 2) > 15/06/03 14:20:18 INFO HttpBroadcast: Started reading broadcast variable 2 > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(2288) called with > curMem=0, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block broadcast_2 stored as values in > memory (estimated size 2.2 KB, free 15.5 GB) > 15/06/03 14:20:18 INFO HttpBroadcast: Reading broadcast variable 2 took > 0.005535572 s > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_2 locally > 15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 908 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 1.0 in stage 1.0 (TID 2). 908 > bytes result sent to driver > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 3 > 15/06/03 14:20:18 INFO Executor: Running task 0.0 in stage 2.0 (TID 3) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 4 > 15/06/03 14:20:18 INFO Executor: Running task 1.0 in stage 2.0 (TID 4) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 5 > 15/06/03 14:20:18 INFO Executor: Running task 2.0 in stage 2.0 (TID 5) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 6 > 15/06/03 14:20:18 INFO Executor: Running task 3.0 in stage 2.0 (TID 6) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 7 > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 8 > 15/06/03 14:20:18 INFO Executor: Running task 4.0 in stage 2.0 (TID 7) > 15/06/03 14:20:18 INFO Executor: Running task 5.0 in stage 2.0 (TID 8) > 15/06/03 14:20:18 INFO HttpBroadcast: Started reading broadcast variable 3 > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(2688) called with > curMem=2288, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block broadcast_3 stored as values in > memory (estimated size 2.6 KB, free 15.5 GB) > 15/06/03 14:20:18 INFO HttpBroadcast: Reading broadcast variable 3 took > 0.008771053 s > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO MapOutputTrackerWorker: Updating epoch to 1 and > clearing cache > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_1 not found, computing it > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_5 not found, computing it > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_2 not found, computing it > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_0 not found, computing it > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_3 not found, computing it > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_4 not found, computing it > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 0, fetching them > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor > = Actor[akka://sparkDriver/user/MapOutputTracker#-495616855] > 15/06/03 14:20:18 INFO MapOutputTrackerMasterActor: Asked to send map output > locations for shuffle 0 to sparkDriver > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Size of output statuses for > shuffle 0 is 82 bytes > 15/06/03 14:20:18 INFO MapOutputTrackerMaster: Got the output locations > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 6 ms > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 6 ms > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 6 ms > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 6 ms > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 5 ms > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 6 ms > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=4976, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_2 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=4992, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_4 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5008, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_5 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5024, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_0 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_2 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5040, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_3 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5056, maxMem=16669841817 > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_4 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_4 > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_2 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_1 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_5 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_5 > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_0 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_0 > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_3 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_3 > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_1 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_1 > 15/06/03 14:20:18 INFO Executor: Finished task 1.0 in stage 2.0 (TID 4). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 2.0 (TID 3). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 4.0 in stage 2.0 (TID 7). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 5.0 in stage 2.0 (TID 8). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 3.0 in stage 2.0 (TID 6). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO Executor: Finished task 2.0 in stage 2.0 (TID 5). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 9 > 15/06/03 14:20:18 INFO Executor: Running task 6.0 in stage 2.0 (TID 9) > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_6 not found, computing it > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5072, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_6 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_6 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_6 > 15/06/03 14:20:18 INFO Executor: Finished task 6.0 in stage 2.0 (TID 9). 1497 > bytes result sent to driver > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 10 > 15/06/03 14:20:18 INFO Executor: Running task 7.0 in stage 2.0 (TID 10) > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_7 not found, computing it > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5088, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_7 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_7 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_7 > 15/06/03 14:20:18 INFO Executor: Finished task 7.0 in stage 2.0 (TID 10). > 1497 bytes result sent to driver > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 11 > 15/06/03 14:20:18 INFO Executor: Running task 8.0 in stage 2.0 (TID 11) > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_8 not found, computing it > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5104, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_8 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_8 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_8 > 15/06/03 14:20:18 INFO Executor: Finished task 8.0 in stage 2.0 (TID 11). > 1497 bytes result sent to driver > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 12 > 15/06/03 14:20:18 INFO Executor: Running task 9.0 in stage 2.0 (TID 12) > 15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_9 not found, computing it > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5120, maxMem=16669841817 > 15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_9 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_9 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 13 > 15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_9 > 15/06/03 14:20:18 INFO Executor: Running task 10.0 in stage 2.0 (TID 13) > 15/06/03 14:20:19 INFO Executor: Finished task 9.0 in stage 2.0 (TID 12). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_10 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5136, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_10 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 14 > 15/06/03 14:20:19 INFO Executor: Running task 11.0 in stage 2.0 (TID 14) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_10 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_10 > 15/06/03 14:20:19 INFO Executor: Finished task 10.0 in stage 2.0 (TID 13). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_11 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5152, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_11 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_11 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_11 > 15/06/03 14:20:19 INFO Executor: Finished task 11.0 in stage 2.0 (TID 14). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 15 > 15/06/03 14:20:19 INFO Executor: Running task 12.0 in stage 2.0 (TID 15) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_12 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5168, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_12 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_12 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_12 > 15/06/03 14:20:19 INFO Executor: Finished task 12.0 in stage 2.0 (TID 15). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 16 > 15/06/03 14:20:19 INFO Executor: Running task 13.0 in stage 2.0 (TID 16) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_13 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5184, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_13 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_13 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_13 > 15/06/03 14:20:19 INFO Executor: Finished task 13.0 in stage 2.0 (TID 16). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 17 > 15/06/03 14:20:19 INFO Executor: Running task 14.0 in stage 2.0 (TID 17) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_14 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5200, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_14 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_14 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_14 > 15/06/03 14:20:19 INFO Executor: Finished task 14.0 in stage 2.0 (TID 17). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 18 > 15/06/03 14:20:19 INFO Executor: Running task 15.0 in stage 2.0 (TID 18) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_15 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5216, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_15 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_15 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_15 > 15/06/03 14:20:19 INFO Executor: Finished task 15.0 in stage 2.0 (TID 18). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 19 > 15/06/03 14:20:19 INFO Executor: Running task 16.0 in stage 2.0 (TID 19) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_16 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5232, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_16 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_16 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_16 > 15/06/03 14:20:19 INFO Executor: Finished task 16.0 in stage 2.0 (TID 19). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 20 > 15/06/03 14:20:19 INFO Executor: Running task 17.0 in stage 2.0 (TID 20) > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 21 > 15/06/03 14:20:19 INFO Executor: Running task 18.0 in stage 2.0 (TID 21) > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_17 not found, computing > it > 15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 22 > 15/06/03 14:20:19 INFO Executor: Running task 19.0 in stage 2.0 (TID 22) > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_18 not found, computing > it > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5248, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_17 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5264, maxMem=16669841817 > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_17 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_17 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_18 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO Executor: Finished task 17.0 in stage 2.0 (TID 20). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_18 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_18 > 15/06/03 14:20:19 INFO Executor: Finished task 18.0 in stage 2.0 (TID 21). > 1497 bytes result sent to driver > 15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_19 not found, computing > it > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=5280, maxMem=16669841817 > 15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_19 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_19 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_19 > 15/06/03 14:20:19 INFO Executor: Finished task 19.0 in stage 2.0 (TID 22). > 1497 bytes result sent to driver > 15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 23 > 15/06/03 14:25:20 INFO Executor: Running task 0.0 in stage 3.0 (TID 23) > 15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 24 > 15/06/03 14:25:20 INFO Executor: Running task 1.0 in stage 3.0 (TID 24) > 15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 25 > 15/06/03 14:25:20 INFO Executor: Running task 2.0 in stage 3.0 (TID 25) > 15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 26 > 15/06/03 14:25:20 INFO Executor: Running task 3.0 in stage 3.0 (TID 26) > 15/06/03 14:25:20 INFO HttpBroadcast: Started reading broadcast variable 4 > 15/06/03 14:25:20 INFO MemoryStore: ensureFreeSpace(11888) called with > curMem=5296, maxMem=16669841817 > 15/06/03 14:25:20 INFO MemoryStore: Block broadcast_4 stored as values in > memory (estimated size 11.6 KB, free 15.5 GB) > 15/06/03 14:25:20 INFO HttpBroadcast: Reading broadcast variable 4 took > 0.007110946 s > 15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally > 15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally > 15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_2 not found, computing it > 15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally > 15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_1 not found, computing it > 15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_3 not found, computing it > 15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_0 not found, computing it > 15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1' > for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct > datacenter name with DCAwareRoundRobinPolicy constructor) > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 > added > 15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster: > clusterName > 15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1' > for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct > datacenter name with DCAwareRoundRobinPolicy constructor) > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 > added > 15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster: > clusterName > 15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1' > for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct > datacenter name with DCAwareRoundRobinPolicy constructor) > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 > added > 15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster: > clusterName > 15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1' > for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct > datacenter name with DCAwareRoundRobinPolicy constructor) > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042 > added > 15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 > added > 15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster: > clusterName > 15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra > cluster: clusterName > 15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra > cluster: clusterName > 15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra > cluster: clusterName > 15/06/03 14:25:22 INFO MemoryStore: ensureFreeSpace(803320) called with > curMem=17184, maxMem=16669841817 > 15/06/03 14:25:22 INFO MemoryStore: Block rdd_1_2 stored as values in memory > (estimated size 784.5 KB, free 15.5 GB) > 15/06/03 14:25:22 INFO BlockManagerInfo: Added rdd_1_2 in memory on > 10.240.241.48:34315 (size: 784.5 KB, free: 15.5 GB) > 15/06/03 14:25:22 INFO BlockManagerMaster: Updated info of block rdd_1_2 > 15/06/03 14:25:22 INFO Executor: Finished task 2.0 in stage 3.0 (TID 25). > 2599 bytes result sent to driver > 15/06/03 14:25:28 INFO MemoryStore: ensureFreeSpace(17551856) called with > curMem=820504, maxMem=16669841817 > 15/06/03 14:25:28 INFO MemoryStore: Block rdd_1_3 stored as values in memory > (estimated size 16.7 MB, free 15.5 GB) > 15/06/03 14:25:28 INFO BlockManagerInfo: Added rdd_1_3 in memory on > 10.240.241.48:34315 (size: 16.7 MB, free: 15.5 GB) > 15/06/03 14:25:28 INFO BlockManagerMaster: Updated info of block rdd_1_3 > 15/06/03 14:25:29 INFO MemoryStore: ensureFreeSpace(17846696) called with > curMem=18372360, maxMem=16669841817 > 15/06/03 14:25:29 INFO MemoryStore: Block rdd_1_0 stored as values in memory > (estimated size 17.0 MB, free 15.5 GB) > 15/06/03 14:25:29 INFO BlockManagerInfo: Added rdd_1_0 in memory on > 10.240.241.48:34315 (size: 17.0 MB, free: 15.5 GB) > 15/06/03 14:25:29 INFO BlockManagerMaster: Updated info of block rdd_1_0 > 15/06/03 14:25:30 INFO MemoryStore: ensureFreeSpace(20620640) called with > curMem=36219056, maxMem=16669841817 > 15/06/03 14:25:30 INFO MemoryStore: Block rdd_1_1 stored as values in memory > (estimated size 19.7 MB, free 15.5 GB) > 15/06/03 14:25:30 INFO BlockManagerInfo: Added rdd_1_1 in memory on > 10.240.241.48:34315 (size: 19.7 MB, free: 15.5 GB) > 15/06/03 14:25:30 INFO BlockManagerMaster: Updated info of block rdd_1_1 > 15/06/03 14:25:30 INFO Executor: Finished task 3.0 in stage 3.0 (TID 26). > 2599 bytes result sent to driver > 15/06/03 14:25:30 INFO Executor: Finished task 0.0 in stage 3.0 (TID 23). > 2599 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 3.0 (TID 24). > 2599 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 27 > 15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 4.0 (TID 27) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 28 > 15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 4.0 (TID 28) > 15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 5 > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 29 > 15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 4.0 (TID 29) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 30 > 15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 4.0 (TID 30) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 31 > 15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 4.0 (TID 31) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 32 > 15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 4.0 (TID 32) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(2632) called with > curMem=56839696, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block broadcast_5 stored as values in > memory (estimated size 2.6 KB, free 15.5 GB) > 15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 5 took > 0.007544338 s > 15/06/03 14:25:31 INFO MapOutputTrackerWorker: Updating epoch to 2 and > clearing cache > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_5 not found, computing > it > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_1 not found, computing > it > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 1, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor > = Actor[akka://sparkDriver/user/MapOutputTracker#-495616855] > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 1, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMasterActor: Asked to send map output > locations for shuffle 1 to sparkDriver > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_3 not found, computing > it > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_2 not found, computing > it > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_0 not found, computing > it > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Size of output statuses for > shuffle 1 is 82 bytes > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 1, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 1, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 1, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Got the output locations > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842328, maxMem=16669841817 > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_5 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_4 not found, computing > it > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842344, maxMem=16669841817 > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_5 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_0 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_5 > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842360, maxMem=16669841817 > 15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 4.0 (TID 32). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_0 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_1 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_0 > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842376, maxMem=16669841817 > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_1 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 4.0 (TID 27). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_1 > 15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 4.0 (TID 28). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_2 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842392, maxMem=16669841817 > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_2 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_3 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_2 > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842408, maxMem=16669841817 > 15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 4.0 (TID 29). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_4 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_3 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_3 > 15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 4.0 (TID 30). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_4 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_4 > 15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 4.0 (TID 31). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 33 > 15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 4.0 (TID 33) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_6 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842424, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_6 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_6 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_6 > 15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 4.0 (TID 33). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 34 > 15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 4.0 (TID 34) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_7 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842440, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_7 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_7 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_7 > 15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 4.0 (TID 34). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 35 > 15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 4.0 (TID 35) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_8 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842456, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_8 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_8 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_8 > 15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 4.0 (TID 35). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 36 > 15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 4.0 (TID 36) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 37 > 15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 4.0 (TID 37) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 38 > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 4.0 (TID 38) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_9 not found, computing > it > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_10 not found, computing > it > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_11 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842472, maxMem=16669841817 > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_10 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842488, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_11 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_10 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842504, maxMem=16669841817 > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_10 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_9 stored as values in memory > (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_11 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 4.0 (TID 37). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_11 > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_9 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 4.0 (TID 38). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_9 > 15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 4.0 (TID 36). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 39 > 15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 4.0 (TID 39) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_12 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842520, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_12 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_12 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_12 > 15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 4.0 (TID 39). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 40 > 15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 4.0 (TID 40) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_13 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842536, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_13 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_13 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_13 > 15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 4.0 (TID 40). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 41 > 15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 4.0 (TID 41) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_14 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842552, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_14 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_14 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_14 > 15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 4.0 (TID 41). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 42 > 15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 4.0 (TID 42) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_15 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842568, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_15 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_15 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_15 > 15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 4.0 (TID 42). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 43 > 15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 4.0 (TID 43) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_16 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842584, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_16 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_16 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_16 > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 44 > 15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 4.0 (TID 43). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 4.0 (TID 44) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_17 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842600, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_17 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_17 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_17 > 15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 4.0 (TID 44). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 45 > 15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 4.0 (TID 45) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_18 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842616, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_18 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_18 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_18 > 15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 4.0 (TID 45). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 46 > 15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 4.0 (TID 46) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally > 15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_19 not found, computing > it > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with > curMem=56842632, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_19 stored as values in > memory (estimated size 16.0 B, free 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_19 in memory on > 10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB) > 15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_19 > 15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 4.0 (TID 46). > 1497 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 47 > 15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 6.0 (TID 47) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 48 > 15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 6.0 (TID 48) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 49 > 15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 6.0 (TID 49) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 50 > 15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 6.0 (TID 50) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 51 > 15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 6.0 (TID 51) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 52 > 15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 6.0 (TID 52) > 15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 6 > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(2760) called with > curMem=56842648, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block broadcast_6 stored as values in > memory (estimated size 2.7 KB, free 15.5 GB) > 15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 6 took > 0.004133133 s > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_1 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_4 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_2 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_3 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_0 locally > 15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 6.0 (TID 48). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 6.0 (TID 50). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 6.0 (TID 47). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 6.0 (TID 52). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 6.0 (TID 51). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 6.0 (TID 49). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 53 > 15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 6.0 (TID 53) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 54 > 15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 6.0 (TID 54) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 55 > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 6.0 (TID 55) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 56 > 15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 6.0 (TID 56) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 6.0 (TID 53). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 6.0 (TID 54). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_8 locally > 15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 6.0 (TID 55). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_9 locally > 15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 6.0 (TID 56). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 57 > 15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 6.0 (TID 57) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_10 locally > 15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 6.0 (TID 57). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 58 > 15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 6.0 (TID 58) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_11 locally > 15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 6.0 (TID 58). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 59 > 15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 6.0 (TID 59) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_12 locally > 15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 6.0 (TID 59). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 60 > 15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 6.0 (TID 60) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_13 locally > 15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 6.0 (TID 60). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 61 > 15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 6.0 (TID 61) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_14 locally > 15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 6.0 (TID 61). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 62 > 15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 6.0 (TID 62) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 63 > 15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 6.0 (TID 63) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_15 locally > 15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 6.0 (TID 62). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_16 locally > 15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 6.0 (TID 63). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 64 > 15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 6.0 (TID 64) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_17 locally > 15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 6.0 (TID 64). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 65 > 15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 6.0 (TID 65) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_18 locally > 15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 6.0 (TID 65). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 66 > 15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 6.0 (TID 66) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_19 locally > 15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 6.0 (TID 66). > 1834 bytes result sent to driver > 15/06/03 14:25:31 INFO CassandraConnector: Disconnected from Cassandra > cluster: clusterName > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 67 > 15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 8.0 (TID 67) > 15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 7 > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 68 > 15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 8.0 (TID 68) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 69 > 15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 8.0 (TID 69) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 70 > 15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 8.0 (TID 70) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 71 > 15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 8.0 (TID 71) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 72 > 15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 8.0 (TID 72) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(3536) called with > curMem=56845408, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block broadcast_7 stored as values in > memory (estimated size 3.5 KB, free 15.5 GB) > 15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 7 took > 0.009490517 s > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_2 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_0 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_1 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_3 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_5 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_4 locally > 15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 8.0 (TID 70). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 8.0 (TID 71). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 8.0 (TID 67). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 8.0 (TID 69). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 8.0 (TID 72). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 8.0 (TID 68). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 73 > 15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 8.0 (TID 73) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_6 locally > 15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 8.0 (TID 73). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 74 > 15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 8.0 (TID 74) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_7 locally > 15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 8.0 (TID 74). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 75 > 15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 8.0 (TID 75) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_8 locally > 15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 8.0 (TID 75). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 76 > 15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 8.0 (TID 76) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_9 locally > 15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 8.0 (TID 76). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 77 > 15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 8.0 (TID 77) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_10 locally > 15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 8.0 (TID 77). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 78 > 15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 8.0 (TID 78) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_11 locally > 15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 8.0 (TID 78). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 79 > 15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 8.0 (TID 79) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_12 locally > 15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 8.0 (TID 79). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 80 > 15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 8.0 (TID 80) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 81 > 15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 8.0 (TID 81) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_14 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_13 locally > 15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 8.0 (TID 81). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 8.0 (TID 80). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 82 > 15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 8.0 (TID 82) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_15 locally > 15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 8.0 (TID 82). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 83 > 15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 8.0 (TID 83) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_16 locally > 15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 8.0 (TID 83). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 84 > 15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 8.0 (TID 84) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_17 locally > 15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 8.0 (TID 84). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 85 > 15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 8.0 (TID 85) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_18 locally > 15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 8.0 (TID 85). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 86 > 15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 8.0 (TID 86) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally > 15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_19 locally > 15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 8.0 (TID 86). > 2030 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 87 > 15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 9.0 (TID 87) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 88 > 15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 9.0 (TID 88) > 15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 8 > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 89 > 15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 9.0 (TID 89) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 90 > 15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 9.0 (TID 90) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 91 > 15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 9.0 (TID 91) > 15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(3736) called with > curMem=56848944, maxMem=16669841817 > 15/06/03 14:25:31 INFO MemoryStore: Block broadcast_8 stored as values in > memory (estimated size 3.6 KB, free 15.5 GB) > 15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 8 took > 0.005282429 s > 15/06/03 14:25:31 INFO MapOutputTrackerWorker: Updating epoch to 3 and > clearing cache > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 2, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor > = Actor[akka://sparkDriver/user/MapOutputTracker#-495616855] > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 2, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 2, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMasterActor: Asked to send map output > locations for shuffle 2 to sparkDriver > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 2, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Size of output statuses for > shuffle 2 is 82 bytes > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for > shuffle 2, fetching them > 15/06/03 14:25:31 INFO MapOutputTrackerMaster: Got the output locations > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 92 > 15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 9.0 (TID 92) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 9.0 (TID 89). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 9.0 (TID 92). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 9.0 (TID 87). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 9.0 (TID 91). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 9.0 (TID 90). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 9.0 (TID 88). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 93 > 15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 9.0 (TID 93) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 9.0 (TID 93). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 94 > 15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 9.0 (TID 94) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 9.0 (TID 94). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 95 > 15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 9.0 (TID 95) > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 96 > 15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 9.0 (TID 96) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 9.0 (TID 95). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 9.0 (TID 96). 928 > bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 97 > 15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 9.0 (TID 97) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 9.0 (TID 97). > 928 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 98 > 15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 9.0 (TID 98) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 9.0 (TID 98). > 928 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 99 > 15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 9.0 (TID 99) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 9.0 (TID 99). > 928 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 100 > 15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 9.0 (TID 100) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 9.0 (TID 100). > 928 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 101 > 15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 9.0 (TID 101) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 9.0 (TID 101). > 928 bytes result sent to driver > 15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 102 > 15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 9.0 (TID 102) > 15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 9.0 (TID 102). > 928 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 103 > 15/06/03 14:25:32 INFO Executor: Running task 16.0 in stage 9.0 (TID 103) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 16.0 in stage 9.0 (TID 103). > 928 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 104 > 15/06/03 14:25:32 INFO Executor: Running task 17.0 in stage 9.0 (TID 104) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 105 > 15/06/03 14:25:32 INFO Executor: Running task 18.0 in stage 9.0 (TID 105) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 17.0 in stage 9.0 (TID 104). > 928 bytes result sent to driver > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 18.0 in stage 9.0 (TID 105). > 928 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 106 > 15/06/03 14:25:32 INFO Executor: Running task 19.0 in stage 9.0 (TID 106) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 19.0 in stage 9.0 (TID 106). > 928 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 107 > 15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 12.0 (TID 107) > 15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 9 > 15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with > curMem=56852680, maxMem=16669841817 > 15/06/03 14:25:32 INFO MemoryStore: Block broadcast_9 stored as values in > memory (estimated size 3.9 KB, free 15.5 GB) > 15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 9 took > 0.010923501 s > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 12.0 (TID 107). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 108 > 15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 15.0 (TID 108) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 109 > 15/06/03 14:25:32 INFO Executor: Running task 1.0 in stage 15.0 (TID 109) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 110 > 15/06/03 14:25:32 INFO Executor: Running task 2.0 in stage 15.0 (TID 110) > 15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 10 > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 111 > 15/06/03 14:25:32 INFO Executor: Running task 3.0 in stage 15.0 (TID 111) > 15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with > curMem=56856664, maxMem=16669841817 > 15/06/03 14:25:32 INFO MemoryStore: Block broadcast_10 stored as values in > memory (estimated size 3.9 KB, free 15.5 GB) > 15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 10 took > 0.003318126 s > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 15.0 (TID 108). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO Executor: Finished task 3.0 in stage 15.0 (TID 111). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 2.0 in stage 15.0 (TID 110). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO Executor: Finished task 1.0 in stage 15.0 (TID 109). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 112 > 15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 18.0 (TID 112) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 113 > 15/06/03 14:25:32 INFO Executor: Running task 1.0 in stage 18.0 (TID 113) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 114 > 15/06/03 14:25:32 INFO Executor: Running task 2.0 in stage 18.0 (TID 114) > 15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 11 > 15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with > curMem=56860648, maxMem=16669841817 > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 115 > 15/06/03 14:25:32 INFO Executor: Running task 3.0 in stage 18.0 (TID 115) > 15/06/03 14:25:32 INFO MemoryStore: Block broadcast_11 stored as values in > memory (estimated size 3.9 KB, free 15.5 GB) > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 116 > 15/06/03 14:25:32 INFO Executor: Running task 4.0 in stage 18.0 (TID 116) > 15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 11 took > 0.005466598 s > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 1 ms > 15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 18.0 (TID 112). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO Executor: Finished task 1.0 in stage 18.0 (TID 113). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO Executor: Finished task 3.0 in stage 18.0 (TID 115). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 117 > 15/06/03 14:25:32 INFO Executor: Finished task 2.0 in stage 18.0 (TID 114). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO Executor: Running task 5.0 in stage 18.0 (TID 117) > 15/06/03 14:25:32 INFO Executor: Finished task 4.0 in stage 18.0 (TID 116). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 5.0 in stage 18.0 (TID 117). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 118 > 15/06/03 14:25:32 INFO Executor: Running task 6.0 in stage 18.0 (TID 118) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 6.0 in stage 18.0 (TID 118). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 119 > 15/06/03 14:25:32 INFO Executor: Running task 7.0 in stage 18.0 (TID 119) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 7.0 in stage 18.0 (TID 119). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 120 > 15/06/03 14:25:32 INFO Executor: Running task 8.0 in stage 18.0 (TID 120) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 121 > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO Executor: Running task 9.0 in stage 18.0 (TID 121) > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 8.0 in stage 18.0 (TID 120). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 9.0 in stage 18.0 (TID 121). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 122 > 15/06/03 14:25:32 INFO Executor: Running task 10.0 in stage 18.0 (TID 122) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 10.0 in stage 18.0 (TID 122). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 123 > 15/06/03 14:25:32 INFO Executor: Running task 11.0 in stage 18.0 (TID 123) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 11.0 in stage 18.0 (TID 123). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 124 > 15/06/03 14:25:32 INFO Executor: Running task 12.0 in stage 18.0 (TID 124) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 12.0 in stage 18.0 (TID 124). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 125 > 15/06/03 14:25:32 INFO Executor: Running task 13.0 in stage 18.0 (TID 125) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 13.0 in stage 18.0 (TID 125). > 908 bytes result sent to driver > 15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 126 > 15/06/03 14:25:32 INFO Executor: Running task 14.0 in stage 18.0 (TID 126) > 15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty > blocks out of 0 blocks > 15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches > in 0 ms > 15/06/03 14:25:32 INFO Executor: Finished task 14.0 in stage 18.0 (TID 126). > 908 bytes result sent to driver > 15/06/03 14:25:34 ERROR CoarseGrainedExecutorBackend: Driver Disassociated > [akka.tcp://sparkExecutor@10.240.241.48:40338] -> > [akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448] > disassociated! Shutting down. > I0603 14:25:34.036478 18289 exec.cpp:379] Executor asked to shutdown > 15/06/03 14:25:34 WARN ReliableDeliverySupervisor: Association with remote > system > [akka.tcp://sparkDriver@mesos-master0.c.vibrant-magpie-705.internal:40448] > has failed, address is now gated for [5000] ms. Reason is: [Disassociated]. > 15/06/03 14:25:34 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL 15: > SIGTERM > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org