Malte Buecken created SPARK-8089:
------------------------------------
Summary: Operations on RDD cause empty collections
Key: SPARK-8089
URL: https://issues.apache.org/jira/browse/SPARK-8089
Project: Spark
Issue Type: Bug
Components: Mesos
Affects Versions: 1.3.1, 1.2.2
Environment: Debian 3.16.7 64bit, mesos 0.22, cassandra 2.1
Reporter: Malte Buecken
I honestly do not know what is happening here:
I use Apache Spark on Mesos and had to switch to httpbroadcast because
Torrentbroadcast fails with SPARK-5594
Now I cannot use any operation on my RDDs without losing all of their data.
I tried both Spark 1.2.2 and Spark 1.3.1, both with the same effect
example:
val data = sc.parallelize(Array(2,3,4,5,6,7,8,9))
val newdata = data.repartition(2)
println(newdata.count()) // 0
same with groupBy and other operations. I always end up with an empty RDD
Attached a log where I try to different operations, all ending up with empty
RDDs before finally failing when I try data.first(). There is a long
Thread.sleep when I wanted to look around the web UI
{code}
I0603 14:19:54.324846 18091 fetcher.cpp:214] Fetching URI
'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz'
I0603 14:19:55.870537 18091 fetcher.cpp:99] Fetching URI
'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz' using Hadoop Client
I0603 14:19:55.870594 18091 fetcher.cpp:109] Downloading resource from
'hdfs://mesos-master0/spark-1.3.1-bin-hadoop2.3.tgz' to
'/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3.tgz'
I0603 14:20:00.973871 18091 fetcher.cpp:78] Extracted resource
'/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3.tgz'
into
'/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e'
I0603 14:20:01.038312 18285 exec.cpp:132] Version: 0.22.0
I0603 14:20:01.040046 18286 exec.cpp:206] Executor registered on slave
20150530-111452-947384330-5050-2394-S3
Using Spark's default log4j profile: org/apache/spark/log4j-defaults.properties
15/06/03 14:20:02 INFO CoarseGrainedExecutorBackend: Registered signal handlers
for [TERM, HUP, INT]
15/06/03 14:20:03 WARN NativeCodeLoader: Unable to load native-hadoop library
for your platform... using builtin-java classes where applicable
15/06/03 14:20:03 INFO SecurityManager: Changing view acls to: admin
15/06/03 14:20:03 INFO SecurityManager: Changing modify acls to: admin
15/06/03 14:20:03 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(admin); users with
modify permissions: Set(admin)
15/06/03 14:20:03 INFO Slf4jLogger: Slf4jLogger started
15/06/03 14:20:04 INFO Remoting: Starting remoting
15/06/03 14:20:04 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://[email protected]:44036]
15/06/03 14:20:04 INFO Utils: Successfully started service 'driverPropsFetcher'
on port 44036.
15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Shutting down
remote daemon.
15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Remote daemon
shut down; proceeding with flushing remote transports.
15/06/03 14:20:04 INFO SecurityManager: Changing view acls to: admin
15/06/03 14:20:04 INFO SecurityManager: Changing modify acls to: admin
15/06/03 14:20:04 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(admin); users with
modify permissions: Set(admin)
15/06/03 14:20:04 INFO Slf4jLogger: Slf4jLogger started
15/06/03 14:20:04 INFO Remoting: Starting remoting
15/06/03 14:20:04 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://[email protected]:40338]
15/06/03 14:20:04 INFO RemoteActorRefProvider$RemotingTerminator: Remoting shut
down.
15/06/03 14:20:04 INFO Utils: Successfully started service 'sparkExecutor' on
port 40338.
15/06/03 14:20:04 INFO AkkaUtils: Connecting to MapOutputTracker:
akka.tcp://[email protected]:40448/user/MapOutputTracker
15/06/03 14:20:04 INFO AkkaUtils: Connecting to BlockManagerMaster:
akka.tcp://[email protected]:40448/user/BlockManagerMaster
15/06/03 14:20:04 INFO DiskBlockManager: Created local directory at
/mnt/spark/active/spark-36127f0b-e5c8-4f39-8137-8ab633b38df1/blockmgr-7d491649-bcb8-4a04-8395-acd7e880c549
15/06/03 14:20:04 INFO MemoryStore: MemoryStore started with capacity 15.5 GB
15/06/03 14:20:05 INFO AkkaUtils: Connecting to OutputCommitCoordinator:
akka.tcp://[email protected]:40448/user/OutputCommitCoordinator
15/06/03 14:20:05 INFO CoarseGrainedExecutorBackend: Connecting to driver:
akka.tcp://[email protected]:40448/user/CoarseGrainedScheduler
15/06/03 14:20:05 INFO CoarseGrainedExecutorBackend: Successfully registered
with driver
15/06/03 14:20:05 INFO Executor: Starting executor ID
20150530-111452-947384330-5050-2394-S3 on host 10.240.241.48
15/06/03 14:20:05 INFO NettyBlockTransferService: Server created on 60337
15/06/03 14:20:05 INFO BlockManagerMaster: Trying to register BlockManager
15/06/03 14:20:05 INFO BlockManagerMaster: Registered BlockManager
15/06/03 14:20:05 INFO AkkaUtils: Connecting to HeartbeatReceiver:
akka.tcp://[email protected]:40448/user/HeartbeatReceiver
15/06/03 14:20:14 INFO CoarseGrainedExecutorBackend: Got assigned task 0
15/06/03 14:20:14 INFO Executor: Running task 0.0 in stage 0.0 (TID 0)
15/06/03 14:20:14 INFO Executor: Fetching
http://10.240.119.56:51874/jars/graphit_2.10-1.0.jar with timestamp
1433366393559
15/06/03 14:20:14 INFO Utils: Fetching
http://10.240.119.56:51874/jars/graphit_2.10-1.0.jar to
/mnt/spark/active/spark-5ab6a324-e4a9-418a-8209-0924b76953e4/fetchFileTemp8154164797092940721.tmp
15/06/03 14:20:14 INFO Utils: Copying
/mnt/spark/active/spark-5ab6a324-e4a9-418a-8209-0924b76953e4/21079050381433366393559_cache
to
/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./graphit_2.10-1.0.jar
15/06/03 14:20:14 INFO Executor: Adding
file:/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./graphit_2.10-1.0.jar
to class loader
15/06/03 14:20:14 INFO Executor: Fetching file:/mnt/spark/spark_1.3.1.fat.jar
with timestamp 1433366393530
15/06/03 14:20:14 INFO Utils: Copying /mnt/spark/spark_1.3.1.fat.jar to
/mnt/spark/active/spark-b084b800-1ede-4a4c-a5d6-9860070ba736/-12338568801433366393530_cache
15/06/03 14:20:14 INFO Utils: Copying
/mnt/spark/active/spark-b084b800-1ede-4a4c-a5d6-9860070ba736/-12338568801433366393530_cache
to
/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./spark_1.3.1.fat.jar
15/06/03 14:20:14 INFO Executor: Adding
file:/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3/./spark_1.3.1.fat.jar
to class loader
15/06/03 14:20:14 INFO HttpBroadcast: Started reading broadcast variable 0
15/06/03 14:20:14 INFO MemoryStore: ensureFreeSpace(11032) called with
curMem=0, maxMem=16669841817
15/06/03 14:20:14 INFO MemoryStore: Block broadcast_0 stored as values in
memory (estimated size 10.8 KB, free 15.5 GB)
15/06/03 14:20:14 INFO HttpBroadcast: Reading broadcast variable 0 took
0.123571332 s
15/06/03 14:20:15 INFO SparkContext: Running Spark version 1.3.1
15/06/03 14:20:15 WARN SparkConf: In Spark 1.0 and later spark.local.dir will
be overridden by the value set by the cluster manager (via SPARK_LOCAL_DIRS in
mesos/standalone and LOCAL_DIRS in YARN).
15/06/03 14:20:15 INFO SecurityManager: Changing view acls to: admin
15/06/03 14:20:15 INFO SecurityManager: Changing modify acls to: admin
15/06/03 14:20:15 INFO SecurityManager: SecurityManager: authentication
disabled; ui acls disabled; users with view permissions: Set(admin); users with
modify permissions: Set(admin)
15/06/03 14:20:15 INFO Slf4jLogger: Slf4jLogger started
15/06/03 14:20:15 INFO Remoting: Starting remoting
15/06/03 14:20:15 INFO Remoting: Remoting started; listening on addresses
:[akka.tcp://[email protected]:59187]
15/06/03 14:20:15 INFO Utils: Successfully started service 'sparkDriver' on
port 59187.
15/06/03 14:20:15 INFO SparkEnv: Registering MapOutputTracker
15/06/03 14:20:15 INFO SparkEnv: Registering BlockManagerMaster
15/06/03 14:20:15 INFO DiskBlockManager: Created local directory at
/mnt/spark/active/spark-563a0744-77e4-4589-8672-6ec20cb5aac2/blockmgr-972a80d1-dfef-4c3a-a679-58a5aae4d331
15/06/03 14:20:15 INFO MemoryStore: MemoryStore started with capacity 15.5 GB
15/06/03 14:20:15 INFO HttpFileServer: HTTP File server directory is
/mnt/spark/active/spark-bb2e728c-88f2-4457-b88b-0b2f4f313e0b/httpd-547ca257-dad4-447e-9473-d090ba76f11f
15/06/03 14:20:15 INFO HttpServer: Starting HTTP Server
15/06/03 14:20:15 INFO Server: jetty-8.y.z-SNAPSHOT
15/06/03 14:20:15 INFO AbstractConnector: Started [email protected]:43459
15/06/03 14:20:15 INFO Utils: Successfully started service 'HTTP file server'
on port 43459.
15/06/03 14:20:15 INFO SparkEnv: Registering OutputCommitCoordinator
15/06/03 14:20:15 INFO Server: jetty-8.y.z-SNAPSHOT
15/06/03 14:20:15 INFO AbstractConnector: Started
[email protected]:4040
15/06/03 14:20:15 INFO Utils: Successfully started service 'SparkUI' on port
4040.
15/06/03 14:20:15 INFO SparkUI: Started SparkUI at http://10.240.241.48:4040
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@712: Client
environment:zookeeper.version=zookeeper C client 3.4.5
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@716: Client
environment:host.name=mesos-slave4
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@723: Client
environment:os.name=Linux
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@724: Client
environment:os.arch=3.16.0-0.bpo.4-amd64
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@725: Client
environment:os.version=#1 SMP Debian 3.16.7-ckt4-3~bpo70+1 (2015-02-12)
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@733: Client
environment:user.name=admin
I0603 14:20:15.705294 18463 sched.cpp:157] Version: 0.22.0
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@741: Client
environment:user.home=/home/admin
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@log_env@753: Client
environment:user.dir=/mnt/spark/slaves/20150530-111452-947384330-5050-2394-S3/frameworks/20150530-111452-947384330-5050-2394-0199/executors/0/runs/7ca5a436-1862-4809-a93d-721c95f1e97e/spark-1.3.1-bin-hadoop2.3
2015-06-03 14:20:15,705:18296(0x7fd4b6bfe700):ZOO_INFO@zookeeper_init@786:
Initiating client connection, host=10.240.119.56:2181 sessionTimeout=10000
watcher=0x7fd4bca88200 sessionId=0 sessionPasswd=<null> context=0x26b7ad0
flags=0
2015-06-03 14:20:15,707:18296(0x7fd4b2bf6700):ZOO_INFO@check_events@1703:
initiated connection to server [10.240.119.56:2181]
2015-06-03 14:20:15,711:18296(0x7fd4b2bf6700):ZOO_INFO@check_events@1750:
session establishment complete on server [10.240.119.56:2181],
sessionId=0x14da609b63100d5, negotiated timeout=10000
I0603 14:20:15.711565 18470 group.cpp:313] Group process
(group(1)@10.240.241.48:45432) connected to ZooKeeper
I0603 14:20:15.711585 18470 group.cpp:790] Syncing group operations: queue size
(joins, cancels, datas) = (0, 0, 0)
I0603 14:20:15.711596 18470 group.cpp:385] Trying to create path '/mesos' in
ZooKeeper
I0603 14:20:15.713131 18470 detector.cpp:138] Detected a new leader: (id='180')
I0603 14:20:15.713296 18466 group.cpp:659] Trying to get
'/mesos/info_0000000180' in ZooKeeper
I0603 14:20:15.713991 18466 detector.cpp:452] A new leading master
([email protected]:5050) is detected
I0603 14:20:15.714061 18466 sched.cpp:254] New master detected at
[email protected]:5050
I0603 14:20:15.714205 18466 sched.cpp:264] No credentials provided. Attempting
to register without authentication
I0603 14:20:15.716712 18466 sched.cpp:448] Framework registered with
20150530-111452-947384330-5050-2394-0200
15/06/03 14:20:15 INFO CoarseMesosSchedulerBackend: Registered as framework ID
20150530-111452-947384330-5050-2394-0200
15/06/03 14:20:15 INFO NettyBlockTransferService: Server created on 34315
15/06/03 14:20:15 INFO BlockManagerMaster: Trying to register BlockManager
15/06/03 14:20:15 INFO BlockManagerMasterActor: Registering block manager
10.240.241.48:34315 with 15.5 GB RAM, BlockManagerId(<driver>, 10.240.241.48,
34315)
15/06/03 14:20:15 INFO BlockManagerMaster: Registered BlockManager
15/06/03 14:20:15 INFO EventLoggingListener: Logging events to
file:///mnt/spark/sparkLogs/20150530-111452-947384330-5050-2394-0200
15/06/03 14:20:15 INFO CoarseMesosSchedulerBackend: SchedulerBackend is ready
for scheduling beginning after reached minRegisteredResourcesRatio: 0.0
15/06/03 14:20:16 INFO NettyUtil: Found Netty's native epoll transport in the
classpath, using it
15/06/03 14:20:17 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1'
for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
datacenter name with DCAwareRoundRobinPolicy constructor)
15/06/03 14:20:17 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 added
15/06/03 14:20:17 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042
added
15/06/03 14:20:17 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 added
15/06/03 14:20:17 INFO CassandraConnector: Connected to Cassandra cluster:
clusterName
15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 0.0 (TID 0). 1935
bytes result sent to driver
15/06/03 14:20:18 INFO CassandraConnector: Disconnected from Cassandra cluster:
clusterName
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 1
15/06/03 14:20:18 INFO Executor: Running task 0.0 in stage 1.0 (TID 1)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 2
15/06/03 14:20:18 INFO Executor: Running task 1.0 in stage 1.0 (TID 2)
15/06/03 14:20:18 INFO HttpBroadcast: Started reading broadcast variable 2
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(2288) called with curMem=0,
maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block broadcast_2 stored as values in
memory (estimated size 2.2 KB, free 15.5 GB)
15/06/03 14:20:18 INFO HttpBroadcast: Reading broadcast variable 2 took
0.005535572 s
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_2 locally
15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 1.0 (TID 1). 908
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 1.0 in stage 1.0 (TID 2). 908
bytes result sent to driver
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 3
15/06/03 14:20:18 INFO Executor: Running task 0.0 in stage 2.0 (TID 3)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 4
15/06/03 14:20:18 INFO Executor: Running task 1.0 in stage 2.0 (TID 4)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 5
15/06/03 14:20:18 INFO Executor: Running task 2.0 in stage 2.0 (TID 5)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 6
15/06/03 14:20:18 INFO Executor: Running task 3.0 in stage 2.0 (TID 6)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 7
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 8
15/06/03 14:20:18 INFO Executor: Running task 4.0 in stage 2.0 (TID 7)
15/06/03 14:20:18 INFO Executor: Running task 5.0 in stage 2.0 (TID 8)
15/06/03 14:20:18 INFO HttpBroadcast: Started reading broadcast variable 3
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(2688) called with
curMem=2288, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block broadcast_3 stored as values in
memory (estimated size 2.6 KB, free 15.5 GB)
15/06/03 14:20:18 INFO HttpBroadcast: Reading broadcast variable 3 took
0.008771053 s
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO MapOutputTrackerWorker: Updating epoch to 1 and clearing
cache
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_1 not found, computing it
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_5 not found, computing it
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_2 not found, computing it
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_0 not found, computing it
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_3 not found, computing it
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_4 not found, computing it
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 0, fetching them
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor =
Actor[akka://sparkDriver/user/MapOutputTracker#-495616855]
15/06/03 14:20:18 INFO MapOutputTrackerMasterActor: Asked to send map output
locations for shuffle 0 to sparkDriver
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Size of output statuses for
shuffle 0 is 82 bytes
15/06/03 14:20:18 INFO MapOutputTrackerMaster: Got the output locations
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
6 ms
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
6 ms
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
6 ms
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
6 ms
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
5 ms
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
6 ms
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=4976, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_2 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=4992, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_4 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5008, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_5 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5024, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_0 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_2 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5040, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_3 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5056, maxMem=16669841817
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_4 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_4
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_2
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_1 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_5 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_5
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_0 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_0
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_3 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_3
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_1 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_1
15/06/03 14:20:18 INFO Executor: Finished task 1.0 in stage 2.0 (TID 4). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 0.0 in stage 2.0 (TID 3). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 4.0 in stage 2.0 (TID 7). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 5.0 in stage 2.0 (TID 8). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 3.0 in stage 2.0 (TID 6). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO Executor: Finished task 2.0 in stage 2.0 (TID 5). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 9
15/06/03 14:20:18 INFO Executor: Running task 6.0 in stage 2.0 (TID 9)
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_6 not found, computing it
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5072, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_6 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_6 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_6
15/06/03 14:20:18 INFO Executor: Finished task 6.0 in stage 2.0 (TID 9). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 10
15/06/03 14:20:18 INFO Executor: Running task 7.0 in stage 2.0 (TID 10)
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_7 not found, computing it
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5088, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_7 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_7 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_7
15/06/03 14:20:18 INFO Executor: Finished task 7.0 in stage 2.0 (TID 10). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 11
15/06/03 14:20:18 INFO Executor: Running task 8.0 in stage 2.0 (TID 11)
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_8 not found, computing it
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5104, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_8 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_8 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_8
15/06/03 14:20:18 INFO Executor: Finished task 8.0 in stage 2.0 (TID 11). 1497
bytes result sent to driver
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 12
15/06/03 14:20:18 INFO Executor: Running task 9.0 in stage 2.0 (TID 12)
15/06/03 14:20:18 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:18 INFO CacheManager: Partition rdd_7_9 not found, computing it
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:18 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:18 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5120, maxMem=16669841817
15/06/03 14:20:18 INFO MemoryStore: Block rdd_7_9 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:18 INFO BlockManagerInfo: Added rdd_7_9 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:18 INFO CoarseGrainedExecutorBackend: Got assigned task 13
15/06/03 14:20:18 INFO BlockManagerMaster: Updated info of block rdd_7_9
15/06/03 14:20:18 INFO Executor: Running task 10.0 in stage 2.0 (TID 13)
15/06/03 14:20:19 INFO Executor: Finished task 9.0 in stage 2.0 (TID 12). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_10 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5136, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_10 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 14
15/06/03 14:20:19 INFO Executor: Running task 11.0 in stage 2.0 (TID 14)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_10 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_10
15/06/03 14:20:19 INFO Executor: Finished task 10.0 in stage 2.0 (TID 13). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_11 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5152, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_11 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_11 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_11
15/06/03 14:20:19 INFO Executor: Finished task 11.0 in stage 2.0 (TID 14). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 15
15/06/03 14:20:19 INFO Executor: Running task 12.0 in stage 2.0 (TID 15)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_12 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5168, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_12 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_12 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_12
15/06/03 14:20:19 INFO Executor: Finished task 12.0 in stage 2.0 (TID 15). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 16
15/06/03 14:20:19 INFO Executor: Running task 13.0 in stage 2.0 (TID 16)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_13 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5184, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_13 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_13 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_13
15/06/03 14:20:19 INFO Executor: Finished task 13.0 in stage 2.0 (TID 16). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 17
15/06/03 14:20:19 INFO Executor: Running task 14.0 in stage 2.0 (TID 17)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_14 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5200, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_14 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_14 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_14
15/06/03 14:20:19 INFO Executor: Finished task 14.0 in stage 2.0 (TID 17). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 18
15/06/03 14:20:19 INFO Executor: Running task 15.0 in stage 2.0 (TID 18)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_15 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5216, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_15 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_15 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_15
15/06/03 14:20:19 INFO Executor: Finished task 15.0 in stage 2.0 (TID 18). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 19
15/06/03 14:20:19 INFO Executor: Running task 16.0 in stage 2.0 (TID 19)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_16 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5232, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_16 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_16 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_16
15/06/03 14:20:19 INFO Executor: Finished task 16.0 in stage 2.0 (TID 19). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 20
15/06/03 14:20:19 INFO Executor: Running task 17.0 in stage 2.0 (TID 20)
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 21
15/06/03 14:20:19 INFO Executor: Running task 18.0 in stage 2.0 (TID 21)
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_17 not found, computing it
15/06/03 14:20:19 INFO CoarseGrainedExecutorBackend: Got assigned task 22
15/06/03 14:20:19 INFO Executor: Running task 19.0 in stage 2.0 (TID 22)
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_18 not found, computing it
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5248, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_17 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO BlockManager: Found block broadcast_3 locally
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5264, maxMem=16669841817
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_17 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_17
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_18 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO Executor: Finished task 17.0 in stage 2.0 (TID 20). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_18 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_18
15/06/03 14:20:19 INFO Executor: Finished task 18.0 in stage 2.0 (TID 21). 1497
bytes result sent to driver
15/06/03 14:20:19 INFO CacheManager: Partition rdd_7_19 not found, computing it
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:20:19 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:20:19 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=5280, maxMem=16669841817
15/06/03 14:20:19 INFO MemoryStore: Block rdd_7_19 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerInfo: Added rdd_7_19 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:20:19 INFO BlockManagerMaster: Updated info of block rdd_7_19
15/06/03 14:20:19 INFO Executor: Finished task 19.0 in stage 2.0 (TID 22). 1497
bytes result sent to driver
15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 23
15/06/03 14:25:20 INFO Executor: Running task 0.0 in stage 3.0 (TID 23)
15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 24
15/06/03 14:25:20 INFO Executor: Running task 1.0 in stage 3.0 (TID 24)
15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 25
15/06/03 14:25:20 INFO Executor: Running task 2.0 in stage 3.0 (TID 25)
15/06/03 14:25:20 INFO CoarseGrainedExecutorBackend: Got assigned task 26
15/06/03 14:25:20 INFO Executor: Running task 3.0 in stage 3.0 (TID 26)
15/06/03 14:25:20 INFO HttpBroadcast: Started reading broadcast variable 4
15/06/03 14:25:20 INFO MemoryStore: ensureFreeSpace(11888) called with
curMem=5296, maxMem=16669841817
15/06/03 14:25:20 INFO MemoryStore: Block broadcast_4 stored as values in
memory (estimated size 11.6 KB, free 15.5 GB)
15/06/03 14:25:20 INFO HttpBroadcast: Reading broadcast variable 4 took
0.007110946 s
15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally
15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally
15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_2 not found, computing it
15/06/03 14:25:20 INFO BlockManager: Found block broadcast_4 locally
15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_1 not found, computing it
15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_3 not found, computing it
15/06/03 14:25:20 INFO CacheManager: Partition rdd_1_0 not found, computing it
15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1'
for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
datacenter name with DCAwareRoundRobinPolicy constructor)
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042
added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 added
15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1'
for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
datacenter name with DCAwareRoundRobinPolicy constructor)
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042
added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 added
15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1'
for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
datacenter name with DCAwareRoundRobinPolicy constructor)
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042
added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 added
15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO DCAwareRoundRobinPolicy: Using data-center name 'DC1'
for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct
datacenter name with DCAwareRoundRobinPolicy constructor)
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse2/10.240.89.240:9042 added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse0/10.240.127.151:9042
added
15/06/03 14:25:21 INFO Cluster: New Cassandra host dse1/10.240.169.24:9042 added
15/06/03 14:25:21 INFO CassandraConnector: Connected to Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra cluster:
clusterName
15/06/03 14:25:21 INFO CassandraConnector: Disconnected from Cassandra cluster:
clusterName
15/06/03 14:25:22 INFO MemoryStore: ensureFreeSpace(803320) called with
curMem=17184, maxMem=16669841817
15/06/03 14:25:22 INFO MemoryStore: Block rdd_1_2 stored as values in memory
(estimated size 784.5 KB, free 15.5 GB)
15/06/03 14:25:22 INFO BlockManagerInfo: Added rdd_1_2 in memory on
10.240.241.48:34315 (size: 784.5 KB, free: 15.5 GB)
15/06/03 14:25:22 INFO BlockManagerMaster: Updated info of block rdd_1_2
15/06/03 14:25:22 INFO Executor: Finished task 2.0 in stage 3.0 (TID 25). 2599
bytes result sent to driver
15/06/03 14:25:28 INFO MemoryStore: ensureFreeSpace(17551856) called with
curMem=820504, maxMem=16669841817
15/06/03 14:25:28 INFO MemoryStore: Block rdd_1_3 stored as values in memory
(estimated size 16.7 MB, free 15.5 GB)
15/06/03 14:25:28 INFO BlockManagerInfo: Added rdd_1_3 in memory on
10.240.241.48:34315 (size: 16.7 MB, free: 15.5 GB)
15/06/03 14:25:28 INFO BlockManagerMaster: Updated info of block rdd_1_3
15/06/03 14:25:29 INFO MemoryStore: ensureFreeSpace(17846696) called with
curMem=18372360, maxMem=16669841817
15/06/03 14:25:29 INFO MemoryStore: Block rdd_1_0 stored as values in memory
(estimated size 17.0 MB, free 15.5 GB)
15/06/03 14:25:29 INFO BlockManagerInfo: Added rdd_1_0 in memory on
10.240.241.48:34315 (size: 17.0 MB, free: 15.5 GB)
15/06/03 14:25:29 INFO BlockManagerMaster: Updated info of block rdd_1_0
15/06/03 14:25:30 INFO MemoryStore: ensureFreeSpace(20620640) called with
curMem=36219056, maxMem=16669841817
15/06/03 14:25:30 INFO MemoryStore: Block rdd_1_1 stored as values in memory
(estimated size 19.7 MB, free 15.5 GB)
15/06/03 14:25:30 INFO BlockManagerInfo: Added rdd_1_1 in memory on
10.240.241.48:34315 (size: 19.7 MB, free: 15.5 GB)
15/06/03 14:25:30 INFO BlockManagerMaster: Updated info of block rdd_1_1
15/06/03 14:25:30 INFO Executor: Finished task 3.0 in stage 3.0 (TID 26). 2599
bytes result sent to driver
15/06/03 14:25:30 INFO Executor: Finished task 0.0 in stage 3.0 (TID 23). 2599
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 3.0 (TID 24). 2599
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 27
15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 4.0 (TID 27)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 28
15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 4.0 (TID 28)
15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 5
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 29
15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 4.0 (TID 29)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 30
15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 4.0 (TID 30)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 31
15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 4.0 (TID 31)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 32
15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 4.0 (TID 32)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(2632) called with
curMem=56839696, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block broadcast_5 stored as values in
memory (estimated size 2.6 KB, free 15.5 GB)
15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 5 took
0.007544338 s
15/06/03 14:25:31 INFO MapOutputTrackerWorker: Updating epoch to 2 and clearing
cache
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_5 not found, computing it
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_1 not found, computing it
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 1, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor =
Actor[akka://sparkDriver/user/MapOutputTracker#-495616855]
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 1, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMasterActor: Asked to send map output
locations for shuffle 1 to sparkDriver
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_3 not found, computing it
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_2 not found, computing it
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_0 not found, computing it
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Size of output statuses for
shuffle 1 is 82 bytes
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 1, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 1, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 1, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Got the output locations
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842328, maxMem=16669841817
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_5 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_4 not found, computing it
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842344, maxMem=16669841817
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_5 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_0 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_5
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842360, maxMem=16669841817
15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 4.0 (TID 32). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_0 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_1 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_0
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842376, maxMem=16669841817
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_1 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 4.0 (TID 27). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_1
15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 4.0 (TID 28). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_2 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842392, maxMem=16669841817
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_2 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_3 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_2
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842408, maxMem=16669841817
15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 4.0 (TID 29). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_4 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_3 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_3
15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 4.0 (TID 30). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_4 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_4
15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 4.0 (TID 31). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 33
15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 4.0 (TID 33)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_6 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842424, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_6 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_6 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_6
15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 4.0 (TID 33). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 34
15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 4.0 (TID 34)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_7 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842440, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_7 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_7 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_7
15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 4.0 (TID 34). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 35
15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 4.0 (TID 35)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_8 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842456, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_8 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_8 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_8
15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 4.0 (TID 35). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 36
15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 4.0 (TID 36)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 37
15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 4.0 (TID 37)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 38
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 4.0 (TID 38)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_9 not found, computing it
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_10 not found, computing it
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_11 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842472, maxMem=16669841817
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_10 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842488, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_11 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_10 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842504, maxMem=16669841817
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_10
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_9 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_11 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 4.0 (TID 37). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_11
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_9 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 4.0 (TID 38). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_9
15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 4.0 (TID 36). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 39
15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 4.0 (TID 39)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_12 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842520, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_12 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_12 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_12
15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 4.0 (TID 39). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 40
15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 4.0 (TID 40)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_13 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842536, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_13 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_13 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_13
15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 4.0 (TID 40). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 41
15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 4.0 (TID 41)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_14 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842552, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_14 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_14 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_14
15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 4.0 (TID 41). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 42
15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 4.0 (TID 42)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_15 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842568, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_15 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_15 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_15
15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 4.0 (TID 42). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 43
15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 4.0 (TID 43)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_16 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842584, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_16 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_16 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_16
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 44
15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 4.0 (TID 43). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 4.0 (TID 44)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_17 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842600, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_17 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_17 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_17
15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 4.0 (TID 44). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 45
15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 4.0 (TID 45)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_18 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842616, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_18 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_18 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_18
15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 4.0 (TID 45). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 46
15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 4.0 (TID 46)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_5 locally
15/06/03 14:25:31 INFO CacheManager: Partition rdd_11_19 not found, computing it
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(16) called with
curMem=56842632, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block rdd_11_19 stored as values in memory
(estimated size 16.0 B, free 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerInfo: Added rdd_11_19 in memory on
10.240.241.48:34315 (size: 16.0 B, free: 15.5 GB)
15/06/03 14:25:31 INFO BlockManagerMaster: Updated info of block rdd_11_19
15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 4.0 (TID 46). 1497
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 47
15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 6.0 (TID 47)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 48
15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 6.0 (TID 48)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 49
15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 6.0 (TID 49)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 50
15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 6.0 (TID 50)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 51
15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 6.0 (TID 51)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 52
15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 6.0 (TID 52)
15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 6
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(2760) called with
curMem=56842648, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block broadcast_6 stored as values in
memory (estimated size 2.7 KB, free 15.5 GB)
15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 6 took
0.004133133 s
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_1 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_4 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_2 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_3 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_0 locally
15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 6.0 (TID 48). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 6.0 (TID 50). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 6.0 (TID 47). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 6.0 (TID 52). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 6.0 (TID 51). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 6.0 (TID 49). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 53
15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 6.0 (TID 53)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 54
15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 6.0 (TID 54)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 55
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 6.0 (TID 55)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 56
15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 6.0 (TID 56)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 6.0 (TID 53). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 6.0 (TID 54). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_8 locally
15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 6.0 (TID 55). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_9 locally
15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 6.0 (TID 56). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 57
15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 6.0 (TID 57)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_10 locally
15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 6.0 (TID 57). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 58
15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 6.0 (TID 58)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_11 locally
15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 6.0 (TID 58). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 59
15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 6.0 (TID 59)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_12 locally
15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 6.0 (TID 59). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 60
15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 6.0 (TID 60)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_13 locally
15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 6.0 (TID 60). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 61
15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 6.0 (TID 61)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_14 locally
15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 6.0 (TID 61). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 62
15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 6.0 (TID 62)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 63
15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 6.0 (TID 63)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_15 locally
15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 6.0 (TID 62). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_16 locally
15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 6.0 (TID 63). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 64
15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 6.0 (TID 64)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_17 locally
15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 6.0 (TID 64). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 65
15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 6.0 (TID 65)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_18 locally
15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 6.0 (TID 65). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 66
15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 6.0 (TID 66)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_6 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_19 locally
15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 6.0 (TID 66). 1834
bytes result sent to driver
15/06/03 14:25:31 INFO CassandraConnector: Disconnected from Cassandra cluster:
clusterName
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 67
15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 8.0 (TID 67)
15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 7
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 68
15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 8.0 (TID 68)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 69
15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 8.0 (TID 69)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 70
15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 8.0 (TID 70)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 71
15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 8.0 (TID 71)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 72
15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 8.0 (TID 72)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(3536) called with
curMem=56845408, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block broadcast_7 stored as values in
memory (estimated size 3.5 KB, free 15.5 GB)
15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 7 took
0.009490517 s
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_2 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_0 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_1 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_3 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_5 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_4 locally
15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 8.0 (TID 70). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 8.0 (TID 71). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 8.0 (TID 67). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 8.0 (TID 69). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 8.0 (TID 72). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 8.0 (TID 68). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 73
15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 8.0 (TID 73)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_6 locally
15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 8.0 (TID 73). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 74
15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 8.0 (TID 74)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_7 locally
15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 8.0 (TID 74). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 75
15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 8.0 (TID 75)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_8 locally
15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 8.0 (TID 75). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 76
15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 8.0 (TID 76)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_9 locally
15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 8.0 (TID 76). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 77
15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 8.0 (TID 77)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_10 locally
15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 8.0 (TID 77). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 78
15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 8.0 (TID 78)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_11 locally
15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 8.0 (TID 78). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 79
15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 8.0 (TID 79)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_12 locally
15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 8.0 (TID 79). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 80
15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 8.0 (TID 80)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 81
15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 8.0 (TID 81)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_14 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_13 locally
15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 8.0 (TID 81). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 8.0 (TID 80). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 82
15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 8.0 (TID 82)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_15 locally
15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 8.0 (TID 82). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 83
15/06/03 14:25:31 INFO Executor: Running task 16.0 in stage 8.0 (TID 83)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_16 locally
15/06/03 14:25:31 INFO Executor: Finished task 16.0 in stage 8.0 (TID 83). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 84
15/06/03 14:25:31 INFO Executor: Running task 17.0 in stage 8.0 (TID 84)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_17 locally
15/06/03 14:25:31 INFO Executor: Finished task 17.0 in stage 8.0 (TID 84). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 85
15/06/03 14:25:31 INFO Executor: Running task 18.0 in stage 8.0 (TID 85)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_18 locally
15/06/03 14:25:31 INFO Executor: Finished task 18.0 in stage 8.0 (TID 85). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 86
15/06/03 14:25:31 INFO Executor: Running task 19.0 in stage 8.0 (TID 86)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_7 locally
15/06/03 14:25:31 INFO BlockManager: Found block rdd_11_19 locally
15/06/03 14:25:31 INFO Executor: Finished task 19.0 in stage 8.0 (TID 86). 2030
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 87
15/06/03 14:25:31 INFO Executor: Running task 0.0 in stage 9.0 (TID 87)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 88
15/06/03 14:25:31 INFO Executor: Running task 1.0 in stage 9.0 (TID 88)
15/06/03 14:25:31 INFO HttpBroadcast: Started reading broadcast variable 8
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 89
15/06/03 14:25:31 INFO Executor: Running task 2.0 in stage 9.0 (TID 89)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 90
15/06/03 14:25:31 INFO Executor: Running task 3.0 in stage 9.0 (TID 90)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 91
15/06/03 14:25:31 INFO Executor: Running task 4.0 in stage 9.0 (TID 91)
15/06/03 14:25:31 INFO MemoryStore: ensureFreeSpace(3736) called with
curMem=56848944, maxMem=16669841817
15/06/03 14:25:31 INFO MemoryStore: Block broadcast_8 stored as values in
memory (estimated size 3.6 KB, free 15.5 GB)
15/06/03 14:25:31 INFO HttpBroadcast: Reading broadcast variable 8 took
0.005282429 s
15/06/03 14:25:31 INFO MapOutputTrackerWorker: Updating epoch to 3 and clearing
cache
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 2, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Doing the fetch; tracker actor =
Actor[akka://sparkDriver/user/MapOutputTracker#-495616855]
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 2, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 2, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMasterActor: Asked to send map output
locations for shuffle 2 to sparkDriver
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 2, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Size of output statuses for
shuffle 2 is 82 bytes
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Don't have map outputs for
shuffle 2, fetching them
15/06/03 14:25:31 INFO MapOutputTrackerMaster: Got the output locations
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 92
15/06/03 14:25:31 INFO Executor: Running task 5.0 in stage 9.0 (TID 92)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 2.0 in stage 9.0 (TID 89). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 5.0 in stage 9.0 (TID 92). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 0.0 in stage 9.0 (TID 87). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 4.0 in stage 9.0 (TID 91). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 3.0 in stage 9.0 (TID 90). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 1.0 in stage 9.0 (TID 88). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 93
15/06/03 14:25:31 INFO Executor: Running task 6.0 in stage 9.0 (TID 93)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 6.0 in stage 9.0 (TID 93). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 94
15/06/03 14:25:31 INFO Executor: Running task 7.0 in stage 9.0 (TID 94)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 7.0 in stage 9.0 (TID 94). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 95
15/06/03 14:25:31 INFO Executor: Running task 8.0 in stage 9.0 (TID 95)
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 96
15/06/03 14:25:31 INFO Executor: Running task 9.0 in stage 9.0 (TID 96)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 8.0 in stage 9.0 (TID 95). 928
bytes result sent to driver
15/06/03 14:25:31 INFO Executor: Finished task 9.0 in stage 9.0 (TID 96). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 97
15/06/03 14:25:31 INFO Executor: Running task 10.0 in stage 9.0 (TID 97)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 10.0 in stage 9.0 (TID 97). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 98
15/06/03 14:25:31 INFO Executor: Running task 11.0 in stage 9.0 (TID 98)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 11.0 in stage 9.0 (TID 98). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 99
15/06/03 14:25:31 INFO Executor: Running task 12.0 in stage 9.0 (TID 99)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 12.0 in stage 9.0 (TID 99). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 100
15/06/03 14:25:31 INFO Executor: Running task 13.0 in stage 9.0 (TID 100)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 13.0 in stage 9.0 (TID 100). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 101
15/06/03 14:25:31 INFO Executor: Running task 14.0 in stage 9.0 (TID 101)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 14.0 in stage 9.0 (TID 101). 928
bytes result sent to driver
15/06/03 14:25:31 INFO CoarseGrainedExecutorBackend: Got assigned task 102
15/06/03 14:25:31 INFO Executor: Running task 15.0 in stage 9.0 (TID 102)
15/06/03 14:25:31 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:31 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:31 INFO Executor: Finished task 15.0 in stage 9.0 (TID 102). 928
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 103
15/06/03 14:25:32 INFO Executor: Running task 16.0 in stage 9.0 (TID 103)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 16.0 in stage 9.0 (TID 103). 928
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 104
15/06/03 14:25:32 INFO Executor: Running task 17.0 in stage 9.0 (TID 104)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 105
15/06/03 14:25:32 INFO Executor: Running task 18.0 in stage 9.0 (TID 105)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 17.0 in stage 9.0 (TID 104). 928
bytes result sent to driver
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 18.0 in stage 9.0 (TID 105). 928
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 106
15/06/03 14:25:32 INFO Executor: Running task 19.0 in stage 9.0 (TID 106)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_8 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 19.0 in stage 9.0 (TID 106). 928
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 107
15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 12.0 (TID 107)
15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 9
15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with
curMem=56852680, maxMem=16669841817
15/06/03 14:25:32 INFO MemoryStore: Block broadcast_9 stored as values in
memory (estimated size 3.9 KB, free 15.5 GB)
15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 9 took
0.010923501 s
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 12.0 (TID 107). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 108
15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 15.0 (TID 108)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 109
15/06/03 14:25:32 INFO Executor: Running task 1.0 in stage 15.0 (TID 109)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 110
15/06/03 14:25:32 INFO Executor: Running task 2.0 in stage 15.0 (TID 110)
15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 10
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 111
15/06/03 14:25:32 INFO Executor: Running task 3.0 in stage 15.0 (TID 111)
15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with
curMem=56856664, maxMem=16669841817
15/06/03 14:25:32 INFO MemoryStore: Block broadcast_10 stored as values in
memory (estimated size 3.9 KB, free 15.5 GB)
15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 10 took
0.003318126 s
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_10 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 15.0 (TID 108). 908
bytes result sent to driver
15/06/03 14:25:32 INFO Executor: Finished task 3.0 in stage 15.0 (TID 111). 908
bytes result sent to driver
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 2.0 in stage 15.0 (TID 110). 908
bytes result sent to driver
15/06/03 14:25:32 INFO Executor: Finished task 1.0 in stage 15.0 (TID 109). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 112
15/06/03 14:25:32 INFO Executor: Running task 0.0 in stage 18.0 (TID 112)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 113
15/06/03 14:25:32 INFO Executor: Running task 1.0 in stage 18.0 (TID 113)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 114
15/06/03 14:25:32 INFO Executor: Running task 2.0 in stage 18.0 (TID 114)
15/06/03 14:25:32 INFO HttpBroadcast: Started reading broadcast variable 11
15/06/03 14:25:32 INFO MemoryStore: ensureFreeSpace(3984) called with
curMem=56860648, maxMem=16669841817
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 115
15/06/03 14:25:32 INFO Executor: Running task 3.0 in stage 18.0 (TID 115)
15/06/03 14:25:32 INFO MemoryStore: Block broadcast_11 stored as values in
memory (estimated size 3.9 KB, free 15.5 GB)
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 116
15/06/03 14:25:32 INFO Executor: Running task 4.0 in stage 18.0 (TID 116)
15/06/03 14:25:32 INFO HttpBroadcast: Reading broadcast variable 11 took
0.005466598 s
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
1 ms
15/06/03 14:25:32 INFO Executor: Finished task 0.0 in stage 18.0 (TID 112). 908
bytes result sent to driver
15/06/03 14:25:32 INFO Executor: Finished task 1.0 in stage 18.0 (TID 113). 908
bytes result sent to driver
15/06/03 14:25:32 INFO Executor: Finished task 3.0 in stage 18.0 (TID 115). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 117
15/06/03 14:25:32 INFO Executor: Finished task 2.0 in stage 18.0 (TID 114). 908
bytes result sent to driver
15/06/03 14:25:32 INFO Executor: Running task 5.0 in stage 18.0 (TID 117)
15/06/03 14:25:32 INFO Executor: Finished task 4.0 in stage 18.0 (TID 116). 908
bytes result sent to driver
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 5.0 in stage 18.0 (TID 117). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 118
15/06/03 14:25:32 INFO Executor: Running task 6.0 in stage 18.0 (TID 118)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 6.0 in stage 18.0 (TID 118). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 119
15/06/03 14:25:32 INFO Executor: Running task 7.0 in stage 18.0 (TID 119)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 7.0 in stage 18.0 (TID 119). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 120
15/06/03 14:25:32 INFO Executor: Running task 8.0 in stage 18.0 (TID 120)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 121
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO Executor: Running task 9.0 in stage 18.0 (TID 121)
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 8.0 in stage 18.0 (TID 120). 908
bytes result sent to driver
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 9.0 in stage 18.0 (TID 121). 908
bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 122
15/06/03 14:25:32 INFO Executor: Running task 10.0 in stage 18.0 (TID 122)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 10.0 in stage 18.0 (TID 122).
908 bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 123
15/06/03 14:25:32 INFO Executor: Running task 11.0 in stage 18.0 (TID 123)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 11.0 in stage 18.0 (TID 123).
908 bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 124
15/06/03 14:25:32 INFO Executor: Running task 12.0 in stage 18.0 (TID 124)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 12.0 in stage 18.0 (TID 124).
908 bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 125
15/06/03 14:25:32 INFO Executor: Running task 13.0 in stage 18.0 (TID 125)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 13.0 in stage 18.0 (TID 125).
908 bytes result sent to driver
15/06/03 14:25:32 INFO CoarseGrainedExecutorBackend: Got assigned task 126
15/06/03 14:25:32 INFO Executor: Running task 14.0 in stage 18.0 (TID 126)
15/06/03 14:25:32 INFO BlockManager: Found block broadcast_11 locally
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Getting 0 non-empty blocks
out of 0 blocks
15/06/03 14:25:32 INFO ShuffleBlockFetcherIterator: Started 0 remote fetches in
0 ms
15/06/03 14:25:32 INFO Executor: Finished task 14.0 in stage 18.0 (TID 126).
908 bytes result sent to driver
15/06/03 14:25:34 ERROR CoarseGrainedExecutorBackend: Driver Disassociated
[akka.tcp://[email protected]:40338] ->
[akka.tcp://[email protected]:40448]
disassociated! Shutting down.
I0603 14:25:34.036478 18289 exec.cpp:379] Executor asked to shutdown
15/06/03 14:25:34 WARN ReliableDeliverySupervisor: Association with remote
system
[akka.tcp://[email protected]:40448] has
failed, address is now gated for [5000] ms. Reason is: [Disassociated].
15/06/03 14:25:34 ERROR CoarseGrainedExecutorBackend: RECEIVED SIGNAL 15:
SIGTERM
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
