Could you find the executor logs on the executor where that task was
scheduled? That may provide more information on what caused the error.
Also take a look at where the block in question was stored, and where the
task was scheduled.
You will need to enabled log4j INFO level logs for this debugging
Not quiet sure, but this can be the case. One of your executor is stuck on
GC pause while the other one asks for the data from it and hence the
request timesout ending in that exception. You can try increasing the akk
framesize and ack wait timeout as follows:
.set("spark.core.connection.ack
My streaming app runs fine for a few hours and then starts spewing "Could
not compute split, block input-xx-xxx not found" errors. After this,
jobs start to fail and batches start to pile up.
My question isn't so much about why this error but rather, how do I trace
what leads to this error? I
