You can list the files in tmpfs in reverse chronological order and remove the oldest until you have enough space.
Cheers On Sun, Jul 26, 2015 at 12:43 AM, Pa Rö <paul.roewer1...@googlemail.com> wrote: > i has seen that the "tempfs" is full, how i can clear this? > > 2015-07-23 13:41 GMT+02:00 Pa Rö <paul.roewer1...@googlemail.com>: > >> hello spark community, >> >> i have build an application with geomesa, accumulo and spark. >> if it run on spark local mode, it is working, but on spark >> cluster not. in short it says: No space left on device. Asked to remove >> non-existent executor XY. >> I´m confused, because there were many GB´s of free space. do i need to >> change my configuration or what else can i do? thanks in advance. >> >> here is the complete exception: >> >> og4j:WARN No appenders could be found for logger >> (org.apache.accumulo.fate.zookeeper.ZooSession). >> log4j:WARN Please initialize the log4j system properly. >> log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for >> more info. >> Using Spark's default log4j profile: >> org/apache/spark/log4j-defaults.properties >> 15/07/23 13:26:39 INFO SparkContext: Running Spark version 1.3.0 >> 15/07/23 13:26:39 WARN NativeCodeLoader: Unable to load native-hadoop >> library for your platform... using builtin-java classes where applicable >> 15/07/23 13:26:39 INFO SecurityManager: Changing view acls to: marcel >> 15/07/23 13:26:39 INFO SecurityManager: Changing modify acls to: marcel >> 15/07/23 13:26:39 INFO SecurityManager: SecurityManager: authentication >> disabled; ui acls disabled; users with view permissions: Set(marcel); users >> with modify permissions: Set(marcel) >> 15/07/23 13:26:39 INFO Slf4jLogger: Slf4jLogger started >> 15/07/23 13:26:40 INFO Remoting: Starting remoting >> 15/07/23 13:26:40 INFO Remoting: Remoting started; listening on addresses >> :[akka.tcp://sparkDriver@node1-scads02:52478] >> 15/07/23 13:26:40 INFO Utils: Successfully started service 'sparkDriver' >> on port 52478. >> 15/07/23 13:26:40 INFO SparkEnv: Registering MapOutputTracker >> 15/07/23 13:26:40 INFO SparkEnv: Registering BlockManagerMaster >> 15/07/23 13:26:40 INFO DiskBlockManager: Created local directory at >> /tmp/spark-ca9319d4-68a2-4add-a21a-48b13ae9cf81/blockmgr-cbf8af23-e113-4732-8c2c-7413ad237b3b >> 15/07/23 13:26:40 INFO MemoryStore: MemoryStore started with capacity >> 1916.2 MB >> 15/07/23 13:26:40 INFO HttpFileServer: HTTP File server directory is >> /tmp/spark-9d4a04d5-3535-49e0-a859-d278a0cc7bf8/httpd-1882aafc-45fe-4490-803d-c04fc67510a2 >> 15/07/23 13:26:40 INFO HttpServer: Starting HTTP Server >> 15/07/23 13:26:40 INFO Server: jetty-8.y.z-SNAPSHOT >> 15/07/23 13:26:40 INFO AbstractConnector: Started >> SocketConnector@0.0.0.0:56499 >> 15/07/23 13:26:40 INFO Utils: Successfully started service 'HTTP file >> server' on port 56499. >> 15/07/23 13:26:40 INFO SparkEnv: Registering OutputCommitCoordinator >> 15/07/23 13:26:40 INFO Server: jetty-8.y.z-SNAPSHOT >> 15/07/23 13:26:40 INFO AbstractConnector: Started >> SelectChannelConnector@0.0.0.0:4040 >> 15/07/23 13:26:40 INFO Utils: Successfully started service 'SparkUI' on >> port 4040. >> 15/07/23 13:26:40 INFO SparkUI: Started SparkUI at >> http://node1-scads02:4040 >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Connecting to master >> akka.tcp://sparkMaster@node1-scads02:7077/user/Master... >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Connected to Spark >> cluster with app ID app-20150723132640-0000 >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor added: >> app-20150723132640-0000/0 on worker-20150723132524-node3-scads06-7078 >> (node3-scads06:7078) with 8 cores >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Granted executor ID >> app-20150723132640-0000/0 on hostPort node3-scads06:7078 with 8 cores, >> 512.0 MB RAM >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor added: >> app-20150723132640-0000/1 on worker-20150723132513-node2-scads05-7078 >> (node2-scads05:7078) with 8 cores >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Granted executor ID >> app-20150723132640-0000/1 on hostPort node2-scads05:7078 with 8 cores, >> 512.0 MB RAM >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/0 is now RUNNING >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/1 is now RUNNING >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/0 is now LOADING >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/1 is now LOADING >> 15/07/23 13:26:40 INFO NettyBlockTransferService: Server created on 45786 >> 15/07/23 13:26:40 INFO BlockManagerMaster: Trying to register BlockManager >> 15/07/23 13:26:40 INFO BlockManagerMasterActor: Registering block manager >> node1-scads02:45786 with 1916.2 MB RAM, BlockManagerId(<driver>, >> node1-scads02, 45786) >> 15/07/23 13:26:40 INFO BlockManagerMaster: Registered BlockManager >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/0 is now FAILED (java.io.IOException: No space left >> on device) >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Executor >> app-20150723132640-0000/0 removed: java.io.IOException: No space left on >> device >> 15/07/23 13:26:40 ERROR SparkDeploySchedulerBackend: Asked to remove >> non-existent executor 0 >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor added: >> app-20150723132640-0000/2 on worker-20150723132524-node3-scads06-7078 >> (node3-scads06:7078) with 8 cores >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Granted executor ID >> app-20150723132640-0000/2 on hostPort node3-scads06:7078 with 8 cores, >> 512.0 MB RAM >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/1 is now FAILED (java.io.IOException: No space left >> on device) >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Executor >> app-20150723132640-0000/1 removed: java.io.IOException: No space left on >> device >> 15/07/23 13:26:40 ERROR SparkDeploySchedulerBackend: Asked to remove >> non-existent executor 1 >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor added: >> app-20150723132640-0000/3 on worker-20150723132513-node2-scads05-7078 >> (node2-scads05:7078) with 8 cores >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: Granted executor ID >> app-20150723132640-0000/3 on hostPort node2-scads05:7078 with 8 cores, >> 512.0 MB RAM >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/2 is now LOADING >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/3 is now LOADING >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/2 is now RUNNING >> 15/07/23 13:26:40 INFO SparkDeploySchedulerBackend: SchedulerBackend is >> ready for scheduling beginning after reached minRegisteredResourcesRatio: >> 0.0 >> 15/07/23 13:26:40 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/3 is now RUNNING >> 15/07/23 13:26:41 INFO AppClient$ClientActor: Executor updated: >> app-20150723132640-0000/2 is now FAILED (java.io.IOException: No space left >> on device) >> 15/07/23 13:26:41 INFO SparkDeploySchedulerBackend: Executor >> app-20150723132640-0000/2 removed: java.io.IOException: No space left on >> device >> 15/07/23 13:26:41 ERROR SparkDeploySchedulerBackend: Asked to remove >> non-existent executor 2... >> > >
