Hi,
First things about safeToDelete.tmp: It looks like that ‘safeToDelete.tmp’ is a file which is owned/controlled by tomcat, but not kylin. It is used to indicate that folder tomcat/temp could be deleted safely. But that should not be true when other program(in this case, that is kylin) use that folder, so, for that reason, please ignore safeToDelete.tmp and delete kylin-related files under tomcat/temp carefully. I am trying to find manual about that file in tomcat document, but I failed. Related links https://stackoverflow.com/questions/7112591/what-is-the-tomcat-temp-directory-in-tomcat-7 https://t246osslab.wordpress.com/2017/04/26/tomcatのsafetodelete-tmpの謎を追う/ Second things about olap_model_XXX.json: These files are created by org.apache.kylin.query.schema.OLAPSchemaFactory. And it is read in sql query analysis procedure by apache calcite(code in method model of class CalciteConnectionConfigImpl and QueryConnection). So it must not deleted. These files are very small, and count of such files is equals to the count of kylin’s model. If you delete them but cannot recover them, please restart your kylin process. It will be created automatically after restart. Third things about folder kylin_job_metaXXX: These files are created by org.apache.kylin.engine.mr.common.AbstractHadoopJob, method name is dumpKylinPropsAndMetadata. Such folder will be created when some step of a kylin job start. When each step finished, these files should be deleted automatically( in finally clause of method run). If you find them in tomcat/temp and it is not fresh (for example, it last-modified timestamp is two days ago), it could be deleted safely. The count of such files will grow rapidly if you have a lot of submited job. You can check source code in github repo if you are interested in them. If your find any mistake, please let me know, thank you. ---------------- Best wishes, Xiaoxiang Yu 发件人: Iñigo Martínez <[email protected]> 答复: "[email protected]" <[email protected]> 日期: 2019年1月18日 星期五 22:35 收件人: "[email protected]" <[email protected]> 主题: Safe deletion of $KYLIN_HOME/tomcat/temp stuff Good morning. We have detected that our Kylin installation has plenty temporary files located at $KYLIN_HOME/tomcat/temp. Now, around 90GB after two months running. Inside this folder, there is a "safeToDelete.tmp" file, so we assumed we could flush the contents of this folder safely. However, this was not true. As soon as we delete contents older than 2 days, a lot of errors appeared in kylin.log complaining about missing files, in particular those of type: Message: java.io.FileNotFoundException: /opt/kylin/current/bin/../tomcat/temp/olap_model_4116748333625199860.json (No such file or directory) We immediately restored all old stuff and service became operational again. Tomcat temp folder is fully of files like these: [Screenshot from 2019-01-18 13-48-14.png] So, question is, how do we deal with this folder cleanup? Can we delete all kylin_jobmetadataXXXXXX folders but maintain olap_modem_XXXXX.json files? Why a safeToDelete.tmp exists here? It only contributes to create confusion about safe cleanup of this folder contents. Thank you!
