Thanks for Xiao Xiang's information, it's quite clear. Best regards,
Shaofeng Shi 史少锋 Apache Kylin PMC Work email: [email protected] Kyligence Inc: https://kyligence.io/ Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html Join Kylin user mail group: [email protected] Join Kylin dev mail group: [email protected] Iñigo Martínez <[email protected]> 于2019年1月21日周一 下午8:44写道： > Thank you, Xiaoxiang. > > Completely clear. We well proceed very carefully with .json deletion and > we will only send to trash kylin_job_metaXXXX created several days ago. > Meanwhile, I will check github in order to find why those folders are not > deleted. > > > El sáb., 19 ene. 2019 a las 12:34, Xiaoxiang Yu (< > [email protected]>) escribió: > >> Hi, >> >> >> >> First things about safeToDelete.tmp: >> >> >> >> It looks like that ‘safeToDelete.tmp’ is a file which is >> owned/controlled by tomcat, but not kylin. It is used to indicate that >> folder tomcat/temp could be deleted safely. But that should not be true >> when other program(in this case, that is kylin) use that folder, so, for >> that reason, please ignore safeToDelete.tmp and delete kylin-related >> files under tomcat/temp carefully. >> >> I am trying to find manual about that file in tomcat document, but I >> failed. Related links >> >> >> https://stackoverflow.com/questions/7112591/what-is-the-tomcat-temp-directory-in-tomcat-7 >> >> https://t246osslab.wordpress.com/2017/04/26/tomcatのsafetodelete-tmpの謎を追う/ >> >> >> >> >> >> >> >> Second things about olap_model_XXX.json: >> >> >> >> These files are created by >> org.apache.kylin.query.schema.OLAPSchemaFactory. And it is read in sql >> query analysis procedure by apache calcite(code in method model of class >> CalciteConnectionConfigImpl and QueryConnection). So it must not >> deleted. These files are very small, and count of such files is equals >> to the count of kylin’s model. >> >> >> >> If you delete them but cannot recover them, please restart your kylin >> process. It will be created automatically after restart. >> >> >> >> >> >> Third things about folder kylin_job_metaXXX: >> >> >> >> These files are created by >> org.apache.kylin.engine.mr.common.AbstractHadoopJob, method name is >> dumpKylinPropsAndMetadata. Such folder will be created when some step of a >> kylin job start. When each step finished, these files should be deleted >> automatically( in finally clause of method run). If you find them in >> tomcat/temp and it is not fresh (for example, it last-modified timestamp >> is two days ago), it could be deleted safely. The count of such files >> will grow rapidly if you have a lot of submited job. >> >> >> >> >> >> You can check source code in github repo if you are interested in them. >> If your find any mistake, please let me know, thank you. >> >> >> >> ---------------- >> >> Best wishes, >> >> Xiaoxiang Yu >> >> >> >> >> >> *发件人**: *Iñigo Martínez <[email protected]> >> *答复**: *"[email protected]" <[email protected]> >> *日期**: *2019年1月18日 星期五 22:35 >> *收件人**: *"[email protected]" <[email protected]> >> *主题**: *Safe deletion of $KYLIN_HOME/tomcat/temp stuff >> >> >> >> Good morning. >> >> >> >> We have detected that our Kylin installation has plenty temporary files >> located at $KYLIN_HOME/tomcat/temp. Now, around 90GB after two months >> running. >> >> >> >> Inside this folder, there is a "safeToDelete.tmp" file, so we assumed we >> could flush the contents of this folder safely. However, this was not true. >> As soon as we delete contents older than 2 days, a lot of errors appeared >> in kylin.log complaining about missing files, in particular those of type: >> >> >> >> Message: java.io.FileNotFoundException: >> /opt/kylin/current/bin/../tomcat/temp/olap_model_4116748333625199860.json >> (No such file or directory) >> >> >> >> We immediately restored all old stuff and service became operational >> again. Tomcat temp folder is fully of files like these: >> >> [image: Screenshot from 2019-01-18 13-48-14.png] >> >> >> >> So, question is, how do we deal with this folder cleanup? >> >> Can we delete all kylin_jobmetadataXXXXXX folders but maintain >> olap_modem_XXXXX.json files? >> >> Why a safeToDelete.tmp exists here? It only contributes to create >> confusion about safe cleanup of this folder contents. >> >> >> >> Thank you! >> >> >> > > > -- > > > > > > Íñigo Martínez > SYSTEMS MANAGER > [email protected] <[email protected]%3C> > > > > > > > [image: Telecoming - Make it digital] > [image: 5000_empresas] > [image: 1000_empresas] > > Paseo de la Castellana, 95. Torre Europa, pl 16. 28046 Madrid, Spain | > telecoming.com <http://www.telecoming.com/> > > > > Este correo electrónico y sus archivos adjuntos están dirigidos > únicamente a la(s) dirección(es) indicada(s) anteriormente. El carácter > confidencial, personal e intransferible del mismo está protegido > legalmente. Cualquier publicación, reproducción, distribución o > retransmisión no autorizada, ya sea completa o en parte, se encuentra > prohibida. Si ha recibido este mensaje por equivocación, notifíquelo > inmediatamente a la persona que lo ha enviado y borre el mensaje original > junto con sus ficheros anexos sin leerlo ni grabarlo en modo alguno. >
