Hi Xiaoxiang， I think you could put these operation info into Kylin FAQ, quite helpful.
With Warm regards Billy Liu ShaoFeng Shi <[email protected]> 于2019年1月21日周一 下午10:22写道： > Thanks for Xiao Xiang's information, it's quite clear. > > Best regards, > > Shaofeng Shi 史少锋 > Apache Kylin PMC > Work email: [email protected] > Kyligence Inc: https://kyligence.io/ > > Apache Kylin FAQ: https://kylin.apache.org/docs/gettingstarted/faq.html > Join Kylin user mail group: [email protected] > Join Kylin dev mail group: [email protected] > > > > > Iñigo Martínez <[email protected]> 于2019年1月21日周一 下午8:44写道： > >> Thank you, Xiaoxiang. >> >> Completely clear. We well proceed very carefully with .json deletion and >> we will only send to trash kylin_job_metaXXXX created several days ago. >> Meanwhile, I will check github in order to find why those folders are not >> deleted. >> >> >> El sáb., 19 ene. 2019 a las 12:34, Xiaoxiang Yu (< >> [email protected]>) escribió: >> >>> Hi, >>> >>> >>> >>> First things about safeToDelete.tmp: >>> >>> >>> >>> It looks like that ‘safeToDelete.tmp’ is a file which is >>> owned/controlled by tomcat, but not kylin. It is used to indicate that >>> folder tomcat/temp could be deleted safely. But that should not be true >>> when other program(in this case, that is kylin) use that folder, so, >>> for that reason, please ignore safeToDelete.tmp and delete >>> kylin-related files under tomcat/temp carefully. >>> >>> I am trying to find manual about that file in tomcat document, but I >>> failed. Related links >>> >>> >>> https://stackoverflow.com/questions/7112591/what-is-the-tomcat-temp-directory-in-tomcat-7 >>> >>> https://t246osslab.wordpress.com/2017/04/26/tomcatのsafetodelete-tmpの謎を追う >>> / >>> >>> >>> >>> >>> >>> >>> >>> Second things about olap_model_XXX.json: >>> >>> >>> >>> These files are created by >>> org.apache.kylin.query.schema.OLAPSchemaFactory. And it is read in sql >>> query analysis procedure by apache calcite(code in method model of >>> class CalciteConnectionConfigImpl and QueryConnection). So it must not >>> deleted. These files are very small, and count of such files is equals >>> to the count of kylin’s model. >>> >>> >>> >>> If you delete them but cannot recover them, please restart your kylin >>> process. It will be created automatically after restart. >>> >>> >>> >>> >>> >>> Third things about folder kylin_job_metaXXX: >>> >>> >>> >>> These files are created by >>> org.apache.kylin.engine.mr.common.AbstractHadoopJob, method name is >>> dumpKylinPropsAndMetadata. Such folder will be created when some step of a >>> kylin job start. When each step finished, these files should be deleted >>> automatically( in finally clause of method run). If you find them in >>> tomcat/temp and it is not fresh (for example, it last-modified >>> timestamp is two days ago), it could be deleted safely. The count of >>> such files will grow rapidly if you have a lot of submited job. >>> >>> >>> >>> >>> >>> You can check source code in github repo if you are interested in them. >>> If your find any mistake, please let me know, thank you. >>> >>> >>> >>> ---------------- >>> >>> Best wishes, >>> >>> Xiaoxiang Yu >>> >>> >>> >>> >>> >>> *发件人**: *Iñigo Martínez <[email protected]> >>> *答复**: *"[email protected]" <[email protected]> >>> *日期**: *2019年1月18日 星期五 22:35 >>> *收件人**: *"[email protected]" <[email protected]> >>> *主题**: *Safe deletion of $KYLIN_HOME/tomcat/temp stuff >>> >>> >>> >>> Good morning. >>> >>> >>> >>> We have detected that our Kylin installation has plenty temporary files >>> located at $KYLIN_HOME/tomcat/temp. Now, around 90GB after two months >>> running. >>> >>> >>> >>> Inside this folder, there is a "safeToDelete.tmp" file, so we assumed we >>> could flush the contents of this folder safely. However, this was not true. >>> As soon as we delete contents older than 2 days, a lot of errors appeared >>> in kylin.log complaining about missing files, in particular those of type: >>> >>> >>> >>> Message: java.io.FileNotFoundException: >>> /opt/kylin/current/bin/../tomcat/temp/olap_model_4116748333625199860.json >>> (No such file or directory) >>> >>> >>> >>> We immediately restored all old stuff and service became operational >>> again. Tomcat temp folder is fully of files like these: >>> >>> [image: Screenshot from 2019-01-18 13-48-14.png] >>> >>> >>> >>> So, question is, how do we deal with this folder cleanup? >>> >>> Can we delete all kylin_jobmetadataXXXXXX folders but maintain >>> olap_modem_XXXXX.json files? >>> >>> Why a safeToDelete.tmp exists here? It only contributes to create >>> confusion about safe cleanup of this folder contents. >>> >>> >>> >>> Thank you! >>> >>> >>> >> >> >> -- >> >> >> >> >> >> Íñigo Martínez >> SYSTEMS MANAGER >> [email protected] <[email protected]%3C> >> >> >> >> >> >> >> [image: Telecoming - Make it digital] >> [image: 5000_empresas] >> [image: 1000_empresas] >> >> Paseo de la Castellana, 95. Torre Europa, pl 16. 28046 Madrid, Spain | >> telecoming.com <http://www.telecoming.com/> >> >> >> >> Este correo electrónico y sus archivos adjuntos están dirigidos >> únicamente a la(s) dirección(es) indicada(s) anteriormente. El carácter >> confidencial, personal e intransferible del mismo está protegido >> legalmente. Cualquier publicación, reproducción, distribución o >> retransmisión no autorizada, ya sea completa o en parte, se encuentra >> prohibida. Si ha recibido este mensaje por equivocación, notifíquelo >> inmediatamente a la persona que lo ha enviado y borre el mensaje original >> junto con sus ficheros anexos sin leerlo ni grabarlo en modo alguno. >> >
