[
https://issues.apache.org/jira/browse/KYLIN-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383231#comment-17383231
]
Xiaoxiang Yu commented on KYLIN-5022:
-------------------------------------
!屏幕快照2.png!
[~star_dev] Hello, from the screenshots you
provided(https://issues.apache.org/jira/secure/attachment/13027509/%E5%B1%8F%E5%B9%95%E5%BF%AB%E7%85%A72.png),
I can see the lastModified of coprocessor.jar are different from each other.
We can see the code from the DeployCoprocessorCLI.java . The uploaded jar on S3
should be consistent with local coprocessor jar.
{code:java}
fileSystem.setTimes(uploadPath, localCoprocessorFile.lastModified(), -1); //
lastModified should be consistent
{code}
>From the method _*isSame*_, if lastModified of two coprocessor jars are
>different from each other, even if they the same jars, they two(remote and
>local) are regards as different one. So a new
>{color:#de350b}*(i++).jar*{color} will be used.
{code:java}
// DeployCoprocessorCLI.java
for (FileStatus fileStatus : fileSystem.listStatus(coprocessorDir)) {
if (isSame(localCoprocessorFile, fileStatus)) {
uploadPath = fileStatus.getPath();
break;
}
String filename = fileStatus.getPath().toString();
if (filename.endsWith(".jar")) {
oldJarPaths.add(filename);
}
}
// compare lastModified of two jars
private static boolean isSame(File localCoprocessorFile, FileStatus fileStatus)
{
return fileStatus.getLen() == localCoprocessorFile.length()
&& fileStatus.getModificationTime() ==
localCoprocessorFile.lastModified();
}
{code}
> kylin升级新版本-/mnt/tmp/hbase-hbase/local/jars/tmp产生大量的kylin-coprocessor文件
> ----------------------------------------------------------------------
>
> Key: KYLIN-5022
> URL: https://issues.apache.org/jira/browse/KYLIN-5022
> Project: Kylin
> Issue Type: Bug
> Components: Metadata, Storage - HBase
> Affects Versions: v3.1.1
> Reporter: star_dev
> Priority: Major
> Attachments: Capture.PNG, 屏幕快照1.png, 屏幕快照2.png, 日志.log
>
>
> kylin版本从3.0.2更新到3.1.1，还是用原来的元数据。
> 发现在EMR的core节点中有大量的kylin-coprocessor文件生成，见附件屏幕快照1，占用了大量的空间，导致hdfs文件系统可用空间变少。路径为/mnt/tmp/hbase-hbase/local/jars/tmp
> 查询官方文档 [http://kylin.apache.org/docs/howto/howto_update_coprocessor.html]
> 执行如下命令仍然不好用，日志信息见附件
> -------------------------
>
> {{$KYLIN_HOME/bin/kylin.sh
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI default all}}
> {{-------------------------}}
> 同时发现kylin元数据
> kylin_metadata/coprocessor/下有大量的kylin-coprocessor-3.1.1-*.jar文件，见附件屏幕快照2
>
> 是什么原因导致的这种现象？
> 如何才能在/mnt/tmp/hbase-hbase/local/jars/tmp路径下不再产生大量的文件？
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
