[
https://issues.apache.org/jira/browse/KYLIN-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383245#comment-17383245
]
Gabor Arki commented on KYLIN-5022:
-----------------------------------
[~xxyu], I did some additional investigation and after finding the
aforementioned {{isSame}} method I also found the root cause.
We are using S3 instead of HDFS and the problem is caused by that. S3 does not
support setting the last modified timestamp. After the coprocessor jar has been
copied to the remote file system, the invoked {{setTimes}} is silently ignored
in the case of an S3 filesystem. Because of that, {{isSame}} will always return
with false and for each and every table a new coprocessor jar is uploaded.
As a manual workaround, I copied the coprocessor jar manually to S3 and then
copied it back from S3 to the local file system. This way the last modified
timestamps are matching and only one coprocessor jar is used.
> kylin升级新版本-/mnt/tmp/hbase-hbase/local/jars/tmp产生大量的kylin-coprocessor文件
> ----------------------------------------------------------------------
>
> Key: KYLIN-5022
> URL: https://issues.apache.org/jira/browse/KYLIN-5022
> Project: Kylin
> Issue Type: Bug
> Components: Metadata, Storage - HBase
> Affects Versions: v3.1.1
> Reporter: star_dev
> Priority: Major
> Attachments: Capture.PNG, 屏幕快照1.png, 屏幕快照2.png, 日志.log
>
>
> kylin版本从3.0.2更新到3.1.1，还是用原来的元数据。
> 发现在EMR的core节点中有大量的kylin-coprocessor文件生成，见附件屏幕快照1，占用了大量的空间，导致hdfs文件系统可用空间变少。路径为/mnt/tmp/hbase-hbase/local/jars/tmp
> 查询官方文档 [http://kylin.apache.org/docs/howto/howto_update_coprocessor.html]
> 执行如下命令仍然不好用，日志信息见附件
> -------------------------
>
> {{$KYLIN_HOME/bin/kylin.sh
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI default all}}
> {{-------------------------}}
> 同时发现kylin元数据
> kylin_metadata/coprocessor/下有大量的kylin-coprocessor-3.1.1-*.jar文件，见附件屏幕快照2
>
> 是什么原因导致的这种现象？
> 如何才能在/mnt/tmp/hbase-hbase/local/jars/tmp路径下不再产生大量的文件？
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
