[
https://issues.apache.org/jira/browse/KYLIN-5022?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17384695#comment-17384695
]
Gabor Arki commented on KYLIN-5022:
-----------------------------------
Maybe using
[checksum|https://hadoop.apache.org/docs/r2.8.2/api/org/apache/hadoop/fs/FileSystem.html#getFileChecksum(org.apache.hadoop.fs.Path)],
although couldn't tell whether this one is applicable to the S3 filesystem.
Also, the implemented algorithm is dependent on the filesystem used.
Another workaround could be to extend the upload method with the solution I am
currently using: after the file has been copied to the
{{hdfsWorkingDirectory}}, copy it back to the local file system.
Or maybe an even easier solution to reverse setting the last modified
timestamp: instead of trying to set it on the filesystem, take the last
modified of the newly uploaded file and set that value as the last modified for
the original jar on the local file system.
> kylin升级新版本-/mnt/tmp/hbase-hbase/local/jars/tmp产生大量的kylin-coprocessor文件
> ----------------------------------------------------------------------
>
> Key: KYLIN-5022
> URL: https://issues.apache.org/jira/browse/KYLIN-5022
> Project: Kylin
> Issue Type: Bug
> Components: Metadata, Storage - HBase
> Affects Versions: v3.1.1
> Reporter: star_dev
> Priority: Major
> Attachments: Capture.PNG, 屏幕快照1.png, 屏幕快照2.png, 日志.log
>
>
> kylin版本从3.0.2更新到3.1.1，还是用原来的元数据。
> 发现在EMR的core节点中有大量的kylin-coprocessor文件生成，见附件屏幕快照1，占用了大量的空间，导致hdfs文件系统可用空间变少。路径为/mnt/tmp/hbase-hbase/local/jars/tmp
> 查询官方文档 [http://kylin.apache.org/docs/howto/howto_update_coprocessor.html]
> 执行如下命令仍然不好用，日志信息见附件
> -------------------------
>
> {{$KYLIN_HOME/bin/kylin.sh
> org.apache.kylin.storage.hbase.util.DeployCoprocessorCLI default all}}
> {{-------------------------}}
> 同时发现kylin元数据
> kylin_metadata/coprocessor/下有大量的kylin-coprocessor-3.1.1-*.jar文件，见附件屏幕快照2
>
> 是什么原因导致的这种现象？
> 如何才能在/mnt/tmp/hbase-hbase/local/jars/tmp路径下不再产生大量的文件？
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
