[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17214421#comment-17214421
]
Zhong Yanghong commented on KYLIN-3271:
---
This change blocks the following command:
{code}
${KYLIN_HOME}/bin/metastore.sh fetch /execute
{code}
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16872041#comment-16872041
]
ASF subversion and git services commented on KYLIN-3271:
Commit 95dc0bf8d5dfa05d840409f5ab82862b25f53d80 in kylin's branch
refs/heads/2.6.x from jie.zou
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=95dc0bf ]
KYLIN-3271 KYLIN-3454 minor refactor,accelerate ResourceTool
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857248#comment-16857248
]
ASF GitHub Bot commented on KYLIN-3271:
---
nichunen commented on pull request #674: KYLIN-3271 KYLIN-3454 minor refactor,
accelerate ResourceTool
URL: https://github.com/apache/kylin/pull/674
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16857249#comment-16857249
]
ASF subversion and git services commented on KYLIN-3271:
Commit 3ec378ce745ee273b47ad9ecee4eeb917b470ca4 in kylin's branch
refs/heads/master from jie.zou
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=3ec378c ]
KYLIN-3271 KYLIN-3454 minor refactor,accelerate ResourceTool
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16856324#comment-16856324
]
ASF GitHub Bot commented on KYLIN-3271:
---
jiezouSH commented on pull request #674: KYLIN-3271 KYLIN-3454 minor refactor,
accelerate ResourceTool
URL: https://github.com/apache/kylin/pull/674
KYLIN-3271 KYLIN-3454 minor refactor, accelerate ResourceTool
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394440#comment-16394440
]
ASF subversion and git services commented on KYLIN-3271:
Commit 24d2d6590d1e31edb5f5b7da11c51c477f9d2dcf in kylin's branch
refs/heads/master from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=24d2d65 ]
KYLIN-3271 Optimize sub-path check of ResourceTool
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394441#comment-16394441
]
ASF subversion and git services commented on KYLIN-3271:
Commit c2ef3c62a96ca52ef673335f621232d6d664c24a in kylin's branch
refs/heads/master from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c2ef3c6 ]
KYLIN-3271 Minor, avoid null pointer exception
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394406#comment-16394406
]
ASF subversion and git services commented on KYLIN-3271:
Commit 24d2d6590d1e31edb5f5b7da11c51c477f9d2dcf in kylin's branch
refs/heads/sync from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=24d2d65 ]
KYLIN-3271 Optimize sub-path check of ResourceTool
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (KYLIN-3271) Optimize sub-path check of ResourceTool
[
https://issues.apache.org/jira/browse/KYLIN-3271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16394408#comment-16394408
]
ASF subversion and git services commented on KYLIN-3271:
Commit c2ef3c62a96ca52ef673335f621232d6d664c24a in kylin's branch
refs/heads/sync from [~nichunen]
[ https://gitbox.apache.org/repos/asf?p=kylin.git;h=c2ef3c6 ]
KYLIN-3271 Minor, avoid null pointer exception
> Optimize sub-path check of ResourceTool
> ---
>
> Key: KYLIN-3271
> URL: https://issues.apache.org/jira/browse/KYLIN-3271
> Project: Kylin
> Issue Type: Improvement
> Components: Metadata
>Affects Versions: v2.2.0
>Reporter: nichunen
>Assignee: nichunen
>Priority: Minor
> Fix For: v2.4.0
>
>
> kylin uses class org.apache.kylin.common.persistence.ResourceTool to do
> metadata download, upload, remove, etc. The algorithm for resource
> transversal is not very effective. For instance, for an "execute_output" with
> key "/execute_output/\{uuid}", the algorithm will try to check whether it's a
> folder with sub-resources, this makes un-necessary time cost, and in cases of
> metadata with lots of jobs, it may last for a long time before the finish.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
