[jira] [Commented] (CARBONDATA-311) Log the data size of blocklet during data load.
[ https://issues.apache.org/jira/browse/CARBONDATA-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15595192#comment-15595192 ] ASF GitHub Bot commented on CARBONDATA-311: --- Github user Zhangshunyu closed the pull request at: https://github.com/apache/incubator-carbondata/pull/231 > Log the data size of blocklet during data load. > --- > > Key: CARBONDATA-311 > URL: https://issues.apache.org/jira/browse/CARBONDATA-311 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 0.1.1-incubating >Reporter: zhangshunyu >Assignee: zhangshunyu >Priority: Minor > Fix For: 0.2.0-incubating > > > Log the data size of blocklet during data load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (CARBONDATA-311) Log the data size of blocklet during data load.
[
https://issues.apache.org/jira/browse/CARBONDATA-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15584415#comment-15584415
]
ASF GitHub Bot commented on CARBONDATA-311:
---
Github user Vimal-Das commented on a diff in the pull request:
https://github.com/apache/incubator-carbondata/pull/231#discussion_r83780338
--- Diff:
processing/src/main/java/org/apache/carbondata/processing/store/writer/CarbonFactDataWriterImplForIntIndexAndAggBlock.java
---
@@ -250,6 +254,7 @@ public NodeHolder
buildDataNodeHolder(IndexStorage[] keyStorageArray, byt
}
long blockletDataSize =
holder.getKeyArray().length + holder.getDataArray().length +
indexBlockSize;
+LOGGER.info("A new blocklet is added, its data size is: " +
blockletDataSize + " Byte");
--- End diff --
Move this line after call to writeDataToFile, because if writeDataToFile
fails, we should not log that a blocklet is added.
> Log the data size of blocklet during data load.
> ---
>
> Key: CARBONDATA-311
> URL: https://issues.apache.org/jira/browse/CARBONDATA-311
> Project: CarbonData
> Issue Type: Improvement
>Affects Versions: 0.1.1-incubating
>Reporter: zhangshunyu
>Assignee: zhangshunyu
>Priority: Minor
> Fix For: 0.2.0-incubating
>
>
> Log the data size of blocklet during data load.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (CARBONDATA-311) Log the data size of blocklet during data load.
[ https://issues.apache.org/jira/browse/CARBONDATA-311?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15570686#comment-15570686 ] ASF GitHub Bot commented on CARBONDATA-311: --- GitHub user Zhangshunyu opened a pull request: https://github.com/apache/incubator-carbondata/pull/231 [CARBONDATA-311]Log the data size of blocklet during data load. ## Why raise this pr? The blocklet size is an important parameter for analyzing data load and query, this info should be logged. ## How to test? Pass all the test case. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Zhangshunyu/incubator-carbondata logblocklet Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-carbondata/pull/231.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #231 commit a110504f58e688e42223e896f7a1cf729463cf9d Author: Zhangshunyu Date: 2016-10-13T03:17:21Z Log the data size of each blocklet > Log the data size of blocklet during data load. > --- > > Key: CARBONDATA-311 > URL: https://issues.apache.org/jira/browse/CARBONDATA-311 > Project: CarbonData > Issue Type: Improvement >Affects Versions: 0.1.1-incubating >Reporter: zhangshunyu >Assignee: zhangshunyu >Priority: Minor > Fix For: 0.2.0-incubating > > > Log the data size of blocklet during data load. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
