Would blocklet division and distribution not help to solve this scenario?
On 2016-09-28 12:36 (+0530), Zhangshunyu wrote:
> For each table, we can set block size consider the data.size, this is because>
> that when execute query, each task will get one block to process one time,>
> when the
For each table, we can set block size consider the data.size, this is because
that when execute query, each task will get one block to process one time,
when the blocks num < parallelism, set a reasonable block size would get
most suitable block num, to make the best of parallelism.
--
View th
What problems we want to solve by making it configurable?
On 2016-09-26 07:36 (+0530), Zhangshunyu wrote:
> Purpose:>
> To configure block file size for each table on column level, so that each>
> table could has its own blocksize.>
> My solution:>
> Add a new parameter in table properties
I have verified that it would not affect the older tables.
--
View this message in context:
http://apache-carbondata-mailing-list-archive.1130556.n5.nabble.com/Discuss-Set-block-size-for-table-on-table-level-tp1472p1531.html
Sent from the Apache CarbonData Mailing List archive mailing list arch
+1, agree with
jihong.--发件人：金铸
发送时间：2016年9月27日(星期二) 08:22收件人：dev
主 题：Re: [Discuss]Set block_size for table
on table level
+1，agree with jihong.
在 2016/9/27 5:12, chenliang613 写道:
> +1, agree with Jihong's comment : ma
oduce this param
as an optional field, as long as it is not a required field, we are fine
with a defined default block size.
Regards.
Jihong
-Original Message-
From: Jacky Li [mailto:
jacky.likun@
]
Sent: Monday, September 26, 2016 7:29 AM
To:
dev@.apache
Subject: Re: [Discuss]Set bloc
s not a required field, we are fine
> with a defined default block size.
>
> Regards.
>
> Jihong
>
> -Original Message-
> From: Jacky Li [mailto:
> jacky.likun@
> ]
> Sent: Monday, September 26, 2016 7:29 AM
> To:
> dev@.apache
> Subject: R
+1
Regards
JB
On 09/26/2016 04:29 PM, Jacky Li wrote:
I am OK with this feature, the only thing I am worrying about is the
compatibility of CarbonData file reader. Can you make it compatible when you
reading old CarbonData file without this property.
We have encountered many times that user n
rds.
>
> Jihong
>
> -Original Message-
> From: Jacky Li [mailto:jacky.li...@qq.com]
> Sent: Monday, September 26, 2016 7:29 AM
> To: dev@carbondata.incubator.apache.org
> Subject: Re: [Discuss]Set block_size for table on table level
>
> I am OK with this featu
26, 2016 7:29 AM
To: dev@carbondata.incubator.apache.org
Subject: Re: [Discuss]Set block_size for table on table level
I am OK with this feature, the only thing I am worrying about is the
compatibility of CarbonData file reader. Can you make it compatible when you
reading old CarbonData file
I am OK with this feature, the only thing I am worrying about is the
compatibility of CarbonData file reader. Can you make it compatible when you
reading old CarbonData file without this property.
We have encountered many times that user need to delete the store and re-load
the data.
Regards,
J
+1
At same time max and min block size should be restricted and validated
while creating table.
On 26 September 2016 at 07:36, Zhangshunyu wrote:
> Purpose:
> To configure block file size for each table on column level, so that each
> table could has its own blocksize.
> My solution:
> Add a new
Purpose:
To configure block file size for each table on column level, so that each
table could has its own blocksize.
My solution:
Add a new parameter in table properties, when create a table, the user can
set it in ddl. Add a parameter in thrift format just like other properties,
and write this in
13 matches
Mail list logo
