[
https://issues.apache.org/jira/browse/KUDU-2671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17772721#comment-17772721
]
ASF subversion and git services commented on KUDU-2671:
-------------------------------------------------------
Commit 959b0e67df6d86431e874eefae48700b627ba273 in kudu's branch
refs/heads/branch-1.17.x from Alexey Serbin
[ https://gitbox.apache.org/repos/asf?p=kudu.git;h=959b0e67d ]
KUDU-3515: fix incompatibility introduced with KUDU-2671
This patch addresses the issue reported in KUDU-3515. The issue has
been originally reported at the #kudu-general Slack channel [1].
The root cause of the issue is an incompatibility in the serialized
representation of tablets' partition keys introduced by changelist [2].
The essence of the fix is to convert the serialized partition keys
representing unbounded ends of the tables' ranges from the legacy
to the new format on-the-fly while loading that information from the
system catalog upon bootstrapping a leader master.
This patch contains unit test scenarios for the function that performs
the conversion from the legacy to the new format.
Also, I verified the fix works as expected for an existing Kudu 1.16.0
cluster with a bunch of tables after upgrading the binaries to Kudu
1.17.0 bits.
This is a follow-up to 8df970f7a6520bb0dc0f9cc89ad7f62ab349e84d.
[1] https://getkudu.slack.com/archives/C0CPXJ3CH/p1695107377230829
[2] https://github.com/apache/kudu/commit/8df970f7a652
Change-Id: I45df424770a09cf7c94f5e1d390757f29f9fb3f4
Reviewed-on: http://gerrit.cloudera.org:8080/20525
Tested-by: Alexey Serbin <[email protected]>
Reviewed-by: Mahesh Reddy <[email protected]>
Reviewed-by: Abhishek Chennaka <[email protected]>
(cherry picked from commit 9ae68af0dcd7a58c9fe754f69fb99861ba26e2ff)
Reviewed-on: http://gerrit.cloudera.org:8080/20536
Tested-by: Kudu Jenkins
> Change hash number for range partitioning
> -----------------------------------------
>
> Key: KUDU-2671
> URL: https://issues.apache.org/jira/browse/KUDU-2671
> Project: Kudu
> Issue Type: Improvement
> Components: client, java, master, server
> Affects Versions: 1.8.0
> Reporter: yangz
> Assignee: Mahesh Reddy
> Priority: Major
> Labels: feature, roadmap-candidate, scalability
> Fix For: 1.17.0
>
> Attachments: 屏幕快照 2019-01-24 下午12.03.41.png
>
>
> For our usage, the kudu schema design isn't flexible enough.
> We create our table for day range such as dt='20181112' as hive table.
> But our data size change a lot every day, for one day it will be 50G， but for
> some other day it will be 500G. For this case, it be hard to set the hash
> schema. If too big, for most case, it will be too wasteful. But too small,
> there is a performance problem in the case of a large amount of data.
>
> So we suggest a solution we can change the hash number by the history data of
> a table.
> for example
> # we create schema with one estimated value.
> # we collect the data size by day range
> # we create new day range partition by our collected day size.
> We use this feature for half a year, and it work well. We hope this feature
> will be useful for the community. Maybe the solution isn't so complete.
> Please help us make it better.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
