[
https://issues.apache.org/jira/browse/KUDU-1861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Serbin updated KUDU-1861:
--------------------------------
Description:
In the context of use case to '...generate as many Kudu blocks as possible...',
the 'kudu test loadgen' tool can do better job if exercising a load pattern
which maximizes throughput and avoids compaction activity on tablet servers.
In short, the default behavior should change for the auto-created table case,
so the tool would:
# create a table with N partitions (where n == number of generator threads)
# let each worker thread insert sequentially into its own partition
Current option of having hash-partioned auto-created table should be preserved,
but turned off by default. For some test scenarios, it makes sense to exercise
data load patterns which involve a lot of compaction activity on the tablet
servers.
was:
In the context of use case to '...generate as many Kudu blocks as possible...',
the 'kudu test loadgen' tool can do better job if exercising a load pattern
which maximizes throughput and avoid compaction on tablet servers.
In short, an additional option (and this should be on by default) should change
the behavior of the tool if running against an auto-created table, so the tool
would:
# create a table with N partitions (where n == number of generator threads)
# let each worker thread insert sequentially into its own partition
Current option of having hash-partioned auto-created table should be preserved.
It allows to exercise data load patterns which involves a lot of compaction
activity on the tablet servers.
> kudu test loadgen: change default behavior to avoid compactions on tablet
> servers
> ----------------------------------------------------------------------------------
>
> Key: KUDU-1861
> URL: https://issues.apache.org/jira/browse/KUDU-1861
> Project: Kudu
> Issue Type: Improvement
> Components: util
> Affects Versions: 1.2.0
> Reporter: Alexey Serbin
>
> In the context of use case to '...generate as many Kudu blocks as
> possible...', the 'kudu test loadgen' tool can do better job if exercising a
> load pattern which maximizes throughput and avoids compaction activity on
> tablet servers.
> In short, the default behavior should change for the auto-created table case,
> so the tool would:
> # create a table with N partitions (where n == number of generator threads)
> # let each worker thread insert sequentially into its own partition
> Current option of having hash-partioned auto-created table should be
> preserved, but turned off by default. For some test scenarios, it makes
> sense to exercise data load patterns which involve a lot of compaction
> activity on the tablet servers.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
