[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17291:
-------------------------------------
Description:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
List<SingleEntryResponse> batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
}
clo.result(new MultipleEntryResponse(batch));{code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
Besides that it's required to update
org.apache.ignite.internal.metastorage.client.CursorImpl with
client-side-iteration over batched data and requesting new portion if nothing
left.
was:
In order to increase metaStorage.range() performance that currently retrieves
entries one by one it's possible to implement simple batching. As an initial
solution we might hardcode the batch-size.
Basically speaking it's required to update CursorNextCommand.
Instead of
{code:java}
Entry e = (Entry) cursorDesc.cursor().next();
clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter())); {code}
we might use something similar to
{code:java}
List<SingleEntryResponse> batch = new ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
Entry e = (Entry) cursorDesc.cursor().next();
batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
e.updateCounter()));
if (! cursorDesc.cursor.hasNext()) {
break;
}
}
clo.result(new MultipleEntryResponse(batch));{code}
It's not trivial to reimplement rocks cursors to also use batching, however
it's not that important because rocks cursors are always local.
> Implement metastorage cursor batching
> -------------------------------------
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
> Reporter: Alexander Lapin
> Priority: Major
> Labels: ignite-3
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List<SingleEntryResponse> batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
