[jira] [Commented] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564351#comment-17564351
]
Alexander Lapin commented on IGNITE-17291:
--
[~Denis Chudov] In general LGTM, please check comments in PR.
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 50m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-13570) Migrate OSGI module to ignite-extensions
[ https://issues.apache.org/jira/browse/IGNITE-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564301#comment-17564301 ] Maxim Muzafarov commented on IGNITE-13570: -- Merged to the master branch. > Migrate OSGI module to ignite-extensions > > > Key: IGNITE-13570 > URL: https://issues.apache.org/jira/browse/IGNITE-13570 > Project: Ignite > Issue Type: Sub-task > Components: extensions >Reporter: Alexey Goncharuk >Assignee: Maxim Muzafarov >Priority: Major > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > Migrate OSGI module to ignite-extensions > https://github.com/apache/ignite-extensions > Details: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-15931) Implement storage for tx states
[ https://issues.apache.org/jira/browse/IGNITE-15931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Denis Chudov reassigned IGNITE-15931: - Assignee: Denis Chudov > Implement storage for tx states > --- > > Key: IGNITE-15931 > URL: https://issues.apache.org/jira/browse/IGNITE-15931 > Project: Ignite > Issue Type: Improvement >Reporter: Alexey Scherbakov >Assignee: Denis Chudov >Priority: Major > Labels: ignite-3, transaction3_rw > > Occurred that ParitionStorage will be removed soon, so it's required to > implement explicit tx state persistent storage. Generally speaking it'll > store map of txId to transaction meta, that will include > txnState(committed/aborted), list of enlisted partition ids and commit > timestamp. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17342) Fix compatibility tests running a new client with the old server
[ https://issues.apache.org/jira/browse/IGNITE-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17342: - Labels: compatibility (was: ) > Fix compatibility tests running a new client with the old server > > > Key: IGNITE-17342 > URL: https://issues.apache.org/jira/browse/IGNITE-17342 > Project: Ignite > Issue Type: Bug >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Labels: compatibility > Fix For: 2.14 > > Time Spent: 20m > Remaining Estimate: 0h > > Tests that are using JavaThinCompatibilityTest#testCurrentClientToOldServer > is not running. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17342) Fix compatibility tests running a new client with the old server
[ https://issues.apache.org/jira/browse/IGNITE-17342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-17342: - Ignite Flags: (was: Docs Required,Release Notes Required) > Fix compatibility tests running a new client with the old server > > > Key: IGNITE-17342 > URL: https://issues.apache.org/jira/browse/IGNITE-17342 > Project: Ignite > Issue Type: Bug >Reporter: Maxim Muzafarov >Assignee: Maxim Muzafarov >Priority: Major > Fix For: 2.14 > > > Tests that are using JavaThinCompatibilityTest#testCurrentClientToOldServer > is not running. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17342) Fix compatibility tests running a new client with the old server
Maxim Muzafarov created IGNITE-17342: Summary: Fix compatibility tests running a new client with the old server Key: IGNITE-17342 URL: https://issues.apache.org/jira/browse/IGNITE-17342 Project: Ignite Issue Type: Bug Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov Fix For: 2.14 Tests that are using JavaThinCompatibilityTest#testCurrentClientToOldServer is not running. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-13570) Migrate OSGI module to ignite-extensions
[ https://issues.apache.org/jira/browse/IGNITE-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564282#comment-17564282 ] Amelchev Nikita commented on IGNITE-13570: -- [~mmuzaf], LGTM. > Migrate OSGI module to ignite-extensions > > > Key: IGNITE-13570 > URL: https://issues.apache.org/jira/browse/IGNITE-13570 > Project: Ignite > Issue Type: Sub-task > Components: extensions >Reporter: Alexey Goncharuk >Assignee: Maxim Muzafarov >Priority: Major > Fix For: 2.14 > > Time Spent: 10m > Remaining Estimate: 0h > > Migrate OSGI module to ignite-extensions > https://github.com/apache/ignite-extensions > Details: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-13570) Migrate OSGI module to ignite-extensions
[ https://issues.apache.org/jira/browse/IGNITE-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Muzafarov updated IGNITE-13570: - Component/s: extensions > Migrate OSGI module to ignite-extensions > > > Key: IGNITE-13570 > URL: https://issues.apache.org/jira/browse/IGNITE-13570 > Project: Ignite > Issue Type: Sub-task > Components: extensions >Reporter: Alexey Goncharuk >Assignee: Maxim Muzafarov >Priority: Major > Fix For: 2.14 > > Time Spent: 10m > Remaining Estimate: 0h > > Migrate OSGI module to ignite-extensions > https://github.com/apache/ignite-extensions > Details: > https://cwiki.apache.org/confluence/display/IGNITE/IEP-36%3A+Modularization#IEP-36:Modularization-IndependentIntegrations -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17215) Write ClusterSnapshotRecord to WAL
[ https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin updated IGNITE-17215: Fix Version/s: 2.14 > Write ClusterSnapshotRecord to WAL > -- > > Key: IGNITE-17215 > URL: https://issues.apache.org/jira/browse/IGNITE-17215 > Project: Ignite > Issue Type: New Feature >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Labels: IEP-89 > Fix For: 2.14 > > Time Spent: 50m > Remaining Estimate: 0h > > For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. > It's required to have a point in WAL which splits whole WAL on 2 areas: > # Before this point all data changes are contained within ClusterSnapshot, > and no need to recover them from WAL archived files. > # After this point all data need to be recovered from WAL archived files. > It's proposed to write ClusterSnapshotRecord while the checkpoint is running > (cp#writeLock has acquired). ClusterSnapshot process guarantees: > # there is no active transactions (or any data changes) in moment of running > checkpoint. > # ClusterSnapshot contains all data pages that will be persisted within this > checkpoint process. > Then every logical record after begin CheckointRecord doesn't belong to > ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the > checkpoint process. > [1] > [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17215) Write ClusterSnapshotRecord to WAL
[ https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Timonin updated IGNITE-17215: Reviewer: Nikita Amelchev > Write ClusterSnapshotRecord to WAL > -- > > Key: IGNITE-17215 > URL: https://issues.apache.org/jira/browse/IGNITE-17215 > Project: Ignite > Issue Type: New Feature >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Labels: IEP-89 > Time Spent: 50m > Remaining Estimate: 0h > > For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. > It's required to have a point in WAL which splits whole WAL on 2 areas: > # Before this point all data changes are contained within ClusterSnapshot, > and no need to recover them from WAL archived files. > # After this point all data need to be recovered from WAL archived files. > It's proposed to write ClusterSnapshotRecord while the checkpoint is running > (cp#writeLock has acquired). ClusterSnapshot process guarantees: > # there is no active transactions (or any data changes) in moment of running > checkpoint. > # ClusterSnapshot contains all data pages that will be persisted within this > checkpoint process. > Then every logical record after begin CheckointRecord doesn't belong to > ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the > checkpoint process. > [1] > [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17269) Java service call interceptor
[
https://issues.apache.org/jira/browse/IGNITE-17269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564228#comment-17564228
]
Ignite TC Bot commented on IGNITE-17269:
{panel:title=Branch: [pull/10128/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10128/head] Base: [master] : New Tests
(12)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Service Grid{color} [[tests
12|https://ci2.ignite.apache.org/viewLog.html?buildId=6511203]]
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testOrder[clusterSingleton=false,
sticky=false] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testException[clusterSingleton=false,
sticky=false] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testOrder[clusterSingleton=true, sticky=true]
- PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testException[clusterSingleton=true,
sticky=true] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testOrder[clusterSingleton=false, sticky=true]
- PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testException[clusterSingleton=false,
sticky=true] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testRedeploy[clusterSingleton=true,
sticky=true] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testRedeploy[clusterSingleton=false,
sticky=true] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testRedeploy[clusterSingleton=false,
sticky=false] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testBasicInterception[clusterSingleton=false,
sticky=false] - PASSED{color}
* {color:#013220}IgniteServiceGridTestSuite:
IgniteServiceCallInterceptorTest.testBasicInterception[clusterSingleton=true,
sticky=true] - PASSED{color}
... and 1 new tests
{panel}
[TeamCity *-- Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6511222buildTypeId=IgniteTests24Java8_RunAll]
> Java service call interceptor
> -
>
> Key: IGNITE-17269
> URL: https://issues.apache.org/jira/browse/IGNITE-17269
> Project: Ignite
> Issue Type: Sub-task
> Components: managed services
>Reporter: Pavel Pereslegin
>Assignee: Pavel Pereslegin
>Priority: Major
> Labels: iep-79, ise
> Fix For: 2.14
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Implement interceptor for java service.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17215) Write ClusterSnapshotRecord to WAL
[ https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564227#comment-17564227 ] Maksim Timonin commented on IGNITE-17215: - [~NSAmelchev] thanks for review. Merged to master > Write ClusterSnapshotRecord to WAL > -- > > Key: IGNITE-17215 > URL: https://issues.apache.org/jira/browse/IGNITE-17215 > Project: Ignite > Issue Type: New Feature >Reporter: Maksim Timonin >Assignee: Maksim Timonin >Priority: Major > Labels: IEP-89 > Time Spent: 50m > Remaining Estimate: 0h > > For PITR [1] process of recovering based on ClusterSnapshot + archived WALs. > It's required to have a point in WAL which splits whole WAL on 2 areas: > # Before this point all data changes are contained within ClusterSnapshot, > and no need to recover them from WAL archived files. > # After this point all data need to be recovered from WAL archived files. > It's proposed to write ClusterSnapshotRecord while the checkpoint is running > (cp#writeLock has acquired). ClusterSnapshot process guarantees: > # there is no active transactions (or any data changes) in moment of running > checkpoint. > # ClusterSnapshot contains all data pages that will be persisted within this > checkpoint process. > Then every logical record after begin CheckointRecord doesn't belong to > ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the > checkpoint process. > [1] > [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17215) Write ClusterSnapshotRecord to WAL
[
https://issues.apache.org/jira/browse/IGNITE-17215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564225#comment-17564225
]
Ignite TC Bot commented on IGNITE-17215:
{panel:title=Branch: [pull/10105/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10105/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6672103buildTypeId=IgniteTests24Java8_RunAll]
> Write ClusterSnapshotRecord to WAL
> --
>
> Key: IGNITE-17215
> URL: https://issues.apache.org/jira/browse/IGNITE-17215
> Project: Ignite
> Issue Type: New Feature
>Reporter: Maksim Timonin
>Assignee: Maksim Timonin
>Priority: Major
> Labels: IEP-89
> Time Spent: 40m
> Remaining Estimate: 0h
>
> For PITR [1] process of recovering based on ClusterSnapshot + archived WALs.
> It's required to have a point in WAL which splits whole WAL on 2 areas:
> # Before this point all data changes are contained within ClusterSnapshot,
> and no need to recover them from WAL archived files.
> # After this point all data need to be recovered from WAL archived files.
> It's proposed to write ClusterSnapshotRecord while the checkpoint is running
> (cp#writeLock has acquired). ClusterSnapshot process guarantees:
> # there is no active transactions (or any data changes) in moment of running
> checkpoint.
> # ClusterSnapshot contains all data pages that will be persisted within this
> checkpoint process.
> Then every logical record after begin CheckointRecord doesn't belong to
> ClusterSnapshot. Then it's safe to write ClusterSnapshotRecord within the
> checkpoint process.
> [1]
> [https://cwiki.apache.org/confluence/pages/editpage.action?pageId=211884314]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17341) Support RAFT configuration with HOCON
Ivan Bessonov created IGNITE-17341: -- Summary: Support RAFT configuration with HOCON Key: IGNITE-17341 URL: https://issues.apache.org/jira/browse/IGNITE-17341 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Currently, the only way to change RAFT settings is to explicitly change them in code, which is not convenient for the actual usage of the product. Some options have to be available -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17340) Disable fsync in RAFT log by default
Ivan Bessonov created IGNITE-17340:
--
Summary: Disable fsync in RAFT log by default
Key: IGNITE-17340
URL: https://issues.apache.org/jira/browse/IGNITE-17340
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
Currently, {{org.apache.ignite.raft.jraft.option.RaftOptions#sync}} leads to
long inserts into tables, Runner TC suite can be made significantly faster with
disabling fsync by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17340) Disable fsync in RAFT log by default
[
https://issues.apache.org/jira/browse/IGNITE-17340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov reassigned IGNITE-17340:
--
Assignee: Ivan Bessonov
> Disable fsync in RAFT log by default
>
>
> Key: IGNITE-17340
> URL: https://issues.apache.org/jira/browse/IGNITE-17340
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
>
> Currently, {{org.apache.ignite.raft.jraft.option.RaftOptions#sync}} leads to
> long inserts into tables, Runner TC suite can be made significantly faster
> with disabling fsync by default.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17339) Implement B+Tree based hash index storage
Ivan Bessonov created IGNITE-17339: -- Summary: Implement B+Tree based hash index storage Key: IGNITE-17339 URL: https://issues.apache.org/jira/browse/IGNITE-17339 Project: Ignite Issue Type: Improvement Reporter: Ivan Bessonov Please refer to IGNITE-17320 and issues from the epic for the gist. It's basically the same thing, but with hash slapped inside of the tree pages and a simplified comparison algorithm. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17046) Move H2RowComparator logic to the core module
[
https://issues.apache.org/jira/browse/IGNITE-17046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564194#comment-17564194
]
Ignite TC Bot commented on IGNITE-17046:
{panel:title=Branch: [pull/10080/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10080/head] Base: [master] : New Tests
(6)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Queries 3{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6673291]]
* {color:#013220}IgniteBinaryCacheQueryTestSuite3:
SqlQueryIndexWithDifferentTypeTest.testQueryIndex - PASSED{color}
{color:#8b}Queries 3 (lazy=true){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6673292]]
* {color:#013220}IgniteBinaryCacheQueryLazyTestSuite3:
SqlQueryIndexWithDifferentTypeTest.testQueryIndex - PASSED{color}
{color:#8b}Index Query API{color} [[tests
4|https://ci.ignite.apache.org/viewLog.html?buildId=6673406]]
* {color:#013220}IndexQueryTestSuite:
IndexQueryAllTypesTest.testCrossTypeComparisonWithOutOfBoundsFilter[useIdxName=true]
- PASSED{color}
* {color:#013220}IndexQueryTestSuite:
IndexQueryAllTypesTest.testRangeCrossTypeConvertion[useIdxName=false] -
PASSED{color}
* {color:#013220}IndexQueryTestSuite:
IndexQueryAllTypesTest.testRangeCrossTypeConvertion[useIdxName=true] -
PASSED{color}
* {color:#013220}IndexQueryTestSuite:
IndexQueryAllTypesTest.testCrossTypeComparisonWithOutOfBoundsFilter[useIdxName=false]
- PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6673317buildTypeId=IgniteTests24Java8_RunAll]
> Move H2RowComparator logic to the core module
> -
>
> Key: IGNITE-17046
> URL: https://issues.apache.org/jira/browse/IGNITE-17046
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
> Time Spent: 2h 40m
> Remaining Estimate: 0h
>
> {{H2RowComparator}} depends on H2 classes and can't be used without
> ignite-indexing module. We should move the logic from this comparator to the
> core module providing backward compatibility, to be able to use indexes
> without H2.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17338) Implement RocksDB based hash index storage
Ivan Bessonov created IGNITE-17338:
--
Summary: Implement RocksDB based hash index storage
Key: IGNITE-17338
URL: https://issues.apache.org/jira/browse/IGNITE-17338
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
Please see IGNITE-17318 for partial description of what needs to be achieved.
I expect that hash index records will have the following structure:
{code:java}
[ indexId | partitionId | hash | tuple | rowId ] -> []{code}
Fixed-length prefix should cover indexId, partitionId and hash value.
Searching rows effectively becomes a scan, but this is fine.
Hashing must be performed internally, hash function already presents somewhere
in the code.
Is far as I understand, PK is going to be implemented as a secondary hash index.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564177#comment-17564177
]
Denis Chudov edited comment on IGNITE-17291 at 7/8/22 9:08 AM:
---
[~sanpwc] could you pls review the code?
was (Author: denis chudov):
[~alapin] could you pls review the code?
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17291) Implement metastorage cursor batching
[
https://issues.apache.org/jira/browse/IGNITE-17291?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17564177#comment-17564177
]
Denis Chudov commented on IGNITE-17291:
---
[~alapin] could you pls review the code?
> Implement metastorage cursor batching
> -
>
> Key: IGNITE-17291
> URL: https://issues.apache.org/jira/browse/IGNITE-17291
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Denis Chudov
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> In order to increase metaStorage.range() performance that currently retrieves
> entries one by one it's possible to implement simple batching. As an initial
> solution we might hardcode the batch-size.
> Basically speaking it's required to update CursorNextCommand.
> Instead of
> {code:java}
> Entry e = (Entry) cursorDesc.cursor().next();
> clo.result(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter())); {code}
> we might use something similar to
> {code:java}
> List batch = new
> ArrayList<>(RANGE_CURSOR_BATCH_SIZE);
> for (int i = 0; i < RANGE_CURSOR_BATCH_SIZE; i++) {
> Entry e = (Entry) cursorDesc.cursor().next();
> batch.add(new SingleEntryResponse(e.key(), e.value(), e.revision(),
> e.updateCounter()));
> if (! cursorDesc.cursor.hasNext()) {
> break;
> }
> }
> clo.result(new MultipleEntryResponse(batch));{code}
> It's not trivial to reimplement rocks cursors to also use batching, however
> it's not that important because rocks cursors are always local.
>
> Besides that it's required to update
> org.apache.ignite.internal.metastorage.client.CursorImpl with
> client-side-iteration over batched data and requesting new portion if nothing
> left.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
