[jira] [Assigned] (IGNITE-20018) Introduce system view definition API
[
https://issues.apache.org/jira/browse/IGNITE-20018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maksim Zhuravkov reassigned IGNITE-20018:
-
Assignee: Maksim Zhuravkov
> Introduce system view definition API
>
>
> Key: IGNITE-20018
> URL: https://issues.apache.org/jira/browse/IGNITE-20018
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> Let's introduce API to define a system view. Below is a few examples of how
> it may look like:
> {code:java}
> SystemView.clusterViewBuilder()
> .name("USERS")
> .addColumn("ID", Integer.class, Table::id)
> .addColumn("SCHEMA_ID", Integer.class, Table::schemaId)
> .addColumn("NAME", String.class, Table::name)
> .addColumn("COLUMN_NAMES", String.class, Table::columnNames)
> .dataProvider(() -> toAsyncCursor(tableManager.allTables()))
> .build();
> SystemView.nodeViewBuilder()
> .name("CONNECTIONS")
> .addColumn("USER", String.class, Connection::user)
> .addColumn("ADDRESS", String.class, Connection::address)
> .addColumn("ATTRS", String.class, Connection::attrs)
> .localNodeColumnAlias("NODE_ID")
> .dataProvider(() -> toAsyncCursor(connManager.connections()))
> .build();
> {code}
> Explanation of the code snippet above:
> * We need to distinguish between views exposing data with cluster-wide
> semantic and views exposing data with node-specific semantic. Such
> segregation is required to properly map queries' fragments to nodes owning
> data.
> * *name* attribute – is a name of the view under which it will be available
> in sql
> * *addColumn* attribute – appends a column to the view definition. View
> definition should have columns in the order of invocation of addColumn
> method. This method accepts three parameters: name, column type represented
> by class, and property reader. Name is a name of the column, under which the
> column under which it will be available in sql. Class is required to properly
> build the view's description. Property reader is a function which accepts
> objects of type specified in generic of builder, and returns value for the
> column of interest.
> * *dataProvider* attribute – provides an access to data this view is supposed
> to expose. Data provider accepts the supplier of AsyncCursor (see
> org.apache.ignite.internal.sql.engine.AsyncCursor; should be moved to core).
> AsyncCursor is required, because some of the data may be not stored locally,
> thus remote call will be required. To avoid blocking of query engine threads,
> it's better to integrate with async primitives, rather than force every
> module to store required state in local collections.
> * *localNodeColumnAlias* attribute – alias for the column exposing node id.
> This attribute is available only for nodeView’s, since node id will be
> injected automatically only for this type of view.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20274) Github actions must rely on environment variables instead of context
Maxim Muzafarov created IGNITE-20274: Summary: Github actions must rely on environment variables instead of context Key: IGNITE-20274 URL: https://issues.apache.org/jira/browse/IGNITE-20274 Project: Ignite Issue Type: Bug Reporter: Maxim Muzafarov Assignee: Maxim Muzafarov Fix For: 2.16 Github actions that are used to perform verification against the source code must use environment variables set directly on the action. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-20272: Assignee: Sergey Uttsel (was: Vyacheslav Koptilin) > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Uttsel >Assignee: Sergey Uttsel >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually, the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need to remove some tests in this class now and later > remove this class when ticket > https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll, and the test passed successfully. This test can be removed > because it repeats the > testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. > # testDataNodesUpdatedOnZoneManagerStart is the happy path of the restart, we > already have such tests. Therefore, this test is not needed and can be > removed. > # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the > zones manager restart, the data nodes for the zone will not be updated to the > value corresponding to the logical topology from the vault, because > zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not > found an analog of this test. I think that when the DZM restart is updated > then we can update this test and move it to the more appropriate class. > h3. *Definition of done* > # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. > # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. > # TODOs with IGNITE-18564 are removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20273) Breakdown for zone-based collocation
[ https://issues.apache.org/jira/browse/IGNITE-20273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov reassigned IGNITE-20273: --- Assignee: Kirill Gusakov > Breakdown for zone-based collocation > > > Key: IGNITE-20273 > URL: https://issues.apache.org/jira/browse/IGNITE-20273 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Kirill Gusakov >Priority: Major > > Based on the current design goals we need to prepare the tickets' breakdown > for the first phase of collocation > [design|https://github.com/gridgain/apache-ignite-3/blob/1abb67c8770179d5d3e295876b73893f3845a08e/modules/distribution-zones/tech-notes/zone-collocation.md#L1]: > Under the first phase we want to implement the following steps: > - Store the table partitions data inside the one zone partition > - Change the RAFT servers and clients management to support the separate > start of zone partitions and start of table state machine listeners > - Refactor the structure of our storage layer to support the many tables > inside the one logical partition > - Implement the way to configure available storages in node configs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20273) Breakdown for zone-based collocation
Kirill Gusakov created IGNITE-20273: --- Summary: Breakdown for zone-based collocation Key: IGNITE-20273 URL: https://issues.apache.org/jira/browse/IGNITE-20273 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov Based on the current design goals we need to prepare the tickets' breakdown for the first phase of collocation [design|https://github.com/gridgain/apache-ignite-3/blob/1abb67c8770179d5d3e295876b73893f3845a08e/modules/distribution-zones/tech-notes/zone-collocation.md#L1]: Under the first phase we want to implement the following steps: - Store the table partitions data inside the one zone partition - Change the RAFT servers and clients management to support the separate start of zone partitions and start of table state machine listeners - Refactor the structure of our storage layer to support the many tables inside the one logical partition - Implement the way to configure available storages in node configs -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-19992) Sql. Rework execution of 2-phase set operators
[
https://issues.apache.org/jira/browse/IGNITE-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov updated IGNITE-19992:
--
Fix Version/s: 3.0.0-beta2
> Sql. Rework execution of 2-phase set operators
> --
>
> Key: IGNITE-19992
> URL: https://issues.apache.org/jira/browse/IGNITE-19992
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 7h
> Remaining Estimate: 0h
>
> As of now, both {{IntersectNode}} and {{MinusNode}} use complex structures as
> result of MAP phase (see
> {{org.apache.ignite.internal.sql.engine.exec.rel.AbstractSetOpNode.Grouping#getOnMapper}};
> it emits rows cardinality of 2, where first column is entire group key, and
> second column is an array of counters). This prevents us from migrating sql
> runtime to BinaryTuple-based rows, because currently BinaryTuple does not
> support nested structures.
> Let's rework those node to inline group key and counters into plain row.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-19942) Mirror Configuration changes to Catalog.
[ https://issues.apache.org/jira/browse/IGNITE-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov resolved IGNITE-19942. --- Resolution: Won't Fix > Mirror Configuration changes to Catalog. > > > Key: IGNITE-19942 > URL: https://issues.apache.org/jira/browse/IGNITE-19942 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 0.5h > Remaining Estimate: 0h > > While moving database schema from Configuration to Catalog, there are ID > generators for a new database objects in Configuration and in Catalog. > Thus, we can't go smoothly mirroring Configuration events to Catalog because > of same things could have different ID. > Configuration ID generators are updated explicitly via changer object, so we > can create object in Catalog first, then pass its id to configuration changer. > Let's make this small refactoring to get Catalog in-sync with Configuration. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-19992) Sql. Rework execution of 2-phase set operators
[
https://issues.apache.org/jira/browse/IGNITE-19992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17758004#comment-17758004
]
Yury Gerzhedovich commented on IGNITE-19992:
[~mzhuravkov] LGTM
> Sql. Rework execution of 2-phase set operators
> --
>
> Key: IGNITE-19992
> URL: https://issues.apache.org/jira/browse/IGNITE-19992
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Konstantin Orlov
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Time Spent: 6h 50m
> Remaining Estimate: 0h
>
> As of now, both {{IntersectNode}} and {{MinusNode}} use complex structures as
> result of MAP phase (see
> {{org.apache.ignite.internal.sql.engine.exec.rel.AbstractSetOpNode.Grouping#getOnMapper}};
> it emits rows cardinality of 2, where first column is entire group key, and
> second column is an array of counters). This prevents us from migrating sql
> runtime to BinaryTuple-based rows, because currently BinaryTuple does not
> support nested structures.
> Let's rework those node to inline group key and counters into plain row.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin reassigned IGNITE-20272: Assignee: Vyacheslav Koptilin > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Uttsel >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually, the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need to remove some tests in this class now and later > remove this class when ticket > https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll, and the test passed successfully. This test can be removed > because it repeats the > testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. > # testDataNodesUpdatedOnZoneManagerStart is the happy path of the restart, we > already have such tests. Therefore, this test is not needed and can be > removed. > # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the > zones manager restart, the data nodes for the zone will not be updated to the > value corresponding to the logical topology from the vault, because > zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not > found an analog of this test. I think that when the DZM restart is updated > then we can update this test and move it to the more appropriate class. > h3. *Definition of done* > # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. > # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. > # TODOs with IGNITE-18564 are removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-20272: - Description: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually, the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need to remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll, and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of the restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found an analog of this test. I think that when the DZM restart is updated then we can update this test and move it to the more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TODOs with IGNITE-18564 are removed. was: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually, the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need to remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll, and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of the restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found an analog of this test. I think that when the DZM restart is updated then we can update this test and move it to the more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TODOs with IGNITE-18564 are removed. > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually, the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need to remove some tests in this class now and later > remove this class when ticket > https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll, and the test passed successfully. This test can be removed > because it repeats the >
[jira] [Updated] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-20272: - Description: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually, the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need to remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll, and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of the restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found an analog of this test. I think that when the DZM restart is updated then we can update this test and move it to the more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TODOs with IGNITE-18564 are removed. was: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 will be implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found a analogue of this test. I think that when the DZM restart is updated then we can update this test and move it to more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TODOs with IGNITE-18564 are removed. > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually, the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need to remove some tests in this class now and later > remove this class when ticket > https://issues.apache.org/jira/browse/IGNITE-19955 is implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll, and the test passed successfully. This test can be removed > because it repeats the >
[jira] [Updated] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Uttsel updated IGNITE-20272: --- Issue Type: Improvement (was: Bug) > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Improvement >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need remove some tests in this class now and later remove > this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 > will be implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll and the test passed successfully. This test can be removed > because it repeats the > testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. > # testDataNodesUpdatedOnZoneManagerStart is the happy path of restart, we > already have such tests. Therefore, this test is not needed and can be > removed. > # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the > zones manager restart, the data nodes for the zone will not be updated to the > value corresponding to the logical topology from the vault, because > zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not > found a analogue of this test. I think that when the DZM restart is updated > then we can update this test and move it to more appropriate class. > h3. *Definition of done* > # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. > # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. > # TODOs with IGNITE-18564 are removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
[ https://issues.apache.org/jira/browse/IGNITE-20272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Uttsel updated IGNITE-20272: --- Description: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 will be implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found a analogue of this test. I think that when the DZM restart is updated then we can update this test and move it to more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TODOs with IGNITE-18564 are removed. was: h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 will be implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found a analogue of this test. I think that when the DZM restart is updated then we can update this test and move it to more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TOTOs with IGNITE-18564 are removed. > Clean up of DistributionZoneManagerWatchListenerTest > > > Key: IGNITE-20272 > URL: https://issues.apache.org/jira/browse/IGNITE-20272 > Project: Ignite > Issue Type: Bug >Reporter: Sergey Uttsel >Priority: Major > Labels: ignite-3, tech-debt > > h3. *Motivation* > Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because > it is not actual anymore. But actually the > DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is > not actual. So we need remove some tests in this class now and later remove > this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 > will be implemented. > DistributionZoneManagerWatchListenerTest has three tests: > # The testStaleWatchEvent is disabled. It fails on an invoke into a > metastorage in which the logical topology and its version are updated. It > fails because the condition of the invoke was incorrect after some changes in > the code. But it is not necessary to use a condition there, I replaced it > with ms.putAll and the test passed successfully. This test can be removed > because it repeats the >
[jira] [Created] (IGNITE-20272) Clean up of DistributionZoneManagerWatchListenerTest
Sergey Uttsel created IGNITE-20272: -- Summary: Clean up of DistributionZoneManagerWatchListenerTest Key: IGNITE-20272 URL: https://issues.apache.org/jira/browse/IGNITE-20272 Project: Ignite Issue Type: Bug Reporter: Sergey Uttsel h3. *Motivation* Ticket https://issues.apache.org/jira/browse/IGNITE-18564 was closed because it is not actual anymore. But actually the DistributionZoneManagerWatchListenerTest that was mentioned in this ticket is not actual. So we need remove some tests in this class now and later remove this class when ticket https://issues.apache.org/jira/browse/IGNITE-19955 will be implemented. DistributionZoneManagerWatchListenerTest has three tests: # The testStaleWatchEvent is disabled. It fails on an invoke into a metastorage in which the logical topology and its version are updated. It fails because the condition of the invoke was incorrect after some changes in the code. But it is not necessary to use a condition there, I replaced it with ms.putAll and the test passed successfully. This test can be removed because it repeats the testScaleUpDidNotChangeDataNodesWhenTriggerKeyWasConcurrentlyChanged test. # testDataNodesUpdatedOnZoneManagerStart is the happy path of restart, we already have such tests. Therefore, this test is not needed and can be removed. # testStaleVaultRevisionOnZoneManagerStart This test simulates that on the zones manager restart, the data nodes for the zone will not be updated to the value corresponding to the logical topology from the vault, because zonesChangeTriggerKey > metaStorageManager.appliedRevision(). I have not found a analogue of this test. I think that when the DZM restart is updated then we can update this test and move it to more appropriate class. h3. *Definition of done* # testStaleWatchEvent and testDataNodesUpdatedOnZoneManagerStart are removed. # testStaleVaultRevisionOnZoneManagerStart marked by IGNITE-19955. # TOTOs with IGNITE-18564 are removed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20271) Error in CLI on SQL repl command
[
https://issues.apache.org/jira/browse/IGNITE-20271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dmitry Baranov reassigned IGNITE-20271:
---
Assignee: Dmitry Baranov
> Error in CLI on SQL repl command
>
>
> Key: IGNITE-20271
> URL: https://issues.apache.org/jira/browse/IGNITE-20271
> Project: Ignite
> Issue Type: Bug
> Components: cli
>Affects Versions: 3.0.0-beta1
>Reporter: Dmitry Baranov
>Assignee: Dmitry Baranov
>Priority: Major
>
> Init ignite 3 cluster, than
> _[defaultNode]> sql_
> *Unknown error*
> {*}Expected{*}:
> enter in sql repl mode
> {*}Actual{*}:
> Cannot invoke
> "org.apache.ignite.internal.cli.core.repl.EventListeningActivationPoint.subscribe()"
> because the return value of
> "org.apache.ignite.internal.cli.core.repl.Repl.getEventListeningActivationPoint()"
> is null
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20271) Error in CLI on SQL repl command
Dmitry Baranov created IGNITE-20271:
---
Summary: Error in CLI on SQL repl command
Key: IGNITE-20271
URL: https://issues.apache.org/jira/browse/IGNITE-20271
Project: Ignite
Issue Type: Bug
Components: cli
Affects Versions: 3.0.0-beta1
Reporter: Dmitry Baranov
Init ignite 3 cluster, than
_[defaultNode]> sql_
*Unknown error*
{*}Expected{*}:
enter in sql repl mode
{*}Actual{*}:
Cannot invoke
"org.apache.ignite.internal.cli.core.repl.EventListeningActivationPoint.subscribe()"
because the return value of
"org.apache.ignite.internal.cli.core.repl.Repl.getEventListeningActivationPoint()"
is null
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-19951) Get rid of @NotNull annotation
[
https://issues.apache.org/jira/browse/IGNITE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757974#comment-17757974
]
Evgeny Stanilovsky commented on IGNITE-19951:
-
[~jooger] merged into main, thanks for your effort!
> Get rid of @NotNull annotation
> --
>
> Key: IGNITE-19951
> URL: https://issues.apache.org/jira/browse/IGNITE-19951
> Project: Ignite
> Issue Type: Improvement
>Reporter: Yury Gerzhedovich
>Assignee: Yury Gerzhedovich
>Priority: Major
> Labels: ignite-3
> Time Spent: 1h 40m
> Remaining Estimate: 0h
>
> Despite our code guide style prohibit to use {{@NotNull}} annotation [1] we
> have many usage of the annotation in code.
> Even more, we have example [2] with {{@NotNull}} in code style guide. it
> could breaks mind.
> Let's do the next:
> 1) fix example in code style guide
> 2) get rid {{@NotNull}} in code base.
> 3) add automation checksyle check to avoid appearance new usage of the
> annotation.
> [1] [ -
> https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2.1Commonannotatingrules|https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-2.1Commonannotatingrules]
> [2] -
> [https://cwiki.apache.org/confluence/display/IGNITE/Java+Code+Style+Guide#JavaCodeStyleGuide-4.1Allasyncmethodsshouldhaveasyncsuffix]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20104) Sql. Unclear weird messages from parser
[
https://issues.apache.org/jira/browse/IGNITE-20104?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757971#comment-17757971
]
Yury Gerzhedovich commented on IGNITE-20104:
[~korlov] LGTM
> Sql. Unclear weird messages from parser
>
>
> Key: IGNITE-20104
> URL: https://issues.apache.org/jira/browse/IGNITE-20104
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Yury Gerzhedovich
>Assignee: Konstantin Orlov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> We have issue with unclear messages from parser, when user can't understand
> what is went wrong and where is error.
> As example of such errors:
> 1. {code:java}
> MERGE INTO TBL_ALL_COLUMNS_SQL(key, str) VALUES(?, ?)
> {code}
> {noformat}
> TraceId:5e59762a-2252-4ac0-bc1f-8ec53112b858 Failed to parse query:
> Encountered "" at line 1, column 35.
> Was expecting one of:
> {noformat}
> 2. {code:java}
> ALTER ZONE TEST_ZONE WITH REPLICAS = 2
> {code}
> {noformat}
> IGN-SQL-3 TraceId:5be71235-eb40-424f-b3f7-0488fd0076ca IGN-SQL-3
> TraceId:5be71235-eb40-424f-b3f7-0488fd0076ca Failed to parse query:
> Encountered "" at line 1, column 24.
> Was expecting one of:
> {noformat}
> Parser can show all alternatives related to syntax to user have ability
> understand is mistake
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20270) Improve RocksDB iterator performance
[
https://issues.apache.org/jira/browse/IGNITE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-20270:
-
Description:
When running benchmarks that insert around 700 rows into Ignite 3, I've
discovered that a lot of time is spent in {{iterator.seek}} calls:
!screenshot-3.png!
I've tried multiple strategies to improve this behavior:
# Introduce Bloom Filters:
https://github.com/facebook/rocksdb/wiki/Prefix-Seek#configure-prefix-bloom-filter
# Use Tailing Iterators:
https://github.com/facebook/rocksdb/wiki/Tailing-Iterator
# Use adaptive prefix mode:
https://github.com/facebook/rocksdb/wiki/Prefix-Seek#adaptive-prefix-mode
This improved some scenarios up to 6 times:
!screenshot-2.png!
> Improve RocksDB iterator performance
>
>
> Key: IGNITE-20270
> URL: https://issues.apache.org/jira/browse/IGNITE-20270
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
> Attachments: screenshot-2.png, screenshot-3.png
>
>
> When running benchmarks that insert around 700 rows into Ignite 3, I've
> discovered that a lot of time is spent in {{iterator.seek}} calls:
> !screenshot-3.png!
> I've tried multiple strategies to improve this behavior:
> # Introduce Bloom Filters:
> https://github.com/facebook/rocksdb/wiki/Prefix-Seek#configure-prefix-bloom-filter
> # Use Tailing Iterators:
> https://github.com/facebook/rocksdb/wiki/Tailing-Iterator
> # Use adaptive prefix mode:
> https://github.com/facebook/rocksdb/wiki/Prefix-Seek#adaptive-prefix-mode
> This improved some scenarios up to 6 times:
> !screenshot-2.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20270) Improve RocksDB iterator performance
[ https://issues.apache.org/jira/browse/IGNITE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Polovtcev updated IGNITE-20270: - Attachment: screenshot-3.png > Improve RocksDB iterator performance > > > Key: IGNITE-20270 > URL: https://issues.apache.org/jira/browse/IGNITE-20270 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Polovtcev >Assignee: Aleksandr Polovtcev >Priority: Major > Labels: ignite-3 > Attachments: screenshot-2.png, screenshot-3.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20270) Improve RocksDB iterator performance
[
https://issues.apache.org/jira/browse/IGNITE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-20270:
-
Attachment: (was: screenshot-1.png)
> Improve RocksDB iterator performance
>
>
> Key: IGNITE-20270
> URL: https://issues.apache.org/jira/browse/IGNITE-20270
> Project: Ignite
> Issue Type: Improvement
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
> Labels: ignite-3
> Attachments: screenshot-2.png, screenshot-3.png
>
>
> When running benchmarks that insert around 700 rows into Ignite 3, I've
> discovered that a lot of time is spent in {{iterator.seek}} calls:
> !screenshot-3.png!
> I've tried multiple strategies to improve this behavior:
> # Introduce Bloom Filters:
> https://github.com/facebook/rocksdb/wiki/Prefix-Seek#configure-prefix-bloom-filter
> # Use Tailing Iterators:
> https://github.com/facebook/rocksdb/wiki/Tailing-Iterator
> # Use adaptive prefix mode:
> https://github.com/facebook/rocksdb/wiki/Prefix-Seek#adaptive-prefix-mode
> This improved some scenarios up to 6 times:
> !screenshot-2.png!
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20270) Improve RocksDB iterator performance
[ https://issues.apache.org/jira/browse/IGNITE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Polovtcev updated IGNITE-20270: - Attachment: screenshot-2.png > Improve RocksDB iterator performance > > > Key: IGNITE-20270 > URL: https://issues.apache.org/jira/browse/IGNITE-20270 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Polovtcev >Assignee: Aleksandr Polovtcev >Priority: Major > Labels: ignite-3 > Attachments: screenshot-1.png, screenshot-2.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20270) Improve RocksDB iterator performance
[ https://issues.apache.org/jira/browse/IGNITE-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr Polovtcev updated IGNITE-20270: - Attachment: screenshot-1.png > Improve RocksDB iterator performance > > > Key: IGNITE-20270 > URL: https://issues.apache.org/jira/browse/IGNITE-20270 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksandr Polovtcev >Assignee: Aleksandr Polovtcev >Priority: Major > Labels: ignite-3 > Attachments: screenshot-1.png > > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20270) Improve RocksDB iterator performance
Aleksandr Polovtcev created IGNITE-20270: Summary: Improve RocksDB iterator performance Key: IGNITE-20270 URL: https://issues.apache.org/jira/browse/IGNITE-20270 Project: Ignite Issue Type: Improvement Reporter: Aleksandr Polovtcev Assignee: Aleksandr Polovtcev -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20269) Check and slightly refactor the validation of tables in the catalog
Kirill Tkalenko created IGNITE-20269: Summary: Check and slightly refactor the validation of tables in the catalog Key: IGNITE-20269 URL: https://issues.apache.org/jira/browse/IGNITE-20269 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 3.0.0-beta2 We need to make sure that all validation for indexes is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20269) Check and slightly refactor the validation of tables in the catalog
[ https://issues.apache.org/jira/browse/IGNITE-20269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-20269: - Description: We need to make sure that all validation for tables is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. was: We need to make sure that all validation for indexes is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. > Check and slightly refactor the validation of tables in the catalog > --- > > Key: IGNITE-20269 > URL: https://issues.apache.org/jira/browse/IGNITE-20269 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > We need to make sure that all validation for tables is present in the catalog. > Slightly refactor the code related to validation, for example, move > validation methods to *CatalogParamsValidationUtils* and tests related to it > in *CatalogManagerValidationTest*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20245) MVCC test/suites removal
[ https://issues.apache.org/jira/browse/IGNITE-20245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20245: -- Ignite Flags: (was: Docs Required,Release Notes Required) > MVCC test/suites removal > > > Key: IGNITE-20245 > URL: https://issues.apache.org/jira/browse/IGNITE-20245 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 4h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20245) MVCC test/suites removal
[ https://issues.apache.org/jira/browse/IGNITE-20245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20245: -- Fix Version/s: 2.16 > MVCC test/suites removal > > > Key: IGNITE-20245 > URL: https://issues.apache.org/jira/browse/IGNITE-20245 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 4h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20265) Check and slightly refactor the validation of indexes in the catalog
[ https://issues.apache.org/jira/browse/IGNITE-20265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-20265: - Summary: Check and slightly refactor the validation of indexes in the catalog (was: Check and slightly refactor the validation of tables and indexes in the catalog) > Check and slightly refactor the validation of indexes in the catalog > > > Key: IGNITE-20265 > URL: https://issues.apache.org/jira/browse/IGNITE-20265 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > We need to make sure that all validation for tables and indexes is present in > the catalog. > Slightly refactor the code related to validation, for example, move > validation methods to *CatalogParamsValidationUtils* and tests related to it > in *CatalogManagerValidationTest*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20265) Check and slightly refactor the validation of indexes in the catalog
[ https://issues.apache.org/jira/browse/IGNITE-20265?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-20265: - Description: We need to make sure that all validation for indexes is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. was: We need to make sure that all validation for tables and indexes is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. > Check and slightly refactor the validation of indexes in the catalog > > > Key: IGNITE-20265 > URL: https://issues.apache.org/jira/browse/IGNITE-20265 > Project: Ignite > Issue Type: Improvement >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > We need to make sure that all validation for indexes is present in the > catalog. > Slightly refactor the code related to validation, for example, move > validation methods to *CatalogParamsValidationUtils* and tests related to it > in *CatalogManagerValidationTest*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19824) Implicit RO should be used in implicit single gets
[
https://issues.apache.org/jira/browse/IGNITE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin reassigned IGNITE-19824:
Assignee: Vladislav Pyatkov (was: Vyacheslav Koptilin)
> Implicit RO should be used in implicit single gets
> --
>
> Key: IGNITE-19824
> URL: https://issues.apache.org/jira/browse/IGNITE-19824
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Currently, all implicit read operations start RW transactions, thus it's
> possible to catch "Failed to acquire a lock due to a conflict" exception.
> Generally speaking, given issue should be resolved by substituting RW with RO
> for all implicit read transactions, however such approach will decrease
> linearizability so it's required to verify it with product management. It's
> still however possible to have special case RO for implicit single-key get
> operation that will set readTimestamp on primary replica instead of
> transaction coordinator and thus provide cluster-wide linearizability even
> for RO transactions (only for single-key implicit get operations). Within
> this ticket, such special RO transactions should be introduced along with
> their usage switch for single-get implicit reads.
> h3. Definition of Done
> * Implicit single-get operations use special RO transactions that provide
> cluster-wide linearizability and thus do not throw "Failed to acquire a lock
> due to a conflict" exception.
> * ItAbstractDataStreamerTest#testAutoFlushByTimer adjusted: catch block
> removed.
> h3. Implementation Notes
> 1. Basically, what we need to do here is to start RO transaction instead of
> RW one in case of single-key implicit get, thus we should add
> {code:java}
> if (tx == null) {
> tx = txManager.begin(true);
> }{code}
> right in front of
> {code:java}
> return enlistInTx({code}
> Please pay attention, that we wan't to start special case RO transaction that
> should go to primary and only primary, so it's not valit to put
> aforementioned tx = txManager.begin(true); at the very beginning of the
> method, because in that case balancer may return non-primary through
> evaluateReadOnlyRecipientNode. Corresponging comment should be added.
> 2. Such specifal case RO transcation doesn't require readTimestamp
> calcualtion on tx.start for the evaluation point of view, however it still
> required it for lowWatermark managerment:
> {code:java}
> readOnlyTxFutureById.compute(new TxIdAndTimestamp(readTimestamp, txId),
> (txIdAndTimestamp, readOnlyTxFuture) -> {
> assert readOnlyTxFuture == null : "previous transaction has not completed
> yet: " + txIdAndTimestamp;
> if (lowWatermark != null && readTimestamp.compareTo(lowWatermark) <= 0) {
> throw new IgniteInternalException(
> TX_READ_ONLY_TOO_OLD_ERR,
> "Timestamp read-only transaction must be greater than the low
> watermark: [txTimestamp={}, lowWatermark={}]",
> readTimestamp, lowWatermark
> );
> }
> return new CompletableFuture<>();
> }); {code}
> So, seems that it worth to leave readTimestamp generatoin at it's current
> place.
> 3. And again in order to have cluster-wide linearizability it's requried to
> use primaryReplica now as readTimestamp instead of the one proposed in
> readOnlyReplicaRequest. Basically that means substitution of
> {code:java}
> HybridTimestamp readTimestamp = request.readTimestamp(); {code}
> with
> {code:java}
> HybridTimestamp readTimestamp;
> if (request.requestType() == RequestType.RO_GET && request.implicit()) {
> readTimestamp = hybridClock.now();
> } else {
> readTimestamp = request.readTimestamp();
> } {code}
> along with
> {code:java}
> CompletableFuture safeReadFuture = isPrimaryInTimestamp(isPrimary,
> readTimestamp) ? completedFuture(null)
> : safeTime.waitFor(readTimestamp); {code}
> in PartitionReplicaListnener. That on its part required adding implicit() to
> ReadOnlySingleRowReplicaRequest that should be properly set on the client
> side.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-19824) Implicit RO should be used in implicit single gets
[
https://issues.apache.org/jira/browse/IGNITE-19824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin reassigned IGNITE-19824:
Assignee: Vyacheslav Koptilin (was: Vladislav Pyatkov)
> Implicit RO should be used in implicit single gets
> --
>
> Key: IGNITE-19824
> URL: https://issues.apache.org/jira/browse/IGNITE-19824
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vyacheslav Koptilin
>Priority: Major
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> h3. Motivation
> Currently, all implicit read operations start RW transactions, thus it's
> possible to catch "Failed to acquire a lock due to a conflict" exception.
> Generally speaking, given issue should be resolved by substituting RW with RO
> for all implicit read transactions, however such approach will decrease
> linearizability so it's required to verify it with product management. It's
> still however possible to have special case RO for implicit single-key get
> operation that will set readTimestamp on primary replica instead of
> transaction coordinator and thus provide cluster-wide linearizability even
> for RO transactions (only for single-key implicit get operations). Within
> this ticket, such special RO transactions should be introduced along with
> their usage switch for single-get implicit reads.
> h3. Definition of Done
> * Implicit single-get operations use special RO transactions that provide
> cluster-wide linearizability and thus do not throw "Failed to acquire a lock
> due to a conflict" exception.
> * ItAbstractDataStreamerTest#testAutoFlushByTimer adjusted: catch block
> removed.
> h3. Implementation Notes
> 1. Basically, what we need to do here is to start RO transaction instead of
> RW one in case of single-key implicit get, thus we should add
> {code:java}
> if (tx == null) {
> tx = txManager.begin(true);
> }{code}
> right in front of
> {code:java}
> return enlistInTx({code}
> Please pay attention, that we wan't to start special case RO transaction that
> should go to primary and only primary, so it's not valit to put
> aforementioned tx = txManager.begin(true); at the very beginning of the
> method, because in that case balancer may return non-primary through
> evaluateReadOnlyRecipientNode. Corresponging comment should be added.
> 2. Such specifal case RO transcation doesn't require readTimestamp
> calcualtion on tx.start for the evaluation point of view, however it still
> required it for lowWatermark managerment:
> {code:java}
> readOnlyTxFutureById.compute(new TxIdAndTimestamp(readTimestamp, txId),
> (txIdAndTimestamp, readOnlyTxFuture) -> {
> assert readOnlyTxFuture == null : "previous transaction has not completed
> yet: " + txIdAndTimestamp;
> if (lowWatermark != null && readTimestamp.compareTo(lowWatermark) <= 0) {
> throw new IgniteInternalException(
> TX_READ_ONLY_TOO_OLD_ERR,
> "Timestamp read-only transaction must be greater than the low
> watermark: [txTimestamp={}, lowWatermark={}]",
> readTimestamp, lowWatermark
> );
> }
> return new CompletableFuture<>();
> }); {code}
> So, seems that it worth to leave readTimestamp generatoin at it's current
> place.
> 3. And again in order to have cluster-wide linearizability it's requried to
> use primaryReplica now as readTimestamp instead of the one proposed in
> readOnlyReplicaRequest. Basically that means substitution of
> {code:java}
> HybridTimestamp readTimestamp = request.readTimestamp(); {code}
> with
> {code:java}
> HybridTimestamp readTimestamp;
> if (request.requestType() == RequestType.RO_GET && request.implicit()) {
> readTimestamp = hybridClock.now();
> } else {
> readTimestamp = request.readTimestamp();
> } {code}
> along with
> {code:java}
> CompletableFuture safeReadFuture = isPrimaryInTimestamp(isPrimary,
> readTimestamp) ? completedFuture(null)
> : safeTime.waitFor(readTimestamp); {code}
> in PartitionReplicaListnener. That on its part required adding implicit() to
> ReadOnlySingleRowReplicaRequest that should be properly set on the client
> side.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20005) Move commitTimestamp generation to the txn coordinator side.
[
https://issues.apache.org/jira/browse/IGNITE-20005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-20005:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Move commitTimestamp generation to the txn coordinator side.
>
>
> Key: IGNITE-20005
> URL: https://issues.apache.org/jira/browse/IGNITE-20005
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction, transaction3_recovery
>
> h3. Motivation
> Please check the context in
> https://issues.apache.org/jira/browse/IGNITE-20033 and
> https://issues.apache.org/jira/browse/IGNITE-20034
> After implemention writeIntentResolution coordinator path it'll be nessessary
> to move commitTimestamp generation to the txn coordinator side.
> h3. Definition of Done
> * Easy-breezy, calculate commitTimestamp as coordinator.now().
> h3. Implementation Notes
> Nothing to update on the "client" side, we already have everything we need.
> {code:java}
> public CompletableFuture finish() {
> ...
> HybridTimestamp commitTimestamp = commit ? clock.now() : null;
> TxFinishReplicaRequest req = FACTORY.txFinishReplicaRequest()
> ...
> .commitTimestampLong(hybridTimestampToLong(commitTimestamp)) // 2
>...
> } {code}
> On the "server" side, we will adjust and simplify code in
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener#finishTransaction
> {code:java}
> private CompletableFuture finishTransaction(List
> aggregatedGroupIds, UUID txId, boolean commit) {
> // TODO: IGNITE-17261 Timestamp from request is not using until the issue
> has not been fixed (request.commitTimestamp())
> var fut = new CompletableFuture();
> txTimestampUpdateMap.put(txId, fut);
> HybridTimestamp currentTimestamp = hybridClock.now();
> HybridTimestamp commitTimestamp = commit ? currentTimestamp : null; {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20233) jdbc: Propagate observable timestamp to sql-engine.
[
https://issues.apache.org/jira/browse/IGNITE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20233:
--
Description:
We need to pass the observable timestamp from jdbc client to the sql engine.
This must be done after the completion of IGNITE-19898.
Investigate possibility of migrating from JDBC_* client messages to SQL_*.
was:
We need to pass the observable timestamp from jdbc client to the sql engine.
This must be done after the completion of IGNITE-19898.
Investigate possibility of migrating from {{JDBC_*}} client messages to
{{SQL_*}}.
> jdbc: Propagate observable timestamp to sql-engine.
> ---
>
> Key: IGNITE-20233
> URL: https://issues.apache.org/jira/browse/IGNITE-20233
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> We need to pass the observable timestamp from jdbc client to the sql engine.
> This must be done after the completion of IGNITE-19898.
> Investigate possibility of migrating from JDBC_* client messages to SQL_*.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20233) jdbc: Propagate observable timestamp to sql-engine.
[
https://issues.apache.org/jira/browse/IGNITE-20233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20233:
--
Description:
We need to pass the observable timestamp from jdbc client to the sql engine.
This must be done after the completion of IGNITE-19898.
Investigate possibility of migrating from {{JDBC_*}} client messages to
{{SQL_*}}.
was:
We need to pass the observable timestamp from jdbc client to the sql engine.
This must be done after the completion of IGNITE-19898.
> jdbc: Propagate observable timestamp to sql-engine.
> ---
>
> Key: IGNITE-20233
> URL: https://issues.apache.org/jira/browse/IGNITE-20233
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> We need to pass the observable timestamp from jdbc client to the sql engine.
> This must be done after the completion of IGNITE-19898.
> Investigate possibility of migrating from {{JDBC_*}} client messages to
> {{SQL_*}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20017) Introduce new command to catalog to create system view
[ https://issues.apache.org/jira/browse/IGNITE-20017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-20017: --- Description: To ensure cluster-wide visibility of the view and seamless integration with sql engine, all system views must be registered to the catalog. Let's extend catalog with new set of params|descriptor to support system view integration. Should be added org.apache.ignite.internal.catalog.CatalogManagerImpl#createView was: To ensure cluster-wide visibility of the view and seamless integration with sql engine, all system views must be registered to the catalog. Let's extend catalog with new set of params|descriptor to support system view integration > Introduce new command to catalog to create system view > -- > > Key: IGNITE-20017 > URL: https://issues.apache.org/jira/browse/IGNITE-20017 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > To ensure cluster-wide visibility of the view and seamless integration with > sql engine, all system views must be registered to the catalog. > Let's extend catalog with new set of params|descriptor to support system view > integration. > Should be added > org.apache.ignite.internal.catalog.CatalogManagerImpl#createView -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20164) Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce aggregates.
[
https://issues.apache.org/jira/browse/IGNITE-20164?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757886#comment-17757886
]
Maksim Zhuravkov commented on IGNITE-20164:
---
[~korlov] [~zstan] Could you please review this ticket?
> Sql. Incorrect propagation of RelCollation trait for Sort-based map/reduce
> aggregates.
> --
>
> Key: IGNITE-20164
> URL: https://issues.apache.org/jira/browse/IGNITE-20164
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Maksim Zhuravkov
>Assignee: Maksim Zhuravkov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> RelCollation propagation does not take into account remapping of group keys
> between MAP/REDUCE phases, hence causes errors in queries that are expected
> to use sort-based MAP/REDUCE - RelCollation uses the same keys on both
> phases. Example:
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[CAST($0):BIGINT NOT NULL], EXPR$1=[$1]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[$SUM0($1)],
> EXPR$1=[COUNT($0)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]), < HERE
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{1}], EXPR$0=[COUNT($0)],
> collation=[[1]]),
> IgniteSort(sort0=[$1], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]],
> requiredColumns=[{1, 2}]),
> {code}
> Error:
> {code:java}
> Caused by: java.lang.ClassCastException: class java.util.ArrayList cannot be
> cast to class java.lang.Comparable (java.util.ArrayList and
> java.lang.Comparable are in module java.base of loader 'bootstrap')
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.compare(ExpressionFactoryImpl.java:247)
> at
> org.apache.ignite.internal.sql.engine.exec.exp.ExpressionFactoryImpl.lambda$comparator$0(ExpressionFactoryImpl.java:178)
> at
> java.base/java.util.Map$Entry.lambda$comparingByKey$6d558cbf$1(Map.java:539)
> at
> java.base/java.util.PriorityQueue.siftUpUsingComparator(PriorityQueue.java:675)
> at java.base/java.util.PriorityQueue.siftUp(PriorityQueue.java:652)
> at java.base/java.util.PriorityQueue.offer(PriorityQueue.java:345)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.pushOrdered(Inbox.java:235)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.push(Inbox.java:188)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.Inbox.onBatchReceived(Inbox.java:168)
> at
> org.apache.ignite.internal.sql.engine.exec.ExchangeServiceImpl.onMessage(ExchangeServiceImpl.java:184)
> ... 7 more
> {code}
> The query below works because position of column b does not change after MAP
> phase.
> {code:java}
> String[] rules = {
> "MapReduceHashAggregateConverterRule",
> "ColocatedHashAggregateConverterRule",
> "ColocatedSortAggregateConverterRule"
> };
> sql("CREATE TABLE testMe40 (a INTEGER, b INTEGER);");
> sql("INSERT INTO testMe40 VALUES (11, 2), (12, 2), (12, 3)");
> assertQuery("SELECT COUNT(a), COUNT(DISTINCT(b)) FROM testMe40")
> .disableRules(rules)
> .returns(3L, 2L)
> .check();
> {code}
> Plan:
> {code:java}
> IgniteProject(EXPR$0=[$0], EXPR$1=[CAST($1):BIGINT NOT NULL]),
> IgniteReduceSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteMapSortAggregate(group=[{}], EXPR$0=[COUNT($0)],
> EXPR$1=[$SUM0($1)], collation=[[]]),
> IgniteReduceSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteExchange(distribution=[single]),
> IgniteMapSortAggregate(group=[{0}], EXPR$1=[COUNT($1)],
> collation=[[0]]),
> IgniteSort(sort0=[$0], dir0=[ASC]),
> IgniteTableScan(table=[[PUBLIC, TESTME40]], projects=[[$t1,
> $t0]], requiredColumns=[{1, 2}]),
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20268) Deduplicate code in V1Encoder/V1Decoder
Roman Puchkovskiy created IGNITE-20268:
--
Summary: Deduplicate code in V1Encoder/V1Decoder
Key: IGNITE-20268
URL: https://issues.apache.org/jira/browse/IGNITE-20268
Project: Ignite
Issue Type: Improvement
Reporter: Roman Puchkovskiy
Fix For: 3.0.0-beta2
V1Encoder and V1Decoder use code very similar to that that in {{VarIntUtils}}.
The latter should be reused.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-17684) Investigate of using BinaryTuple instead of array of objects in SQL execution
[ https://issues.apache.org/jira/browse/IGNITE-17684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich resolved IGNITE-17684. Resolution: Fixed > Investigate of using BinaryTuple instead of array of objects in SQL execution > - > > Key: IGNITE-17684 > URL: https://issues.apache.org/jira/browse/IGNITE-17684 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: ignite-3, perfomance, tech-debt > > Right now internaly we use array of objects to represent row in SQL instead > of BinaryTuple and do unnecessary convertation. > Let's investigate possibility migration from array of objects to direct usage > of BinaryTuple in execution tree. There are possible issue for some type > execution, like a twophase aggregates and potentialy we should use different > representation of row for different parts. > Start points are: > org.apache.ignite.internal.sql.engine.exec.ArrayRowHandler > org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl#toRow > As result of the task - list of issues which need to be resolved to implement > reuse BinaryTuple -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20267) Infinite loop of SockeException
[
https://issues.apache.org/jira/browse/IGNITE-20267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757852#comment-17757852
]
Sebastian Fabisz commented on IGNITE-20267:
---
I'm unable to upload files so I added trace logs in comment
> Infinite loop of SockeException
> ---
>
> Key: IGNITE-20267
> URL: https://issues.apache.org/jira/browse/IGNITE-20267
> Project: Ignite
> Issue Type: Bug
> Components: thin client
>Affects Versions: 2.15
>Reporter: Sebastian Fabisz
>Priority: Major
>
> Some of our ignite instances are experiencing infinite loop of same error:
> {{ERROR 2023-07-27 08:26:44,876
> [grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]
> o.a.i.s.c.t.TcpCommunicationSpi traceId="" spanId="" - Failed to process
> selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
> [super=AbstractNioClientWorker [idx=2, bytesRcvd=21528, bytesSent=15345,
> bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker
> [name=grid-nio-worker-tcp-comm-2, igniteInstanceName=TcpCommunicationSpi,
> finished=false, heartbeatTs=1690442803865, hashCode=2102759141,
> interrupted=false,
> runner=grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]]],
> writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
> readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
> inRecovery=null, outRecovery=null, closeSocket=true,
> outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
> super=GridNioSessionImpl [locAddr=\{removed}, rmtAddr=\{removed},
> createTime=1690249023154, closeTime=0, bytesSent=18, bytesRcvd=3,
> bytesSent0=0, bytesRcvd0=0, sndSchedTime=1690442567813,
> lastSndTime=1690249023154, lastRcvTime=1690442567813, readsPaused=false,
> filterChain=FilterChain[filters=[GridNioCodecFilter
> [parser=o.a.i.i.util.nio.GridDirectParser@1fff7116, directMode=true],
> GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
> java.net.SocketException: Connection reset at
> java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
> at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:411)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1351)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeys(GridNioServer.java:2575)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2271)
> at
> org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125) at
> java.base/java.lang.Thread.run(Thread.java:833)}}
> Each error contains same message except of heartbeatTs field.
> This error repeats approximately every second. Not all of Ignite instances
> are affected. We have figured out that problem is caused by Nessus security
> scanner. It walks over all boxes and runs some security checks. It looks like
> one of security checks (which can be http request) causes Ignite to fall into
> infinite loop of errors. We think that nessus opens a connection to Ignite,
> then connection is closed by nessus, but Iginite won't kill the socket.
>
> We have already updated Ignite to latest version.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-20267) Infinite loop of SockeException
[
https://issues.apache.org/jira/browse/IGNITE-20267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17757851#comment-17757851
]
Sebastian Fabisz commented on IGNITE-20267:
---
{{ERROR 2023-08-02 14:19:53,500
[grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%]
o.a.i.s.c.t.TcpCommunicationSpi traceId="" spanId="" - Failed to process
selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
[super=AbstractNioClientWorker [idx=0, bytesRcvd=8197316, bytesSent=105099,
bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker
[name=grid-nio-worker-tcp-comm-0, igniteInstanceName=TcpCommunicationSpi,
finished=false, heartbeatTs=1690982392494, hashCode=744530904,
interrupted=false,
runner=grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=null,
outRecovery=null, closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=\{removed}, rmtAddr=\{removed},
createTime=1690978654235, closeTime=0, bytesSent=18, bytesRcvd=0, bytesSent0=0,
bytesRcvd0=0, sndSchedTime=1690982259458, lastSndTime=1690978654235,
lastRcvTime=1690982259458, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@707a270e, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]}}
{{java.net.SocketException: Connection reset}}
{{ at
java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)}}
{{ at
java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:411)}}
{{ at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1351)}}
{{ at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeys(GridNioServer.java:2575)}}
{{ at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2271)}}
{{ at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)}}
{{ at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)}}
{{ at java.base/java.lang.Thread.run(Thread.java:833)}}
{{TRACE 2023-08-02 14:19:53,500
[grid-nio-worker-tcp-comm-0-#23%TcpCommunicationSpi%]
o.a.i.s.c.t.TcpCommunicationSpi traceId="" spanId="" - Processing keys in
client worker: 1}}
{{DEBUG 2023-08-02 14:19:53,563 [grid-timeout-worker-#22%GRID_NAME_REPLACED%]
o.a.i.i.p.t.GridTimeoutProcessor traceId="" spanId="" - Timeout has occurred
[obj=CancelableTask [id=82140b0b981-8bdf698a-49e0-48e8-a6db-4901eb305943,
endTime=1690982393559, period=5000, cancel=false,
task=org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager$BackupCleaner@327e8671],
process=true]}}
{{DEBUG 2023-08-02 14:19:53,563 [grid-timeout-worker-#22%GRID_NAME_REPLACED%]
o.a.i.i.p.t.GridTimeoutProcessor traceId="" spanId="" - Timeout has occurred
[obj=CancelableTask [id=92140b0b981-8bdf698a-49e0-48e8-a6db-4901eb305943,
endTime=1690982393559, period=5000, cancel=false,
task=org.apache.ignite.internal.processors.cache.query.continuous.CacheContinuousQueryManager$BackupCleaner@208b506],
process=true]}}
{{INFO 2023-08-02 14:19:53,824 [Thread-82414] o.a.i.s.d.t.TcpDiscoverySpi
traceId="" spanId="" - Pinging node: bf79c9d1-746b-4454-bad2-a74ce36fe6c1}}
{{INFO 2023-08-02 14:19:53,827 [Thread-82414] o.a.i.s.d.t.TcpDiscoverySpi
traceId="" spanId="" - Finished node ping
[nodeId=bf79c9d1-746b-4454-bad2-a74ce36fe6c1, res=true, time=5ms]}}
{{INFO 2023-08-02 14:19:53,827 [Thread-82414] o.a.i.s.d.t.TcpDiscoverySpi
traceId="" spanId="" - Pinging node: ad61ff66-d298-4311-8785-193c1dbe443b}}
{{INFO 2023-08-02 14:19:53,828 [Thread-82414] o.a.i.s.d.t.TcpDiscoverySpi
traceId="" spanId="" - Finished node ping
[nodeId=ad61ff66-d298-4311-8785-193c1dbe443b, res=true, time=6ms]}}
{{DEBUG 2023-08-02 14:19:53,829 [ClientMessageReceptor2]
o.j.o.i.ClientIIOPConnection traceId="" spanId="" - read 12 bytes from
\{removed}}}
{{DEBUG 2023-08-02 14:19:53,829 [ClientMessageReceptor2]
o.j.o.i.ClientIIOPConnection traceId="" spanId="" - read 23 bytes from
\{removed}}}
{{DEBUG 2023-08-02 14:19:53,829 [ClientMessageReceptor2] o.j.g.conn traceId=""
spanId="" - read GIOP message of size 35 from ClientGIOPConnection to
\{removed} (e8696fa)}}
{{TRACE 2023-08-02 14:19:53,924 [tcp-disco-msg-worker-[ad61ff66
\{removed}]-#2%GRID_NAME_REPLACED%-#42%GRID_NAME_REPLACED%]
o.a.i.s.d.t.TcpDiscoverySpi traceId="" spanId="" - Next node remains the same
[nextId=ad61ff66-d298-4311-8785-193c1dbe443b, nextOrder=3]}}
{{TRACE 2023-08-02 14:19:53,925 [tcp-disco-msg-worker-[ad61ff66
\{removed}]-#2%GRID_NAME_REPLACED%-#42%GRID_NAME_REPLACED%]
[jira] [Updated] (IGNITE-20194) Calcite engine. Dependency common-codec required for some functions
[ https://issues.apache.org/jira/browse/IGNITE-20194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yury Gerzhedovich updated IGNITE-20194: --- Labels: calcite calcite3-required ise (was: calcite ise) > Calcite engine. Dependency common-codec required for some functions > --- > > Key: IGNITE-20194 > URL: https://issues.apache.org/jira/browse/IGNITE-20194 > Project: Ignite > Issue Type: Bug >Reporter: Aleksey Plekhanov >Assignee: Aleksey Plekhanov >Priority: Major > Labels: calcite, calcite3-required, ise > Fix For: 2.16 > > Time Spent: 0.5h > Remaining Estimate: 0h > > Some functions (md5, soundex) require common-codec dependency, but this > dependency is not included explicitly to dependencies list and doesn't added > to calcite library folder on build. Tests run with transitive dependencies > and don't trigger the problem. Queries fail only when run on Ignite started > from binary release package. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20267) Infinite loop of SockeException
Sebastian Fabisz created IGNITE-20267:
-
Summary: Infinite loop of SockeException
Key: IGNITE-20267
URL: https://issues.apache.org/jira/browse/IGNITE-20267
Project: Ignite
Issue Type: Bug
Components: thin client
Affects Versions: 2.15
Reporter: Sebastian Fabisz
Some of our ignite instances are experiencing infinite loop of same error:
{{ERROR 2023-07-27 08:26:44,876
[grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]
o.a.i.s.c.t.TcpCommunicationSpi traceId="" spanId="" - Failed to process
selector key [ses=GridSelectorNioSessionImpl [worker=DirectNioClientWorker
[super=AbstractNioClientWorker [idx=2, bytesRcvd=21528, bytesSent=15345,
bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker
[name=grid-nio-worker-tcp-comm-2, igniteInstanceName=TcpCommunicationSpi,
finished=false, heartbeatTs=1690442803865, hashCode=2102759141,
interrupted=false,
runner=grid-nio-worker-tcp-comm-2-#25%TcpCommunicationSpi%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768], inRecovery=null,
outRecovery=null, closeSocket=true,
outboundMessagesQueueSizeMetric=o.a.i.i.processors.metric.impl.LongAdderMetric@69a257d1,
super=GridNioSessionImpl [locAddr=\{removed}, rmtAddr=\{removed},
createTime=1690249023154, closeTime=0, bytesSent=18, bytesRcvd=3, bytesSent0=0,
bytesRcvd0=0, sndSchedTime=1690442567813, lastSndTime=1690249023154,
lastRcvTime=1690442567813, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=o.a.i.i.util.nio.GridDirectParser@1fff7116, directMode=true],
GridConnectionBytesVerifyFilter], accepted=true, markedForClose=false]]]
java.net.SocketException: Connection reset at
java.base/sun.nio.ch.SocketChannelImpl.throwConnectionReset(SocketChannelImpl.java:394)
at java.base/sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:411) at
org.apache.ignite.internal.util.nio.GridNioServer$DirectNioClientWorker.processRead(GridNioServer.java:1351)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.processSelectedKeys(GridNioServer.java:2575)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.bodyInternal(GridNioServer.java:2271)
at
org.apache.ignite.internal.util.nio.GridNioServer$AbstractNioClientWorker.body(GridNioServer.java:1910)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.base/java.lang.Thread.run(Thread.java:833)}}
Each error contains same message except of heartbeatTs field.
This error repeats approximately every second. Not all of Ignite instances are
affected. We have figured out that problem is caused by Nessus security
scanner. It walks over all boxes and runs some security checks. It looks like
one of security checks (which can be http request) causes Ignite to fall into
infinite loop of errors. We think that nessus opens a connection to Ignite,
then connection is closed by nessus, but Iginite won't kill the socket.
We have already updated Ignite to latest version.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20266) .NET: Thin 3.0: TestDataStreamerMetrics is flaky
[
https://issues.apache.org/jira/browse/IGNITE-20266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-20266:
Description:
https://ci.ignite.apache.org/test/5503643215833216881?currentProjectId=ApacheIgnite3xGradle_Test=true=
{code}
streamer-batches-active
Expected: 1
But was: 2
at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.RecordView`1.StreamDataAsync(IAsyncEnumerable`1
data, DataStreamerOptions options, CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/RecordView.cs:line
300
at Apache.Ignite.Tests.MetricsTests.TestDataStreamerMetrics() in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
224
at
NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
at
NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter
awaiter)
at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.b__0()
at
NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext
context, Action action)
1)at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore()
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncIteratorBase`1.MoveNextAsync()
at
System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.Enumerator.MoveNextAsync()
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken)
at
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object
s)
{code}
was:
https://ci.ignite.apache.org/test/5503643215833216881?currentProjectId=ApacheIgnite3xGradle_Test=true=%3Cdefault%3E
{code}
streamer-batches-active
Expected: 1
But was: 2
at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at
[jira] [Updated] (IGNITE-20266) .NET: Thin 3.0: TestDataStreamerMetrics is flaky
[
https://issues.apache.org/jira/browse/IGNITE-20266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-20266:
Description:
https://ci.ignite.apache.org/test/5503643215833216881?currentProjectId=ApacheIgnite3xGradle_Test=true=%3Cdefault%3E
{code}
streamer-batches-active
Expected: 1
But was: 2
at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.RecordView`1.StreamDataAsync(IAsyncEnumerable`1
data, DataStreamerOptions options, CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/RecordView.cs:line
300
at Apache.Ignite.Tests.MetricsTests.TestDataStreamerMetrics() in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
224
at
NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
at
NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter
awaiter)
at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.b__0()
at
NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext
context, Action action)
1)at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore()
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncIteratorBase`1.MoveNextAsync()
at
System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.Enumerator.MoveNextAsync()
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken)
at
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object
s)
{code}
was:
{code}
streamer-batches-active
Expected: 1
But was: 2
at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at
[jira] [Created] (IGNITE-20266) .NET: Thin 3.0: TestDataStreamerMetrics is flaky
Pavel Tupitsyn created IGNITE-20266: --- Summary: .NET: Thin 3.0: TestDataStreamerMetrics is flaky Key: IGNITE-20266 URL: https://issues.apache.org/jira/browse/IGNITE-20266 Project: Ignite Issue Type: Bug Components: platforms, thin client Affects Versions: 3.0.0-beta1 Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn Fix For: 3.0.0-beta2 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20266) .NET: Thin 3.0: TestDataStreamerMetrics is flaky
[
https://issues.apache.org/jira/browse/IGNITE-20266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-20266:
Description:
{code}
streamer-batches-active
Expected: 1
But was: 2
at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 75
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/DataStreamer.cs:line
92
at
Apache.Ignite.Internal.Table.RecordView`1.StreamDataAsync(IAsyncEnumerable`1
data, DataStreamerOptions options, CancellationToken cancellationToken) in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite/Internal/Table/RecordView.cs:line
300
at Apache.Ignite.Tests.MetricsTests.TestDataStreamerMetrics() in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
224
at
NUnit.Framework.Internal.TaskAwaitAdapter.GenericAdapter`1.BlockUntilCompleted()
at
NUnit.Framework.Internal.MessagePumpStrategy.NoMessagePumpStrategy.WaitForCompletion(AwaitAdapter
awaiter)
at NUnit.Framework.Internal.AsyncToSyncAdapter.Await(Func`1 invoke)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.RunTestMethod(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.TestMethodCommand.Execute(TestExecutionContext
context)
at
NUnit.Framework.Internal.Commands.BeforeAndAfterTestCommand.<>c__DisplayClass1_0.b__0()
at
NUnit.Framework.Internal.Commands.DelegatingTestCommand.RunTestMethodInThreadAbortSafeZone(TestExecutionContext
context, Action action)
1)at
Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
in
/opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
248
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/Operators/ToAsyncEnumerable.cs:line
68
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncEnumerable.AsyncEnumerableAdapter`1.MoveNextCore()
at System.Linq.AsyncIteratorBase`1.MoveNextAsync() in
/_/Ix.NET/Source/System.Linq.Async/System/Linq/AsyncIterator.cs:line 70
at
System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine&
stateMachine)
at System.Linq.AsyncIteratorBase`1.MoveNextAsync()
at
System.Runtime.CompilerServices.ConfiguredCancelableAsyncEnumerable`1.Enumerator.MoveNextAsync()
at
Apache.Ignite.Internal.Table.DataStreamer.StreamDataAsync[T](IAsyncEnumerable`1
data, Func`4 sender, IRecordSerializerHandler`1 writer, Func`1 schemaProvider,
Func`1 partitionAssignmentProvider, DataStreamerOptions options,
CancellationToken cancellationToken)
at
System.Runtime.CompilerServices.AsyncTaskMethodBuilder`1.AsyncStateMachineBox`1.ExecutionContextCallback(Object
s)
{code}
> .NET: Thin 3.0: TestDataStreamerMetrics is flaky
>
>
> Key: IGNITE-20266
> URL: https://issues.apache.org/jira/browse/IGNITE-20266
> Project: Ignite
> Issue Type: Bug
> Components: platforms, thin client
>Affects Versions: 3.0.0-beta1
>Reporter: Pavel Tupitsyn
>Assignee: Pavel Tupitsyn
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> {code}
> streamer-batches-active
> Expected: 1
> But was: 2
>at
> Apache.Ignite.Tests.MetricsTests.g__GetTuples|13_0()+MoveNext()
> in
> /opt/buildagent/work/b8d4df1365f1f1e5/modules/platforms/dotnet/Apache.Ignite.Tests/MetricsTests.cs:line
> 248
>at
[jira] [Created] (IGNITE-20265) Check and slightly refactor the validation of tables and indexes in the catalog
Kirill Tkalenko created IGNITE-20265: Summary: Check and slightly refactor the validation of tables and indexes in the catalog Key: IGNITE-20265 URL: https://issues.apache.org/jira/browse/IGNITE-20265 Project: Ignite Issue Type: Improvement Reporter: Kirill Tkalenko Assignee: Kirill Tkalenko Fix For: 3.0.0-beta2 We need to make sure that all validation for tables and indexes is present in the catalog. Slightly refactor the code related to validation, for example, move validation methods to *CatalogParamsValidationUtils* and tests related to it in *CatalogManagerValidationTest*. -- This message was sent by Atlassian Jira (v8.20.10#820010)
