[jira] [Created] (IGNITE-20462) Idle_verify prints partitions hash conflicts when entries are expiring concurrently
Aleksey Plekhanov created IGNITE-20462: -- Summary: Idle_verify prints partitions hash conflicts when entries are expiring concurrently Key: IGNITE-20462 URL: https://issues.apache.org/jira/browse/IGNITE-20462 Project: Ignite Issue Type: Bug Reporter: Aleksey Plekhanov Assignee: Aleksey Plekhanov Background process of entries expire (ttl-cleaner-worker) always working on activated cluster, so, during idle_verify execution entries still can be expired even without workload on cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-20433) cctx.mvccEnabled() removal
[ https://issues.apache.org/jira/browse/IGNITE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov resolved IGNITE-20433. --- Resolution: Fixed Merged to the master. [~vladsz83], thanks for the review! > cctx.mvccEnabled() removal > -- > > Key: IGNITE-20433 > URL: https://issues.apache.org/jira/browse/IGNITE-20433 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 3h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20433) cctx.mvccEnabled() removal
[ https://issues.apache.org/jira/browse/IGNITE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20433: -- Fix Version/s: 2.16 > cctx.mvccEnabled() removal > -- > > Key: IGNITE-20433 > URL: https://issues.apache.org/jira/browse/IGNITE-20433 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 3h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20433) cctx.mvccEnabled() removal
[ https://issues.apache.org/jira/browse/IGNITE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anton Vinogradov updated IGNITE-20433: -- Ignite Flags: (was: Docs Required,Release Notes Required) > cctx.mvccEnabled() removal > -- > > Key: IGNITE-20433 > URL: https://issues.apache.org/jira/browse/IGNITE-20433 > Project: Ignite > Issue Type: Sub-task >Reporter: Anton Vinogradov >Assignee: Anton Vinogradov >Priority: Major > Fix For: 2.16 > > Time Spent: 3h 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20433) cctx.mvccEnabled() removal
[
https://issues.apache.org/jira/browse/IGNITE-20433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767185#comment-17767185
]
Ignite TC Bot commented on IGNITE-20433:
{panel:title=Branch: [pull/10938/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10938/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=7342315buildTypeId=IgniteTests24Java8_RunAll]
> cctx.mvccEnabled() removal
> --
>
> Key: IGNITE-20433
> URL: https://issues.apache.org/jira/browse/IGNITE-20433
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20461) Sql. Document SourceAwareIgniteRel.
[ https://issues.apache.org/jira/browse/IGNITE-20461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20461: -- Description: Update javadocs of SourceAwareIgniteRel. Docs should answer the folloing questions: - Why do we need this interface. - How source id property is used. - Want does source = -1 mean. was: Update javadoc to SourceAwareIgniteRel. Docs should answer the folloing questions: - Why do we need this interface. - How source id is used. - Want does source = -1 means. > Sql. Document SourceAwareIgniteRel. > --- > > Key: IGNITE-20461 > URL: https://issues.apache.org/jira/browse/IGNITE-20461 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Update javadocs of SourceAwareIgniteRel. Docs should answer the folloing > questions: > - Why do we need this interface. > - How source id property is used. > - Want does source = -1 mean. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20461) Sql. Document SourceAwareIgniteRel.
[ https://issues.apache.org/jira/browse/IGNITE-20461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20461: -- Fix Version/s: 3.0.0-beta2 > Sql. Document SourceAwareIgniteRel. > --- > > Key: IGNITE-20461 > URL: https://issues.apache.org/jira/browse/IGNITE-20461 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Update javadoc to SourceAwareIgniteRel. Docs should answer the folloing > questions: > - Why do we need this interface. > - How source id is used. > - Want does source = -1 means. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20461) Sql. Document SourceAwareIgniteRel.
Maksim Zhuravkov created IGNITE-20461: - Summary: Sql. Document SourceAwareIgniteRel. Key: IGNITE-20461 URL: https://issues.apache.org/jira/browse/IGNITE-20461 Project: Ignite Issue Type: Improvement Reporter: Maksim Zhuravkov Update javadoc to SourceAwareIgniteRel. Docs should answer the folloing questions: - Why do we need this interface. - How source id is used. - Want does source = -1 means. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20453) Basic Multi Statement Support
[ https://issues.apache.org/jira/browse/IGNITE-20453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-20453: -- Labels: ignite-3 (was: ) > Basic Multi Statement Support > - > > Key: IGNITE-20453 > URL: https://issues.apache.org/jira/browse/IGNITE-20453 > Project: Ignite > Issue Type: Epic > Components: sql >Reporter: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > h1. Motivation > A multi-statement query is a collection of SQL statements that can be > executed in one request. Supporting multi-statement queries may result in > several benefits: > * It helps to decrease number of round trips between the application and the > database server, which is positively affects performance (this can be > achieved by using batching though) > * It may significantly improve UX: during maintaining, user may submit an > entire migration/initialization script to the database server without need to > split this script on independent statements by hand > * In distributed system, some features (like shared mutable state, system and > user defined variables) are easier to introduce for multi-statement only, > rather than for general case > Most popular RDBMS systems, such as Oracle, MySQL, and PostgreSQL, already > support multi-statement execution. > Let's support multi statement queries in Apache Ignite 3 to ease the > migration to Ignite and improve UX by providing familiar and convenient way > of working with database. > h1. Requirements > # It should be possible to start new transaction and commit it from a script > # If there is no explicit active transaction (either passed as parameter to > the API call or started from script), then every statement should be wrapped > in its own transaction > # It should not be possible to commit transaction passed as parameter to the > API call > # It should not be possible to start another transaction if there is an > active transaction > # It should not be possible to start transaction by executing a tx management > statement in single statement mode > # The execution of a multi statement query should emulates serial execution > for all statements in the order they are enumerated in the script; as if > statements had been executed one after another, serially, rather than > concurrently > # Multi statement query should do progress even if no one consumes the result -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20443) Sql. Implement script processing logic.
[
https://issues.apache.org/jira/browse/IGNITE-20443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20443:
--
Description:
The main goal of this task - introduce a {{script processor}} that will be
responsible for managing the execution of multi-statement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
{{script processor}} should use to control script execution.
Integration with transaction management (using script commands) should not be
part of this ticket.
was:
The main goal of this task - introduce a script processor that will be
responsible for managing the execution of mulstistatement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
script processor should use to control script execution.
Integration with transaction management (using script commands) should not be
part of this ticket.
> Sql. Implement script processing logic.
> ---
>
> Key: IGNITE-20443
> URL: https://issues.apache.org/jira/browse/IGNITE-20443
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> The main goal of this task - introduce a {{script processor}} that will be
> responsible for managing the execution of multi-statement queries.
> In order to integrate it with oublic API (session#executeScriptAsync) we need
> add a new internal method to QueryProcessor.
> Since we need the ability to switch forward between cursors, the signature
> should essentially look something like this:
> {code:java}
> CompletableFuture> queryScriptAsync(...)
> {code}
> (the decision on the final form of the signature must be made during the
> implementation of this task)
> The entire script will be parsed at once.
> Statements must be executed one by one in the order they are specified in the
> script.
> Due to the lazy nature of SQL engine, the moment when the current statement
> is "complete" depends on the user who drains the cursor.
> To avoid dependency on a user's actions, it proposed to consider statement
> being "complete" as soon as first page is ready to be returned to the user.
> IGNITE-20454 should introduce a notification for cursor prefetching which the
> {{script processor}} should use to control script execution.
> Integration with transaction management (using script commands) should not be
> part of this ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20454) Sql. Extend SQL cursor with ability to check if first page is ready
Konstantin Orlov created IGNITE-20454:
-
Summary: Sql. Extend SQL cursor with ability to check if first
page is ready
Key: IGNITE-20454
URL: https://issues.apache.org/jira/browse/IGNITE-20454
Project: Ignite
Issue Type: Improvement
Components: sql
Reporter: Konstantin Orlov
For multi statement queries, in order to advance to the next statement we have
to get sure that the first page of result for current statement is ready to be
served. This allows not to depend on a user and finish the script even if no
one consumes the results.
Definition of done: there is an API available from within {{SqlQueryProcessor}}
such that will allow to be notified about completion of prefetch
({{AsyncRootNode#prefetch}}).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20461) Sql. Document SourceAwareIgniteRel.
[ https://issues.apache.org/jira/browse/IGNITE-20461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20461: -- Component/s: sql > Sql. Document SourceAwareIgniteRel. > --- > > Key: IGNITE-20461 > URL: https://issues.apache.org/jira/browse/IGNITE-20461 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > > Update javadoc to SourceAwareIgniteRel. Docs should answer the folloing > questions: > - Why do we need this interface. > - How source id is used. > - Want does source = -1 means. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20461) Sql. Document SourceAwareIgniteRel.
[ https://issues.apache.org/jira/browse/IGNITE-20461?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20461: -- Labels: ignite-3 (was: ) > Sql. Document SourceAwareIgniteRel. > --- > > Key: IGNITE-20461 > URL: https://issues.apache.org/jira/browse/IGNITE-20461 > Project: Ignite > Issue Type: Improvement >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > > Update javadoc to SourceAwareIgniteRel. Docs should answer the folloing > questions: > - Why do we need this interface. > - How source id is used. > - Want does source = -1 means. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20455) Sql. Test Framework. Add System View support.
[ https://issues.apache.org/jira/browse/IGNITE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20455: -- Fix Version/s: 3.0.0-beta2 > Sql. Test Framework. Add System View support. > -- > > Key: IGNITE-20455 > URL: https://issues.apache.org/jira/browse/IGNITE-20455 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Fix For: 3.0.0-beta2 > > > Implement test system view builders - standalone one > (TestBuilders::systemView()) and one for test cluster > (CLusterBuilder::addSystemView()). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20453) Sql. Basic Multi Statement Support
[ https://issues.apache.org/jira/browse/IGNITE-20453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-20453: -- Summary: Sql. Basic Multi Statement Support (was: Basic Multi Statement Support) > Sql. Basic Multi Statement Support > -- > > Key: IGNITE-20453 > URL: https://issues.apache.org/jira/browse/IGNITE-20453 > Project: Ignite > Issue Type: Epic > Components: sql >Reporter: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > h1. Motivation > A multi-statement query is a collection of SQL statements that can be > executed in one request. Supporting multi-statement queries may result in > several benefits: > * It helps to decrease number of round trips between the application and the > database server, which is positively affects performance (this can be > achieved by using batching though) > * It may significantly improve UX: during maintaining, user may submit an > entire migration/initialization script to the database server without need to > split this script on independent statements by hand > * In distributed system, some features (like shared mutable state, system and > user defined variables) are easier to introduce for multi-statement only, > rather than for general case > Most popular RDBMS systems, such as Oracle, MySQL, and PostgreSQL, already > support multi-statement execution. > Let's support multi statement queries in Apache Ignite 3 to ease the > migration to Ignite and improve UX by providing familiar and convenient way > of working with database. > h1. Requirements > # It should be possible to start new transaction and commit it from a script > # If there is no explicit active transaction (either passed as parameter to > the API call or started from script), then every statement should be wrapped > in its own transaction > # It should not be possible to commit transaction passed as parameter to the > API call > # It should not be possible to start another transaction if there is an > active transaction > # It should not be possible to start transaction by executing a tx management > statement in single statement mode > # The execution of a multi statement query should emulates serial execution > for all statements in the order they are enumerated in the script; as if > statements had been executed one after another, serially, rather than > concurrently > # Multi statement query should do progress even if no one consumes the result -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20369) Breakdown tasks for initial implementation of FailureHandler
[ https://issues.apache.org/jira/browse/IGNITE-20369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-20369: - Reviewer: Sergey Uttsel > Breakdown tasks for initial implementation of FailureHandler > > > Key: IGNITE-20369 > URL: https://issues.apache.org/jira/browse/IGNITE-20369 > Project: Ignite > Issue Type: Task >Reporter: Vyacheslav Koptilin >Assignee: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > The initial implementation can be broken down into the following tasks: > - IGNITE-20447 Introduce a new failure-handling component > - IGNITE-20448 Implement strategies for failure handling > - IGNITE-20451 Introduce Introduce WorkerRegistery > - IGNITE-20449 OutOfMemory error should be covered by failure handling > - IGNITE-20450 Failure handler configuration > - IGNITE-20452 Integrate failure handler processor into all components -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20455) Sql. Test Framework. Add System View support.
[ https://issues.apache.org/jira/browse/IGNITE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20455: -- Labels: ignite-3 (was: ) > Sql. Test Framework. Add System View support. > -- > > Key: IGNITE-20455 > URL: https://issues.apache.org/jira/browse/IGNITE-20455 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Implement test system view builders - standalone one > (TestBuilders::systemView()) and one for test cluster > (CLusterBuilder::addSystemView()). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20444) Sql. Add restrictions for execution tx related statements with single statement mode.
[ https://issues.apache.org/jira/browse/IGNITE-20444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-20444: -- Epic Link: IGNITE-20453 > Sql. Add restrictions for execution tx related statements with single > statement mode. > - > > Key: IGNITE-20444 > URL: https://issues.apache.org/jira/browse/IGNITE-20444 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 3.0.0-beta1 >Reporter: Evgeny Stanilovsky >Assignee: Evgeny Stanilovsky >Priority: Major > Labels: ignite-3 > > Append single statement restrictions for using with new tx related sql syntax > [1] implementation. > [1] https://issues.apache.org/jira/browse/IGNITE-20442 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20443) Sql. Implement script processing logic.
[
https://issues.apache.org/jira/browse/IGNITE-20443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20443:
--
Description:
The main goal of this task - introduce a {{script processor}} that will be
responsible for managing the execution of multi-statement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture>> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
{{script processor}} should use to control script execution.
Integration with transaction management (using script commands) must be
implemented in another ticket and should not be part of this ticket.
was:
The main goal of this task - introduce a {{script processor}} that will be
responsible for managing the execution of multi-statement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
{{script processor}} should use to control script execution.
Integration with transaction management (using script commands) must be
implemented in another ticket and should not be part of this ticket.
> Sql. Implement script processing logic.
> ---
>
> Key: IGNITE-20443
> URL: https://issues.apache.org/jira/browse/IGNITE-20443
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> The main goal of this task - introduce a {{script processor}} that will be
> responsible for managing the execution of multi-statement queries.
> In order to integrate it with oublic API (session#executeScriptAsync) we need
> add a new internal method to QueryProcessor.
> Since we need the ability to switch forward between cursors, the signature
> should essentially look something like this:
> {code:java}
> CompletableFuture>> queryScriptAsync(...)
> {code}
> (the decision on the final form of the signature must be made during the
> implementation of this task)
> The entire script will be parsed at once.
> Statements must be executed one by one in the order they are specified in the
> script.
> Due to the lazy nature of SQL engine, the moment when the current statement
> is "complete" depends on the user who drains the cursor.
> To avoid dependency on a user's actions, it proposed to consider statement
> being "complete" as soon as first page is ready to be returned to the user.
> IGNITE-20454 should introduce a notification for cursor prefetching which the
> {{script processor}} should use to control script execution.
> Integration with transaction management (using script commands) must be
> implemented in another ticket and should not be part of this ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20455) Sql. Test Framework. Add System View support.
Maksim Zhuravkov created IGNITE-20455: - Summary: Sql. Test Framework. Add System View support. Key: IGNITE-20455 URL: https://issues.apache.org/jira/browse/IGNITE-20455 Project: Ignite Issue Type: Improvement Components: sql Reporter: Maksim Zhuravkov Implement test system view builders - standalone one (TestBuilders::systemView()) and one for test cluster (CLusterBuilder::addSystemView()). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20442) Sql. Extend grammar with transaction related statements.
[
https://issues.apache.org/jira/browse/IGNITE-20442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov updated IGNITE-20442:
--
Epic Link: IGNITE-20453
> Sql. Extend grammar with transaction related statements.
>
>
> Key: IGNITE-20442
> URL: https://issues.apache.org/jira/browse/IGNITE-20442
> Project: Ignite
> Issue Type: Task
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> In order to process multistatement queries we need to support the following
> sql grammar to start/finish transactions.
> {code}
> ::=
> START TRANSACTION []
> ::= READ ONLY | READ WRITE
> {code}
> {code}
> ::=
> COMMIT
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20453) Basic Multi Statement Support
Konstantin Orlov created IGNITE-20453: - Summary: Basic Multi Statement Support Key: IGNITE-20453 URL: https://issues.apache.org/jira/browse/IGNITE-20453 Project: Ignite Issue Type: Epic Components: sql Reporter: Konstantin Orlov h1. Motivation A multi-statement query is a collection of SQL statements that can be executed in one request. Supporting multi-statement queries may result in several benefits: * It helps to decrease number of round trips between the application and the database server, which is positively affects performance (this can be achieved by using batching though) * It may significantly improve UX: during maintaining, user may submit an entire migration/initialization script to the database server without need to split this script on independent statements by hand * In distributed system, some features (like shared mutable state, system and user defined variables) are easier to introduce for multi-statement only, rather than for general case Most popular RDBMS systems, such as Oracle, MySQL, and PostgreSQL, already support multi-statement execution. Let's support multi statement queries in Apache Ignite 3 to ease the migration to Ignite and improve UX by providing familiar and convenient way of working with database. h1. Requirements # It should be possible to start new transaction and commit it from a script # If there is no explicit active transaction (either passed as parameter to the API call or started from script), then every statement should be wrapped in its own transaction # It should not be possible to commit transaction passed as parameter to the API call # It should not be possible to start another transaction if there is an active transaction # It should not be possible to start transaction by executing a tx management statement in single statement mode # The execution of a multi statement query should emulates serial execution for all statements in the order they are enumerated in the script; as if statements had been executed one after another, serially, rather than concurrently # Multi statement query should do progress even if no one consumes the result -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20443) Sql. Implement script processing logic.
[
https://issues.apache.org/jira/browse/IGNITE-20443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20443:
--
Description:
The main goal of this task - introduce a {{script processor}} that will be
responsible for managing the execution of multi-statement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
{{script processor}} should use to control script execution.
Integration with transaction management (using script commands) must be
implemented in another ticket and should not be part of this ticket.
was:
The main goal of this task - introduce a {{script processor}} that will be
responsible for managing the execution of multi-statement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
{{script processor}} should use to control script execution.
Integration with transaction management (using script commands) should not be
part of this ticket.
> Sql. Implement script processing logic.
> ---
>
> Key: IGNITE-20443
> URL: https://issues.apache.org/jira/browse/IGNITE-20443
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> The main goal of this task - introduce a {{script processor}} that will be
> responsible for managing the execution of multi-statement queries.
> In order to integrate it with oublic API (session#executeScriptAsync) we need
> add a new internal method to QueryProcessor.
> Since we need the ability to switch forward between cursors, the signature
> should essentially look something like this:
> {code:java}
> CompletableFuture> queryScriptAsync(...)
> {code}
> (the decision on the final form of the signature must be made during the
> implementation of this task)
> The entire script will be parsed at once.
> Statements must be executed one by one in the order they are specified in the
> script.
> Due to the lazy nature of SQL engine, the moment when the current statement
> is "complete" depends on the user who drains the cursor.
> To avoid dependency on a user's actions, it proposed to consider statement
> being "complete" as soon as first page is ready to be returned to the user.
> IGNITE-20454 should introduce a notification for cursor prefetching which the
> {{script processor}} should use to control script execution.
> Integration with transaction management (using script commands) must be
> implemented in another ticket and should not be part of this ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Resolved] (IGNITE-19967) Design restart of Distribution Zone Manager in accordance with the new local recovery process
[ https://issues.apache.org/jira/browse/IGNITE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin resolved IGNITE-19967. -- Resolution: Fixed > Design restart of Distribution Zone Manager in accordance with the new local > recovery process > - > > Key: IGNITE-19967 > URL: https://issues.apache.org/jira/browse/IGNITE-19967 > Project: Ignite > Issue Type: Task >Reporter: Mirza Aliev >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > > In https://issues.apache.org/jira/browse/IGNITE-19061 we have provided and > later implemented algorithm for restoring zones states after restart. > In https://issues.apache.org/jira/browse/IGNITE-19778 and other tickets new > invariants for recovery were introduced, in particular, in the new proposal > node state is recovered not until applied revision, but until some higher > revision, which brings inconsistency for restoring Distribution Zone Manager > because the process was based on saving states to the Vault and on mechanism > with restoring from the Vault. Previously we could based our recovery on the > fact, that all events, that node missed during absence, will be applied to > starting from metastorage applied revision, but in the new recovery process, > node receives only snapshot of the data and we cannot restore the history of > data nodes related events. > In this task we need to provide a new approach, taking into account new > circumstances. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20336) Sql. Remove conversion from java types to TypeSpec.
[ https://issues.apache.org/jira/browse/IGNITE-20336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Evgeny Stanilovsky reassigned IGNITE-20336: --- Assignee: Evgeny Stanilovsky > Sql. Remove conversion from java types to TypeSpec. > --- > > Key: IGNITE-20336 > URL: https://issues.apache.org/jira/browse/IGNITE-20336 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Assignee: Evgeny Stanilovsky >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > JavaTypes are no longer used by execution engine, but they remain in > execution node tests and TypeUtils::convertToTypeSpec. > - Update unit not to use TypeUtils::createRowType(TypeFactory, Class... > fields). > - Remove conversion from JavaType from TypeUtils::convertToTypeSpec. > - Remove TypeUtils::createRowType(TypeFactory, Class... fields) as it is > no longer needed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20459) Preapre test plan for the Transactions on Stable Topology feature
[ https://issues.apache.org/jira/browse/IGNITE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20459: - Labels: ignite-3 (was: ) > Preapre test plan for the Transactions on Stable Topology feature > - > > Key: IGNITE-20459 > URL: https://issues.apache.org/jira/browse/IGNITE-20459 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20457) Verify commitTimestamp against enlisted partitions expiration timestamps
[ https://issues.apache.org/jira/browse/IGNITE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20457: - Labels: ignite-3 tech-debt (was: ) > Verify commitTimestamp against enlisted partitions expiration timestamps > > > Key: IGNITE-20457 > URL: https://issues.apache.org/jira/browse/IGNITE-20457 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, tech-debt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20457) Verify commitTimestamp against enlisted partitions expiration timestamps
[ https://issues.apache.org/jira/browse/IGNITE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-20457: Assignee: Alexander Lapin > Verify commitTimestamp against enlisted partitions expiration timestamps > > > Key: IGNITE-20457 > URL: https://issues.apache.org/jira/browse/IGNITE-20457 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Blocker > Labels: ignite-3, tech-debt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20457) Verify commitTimestamp against enlisted partitions expiration timestamps
[ https://issues.apache.org/jira/browse/IGNITE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20457: - Ignite Flags: (was: Docs Required,Release Notes Required) > Verify commitTimestamp against enlisted partitions expiration timestamps > > > Key: IGNITE-20457 > URL: https://issues.apache.org/jira/browse/IGNITE-20457 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20458) Preapre test plan for the DistributionZones feature
[ https://issues.apache.org/jira/browse/IGNITE-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-20458: Assignee: Mirza Aliev > Preapre test plan for the DistributionZones feature > --- > > Key: IGNITE-20458 > URL: https://issues.apache.org/jira/browse/IGNITE-20458 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20459) Preapre test plan for the Transactions on Stable Topology feature
[ https://issues.apache.org/jira/browse/IGNITE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-20459: Assignee: Alexander Lapin > Preapre test plan for the Transactions on Stable Topology feature > - > > Key: IGNITE-20459 > URL: https://issues.apache.org/jira/browse/IGNITE-20459 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Assignee: Alexander Lapin >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20443) Sql. Implement script processing logic.
[
https://issues.apache.org/jira/browse/IGNITE-20443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov updated IGNITE-20443:
--
Epic Link: IGNITE-20453
> Sql. Implement script processing logic.
> ---
>
> Key: IGNITE-20443
> URL: https://issues.apache.org/jira/browse/IGNITE-20443
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> We need to add a new method to {{QueryProcessor}} that will be responsible
> for processing multi-statement queries.
> {{// TBD suggested method signature}}
> In general, statements must be executed one by one in the order they are
> specified in the script.
> {{// TBD suggestion for parsing}}
> Due to the lazy nature of SQL engine, the moment when the current statement
> is "complete" depends on the user who drains the cursor.
> To avoid dependency on a user's actions, it proposed to consider statement
> being "complete" as soon as first page is ready to be returned to the user.
> {{// TBD reference to ticket about SqlCursor event}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20395) Clean up write intents for RW transaction on primary
[
https://issues.apache.org/jira/browse/IGNITE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-20395:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Clean up write intents for RW transaction on primary
>
>
> Key: IGNITE-20395
> URL: https://issues.apache.org/jira/browse/IGNITE-20395
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Sizov
>Priority: Major
> Labels: ignite-3
>
> If a transaction was committed/aborted, but for any reason the cleanup
> operation was not performed on a node, the write intent entries would still
> be present in the storage.
> For example, the primary node crashed before executing the cleanup. In this
> case the storage will still have write intents even if the transaction is
> finished.
> When an RO transaction sees write intents, no matter on primary or on any
> other node, it performs write intent resolution and returns the correct
> result.
> When an RW transaction sees write intents (happens on primary only), it also
> performs write intent resolution. But any following change of the rows having
> write intents will result in a storage exception because the storage doesn't
> support more than one write intent per row.
> IGNITE-20041 added a way to trigger async write intent cleanup on the node
> that executes write intent resolution. But RW transaction does not wait the
> completion of the cleanup.
> *Definition of done:*
> If an RW transaction performs write intent resolution for a row and this is
> a modifying action (not GET), we should synchronously cleanup the write
> intent before proceeding with executing UpdateCommand.
> *Implementation details*
> We can either call cleanup directly (like in {{processTxCleanupAction}} or
> wait for the cleanup future (the result of async cleanup execution method) to
> finish.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20443) Sql. Implement script processing logic.
[
https://issues.apache.org/jira/browse/IGNITE-20443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-20443:
--
Description:
The main goal of this task - introduce a script processor that will be
responsible for managing the execution of mulstistatement queries.
In order to integrate it with oublic API (session#executeScriptAsync) we need
add a new internal method to QueryProcessor.
Since we need the ability to switch forward between cursors, the signature
should essentially look something like this:
{code:java}
CompletableFuture> queryScriptAsync(...)
{code}
(the decision on the final form of the signature must be made during the
implementation of this task)
The entire script will be parsed at once.
Statements must be executed one by one in the order they are specified in the
script.
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
IGNITE-20454 should introduce a notification for cursor prefetching which the
script processor should use to control script execution.
Integration with transaction management (using script commands) should not be
part of this ticket.
was:
We need to add a new method to {{QueryProcessor}} that will be responsible for
processing multi-statement queries.
{{// TBD suggested method signature}}
In general, statements must be executed one by one in the order they are
specified in the script.
{{// TBD suggestion for parsing}}
Due to the lazy nature of SQL engine, the moment when the current statement is
"complete" depends on the user who drains the cursor.
To avoid dependency on a user's actions, it proposed to consider statement
being "complete" as soon as first page is ready to be returned to the user.
{{// TBD reference to ticket about SqlCursor event}}
> Sql. Implement script processing logic.
> ---
>
> Key: IGNITE-20443
> URL: https://issues.apache.org/jira/browse/IGNITE-20443
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Pavel Pereslegin
>Priority: Major
> Labels: ignite-3
>
> The main goal of this task - introduce a script processor that will be
> responsible for managing the execution of mulstistatement queries.
> In order to integrate it with oublic API (session#executeScriptAsync) we need
> add a new internal method to QueryProcessor.
> Since we need the ability to switch forward between cursors, the signature
> should essentially look something like this:
> {code:java}
> CompletableFuture> queryScriptAsync(...)
> {code}
> (the decision on the final form of the signature must be made during the
> implementation of this task)
> The entire script will be parsed at once.
> Statements must be executed one by one in the order they are specified in the
> script.
> Due to the lazy nature of SQL engine, the moment when the current statement
> is "complete" depends on the user who drains the cursor.
> To avoid dependency on a user's actions, it proposed to consider statement
> being "complete" as soon as first page is ready to be returned to the user.
> IGNITE-20454 should introduce a notification for cursor prefetching which the
> script processor should use to control script execution.
> Integration with transaction management (using script commands) should not be
> part of this ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20458) Preapre test plan for the DistributionZones feature
[ https://issues.apache.org/jira/browse/IGNITE-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20458: - Labels: ignite-3 (was: ) > Preapre test plan for the DistributionZones feature > --- > > Key: IGNITE-20458 > URL: https://issues.apache.org/jira/browse/IGNITE-20458 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20445) Clean up write intents for RW transaction on replication group nodes
[ https://issues.apache.org/jira/browse/IGNITE-20445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-20445: Assignee: Kirill Sizov > Clean up write intents for RW transaction on replication group nodes > > > Key: IGNITE-20445 > URL: https://issues.apache.org/jira/browse/IGNITE-20445 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted and for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents, we can get an exception. > _Imagine the case where a finished transaction left its write intents in the > storage of all nodes A, B and C. A is primary._ > _An RO transaction is executed on the primary A, it kicks off an async > cleanup (IGNITE-20041)._ > _The cleanup is a local task (not distributed to the replication group), thus > only the A's storage is cleaned. B and C storages still contain the same > write intent._ > _Now an RW transaction starts. It sees no write intents on A, executes its > action and the action is replicated to B and C. Execution of this task on B > and C will result in a storage exception since it's not allowed to have more > than one write intent per row._ > *Definition of Done* > The nodes of the replication group should perform cleanup of their storages > when they receive an UpdateCommand, before adding new write intents. > *Implementation details* > We can extend the update command with the timestamp of the latest commit on > primary. > If the nodes of the replication group see a write intent in their storage, > they will: > * +commit+ the write intent if the UpdateCommand`s latestCommitTimestamp is > greater than the commit timestamp of latest committed entry. > * +abort+ the write intent if he UpdateCommand`s latestCommitTimestamp is > equal to the commit timestamp of latest committed entry. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20457) Verify commitTimestamp against enlisted partitions expiration timestamps
Alexander Lapin created IGNITE-20457: Summary: Verify commitTimestamp against enlisted partitions expiration timestamps Key: IGNITE-20457 URL: https://issues.apache.org/jira/browse/IGNITE-20457 Project: Ignite Issue Type: Bug Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20455) Sql. Test Framework. Add System View support.
[ https://issues.apache.org/jira/browse/IGNITE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20455: -- Description: Implement test system view builders - standalone one (TestBuilders::systemView()) and one for test cluster (ClusterBuilder::addSystemView()). was: Implement test system view builders - standalone one (TestBuilders::systemView()) and one for test cluster (CLusterBuilder::addSystemView()). > Sql. Test Framework. Add System View support. > -- > > Key: IGNITE-20455 > URL: https://issues.apache.org/jira/browse/IGNITE-20455 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Implement test system view builders - standalone one > (TestBuilders::systemView()) and one for test cluster > (ClusterBuilder::addSystemView()). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20445) Clean up write intents for RW transaction on replication group nodes
[ https://issues.apache.org/jira/browse/IGNITE-20445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20445: - Ignite Flags: (was: Docs Required,Release Notes Required) > Clean up write intents for RW transaction on replication group nodes > > > Key: IGNITE-20445 > URL: https://issues.apache.org/jira/browse/IGNITE-20445 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted and for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents, we can get an exception. > _Imagine the case where a finished transaction left its write intents in the > storage of all nodes A, B and C. A is primary._ > _An RO transaction is executed on the primary A, it kicks off an async > cleanup (IGNITE-20041)._ > _The cleanup is a local task (not distributed to the replication group), thus > only the A's storage is cleaned. B and C storages still contain the same > write intent._ > _Now an RW transaction starts. It sees no write intents on A, executes its > action and the action is replicated to B and C. Execution of this task on B > and C will result in a storage exception since it's not allowed to have more > than one write intent per row._ > *Definition of Done* > The nodes of the replication group should perform cleanup of their storages > when they receive an UpdateCommand, before adding new write intents. > *Implementation details* > We can extend the update command with the timestamp of the latest commit on > primary. > If the nodes of the replication group see a write intent in their storage, > they will: > * +commit+ the write intent if the UpdateCommand`s latestCommitTimestamp is > greater than the commit timestamp of latest committed entry. > * +abort+ the write intent if he UpdateCommand`s latestCommitTimestamp is > equal to the commit timestamp of latest committed entry. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20459) Preapre test plan for the Transactions on Stable Topology feature
[ https://issues.apache.org/jira/browse/IGNITE-20459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20459: - Ignite Flags: (was: Docs Required,Release Notes Required) > Preapre test plan for the Transactions on Stable Topology feature > - > > Key: IGNITE-20459 > URL: https://issues.apache.org/jira/browse/IGNITE-20459 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-19880) Fix negative duration in the SQL query system view
[ https://issues.apache.org/jira/browse/IGNITE-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Anastasia Iakimova reassigned IGNITE-19880: --- Assignee: Anastasia Iakimova (was: Nikita Amelchev) > Fix negative duration in the SQL query system view > -- > > Key: IGNITE-19880 > URL: https://issues.apache.org/jira/browse/IGNITE-19880 > Project: Ignite > Issue Type: Bug >Reporter: Nikita Amelchev >Assignee: Anastasia Iakimova >Priority: Minor > Labels: ise > > Negative duration in the SQL query system view happens due to different > measure of start and end times: System.currentTimeMillis() and > U.currentTimeMillis(). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20395) Clean up write intents for RW transaction on primary
[
https://issues.apache.org/jira/browse/IGNITE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-20395:
Assignee: Alexander Lapin (was: Kirill Sizov)
> Clean up write intents for RW transaction on primary
>
>
> Key: IGNITE-20395
> URL: https://issues.apache.org/jira/browse/IGNITE-20395
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Sizov
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3
>
> If a transaction was committed/aborted, but for any reason the cleanup
> operation was not performed on a node, the write intent entries would still
> be present in the storage.
> For example, the primary node crashed before executing the cleanup. In this
> case the storage will still have write intents even if the transaction is
> finished.
> When an RO transaction sees write intents, no matter on primary or on any
> other node, it performs write intent resolution and returns the correct
> result.
> When an RW transaction sees write intents (happens on primary only), it also
> performs write intent resolution. But any following change of the rows having
> write intents will result in a storage exception because the storage doesn't
> support more than one write intent per row.
> IGNITE-20041 added a way to trigger async write intent cleanup on the node
> that executes write intent resolution. But RW transaction does not wait the
> completion of the cleanup.
> *Definition of done:*
> If an RW transaction performs write intent resolution for a row and this is
> a modifying action (not GET), we should synchronously cleanup the write
> intent before proceeding with executing UpdateCommand.
> *Implementation details*
> We can either call cleanup directly (like in {{processTxCleanupAction}} or
> wait for the cleanup future (the result of async cleanup execution method) to
> finish.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20446) Perform read before executing the Upsert operation
[ https://issues.apache.org/jira/browse/IGNITE-20446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20446: - Epic Link: IGNITE-20001 > Perform read before executing the Upsert operation > -- > > Key: IGNITE-20446 > URL: https://issues.apache.org/jira/browse/IGNITE-20446 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted, but for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents (happens on primary only), it also > performs write intent resolution and cleanup (done in IGNITE-20395, > IGNITE-20445), so we are safe. > But it works only for those RW transactions that perform READ operation > before making any changes. This is not the case for Upsert. So we need to > change it to read data (hence write intent resolution and cleanup). > *Definition of Done* > Upsert operation execution should result in write intent resolution and > cleanup if the affected rows still contain write intents of a finished > transaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20446) Perform read before executing the Upsert operation
[ https://issues.apache.org/jira/browse/IGNITE-20446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20446: - Ignite Flags: (was: Docs Required,Release Notes Required) > Perform read before executing the Upsert operation > -- > > Key: IGNITE-20446 > URL: https://issues.apache.org/jira/browse/IGNITE-20446 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted, but for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents (happens on primary only), it also > performs write intent resolution and cleanup (done in IGNITE-20395, > IGNITE-20445), so we are safe. > But it works only for those RW transactions that perform READ operation > before making any changes. This is not the case for Upsert. So we need to > change it to read data (hence write intent resolution and cleanup). > *Definition of Done* > Upsert operation execution should result in write intent resolution and > cleanup if the affected rows still contain write intents of a finished > transaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20395) Clean up write intents for RW transaction on primary
[
https://issues.apache.org/jira/browse/IGNITE-20395?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin reassigned IGNITE-20395:
Assignee: Kirill Sizov
> Clean up write intents for RW transaction on primary
>
>
> Key: IGNITE-20395
> URL: https://issues.apache.org/jira/browse/IGNITE-20395
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Sizov
>Assignee: Kirill Sizov
>Priority: Major
> Labels: ignite-3
>
> If a transaction was committed/aborted, but for any reason the cleanup
> operation was not performed on a node, the write intent entries would still
> be present in the storage.
> For example, the primary node crashed before executing the cleanup. In this
> case the storage will still have write intents even if the transaction is
> finished.
> When an RO transaction sees write intents, no matter on primary or on any
> other node, it performs write intent resolution and returns the correct
> result.
> When an RW transaction sees write intents (happens on primary only), it also
> performs write intent resolution. But any following change of the rows having
> write intents will result in a storage exception because the storage doesn't
> support more than one write intent per row.
> IGNITE-20041 added a way to trigger async write intent cleanup on the node
> that executes write intent resolution. But RW transaction does not wait the
> completion of the cleanup.
> *Definition of done:*
> If an RW transaction performs write intent resolution for a row and this is
> a modifying action (not GET), we should synchronously cleanup the write
> intent before proceeding with executing UpdateCommand.
> *Implementation details*
> We can either call cleanup directly (like in {{processTxCleanupAction}} or
> wait for the cleanup future (the result of async cleanup execution method) to
> finish.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20359) Expose node storage as a node attribute
[ https://issues.apache.org/jira/browse/IGNITE-20359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-20359: Description: *Motivation* To introduce the nodes' filtering by storage type and profile we need to expose the appropriate node storage configurations as node attributes (kind of inner attributes, not suitable for general filters). *Definition of done* - Node storage and storage profile exposed as node attributes (TODO: clarify the details, when design will be finished) was: *Motivation* To introduce the nodes' filtering by storage type and profile we need to expose the appropriate node storage configurations as node attributes. *Definition of done* - Node storage and storage profile exposed as node attributes (TODO: clarify the details, when design will be finished) > Expose node storage as a node attribute > --- > > Key: IGNITE-20359 > URL: https://issues.apache.org/jira/browse/IGNITE-20359 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Motivation* > To introduce the nodes' filtering by storage type and profile we need to > expose the appropriate node storage configurations as node attributes (kind > of inner attributes, not suitable for general filters). > *Definition of done* > - Node storage and storage profile exposed as node attributes (TODO: clarify > the details, when design will be finished) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-20446) Perform read before executing the Upsert operation
[ https://issues.apache.org/jira/browse/IGNITE-20446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin reassigned IGNITE-20446: Assignee: Kirill Sizov > Perform read before executing the Upsert operation > -- > > Key: IGNITE-20446 > URL: https://issues.apache.org/jira/browse/IGNITE-20446 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Assignee: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted, but for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents (happens on primary only), it also > performs write intent resolution and cleanup (done in IGNITE-20395, > IGNITE-20445), so we are safe. > But it works only for those RW transactions that perform READ operation > before making any changes. This is not the case for Upsert. So we need to > change it to read data (hence write intent resolution and cleanup). > *Definition of Done* > Upsert operation execution should result in write intent resolution and > cleanup if the affected rows still contain write intents of a finished > transaction. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20459) Preapre test plan for the Transactions on Stable Topology feature
Alexander Lapin created IGNITE-20459: Summary: Preapre test plan for the Transactions on Stable Topology feature Key: IGNITE-20459 URL: https://issues.apache.org/jira/browse/IGNITE-20459 Project: Ignite Issue Type: Task Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20360) Implement the set of zone supported storages
[ https://issues.apache.org/jira/browse/IGNITE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-20360: Description: *Motivation* According to IGNITE-20357 we need to have an appropriate zone filter, which filters the nodes based on their available storages. *Definition of done* - Zone has the filters, which can be unambiguously used to check if table can be "deployed" in this zone *Notes* - Avoid filter altering for now (but add the appropriate event types) was: *Motivation* According to IGNITE-20357 we need to have an appropriate zone filter, which filters the nodes based on their available storages. *Definition of done* - Zone has the filters, which can be unambiguously used to check if table can be "deployed" in this zone > Implement the set of zone supported storages > > > Key: IGNITE-20360 > URL: https://issues.apache.org/jira/browse/IGNITE-20360 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Motivation* > According to IGNITE-20357 we need to have an appropriate zone filter, which > filters the nodes based on their available storages. > *Definition of done* > - Zone has the filters, which can be unambiguously used to check if table can > be "deployed" in this zone > *Notes* > - Avoid filter altering for now (but add the appropriate event types) -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20460) Too many files with unapproved license
Artem Egorov created IGNITE-20460:
-
Summary: Too many files with unapproved license
Key: IGNITE-20460
URL: https://issues.apache.org/jira/browse/IGNITE-20460
Project: Ignite
Issue Type: Bug
Components: build
Reporter: Artem Egorov
Assignee: Igor Gusev
After merging IGNITE-20431, the project build began to fail with an "Too many
files with unapproved license" error:
{quote}1 Unknown Licenses *
Files with unapproved licenses: docs/_docs/installation/upgrades.adoc
{quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-20458) Preapre test plan for the DistributionZones feature
Alexander Lapin created IGNITE-20458: Summary: Preapre test plan for the DistributionZones feature Key: IGNITE-20458 URL: https://issues.apache.org/jira/browse/IGNITE-20458 Project: Ignite Issue Type: Task Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20458) Preapre test plan for the DistributionZones feature
[ https://issues.apache.org/jira/browse/IGNITE-20458?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20458: - Ignite Flags: (was: Docs Required,Release Notes Required) > Preapre test plan for the DistributionZones feature > --- > > Key: IGNITE-20458 > URL: https://issues.apache.org/jira/browse/IGNITE-20458 > Project: Ignite > Issue Type: Task >Reporter: Alexander Lapin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20445) Clean up write intents for RW transaction on replication group nodes
[ https://issues.apache.org/jira/browse/IGNITE-20445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20445: - Epic Link: IGNITE-20001 > Clean up write intents for RW transaction on replication group nodes > > > Key: IGNITE-20445 > URL: https://issues.apache.org/jira/browse/IGNITE-20445 > Project: Ignite > Issue Type: Task >Reporter: Kirill Sizov >Priority: Major > Labels: ignite-3 > > If a transaction was committed/aborted and for any reason the cleanup > operation was not performed on a node, the write intent entries would still > be present in the storage. > When an RO transaction sees write intents, no matter on primary or on any > other node, it performs write intent resolution and returns the correct > result. > When an RW transaction sees write intents, we can get an exception. > _Imagine the case where a finished transaction left its write intents in the > storage of all nodes A, B and C. A is primary._ > _An RO transaction is executed on the primary A, it kicks off an async > cleanup (IGNITE-20041)._ > _The cleanup is a local task (not distributed to the replication group), thus > only the A's storage is cleaned. B and C storages still contain the same > write intent._ > _Now an RW transaction starts. It sees no write intents on A, executes its > action and the action is replicated to B and C. Execution of this task on B > and C will result in a storage exception since it's not allowed to have more > than one write intent per row._ > *Definition of Done* > The nodes of the replication group should perform cleanup of their storages > when they receive an UpdateCommand, before adding new write intents. > *Implementation details* > We can extend the update command with the timestamp of the latest commit on > primary. > If the nodes of the replication group see a write intent in their storage, > they will: > * +commit+ the write intent if the UpdateCommand`s latestCommitTimestamp is > greater than the commit timestamp of latest committed entry. > * +abort+ the write intent if he UpdateCommand`s latestCommitTimestamp is > equal to the commit timestamp of latest committed entry. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20000) Sql. Add planner test to verify numeric type coercion in binary arithmetic
[ https://issues.apache.org/jira/browse/IGNITE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-2: -- Fix Version/s: 3.0.0-beta2 > Sql. Add planner test to verify numeric type coercion in binary arithmetic > --- > > Key: IGNITE-2 > URL: https://issues.apache.org/jira/browse/IGNITE-2 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Assignee: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Let's add new tests to validate numeric type coercion in binary arithmetic. > The tests should be as easy as possible, every test case must answer a > question "what are expected types of each operand when we do arithmetic with > a given numeric type pair?" > Look to NumericComparisonTypeCoercionTest as example. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-10516) Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables
[
https://issues.apache.org/jira/browse/IGNITE-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766326#comment-17766326
]
Ilya Shishkov edited comment on IGNITE-10516 at 9/20/23 1:08 PM:
-
As I see, problem is still actual both for Calcite and H2 engines, because it
does not relate to SQL engine at all. Problem is caused during processing of
{{SchemaFinishDiscoveryMessage}} in
{{GridQueryProcessor#onSchemaFinishDiscovery}} [1]: cache configuration is
saved always if there are no errors.
According to [2], SQL indexes was not described by standard. Also I did not
find any mention of index command in SQL-99 specification [3, 4]
Links:
#
https://github.com/apache/ignite/blob/9b5d30c8556323a41ea18a3e3f326aa745322ba2/modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryProcessor.java#L884
# https://en.wikipedia.org/wiki/Database_index#Standardization
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-1-1999.pdf
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-2-1999.pdf
was (Author: shishkovilja):
As I see, problem is still actual both for Calcite and H2 engines, because it
does not relate to SQL engine at all. Problem is caused by processing of
{{SchemaFinishDiscoveryMessage}} in
{{GridQueryProcessor#onSchemaFinishDiscovery}} [1].
According to [2], SQL indexes was not described by standard. Also I did not
find any mention of index command in SQL-99 specification [3, 4]
Links:
#
https://github.com/apache/ignite/blob/9b5d30c8556323a41ea18a3e3f326aa745322ba2/modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryProcessor.java#L884
# https://en.wikipedia.org/wiki/Database_index#Standardization
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-1-1999.pdf
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-2-1999.pdf
> Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables
> -
>
> Key: IGNITE-10516
> URL: https://issues.apache.org/jira/browse/IGNITE-10516
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Stanislav Lukyanov
>Assignee: Ilya Shishkov
>Priority: Major
> Labels: ise
>
> Given two tables in the same schema, we can't create an index with the same
> name for both tables. In other words, the following code leads to an error -
> which is good.
> {code}
> CREATE INDEX IDX on T1 (COL);
> CREATE INDEX IDX on T2 (COL);
> {code}
> If used with `IF NOT EXISTS`, the queries pass. It might be OK or not - one
> needs to look into SQL spec to check if the second operation should be a
> no-op (because IDX exists) or fail (because IDX exists for a different table,
> so the caller is probably doing something wrong)
> {code}
> CREATE INDEX IDX on T1 (COL);
> CREATE INDEX IF NOT EXISTS IDX on T2 (COL);
> {code}
> However, if persistence is enabled, the node will fail to restart complaining
> about duplicate index names.
> {code}
> class org.apache.ignite.IgniteCheckedException: Duplicate index name
> [cache=SQL_PUBLIC_T2, schemaName=PUBLIC, idxName=IDX, existingTable=T,
> table=T2]
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1183)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2040)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1732)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:959)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:900)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:888)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:854)
> at
> org.apache.ignite.IndexWithSameNameTest.test(IndexWithSameNameTest.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.access$001(GridAbstractTest.java:150)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.evaluate(GridAbstractTest.java:2104)
> at
>
[jira] [Updated] (IGNITE-20457) Verify commitTimestamp against enlisted partitions expiration timestamps
[ https://issues.apache.org/jira/browse/IGNITE-20457?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-20457: - Priority: Blocker (was: Major) > Verify commitTimestamp against enlisted partitions expiration timestamps > > > Key: IGNITE-20457 > URL: https://issues.apache.org/jira/browse/IGNITE-20457 > Project: Ignite > Issue Type: Bug >Reporter: Alexander Lapin >Priority: Blocker > Labels: ignite-3, tech-debt > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20441) ItRebalanceRecoveryTest is flaky
[
https://issues.apache.org/jira/browse/IGNITE-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-20441:
-
Description:
{{org.apache.ignite.internal.rebalance.ItRebalanceRecoveryTest}} is flaky on
TC, see [TC
history|https://ci.ignite.apache.org/test/5525445229026025351?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true=%3Cdefault%3E].
After some invsetigation, I found out that the problem is the following:
# Imagine two threads: A and B.
# Thread A executes an update from the test. Primary replica generates a
timestamp, creates a Raft command and tries to apply it, but the thread stalls
for any reason.
# Thread B performs idle safe time sync. Primary replica generates a timestamp
(larger than the timestamp from the previous step), creates a Raft command and
successfully applies it.
# Thread A resumes its execution and applies its command. This means that the
Raft command from thread B will be applied before the command from thread A,
despite their timestamps being ordered differently.
# According to the test protocol, a node gets restarted and needs to apply the
missing Raft log part, which means re-applying the two commands above. However,
the second command (which inserts data) will be ignored, because there's code
in {{PartitionListener}} that ignores storage updates if their timestamp is
smaller than the current timestamp (which got updated by the first command).
Therefore, this bug is definitely caused by IGNITE-20116
was:
{{org.apache.ignite.internal.rebalance.ItRebalanceRecoveryTest}} is flaky on
TC, see [TC
history|https://ci.ignite.apache.org/test/5525445229026025351?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true=%3Cdefault%3E].
After some invsetigation, I found out that the problem is the following:
# Imagine two threads: A and B.
# Thread A executes an update from the test. Primary replica generates a
timestamp, creates a Raft command and tries to apply it, but the thread stalls
for any reason.
# Thread B performs idle safe time sync. Primary replica generates a timestamp
(larger than the timestamp from the previous step), creates a Raft command and
successfully applies it.
# Thread A resumes its execution and applies its command. This means that the
Raft command from thread B will be applied before the command from thread A,
despite their timestamps being ordered differently.
# According to the test protocol, a node gets restarted and needs to apply the
missing Raft log part, which means re-applying the two commands above. However,
the second command (which inserts data) will be ignored, because there's code
in {{PartitionListener}} that ignores storage updates if their timestamp is
smaller than the current timestamp (which got updated by the first command).
Therefore, this bug is definitely caused by
https://issues.apache.org/jira/browse/IGNITE-20116
> ItRebalanceRecoveryTest is flaky
>
>
> Key: IGNITE-20441
> URL: https://issues.apache.org/jira/browse/IGNITE-20441
> Project: Ignite
> Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Blocker
> Labels: ignite-3
>
> {{org.apache.ignite.internal.rebalance.ItRebalanceRecoveryTest}} is flaky on
> TC, see [TC
> history|https://ci.ignite.apache.org/test/5525445229026025351?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true=%3Cdefault%3E].
> After some invsetigation, I found out that the problem is the following:
> # Imagine two threads: A and B.
> # Thread A executes an update from the test. Primary replica generates a
> timestamp, creates a Raft command and tries to apply it, but the thread
> stalls for any reason.
> # Thread B performs idle safe time sync. Primary replica generates a
> timestamp (larger than the timestamp from the previous step), creates a Raft
> command and successfully applies it.
> # Thread A resumes its execution and applies its command. This means that the
> Raft command from thread B will be applied before the command from thread A,
> despite their timestamps being ordered differently.
> # According to the test protocol, a node gets restarted and needs to apply
> the missing Raft log part, which means re-applying the two commands above.
> However, the second command (which inserts data) will be ignored, because
> there's code in {{PartitionListener}} that ignores storage updates if their
> timestamp is smaller than the current timestamp (which got updated by the
> first command).
>
> Therefore, this bug is definitely caused by IGNITE-20116
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20361) Implement the table storage description
[ https://issues.apache.org/jira/browse/IGNITE-20361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-20361: Description: *Motivation* According to IGNITE-20357 we need to have an appropriate table storage configuration, which can be used on the table create/alter to check if the chosen zone has appropriate storage requirements. *Definition of done* - Table has the storage configuration, which can be used for early validation if table can be "deployed" in its zone correctly. - Data storage configuration removed from zone configuration *Notes* - Avoid altering of this field for now with appropriate exception was: *Motivation* According to IGNITE-20357 we need to have an appropriate table storage configuration, which can be used on the table create/alter to check if the chosen zone has appropriate storage requirements. *Definition of done* - Table has the storage configuration, which can be used for early validation if table can be "deployed" in its zone correctly. - Data storage configuration removed from zone configuration > Implement the table storage description > --- > > Key: IGNITE-20361 > URL: https://issues.apache.org/jira/browse/IGNITE-20361 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Motivation* > According to IGNITE-20357 we need to have an appropriate table storage > configuration, which can be used on the table create/alter to check if the > chosen zone has appropriate storage requirements. > *Definition of done* > - Table has the storage configuration, which can be used for early validation > if table can be "deployed" in its zone correctly. > - Data storage configuration removed from zone configuration > *Notes* > - Avoid altering of this field for now with appropriate exception -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20455) Sql. Test Framework. Add System View support.
[ https://issues.apache.org/jira/browse/IGNITE-20455?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20455: -- Epic Link: IGNITE-20014 > Sql. Test Framework. Add System View support. > -- > > Key: IGNITE-20455 > URL: https://issues.apache.org/jira/browse/IGNITE-20455 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > Implement test system view builders - standalone one > (TestBuilders::systemView()) and one for test cluster > (ClusterBuilder::addSystemView()). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20456) Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable.
[ https://issues.apache.org/jira/browse/IGNITE-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20456: -- Description: TableDescriptor defines methods for accessing table columns (1), data distribution (2), and row type building for DML operations (3). Since methods (1) and (2) are valid for both table and system views, but (3) are only legal for tables, it would be better to move methods (3) to `IgniteTable`. We may also rename `TableDescriptor` as part of this issue, because it is shared for both tables and system views. was: TableDescriptor defines methods for accessing table columns (1), data distribution (2), and row type building for DML operations (3). Since methods (1) and (2) are valid for both table and system views, but (3) are only legal for tables, it would be better to move methods (3) to `IgniteTable`. We may also rename `TableDescriptor`, because it is shared for both tables and system views. > Sql. Move insert/update/delete rows methods from TableDescriptor to > IgniteTable. > > > Key: IGNITE-20456 > URL: https://issues.apache.org/jira/browse/IGNITE-20456 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > > TableDescriptor defines methods for accessing table columns (1), data > distribution (2), and row type building for DML operations (3). > Since methods (1) and (2) are valid for both table and system views, but (3) > are only legal for tables, it would be better to move methods (3) to > `IgniteTable`. > We may also rename `TableDescriptor` as part of this issue, because it is > shared for both tables and system views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-10516) Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables
[
https://issues.apache.org/jira/browse/IGNITE-10516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17766326#comment-17766326
]
Ilya Shishkov edited comment on IGNITE-10516 at 9/20/23 1:13 PM:
-
As I see, problem is still actual both for Calcite and H2 engines, because it
does not relate to SQL engine at all. Problem caused by
{{GridQueryProcessor#onSchemaFinishDiscovery}} [1]: cache configuration is
saved always if there are no errors during index creation on all nodes.
According to [2], SQL indexes was not described by standard. Also I did not
find any mention of index command in SQL-99 specification [3, 4]
Links:
#
https://github.com/apache/ignite/blob/9b5d30c8556323a41ea18a3e3f326aa745322ba2/modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryProcessor.java#L884
# https://en.wikipedia.org/wiki/Database_index#Standardization
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-1-1999.pdf
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-2-1999.pdf
was (Author: shishkovilja):
As I see, problem is still actual both for Calcite and H2 engines, because it
does not relate to SQL engine at all. Problem is caused during processing of
{{SchemaFinishDiscoveryMessage}} in
{{GridQueryProcessor#onSchemaFinishDiscovery}} [1]: cache configuration is
saved always if there are no errors.
According to [2], SQL indexes was not described by standard. Also I did not
find any mention of index command in SQL-99 specification [3, 4]
Links:
#
https://github.com/apache/ignite/blob/9b5d30c8556323a41ea18a3e3f326aa745322ba2/modules/core/src/main/java/org/apache/ignite/internal/processors/query/GridQueryProcessor.java#L884
# https://en.wikipedia.org/wiki/Database_index#Standardization
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-1-1999.pdf
# http://courses.cms.caltech.edu/cs123/sql99std/ansi-iso-9075-2-1999.pdf
> Storage is corrupted after CREATE INDEX IF NOT EXISTS on different tables
> -
>
> Key: IGNITE-10516
> URL: https://issues.apache.org/jira/browse/IGNITE-10516
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.15
>Reporter: Stanislav Lukyanov
>Assignee: Ilya Shishkov
>Priority: Major
> Labels: ise
>
> Given two tables in the same schema, we can't create an index with the same
> name for both tables. In other words, the following code leads to an error -
> which is good.
> {code}
> CREATE INDEX IDX on T1 (COL);
> CREATE INDEX IDX on T2 (COL);
> {code}
> If used with `IF NOT EXISTS`, the queries pass. It might be OK or not - one
> needs to look into SQL spec to check if the second operation should be a
> no-op (because IDX exists) or fail (because IDX exists for a different table,
> so the caller is probably doing something wrong)
> {code}
> CREATE INDEX IDX on T1 (COL);
> CREATE INDEX IF NOT EXISTS IDX on T2 (COL);
> {code}
> However, if persistence is enabled, the node will fail to restart complaining
> about duplicate index names.
> {code}
> class org.apache.ignite.IgniteCheckedException: Duplicate index name
> [cache=SQL_PUBLIC_T2, schemaName=PUBLIC, idxName=IDX, existingTable=T,
> table=T2]
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1183)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2040)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1732)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1158)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:656)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:959)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:900)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:888)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:854)
> at
> org.apache.ignite.IndexWithSameNameTest.test(IndexWithSameNameTest.java:77)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at junit.framework.TestCase.runTest(TestCase.java:176)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.access$001(GridAbstractTest.java:150)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.evaluate(GridAbstractTest.java:2104)
> at
>
[jira] [Updated] (IGNITE-20456) Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable.
[ https://issues.apache.org/jira/browse/IGNITE-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20456: -- Epic Link: IGNITE-20014 > Sql. Move insert/update/delete rows methods from TableDescriptor to > IgniteTable. > > > Key: IGNITE-20456 > URL: https://issues.apache.org/jira/browse/IGNITE-20456 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > TableDescriptor defines methods for accessing table columns (1), data > distribution (2), and row type building for DML operations (3). > Since methods (1) and (2) are valid for both table and system views, but (3) > are only legal for tables, it would be better to move methods (3) to > `IgniteTable`. > We may also rename `TableDescriptor` as part of this issue, because it is > shared for both tables and system views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20441) ItRebalanceRecoveryTest is flaky
[
https://issues.apache.org/jira/browse/IGNITE-20441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-20441:
-
Description:
{{org.apache.ignite.internal.rebalance.ItRebalanceRecoveryTest}} is flaky on
TC, see [TC
history|https://ci.ignite.apache.org/test/5525445229026025351?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true=%3Cdefault%3E].
After some invsetigation, I found out that the problem is the following:
# Imagine two threads: A and B.
# Thread A executes an update from the test. Primary replica generates a
timestamp, creates a Raft command and tries to apply it, but the thread stalls
for any reason.
# Thread B performs idle safe time sync. Primary replica generates a timestamp
(larger than the timestamp from the previous step), creates a Raft command and
successfully applies it.
# Thread A resumes its execution and applies its command. This means that the
Raft command from thread B will be applied before the command from thread A,
despite their timestamps being ordered differently.
# According to the test protocol, a node gets restarted and needs to apply the
missing Raft log part, which means re-applying the two commands above. However,
the second command (which inserts data) will be ignored, because there's code
in {{PartitionListener}} that ignores storage updates if their timestamp is
smaller than the current timestamp (which got updated by the first command).
Therefore, this bug is definitely caused by
https://issues.apache.org/jira/browse/IGNITE-20116
> ItRebalanceRecoveryTest is flaky
>
>
> Key: IGNITE-20441
> URL: https://issues.apache.org/jira/browse/IGNITE-20441
> Project: Ignite
> Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Blocker
> Labels: ignite-3
>
> {{org.apache.ignite.internal.rebalance.ItRebalanceRecoveryTest}} is flaky on
> TC, see [TC
> history|https://ci.ignite.apache.org/test/5525445229026025351?currentProjectId=ApacheIgnite3xGradle_Test_IntegrationTests=true=%3Cdefault%3E].
> After some invsetigation, I found out that the problem is the following:
> # Imagine two threads: A and B.
> # Thread A executes an update from the test. Primary replica generates a
> timestamp, creates a Raft command and tries to apply it, but the thread
> stalls for any reason.
> # Thread B performs idle safe time sync. Primary replica generates a
> timestamp (larger than the timestamp from the previous step), creates a Raft
> command and successfully applies it.
> # Thread A resumes its execution and applies its command. This means that the
> Raft command from thread B will be applied before the command from thread A,
> despite their timestamps being ordered differently.
> # According to the test protocol, a node gets restarted and needs to apply
> the missing Raft log part, which means re-applying the two commands above.
> However, the second command (which inserts data) will be ignored, because
> there's code in {{PartitionListener}} that ignores storage updates if their
> timestamp is smaller than the current timestamp (which got updated by the
> first command).
>
> Therefore, this bug is definitely caused by
> https://issues.apache.org/jira/browse/IGNITE-20116
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15996) Node fails with "Node with the same ID was found" while connecting to the cluster in Docker container if previous container was stopped
[
https://issues.apache.org/jira/browse/IGNITE-15996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767097#comment-17767097
]
Łukasz Bigorajski commented on IGNITE-15996:
I wonder if you are going to fix this problem as I have the same problem on
openshift. If not, I need to consider some workaround.
> Node fails with "Node with the same ID was found" while connecting to the
> cluster in Docker container if previous container was stopped
> ---
>
> Key: IGNITE-15996
> URL: https://issues.apache.org/jira/browse/IGNITE-15996
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.10
> Environment: Windows 10, Docker+WSL2
>Reporter: Ksenia Rybakova
>Priority: Major
> Attachments: ignite-47b5227b.0.log, ignite-c072978e.0.log,
> ignite-c62bc58e.0.log, image-2023-09-20-13-57-30-131.png
>
>
> Node in Docker container fails to connect to existing cluster if previously
> connected node (container) was stopped:
> {noformat}
> [11:27:38,272][SEVERE][main][IgniteKernal] Got exception while starting (will
> rollback startup routine).
> class org.apache.ignite.IgniteCheckedException: Failed to start manager:
> GridManagerAdapter [enabled=true,
> name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1990)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1331)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2141)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1787)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1172)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1066)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:952)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
> at org.apache.ignite.Ignition.start(Ignition.java:353)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start
> SPI: TcpDiscoverySpi [addrRslvr=null, addressFilter=null, sockTimeout=5000,
> ackTimeout=5000, marsh=JdkMarshaller
> [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@21f9277b],
> reconCnt=10, reconDelay=2000, maxAckTimeout=60, soLinger=0,
> forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null,
> skipAddrsRandomization=false]
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:281)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:980)
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1985)
> ... 11 more
> Caused by: class org.apache.ignite.spi.IgniteSpiException: Node with the same
> ID was found in node IDs history or existing node in topology has the same ID
> (fix configuration and restart local node) [localNode=TcpDiscoveryNode
> [id=c62bc58e-102a-4928-8e54-ac8a56bf4d44,
> consistentId=127.0.0.1,172.17.0.4:47500, addrs=ArrayList [127.0.0.1,
> 172.17.0.4], sockAddrs=HashSet [402b337a50dd/172.17.0.4:47500,
> /127.0.0.1:47500], discPort=47500, order=0, intOrder=3,
> lastExchangeTime=1637839658247, loc=true, ver=2.11.0#20210911-sha1:8f3f07d3,
> isClient=false], existingNode=c62bc58e-102a-4928-8e54-ac8a56bf4d44]
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.duplicateIdError(TcpDiscoverySpi.java:2083)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1201)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:473)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2207)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:278)
> ... 13 more{noformat}
> Steps to reproduce:
> 1) Download ignite Docker image
> {code:java}
> docker pull apacheignite/ignite:2.11.0{code}
> 2) Start node 1 (local directory is mounted to save logs)
> {code:java}
> docker run -d -v ${PWD}/docker_ignite_w1:/opt/ignite/apache-ignite/work
> apacheignite/ignite:2.11.0
> c5219b095c93ec56731eec9fa871ffb722ddead987256198d76889f4a1a8ea3e{code}
> 3) Start node 2
> {code:java}
> docker run -d -v ${PWD}/docker_ignite_w2:/opt/ignite/apache-ignite/work
> apacheignite/ignite:2.11.0
>
[jira] [Updated] (IGNITE-20460) Too many files with unapproved license
[
https://issues.apache.org/jira/browse/IGNITE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Gusev updated IGNITE-20460:
Component/s: documentation
> Too many files with unapproved license
> --
>
> Key: IGNITE-20460
> URL: https://issues.apache.org/jira/browse/IGNITE-20460
> Project: Ignite
> Issue Type: Bug
> Components: build, documentation
>Reporter: Artem Egorov
>Assignee: Igor Gusev
>Priority: Critical
>
> After merging IGNITE-20431, the project build began to fail with an "Too many
> files with unapproved license" error:
> {quote}1 Unknown Licenses
> * Files with unapproved
> licenses: docs/_docs/installation/upgrades.adoc
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-20456) Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable.
[ https://issues.apache.org/jira/browse/IGNITE-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20456: -- Fix Version/s: 3.0.0-beta2 > Sql. Move insert/update/delete rows methods from TableDescriptor to > IgniteTable. > > > Key: IGNITE-20456 > URL: https://issues.apache.org/jira/browse/IGNITE-20456 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Fix For: 3.0.0-beta2 > > > TableDescriptor defines methods for accessing table columns (1), data > distribution (2), and row type building for DML operations (3). > Since methods (1) and (2) are valid for both table and system views, but (3) > are only legal for tables, it would be better to move methods (3) to > `IgniteTable`. > We may also rename `TableDescriptor` as part of this issue, because it is > shared for both tables and system views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-20460) Too many files with unapproved license
[
https://issues.apache.org/jira/browse/IGNITE-20460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17767098#comment-17767098
]
Igor Gusev commented on IGNITE-20460:
-
[~artyeshi] please review.
> Too many files with unapproved license
> --
>
> Key: IGNITE-20460
> URL: https://issues.apache.org/jira/browse/IGNITE-20460
> Project: Ignite
> Issue Type: Bug
> Components: build, documentation
>Reporter: Artem Egorov
>Assignee: Igor Gusev
>Priority: Critical
>
> After merging IGNITE-20431, the project build began to fail with an "Too many
> files with unapproved license" error:
> {quote}1 Unknown Licenses
> * Files with unapproved
> licenses: docs/_docs/installation/upgrades.adoc
> {quote}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-15996) Node fails with "Node with the same ID was found" while connecting to the cluster in Docker container if previous container was stopped
[
https://issues.apache.org/jira/browse/IGNITE-15996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Łukasz Bigorajski updated IGNITE-15996:
---
Attachment: image-2023-09-20-13-57-30-131.png
> Node fails with "Node with the same ID was found" while connecting to the
> cluster in Docker container if previous container was stopped
> ---
>
> Key: IGNITE-15996
> URL: https://issues.apache.org/jira/browse/IGNITE-15996
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 2.10
> Environment: Windows 10, Docker+WSL2
>Reporter: Ksenia Rybakova
>Priority: Major
> Attachments: ignite-47b5227b.0.log, ignite-c072978e.0.log,
> ignite-c62bc58e.0.log, image-2023-09-20-13-57-30-131.png
>
>
> Node in Docker container fails to connect to existing cluster if previously
> connected node (container) was stopped:
> {noformat}
> [11:27:38,272][SEVERE][main][IgniteKernal] Got exception while starting (will
> rollback startup routine).
> class org.apache.ignite.IgniteCheckedException: Failed to start manager:
> GridManagerAdapter [enabled=true,
> name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1990)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1331)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2141)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1787)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1172)
> at
> org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1066)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:952)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:851)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:721)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:690)
> at org.apache.ignite.Ignition.start(Ignition.java:353)
> at
> org.apache.ignite.startup.cmdline.CommandLineStartup.main(CommandLineStartup.java:367)
> Caused by: class org.apache.ignite.IgniteCheckedException: Failed to start
> SPI: TcpDiscoverySpi [addrRslvr=null, addressFilter=null, sockTimeout=5000,
> ackTimeout=5000, marsh=JdkMarshaller
> [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@21f9277b],
> reconCnt=10, reconDelay=2000, maxAckTimeout=60, soLinger=0,
> forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null,
> skipAddrsRandomization=false]
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:281)
> at
> org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:980)
> at
> org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1985)
> ... 11 more
> Caused by: class org.apache.ignite.spi.IgniteSpiException: Node with the same
> ID was found in node IDs history or existing node in topology has the same ID
> (fix configuration and restart local node) [localNode=TcpDiscoveryNode
> [id=c62bc58e-102a-4928-8e54-ac8a56bf4d44,
> consistentId=127.0.0.1,172.17.0.4:47500, addrs=ArrayList [127.0.0.1,
> 172.17.0.4], sockAddrs=HashSet [402b337a50dd/172.17.0.4:47500,
> /127.0.0.1:47500], discPort=47500, order=0, intOrder=3,
> lastExchangeTime=1637839658247, loc=true, ver=2.11.0#20210911-sha1:8f3f07d3,
> isClient=false], existingNode=c62bc58e-102a-4928-8e54-ac8a56bf4d44]
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.duplicateIdError(TcpDiscoverySpi.java:2083)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.joinTopology(ServerImpl.java:1201)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl.spiStart(ServerImpl.java:473)
> at
> org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.spiStart(TcpDiscoverySpi.java:2207)
> at
> org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:278)
> ... 13 more{noformat}
> Steps to reproduce:
> 1) Download ignite Docker image
> {code:java}
> docker pull apacheignite/ignite:2.11.0{code}
> 2) Start node 1 (local directory is mounted to save logs)
> {code:java}
> docker run -d -v ${PWD}/docker_ignite_w1:/opt/ignite/apache-ignite/work
> apacheignite/ignite:2.11.0
> c5219b095c93ec56731eec9fa871ffb722ddead987256198d76889f4a1a8ea3e{code}
> 3) Start node 2
> {code:java}
> docker run -d -v ${PWD}/docker_ignite_w2:/opt/ignite/apache-ignite/work
> apacheignite/ignite:2.11.0
> 65fdae68a40b2d3d17ab7e560320ef6757713d8efacbc25a26aecca03be6f975{code}
> 4) Stop container for node 2
> {code:java}
> docker stop 65fdae68a40b{code}
> 5) Start
[jira] [Updated] (IGNITE-20456) Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable.
[ https://issues.apache.org/jira/browse/IGNITE-20456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maksim Zhuravkov updated IGNITE-20456: -- Labels: ignite-3 (was: ) > Sql. Move insert/update/delete rows methods from TableDescriptor to > IgniteTable. > > > Key: IGNITE-20456 > URL: https://issues.apache.org/jira/browse/IGNITE-20456 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Maksim Zhuravkov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > TableDescriptor defines methods for accessing table columns (1), data > distribution (2), and row type building for DML operations (3). > Since methods (1) and (2) are valid for both table and system views, but (3) > are only legal for tables, it would be better to move methods (3) to > `IgniteTable`. > We may also rename `TableDescriptor` as part of this issue, because it is > shared for both tables and system views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-20456) Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable.
Maksim Zhuravkov created IGNITE-20456: - Summary: Sql. Move insert/update/delete rows methods from TableDescriptor to IgniteTable. Key: IGNITE-20456 URL: https://issues.apache.org/jira/browse/IGNITE-20456 Project: Ignite Issue Type: Improvement Components: sql Reporter: Maksim Zhuravkov TableDescriptor defines methods for accessing table columns (1), data distribution (2), and row type building for DML operations (3). Since methods (1) and (2) are valid for both table and system views, but (3) are only legal for tables, it would be better to move methods (3) to `IgniteTable`. We may also rename `TableDescriptor`, because it is shared for both tables and system views. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20420) Streamer metrics missing in ClientMetricSource.Holder.metrics
[ https://issues.apache.org/jira/browse/IGNITE-20420?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-20420: Description: *ClientMetricSource.Holder.metrics* does not have streamer metrics (*streamerBatchesSent* etc). Add a test to check that all metrics are registered (via reflection). was: ClientMetricSource.Holder.metrics does not have streamer metrics (streamerBatchesSent etc). Add a test to check that all metrics are registered (via reflection). > Streamer metrics missing in ClientMetricSource.Holder.metrics > - > > Key: IGNITE-20420 > URL: https://issues.apache.org/jira/browse/IGNITE-20420 > Project: Ignite > Issue Type: Bug > Components: thin client >Affects Versions: 3.0.0-beta1 >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > > *ClientMetricSource.Holder.metrics* does not have streamer metrics > (*streamerBatchesSent* etc). > Add a test to check that all metrics are registered (via reflection). -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-20116) Linearize storage updates with safeTime adjustment rules
[
https://issues.apache.org/jira/browse/IGNITE-20116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-20116:
-
Description:
h3. Motivation
The logic of setting safeTime explicitly prohibits setting a larger time ahead
of a smaller one. In other words, all data updates within storages should be
strictly ordered by the safeTime associated with such updates. Currently it's
not true:
* We associate update and safe time during update command creation (see
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener)
{code:java}
UpdateCommandBuilder bldr = MSG_FACTORY.updateCommand()
...
.safeTimeLong(hybridClock.nowLong()); {code}
* However, neither applying a given command locally nor sending it to the raft
isn't linearized with associated safeTime value. In other words, it's possible
that we will assign t0 to the cmd0 and t1 to the cmd1 but will apply cmd1 prior
to cmd0 locally.
Simply speaking, we lack some sort of synchronization here.
h3. Definition of Done
* It's required to linearize updates application to preserve guarantees of the
monotonicity of a safeTime's adjustment.
h3. Implementation Notes
Different options are possible:
# We may reject a command that is associated with safeTime < already applied
one. Such approach requires
## To resend the command with new safeTime in case of 1pc.
## Adjust local safeTime, and resend command with new safe time in case of 2pc.
# Add proper synchronization both on client and server side.
# Send pending safeTime instances with each command. More details below:
Let’s assume that there were two updateCommands cmd1(safeTime: t1) and
cmd2(safeTime: t2). Let’s also assume that cmd2 was send prior to cmd2 (meaning
that it was reordered). In that case, assuming that cmd2 has both t1 and t2
within its data bag, it will wait for cmd1 to bring it data in a queue or
formally it will wait previous commands to apply themselves.
was:
h3. Motivation
The logic of setting safeTime explicitly prohibits setting a larger time ahead
of a smaller one. In other words, all data updates within storages should be
strictly ordered by the safeTime associated with such updates. Currently it's
not true:
* We associate update and safe time during update command creation (see
org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener)
{code:java}
UpdateCommandBuilder bldr = MSG_FACTORY.updateCommand()
...
.safeTimeLong(hybridClock.nowLong()); {code}
* However, neither applying a given command locally nor sending it to the raft
isn't linearized with associated safeTime value. In other words, it's possible
that we will assign t0 to the cmd0 and t1 to the cmd1 but will apply cmd1 prior
to cmd0 locally.
Simply speaking, we lack some sort of synchronization here.
h3. Definition of Done
* It's required to linearize updates application to preserve guarantees of the
monotonicity of a safeTime's adjustment.
> Linearize storage updates with safeTime adjustment rules
>
>
> Key: IGNITE-20116
> URL: https://issues.apache.org/jira/browse/IGNITE-20116
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Priority: Blocker
> Labels: ignite-3, transactions
>
> h3. Motivation
> The logic of setting safeTime explicitly prohibits setting a larger time
> ahead of a smaller one. In other words, all data updates within storages
> should be strictly ordered by the safeTime associated with such updates.
> Currently it's not true:
> * We associate update and safe time during update command creation (see
> org.apache.ignite.internal.table.distributed.replicator.PartitionReplicaListener)
> {code:java}
> UpdateCommandBuilder bldr = MSG_FACTORY.updateCommand()
> ...
> .safeTimeLong(hybridClock.nowLong()); {code}
> * However, neither applying a given command locally nor sending it to the
> raft isn't linearized with associated safeTime value. In other words, it's
> possible that we will assign t0 to the cmd0 and t1 to the cmd1 but will apply
> cmd1 prior to cmd0 locally.
> Simply speaking, we lack some sort of synchronization here.
> h3. Definition of Done
> * It's required to linearize updates application to preserve guarantees of
> the monotonicity of a safeTime's adjustment.
> h3. Implementation Notes
> Different options are possible:
> # We may reject a command that is associated with safeTime < already applied
> one. Such approach requires
> ## To resend the command with new safeTime in case of 1pc.
> ## Adjust local safeTime, and resend command with new safe time in case of
> 2pc.
> # Add proper synchronization both on client and server side.
> # Send
[jira] [Updated] (IGNITE-20360) Implement the set of zone supported storages
[ https://issues.apache.org/jira/browse/IGNITE-20360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-20360: Summary: Implement the set of zone supported storages (was: Implement the zone filter for storage types and profiles) > Implement the set of zone supported storages > > > Key: IGNITE-20360 > URL: https://issues.apache.org/jira/browse/IGNITE-20360 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Motivation* > According to IGNITE-20357 we need to have an appropriate zone filter, which > filters the nodes based on their available storages. > *Definition of done* > - Zone has the filters, which can be unambiguously used to check if table can > be "deployed" in this zone -- This message was sent by Atlassian Jira (v8.20.10#820010)
