[
https://issues.apache.org/jira/browse/IGNITE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Scherbakov updated IGNITE-21077:
---------------------------------------
Description:
Currently KV and SQL engines use different approach to handling transactions,
which leads to different execution paths. As a consequence, performance
improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to
KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data
using KV API and starts nested transaction having the ID of a parent
transaction.
# Nested transaction can't be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE
step. On PREPARE step query initiator sends prepare request to each node
containing nested transaction, which is ACKED when a nested txn is safe to
commit.
# On receiving all acks parent transaction sends commit request to commit
partition. Commit partition does async cleanup
# PREPARE step introduces additional latency, so it's always good to try to
map query to a single execution node. In this case PREPARE step can be avoided.
Thich is true for any SQL transaction which can be executed by calling only KV
operations. This is not applicable, for example, to grouping queries.
The same approach can be used to implement things like transactional Compute.
was:
Currently KV and SQL engines use different approach to handling transactions,
which leads to different execution paths. As a consequence, performance
improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to
KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data
using KV API and starts nested transaction having the ID of a parent
transaction.
# Nested transaction can'be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE
step. On PREPARE step query initiator sends prepare request to each node
containing nested transaction, which is ACKED when a nested txn is safe to
commit.
# On receiving all acks parent transaction sends commit request to commit
partition. Commit partition does async cleanup
# PREPARE step introduces additional latency, so it's always good to try to
map query to a single execution node. In this case PREPARE step can be avoided.
Thich is true for any SQL transaction which can be executed by calling only KV
operations. This is not applicable, for example, to grouping queries.
The same approach can be used to implement things like transactional Compute.
> Unify transactions for SQL and KV layers
> ----------------------------------------
>
> Key: IGNITE-21077
> URL: https://issues.apache.org/jira/browse/IGNITE-21077
> Project: Ignite
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Alexey Scherbakov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0
>
>
> Currently KV and SQL engines use different approach to handling transactions,
> which leads to different execution paths. As a consequence, performance
> improvements for KV does not apply to SQL.
> The problem here is due to distributed nature of SQL execution, in contrary
> to KV API, which can be coordinated from a single txn coordinator.
> The idea is to introduce nested transactions, which works in the following
> way:
> # Query initiator starts parent transaction.
> # SQL engine should always use KV api to access data for further SQL
> execution.
> # If SQL query is mapped to multiple execution nodes, each node accesses
> data using KV API and starts nested transaction having the ID of a parent
> transaction.
> # Nested transaction can't be directly committed or rolled back.
> # If at least one nested transaction is created, we need additional PREPARE
> step. On PREPARE step query initiator sends prepare request to each node
> containing nested transaction, which is ACKED when a nested txn is safe to
> commit.
> # On receiving all acks parent transaction sends commit request to commit
> partition. Commit partition does async cleanup
> # PREPARE step introduces additional latency, so it's always good to try to
> map query to a single execution node. In this case PREPARE step can be
> avoided. Thich is true for any SQL transaction which can be executed by
> calling only KV operations. This is not applicable, for example, to grouping
> queries.
> The same approach can be used to implement things like transactional Compute.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
