[
https://issues.apache.org/jira/browse/IGNITE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexey Scherbakov updated IGNITE-21077:
---------------------------------------
Description:
Currently KV and SQL engines use different approach to handling transactions,
which leads to different execution paths. As a consequence, performance
improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to
KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data
using KV API and starts nested transaction having the ID of a parent
transaction.
# Nested transaction can'be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE
step. On PREPARE step query initiator sends prepare request to each node
containing nested transaction, which is ACKED when a nested txn is safe to
commit.
# On receiving all acks parent transaction sends commit request to commit
partition. Commit partition does async cleanup
# PREPARE step introduces additional latency, so it's always good to try to
map query to a single execution node. In this case PREPARE step can be avoided.
For example, SQL transaction which can build the result set on a single node,
can be always executed from a initiator. This is not applicable, for example,
to grouping queries.
The same approach can be used to implement things like transactional Compute.
was:
Currently KV and SQL engines use different approach to handling transactions,
which leads to different execution paths. As a consequence, performance
improvements for KV does not apply to SQL.
The problem here is due to distributed nature of SQL execution, in contrary to
KV API, which can be coordinated from a single txn coordinator.
The idea is to introduce nested transactions, which works in the following way:
# Query initiator starts parent transaction.
# SQL engine should always use KV api to access data for further SQL execution.
# If SQL query is mapped to multiple execution nodes, each node accesses data
using KV API and starts nested transaction having the ID of a parent
transaction.
# Nested transaction can'be directly committed or rolled back.
# If at least one nested transaction is created, we need additional PREPARE
step. On PREPARE step query initiator sends prepare request to each node
containing nested transaction, which is ACKED when a nested txn is safe to
commit.
# On receiving all acks parent transaction sends commit request to commit
partition.
# PREPARE step introduces additional latency, so it's always OK to map query
to a single execution node. For example, SQL transaction containing only
unicast operations, can be always executed from a initiator, so PREPARE step is
not required.
The same approach can be used to implement things like transactional Compute.
> Unify transactions for SQL and KV layers
> ----------------------------------------
>
> Key: IGNITE-21077
> URL: https://issues.apache.org/jira/browse/IGNITE-21077
> Project: Ignite
> Issue Type: Improvement
> Affects Versions: 3.0
> Reporter: Alexey Scherbakov
> Priority: Major
> Labels: ignite-3
> Fix For: 3.0
>
>
> Currently KV and SQL engines use different approach to handling transactions,
> which leads to different execution paths. As a consequence, performance
> improvements for KV does not apply to SQL.
> The problem here is due to distributed nature of SQL execution, in contrary
> to KV API, which can be coordinated from a single txn coordinator.
> The idea is to introduce nested transactions, which works in the following
> way:
> # Query initiator starts parent transaction.
> # SQL engine should always use KV api to access data for further SQL
> execution.
> # If SQL query is mapped to multiple execution nodes, each node accesses
> data using KV API and starts nested transaction having the ID of a parent
> transaction.
> # Nested transaction can'be directly committed or rolled back.
> # If at least one nested transaction is created, we need additional PREPARE
> step. On PREPARE step query initiator sends prepare request to each node
> containing nested transaction, which is ACKED when a nested txn is safe to
> commit.
> # On receiving all acks parent transaction sends commit request to commit
> partition. Commit partition does async cleanup
> # PREPARE step introduces additional latency, so it's always good to try to
> map query to a single execution node. In this case PREPARE step can be
> avoided. For example, SQL transaction which can build the result set on a
> single node, can be always executed from a initiator. This is not applicable,
> for example, to grouping queries.
> The same approach can be used to implement things like transactional Compute.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
