[jira] [Created] (IGNITE-18013) Unmute tests after IGNITE-17968
Semyon Danilov created IGNITE-18013: --- Summary: Unmute tests after IGNITE-17968 Key: IGNITE-18013 URL: https://issues.apache.org/jira/browse/IGNITE-18013 Project: Ignite Issue Type: Improvement Reporter: Semyon Danilov Assignee: Semyon Danilov Fix For: 3.0.0-beta2 Need to unmute tests muted for IGNITE-17968 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-18012) Add connect command to start CLI in REPL mode
Vadim Pakhnushev created IGNITE-18012:
-
Summary: Add connect command to start CLI in REPL mode
Key: IGNITE-18012
URL: https://issues.apache.org/jira/browse/IGNITE-18012
Project: Ignite
Issue Type: Task
Components: cli
Reporter: Vadim Pakhnushev
Assignee: Vadim Pakhnushev
Allow user to run a CLI in REPL mode starting in connected state with specified
node like
{{ignite3 connect http://localhost:10300}}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-18011) Avoid hacky way to get LogManager for streaming RAFT snapshot reading
Roman Puchkovskiy created IGNITE-18011: -- Summary: Avoid hacky way to get LogManager for streaming RAFT snapshot reading Key: IGNITE-18011 URL: https://issues.apache.org/jira/browse/IGNITE-18011 Project: Ignite Issue Type: Improvement Components: persistence Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 3.0.0-beta2 TBD -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-18009) Fix gradle build
[ https://issues.apache.org/jira/browse/IGNITE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov resolved IGNITE-18009. - Fix Version/s: 3.0.0-beta2 Resolution: Fixed LGTM, build is green again. Thank your for the contribution, merged to the main branch. > Fix gradle build > > > Key: IGNITE-18009 > URL: https://issues.apache.org/jira/browse/IGNITE-18009 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18009) Fix gradle build
[ https://issues.apache.org/jira/browse/IGNITE-18009?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov updated IGNITE-18009: Reviewer: Semyon Danilov > Fix gradle build > > > Key: IGNITE-18009 > URL: https://issues.apache.org/jira/browse/IGNITE-18009 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-18010) Migrate control.sh to IgniteLogger
[ https://issues.apache.org/jira/browse/IGNITE-18010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov reassigned IGNITE-18010: Assignee: Nikolay Izhikov > Migrate control.sh to IgniteLogger > -- > > Key: IGNITE-18010 > URL: https://issues.apache.org/jira/browse/IGNITE-18010 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-81 > > Right now control.sh utility uses own logging mechanism built on > java.util.logging while other parts of Ignite uses IgniteLogger facade. > It will be more convenient and consistent to use IgniteLogger in control.sh > code too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-18010) Migrate control.sh to IgniteLogger
Nikolay Izhikov created IGNITE-18010: Summary: Migrate control.sh to IgniteLogger Key: IGNITE-18010 URL: https://issues.apache.org/jira/browse/IGNITE-18010 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Right now control.sh utility uses own logging mechanism built on java.util.logging while other parts of Ignite uses IgniteLogger facade. It will be more convenient and consistent to use IgniteLogger in control.sh code too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15629) Cluster Management API
[ https://issues.apache.org/jira/browse/IGNITE-15629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-15629: - Labels: IEP-81 important ise (was: iep-81 important ise) > Cluster Management API > -- > > Key: IGNITE-15629 > URL: https://issues.apache.org/jira/browse/IGNITE-15629 > Project: Ignite > Issue Type: Task >Reporter: Maxim Muzafarov >Assignee: Nikolay Izhikov >Priority: Major > Labels: IEP-81, important, ise > > Ignite must provide cluster management internal API in that way the other > available user APIs like REST, CLI or JMX won't require the source code > changes. > The following features must be available out of the box: > - man pages and documentation autogeneration > - registering new commands from plugins -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-18009) Fix gradle build
Mikhail Pochatkin created IGNITE-18009: -- Summary: Fix gradle build Key: IGNITE-18009 URL: https://issues.apache.org/jira/browse/IGNITE-18009 Project: Ignite Issue Type: Bug Components: build Reporter: Mikhail Pochatkin Assignee: Mikhail Pochatkin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17302) Raft listener multi-storage support
[ https://issues.apache.org/jira/browse/IGNITE-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625672#comment-17625672 ] Alexander Lapin commented on IGNITE-17302: -- [~maliev] LGTM from my side. > Raft listener multi-storage support > > > Key: IGNITE-17302 > URL: https://issues.apache.org/jira/browse/IGNITE-17302 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3, transaction3_rw > Fix For: 3.0.0-beta2 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > It's supposed that replication state machine (e.g. RaftGroupListener) will > handle multiple storages. For example PartitionListener will use: > * MvPartitionStorage - common multi-version partition storage for core > partition data. > * Both volatile and persistent lock storages IGNITE-15932. > * Persistent storage for txn state IGNITE-15931. > * Index storages. > Generally speaking replication logic is decoupled from data-storages, and > there are only few intersection points: > * Raft log truncation - in order to safely truncate raft log it's required > to take into consideration all checkpointIndexes for all storages of the > given state machine and perform min(storage1.checkpointIndex, > storage2.checkpointIndex, ... ) truncation. Besides that it's important for > storages to skip already applied commands on raft log replay - storage > indexes to the rescue. See IGNITE-16907 for more details. Such modifications > will also effect PartitionListener.onSnapshotLoad()/onSnapshotSave() methods. > * It's required to update local node recovery process a bit: a node must > enter the logical topology only after restoring all local raft nodes, > including both restoring storages from corresponding snapshot and applying > raft log. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-18005:
-
Reviewer: Alexander Lapin
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17967) RO writeIntent resolution tests hang up in case of multi node cluster
[
https://issues.apache.org/jira/browse/IGNITE-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-17967:
-
Reviewer: Alexander Lapin
> RO writeIntent resolution tests hang up in case of multi node cluster
> -
>
> Key: IGNITE-17967
> URL: https://issues.apache.org/jira/browse/IGNITE-17967
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Labels: transaction3_ro
> Fix For: 3.0.0-beta1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> TxAbstractTest#testReadOnlyGetWriteIntentResolutionUpdate{code}
> works in case of single node cluster but hangs up in case of multi node one.
> It also worth to mention that simple get tests without writeIntent resolution
> e.g.
> {code:java}
> TxAbstractTest#testReadOnlyGet{code}
> works both on sinle and multi node cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17967) RO writeIntent resolution tests hang up in case of multi node cluster
[
https://issues.apache.org/jira/browse/IGNITE-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625658#comment-17625658
]
Alexander Lapin commented on IGNITE-17967:
--
[~v.pyatkov] Besides minor comments, that we've discussed -LGTM.
> RO writeIntent resolution tests hang up in case of multi node cluster
> -
>
> Key: IGNITE-17967
> URL: https://issues.apache.org/jira/browse/IGNITE-17967
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Labels: transaction3_ro
> Fix For: 3.0.0-beta1
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> TxAbstractTest#testReadOnlyGetWriteIntentResolutionUpdate{code}
> works in case of single node cluster but hangs up in case of multi node one.
> It also worth to mention that simple get tests without writeIntent resolution
> e.g.
> {code:java}
> TxAbstractTest#testReadOnlyGet{code}
> works both on sinle and multi node cluster.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17965) Enable remote build cache for Gradle
[ https://issues.apache.org/jira/browse/IGNITE-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov updated IGNITE-17965: Fix Version/s: 3.0.0-beta2 Reviewer: Semyon Danilov > Enable remote build cache for Gradle > > > Key: IGNITE-17965 > URL: https://issues.apache.org/jira/browse/IGNITE-17965 > Project: Ignite > Issue Type: Improvement > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 20m > Remaining Estimate: 0h > > Need to enable remote build cache feature for Gradle on CI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-17965) Enable remote build cache for Gradle
[ https://issues.apache.org/jira/browse/IGNITE-17965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov resolved IGNITE-17965. - Resolution: Fixed The patch looks good to me. Thank you for the contribution, merged to the main branch. > Enable remote build cache for Gradle > > > Key: IGNITE-17965 > URL: https://issues.apache.org/jira/browse/IGNITE-17965 > Project: Ignite > Issue Type: Improvement > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 20m > Remaining Estimate: 0h > > Need to enable remote build cache feature for Gradle on CI. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (IGNITE-17966) Fix problem with stuck Gradle processes in .NET tests
[ https://issues.apache.org/jira/browse/IGNITE-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov resolved IGNITE-17966. - Fix Version/s: 3.0.0-beta2 Resolution: Fixed The patch looks good to me. Thank you for the contribution, merged to the main branch. > Fix problem with stuck Gradle processes in .NET tests > - > > Key: IGNITE-17966 > URL: https://issues.apache.org/jira/browse/IGNITE-17966 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Blocker > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently, when .NET test suite finished some Gradle processes will alive and > after that CI agents start be in inconsistance state. The main problem is > using ports and next build will be failed because of unreachable port. So, > need to fix this problem and kill all processes which producing in build > time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17966) Fix problem with stuck Gradle processes in .NET tests
[ https://issues.apache.org/jira/browse/IGNITE-17966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov updated IGNITE-17966: Reviewer: Semyon Danilov > Fix problem with stuck Gradle processes in .NET tests > - > > Key: IGNITE-17966 > URL: https://issues.apache.org/jira/browse/IGNITE-17966 > Project: Ignite > Issue Type: Bug > Components: build >Reporter: Mikhail Pochatkin >Assignee: Mikhail Pochatkin >Priority: Blocker > Labels: ignite-3 > Time Spent: 10m > Remaining Estimate: 0h > > Currently, when .NET test suite finished some Gradle processes will alive and > after that CI agents start be in inconsistance state. The main problem is > using ports and next build will be failed because of unreachable port. So, > need to fix this problem and kill all processes which producing in build > time. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17990) Download RAFT snapshot without deleting original partition data
[
https://issues.apache.org/jira/browse/IGNITE-17990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-17990:
-
Ignite Flags: (was: Docs Required,Release Notes Required)
> Download RAFT snapshot without deleting original partition data
> ---
>
> Key: IGNITE-17990
> URL: https://issues.apache.org/jira/browse/IGNITE-17990
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> h3. Description
> In first design, full rebalance is implemented this way:
> * we drop partition data
> * we download partition data from the leader
> * we're done
> There's a problem with this approach - if download part failed, we lost one
> follower. This is bad, because technically new leader may have more data in
> the log and it could have uploaded it the follower, but now it makes no sense.
> Not only can it lead to hard-to-catch errors and introducing custom code to
> JRaft, it's also an unconditional data deletion without neither explicit
> user approval nor a copy of the data preserved durably.
> Such implementation is fine for POC and some tests, but it cannot be allowed
> in the release version of the product.
> h3. New proposed solution
> As trivial as it may seem, new solution is to _not deleting data_ before
> snapshot is fully downloaded and ready for swap. Why is it trivial? Because
> this is literally what RAFT demands to be done.
> Of course, there's a {*}but{*}. Snapshot application, when it's downloaded,
> should be {{O(1)}} when it comes to the number of rows in the partition and a
> number of transactions in a tx state store. This may not be fully achievable,
> depending on the implementation that we chose, more on that later.
> Following sections will describe all my concerns and possible
> implementations. Some sections can be skipped while reading. For example, if
> you're not interested in a specific storage engine, but want to read
> everything else.
> h3. TX state storage
> There's one really good thing about TX state storage. It has no storage
> engine, there's only a single RocksDB-based implementation. This makes
> possible the following approach:
> * when we stream data, we can write it into a SST file, almost like in
> snapshots of meta-storage ans CMG storages
> * once snapshot is downloaded, we ingest it into a storage
> What I like about this solution is that it's very simple. But, there are
> concerns:
> * ingesting leads to additional implicit data flush. Maybe it can be
> avoided, more on that later
> * it's not clear whether RocksDB creates a copy of SST file or not. I would
> assume that it does, because the file might be in other folder or on another
> device, for example. Although copying files is fast, it still takes time. Add
> to this a time required for the flush and we see a problem - operation may
> become unnecessarily long
> For these reasons, I don't think that such solution should be implemented.
> The point of this description was to show, that I thought about this
> alternative and consciously decided to use another one.
> I believe that TX state storage should use the same approach as a
> RocksDB-based partition storage. Its description can be found later in this
> issue.
> h3. MV storage - Test engine
> Test uses concurrent skip-list map for MV data and a bunch of other maps for
> indexes. While snapshots is being downloaded, we should insert all data into
> new maps, that have the same structure. In the end, we should have two
> versions of the partition: old and new.
> {{onSnapshotLoad}} should just swap all objects. After that, old partition
> data can be cleaned by the garbage collector.
> This is a good place to start implementation. I assume that some new API will
> be introduced. I have thoughts about it as well, they are described later in
> *API* section.
> h3. MV storage - RocksDB engine
> SST-based approach is described in a *TX state storage* section. There I
> describe why I don't think that this is a good solution. Same reasoning can
> be applied here just as effectively. This means that we should write data in
> the same RocksDB instance. This is a little bit tricky.
> The reason is that all stored data is merged together, and Columns Families
> are shared between different partitions. This makes it harder to find a place
> to write partition data while old partition data persists. As a reminder and
> an example, let's take a look at how data is stored in row storage:
> {code:java}
> +---+-+---+
> |Partition 0| ... | Partition MAX |
> +---+-+---+
> | Row1 | ... | RowN | ... | Row1 | ... | RowN |
>
[jira] [Updated] (IGNITE-17990) Download RAFT snapshot without deleting original partition data
[
https://issues.apache.org/jira/browse/IGNITE-17990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kirill Tkalenko updated IGNITE-17990:
-
Fix Version/s: 3.0.0-beta2
> Download RAFT snapshot without deleting original partition data
> ---
>
> Key: IGNITE-17990
> URL: https://issues.apache.org/jira/browse/IGNITE-17990
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
> Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>
> h3. Description
> In first design, full rebalance is implemented this way:
> * we drop partition data
> * we download partition data from the leader
> * we're done
> There's a problem with this approach - if download part failed, we lost one
> follower. This is bad, because technically new leader may have more data in
> the log and it could have uploaded it the follower, but now it makes no sense.
> Not only can it lead to hard-to-catch errors and introducing custom code to
> JRaft, it's also an unconditional data deletion without neither explicit
> user approval nor a copy of the data preserved durably.
> Such implementation is fine for POC and some tests, but it cannot be allowed
> in the release version of the product.
> h3. New proposed solution
> As trivial as it may seem, new solution is to _not deleting data_ before
> snapshot is fully downloaded and ready for swap. Why is it trivial? Because
> this is literally what RAFT demands to be done.
> Of course, there's a {*}but{*}. Snapshot application, when it's downloaded,
> should be {{O(1)}} when it comes to the number of rows in the partition and a
> number of transactions in a tx state store. This may not be fully achievable,
> depending on the implementation that we chose, more on that later.
> Following sections will describe all my concerns and possible
> implementations. Some sections can be skipped while reading. For example, if
> you're not interested in a specific storage engine, but want to read
> everything else.
> h3. TX state storage
> There's one really good thing about TX state storage. It has no storage
> engine, there's only a single RocksDB-based implementation. This makes
> possible the following approach:
> * when we stream data, we can write it into a SST file, almost like in
> snapshots of meta-storage ans CMG storages
> * once snapshot is downloaded, we ingest it into a storage
> What I like about this solution is that it's very simple. But, there are
> concerns:
> * ingesting leads to additional implicit data flush. Maybe it can be
> avoided, more on that later
> * it's not clear whether RocksDB creates a copy of SST file or not. I would
> assume that it does, because the file might be in other folder or on another
> device, for example. Although copying files is fast, it still takes time. Add
> to this a time required for the flush and we see a problem - operation may
> become unnecessarily long
> For these reasons, I don't think that such solution should be implemented.
> The point of this description was to show, that I thought about this
> alternative and consciously decided to use another one.
> I believe that TX state storage should use the same approach as a
> RocksDB-based partition storage. Its description can be found later in this
> issue.
> h3. MV storage - Test engine
> Test uses concurrent skip-list map for MV data and a bunch of other maps for
> indexes. While snapshots is being downloaded, we should insert all data into
> new maps, that have the same structure. In the end, we should have two
> versions of the partition: old and new.
> {{onSnapshotLoad}} should just swap all objects. After that, old partition
> data can be cleaned by the garbage collector.
> This is a good place to start implementation. I assume that some new API will
> be introduced. I have thoughts about it as well, they are described later in
> *API* section.
> h3. MV storage - RocksDB engine
> SST-based approach is described in a *TX state storage* section. There I
> describe why I don't think that this is a good solution. Same reasoning can
> be applied here just as effectively. This means that we should write data in
> the same RocksDB instance. This is a little bit tricky.
> The reason is that all stored data is merged together, and Columns Families
> are shared between different partitions. This makes it harder to find a place
> to write partition data while old partition data persists. As a reminder and
> an example, let's take a look at how data is stored in row storage:
> {code:java}
> +---+-+---+
> |Partition 0| ... | Partition MAX |
> +---+-+---+
> | Row1 | ... | RowN | ... | Row1 | ... | RowN |
>
[jira] [Updated] (IGNITE-17302) Raft listener multi-storage support
[ https://issues.apache.org/jira/browse/IGNITE-17302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Tkalenko updated IGNITE-17302: - Fix Version/s: 3.0.0-beta2 > Raft listener multi-storage support > > > Key: IGNITE-17302 > URL: https://issues.apache.org/jira/browse/IGNITE-17302 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Mirza Aliev >Priority: Major > Labels: ignite-3, transaction3_rw > Fix For: 3.0.0-beta2 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > It's supposed that replication state machine (e.g. RaftGroupListener) will > handle multiple storages. For example PartitionListener will use: > * MvPartitionStorage - common multi-version partition storage for core > partition data. > * Both volatile and persistent lock storages IGNITE-15932. > * Persistent storage for txn state IGNITE-15931. > * Index storages. > Generally speaking replication logic is decoupled from data-storages, and > there are only few intersection points: > * Raft log truncation - in order to safely truncate raft log it's required > to take into consideration all checkpointIndexes for all storages of the > given state machine and perform min(storage1.checkpointIndex, > storage2.checkpointIndex, ... ) truncation. Besides that it's important for > storages to skip already applied commands on raft log replay - storage > indexes to the rescue. See IGNITE-16907 for more details. Such modifications > will also effect PartitionListener.onSnapshotLoad()/onSnapshotSave() methods. > * It's required to update local node recovery process a bit: a node must > enter the logical topology only after restoring all local raft nodes, > including both restoring storages from corresponding snapshot and applying > raft log. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625617#comment-17625617
]
Vladislav Pyatkov commented on IGNITE-18005:
Merged c8340a79cc29cb9ff0bcc14f800cde403016658f
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-18007) macOS support for Ignite 3 C++ client
[ https://issues.apache.org/jira/browse/IGNITE-18007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Semyon Danilov updated IGNITE-18007: Fix Version/s: 3.0.0-beta2 > macOS support for Ignite 3 C++ client > - > > Key: IGNITE-18007 > URL: https://issues.apache.org/jira/browse/IGNITE-18007 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Semyon Danilov >Assignee: Semyon Danilov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 10m > Remaining Estimate: 0h > > Currently there's support for Windows and Linux. MacOS (and other BSD > relatives) doesn't have epoll for async IO operations. It is possible, > however, to add support with relatively easy approach: using epoll-shim > library, that lets user epoll api over kqueue -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625612#comment-17625612
]
Alexander Lapin commented on IGNITE-18005:
--
[~v.pyatkov] LGTM
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-17917) Use unified start script for all distributions
[ https://issues.apache.org/jira/browse/IGNITE-17917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Aleksandr reassigned IGNITE-17917: -- Assignee: Aleksandr > Use unified start script for all distributions > -- > > Key: IGNITE-17917 > URL: https://issues.apache.org/jira/browse/IGNITE-17917 > Project: Ignite > Issue Type: Task > Components: build >Reporter: Aleksandr >Assignee: Aleksandr >Priority: Major > Labels: ignite-3, tech-debt > > Now db zip distribution uses its own start script that makes refactoring > difficult. Changes in zip distribution are not applied to rpm/deb and docker > and visa-versa. > We have to unify at least shell functions. And if possible use a single > script for all distributions. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625603#comment-17625603
]
Vladislav Pyatkov commented on IGNITE-18005:
It is nor exactly right, because the tombstone transforms to null value (it is
not had to filter some specific way).
But the issue is in delete batch option, that passes a binary row with tuple
contains null value and a key, but the removed value should pass as a null.
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625603#comment-17625603
]
Vladislav Pyatkov edited comment on IGNITE-18005 at 10/28/22 10:56 AM:
---
It is not exactly right, because the tombstone transforms to null value (it is
not had to filter some specific way).
But the issue is in delete batch option, that passes a binary row with tuple
contains null value and a key, but the removed value should pass as a null.
was (Author: v.pyatkov):
It is nor exactly right, because the tombstone transforms to null value (it is
not had to filter some specific way).
But the issue is in delete batch option, that passes a binary row with tuple
contains null value and a key, but the removed value should pass as a null.
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vladislav Pyatkov reassigned IGNITE-18005:
--
Assignee: Vladislav Pyatkov
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17943) Implement RAFT snapshot TX data sender
[ https://issues.apache.org/jira/browse/IGNITE-17943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625581#comment-17625581 ] Roman Puchkovskiy commented on IGNITE-17943: Thanks! > Implement RAFT snapshot TX data sender > -- > > Key: IGNITE-17943 > URL: https://issues.apache.org/jira/browse/IGNITE-17943 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Roman Puchkovskiy >Assignee: Roman Puchkovskiy >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-beta2 > > Time Spent: 7h > Remaining Estimate: 0h > > OutgoingSnapshot#handleSnapshotTxDataRequest() needs to be implemented > similar to how OutgoingSnapshot#handleSnapshotMvDataRequest() works. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-18008) Slow dns cause slow cli startup
Aleksandr created IGNITE-18008: -- Summary: Slow dns cause slow cli startup Key: IGNITE-18008 URL: https://issues.apache.org/jira/browse/IGNITE-18008 Project: Ignite Issue Type: Task Components: cli Reporter: Aleksandr It seems like CLI application goes to the Internet for some reason and it might cause a slow startup. I think we have to disable this behavior because there is no reason to go to the Internet for CLI application. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17357) JMX metric exporter for Ignite 3
[ https://issues.apache.org/jira/browse/IGNITE-17357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625558#comment-17625558 ] Kirill Gusakov commented on IGNITE-17357: - https://github.com/apache/ignite-3/pull/1234 > JMX metric exporter for Ignite 3 > > > Key: IGNITE-17357 > URL: https://issues.apache.org/jira/browse/IGNITE-17357 > Project: Ignite > Issue Type: New Feature >Reporter: Denis Chudov >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > Metrics should be able to be exported via JMX as a first stage of metrics > exposing. > Exporter implementation must provide the following behavior: > * for each MetricSource we need to provide separate MBean with attribute per > metric > * each MBean attribute must have the same name as corresponding metric > * on enable/disable event for MetricSource corresponding MBean must be > registered/unregistered -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky reassigned IGNITE-18005:
---
Assignee: (was: Evgeny Stanilovsky)
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17997) Whitespaces at the end are ignored during string comparison
[
https://issues.apache.org/jira/browse/IGNITE-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625538#comment-17625538
]
Andrey Khitrin commented on IGNITE-17997:
-
Thank you, I see expected behavior with `VARCHAR`:
{code:sql}
sql-cli> select cast('a' AS VARCHAR) = cast('a' AS VARCHAR) as test;
╔═╗
║ TEST║
╠═╣
║ true║
╚═╝
sql-cli> select cast('a' AS VARCHAR) = cast('a ' AS VARCHAR) as test;
╔═╗
║ TEST║
╠═╣
║ false ║
╚═╝
{code}
> Whitespaces at the end are ignored during string comparison
> ---
>
> Key: IGNITE-17997
> URL: https://issues.apache.org/jira/browse/IGNITE-17997
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 3.0.0-beta1
>Reporter: Andrey Khitrin
>Priority: Major
>
> In 3.0.0-Beta-1:
> {code:java}
> sql-cli> select 'a' = 'a' as t1, 'a' = 'b' as t2, 'a' = 'a ' as t3, 'a' = '
> a' as t4;
> ╔══╤═══╤══╤═══╗
> ║ T1 │ T2 │ T3 │ T4 ║
> ╠══╪═══╪══╪═══╣
> ║ true │ false │ true │ false ║
> ╚══╧═══╧══╧═══╝
> {code}
> Tests T1, T2, and T4 show correct behavior. But in test T2 we see that string
> 'a' is considered being equal to string 'a ' (same string but with
> arbitrary amount of whitespaces at the end). This is incorrect behavior.
> This issue may have the same nature as IGNITE-17996, but it's just a
> hypothesis.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-18007) macOS support for Ignite 3 C++ client
Semyon Danilov created IGNITE-18007: --- Summary: macOS support for Ignite 3 C++ client Key: IGNITE-18007 URL: https://issues.apache.org/jira/browse/IGNITE-18007 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Semyon Danilov Assignee: Semyon Danilov Currently there's support for Windows and Linux. MacOS (and other BSD relatives) doesn't have epoll for async IO operations. It is possible, however, to add support with relatively easy approach: using epoll-shim library, that lets user epoll api over kqueue -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-15574) Calcite. JOIN with DISTINCT FROM fails.
[
https://issues.apache.org/jira/browse/IGNITE-15574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksey Plekhanov updated IGNITE-15574:
---
Labels: calcite calcite3-required ignite-3 (was: calcite calcite2-required
calcite3-required ignite-3)
> Calcite. JOIN with DISTINCT FROM fails.
> ---
>
> Key: IGNITE-15574
> URL: https://issues.apache.org/jira/browse/IGNITE-15574
> Project: Ignite
> Issue Type: Improvement
> Components: sql
>Reporter: Evgeny Stanilovsky
>Assignee: Aleksey Plekhanov
>Priority: Major
> Labels: calcite, calcite3-required, ignite-3
> Fix For: 2.15
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {noformat}
> statement ok
> create table tbl_1 (a integer, b integer)
> statement ok
> insert into tbl_1 values (1,NULL),(2,3),(NULL,NULL)
> statement ok
> create table tbl_2 (b integer)
> statement ok
> insert into tbl_2 values (1),(2),(NULL)
> query II
> select a,tbl_2.b from tbl_1 inner join tbl_2 on (a IS NOT DISTINCT FROM
> tbl_2.b)
>
> 1 1
> 2 2
> NULL NULL
> {noformat}
> failed with
> {noformat}
> java.lang.RuntimeException: cannot translate call IS NOT DISTINCT FROM($t0,
> $t1)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitCall(RexToLixTranslator.java:980)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitCall(RexToLixTranslator.java:79)
> at org.apache.calcite.rex.RexCall.accept(RexCall.java:189)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:886)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.exp.RexToLixTranslator.visitLocalRef(RexToLixTranslator.java:79)
> {noformat}
> {noformat}
> /join/test_not_distinct_from.test[_ignore]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Evgeny Stanilovsky reassigned IGNITE-18005:
---
Assignee: Evgeny Stanilovsky
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Assignee: Evgeny Stanilovsky
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-18005) ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
[
https://issues.apache.org/jira/browse/IGNITE-18005?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625481#comment-17625481
]
Evgeny Stanilovsky commented on IGNITE-18005:
-
Seems that root case is from : IGNITE-17968 Fix write-intents being filtered
out in case if it's a tombstone
No one filters tombstone`s for now.
> ItDataTypesTest#testUnicodeStrings fails with IndexOutOfBoundsException
> ---
>
> Key: IGNITE-18005
> URL: https://issues.apache.org/jira/browse/IGNITE-18005
> Project: Ignite
> Issue Type: Bug
>Reporter: Alexander Lapin
>Priority: Major
>
> {code:java}
> Caused by: java.lang.IndexOutOfBoundsException: 4
> at java.base/java.nio.HeapByteBuffer.get(HeapByteBuffer.java:166)
> at org.apache.ignite.internal.schema.row.Row.findColumn(Row.java:516)
> at org.apache.ignite.internal.schema.row.Row.stringValue(Row.java:288)
> at
> org.apache.ignite.internal.schema.NativeTypeSpec$9.objectValue(NativeTypeSpec.java:134)
> at org.apache.ignite.internal.schema.row.Row.value(Row.java:122)
> at
> org.apache.ignite.internal.sql.engine.schema.IgniteTableImpl.toRow(IgniteTableImpl.java:284)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode.convert(TableScanNode.java:279)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:246)
> at
> org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode$SubscriberImpl.onNext(TableScanNode.java:233)
> at java.base/java.util.ArrayList.forEach(ArrayList.java:1540)
> at
> org.apache.ignite.internal.table.distributed.storage.InternalTableImpl$PartitionScanPublisher$PartitionScanSubscription.lambda$scanBatch$2(InternalTableImpl.java:1182)
> at
> java.base/java.util.concurrent.CompletableFuture$UniAccept.tryFire(CompletableFuture.java:714)
> ... 11 more
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-18006) Support timeouts in async version of python client
Ivan Daschinsky created IGNITE-18006: Summary: Support timeouts in async version of python client Key: IGNITE-18006 URL: https://issues.apache.org/jira/browse/IGNITE-18006 Project: Ignite Issue Type: Improvement Components: python, thin client Reporter: Ivan Daschinsky Assignee: Ivan Daschinsky Fix For: python-0.6.0 Need to support timeouts in async version of clients. All caches operations, cursor fetch operations. Transaction start, close, commit and cursors start and close should not be affected. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: HeapConsumptionDataStreamerTest.src > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: (was: HeapConsumptionDataStreamerTest.src) > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: (was: HeapConsumptionDataStreamerTest-1.src) > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: HeapConsumptionDataStreamerTest-1.src > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: (was: HeapConsumptionDataStreamerTest.src) > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: HeapConsumptionDataStreamerTest.src > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17969) .NET: Thin 3.0: Partition Awareness - support all key types
[ https://issues.apache.org/jira/browse/IGNITE-17969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17625455#comment-17625455 ] Igor Sapego commented on IGNITE-17969: -- [~ptupitsyn] looks good to me. > .NET: Thin 3.0: Partition Awareness - support all key types > --- > > Key: IGNITE-17969 > URL: https://issues.apache.org/jira/browse/IGNITE-17969 > Project: Ignite > Issue Type: Improvement > Components: platforms, thin client >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Major > Labels: .NET, ignite-3 > Fix For: 3.0.0-beta2 > > > IGNITE-17750 implements Partition Awareness in .NET thin client for int keys. > Extend this implementation: > * Support all key types > * Support composite keys > * Support custom colocation columns -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17607891#comment-17607891 ] Vladimir Steshin edited comment on IGNITE-17735 at 10/28/22 6:04 AM: - Datastreamer with 'allowOverwrite==true' and PRIMARY_SYNC persistent cache may cause heap issue or consume increased heap amount with persistent caches. There is related fixing 'perNodeParallelOperations()' setting. But I met this issue with trivial research like `HeapConsumptionDataStreamerTest.src`. Default parallel batches amount for persistent caches looks too much. The setting is historically for in-memory caches. What happens: streamer keep sending more and more streamer batches to process while receiving node collects backup updates futures, requests. Similarly, backup node accumulates incoming update requests stucking at disk writes. See 'DS_heap_consumption.png' for example. Suggestion. We might bring reduced default parallel batches number for persistent caches `IgniteDataStreamer#DFLT_PARALLEL_OPS_PERSISTENT_MULTIPLIER`(PR #10343). Why sending more? Helps a lot, reduces heap utilization even if there is no OOMe. Better solution would be a backpressure. Not sure it worth the case. Did some benchmarks. For persistent caches the setting `CPUs x 2` seems enough. was (Author: vladsz83): Datastreamer with 'allowOverwrite==true' and PRIMARY_SYNC persistent cache may cause heap issue or consume increased heap amount with persistent caches. There is related fixing 'perNodeParallelOperations()' setting. But I met this issue with trivial research like `HeapConsumptionDataStreamerTest.src`. Default parallel batches amount for persistent caches looks too much. The setting is historically for in-memory caches. What happens: streamer keep sending more and more streamer batches to process while receiving node collects backup updates futures, requests. Similarly, backup node accumulates incoming update requests stucking at disk writes. See 'DS_heap_consumption.png' for example. Suggestion. We might bring reduced default parallel batches number for persistent caches `IgniteDataStreamer#DFLT_PARALLEL_OPS_PERSISTENT_MULTIPLIER`(PR #10343). Why sending more? Helps a lot, reduces heap utilization even if there is no OOMe. Better solution would be a backpressure. Not sure it worth the case. Did some benchmarks. For persistent caches `CPUs x 2` seems enough. > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17735) Datastreamer may consume heap with allowOverwtire=='true'.
[ https://issues.apache.org/jira/browse/IGNITE-17735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vladimir Steshin updated IGNITE-17735: -- Attachment: bench_results_defaultIsolated_longer.txt bench_results_defaultIsolated.txt bench_results_Individual.txt > Datastreamer may consume heap with allowOverwtire=='true'. > -- > > Key: IGNITE-17735 > URL: https://issues.apache.org/jira/browse/IGNITE-17735 > Project: Ignite > Issue Type: Sub-task >Reporter: Vladimir Steshin >Assignee: Vladimir Steshin >Priority: Major > Labels: ise > Attachments: DS_heap_consumption.png, DS_heap_consumption_2.png, > HeapConsumptionDataStreamerTest.src, bench_results_Individual.txt, > bench_results_defaultIsolated.txt, bench_results_defaultIsolated_longer.txt > > Time Spent: 1h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.20.10#820010)
