[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity mapping for the partition awareness usage
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17316:
-
Ignite Flags: Release Notes Required (was: Docs Required,Release Notes
Required)
> Thin client pluggable affnity mapping for the partition awareness usage
> ---
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.13
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17316) Thin client pluggable affnity mapping for the partition awareness usage
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maxim Muzafarov updated IGNITE-17316:
-
Summary: Thin client pluggable affnity mapping for the partition awareness
usage (was: Thin client pluggable affnity function to use partition awareness)
> Thin client pluggable affnity mapping for the partition awareness usage
> ---
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.13
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 5h
> Remaining Estimate: 0h
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17316) Thin client pluggable affnity function to use partition awareness
[
https://issues.apache.org/jira/browse/IGNITE-17316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577658#comment-17577658
]
Ignite TC Bot commented on IGNITE-17316:
{panel:title=Branch: [pull/10140/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10140/head] Base: [master] : New Tests
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Thin Client: Java{color} [[tests
2|https://ci.ignite.apache.org/viewLog.html?buildId=6720305]]
* {color:#013220}ClientTestSuite:
ThinClientPartitionAwarenessResourceReleaseTest.testResourcesReleasedAfterCacheDestroyed
- PASSED{color}
* {color:#013220}ClientTestSuite:
ThinClientPartitionAwarenessStableTopologyTest.testPartitionedCustomAffinityCacheWithMapper
- PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6701404buildTypeId=IgniteTests24Java8_RunAll]
> Thin client pluggable affnity function to use partition awareness
> -
>
> Key: IGNITE-17316
> URL: https://issues.apache.org/jira/browse/IGNITE-17316
> Project: Ignite
> Issue Type: Improvement
> Components: thin client
>Affects Versions: 2.13
>Reporter: Maxim Muzafarov
>Assignee: Maxim Muzafarov
>Priority: Major
> Fix For: 2.14
>
> Time Spent: 4h 50m
> Remaining Estimate: 0h
>
> We should provie a pluggable affinity for the thin client, thus using the
> partition awareness features become possible on only for
> {{RendezvousAffinityFunction}}.
> Dev-list discussion:
> https://lists.apache.org/thread/7n7zbcgw59voxyr08ct3zx2ss5g8q9wh
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (IGNITE-15759) [IEP-80] Removal of LOCAL caches support
[ https://issues.apache.org/jira/browse/IGNITE-15759 ]
Maxim Muzafarov deleted comment on IGNITE-15759:
--
was (Author: ignitetcbot):
{panel:title=Branch: [pull/10157/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10157/head] Base: [master] : New Tests
(3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 8{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=6712991]]
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsFifo - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsSorted - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsLru - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6713084buildTypeId=IgniteTests24Java8_RunAll]
> [IEP-80] Removal of LOCAL caches support
>
>
> Key: IGNITE-15759
> URL: https://issues.apache.org/jira/browse/IGNITE-15759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maxim Muzafarov
>Priority: Major
> Labels: IEP-80
> Fix For: 2.14
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> LOCAL cachens has huge amount of known limitation and not intended to be used
> in production environment.
> Should be removed in 2.13
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] (IGNITE-15759) [IEP-80] Removal of LOCAL caches support
[ https://issues.apache.org/jira/browse/IGNITE-15759 ]
Maxim Muzafarov deleted comment on IGNITE-15759:
--
was (Author: ignitetcbot):
{panel:title=Branch: [pull/10157/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10157/head] Base: [master] : New Tests
(3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 8{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=6712991]]
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsFifo - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsSorted - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsLru - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6713084buildTypeId=IgniteTests24Java8_RunAll]
> [IEP-80] Removal of LOCAL caches support
>
>
> Key: IGNITE-15759
> URL: https://issues.apache.org/jira/browse/IGNITE-15759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maxim Muzafarov
>Priority: Major
> Labels: IEP-80
> Fix For: 2.14
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> LOCAL cachens has huge amount of known limitation and not intended to be used
> in production environment.
> Should be removed in 2.13
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15759) [IEP-80] Removal of LOCAL caches support
[
https://issues.apache.org/jira/browse/IGNITE-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577546#comment-17577546
]
Ignite TC Bot commented on IGNITE-15759:
{panel:title=Branch: [pull/10157/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10157/head] Base: [master] : New Tests
(3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 8{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=6712991]]
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsFifo - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsSorted - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsLru - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6713084buildTypeId=IgniteTests24Java8_RunAll]
> [IEP-80] Removal of LOCAL caches support
>
>
> Key: IGNITE-15759
> URL: https://issues.apache.org/jira/browse/IGNITE-15759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maxim Muzafarov
>Priority: Major
> Labels: IEP-80
> Fix For: 2.14
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> LOCAL cachens has huge amount of known limitation and not intended to be used
> in production environment.
> Should be removed in 2.13
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15759) [IEP-80] Removal of LOCAL caches support
[
https://issues.apache.org/jira/browse/IGNITE-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577545#comment-17577545
]
Ignite TC Bot commented on IGNITE-15759:
{panel:title=Branch: [pull/10157/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10157/head] Base: [master] : New Tests
(3)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 8{color} [[tests
3|https://ci.ignite.apache.org/viewLog.html?buildId=6712991]]
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsFifo - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsSorted - PASSED{color}
* {color:#013220}IgniteCacheTestSuite8:
GridCacheConcurrentEvictionsSelfTest.testConcurrentPutsLru - PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6713084buildTypeId=IgniteTests24Java8_RunAll]
> [IEP-80] Removal of LOCAL caches support
>
>
> Key: IGNITE-15759
> URL: https://issues.apache.org/jira/browse/IGNITE-15759
> Project: Ignite
> Issue Type: Improvement
>Reporter: Nikolay Izhikov
>Assignee: Maxim Muzafarov
>Priority: Major
> Labels: IEP-80
> Fix For: 2.14
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> LOCAL cachens has huge amount of known limitation and not intended to be used
> in production environment.
> Should be removed in 2.13
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-17500) RocksDbMvPartitionStorage#destroy should wait for the flush operation to complete
[
https://issues.apache.org/jira/browse/IGNITE-17500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577511#comment-17577511
]
Kirill Tkalenko commented on IGNITE-17500:
--
[~apolovtcev] Looks good.
> RocksDbMvPartitionStorage#destroy should wait for the flush operation to
> complete
> -
>
> Key: IGNITE-17500
> URL: https://issues.apache.org/jira/browse/IGNITE-17500
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Minor
> Labels: ignite-3
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Currently {{RocksDbMvPartitionStorage#destroy}} method returns a
> CompletableFuture that is immediately completed after the data has been
> written to the memtable. Instead, it should only complete after the data has
> been flushed to the disk.
> There's also another issue that {{RocksDbMvPartitionStorage}} is not closed
> after {{destroy}} is called, which is a resource leak.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17500) RocksDbMvPartitionStorage#destroy should wait for the flush operation to complete
[
https://issues.apache.org/jira/browse/IGNITE-17500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr Polovtcev updated IGNITE-17500:
-
Description:
Currently {{RocksDbMvPartitionStorage#destroy}} method returns a
CompletableFuture that is immediately completed after the data has been written
to the memtable. Instead, it should only complete after the data has been
flushed to the disk.
There's also another issue that {{RocksDbMvPartitionStorage}} is not closed
after {{destroy}} is called, which is a resource leak.
> RocksDbMvPartitionStorage#destroy should wait for the flush operation to
> complete
> -
>
> Key: IGNITE-17500
> URL: https://issues.apache.org/jira/browse/IGNITE-17500
> Project: Ignite
> Issue Type: Bug
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Minor
> Labels: ignite-3
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Currently {{RocksDbMvPartitionStorage#destroy}} method returns a
> CompletableFuture that is immediately completed after the data has been
> written to the memtable. Instead, it should only complete after the data has
> been flushed to the disk.
> There's also another issue that {{RocksDbMvPartitionStorage}} is not closed
> after {{destroy}} is called, which is a resource leak.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17500) RocksDbMvPartitionStorage#destroy should wait for the flush operation to complete
Aleksandr Polovtcev created IGNITE-17500: Summary: RocksDbMvPartitionStorage#destroy should wait for the flush operation to complete Key: IGNITE-17500 URL: https://issues.apache.org/jira/browse/IGNITE-17500 Project: Ignite Issue Type: Bug Reporter: Aleksandr Polovtcev Assignee: Aleksandr Polovtcev -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17378) Check the replica is alive during before that a method will be invoked
[ https://issues.apache.org/jira/browse/IGNITE-17378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17378: - Epic Link: IGNITE-15081 > Check the replica is alive during before that a method will be invoked > -- > > Key: IGNITE-17378 > URL: https://issues.apache.org/jira/browse/IGNITE-17378 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3, transaction3_rw > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17499) Service method invocation excepition is not propagated to thin client side.
Mikhail Petrov created IGNITE-17499: --- Summary: Service method invocation excepition is not propagated to thin client side. Key: IGNITE-17499 URL: https://issues.apache.org/jira/browse/IGNITE-17499 Project: Ignite Issue Type: Bug Reporter: Mikhail Petrov https://issues.apache.org/jira/browse/IGNITE-13389 introduced dedicated flag that make it possible to propagate server side stacktrace to a thin client side. The mentoined above propagation does not work for exceptions that arises during Ignite Service invocation. Steps to reproduce: 1. Start .Net Ignite node 2. Deploy service which invocation throws an arbitrary uncaught exception 3. Invoke previously deployed services via Java thin client As a result, information about the custom code exception is not present in the exception stacktrace that is thrown after the service call. The main reason of such behaviour is that ClientServiceInvokeRequest.java:198 does not propagate initial exception. So ClientRequestHandler#handleException could not handle exception properly even if ThinClientConfiguration#sendServerExceptionStackTraceToClient() is enabled. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17457) Cluster locks after the transaction recovery procedure if the tx primary node fail
[
https://issues.apache.org/jira/browse/IGNITE-17457?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577473#comment-17577473
]
Ignite TC Bot commented on IGNITE-17457:
{panel:title=Branch: [pull/10178/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10178/head] Base: [master] : New Tests
(1)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Cache 12{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6720371]]
* {color:#013220}IgniteCacheTestSuite12:
TxRecoveryConcurrentOnPrimaryFailTest.testRecoveryNotDeadLockOnPrimaryFail -
PASSED{color}
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6710922buildTypeId=IgniteTests24Java8_RunAll]
> Cluster locks after the transaction recovery procedure if the tx primary node
> fail
> --
>
> Key: IGNITE-17457
> URL: https://issues.apache.org/jira/browse/IGNITE-17457
> Project: Ignite
> Issue Type: Bug
>Reporter: Sergey Korotkov
>Assignee: Sergey Korotkov
>Priority: Major
> Time Spent: 4.5h
> Remaining Estimate: 0h
>
> Ignite cluster may be locked (all client operations would block) after the tx
> recovery procedure executed on the tx primary node failure.
> The prepared transaction may remain un-commited on the backup node after the
> tx recovery. So the partition exchange wouldn't complete. So cluster would
> be locked.
>
> The Immediate reason is the race condition in the method:
> {code:java}
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxAdapter::markFinalizing(RECOVERY_FINISH){code}
> If 2 or more backups are configured It may be called concurrently for the
> same transaction both from the recovery procedure:
> {code:java}
> IgniteTxManager::commitIfPrepared{code}
> and from the tx recovery request handler:
> {code:java}
> IgniteTxHandler::processCheckPreparedTxRequest{code}
> Problem occur if thread context is switched between old finalization status
> request and status update.
>
> The problematic sequence of events is as follows (the lock will be in the
> node1):
> 1. Start cluster with 3 nodes (node0, node1, node2) and cache with 2 backups.
> 2. On node2 start and prepare transaction choosing key with primary partition
> stored on node2.
> 3. Kill node2
> 4. The tx recovery procedure is started both on node0 and node1
> 5. In scope of the recovery procedure node0 sends tx recovery request to node1
> 6. The following steps are executed on the node1 in two threads ("procedure"
> which is a system pool thread executing the tx recovery procedure and
> "handler" which is a striped pool thread processing the tx recovery request
> sent from node0):
> - tx.finalization == NONE
> - "procedure": calls markFinalizing(RECOVERY_FINISH)
> - "handler": calls markFinalizing(RECOVERY_FINISH)
> - "procedure": gets old tx.finlalization - it's NONE
> - "handler": gets old tx.finalization - it's NONE
> - "handler": updates tx.finalization - now it's RECOVERY_FINISH
> - "procedure": tries to update tx.finalization via compareAndSet and fails
> since compare fails.
> - "procedure": stops transaction processing and does not try to commit it.
> - Transaction remains not finished on node1.
>
> Reproducer is in the pull request.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (IGNITE-15759) [IEP-80] Removal of LOCAL caches support
[ https://issues.apache.org/jira/browse/IGNITE-15759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577453#comment-17577453 ] Evgeny Stanilovsky commented on IGNITE-15759: - fill some minor comments, overall looks good. > [IEP-80] Removal of LOCAL caches support > > > Key: IGNITE-15759 > URL: https://issues.apache.org/jira/browse/IGNITE-15759 > Project: Ignite > Issue Type: Improvement >Reporter: Nikolay Izhikov >Assignee: Maxim Muzafarov >Priority: Major > Labels: IEP-80 > Fix For: 2.14 > > Time Spent: 1h > Remaining Estimate: 0h > > LOCAL cachens has huge amount of known limitation and not intended to be used > in production environment. > Should be removed in 2.13 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17476) Implement configurations event handling by index manager
[ https://issues.apache.org/jira/browse/IGNITE-17476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov updated IGNITE-17476: -- Fix Version/s: 3.0.0-alpha6 > Implement configurations event handling by index manager > > > Key: IGNITE-17476 > URL: https://issues.apache.org/jira/browse/IGNITE-17476 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Assignee: Konstantin Orlov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > > Need to implement listener for configurations event to reflect the state of > configuration an create all necessary runtime structures. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17469) cli profile commands unification
[
https://issues.apache.org/jira/browse/IGNITE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mikhail Pochatkin updated IGNITE-17469:
---
Description:
cli config profile now has two commands show and create, while setting a
profile is done through a parameter, which is confusing.
There should be a separate command to set the profile.
In general I would suggest the commands there to look like:
cli config show
cli config set
cli config get
All repl cli commands should not accept --profile or -p parameters, all setting
and getting of the params in a profile should go through its activation
instead. I.e. cli config profile create newprofile ; cli config profile
activate newprofile; cli config set
ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
We should retain --profile in non-repl mode.
Profile manipulation should go like below:
cli config profile show []
cli config profile list
cli config profile create --copy-from
cli config profile activate
Let’s please update IEP with the corresponding changes as well.
Let's also make sure non-repl commands do have --profile option and
auto-completion scripts are updated accordingly. Currently I don't see any
suggestions on --profile there:
{noformat}
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$ ./ignite cluster config show
--profile te
st
Unknown options: '--profile', 'test'
Usage: ignite cluster config show [-h] [--cluster-url=]
[--selector=]
Shows cluster configuration.
--cluster-url=
Url to ignite node.
-h, --help Show this help message and exit.
--selector=
Configuration path selector.
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$
{noformat}
was:
cli config profile now has two commands show and create, while setting a
profile is done through a parameter, which is confusing.
There should be a separate command to set the profile.
In general I would suggest the commands there to look like:
cli config show
cli config set
cli config get
All repl cli commands should not accept --profile or -p parameters, all setting
and getting of the params in a profile should go through its activation
instead. I.e. cli config profile create newprofile ; cli config profile
activate newprofile; cli config set
ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
We should retain --profile in non-repl mode.
Profile manipulation should go like below:
cli config profile show []
cli config profile list
cli config profile create --copy-from
cli config profile activate
Let’s please update IEP with the corresponding changes as well.
Let's also make sure non-cli commands do have --profile option and
auto-completion scripts are updated accordingly. Currently I don't see any
suggestions on --profile there:
{noformat}
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$ ./ignite cluster config show
--profile te
st
Unknown options: '--profile', 'test'
Usage: ignite cluster config show [-h] [--cluster-url=]
[--selector=]
Shows cluster configuration.
--cluster-url=
Url to ignite node.
-h, --help Show this help message and exit.
--selector=
Configuration path selector.
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$
{noformat}
> cli profile commands unification
>
>
> Key: IGNITE-17469
> URL: https://issues.apache.org/jira/browse/IGNITE-17469
> Project: Ignite
> Issue Type: Improvement
> Components: cli, ignite-3
>Reporter: Yury Yudin
>Assignee: Vadim Pakhnushev
>Priority: Major
> Labels: ignite-3
>
> cli config profile now has two commands show and create, while setting a
> profile is done through a parameter, which is confusing.
> There should be a separate command to set the profile.
> In general I would suggest the commands there to look like:
> cli config show
> cli config set
> cli config get
> All repl cli commands should not accept --profile or -p parameters, all
> setting and getting of the params in a profile should go through its
> activation instead. I.e. cli config profile create newprofile ; cli config
> profile activate newprofile; cli config set
> ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
> We should retain --profile in non-repl mode.
> Profile manipulation should go like below:
> cli config profile show []
> cli config profile list
> cli config profile create --copy-from
> cli config profile activate
>
> Let’s please update IEP with the corresponding changes as well.
>
> Let's also make sure non-repl commands do have --profile option and
> auto-completion scripts are updated accordingly. Currently I don't see any
> suggestions on --profile there:
> {noformat}
>
[jira] [Commented] (IGNITE-16892) Introduce new LockManager interface in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577406#comment-17577406
]
Vladislav Pyatkov commented on IGNITE-16892:
LGTM
> Introduce new LockManager interface in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
> h3. Upd
> For the purposes of unblocking IGNITE-17255 it's reasonable to split given
> ticket into two parts
> * The one that will introduce new LockManager interface and integrate it
> into existing tx logic.
> * The one that will add intention lock support from the HeapLockManager
> point of view.
> Only first part will be implemented within the scope of given ticket.
> HeapLockManager updates will be implemented in
> https://issues.apache.org/jira/browse/IGNITE-17498
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17498) Update HeapLockManager in order to support Intention locks
[ https://issues.apache.org/jira/browse/IGNITE-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17498: - Description: It's required to implement new lock upgrade logic that will consider not only S and X locks but also IS, IX and SIX. > Update HeapLockManager in order to support Intention locks > -- > > Key: IGNITE-17498 > URL: https://issues.apache.org/jira/browse/IGNITE-17498 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3, transaction3_rw > > It's required to implement new lock upgrade logic that will consider not only > S and X locks but also IS, IX and SIX. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16892) Introduce new LockManager interface in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-16892:
-
Summary: Introduce new LockManager interface in order to soupport S, X and
I locks (was: Update lock manager in order to soupport S, X and I locks)
> Introduce new LockManager interface in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
> h3. Upd
> For the purposes of unblocking IGNITE-17255 it's reasonable to split given
> ticket into two parts
> * The one that will introduce new LockManager interface and integrate it
> into existing tx logic.
> * The one that will add intention lock support from the HeapLockManager
> point of view.
> Only first part will be implemented within the scope of given ticket.
> HeapLockManager updates will be implemented in
> https://issues.apache.org/jira/browse/IGNITE-17498
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17255) Implement ReplicaService
[ https://issues.apache.org/jira/browse/IGNITE-17255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-17255: - Reviewer: Vladislav Pyatkov > Implement ReplicaService > > > Key: IGNITE-17255 > URL: https://issues.apache.org/jira/browse/IGNITE-17255 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Sergey Uttsel >Priority: Major > Labels: ignite-3, transaction3_rw > > For general context please check IGNITE-17252 > Within given ticket it's required to > * Implement ReplicaService itself. > * Substitute RaftGroupService with ReplicaService from within > InternalTableImpl and others > Please pay attention that according to tx protocol it's valid to fail the > transaction in case of primary replica change - we'll introduce support of > graceful primary replica switch later on. For now, within the scope of RW > transactions it's enough to detect where primary replica is and enlist it to > transaction with corresponding partition in order to reuse for further > in-partition communication. > We should make it very clear, that any replicaService.invoke(nodeId) might > fail with primaryReplicaMiss or replicaUnavailable, it's up to the outer > logic to remap such failed requests. > However it's still required to detect proper primary replica initially and > check whether it's still primary during further queries. Proper lease-based > primary replica stability engine will be introduced within > [IGNITE-17256|https://issues.apache.org/jira/browse/IGNITE-17256] , as a > staring point it's possible to reuse sendWithRetry logic with true readIndex > leader checks, meaning that primary replica is the replica collocated with > the current leader (not the node that is thinking that it's a leader, but a > node that was proved to be a leader based on readIndex logic). > In addition to all points mentioned above we should be aware that lot's of > tests will become flaky because sendWithRetry logic will only be available > with initial primaryReplica detection method and not within common invokes. > Generally speaking I believe that it's a good chance to rework them and thus > make them stable. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16892) Update lock manager in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-16892:
-
Description:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
h3. Upd
For the purposes of unblocking IGNITE-17255 it's reasonable to split given
ticket into two parts
* The one that will introduce new LockManager interface and integrate it into
existing tx logic.
* The one that will add intention lock support from the HeapLockManager point
of view.
Only first part will be implemented within the scope of given ticket.
HeapLockManager updates will be implemented in
https://issues.apache.org/jira/browse/IGNITE-17498
was:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
h3. Upd
For the purposes of unblocking IGNITE-17255 it's reasonable to split given
ticket into two parts
* The one that will introduce new LockManager interface and integrate it into
existing tx logic.
* The one that will add intention lock support from the HeapLockManager point
of view.
Only first part will be implemented within the scope of given ticket.
> Update lock manager in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
> h3. Upd
> For the purposes of unblocking IGNITE-17255 it's reasonable to split given
> ticket into two parts
> * The one that will introduce new LockManager interface and integrate it
> into existing tx logic.
> * The one that will add intention lock support from the HeapLockManager
> point of view.
> Only first part will be implemented within the scope of given ticket.
> HeapLockManager updates will be implemented in
> https://issues.apache.org/jira/browse/IGNITE-17498
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (IGNITE-17498) Update HeapLockManager in order to support Intention locks
Alexander Lapin created IGNITE-17498: Summary: Update HeapLockManager in order to support Intention locks Key: IGNITE-17498 URL: https://issues.apache.org/jira/browse/IGNITE-17498 Project: Ignite Issue Type: Improvement Reporter: Alexander Lapin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-16892) Update lock manager in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-16892:
-
Description:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
h3. Upd
For the purposes of unblocking IGNITE-17255 it's reasonable to split given
ticket into two parts
* The one that will introduce new LockManager interface and integrate it into
existing tx logic.
* The one that will add intention lock support from the HeapLockManager point
of view.
Only first part will be implemented within the scope of given ticket.
was:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
h3. Upd
For the purposes of unblocking
> Update lock manager in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
> h3. Upd
> For the purposes of unblocking IGNITE-17255 it's reasonable to split given
> ticket into two parts
> * The one that will introduce new LockManager interface and integrate it
> into existing tx logic.
> * The one that will add intention lock support from the HeapLockManager
> point of view.
> Only first part will be implemented within the scope of given ticket.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17496) LWM may be after HWM (reserved) on primary after the node restart
[
https://issues.apache.org/jira/browse/IGNITE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-17496:
--
Description:
{code:java}
java.lang.AssertionError: LWM after HWM: lwm=10010, hwm=10003, cntr=Counter
[lwm=10010, missed=[10011 - 10012, 10021, 10031 - 10032, 10043 - 10044],
maxApplied=10047, hwm=10004]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:265)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getAndIncrementUpdateCounter(GridCacheOffheapManager.java:2538)
at
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.getAndIncrementUpdateCounter(GridDhtLocalPartition.java:942)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.calculatePartitionUpdateCounters(IgniteTxLocalAdapter.java:510)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1360)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:730)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1136)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:581)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:378)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:201)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:175)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:135)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:221)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750) {code}
It looks like we have incorrect initialization problem here.
For example, at startup primary has the following counter
{code:java}
[lwm=10006, missed=[10007 - 10008, 10017 - 10020, 10031 - 10033, 10039 - 10042,
10055], hwm=10059, reserved=10006]{code}
but when updates started we'll got an exception
{code:java}
LWM after reserved: lwm=10016, reserved=10008, cntr=Counter [lwm=10016,
missed=[10017 - 10020, 10031 - 10033, 10039 - 10042, 10055], hwm=10059,
reserved=10009]{code}
this happens because first gap was closed and {{lwm}} changed from {{10006}} to
{{10016}} because of closed {{{}0007 - 10008{}}}.
And main prodlem here that we're trying to reuse already used counters, so,
{{reserved}} should be set to {{hwm}} at initialization.
was:
{code:java}
java.lang.AssertionError: LWM after HWM:
[jira] [Updated] (IGNITE-16892) Update lock manager in order to soupport S, X and I locks
[
https://issues.apache.org/jira/browse/IGNITE-16892?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-16892:
-
Description:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
h3. Upd
For the purposes of unblocking
was:
1. Lock management logic will be introduced in IGNITE-17255
2. Lock storages will be introduced in IGNITE-15932
Given ticket is a sort of a bridge between 1 and 2 that will adjust LockManager
methods along with corresponding implementation with LockMode parameter that
will clarify whether lock is
* Exclusive
* Shared
* IntentExclusive
* IntentShared
* SharedAndIntentExclusive
{code:java}
CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
LockException;
Iterator locks(UUID txId) {code}
lockName is a sort of lock locator, that either:
* RowId for data storage locks.
* UUID or similar to table/index commonly intent locks.
* Index keys.
> Update lock manager in order to soupport S, X and I locks
> -
>
> Key: IGNITE-16892
> URL: https://issues.apache.org/jira/browse/IGNITE-16892
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Alexander Lapin
>Priority: Major
> Labels: ignite-3, transaction3_rw
> Time Spent: 20m
> Remaining Estimate: 0h
>
> 1. Lock management logic will be introduced in IGNITE-17255
> 2. Lock storages will be introduced in IGNITE-15932
> Given ticket is a sort of a bridge between 1 and 2 that will adjust
> LockManager methods along with corresponding implementation with LockMode
> parameter that will clarify whether lock is
> * Exclusive
> * Shared
> * IntentExclusive
> * IntentShared
> * SharedAndIntentExclusive
> {code:java}
> CompletableFuture acquire(UUID txId, lockName LockMode lockMode);
> void release(Object key, UUID txId, Lock lock, LockMode lockMode) throws
> LockException;
> Iterator locks(UUID txId) {code}
> lockName is a sort of lock locator, that either:
> * RowId for data storage locks.
> * UUID or similar to table/index commonly intent locks.
> * Index keys.
> h3. Upd
> For the purposes of unblocking
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17470) Add initial support of Spark 3.2
[ https://issues.apache.org/jira/browse/IGNITE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-17470: --- Description: Update ignite-spark module to spark-3.2 and scala 12 (was: |Update ignite-spark module to spark-3.2 and scala 12|) > Add initial support of Spark 3.2 > > > Key: IGNITE-17470 > URL: https://issues.apache.org/jira/browse/IGNITE-17470 > Project: Ignite > Issue Type: Task > Components: spark >Reporter: Ivan Gagarkin >Priority: Major > > Update ignite-spark module to spark-3.2 and scala 12 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17470) Add initial support of Spark 3.2
[ https://issues.apache.org/jira/browse/IGNITE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-17470: --- Description: |Update ignite-spark module to spark-3.2 and scala 12| > Add initial support of Spark 3.2 > > > Key: IGNITE-17470 > URL: https://issues.apache.org/jira/browse/IGNITE-17470 > Project: Ignite > Issue Type: Task > Components: spark >Reporter: Ivan Gagarkin >Priority: Major > > |Update ignite-spark module to spark-3.2 and scala 12| -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17470) Add initial support of Spark 3.2
[ https://issues.apache.org/jira/browse/IGNITE-17470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-17470: --- Summary: Add initial support of Spark 3.2 (was: Update ignite-spark module to spark-3.2 and scala 12) > Add initial support of Spark 3.2 > > > Key: IGNITE-17470 > URL: https://issues.apache.org/jira/browse/IGNITE-17470 > Project: Ignite > Issue Type: Task > Components: spark >Reporter: Ivan Gagarkin >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17490) Packaging: SDKman package
[ https://issues.apache.org/jira/browse/IGNITE-17490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17490: - Labels: ignite-3 (was: ) > Packaging: SDKman package > - > > Key: IGNITE-17490 > URL: https://issues.apache.org/jira/browse/IGNITE-17490 > Project: Ignite > Issue Type: New Feature >Reporter: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17488) Packaging: Zip archive
[ https://issues.apache.org/jira/browse/IGNITE-17488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17488: - Labels: ignite-3 (was: ) > Packaging: Zip archive > --- > > Key: IGNITE-17488 > URL: https://issues.apache.org/jira/browse/IGNITE-17488 > Project: Ignite > Issue Type: New Feature >Reporter: Mikhail Pochatkin >Priority: Major > Labels: ignite-3 > > Create zip archive target with all Apache Ignite binaries and > install\uninstall\update scripts. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17476) Implement configurations event handling by index manager
[ https://issues.apache.org/jira/browse/IGNITE-17476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17476: - Labels: ignite-3 (was: ) > Implement configurations event handling by index manager > > > Key: IGNITE-17476 > URL: https://issues.apache.org/jira/browse/IGNITE-17476 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Assignee: Konstantin Orlov >Priority: Major > Labels: ignite-3 > > Need to implement listener for configurations event to reflect the state of > configuration an create all necessary runtime structures. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17473) Support transactional scan for RW transaction
[ https://issues.apache.org/jira/browse/IGNITE-17473?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17473: - Labels: ignite-3 (was: ) > Support transactional scan for RW transaction > - > > Key: IGNITE-17473 > URL: https://issues.apache.org/jira/browse/IGNITE-17473 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Priority: Major > Labels: ignite-3 > > Right now transaction not sending to remote nodes, so we don't use single > transaction for whole query (transaction scan is not possible). > Need to transfer transaction metadata to remote nodes and use it during scan. > Seems required transfer just transaction id and recover transaction by the id > (build transaction instance limited transaction, don't go to transaction > coordinator) on remote nodes, however need to understand should we do enlist > partitions for scan. > Start point: > org.apache.ignite.internal.sql.engine.exec.rel.TableScanNode#request -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17453) Calcite Engine: ORDER BY Optimization
[
https://issues.apache.org/jira/browse/IGNITE-17453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-17453:
-
Labels: calcite3-required (was: )
> Calcite Engine: ORDER BY Optimization
> -
>
> Key: IGNITE-17453
> URL: https://issues.apache.org/jira/browse/IGNITE-17453
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.13
>Reporter: YuJue Li
>Priority: Critical
> Labels: calcite3-required
> Fix For: 2.14
>
>
> 1.start a node;
> 2.create table:
> CREATE TABLE PI_COM_DAY
> (COM_ID VARCHAR(30) NOT NULL ,
> ITEM_ID VARCHAR(30) NOT NULL ,
> DATE1 VARCHAR(8) NOT NULL ,
> KIND VARCHAR(1),
> QTY_IOD DECIMAL(18, 6) ,
> AMT_IOD DECIMAL(18, 6) ,
> PRIMARYKEY (COM_ID,ITEM_ID,DATE1))
> WITH"template=partitioned,CACHE_NAME=PI_COM_DAY";
> CREATE INDEX IDX_PI_COM_DAY_ITEM_DATE ON PI_COM_DAY(ITEM_ID);
>
> 3.insert some data, for example, 1m;
> 4.execute sql:
> select * from pi_com_day order by item_id desc limit 10;
> normal.
> select /*+ QUERY_ENGINE('calcite') */ * from pi_com_day order by item_id desc
> limit 10;
> then OOM.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17445) RocksDbKeyValueStorage recreates DB on start, so data can't be found until Raft log is replayed
[
https://issues.apache.org/jira/browse/IGNITE-17445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-17445:
-
Labels: ignite-3 (was: )
> RocksDbKeyValueStorage recreates DB on start, so data can't be found until
> Raft log is replayed
> ---
>
> Key: IGNITE-17445
> URL: https://issues.apache.org/jira/browse/IGNITE-17445
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Chudov
>Priority: Major
> Labels: ignite-3
>
> RocksDbKeyValueStorage recreates DB on start. This means that entries that
> were put to this storage earlier, can or cant be found until raft log is
> replayed, i.e. the behavior is undefined. For example, this can cause
> assertion on node recovery:
> {code:java}
> java.lang.AssertionError: Configuration revision must be greater than local
> node applied revision [msRev=0, appliedRev=1
> {code}
> which means that applied revision in vault is 1 but only 0 is found in meta
> storage, as the storage of meta storage is being recreated.
> For now, the only thing that saves us from this assertion to be thrown every
> time, is that operations related to node recovery, applied from distributed
> configuration (see IgniteImpl#notifyConfigurationListeners ), take some time
> and raft log is small enough to replay faster than the performing of recovery
> operations.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17439) Small enhancements for BinaryTuple
[ https://issues.apache.org/jira/browse/IGNITE-17439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17439: - Labels: ignite-3 (was: ) > Small enhancements for BinaryTuple > -- > > Key: IGNITE-17439 > URL: https://issues.apache.org/jira/browse/IGNITE-17439 > Project: Ignite > Issue Type: Improvement > Components: ignite-3 >Reporter: Aleksey Demakov >Assignee: Aleksey Demakov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > In the initial BinaryTuple implementation there are a few places where the > code could be improved. E.g. in one place there is a field that could be > declared final, in another place there is a method that is quite useless, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17439) Small enhancements for BinaryTuple
[ https://issues.apache.org/jira/browse/IGNITE-17439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17439: - Component/s: (was: ignite-3) > Small enhancements for BinaryTuple > -- > > Key: IGNITE-17439 > URL: https://issues.apache.org/jira/browse/IGNITE-17439 > Project: Ignite > Issue Type: Improvement >Reporter: Aleksey Demakov >Assignee: Aleksey Demakov >Priority: Minor > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > In the initial BinaryTuple implementation there are a few places where the > code could be improved. E.g. in one place there is a field that could be > declared final, in another place there is a method that is quite useless, etc. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17424) Thin 3.0: C++ client
[ https://issues.apache.org/jira/browse/IGNITE-17424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17424: - Labels: ignite-3 (was: ) > Thin 3.0: C++ client > > > Key: IGNITE-17424 > URL: https://issues.apache.org/jira/browse/IGNITE-17424 > Project: Ignite > Issue Type: Epic > Components: thin client >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Labels: ignite-3 > > We need to implement C++ client for Ignite 3.x. > Let's adopt C++14 for a new client and consider adopting some library > repository for it -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17425) C++ 3.0: Basic C++ client
[ https://issues.apache.org/jira/browse/IGNITE-17425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17425: - Labels: ignite-3 (was: ) > C++ 3.0: Basic C++ client > - > > Key: IGNITE-17425 > URL: https://issues.apache.org/jira/browse/IGNITE-17425 > Project: Ignite > Issue Type: New Feature > Components: thin client >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Labels: ignite-3 > > Implement basic version of C++ client that will be able to get a table and > perform put and get operation on it. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17426) C++ 3.0: Implement table API
[ https://issues.apache.org/jira/browse/IGNITE-17426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17426: - Labels: ignite-3 (was: ) > C++ 3.0: Implement table API > > > Key: IGNITE-17426 > URL: https://issues.apache.org/jira/browse/IGNITE-17426 > Project: Ignite > Issue Type: New Feature > Components: thin client >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Labels: ignite-3 > > Implement basic table API -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17497) Support inheritance of polymorphic configurations
Roman Puchkovskiy created IGNITE-17497: -- Summary: Support inheritance of polymorphic configurations Key: IGNITE-17497 URL: https://issues.apache.org/jira/browse/IGNITE-17497 Project: Ignite Issue Type: Improvement Reporter: Roman Puchkovskiy Assignee: Roman Puchkovskiy Fix For: 3.0.0-alpha6 Currently, polymorphic configuration schemas must have exactly one parent class (not Object). It is suggested to implement the following logic: # Top config schema must be annotated with PolymorphicConfig (it already works as described here, so nothing needs to be done) # Leaf config schema must be annotated with PolymorphicConfigInstance (it already works as described here, so nothing needs to be done) # Intermediary config schema classes (extending, directly or indirectly, the top config schema and extended, directly or indirectly by leaf config schemas) are allowed. They do not need to be annotated. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17496) LWM may be after HWM (reserved) on primary after the node restart
[
https://issues.apache.org/jira/browse/IGNITE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-17496:
--
Description:
{code:java}
java.lang.AssertionError: LWM after HWM: lwm=10010, hwm=10003, cntr=Counter
[lwm=10010, missed=[10011 - 10012, 10021, 10031 - 10032, 10043 - 10044],
maxApplied=10047, hwm=10004]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:265)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getAndIncrementUpdateCounter(GridCacheOffheapManager.java:2538)
at
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.getAndIncrementUpdateCounter(GridDhtLocalPartition.java:942)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.calculatePartitionUpdateCounters(IgniteTxLocalAdapter.java:510)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1360)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:730)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1136)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:581)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:378)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:201)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:175)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:135)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:221)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750) {code}
was:
{code}
java.lang.AssertionError: LWM after reserved: lwm=10018, reserved=10008,
cntr=Counter [lwm=10018, missed=[10019 - 10020, 10025 - 10028, 10039 - 10042,
10049 - 10052], hwm=10058, reserved=10009]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:268)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
at
[jira] [Assigned] (IGNITE-17090) Map sql errors to messages that user see
[
https://issues.apache.org/jira/browse/IGNITE-17090?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Aleksandr reassigned IGNITE-17090:
--
Assignee: Aleksandr (was: Mikhail Pochatkin)
> Map sql errors to messages that user see
>
>
> Key: IGNITE-17090
> URL: https://issues.apache.org/jira/browse/IGNITE-17090
> Project: Ignite
> Issue Type: Task
>Reporter: Aleksandr
>Assignee: Aleksandr
>Priority: Major
> Labels: ignite-3, ignite-3-cli-tool
>
> There is a number of errors that user can get during sql command execution.
> Map all types of errors and map them to exit error codes and user messages.
> Also, a user has to understand what is wrong with the query. Now it displays
> jusn an abstract message.
>
> How it works now:
> {code:java}
> sql-cli> create table myta
> SQL query parsing error: Sql query execution failed.
> sql-cli> {code}
> {code:java}
> sql-cli> create table mytable(i int, j int);
> Unrecognized error while process SQL query.
> sql-cli> {code}
> "SQL query parsing error: Sql query execution failed.", "Unrecognized error
> while process SQL query." say nothing about {*}what exactly is wrong{*}.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17496) LWM may be after HWM (reserved) on primary after the node restart
[
https://issues.apache.org/jira/browse/IGNITE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-17496:
--
Fix Version/s: 2.14
> LWM may be after HWM (reserved) on primary after the node restart
> -
>
> Key: IGNITE-17496
> URL: https://issues.apache.org/jira/browse/IGNITE-17496
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Anton Vinogradov
>Assignee: Anton Vinogradov
>Priority: Major
> Labels: iep-31
> Fix For: 2.14
>
>
> {code}
> java.lang.AssertionError: LWM after reserved: lwm=10018, reserved=10008,
> cntr=Counter [lwm=10018, missed=[10019 - 10020, 10025 - 10028, 10039 - 10042,
> 10049 - 10052], hwm=10058, reserved=10009]
> at
> org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:268)
> at
> org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
> at
> org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getAndIncrementUpdateCounter(GridCacheOffheapManager.java:2538)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.getAndIncrementUpdateCounter(GridDhtLocalPartition.java:942)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.calculatePartitionUpdateCounters(IgniteTxLocalAdapter.java:510)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1360)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:730)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1136)
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:581)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:378)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:201)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:175)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:135)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:221)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
> at
> org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
> at
> org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
> at
> org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
> at
> org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
> at java.lang.Thread.run(Thread.java:750)
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17496) LWM may be after HWM (reserved) on primary after the node restart
[
https://issues.apache.org/jira/browse/IGNITE-17496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anton Vinogradov updated IGNITE-17496:
--
Description:
{code}
java.lang.AssertionError: LWM after reserved: lwm=10018, reserved=10008,
cntr=Counter [lwm=10018, missed=[10019 - 10020, 10025 - 10028, 10039 - 10042,
10049 - 10052], hwm=10058, reserved=10009]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:268)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getAndIncrementUpdateCounter(GridCacheOffheapManager.java:2538)
at
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.getAndIncrementUpdateCounter(GridDhtLocalPartition.java:942)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.calculatePartitionUpdateCounters(IgniteTxLocalAdapter.java:510)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1360)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:730)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1136)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:581)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:378)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:201)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:175)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:135)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:221)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750)
{code}
was:
{{code}}
java.lang.AssertionError: LWM after reserved: lwm=10018, reserved=10008,
cntr=Counter [lwm=10018, missed=[10019 - 10020, 10025 - 10028, 10039 - 10042,
10049 - 10052], hwm=10058, reserved=10009]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:268)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
[jira] [Created] (IGNITE-17496) LWM may be after HWM (reserved) on primary after the node restart
Anton Vinogradov created IGNITE-17496:
-
Summary: LWM may be after HWM (reserved) on primary after the node
restart
Key: IGNITE-17496
URL: https://issues.apache.org/jira/browse/IGNITE-17496
Project: Ignite
Issue Type: Sub-task
Reporter: Anton Vinogradov
Assignee: Anton Vinogradov
{{code}}
java.lang.AssertionError: LWM after reserved: lwm=10018, reserved=10008,
cntr=Counter [lwm=10018, missed=[10019 - 10020, 10025 - 10028, 10039 - 10042,
10049 - 10052], hwm=10058, reserved=10009]
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterTrackingImpl.reserve(PartitionUpdateCounterTrackingImpl.java:268)
at
org.apache.ignite.internal.processors.cache.PartitionUpdateCounterErrorWrapper.reserve(PartitionUpdateCounterErrorWrapper.java:58)
at
org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.getAndIncrementUpdateCounter(IgniteCacheOffheapManagerImpl.java:1620)
at
org.apache.ignite.internal.processors.cache.persistence.GridCacheOffheapManager$GridCacheDataStore.getAndIncrementUpdateCounter(GridCacheOffheapManager.java:2538)
at
org.apache.ignite.internal.processors.cache.distributed.dht.topology.GridDhtLocalPartition.getAndIncrementUpdateCounter(GridDhtLocalPartition.java:942)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.calculatePartitionUpdateCounters(IgniteTxLocalAdapter.java:510)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare0(GridDhtTxPrepareFuture.java:1360)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.mapIfLocked(GridDhtTxPrepareFuture.java:730)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxPrepareFuture.prepare(GridDhtTxPrepareFuture.java:1136)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.prepareAsync(GridDhtTxLocal.java:400)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:581)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.prepareNearTx(IgniteTxHandler.java:378)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest0(IgniteTxHandler.java:201)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.processNearTxPrepareRequest(IgniteTxHandler.java:175)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler.access$000(IgniteTxHandler.java:135)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:223)
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxHandler$1.apply(IgniteTxHandler.java:221)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1151)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:592)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:393)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:319)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:110)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:309)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1907)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1528)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$5300(GridIoManager.java:243)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.execute(GridIoManager.java:1421)
at
org.apache.ignite.internal.managers.communication.TraceRunnable.run(TraceRunnable.java:55)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.body(StripedExecutor.java:637)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:125)
at java.lang.Thread.run(Thread.java:750)
{{code}}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17469) cli profile commands unification
[
https://issues.apache.org/jira/browse/IGNITE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yury Yudin updated IGNITE-17469:
Description:
cli config profile now has two commands show and create, while setting a
profile is done through a parameter, which is confusing.
There should be a separate command to set the profile.
In general I would suggest the commands there to look like:
cli config show
cli config set
cli config get
All repl cli commands should not accept --profile or -p parameters, all setting
and getting of the params in a profile should go through its activation
instead. I.e. cli config profile create newprofile ; cli config profile
activate newprofile; cli config set
ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
We should retain --profile in non-repl mode.
Profile manipulation should go like below:
cli config profile show []
cli config profile list
cli config profile create --copy-from
cli config profile activate
Let’s please update IEP with the corresponding changes as well.
Let's also make sure non-cli commands do have --profile option and
auto-completion scripts are updated accordingly. Currently I don't see any
suggestions on --profile there:
{noformat}
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$ ./ignite cluster config show
--profile te
st
Unknown options: '--profile', 'test'
Usage: ignite cluster config show [-h] [--cluster-url=]
[--selector=]
Shows cluster configuration.
--cluster-url=
Url to ignite node.
-h, --help Show this help message and exit.
--selector=
Configuration path selector.
tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$
{noformat}
was:
cli config profile now has two commands show and create, while setting a
profile is done through a parameter, which is confusing.
There should be a separate command to set the profile.
In general I would suggest the commands there to look like:
cli config show
cli config set
cli config get
All repl cli commands should not accept --profile or -p parameters, all setting
and getting of the params in a profile should go through its activation
instead. I.e. cli config profile create newprofile ; cli config profile
activate newprofile; cli config set
ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
We should retain --profile in non-repl mode.
Profile manipulation should go like below:
cli config profile show []
cli config profile list
cli config profile create --copy-from
cli config profile activate
Let’s please update IEP with the corresponding changes as well.
> cli profile commands unification
>
>
> Key: IGNITE-17469
> URL: https://issues.apache.org/jira/browse/IGNITE-17469
> Project: Ignite
> Issue Type: Improvement
> Components: cli, ignite-3
>Reporter: Yury Yudin
>Assignee: Vadim Pakhnushev
>Priority: Major
> Labels: ignite-3
>
> cli config profile now has two commands show and create, while setting a
> profile is done through a parameter, which is confusing.
> There should be a separate command to set the profile.
> In general I would suggest the commands there to look like:
> cli config show
> cli config set
> cli config get
> All repl cli commands should not accept --profile or -p parameters, all
> setting and getting of the params in a profile should go through its
> activation instead. I.e. cli config profile create newprofile ; cli config
> profile activate newprofile; cli config set
> ignite.cluster-url=[http://localhost:10300|http://localhost:10300/]
> We should retain --profile in non-repl mode.
> Profile manipulation should go like below:
> cli config profile show []
> cli config profile list
> cli config profile create --copy-from
> cli config profile activate
>
> Let’s please update IEP with the corresponding changes as well.
>
> Let's also make sure non-cli commands do have --profile option and
> auto-completion scripts are updated accordingly. Currently I don't see any
> suggestions on --profile there:
> {noformat}
> tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$ ./ignite cluster config
> show --profile te
> st
> Unknown options: '--profile', 'test'
> Usage: ignite cluster config show [-h] [--cluster-url=]
> [--selector=]
> Shows cluster configuration.
> --cluster-url=
> Url to ignite node.
> -h, --help Show this help message and exit.
> --selector=
> Configuration path selector.
> tests@ubuntu:~/Work/apache-ignite-3.0.0-SNAPSHOT$
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-17076) Unify RowId format for different storages
[ https://issues.apache.org/jira/browse/IGNITE-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Bessonov updated IGNITE-17076: --- Description: Current MV store bridge API has a fatal flaw, born from a misunderstanding. There's a method called "insert" that generates RowId by itself. This is wrong, because it can lead to different id for the same row on the replica storage. This completely breaks everything. Every replicated write command, that inserts new value, should produce same row ids. There are several ways to achieve this: * Use timestamps as identifiers. This is not very convenient, because we would have to attach partition id on top of it. It's mandatory to know the partition of the row. * Use more complicated structure, for example a tuple of (raftCommitIndex, partitionId, batchCounter), where * ** raftCommitIndex is the index of write command that performs insertion. ** partitionId is an integer identifier of the partition. Could be 4 bytes, considering that there are plans to support more than 65000 partitions per table. ** batchCounter is used to differentiate insertions made in a single write command. We can limit it with 2 bytes to save a little bit of space, if it's necessary. I prefer the second option, but maybe it could be revised during the implementation. Of course, method "insert" should be removed from bridge API. Tests have to be updated. With the lack of RAFT group in storage tests, we can generate row ids artificially, it's not a big deal. EDIT: second option makes it difficult to use row ids in action request processor in cases when data is inserted. So, hybrid clock + partition id is a better option. EDIT 2: removing "insert" method from the API is out of scope for now. was: Current MV store bridge API has a fatal flaw, born from a misunderstanding. There's a method called "insert" that generates RowId by itself. This is wrong, because it can lead to different id for the same row on the replica storage. This completely breaks everything. Every replicated write command, that inserts new value, should produce same row ids. There are several ways to achieve this: * Use timestamps as identifiers. This is not very convenient, because we would have to attach partition id on top of it. It's mandatory to know the partition of the row. * Use more complicated structure, for example a tuple of (raftCommitIndex, partitionId, batchCounter), where * ** raftCommitIndex is the index of write command that performs insertion. ** partitionId is an integer identifier of the partition. Could be 4 bytes, considering that there are plans to support more than 65000 partitions per table. ** batchCounter is used to differentiate insertions made in a single write command. We can limit it with 2 bytes to save a little bit of space, if it's necessary. I prefer the second option, but maybe it could be revised during the implementation. Of course, method "insert" should be removed from bridge API. Tests have to be updated. With the lack of RAFT group in storage tests, we can generate row ids artificially, it's not a big deal. EDIT: second option makes it difficult to use row ids in action request processor in cases when data is inserted. So, hybrid clock + partition id is a better option. > Unify RowId format for different storages > - > > Key: IGNITE-17076 > URL: https://issues.apache.org/jira/browse/IGNITE-17076 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Current MV store bridge API has a fatal flaw, born from a misunderstanding. > There's a method called "insert" that generates RowId by itself. This is > wrong, because it can lead to different id for the same row on the replica > storage. This completely breaks everything. > Every replicated write command, that inserts new value, should produce same > row ids. There are several ways to achieve this: > * Use timestamps as identifiers. This is not very convenient, because we > would have to attach partition id on top of it. It's mandatory to know the > partition of the row. > * Use more complicated structure, for example a tuple of (raftCommitIndex, > partitionId, batchCounter), where > * > ** raftCommitIndex is the index of write command that performs insertion. > ** partitionId is an integer identifier of the partition. Could be 4 bytes, > considering that there are plans to support more than 65000 partitions per > table. > ** batchCounter is used to differentiate insertions made in a single write > command. We can limit it with 2 bytes to save a little bit of space, if it's > necessary. > I prefer the second option, but maybe it could be revised during the >
[jira] [Assigned] (IGNITE-17076) Unify RowId format for different storages
[ https://issues.apache.org/jira/browse/IGNITE-17076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Bessonov reassigned IGNITE-17076: -- Assignee: Ivan Bessonov > Unify RowId format for different storages > - > > Key: IGNITE-17076 > URL: https://issues.apache.org/jira/browse/IGNITE-17076 > Project: Ignite > Issue Type: Improvement >Reporter: Ivan Bessonov >Assignee: Ivan Bessonov >Priority: Major > Labels: ignite-3 > > Current MV store bridge API has a fatal flaw, born from a misunderstanding. > There's a method called "insert" that generates RowId by itself. This is > wrong, because it can lead to different id for the same row on the replica > storage. This completely breaks everything. > Every replicated write command, that inserts new value, should produce same > row ids. There are several ways to achieve this: > * Use timestamps as identifiers. This is not very convenient, because we > would have to attach partition id on top of it. It's mandatory to know the > partition of the row. > * Use more complicated structure, for example a tuple of (raftCommitIndex, > partitionId, batchCounter), where > * > ** raftCommitIndex is the index of write command that performs insertion. > ** partitionId is an integer identifier of the partition. Could be 4 bytes, > considering that there are plans to support more than 65000 partitions per > table. > ** batchCounter is used to differentiate insertions made in a single write > command. We can limit it with 2 bytes to save a little bit of space, if it's > necessary. > I prefer the second option, but maybe it could be revised during the > implementation. > Of course, method "insert" should be removed from bridge API. Tests have to > be updated. With the lack of RAFT group in storage tests, we can generate row > ids artificially, it's not a big deal. > EDIT: second option makes it difficult to use row ids in action request > processor in cases when data is inserted. So, hybrid clock + partition id is > a better option. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (IGNITE-17495) Create start, stop, restart scripts for ingite-db
Mikhail Pochatkin created IGNITE-17495: -- Summary: Create start, stop, restart scripts for ingite-db Key: IGNITE-17495 URL: https://issues.apache.org/jira/browse/IGNITE-17495 Project: Ignite Issue Type: Improvement Components: build Reporter: Mikhail Pochatkin -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17233) Clarify cluster URL parameter name
[ https://issues.apache.org/jira/browse/IGNITE-17233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-17233: - Fix Version/s: 3.0.0-alpha6 > Clarify cluster URL parameter name > -- > > Key: IGNITE-17233 > URL: https://issues.apache.org/jira/browse/IGNITE-17233 > Project: Ignite > Issue Type: Task > Components: cli >Reporter: Vadim Pakhnushev >Assignee: Vadim Pakhnushev >Priority: Major > Labels: ignite-3, ignite-3-cli-tool > Fix For: 3.0.0-alpha6 > > Time Spent: 0.5h > Remaining Estimate: 0h > > [IEP-88|https://cwiki.apache.org/confluence/display/IGNITE/IEP-88%3A+CLI+Tool] > states that commands for cluster management use --cluster-url parameter for > the endpoint which is unclear. In fact --cluster-url could be a URL to any > node in the cluster. > We could use --cluster-endpoint-url in cluster commands to clarify that it is > an endpoint to the cluster. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17469) cli profile commands unification
[ https://issues.apache.org/jira/browse/IGNITE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Pakhnushev updated IGNITE-17469: -- Description: cli config profile now has two commands show and create, while setting a profile is done through a parameter, which is confusing. There should be a separate command to set the profile. In general I would suggest the commands there to look like: cli config show cli config set cli config get All repl cli commands should not accept --profile or -p parameters, all setting and getting of the params in a profile should go through its activation instead. I.e. cli config profile create newprofile ; cli config profile activate newprofile; cli config set ignite.cluster-url=[http://localhost:10300|http://localhost:10300/] We should retain --profile in non-repl mode. Profile manipulation should go like below: cli config profile show [] cli config profile list cli config profile create --copy-from cli config profile activate Let’s please update IEP with the corresponding changes as well. was: cli config profile now has two commands show and create, while setting a profile is done through a parameter, which is confusing. There should be a separate command to set the profile. In general I would suggest the commands there to look like: cli config show cli config set cli config get All cli commands should not accept --profile or -p parameters, all setting and getting of the params in a profile should go through its activation instead. I.e. cli config profile create newprofile ; cli config profile activate newprofile; cli config set ignite.cluster-url=[http://localhost:10300|http://localhost:10300/] Profile manipulation should go like below: cli config profile show [] cli config profile list cli config profile create --copy-from cli config profile activate > cli profile commands unification > > > Key: IGNITE-17469 > URL: https://issues.apache.org/jira/browse/IGNITE-17469 > Project: Ignite > Issue Type: Improvement > Components: cli, ignite-3 >Reporter: Yury Yudin >Assignee: Vadim Pakhnushev >Priority: Major > Labels: ignite-3 > > cli config profile now has two commands show and create, while setting a > profile is done through a parameter, which is confusing. > There should be a separate command to set the profile. > In general I would suggest the commands there to look like: > cli config show > cli config set > cli config get > All repl cli commands should not accept --profile or -p parameters, all > setting and getting of the params in a profile should go through its > activation instead. I.e. cli config profile create newprofile ; cli config > profile activate newprofile; cli config set > ignite.cluster-url=[http://localhost:10300|http://localhost:10300/] > We should retain --profile in non-repl mode. > Profile manipulation should go like below: > cli config profile show [] > cli config profile list > cli config profile create --copy-from > cli config profile activate > > Let’s please update IEP with the corresponding changes as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17469) cli profile commands unification
[ https://issues.apache.org/jira/browse/IGNITE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Pakhnushev updated IGNITE-17469: -- Labels: ignite-3 (was: ) > cli profile commands unification > > > Key: IGNITE-17469 > URL: https://issues.apache.org/jira/browse/IGNITE-17469 > Project: Ignite > Issue Type: Improvement > Components: cli, ignite-3 >Reporter: Yury Yudin >Assignee: Vadim Pakhnushev >Priority: Major > Labels: ignite-3 > > cli config profile now has two commands show and create, while setting a > profile is done through a parameter, which is confusing. > There should be a separate command to set the profile. > In general I would suggest the commands there to look like: > cli config show > cli config set > cli config get > All cli commands should not accept --profile or -p parameters, all setting > and getting of the params in a profile should go through its activation > instead. I.e. cli config profile create newprofile ; cli config profile > activate newprofile; cli config set > ignite.cluster-url=[http://localhost:10300|http://localhost:10300/] > Profile manipulation should go like below: > cli config profile show [] > cli config profile list > cli config profile create --copy-from > cli config profile activate > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (IGNITE-17469) cli profile commands unification
[ https://issues.apache.org/jira/browse/IGNITE-17469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vadim Pakhnushev reassigned IGNITE-17469: - Assignee: Vadim Pakhnushev > cli profile commands unification > > > Key: IGNITE-17469 > URL: https://issues.apache.org/jira/browse/IGNITE-17469 > Project: Ignite > Issue Type: Improvement > Components: cli, ignite-3 >Reporter: Yury Yudin >Assignee: Vadim Pakhnushev >Priority: Major > > cli config profile now has two commands show and create, while setting a > profile is done through a parameter, which is confusing. > There should be a separate command to set the profile. > In general I would suggest the commands there to look like: > cli config show > cli config set > cli config get > All cli commands should not accept --profile or -p parameters, all setting > and getting of the params in a profile should go through its activation > instead. I.e. cli config profile create newprofile ; cli config profile > activate newprofile; cli config set > ignite.cluster-url=[http://localhost:10300|http://localhost:10300/] > Profile manipulation should go like below: > cli config profile show [] > cli config profile list > cli config profile create --copy-from > cli config profile activate > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-17494) Python thin: Fix client authentication/TLS defaults
[ https://issues.apache.org/jira/browse/IGNITE-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577250#comment-17577250 ] Evgeny Stanilovsky commented on IGNITE-17494: - looks good, plz proceed. > Python thin: Fix client authentication/TLS defaults > --- > > Key: IGNITE-17494 > URL: https://issues.apache.org/jira/browse/IGNITE-17494 > Project: Ignite > Issue Type: New Feature > Components: python, thin client >Affects Versions: python-0.5.2 >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Fix For: python-0.6.0 > > > If you specify a username/password to the Python thin-client it automatically > switches use_ssl to True. The logic here is understandable but it’s confusing > when two orthogonal settings are related like this. Some users mentioned it > too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12243: --- Epic Link: IGNITE-17460 > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 3.0 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12243) [Spark] Add support of HAVING without GROUP BY
[ https://issues.apache.org/jira/browse/IGNITE-12243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12243: --- Parent: (was: IGNITE-12054) Issue Type: Bug (was: Sub-task) > [Spark] Add support of HAVING without GROUP BY > -- > > Key: IGNITE-12243 > URL: https://issues.apache.org/jira/browse/IGNITE-12243 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Critical > Labels: await > Fix For: 3.0 > > > Also the semantic of Having support was changed in Spark > [https://github.com/apache/spark/pull/22696/files] > https://issues.apache.org/jira/browse/SPARK-25708 > Now Having could be a legal operation without GroupBY > > Rewrite the test "SELECT id FROM city HAVING id > 1" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12244) [Spark] Fix test with Multiple Joins
[ https://issues.apache.org/jira/browse/IGNITE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12244: --- Epic Link: IGNITE-17460 > [Spark] Fix test with Multiple Joins > > > Key: IGNITE-12244 > URL: https://issues.apache.org/jira/browse/IGNITE-12244 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Minor > Fix For: 3.0 > > > Fix test "JOIN 3 TABLE" after investigation in strange join plan generation > in spark 2.4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12244) [Spark] Fix test with Multiple Joins
[ https://issues.apache.org/jira/browse/IGNITE-12244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12244: --- Parent: (was: IGNITE-12054) Issue Type: Bug (was: Sub-task) > [Spark] Fix test with Multiple Joins > > > Key: IGNITE-12244 > URL: https://issues.apache.org/jira/browse/IGNITE-12244 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Minor > Fix For: 3.0 > > > Fix test "JOIN 3 TABLE" after investigation in strange join plan generation > in spark 2.4. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12246) [Spark] Add support of changes in External Catalog
[ https://issues.apache.org/jira/browse/IGNITE-12246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12246: --- Epic Link: IGNITE-17460 > [Spark] Add support of changes in External Catalog > -- > > Key: IGNITE-12246 > URL: https://issues.apache.org/jira/browse/IGNITE-12246 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 3.0 > > > It leads to problems with schemas > > For example, next tests are muted now in Spark 2.4 > "Additional features for IgniteSparkSession" > and > "Should allow Spark SQL to create a table" > "Should disallow creation of tables in non-PUBLIC schemas" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-17494) Python thin: Fix client authentication/TLS defaults
[ https://issues.apache.org/jira/browse/IGNITE-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Igor Sapego updated IGNITE-17494: - Reviewer: Evgeny Stanilovsky > Python thin: Fix client authentication/TLS defaults > --- > > Key: IGNITE-17494 > URL: https://issues.apache.org/jira/browse/IGNITE-17494 > Project: Ignite > Issue Type: New Feature > Components: python, thin client >Affects Versions: python-0.5.2 >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Fix For: python-0.6.0 > > > If you specify a username/password to the Python thin-client it automatically > switches use_ssl to True. The logic here is understandable but it’s confusing > when two orthogonal settings are related like this. Some users mentioned it > too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12246) [Spark] Add support of changes in External Catalog
[ https://issues.apache.org/jira/browse/IGNITE-12246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12246: --- Parent: (was: IGNITE-12054) Issue Type: Bug (was: Sub-task) > [Spark] Add support of changes in External Catalog > -- > > Key: IGNITE-12246 > URL: https://issues.apache.org/jira/browse/IGNITE-12246 > Project: Ignite > Issue Type: Bug > Components: spark >Affects Versions: 2.9 >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 3.0 > > > It leads to problems with schemas > > For example, next tests are muted now in Spark 2.4 > "Additional features for IgniteSparkSession" > and > "Should allow Spark SQL to create a table" > "Should disallow creation of tables in non-PUBLIC schemas" -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Comment Edited] (IGNITE-17494) Python thin: Fix client authentication/TLS defaults
[ https://issues.apache.org/jira/browse/IGNITE-17494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577121#comment-17577121 ] Igor Sapego edited comment on IGNITE-17494 at 8/9/22 7:48 AM: -- [~zstan], can you take a look? was (Author: isapego): [~jooger], can you take a look? > Python thin: Fix client authentication/TLS defaults > --- > > Key: IGNITE-17494 > URL: https://issues.apache.org/jira/browse/IGNITE-17494 > Project: Ignite > Issue Type: New Feature > Components: python, thin client >Affects Versions: python-0.5.2 >Reporter: Igor Sapego >Assignee: Igor Sapego >Priority: Major > Fix For: python-0.6.0 > > > If you specify a username/password to the Python thin-client it automatically > switches use_ssl to True. The logic here is understandable but it’s confusing > when two orthogonal settings are related like this. Some users mentioned it > too. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (IGNITE-12519) Spark SQL not working with NON upper case column names
[
https://issues.apache.org/jira/browse/IGNITE-12519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Gagarkin updated IGNITE-12519:
---
Epic Link: IGNITE-17460
> Spark SQL not working with NON upper case column names
> ---
>
> Key: IGNITE-12519
> URL: https://issues.apache.org/jira/browse/IGNITE-12519
> Project: Ignite
> Issue Type: Bug
> Components: spark
>Affects Versions: 2.7.6
> Environment: 1) Spark 2.3.0 (Tried on Mesos Master and Local Master)
> 2) Ignite 2.7.6 (10 Nodes Cluster on Kubernetes)
> 3) Spark Ignite 2.7.6
>Reporter: Praneeth Ramesh
>Priority: Major
> Labels: dataframe, spark, spark-shell
>
> I created a simple table as below.
> {code:java}
> CREATE TABLE acc (
> "accId" VARCHAR PRIMARY KEY,
> "accCol1" VARCHAR,
> "accCol2" INT,
> "accCol3" VARCHAR,
> "accCol4" BOOLEAN
> );{code}
> And trying to read the data from table from Ignite Spark as below.
>
> {code:java}
> val igniteDF = spark.read
> .format(FORMAT_IGNITE)
> .option(OPTION_TABLE, "acc")
> .option(OPTION_CONFIG_FILE, "example-config.xml")
> .load()
> igniteDF.show(100, false)
> {code}
>
> But I see an exception as below.
> {code:java}
> Caused by: org.h2.jdbc.JdbcSQLException: Column "ACCCOL1" not found; SQL
> statement:
> SELECT accCol4, CAST(accCol1 AS VARCHAR) AS accCol1, accCol2, CAST(accCol3 AS
> VARCHAR) AS accCol3, accId FROM ACC LIMIT 21 [42122-197]
> at org.h2.message.DbException.getJdbcSQLException(DbException.java:357)
> at org.h2.message.DbException.get(DbException.java:179)
> at org.h2.message.DbException.get(DbException.java:155)
> at org.h2.expression.ExpressionColumn.optimize(ExpressionColumn.java:150)
> at org.h2.command.dml.Select.prepare(Select.java:858)
> at org.h2.command.Parser.prepareCommand(Parser.java:283)
> at org.h2.engine.Session.prepareLocal(Session.java:611)
> at org.h2.engine.Session.prepareCommand(Session.java:549)
> at org.h2.jdbc.JdbcConnection.prepareCommand(JdbcConnection.java:1247)
> at org.h2.jdbc.JdbcPreparedStatement.(JdbcPreparedStatement.java:76)
> at org.h2.jdbc.JdbcConnection.prepareStatement(JdbcConnection.java:694)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepare0(IgniteH2Indexing.java:539)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:509)
> at
> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing.prepareStatement(IgniteH2Indexing.java:476){code}
>
> When I try naming the TABLE cols with UPPER CASE everything works fine. But
> when I use the quotes in the Column Names to preserve the case, then it
> breaks with the exception.
> From exception I can see query built is still having the UPPER case column
> name ACCCOL1 instead of the camel case column names.
> Is there any workaround for this.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-12432) [Spark] Need to add test for AVG function in IgniteOptimizationAggregationFuncSpec
[
https://issues.apache.org/jira/browse/IGNITE-12432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Gagarkin updated IGNITE-12432:
---
Epic Link: IGNITE-17460
> [Spark] Need to add test for AVG function in
> IgniteOptimizationAggregationFuncSpec
> --
>
> Key: IGNITE-12432
> URL: https://issues.apache.org/jira/browse/IGNITE-12432
> Project: Ignite
> Issue Type: Test
> Components: spark
>Reporter: Alexey Zinoviev
>Assignee: Alexey Zinoviev
>Priority: Major
> Fix For: 3.0
>
>
> The test is skipped with TODO: write me
> it("AVG - DECIMAL") {
> //TODO: write me
> }
> It should be merged to 2.3 and 2.4 Spark together
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (IGNITE-12435) [Spark] Add support for saving to existing table via saveAsTable
[ https://issues.apache.org/jira/browse/IGNITE-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12435: --- Epic Link: IGNITE-17460 > [Spark] Add support for saving to existing table via saveAsTable > > > Key: IGNITE-12435 > URL: https://issues.apache.org/jira/browse/IGNITE-12435 > Project: Ignite > Issue Type: Bug > Components: spark >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 3.0 > > > Tests in IgniteSQLDataFrameIgniteSessionWriteSpec are muted due to strange > error related to working with filesystems and schemas > > All three tests generates the same error when you are trying to call > saveAsTable as a terminal operation on dataframe write: > java.io.IOException: No FileSystem for scheme: ignitejava.io.IOException: No > FileSystem for scheme: ignite > at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2586) > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2593) at > org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632) at > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614) at > org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at > org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at > org.apache.spark.sql.catalyst.catalog.SessionCatalog.validateTableLocation(SessionCatalog.scala:333) > at > org.apache.spark.sql.execution.command.CreateDataSourceTableAsSelectCommand.run(createDataSourceTables.scala:170) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:104) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:102) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:122) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155) > at > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) > at > org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152) at > org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127) at > org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80) > at > org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80) > at > org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676) > at > org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676) > at > org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78) > at > org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125) > at > org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73) > at > org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676) at > org.apache.spark.sql.DataFrameWriter.createTable(DataFrameWriter.scala:474) > at > org.apache.spark.sql.DataFrameWriter.saveAsTable(DataFrameWriter.scala:449) > at > org.apache.spark.sql.DataFrameWriter.saveAsTable(DataFrameWriter.scala:409) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:45) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:35) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:35) > at > org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) > at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at > org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at > org.scalatest.Transformer.apply(Transformer.scala:22) at > org.scalatest.Transformer.apply(Transformer.scala:20) at > org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:422) at > org.scalatest.Suite$class.withFixture(Suite.scala:1122) at > org.scalatest.FunSpec.withFixture(FunSpec.scala:1626) at > org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:419) at > org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:431) at > org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:431) at >
[jira] [Updated] (IGNITE-12435) [Spark] Add support for saving to existing table via saveAsTable
[ https://issues.apache.org/jira/browse/IGNITE-12435?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Gagarkin updated IGNITE-12435: --- Parent: (was: IGNITE-12054) Issue Type: Bug (was: Sub-task) > [Spark] Add support for saving to existing table via saveAsTable > > > Key: IGNITE-12435 > URL: https://issues.apache.org/jira/browse/IGNITE-12435 > Project: Ignite > Issue Type: Bug > Components: spark >Reporter: Alexey Zinoviev >Assignee: Alexey Zinoviev >Priority: Major > Fix For: 3.0 > > > Tests in IgniteSQLDataFrameIgniteSessionWriteSpec are muted due to strange > error related to working with filesystems and schemas > > All three tests generates the same error when you are trying to call > saveAsTable as a terminal operation on dataframe write: > java.io.IOException: No FileSystem for scheme: ignitejava.io.IOException: No > FileSystem for scheme: ignite > at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2586) > at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2593) at > org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91) at > org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2632) at > org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2614) at > org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370) at > org.apache.hadoop.fs.Path.getFileSystem(Path.java:296) at > org.apache.spark.sql.catalyst.catalog.SessionCatalog.validateTableLocation(SessionCatalog.scala:333) > at > org.apache.spark.sql.execution.command.CreateDataSourceTableAsSelectCommand.run(createDataSourceTables.scala:170) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult$lzycompute(commands.scala:104) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.sideEffectResult(commands.scala:102) > at > org.apache.spark.sql.execution.command.DataWritingCommandExec.doExecute(commands.scala:122) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:131) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$execute$1.apply(SparkPlan.scala:127) > at > org.apache.spark.sql.execution.SparkPlan$$anonfun$executeQuery$1.apply(SparkPlan.scala:155) > at > org.apache.spark.rdd.RDDOperationScope$.withScope(RDDOperationScope.scala:151) > at > org.apache.spark.sql.execution.SparkPlan.executeQuery(SparkPlan.scala:152) at > org.apache.spark.sql.execution.SparkPlan.execute(SparkPlan.scala:127) at > org.apache.spark.sql.execution.QueryExecution.toRdd$lzycompute(QueryExecution.scala:80) > at > org.apache.spark.sql.execution.QueryExecution.toRdd(QueryExecution.scala:80) > at > org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676) > at > org.apache.spark.sql.DataFrameWriter$$anonfun$runCommand$1.apply(DataFrameWriter.scala:676) > at > org.apache.spark.sql.execution.SQLExecution$$anonfun$withNewExecutionId$1.apply(SQLExecution.scala:78) > at > org.apache.spark.sql.execution.SQLExecution$.withSQLConfPropagated(SQLExecution.scala:125) > at > org.apache.spark.sql.execution.SQLExecution$.withNewExecutionId(SQLExecution.scala:73) > at > org.apache.spark.sql.DataFrameWriter.runCommand(DataFrameWriter.scala:676) at > org.apache.spark.sql.DataFrameWriter.createTable(DataFrameWriter.scala:474) > at > org.apache.spark.sql.DataFrameWriter.saveAsTable(DataFrameWriter.scala:449) > at > org.apache.spark.sql.DataFrameWriter.saveAsTable(DataFrameWriter.scala:409) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:45) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:35) > at > org.apache.ignite.spark.IgniteSQLDataFrameIgniteSessionWriteSpec$$anonfun$1$$anonfun$apply$mcV$sp$1.apply(IgniteSQLDataFrameIgniteSessionWriteSpec.scala:35) > at > org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22) > at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85) at > org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104) at > org.scalatest.Transformer.apply(Transformer.scala:22) at > org.scalatest.Transformer.apply(Transformer.scala:20) at > org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:422) at > org.scalatest.Suite$class.withFixture(Suite.scala:1122) at > org.scalatest.FunSpec.withFixture(FunSpec.scala:1626) at > org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:419) at > org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:431) at >
[jira] [Assigned] (IGNITE-17476) Implement configurations event handling by index manager
[ https://issues.apache.org/jira/browse/IGNITE-17476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov reassigned IGNITE-17476: - Assignee: Konstantin Orlov > Implement configurations event handling by index manager > > > Key: IGNITE-17476 > URL: https://issues.apache.org/jira/browse/IGNITE-17476 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Konstantin Orlov >Assignee: Konstantin Orlov >Priority: Major > > Need to implement listener for configurations event to reflect the state of > configuration an create all necessary runtime structures. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (IGNITE-14937) Introduce index management
[ https://issues.apache.org/jira/browse/IGNITE-14937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577190#comment-17577190 ] Evgeny Stanilovsky commented on IGNITE-14937: - looks goot, plz proceed. > Introduce index management > -- > > Key: IGNITE-14937 > URL: https://issues.apache.org/jira/browse/IGNITE-14937 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Taras Ledkov >Assignee: Konstantin Orlov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha6 > > Time Spent: 3h > Remaining Estimate: 0h > > We need to introduce component to handle index-related manipulation commands > (like CREATE or DROP) as well as manage indices' lifecycle. > As a first phase let's introduce index manager with functionality reduced to > the following duties: > * accepting CREATE|DROP command and changing the schema > * --handling the event from configuration and creating reduced set of > index-related structures (index description only for now) on all necessary > nodes-- -- This message was sent by Atlassian Jira (v8.20.10#820010)
