[jira] [Updated] (IGNITE-18439) Calcite engine. Expired entries are not filtered by scans

2023-01-12 Thread Aleksey Plekhanov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksey Plekhanov updated IGNITE-18439:
---
Labels: calcite  (was: calcite calcite2-required)

> Calcite engine. Expired entries are not filtered by scans
> -
>
> Key: IGNITE-18439
> URL: https://issues.apache.org/jira/browse/IGNITE-18439
> Project: Ignite
>  Issue Type: Bug
>Reporter: Aleksey Plekhanov
>Assignee: Aleksey Plekhanov
>Priority: Major
>  Labels: calcite
> Fix For: 2.15
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> Expired entries should be filtered when it's possible (table scans, regular 
> index scans). Some types of optimizations (inline scans) don't have access to 
> expireTime field, perhaps we should disable these optimizations for caches 
> with {{{}eagerTtl == false{}}}.
> Reproducer:
> {code:java}
> public void testExpiration() throws Exception {
> IgniteCache cache = client.getOrCreateCache(new 
> CacheConfiguration()
> .setName("dev")
> .setSqlSchema("PUBLIC")
> .setIndexedTypes(Integer.class, Developer.class)
> .setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new 
> Duration(TimeUnit.MILLISECONDS, 1)))
> .setEagerTtl(false)
> );
>  
> for (int i = 0; i < 10; i++)
> cache.put(i, new Developer("name" + i, i));
>         doSleep(100);
> assertQuery("SELECT * FROM PUBLIC.DEVELOPER").resultSize(0).check();
> }
> {code}
>  



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18546) Cache clean control.sh command

2023-01-12 Thread Maksim Timonin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Maksim Timonin updated IGNITE-18546:

Labels: ise  (was: )

> Cache clean control.sh command 
> ---
>
> Key: IGNITE-18546
> URL: https://issues.apache.org/jira/browse/IGNITE-18546
> Project: Ignite
>  Issue Type: New Feature
>Reporter: Maksim Timonin
>Assignee: Maksim Timonin
>Priority: Major
>  Labels: ise
>
> IgniteCache provides a functionality for cleaning cache (IgniteCache#clean).
> Users might want clear cache for test purposes by remote job (Jenkins, etc.). 
> Then it will be useful to provide opportunity to clean cache by control.sh 
> util.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18546) Cache clean control.sh command

2023-01-12 Thread Maksim Timonin (Jira)
Maksim Timonin created IGNITE-18546:
---

 Summary: Cache clean control.sh command 
 Key: IGNITE-18546
 URL: https://issues.apache.org/jira/browse/IGNITE-18546
 Project: Ignite
  Issue Type: New Feature
Reporter: Maksim Timonin
Assignee: Maksim Timonin


IgniteCache provides a functionality for cleaning cache (IgniteCache#clean).

Users might want clear cache for test purposes by remote job (Jenkins, etc.). 
Then it will be useful to provide opportunity to clean cache by control.sh util.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17915) Generalize support for inversion of control mechanisms

2023-01-12 Thread Maxim Muzafarov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676388#comment-17676388
 ] 

Maxim Muzafarov commented on IGNITE-17915:
--

[~ldywicki] I've added you to the Contributors list. Please, check that you're 
able to assign the issue.

> Generalize support for inversion of control  mechanisms
> ---
>
> Key: IGNITE-17915
> URL: https://issues.apache.org/jira/browse/IGNITE-17915
> Project: Ignite
>  Issue Type: Improvement
>  Components: spring
>Affects Versions: 2.14
>Reporter: Łukasz Dywicki
>Priority: Major
>  Labels: patch-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Current inversion of control support for Ignite is bound to Spring framework 
> and Spring ApplicationContext interface which is used to lookup beans by type 
> or name.
> Above lookup criteria are fairly generic and can be used to first - extract, 
> second - generalize support for foreign injections within Ignite itself. With 
> little work it is possible to open this for further extensions. This is known 
> pattern used in other Apache projects, ie. 
> [{{BeanRepository}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/BeanRepository.java]
>  and 
> [{{Registry}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/Registry.java].
> Main motivation is Ignite running under OSGi which does not host Spring 
> (Spring framework dropped support for OSGi a while ago), and has alternative 
> ways to provide similar functionality. For OSGi there are mechanisms such as 
> OSGi Blueprint and OSGi Service Registry itself.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17915) Generalize support for inversion of control mechanisms

2023-01-12 Thread Jira


[ 
https://issues.apache.org/jira/browse/IGNITE-17915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676366#comment-17676366
 ] 

Łukasz Dywicki commented on IGNITE-17915:
-

[~mmuzaf] since I am not contributor in project I can not update issue. Feel 
free to change assignee to me if needed.

> Generalize support for inversion of control  mechanisms
> ---
>
> Key: IGNITE-17915
> URL: https://issues.apache.org/jira/browse/IGNITE-17915
> Project: Ignite
>  Issue Type: Improvement
>  Components: spring
>Affects Versions: 2.14
>Reporter: Łukasz Dywicki
>Priority: Major
>  Labels: patch-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Current inversion of control support for Ignite is bound to Spring framework 
> and Spring ApplicationContext interface which is used to lookup beans by type 
> or name.
> Above lookup criteria are fairly generic and can be used to first - extract, 
> second - generalize support for foreign injections within Ignite itself. With 
> little work it is possible to open this for further extensions. This is known 
> pattern used in other Apache projects, ie. 
> [{{BeanRepository}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/BeanRepository.java]
>  and 
> [{{Registry}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/Registry.java].
> Main motivation is Ignite running under OSGi which does not host Spring 
> (Spring framework dropped support for OSGi a while ago), and has alternative 
> ways to provide similar functionality. For OSGi there are mechanisms such as 
> OSGi Blueprint and OSGi Service Registry itself.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-17915) Generalize support for inversion of control mechanisms

2023-01-12 Thread Maxim Muzafarov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-17915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676364#comment-17676364
 ] 

Maxim Muzafarov commented on IGNITE-17915:
--

I will take a look within the next few weeks. 
[~ldywicki] you can assign the issue to yourself and set the right status. 

> Generalize support for inversion of control  mechanisms
> ---
>
> Key: IGNITE-17915
> URL: https://issues.apache.org/jira/browse/IGNITE-17915
> Project: Ignite
>  Issue Type: Improvement
>  Components: spring
>Affects Versions: 2.14
>Reporter: Łukasz Dywicki
>Priority: Major
>  Labels: patch-available
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> Current inversion of control support for Ignite is bound to Spring framework 
> and Spring ApplicationContext interface which is used to lookup beans by type 
> or name.
> Above lookup criteria are fairly generic and can be used to first - extract, 
> second - generalize support for foreign injections within Ignite itself. With 
> little work it is possible to open this for further extensions. This is known 
> pattern used in other Apache projects, ie. 
> [{{BeanRepository}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/BeanRepository.java]
>  and 
> [{{Registry}}|https://github.com/apache/camel/blob/camel-3.19.0/core/camel-api/src/main/java/org/apache/camel/spi/Registry.java].
> Main motivation is Ignite running under OSGi which does not host Spring 
> (Spring framework dropped support for OSGi a while ago), and has alternative 
> ways to provide similar functionality. For OSGi there are mechanisms such as 
> OSGi Blueprint and OSGi Service Registry itself.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18494) Near cache not created with getCache

2023-01-12 Thread Ivan Daschinsky (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Daschinsky updated IGNITE-18494:
-
Labels: .NET ise  (was: ise)

> Near cache not created with getCache
> 
>
> Key: IGNITE-18494
> URL: https://issues.apache.org/jira/browse/IGNITE-18494
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: .NET, ise
> Fix For: 2.15
>
> Attachments: Issue.java, NearCacheTest.cs, NearCacheTest.java
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The problem:
> On affinity nodes everything works.
> On client nodes and non-affinity nodes (i.e. filtered by some node filter) -- 
> near configuration ignored when starting cache on demand from descriptor.
> ---
> _The documentation 
> (https://ignite.apache.org/docs/latest/configuring-caches/near-cache#configuring-near-cache)
>  says:
> "Once configured in this way, the near cache is created on any node that 
> requests data from the underlying cache, including both server nodes and 
> client nodes."
> We expect the Near cache to be created for the cache proxy obtained with 
> getCache, and getOrCreateNearCache does not need to be called because the 
> NearConfiguration was initialized when the cache was created.
> Reproducers show that this is not so.
> Java reproducer [^Issue.java] without platformcache._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18494) Near cache not created with getCache

2023-01-12 Thread Ivan Daschinsky (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Daschinsky updated IGNITE-18494:
-
Fix Version/s: 2.15

> Near cache not created with getCache
> 
>
> Key: IGNITE-18494
> URL: https://issues.apache.org/jira/browse/IGNITE-18494
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: ise
> Fix For: 2.15
>
> Attachments: Issue.java, NearCacheTest.cs, NearCacheTest.java
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The problem:
> On affinity nodes everything works.
> On client nodes and non-affinity nodes (i.e. filtered by some node filter) -- 
> near configuration ignored when starting cache on demand from descriptor.
> ---
> _The documentation 
> (https://ignite.apache.org/docs/latest/configuring-caches/near-cache#configuring-near-cache)
>  says:
> "Once configured in this way, the near cache is created on any node that 
> requests data from the underlying cache, including both server nodes and 
> client nodes."
> We expect the Near cache to be created for the cache proxy obtained with 
> getCache, and getOrCreateNearCache does not need to be called because the 
> NearConfiguration was initialized when the cache was created.
> Reproducers show that this is not so.
> Java reproducer [^Issue.java] without platformcache._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18494) Near cache not created with getCache

2023-01-12 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676333#comment-17676333
 ] 

Ignite TC Bot commented on IGNITE-18494:


{panel:title=Branch: [pull/10476/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10476/head] Base: [master] : New Tests 
(11)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Core Linux){color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6995351]]
* {color:#013220}DotNetCore: 
PlatformCacheTopologyChangeTest.TestClientNodeReconnectWithClusterRestartKeepsPlatformCache
 - PASSED{color}

{color:#8b}Platform .NET (Windows){color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6995619]]
* {color:#013220}exe: 
PlatformCacheTopologyChangeTest.TestClientNodeReconnectWithClusterRestartKeepsPlatformCache
 - PASSED{color}

{color:#8b}Cache 2{color} [[tests 
9|https://ci2.ignite.apache.org/viewLog.html?buildId=6996193]]
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= CLIENT] - PASSED{color}

{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6994467buildTypeId=IgniteTests24Java8_RunAll]

> Near cache not created with getCache
> 
>
> Key: IGNITE-18494
> URL: https://issues.apache.org/jira/browse/IGNITE-18494
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: ise
> Attachments: Issue.java, NearCacheTest.cs, NearCacheTest.java
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The problem:
> On affinity nodes everything works.
> On client nodes and non-affinity nodes (i.e. filtered by some node filter) -- 
> near configuration ignored when starting cache on demand from descriptor.
> ---
> _The documentation 
> (https://ignite.apache.org/docs/latest/configuring-caches/near-cache#configuring-near-cache)
>  says:
> "Once configured in this way, the near cache is created on any node that 
> requests data from the underlying cache, including both server nodes and 
> client nodes."
> We expect the Near cache to be created for the cache proxy obtained with 
> getCache, and getOrCreateNearCache does not need to be called because the 
> NearConfiguration was initialized when the cache was created.
> Reproducers show that this is not so.
> Java reproducer [^Issue.java] without platformcache._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18494) Near cache not created with getCache

2023-01-12 Thread Ignite TC Bot (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18494?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676332#comment-17676332
 ] 

Ignite TC Bot commented on IGNITE-18494:


{panel:title=Branch: [pull/10476/head] Base: [master] : No blockers 
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/10476/head] Base: [master] : New Tests 
(11)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Core Linux){color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6995351]]
* {color:#013220}DotNetCore: 
PlatformCacheTopologyChangeTest.TestClientNodeReconnectWithClusterRestartKeepsPlatformCache
 - PASSED{color}

{color:#8b}Platform .NET (Windows){color} [[tests 
1|https://ci2.ignite.apache.org/viewLog.html?buildId=6995619]]
* {color:#013220}exe: 
PlatformCacheTopologyChangeTest.TestClientNodeReconnectWithClusterRestartKeepsPlatformCache
 - PASSED{color}

{color:#8b}Cache 2{color} [[tests 
9|https://ci2.ignite.apache.org/viewLog.html?buildId=6996193]]
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = AFFINITY, nodeNearCheck = 
NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
NON_AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = CLIENT, nodeNearCheck = 
AFFINITY] - PASSED{color}
* {color:#013220}IgniteCacheTestSuite2: 
GridCacheNearDynamicStartTest.test[nodeCacheStart = NON_AFFINITY, nodeNearCheck 
= CLIENT] - PASSED{color}

{panel}
[TeamCity *-- Run :: All* 
Results|https://ci2.ignite.apache.org/viewLog.html?buildId=6994467buildTypeId=IgniteTests24Java8_RunAll]

> Near cache not created with getCache
> 
>
> Key: IGNITE-18494
> URL: https://issues.apache.org/jira/browse/IGNITE-18494
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Assignee: Ivan Daschinsky
>Priority: Major
>  Labels: ise
> Attachments: Issue.java, NearCacheTest.cs, NearCacheTest.java
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The problem:
> On affinity nodes everything works.
> On client nodes and non-affinity nodes (i.e. filtered by some node filter) -- 
> near configuration ignored when starting cache on demand from descriptor.
> ---
> _The documentation 
> (https://ignite.apache.org/docs/latest/configuring-caches/near-cache#configuring-near-cache)
>  says:
> "Once configured in this way, the near cache is created on any node that 
> requests data from the underlying cache, including both server nodes and 
> client nodes."
> We expect the Near cache to be created for the cache proxy obtained with 
> getCache, and getOrCreateNearCache does not need to be called because the 
> NearConfiguration was initialized when the cache was created.
> Reproducers show that this is not so.
> Java reproducer [^Issue.java] without platformcache._



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18545) Compute "withXXX" modifiers in some cases are not applied to the user compute method call.

2023-01-12 Thread Mikhail Petrov (Jira)
Mikhail Petrov created IGNITE-18545:
---

 Summary: Compute "withXXX" modifiers in some cases are not applied 
to the user compute method call.
 Key: IGNITE-18545
 URL: https://issues.apache.org/jira/browse/IGNITE-18545
 Project: Ignite
  Issue Type: Bug
Reporter: Mikhail Petrov


When the "withXXX" IgniteCompute methods are called, we set the corresponding 
thread-local parameters that are applied to the first task executed (see 
IgniteComputeImpl#withName).
When we run the task, we clear all the thread-local settings that were set 
earlier.

Consider the IgniteCompute#affinityCall method.
During its execution, in some cases we run a system task to get the affinity 
distribution from other nodes (see GridAffinityProcessor#affinityKey and 
GridAffinityProcessor#affinityInfoFromNode). As a result, the mentioned above 
modifiers are applied to the system task that requests the affinity 
distribution but not the user task.

It seems that the current approach to setting task parameters via thread-local 
variables is  error-prone and should be refactored.





--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18544) Reader leak for near caches even with near eviciton policy

2023-01-12 Thread Ivan Daschinsky (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Daschinsky updated IGNITE-18544:
-
Description: 
It was suddenly discovered while fixing IGNITE-18494 that 
`Random2LruNearEnabledPageEvictionMultinodeTest` actually had been tested 
without near caching. When it was enabled, it was found that actually page 
eviction didn't happen. 
The main reason of that behaviour is a leakage of readers on primary nodes 
(added but not removed) even with the near eviction policy.

  was:It was suddenly discovered while fixing IGNITE-18494


> Reader leak for near caches even  with near eviciton policy
> ---
>
> Key: IGNITE-18544
> URL: https://issues.apache.org/jira/browse/IGNITE-18544
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Daschinsky
>Priority: Major
>
> It was suddenly discovered while fixing IGNITE-18494 that 
> `Random2LruNearEnabledPageEvictionMultinodeTest` actually had been tested 
> without near caching. When it was enabled, it was found that actually page 
> eviction didn't happen. 
> The main reason of that behaviour is a leakage of readers on primary nodes 
> (added but not removed) even with the near eviction policy.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18544) Reader leak for near caches even with near eviciton policy

2023-01-12 Thread Ivan Daschinsky (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18544?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Daschinsky updated IGNITE-18544:
-
Description: It was suddenly discovered while fixing IGNITE-18494

> Reader leak for near caches even  with near eviciton policy
> ---
>
> Key: IGNITE-18544
> URL: https://issues.apache.org/jira/browse/IGNITE-18544
> Project: Ignite
>  Issue Type: Bug
>Reporter: Ivan Daschinsky
>Priority: Major
>
> It was suddenly discovered while fixing IGNITE-18494



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18544) Reader leak for near caches even with near eviciton policy

2023-01-12 Thread Ivan Daschinsky (Jira)
Ivan Daschinsky created IGNITE-18544:


 Summary: Reader leak for near caches even  with near eviciton 
policy
 Key: IGNITE-18544
 URL: https://issues.apache.org/jira/browse/IGNITE-18544
 Project: Ignite
  Issue Type: Bug
Reporter: Ivan Daschinsky






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18542) Remove duplicate server-side classes from Meta Storage

2023-01-12 Thread Aleksandr Polovtcev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-18542:
-
Description: Before https://issues.apache.org/jira/browse/IGNITE-18413 , we 
had a separate server and client parts of Meta Storage. These parts were fully 
independent and had to declare very similar classes (like Entry, Value, 
Operation, etc). Since this separation has been removed, we don't need to have 
these duplicates.  (was: Before 
https://issues.apache.org/jira/browse/IGNITE-18413 , we had a separate server 
and client parts of Meta Storage. These parts were fully independent and had to 
declare very similar classes (like Entry, Value, Operation, etc). Since this 
separation has been removed, we don't need to have these duplicates)

> Remove duplicate server-side classes from Meta Storage
> --
>
> Key: IGNITE-18542
> URL: https://issues.apache.org/jira/browse/IGNITE-18542
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before https://issues.apache.org/jira/browse/IGNITE-18413 , we had a separate 
> server and client parts of Meta Storage. These parts were fully independent 
> and had to declare very similar classes (like Entry, Value, Operation, etc). 
> Since this separation has been removed, we don't need to have these 
> duplicates.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18542) Remove duplicate server-side classes from Meta Storage

2023-01-12 Thread Aleksandr Polovtcev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aleksandr Polovtcev updated IGNITE-18542:
-
Description: Before https://issues.apache.org/jira/browse/IGNITE-18413 , we 
had a separate server and client parts of Meta Storage. These parts were fully 
independent and had to declare very similar classes (like Entry, Value, 
Operation, etc). Since this separation has been removed, we don't need to have 
these duplicates

> Remove duplicate server-side classes from Meta Storage
> --
>
> Key: IGNITE-18542
> URL: https://issues.apache.org/jira/browse/IGNITE-18542
> Project: Ignite
>  Issue Type: Task
>Reporter: Aleksandr Polovtcev
>Assignee: Aleksandr Polovtcev
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> Before https://issues.apache.org/jira/browse/IGNITE-18413 , we had a separate 
> server and client parts of Meta Storage. These parts were fully independent 
> and had to declare very similar classes (like Entry, Value, Operation, etc). 
> Since this separation has been removed, we don't need to have these duplicates



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18541) Limit number of threads and reduce logs in B+Tree IT tests

2023-01-12 Thread Semyon Danilov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17676046#comment-17676046
 ] 

Semyon Danilov commented on IGNITE-18541:
-

The patch looks good to me!

> Limit number of threads and reduce logs in B+Tree IT tests
> --
>
> Key: IGNITE-18541
> URL: https://issues.apache.org/jira/browse/IGNITE-18541
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nuff said



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18541) Limit number of threads and reduce logs in B+Tree IT tests

2023-01-12 Thread Semyon Danilov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Semyon Danilov updated IGNITE-18541:

Reviewer: Semyon Danilov

> Limit number of threads and reduce logs in B+Tree IT tests
> --
>
> Key: IGNITE-18541
> URL: https://issues.apache.org/jira/browse/IGNITE-18541
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> nuff said



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-18541) Limit number of threads and reduce logs in B+Tree IT tests

2023-01-12 Thread Ivan Bessonov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18541?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ivan Bessonov reassigned IGNITE-18541:
--

Assignee: Ivan Bessonov

> Limit number of threads and reduce logs in B+Tree IT tests
> --
>
> Key: IGNITE-18541
> URL: https://issues.apache.org/jira/browse/IGNITE-18541
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
>  Labels: ignite-3
>
> nuff said



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18542) Remove duplicate server-side classes from Meta Storage

2023-01-12 Thread Aleksandr Polovtcev (Jira)
Aleksandr Polovtcev created IGNITE-18542:


 Summary: Remove duplicate server-side classes from Meta Storage
 Key: IGNITE-18542
 URL: https://issues.apache.org/jira/browse/IGNITE-18542
 Project: Ignite
  Issue Type: Task
Reporter: Aleksandr Polovtcev
Assignee: Aleksandr Polovtcev






--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18543) Remove missed pom.xml

2023-01-12 Thread Mikhail Pochatkin (Jira)
Mikhail Pochatkin created IGNITE-18543:
--

 Summary: Remove missed pom.xml
 Key: IGNITE-18543
 URL: https://issues.apache.org/jira/browse/IGNITE-18543
 Project: Ignite
  Issue Type: Improvement
  Components: build
Reporter: Mikhail Pochatkin
Assignee: Mikhail Pochatkin


Remove missed pom.xml from distribution-zones module



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18541) Limit number of threads and reduce logs in B+Tree IT tests

2023-01-12 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-18541:
--

 Summary: Limit number of threads and reduce logs in B+Tree IT tests
 Key: IGNITE-18541
 URL: https://issues.apache.org/jira/browse/IGNITE-18541
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


nuff said



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-18540) It is necessary to process the default distribution zone like all other zones

2023-01-12 Thread Sergey Uttsel (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Uttsel reassigned IGNITE-18540:
--

Assignee: Sergey Uttsel

> It is necessary to process the default distribution zone like all other zones
> -
>
> Key: IGNITE-18540
> URL: https://issues.apache.org/jira/browse/IGNITE-18540
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Uttsel
>Assignee: Sergey Uttsel
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> The default distribution zone is separated from other zones. So need to check 
> all places where is processed zonesConfiguration.distributionZones() and add 
> processing of zonesConfiguration.defaultDistributionZone().
> h3. Definition of Done
> The default distribution zone is processed like all other zones.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18540) It is necessary to process the default distribution zone like all other zones

2023-01-12 Thread Sergey Uttsel (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Uttsel updated IGNITE-18540:
---
Description: 
h3. Motivation

The default distribution zone is separated from other zones. So need to check 
all places where is processed zonesConfiguration.distributionZones() and add 
processing of zonesConfiguration.defaultDistributionZone().
h3. Definition of Done

The default distribution zone is processed like all other zones.

  was:
h3. Motivation

The default distribution zone is separated from other zones. So need to check 
all places where is processed zonesConfiguration.distributionZones() and add 
processing of 
zonesConfiguration.defaultDistributionZone().
h3. Definition of Done

The default distribution zone is processed like all other zones.


> It is necessary to process the default distribution zone like all other zones
> -
>
> Key: IGNITE-18540
> URL: https://issues.apache.org/jira/browse/IGNITE-18540
> Project: Ignite
>  Issue Type: Bug
>Reporter: Sergey Uttsel
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> The default distribution zone is separated from other zones. So need to check 
> all places where is processed zonesConfiguration.distributionZones() and add 
> processing of zonesConfiguration.defaultDistributionZone().
> h3. Definition of Done
> The default distribution zone is processed like all other zones.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18540) It is necessary to process the default distribution zone like all other zones

2023-01-12 Thread Sergey Uttsel (Jira)
Sergey Uttsel created IGNITE-18540:
--

 Summary: It is necessary to process the default distribution zone 
like all other zones
 Key: IGNITE-18540
 URL: https://issues.apache.org/jira/browse/IGNITE-18540
 Project: Ignite
  Issue Type: Bug
Reporter: Sergey Uttsel


h3. Motivation

The default distribution zone is separated from other zones. So need to check 
all places where is processed zonesConfiguration.distributionZones() and add 
processing of 
zonesConfiguration.defaultDistributionZone().
h3. Definition of Done

The default distribution zone is processed like all other zones.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-18525) Introduce new PlacementDriver Ignite component

2023-01-12 Thread Vladislav Pyatkov (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18525?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vladislav Pyatkov reassigned IGNITE-18525:
--

Assignee: Vladislav Pyatkov

> Introduce new PlacementDriver Ignite component
> --
>
> Key: IGNITE-18525
> URL: https://issues.apache.org/jira/browse/IGNITE-18525
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> Placement Driver  is expected to be used as 
>  * A leaseholder tracker, or in other words, an engine for replication groups 
> leases selection and fail-over.
>  * Distributed meta data propagator, such as safe time, will be implemented 
> in Placement Driver [Part 2] though.
> This is an initial ticket, a sort of  utility one, that will introduce new 
> PlacementDriver Ignite component, add corresponding dependencies wihtin mvn 
> and gradle and incorporate the component into node start/stop flow.
> h3. Definition of Done
>  * New Placement Driver  Ignite Component is introduced along with 
> corresponding decencies (e.g. meta storage), busy locks, etc.
>  * It's properly started and stopped withing Ignite node start/stop process.
> h3. Implementation Notes
> Pretty straightforward. The one to mention is that introducing new component 
> will required to update tests and mocks, however because at this point 
> placement driver does nothing it'll possible to use nulls almost everywhere.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18456) Explain treading model in corresponding README.md file for MetaStorageManager

2023-01-12 Thread Vladislav Pyatkov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675977#comment-17675977
 ] 

Vladislav Pyatkov commented on IGNITE-18456:


Merged d4de4f6a714df05905db9eafa4e530c55adc207d

> Explain treading model in corresponding README.md file for MetaStorageManager
> -
>
> Key: IGNITE-18456
> URL: https://issues.apache.org/jira/browse/IGNITE-18456
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Use ignite-3/modules/raft/README.md as a reference thread-model explanation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18511) Incomprehensible error when using a reserved word in ddl via jdbc

2023-01-12 Thread Ilhom Ulmasov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18511?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675965#comment-17675965
 ] 

Ilhom Ulmasov commented on IGNITE-18511:


[~alexpl] could you, please, take a look on the PR?

>  Incomprehensible error when using a reserved word in ddl via jdbc
> --
>
> Key: IGNITE-18511
> URL: https://issues.apache.org/jira/browse/IGNITE-18511
> Project: Ignite
>  Issue Type: Improvement
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Assignee: Ilhom Ulmasov
>Priority: Minor
>  Labels: ise
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> When creating a table via sqlline, the column name of which uses a reserved 
> word (in our case, the name is "type") on the calcite engine, we fall with an 
> error due to which the cause cannot be determined.
> When using the h2 engine, the table is created.
> {code:java}
> ./sqlline.sh --verbose=true -u 
> jdbc:ignite:thin://127.0.0.1:10800?queryEngine=calcite
> 0: jdbc:ignite:thin://127.0.0.1:10800> CREATE TABLE test (
> . . . . . . . . . . . . . . . . . . )>  id BIGINT,
> . . . . . . . . . . . . . . . . . . )>  name VARCHAR,
> . . . . . . . . . . . . . . . . . . )>  type VARCHAR,
> . . . . . . . . . . . . . . . . . . )>  PRIMARY KEY (id)
> . . . . . . . . . . . . . . . . . . )>  );
> Error: Failed to parse query. (state=42000,code=1001)
> java.sql.SQLException: Failed to parse query.
>   at 
> org.apache.ignite.internal.jdbc.thin.JdbcThinConnection.sendRequest(JdbcThinConnection.java:1010)
>   at 
> org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute0(JdbcThinStatement.java:234)
>   at 
> org.apache.ignite.internal.jdbc.thin.JdbcThinStatement.execute(JdbcThinStatement.java:560)
>   at sqlline.Commands.executeSingleQuery(Commands.java:1054)
>   at sqlline.Commands.execute(Commands.java:1003)
>   at sqlline.Commands.sql(Commands.java:967)
>   at sqlline.SqlLine.dispatch(SqlLine.java:734)
>   at sqlline.SqlLine.begin(SqlLine.java:541)
>   at sqlline.SqlLine.start(SqlLine.java:267)
>   at sqlline.SqlLine.main(SqlLine.java:206)
> {code}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18539) Implement build procedure for new indexes

2023-01-12 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-18539:
--

 Summary: Implement build procedure for new indexes
 Key: IGNITE-18539
 URL: https://issues.apache.org/jira/browse/IGNITE-18539
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


First of all, "build" is the process of moving the index from "WO" to "RW" 
state in schema definitions. Finished build creates a new schema version. This 
means that new transactions, started after a specific timestamp, will be able 
to use this index in their plans.

Now, when index is created, we also create a table somewhere that looks similar 
to this:

 
{code:java}
indexComplation..0 = false
...
indexComplation..N = false {code}
It is used to track completion of index build for each individual partition. 
Once it's built bor each partition, WO -> RW state transition may occur.

 

For each individual partition, leader/coordinator/leaseholder/whatever node 
starts iterating all rowIds in the partition and replicates the "index build 
command" for each row (or using batches). This procedure can be described with 
this pseudocode:
{code:java}
RowId curRowId = minRowId();

// Get existing row id higher or equal than the parameter.
curRowId = partition.findRowId(curRowId);

while (curRowId != null) {
// Put that into a state machine.
replicateIndexRowRebuild(indexId, curRowId);

// Compute the next row id candidate.
curRowId = increment(curRowId);

// Get the real next row id value.
if (curRowId != null) {
curRowId = partition.findRowId(curRowId);
}
}

indexBuildCompleted(indexId, partitionId);{code}
Such replication doesn't require too much traffic, and it guarantees 
consistency between replicas, which is crucial.

Once index is created and WO, all insertions should update it 
"unconditionally", independently from the build process. This gives us 
cooperation and simplifies the way we guarantee consistency between row store 
and index.

Easy optimization: during regular load, we can avoid updating WO indexes if 
rowId > lastRowId for that particular index, because it will be updated later 
anyway.

 

About that: we should store "lastUpdatedRowId" for every index that's not yet 
completed its build. We need it for 2 reasons:
 * leader might be changed
 * cluster may be restarted

In both cases we simply start where we stopped last time, instead of starting 
from scratch, like it was in Ignite 2.x.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18538) Calcite engine does not allow creating tables with column names type, options

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18538:
-
Affects Version/s: 2.14

>  Calcite engine does not allow creating tables with column names type, options
> --
>
> Key: IGNITE-18538
> URL: https://issues.apache.org/jira/browse/IGNITE-18538
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: ColumnNameKeywordsTest.java
>
>
> Calcite engine does not allow creating tables with column names type, options.
> Reproducer  [^ColumnNameKeywordsTest.java] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18538) Calcite engine does not allow creating tables with column names type, options

2023-01-12 Thread Luchnikov Alexander (Jira)
Luchnikov Alexander created IGNITE-18538:


 Summary:  Calcite engine does not allow creating tables with 
column names type, options
 Key: IGNITE-18538
 URL: https://issues.apache.org/jira/browse/IGNITE-18538
 Project: Ignite
  Issue Type: Bug
Reporter: Luchnikov Alexander
 Attachments: ColumnNameKeywordsTest.java

Calcite engine does not allow creating tables with column names type, options.

Reproducer  [^ColumnNameKeywordsTest.java] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Commented] (IGNITE-18456) Explain treading model in corresponding README.md file for MetaStorageManager

2023-01-12 Thread Denis Chudov (Jira)


[ 
https://issues.apache.org/jira/browse/IGNITE-18456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17675937#comment-17675937
 ] 

Denis Chudov commented on IGNITE-18456:
---

[~v.pyatkov] lgtm

> Explain treading model in corresponding README.md file for MetaStorageManager
> -
>
> Key: IGNITE-18456
> URL: https://issues.apache.org/jira/browse/IGNITE-18456
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Vladislav Pyatkov
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 2h 10m
>  Remaining Estimate: 0h
>
> Use ignite-3/modules/raft/README.md as a reference thread-model explanation.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-18496) Handle documentation feedback

2023-01-12 Thread Igor Gusev (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Igor Gusev reassigned IGNITE-18496:
---

Assignee: Igor Gusev

> Handle documentation feedback
> -
>
> Key: IGNITE-18496
> URL: https://issues.apache.org/jira/browse/IGNITE-18496
> Project: Ignite
>  Issue Type: Task
>Reporter: Igor Gusev
>Assignee: Igor Gusev
>Priority: Major
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> We have had bugyard for a while, and there is a lot of useful feedback on 
> documentation. Its time to go through it and fix all issues.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18537) Integrate versioned schema definitions into create/alter/drop operations

2023-01-12 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-18537:
--

 Summary: Integrate versioned schema definitions into 
create/alter/drop operations
 Key: IGNITE-18537
 URL: https://issues.apache.org/jira/browse/IGNITE-18537
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


Every CRETE/ALTER/DROP operation should lead to a new schema definition.

Most likely, this must be the golden source of truth, instead of configuration.

Please keep in mind that the affinity is not the part of the schema definition. 
Data storage parameters are not the part of it as well.

TBD



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18536) Versioned schema definitions synchronization

2023-01-12 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-18536:
--

 Summary: Versioned schema definitions synchronization
 Key: IGNITE-18536
 URL: https://issues.apache.org/jira/browse/IGNITE-18536
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


TBD.

Metadata synchronization should be used, implemented (most likely) on top of 
"transactions in the past", defined in the transactions IEP (see "Lock-free RW 
Transactions" in 
https://cwiki.apache.org/confluence/display/IGNITE/IEP-91:+Transaction+protocol).



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18535) Define new classes for versioned tables/indexes schemas

2023-01-12 Thread Ivan Bessonov (Jira)
Ivan Bessonov created IGNITE-18535:
--

 Summary: Define new classes for versioned tables/indexes schemas
 Key: IGNITE-18535
 URL: https://issues.apache.org/jira/browse/IGNITE-18535
 Project: Ignite
  Issue Type: Improvement
Reporter: Ivan Bessonov


Current approach with schema management is faulty and can't support indexes. On 
top of that, it doesn't allow us to truly have multi-versioned historical data. 
Once the table is removed, it's removed for good, meaning that "current" RO 
transactions will not be able to finish. This is not acceptable.
h3. Schema definitions

What we need to have is the following:
{code:java}
SchemaDefinitions = map {version -> SchemaDefinition}

SchemaDefinition = {timestamp, set {TableDefinition}, set{IndexDefinition}}

TableDefinition = {name, id, array[ColumnDefinition], ...}

IndexDefinition = {name, id, tableId, state, array[IdxColumnDefinition], 
...}{code}
Schema must be versioned, that's the first point. Well, it's already versioned 
in "main", here I mean the global versioning to tie everything to transactions 
and management of SQL indexes.

Each definition correspond to a time period, where it represents the "actual" 
state of things. It must be used for RO queries, for example. RW transactions 
always use LATEST schema, obviously.

Now, the meaning of defined values:
 * version - a simple auto-incrementing integer value;
 * "timestamp" - the schema is considered to be valid from this timestamp until 
the timestamp of "next" version (or "inifinity" if the next version doesn't yet 
exist);
 * most of tables and indexes properties are self-explanatory;
 * index state - RO or RW. We should differentiate the indexes that are not yet 
built frome indexes that are fully available.

Currently, it's not too clear where to store this structure. The problem lies 
in the realm of metadata synchronization, that's not yet designed. But the 
thing is that all nodes must eventually have an up-to-date state and every 
data/index update must be consistent with the version that belongs to a current 
operation's timestamp.

There are two likely candidates - Meta-Storage or Configuration. We'll figure 
it out later.
h3. Seralization / storage

It would be convenient to only store the oldest version + the collection of 
diffs. Every node would unpack that locally, but we would save a lot on the 
storage space in meta-storage in case when user has a lot of tables/indexes.

This approach would also be beneficial for another reason: we need to know, 
what's changed between versions. It may be hard to calculate if all that we 
have are definitions themselves.
h3. General thoughts

This may be a good place to start using integer tableId and indexId more often. 
UUIDs are too much. What's good is that "serializability" of schemas gives us 
easy way of generating integer ids, just like it's don right now with 
configuration.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17487) Remove Maven build scripts

2023-01-12 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17487:
-
Fix Version/s: 3.0.0-beta2

> Remove Maven build scripts
> --
>
> Key: IGNITE-17487
> URL: https://issues.apache.org/jira/browse/IGNITE-17487
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
> Fix For: 3.0.0-beta2
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 1. Remove Maven project from Apache Ignite 3 repo.
> 2. Remove Maven project from CI.
> 3. Start use Gradle project verification as default for PR visa.
> 4. Change WorkDirectoryExtension to write log files for tests to the build 
> directory.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17487) Remove Maven build scripts

2023-01-12 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17487:
-
Reviewer: Igor Sapego

> Remove Maven build scripts
> --
>
> Key: IGNITE-17487
> URL: https://issues.apache.org/jira/browse/IGNITE-17487
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 1. Remove Maven project from Apache Ignite 3 repo.
> 2. Remove Maven project from CI.
> 3. Start use Gradle project verification as default for PR visa.
> 4. Change WorkDirectoryExtension to write log files for tests to the build 
> directory.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-17487) Remove Maven build scripts

2023-01-12 Thread Vyacheslav Koptilin (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-17487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vyacheslav Koptilin updated IGNITE-17487:
-
Ignite Flags:   (was: Docs Required,Release Notes Required)

> Remove Maven build scripts
> --
>
> Key: IGNITE-17487
> URL: https://issues.apache.org/jira/browse/IGNITE-17487
> Project: Ignite
>  Issue Type: New Feature
>  Components: build
>Reporter: Mikhail Pochatkin
>Assignee: Mikhail Pochatkin
>Priority: Major
>  Labels: ignite-3
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> 1. Remove Maven project from Apache Ignite 3 repo.
> 2. Remove Maven project from CI.
> 3. Start use Gradle project verification as default for PR visa.
> 4. Change WorkDirectoryExtension to write log files for tests to the build 
> directory.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Assigned] (IGNITE-18088) Trigger rebalance on zone.dataNodes change

2023-01-12 Thread Sergey Uttsel (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Uttsel reassigned IGNITE-18088:
--

Assignee: Sergey Uttsel

> Trigger rebalance on zone.dataNodes change
> --
>
> Key: IGNITE-18088
> URL: https://issues.apache.org/jira/browse/IGNITE-18088
> Project: Ignite
>  Issue Type: Improvement
>Reporter: Alexander Lapin
>Assignee: Sergey Uttsel
>Priority: Major
>  Labels: ignite-3
>
> h3. Motivation
> Core idea is to rebalance table data after some topology events. Distribution 
> zones both provide corresponding triggers and set of nodes to perform 
> affinity recalculation, meaning that dataNodes _change_ is a trigger and 
> dataNode _state_ is an input for an affinity recalculation. Thus it's 
> required for partition to listen for such events though ms watches and call 
> old-fashioned rebalance logic.
> *Definition of Done*
> Rebalance is performed after dataNodes changes.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Description: 
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].
Quick fix patch 


  was:
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].


> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java, patch.patch
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].
> Quick fix patch 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Attachment: patch.patch

> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java, patch.patch
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].
> Quick fix patch 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Description: 
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].
Quick fix patch  [^patch.patch].


  was:
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].
Quick fix patch  [^patch.patch] 



> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java, patch.patch
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].
> Quick fix patch  [^patch.patch].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Description: 
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].
Quick fix patch  [^patch.patch] 


  was:
The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].
Quick fix patch 



> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java, patch.patch
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].
> Quick fix patch  [^patch.patch] 



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Affects Version/s: 2.14

> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Affects Versions: 2.14
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Ignite Flags: Release Notes Required  (was: Docs Required,Release Notes 
Required)

> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Updated] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)


 [ 
https://issues.apache.org/jira/browse/IGNITE-18534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Luchnikov Alexander updated IGNITE-18534:
-
Labels: ise  (was: )

> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND
> --
>
> Key: IGNITE-18534
> URL: https://issues.apache.org/jira/browse/IGNITE-18534
> Project: Ignite
>  Issue Type: Bug
>Reporter: Luchnikov Alexander
>Priority: Major
>  Labels: ise
> Attachments: IoDatastorageMetricsTest.java
>
>
> The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
> BACKGROUND.
> Reproducer  [^IoDatastorageMetricsTest.java].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)


[jira] [Created] (IGNITE-18534) The WalWritingRate metric is not calculated when walMode is LOG_ONLY or BACKGROUND

2023-01-12 Thread Luchnikov Alexander (Jira)
Luchnikov Alexander created IGNITE-18534:


 Summary: The WalWritingRate metric is not calculated when walMode 
is LOG_ONLY or BACKGROUND
 Key: IGNITE-18534
 URL: https://issues.apache.org/jira/browse/IGNITE-18534
 Project: Ignite
  Issue Type: Bug
Reporter: Luchnikov Alexander
 Attachments: IoDatastorageMetricsTest.java

The WalWritingRate metric is not calculated when walMode is LOG_ONLY or 
BACKGROUND.
Reproducer  [^IoDatastorageMetricsTest.java].



--
This message was sent by Atlassian Jira
(v8.20.10#820010)