[jira] [Created] (IGNITE-15770) SQL-escaped similar columns are ignored by BinaryMapper
Maksim Timonin created IGNITE-15770:
---
Summary: SQL-escaped similar columns are ignored by BinaryMapper
Key: IGNITE-15770
URL: https://issues.apache.org/jira/browse/IGNITE-15770
Project: Ignite
Issue Type: Task
Reporter: Maksim Timonin
create table(id, "aA", "Aa")
insert into table("aA", "Aa") values (1, 100)
select "aA", "Aa" from table --> (1, 1)
Error is within BinaryBasicIdMapper#isLowerCase - SQL supports escaped имена,
but BinaryInternalMapper always uses lower case for fields resolution.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430181#comment-17430181
]
Pavel Tupitsyn commented on IGNITE-14776:
-
Cherry-picked to ignite-2.12: 3c8bdfaef5697cbff35acd76cd15d2f9b7a12a89
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430180#comment-17430180
]
Pavel Tupitsyn commented on IGNITE-14776:
-
Merged to master: 75036d28ebce294ba2aad68d3495e179ccb7409a
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn resolved IGNITE-14776.
-
Resolution: Fixed
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430118#comment-17430118
]
Ignite TC Bot commented on IGNITE-14776:
{panel:title=Branch: [pull/9499/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/9499/head] Base: [master] : New Tests
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Windows){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228818]]
* {color:#013220}exe:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{color:#8b}Platform .NET (Core Linux){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=622]]
* {color:#013220}dll:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6228819&buildTypeId=IgniteTests24Java8_RunAll]
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Issue Comment Deleted] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-14776:
Comment: was deleted
(was: {panel:title=Branch: [pull/9499/head] Base: [master] : Possible Blockers
(23)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Examples{color} [[tests
21|https://ci.ignite.apache.org/viewLog.html?buildId=6228732]]
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheTransactionExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheQueryExample -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheAtomicLongExample
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheContinuousQueryExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheAtomicReferenceExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCachePutGetExample -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheDataStreamerExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheAtomicStampedExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheCountDownLatchExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testSnowflakeSchemaExample
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheQueueExample -
Test has low fail rate in base branch 0,0% and is not flaky
... and 10 tests blockers
{color:#d04437}Platform .NET (Core Linux){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6228795]]
{color:#d04437}PDS (Compatibility)*{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228788]]
* IgniteCompatibilityBasicTestSuite:
JavaThinCompatibilityTest.testOldClientToCurrentServer[Version 2.12.0-SNAPSHOT]
- Test has low fail rate in base branch 0,0% and is not flaky
{panel}
{panel:title=Branch: [pull/9499/head] Base: [master] : New Tests
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Windows){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228818]]
* {color:#013220}exe:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{color:#8b}Platform .NET (Core Linux){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228795]]
* {color:#013220}dll:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6228819&buildTypeId=IgniteTests24Java8_RunAll])
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClie
[jira] [Updated] (IGNITE-14872) Recalculation affinity based on topology events
[ https://issues.apache.org/jira/browse/IGNITE-14872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-14872: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Recalculation affinity based on topology events > --- > > Key: IGNITE-14872 > URL: https://issues.apache.org/jira/browse/IGNITE-14872 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-14873) Failover scenarios
[ https://issues.apache.org/jira/browse/IGNITE-14873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-14873: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Failover scenarios > -- > > Key: IGNITE-14873 > URL: https://issues.apache.org/jira/browse/IGNITE-14873 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to implement a comprehensive set of tests in order to cover data > rebalancing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15125) Implement naive partitions reassignment
[ https://issues.apache.org/jira/browse/IGNITE-15125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15125: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Implement naive partitions reassignment > --- > > Key: IGNITE-15125 > URL: https://issues.apache.org/jira/browse/IGNITE-15125 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > Original Estimate: 96h > Time Spent: 0.5h > Remaining Estimate: 95.5h > > TBD -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15514) Add guaranties of using appropriate affinity assignments from lagged node after reassignment.
[ https://issues.apache.org/jira/browse/IGNITE-15514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15514: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Add guaranties of using appropriate affinity assignments from lagged node > after reassignment. > - > > Key: IGNITE-15514 > URL: https://issues.apache.org/jira/browse/IGNITE-15514 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > Let's consider following example: > # tbl1 true assignment(ms hosted) is A,B,C. Raft is up and running on A,B,C. > On node A it's known that tbl1 assignment is A,B,C. > # Assignment recalculation was triggered that moved true assignment from > A,B,C to D,E,F(ms hosted). Raft is up and running on D,E,F. Node A still > assumes that tbl1 assignment is A,B,C cause it didn't get update from meta > storage about reassignment yet. > # User tries to put or get data to/from tbl1 from node A. Internally he > tries to retrieve raft leader, assuming that it's either on A, B or C. Given > assumption is wrong, so that he will fail with something similar to > TimeoutException. > Seems that we need a solution that will guarantee that appropriate true > assignment is used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
@Override public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void serviceMethod() {
String user = svcCtx.attribute("login");
// and/or
String user = (String)svcCtx.attributes().get("login");
}
...
}{code}
h4. The next two options require wrapping Map into a new
ServiceRequestContext class.
h4. 3. Read context "wrapper" using special annotation and supplier.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier ctxSupplier;
@Override public void serviceMethod() {
String login = ctxSupplier.get().attribute("login");
}
...
}
{code}
h4. 4. Using the special static method of the "wrapper" class.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void serviceMethod() {
String login = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
// and/or
String user = (String)svcCtx.attributes().get("login");
}
...
}{code}
h4. The next two options require wrapping Map into a new
ServiceRequestContext class.
h4. 3. Read context "wrapper" using special annotation and supplier.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier ctxSupplier;
@Override public void businessMethod() {
[jira] [Updated] (IGNITE-15495) Rebalance via raft log
[ https://issues.apache.org/jira/browse/IGNITE-15495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15495: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Rebalance via raft log > -- > > Key: IGNITE-15495 > URL: https://issues.apache.org/jira/browse/IGNITE-15495 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to be sure that rebalance can be done in two possible ways. > When a new node joins the cluster, the leader might need to send snapshots to > a new follower that doesn’t have the necessary log information and the leader > might already created a snapshot of its log. However, it looks like the > rebalance can be done via raft log without a snapshot. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15494) Need to provide internal mechanism that allows to wait for rebalance
[ https://issues.apache.org/jira/browse/IGNITE-15494?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15494: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Need to provide internal mechanism that allows to wait for rebalance > > > Key: IGNITE-15494 > URL: https://issues.apache.org/jira/browse/IGNITE-15494 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > It would be nice to have an internal mechanism that allows waiting for > rebalancing. This method can be extremely useful for testing purposes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15493) Need to clarify changePeers behavior to support the case with only one replica which should be moved during rebalance
[ https://issues.apache.org/jira/browse/IGNITE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15493: - Issue Type: Task (was: Improvement) > Need to clarify changePeers behavior to support the case with only one > replica which should be moved during rebalance > - > > Key: IGNITE-15493 > URL: https://issues.apache.org/jira/browse/IGNITE-15493 > Project: Ignite > Issue Type: Task >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to clarify the behavior of changePeers (IGNITE-15288) method in case of > a raft group contains only one raw node and it should be moved to a new > Ignite node. > h4. Upd 1 > Following open points should be clarified: > * Is it possible to change peers for the case when the old and new sets of > raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s > possible, see ITJRaftCounterServerTest#testRebalance > * When changePeers() returns to the client? We assume that it returns after > appliance on both old and new raft group typologies. It’s also expected that > changePeers is a raft command that’s as any other raft commands have an index > and will be applied after commands with lower index, which actually means > that data rebalance to the majority of nodes within new topology will be > finished before applying change peers. Let’s check whether it’s true: > ** Let’s check whether dataRebalance is a raft command that works just as > any other raft commands and do not expect index gaps. > ** Let’s check that snapshot works the same way as log-rebalance in the > context of index moving. > ** If all true, how it’s possible to have changePeers invoke that will last > for “new majority time“ that may take hours or event days? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15493) Need to clarify changePeers behavior to support the case with only one replica which should be moved during rebalance
[ https://issues.apache.org/jira/browse/IGNITE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15493: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Need to clarify changePeers behavior to support the case with only one > replica which should be moved during rebalance > - > > Key: IGNITE-15493 > URL: https://issues.apache.org/jira/browse/IGNITE-15493 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to clarify the behavior of changePeers (IGNITE-15288) method in case of > a raft group contains only one raw node and it should be moved to a new > Ignite node. > h4. Upd 1 > Following open points should be clarified: > * Is it possible to change peers for the case when the old and new sets of > raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s > possible, see ITJRaftCounterServerTest#testRebalance > * When changePeers() returns to the client? We assume that it returns after > appliance on both old and new raft group typologies. It’s also expected that > changePeers is a raft command that’s as any other raft commands have an index > and will be applied after commands with lower index, which actually means > that data rebalance to the majority of nodes within new topology will be > finished before applying change peers. Let’s check whether it’s true: > ** Let’s check whether dataRebalance is a raft command that works just as > any other raft commands and do not expect index gaps. > ** Let’s check that snapshot works the same way as log-rebalance in the > context of index moving. > ** If all true, how it’s possible to have changePeers invoke that will last > for “new majority time“ that may take hours or event days? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15492) Check schema availability on local node
[ https://issues.apache.org/jira/browse/IGNITE-15492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15492: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Check schema availability on local node > --- > > Key: IGNITE-15492 > URL: https://issues.apache.org/jira/browse/IGNITE-15492 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > Time Spent: 0.5h > Remaining Estimate: 0h > > Currently, we do not check that a schema that is needed for deserialization > returned tuple (for example, the end-user calls tableKvView.get(key) method) > is available on the local node that performs table operation. > Need to implement a synchronization mechanism that allows overcoming an issue > related to the deserialization of tuples. Looks like it is an actual issue > regardless of rebalancing. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15491) Introduce IgniteCluster interface to manipulate baseline nodes
[ https://issues.apache.org/jira/browse/IGNITE-15491?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15491: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Introduce IgniteCluster interface to manipulate baseline nodes > --- > > Key: IGNITE-15491 > URL: https://issues.apache.org/jira/browse/IGNITE-15491 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha3 > > Time Spent: 9h > Remaining Estimate: 0h > > Introduce IgniteCluster API as a temporary solution in order to trigger > rebalance in a manual way. Perhaps it will be easier to take the approach > introduced by IGNITE-15125 (this approach is based on events and changes > topology baseline in an automatic way) > Need to choose the most appropriate for the first phase (IGNITE-14209) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
// and/or
String user = (String)svcCtx.attributes().get("login");
}
...
}{code}
h4. The next two options require wrapping Map into a new
ServiceRequestContext class.
h4. 3. Read context "wrapper" using special annotation and supplier.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier ctxSupplier;
@Override public void businessMethod() {
String login = ctxSupplier.get().attribute("login");
}
...
}
{code}
h4. 4. Using the special static method of the "wrapper" class.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String login = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
// and/or
String user = (String)svcCtx.attributes().get("login");
}
...
}{code}
h4. The next two options require wrapping Map into a new
ServiceRequestContext class.
h4. 3. Read context "wrapper" using special annotation and supplier.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier ctxSupplier;
@Override public void businessMethod() {
Str
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
// and/or
String user = (String)svcCtx.attributes().get("login");
}
...
}{code}
h4. The next two options require wrapping Map into a new
ServiceRequestContext class.
h4. 3. Read context "wrapper" using special annotation and supplier.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier ctxSupplier;
@Override public void businessMethod() {
String login = ctxSupplier.get().attribute("login");
}
...
}
{code}
h4. 4. Using special static method of the "wrapper" class.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ... new
ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String login = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = (String)svcCtx.attributes().get("login");
// and/or
String user = svcCtx.attribute("login");
}
...
}{code}
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String user = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ig
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using a custom annotation (ServiceRequestContextResource) and reading
context attributes with a function.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", ...
Collections.singletonMap("login", "user1"));
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
private Function ctxFunc;
public void serviceMethod() {
String login = (String)ctxFunc.apply("login");
}
...
}{code}
h4. 2. Using a new method of the existing ServiceContext.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = (String)svcCtx.attributes().get("login");
// and/or
String user = svcCtx.attribute("login");
}
...
}{code}
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String user = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Function ctxSupplier.
@Override public void businessMethod() {
String user = (String)ctxSupplier.apply("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String user = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Function ctxSupplier.
@Override public void businessMethod() {
String user = (String)ctxSupplier.apply("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Colle
[jira] [Updated] (IGNITE-14209) Data rebalancing
[ https://issues.apache.org/jira/browse/IGNITE-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-14209: - Issue Type: Epic (was: New Feature) > Data rebalancing > > > Key: IGNITE-14209 > URL: https://issues.apache.org/jira/browse/IGNITE-14209 > Project: Ignite > Issue Type: Epic >Reporter: Vyacheslav Koptilin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Overview* > The lifecycle of the Ignite cluster assumes that new nodes can join this > cluster and the old ones can leave it. In order for the data to remain > distributed equally in the cluster, some replicas have to be moved from one > Ignite node to another. This process is called data rebalancing. > The main goal of this umbrella ticket is an approach that can be implemented > based on Ignite Raft module. > > The first implementation phase has the following constraints in order to > simplify the rebalance process until the table group concept is not defined: > - only "empty" nodes can join the cluster (in other words, node restart with > already exists data is not supported) > - rebalance can be triggered by NODE_JOIN/NODE_FAIL events only in an > automatic way (alternative option is to introduce IgniteCluster interface > which allows triggering rebalance in a manual way) > - reconfiguration of meta-storage nodes is not supported > Phase 1 includes the following sub-tasks: > - IGNITE-15125 || IGNITE-15491 > - IGNITE-15490 > - IGNITE-15492 > - IGNITE-15493 > - IGNITE-15288 > - IGNITE-15495 > - IGNITE-15494 (optional) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-14209) Data rebalancing
[ https://issues.apache.org/jira/browse/IGNITE-14209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-14209: - Epic Name: Data rebalancing > Data rebalancing > > > Key: IGNITE-14209 > URL: https://issues.apache.org/jira/browse/IGNITE-14209 > Project: Ignite > Issue Type: Epic >Reporter: Vyacheslav Koptilin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > *Overview* > The lifecycle of the Ignite cluster assumes that new nodes can join this > cluster and the old ones can leave it. In order for the data to remain > distributed equally in the cluster, some replicas have to be moved from one > Ignite node to another. This process is called data rebalancing. > The main goal of this umbrella ticket is an approach that can be implemented > based on Ignite Raft module. > > The first implementation phase has the following constraints in order to > simplify the rebalance process until the table group concept is not defined: > - only "empty" nodes can join the cluster (in other words, node restart with > already exists data is not supported) > - rebalance can be triggered by NODE_JOIN/NODE_FAIL events only in an > automatic way (alternative option is to introduce IgniteCluster interface > which allows triggering rebalance in a manual way) > - reconfiguration of meta-storage nodes is not supported > Phase 1 includes the following sub-tasks: > - IGNITE-15125 || IGNITE-15491 > - IGNITE-15490 > - IGNITE-15492 > - IGNITE-15493 > - IGNITE-15288 > - IGNITE-15495 > - IGNITE-15494 (optional) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15705) Investigate raft client timeouts
[
https://issues.apache.org/jira/browse/IGNITE-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15705:
-
Description:
h3. Problem
Raft client timeout should be large enough for the operation to be performed
even if it falls on several consecutive rounds of choosing a new leader of the
raft group. Most of jraft timeouts are based on electionTimeoutMs.
{code:java}
// A follower would become a candidate if it doesn't receive any message
// from the leader in |election_timeout_ms| milliseconds
// Default: 1000 (1s)
private int electionTimeoutMs = 1000; // follower to candidate timeout
{code}
For example both voteTime and electionTime use exact value of
getElectionTimeoutMs (1000 ms):
{code:java}
String name = "JRaft-VoteTimer-" + suffix;
this.voteTimer = new RepeatedTimer(name,
options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
name = "JRaft-ElectionTimer-" + suffix;
electionTimer = new RepeatedTimer(name, options.getElectionTimeoutMs(),
timerFactory.getElectionTimer(name)) {...};
{code}
Going back to client timeout, seems that it should be greater than
reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
set client timeout to the corresponding value.
Not sure whether it’s a good idea but let’s also consider raft client timeout
to be derivative of leader election timeout not only semantically but also
within code:
{code:java}
private static final int TIMEOUT = 10 * leaderElectionTimeout;{code}
was:
h3. Problem
Raft client timeout should be large enough for the operation to be performed
even if it falls on several consecutive rounds of choosing a new leader of the
raft group. Most of jraft timeouts are based on electionTimeoutMs.
{code:java}
// A follower would become a candidate if it doesn't receive any message
// from the leader in |election_timeout_ms| milliseconds
// Default: 1000 (1s)
private int electionTimeoutMs = 1000; // follower to candidate timeout
{code}
For example both voteTime and electionTime use exact value of
getElectionTimeoutMs (1000 ms):
{{}}
{code:java}
String name = "JRaft-VoteTimer-" + suffix;
this.voteTimer = new RepeatedTimer(name,
options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
name = "JRaft-ElectionTimer-" + suffix;
electionTimer = new RepeatedTimer(name, options.getElectionTimeoutMs(),
timerFactory.getElectionTimer(name)) {...};
{code}
{{}}
Going back to client timeout, seems that it should be greater than
reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
set client timeout to the corresponding value.
Not sure whether it’s a good idea but let’s also consider raft client timeout
to be derivative of leader election timeout not only semantically but also
within code:
{{}}
{code:java}
private static final int TIMEOUT = 10 * leaderElectionTimeout;{code}
{{}}
> Investigate raft client timeouts
>
>
> Key: IGNITE-15705
> URL: https://issues.apache.org/jira/browse/IGNITE-15705
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Gusakov
>Priority: Critical
> Labels: ignite-3
>
> h3. Problem
> Raft client timeout should be large enough for the operation to be performed
> even if it falls on several consecutive rounds of choosing a new leader of
> the raft group. Most of jraft timeouts are based on electionTimeoutMs.
> {code:java}
> // A follower would become a candidate if it doesn't receive any message
> // from the leader in |election_timeout_ms| milliseconds
> // Default: 1000 (1s)
> private int electionTimeoutMs = 1000; // follower to candidate timeout
> {code}
> For example both voteTime and electionTime use exact value of
> getElectionTimeoutMs (1000 ms):
> {code:java}
> String name = "JRaft-VoteTimer-" + suffix;
> this.voteTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
> name = "JRaft-ElectionTimer-" + suffix;
> electionTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getElectionTimer(name)) {...};
> {code}
> Going back to client timeout, seems that it should be greater than
> reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
> So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
> set client timeout to the corresponding value.
> Not sure whether it’s a good idea but let’s also consider raft client timeout
> to be derivative of leader election timeout not only semantically but also
> within code:
> {code:java}
> private static final int TIMEOUT = 10 *
[jira] [Updated] (IGNITE-15705) Investigate raft client timeouts
[
https://issues.apache.org/jira/browse/IGNITE-15705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin updated IGNITE-15705:
-
Description:
h3. Problem
Raft client timeout should be large enough for the operation to be performed
even if it falls on several consecutive rounds of choosing a new leader of the
raft group. Most of jraft timeouts are based on electionTimeoutMs.
{code:java}
// A follower would become a candidate if it doesn't receive any message
// from the leader in |election_timeout_ms| milliseconds
// Default: 1000 (1s)
private int electionTimeoutMs = 1000; // follower to candidate timeout
{code}
For example both voteTime and electionTime use exact value of
getElectionTimeoutMs (1000 ms):
{{}}
{code:java}
String name = "JRaft-VoteTimer-" + suffix;
this.voteTimer = new RepeatedTimer(name,
options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
name = "JRaft-ElectionTimer-" + suffix;
electionTimer = new RepeatedTimer(name, options.getElectionTimeoutMs(),
timerFactory.getElectionTimer(name)) {...};
{code}
{{}}
Going back to client timeout, seems that it should be greater than
reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
set client timeout to the corresponding value.
Not sure whether it’s a good idea but let’s also consider raft client timeout
to be derivative of leader election timeout not only semantically but also
within code:
{{}}
{code:java}
private static final int TIMEOUT = 10 * leaderElectionTimeout;{code}
{{}}
was:TODO
> Investigate raft client timeouts
>
>
> Key: IGNITE-15705
> URL: https://issues.apache.org/jira/browse/IGNITE-15705
> Project: Ignite
> Issue Type: Task
>Reporter: Kirill Gusakov
>Priority: Critical
> Labels: ignite-3
>
> h3. Problem
> Raft client timeout should be large enough for the operation to be performed
> even if it falls on several consecutive rounds of choosing a new leader of
> the raft group. Most of jraft timeouts are based on electionTimeoutMs.
> {code:java}
> // A follower would become a candidate if it doesn't receive any message
> // from the leader in |election_timeout_ms| milliseconds
> // Default: 1000 (1s)
> private int electionTimeoutMs = 1000; // follower to candidate timeout
> {code}
> For example both voteTime and electionTime use exact value of
> getElectionTimeoutMs (1000 ms):
> {{}}
> {code:java}
> String name = "JRaft-VoteTimer-" + suffix;
> this.voteTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getVoteTimer(name)) {...};
> name = "JRaft-ElectionTimer-" + suffix;
> electionTimer = new RepeatedTimer(name,
> options.getElectionTimeoutMs(), timerFactory.getElectionTimer(name)) {...};
> {code}
> {{}}
> Going back to client timeout, seems that it should be greater than
> reasonableAmountOfElecionRounds(electionTime + networkTimeoutToRetrieveAcks).
> So seems that we should check the value of “networkTimeoutToRetrieveAcks” and
> set client timeout to the corresponding value.
> Not sure whether it’s a good idea but let’s also consider raft client timeout
> to be derivative of leader election timeout not only semantically but also
> within code:
> {{}}
> {code:java}
> private static final int TIMEOUT = 10 * leaderElectionTimeout;{code}
> {{}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15493) Need to clarify changePeers behavior to support the case with only one replica which should be moved during rebalance
[ https://issues.apache.org/jira/browse/IGNITE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15493: - Description: Need to clarify the behavior of changePeers (IGNITE-15288) method in case of a raft group contains only one raw node and it should be moved to a new Ignite node. h4. Upd 1 Following open points should be clarified: * Is it possible to change peers for the case when the old and new sets of raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s possible, see ITJRaftCounterServerTest#testRebalance * When changePeers() returns to the client? We assume that it returns after appliance on both old and new raft group typologies. It’s also expected that changePeers is a raft command that’s as any other raft commands have an index and will be applied after commands with lower index, which actually means that data rebalance to the majority of nodes within new topology will be finished before applying change peers. Let’s check whether it’s true: ** Let’s check whether dataRebalance is a raft command that works just as any other raft commands and do not expect index gaps. ** Let’s check that snapshot works the same way as log-rebalance in the context of index moving. ** If all true, how it’s possible to have changePeers invoke that will last for “new majority time“ that may take hours or event days? was: Need to clarify the behavior of changePeers (IGNITE-15288) method in case of a raft group contains only one raw node and it should be moved to a new Ignite node. h4. Upd 1 Following open points should be clarified: * Is it possible to change peers for the case when the old and new sets of raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s possible, see ITJRaftCounterServerTest#testRebalance * When changePeers() returns to the client? We assume that it returns after appliance on both old and new raft group typologies. It’s also expected that changePeers is a raft command that’s as any other raft commands have an index and will be applied after commands with lower index, which actually means that data rebalance to the majority of nodes within new topology will be finished before applying change peers. Let’s check whether it’s true: ** Let’s check whether dataRebalance is a raft command that works just as any other raft commands and do not expect index gaps. ** Let’s check that snapshot works the same way as log-rebalance in the context of index moving. ** If all true, how it’s possible to have changePeers invoke that will last for “new majority time“ that may take hours or event days? > Need to clarify changePeers behavior to support the case with only one > replica which should be moved during rebalance > - > > Key: IGNITE-15493 > URL: https://issues.apache.org/jira/browse/IGNITE-15493 > Project: Ignite > Issue Type: Sub-task >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to clarify the behavior of changePeers (IGNITE-15288) method in case of > a raft group contains only one raw node and it should be moved to a new > Ignite node. > h4. Upd 1 > Following open points should be clarified: > * Is it possible to change peers for the case when the old and new sets of > raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s > possible, see ITJRaftCounterServerTest#testRebalance > * When changePeers() returns to the client? We assume that it returns after > appliance on both old and new raft group typologies. It’s also expected that > changePeers is a raft command that’s as any other raft commands have an index > and will be applied after commands with lower index, which actually means > that data rebalance to the majority of nodes within new topology will be > finished before applying change peers. Let’s check whether it’s true: > ** Let’s check whether dataRebalance is a raft command that works just as > any other raft commands and do not expect index gaps. > ** Let’s check that snapshot works the same way as log-rebalance in the > context of index moving. > ** If all true, how it’s possible to have changePeers invoke that will last > for “new majority time“ that may take hours or event days? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15493) Need to clarify changePeers behavior to support the case with only one replica which should be moved during rebalance
[ https://issues.apache.org/jira/browse/IGNITE-15493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Lapin updated IGNITE-15493: - Description: Need to clarify the behavior of changePeers (IGNITE-15288) method in case of a raft group contains only one raw node and it should be moved to a new Ignite node. h4. Upd 1 Following open points should be clarified: * Is it possible to change peers for the case when the old and new sets of raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s possible, see ITJRaftCounterServerTest#testRebalance * When changePeers() returns to the client? We assume that it returns after appliance on both old and new raft group typologies. It’s also expected that changePeers is a raft command that’s as any other raft commands have an index and will be applied after commands with lower index, which actually means that data rebalance to the majority of nodes within new topology will be finished before applying change peers. Let’s check whether it’s true: ** Let’s check whether dataRebalance is a raft command that works just as any other raft commands and do not expect index gaps. ** Let’s check that snapshot works the same way as log-rebalance in the context of index moving. ** If all true, how it’s possible to have changePeers invoke that will last for “new majority time“ that may take hours or event days? was:Need to clarify the behavior of changePeers (IGNITE-15288) method in case of a raft group contains only one raw node and it should be moved to a new Ignite node. > Need to clarify changePeers behavior to support the case with only one > replica which should be moved during rebalance > - > > Key: IGNITE-15493 > URL: https://issues.apache.org/jira/browse/IGNITE-15493 > Project: Ignite > Issue Type: Sub-task >Reporter: Vyacheslav Koptilin >Priority: Major > Labels: ignite-3 > > Need to clarify the behavior of changePeers (IGNITE-15288) method in case of > a raft group contains only one raw node and it should be moved to a new > Ignite node. > h4. Upd 1 > Following open points should be clarified: > * Is it possible to change peers for the case when the old and new sets of > raft nodes do not intersect? As Kirill Gusakov mentioned seems that it’s > possible, see ITJRaftCounterServerTest#testRebalance > * When changePeers() returns to the client? We assume that it returns after > appliance on both old and new raft group typologies. It’s also expected that > changePeers is a raft command that’s as any other raft commands have an index > and will be applied after commands with lower index, which actually means > that data rebalance to the majority of nodes within new topology will be > finished before applying change peers. Let’s check whether it’s true: > ** Let’s check whether dataRebalance is a raft command that works just as > any other raft commands and do not expect index gaps. > ** Let’s check that snapshot works the same way as log-rebalance in the > context of index moving. > ** If all true, how it’s possible to have changePeers invoke that will last > for “new majority time“ that may take hours or event days? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430067#comment-17430067
]
Ignite TC Bot commented on IGNITE-14776:
{panel:title=Branch: [pull/9499/head] Base: [master] : Possible Blockers
(23)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}
{color:#d04437}Examples{color} [[tests
21|https://ci.ignite.apache.org/viewLog.html?buildId=6228732]]
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheTransactionExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheQueryExample -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheAtomicLongExample
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheContinuousQueryExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheAtomicReferenceExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCachePutGetExample -
Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheDataStreamerExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheAtomicStampedExample - Test has low fail rate in
base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite:
CacheExamplesSelfTest.testCacheCountDownLatchExample - Test has low fail rate
in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testSnowflakeSchemaExample
- Test has low fail rate in base branch 0,0% and is not flaky
* IgniteExamplesSelfTestSuite: CacheExamplesSelfTest.testCacheQueueExample -
Test has low fail rate in base branch 0,0% and is not flaky
... and 10 tests blockers
{color:#d04437}Platform .NET (Core Linux){color} [[tests 0 TIMEOUT , Exit Code
|https://ci.ignite.apache.org/viewLog.html?buildId=6228795]]
{color:#d04437}PDS (Compatibility)*{color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228788]]
* IgniteCompatibilityBasicTestSuite:
JavaThinCompatibilityTest.testOldClientToCurrentServer[Version 2.12.0-SNAPSHOT]
- Test has low fail rate in base branch 0,0% and is not flaky
{panel}
{panel:title=Branch: [pull/9499/head] Base: [master] : New Tests
(2)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Platform .NET (Windows){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228818]]
* {color:#013220}exe:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{color:#8b}Platform .NET (Core Linux){color} [[tests
1|https://ci.ignite.apache.org/viewLog.html?buildId=6228795]]
* {color:#013220}dll:
ClientConnectionTest.TestInvalidProtocolThrowsSocketException - PASSED{color}
{panel}
[TeamCity *--> Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=6228819&buildTypeId=IgniteTests24Java8_RunAll]
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailover
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context. For example, a REST service may obtain the session ID from the
request. We can say that each client request, in this case, has a set of
explicit and implicit parameters. One of the implicit parameters is a session
identifier that can be passed to the service using request headers. It would be
nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
String user = ServiceRequestContext.current().attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Function ctxSupplier.
@Override public void businessMethod() {
String user = (String)ctxSupplier.apply("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
String user = svcCtx.attribute("login");
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.get("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = svcCtx.attributes();
String user = ctx.get("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy(
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.get("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = svcCtx.attributes();
String user = ctx.get("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special clas
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.svcCtx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = svcCtx.attributes();
String user = ctx.get("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using specia
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.get("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special clas
[jira] [Commented] (IGNITE-15443) Add Map with primitive keys implementations.
[ https://issues.apache.org/jira/browse/IGNITE-15443?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430042#comment-17430042 ] Andrey Mashenkov commented on IGNITE-15443: --- [~aholod], the ticket is not in "in progress" state, do you intend to fix this? BTW, the ticket is not about choosing the best-performed implementation, but more about API. We don't need hash maps size of 2^32 items (up to 64k, maybe sparsed) and don't expect high contention here. > Add Map with primitive keys implementations. > > > Key: IGNITE-15443 > URL: https://issues.apache.org/jira/browse/IGNITE-15443 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Anton Kholodkov >Priority: Major > Labels: ignite-3, newbie, perfomance > > There are a number of places where Map and Map Object> are used. > Let's add an optimized Map with a primitive key (IntMap, IntIntMap) to the > ignite-core module. > Let's benchmark next candidates and choose one > * IntMap from Ignite 2 with robinhood hashing. > * FastUtils > * Netty > * OpenHFT -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
MyService proxy = ignite.services().serviceProxy("svc", MyService.class, false,
Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using specia
[jira] [Updated] (IGNITE-15096) Schema events processing refactoring.
[ https://issues.apache.org/jira/browse/IGNITE-15096?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15096: -- Description: All table events with schema events are processed in TableManager listener. Let's * -Replace ConfigurationListener with ConfigurationNamedListListener to simplify the diff calculation between old/new configs.- Resolved in IGNITE-15404 * -Register a separate per-table listener for schema events on table creation.- Resolved in IGNITE-15409 Update: Now we register a per-table listener for schema changes + listener for partition affinity assignment. A single listener for schema changes and single listener for affinity changes will be enough, but table registration may be tricky a bit. Let's discuss and fix this. was: All table events with schema events are processed in TableManager listener. Let's * -Replace ConfigurationListener with ConfigurationNamedListListener to simplify the diff calculation between old/new configs.- Resolved in IGNITE-15404 * -Register a separate per-table listener for schema events on table creation. Resolved in IGNITE-15409 > Schema events processing refactoring. > - > > Key: IGNITE-15096 > URL: https://issues.apache.org/jira/browse/IGNITE-15096 > Project: Ignite > Issue Type: Improvement >Reporter: Andrey Mashenkov >Assignee: Andrey Mashenkov >Priority: Minor > Labels: ignite-3 > Time Spent: 40m > Remaining Estimate: 0h > > All table events with schema events are processed in TableManager listener. > Let's > * -Replace ConfigurationListener with ConfigurationNamedListListener to > simplify the diff calculation between old/new configs.- > Resolved in IGNITE-15404 > * -Register a separate per-table listener for schema events on table > creation.- > Resolved in IGNITE-15409 > Update: > Now we register a per-table listener for schema changes + listener for > partition affinity assignment. > A single listener for schema changes and single listener for affinity changes > will be enough, > but table registration may be tricky a bit. > Let's discuss and fix this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContextResource
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyServ
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContext
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Pass Map directly and read it using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.clas
[jira] [Assigned] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-15769:
Assignee: Nikolay Izhikov
> IgniteToIgniteCdcStreamer fails on complex keys
> ---
>
> Key: IGNITE-15769
> URL: https://issues.apache.org/jira/browse/IGNITE-15769
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Shishkov
>Assignee: Nikolay Izhikov
>Priority: Blocker
> Attachments: CdcFailureOnComplexKey.html, CdcFailureOnComplexKey.patch
>
>
> In case if complex key is used (i.e. custom class or composite primary key in
> terms of SQL), when data is put into cache, below error occurs in
> IgniteToIgniteCdcStreamer which streams data from updated datacenter:
> {code}
> [18:46:36,757][SEVERE][Thread-1][] Cdc error
> class org.apache.ignite.IgniteException: Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
> at java.util.ArrayList.forEach(ArrayList.java:1257)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
> at
> org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
> at
> org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
> ... 20 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
> object class [type=class
> org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
> at
> org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
>
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Priority: Blocker (was: Major)
> IgniteToIgniteCdcStreamer fails on complex keys
> ---
>
> Key: IGNITE-15769
> URL: https://issues.apache.org/jira/browse/IGNITE-15769
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Shishkov
>Priority: Blocker
> Attachments: CdcFailureOnComplexKey.html, CdcFailureOnComplexKey.patch
>
>
> In case if complex key is used (i.e. custom class or composite primary key in
> terms of SQL), when data is put into cache, below error occurs in
> IgniteToIgniteCdcStreamer which streams data from updated datacenter:
> {code}
> [18:46:36,757][SEVERE][Thread-1][] Cdc error
> class org.apache.ignite.IgniteException: Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
> at java.util.ArrayList.forEach(ArrayList.java:1257)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
> at
> org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
> at
> org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
> ... 20 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
> object class [type=class
> org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
> at
> org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
> at
> org.apache.ignite.internal.process
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Description:
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data is put into cache, below error occurs in
IgniteToIgniteCdcStreamer which streams data from updated datacenter:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extensions): [^CdcFailureOnComplexKey.patch]
There are 3 test failures (total amount of test - 4):
# Cache create via DDL with not null f
[jira] [Commented] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430036#comment-17430036 ] Andrey Mashenkov commented on IGNITE-15414: --- [~jooger], done. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify all linked properties > and the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. > 4. Because of table configuration public keys and internal keys are written > within the same transaction, and validation happens just before the > transaction commit, it is possible internal keys calculation will be failed > with IllegalArgumentException. > It is expected the only Validation exception will be thrown, but there is no > way, for now, to force configuration validation on incomplete change. > Let's just rethrow the correct exception and left a TODO with a link to > IGNITE-15747. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15414: -- Description: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. 4. Because of table configuration public keys and internal keys are written within the same transaction, and validation happens just before the transaction commit, it is possible internal keys calculation will be failed with IllegalArgumentException. It is expected the only Validation exception will be thrown, but there is no way, for now, to force configuration validation on incomplete change. Let's just rethrow the correct exception and left a TODO with a link to IGNITE-15747. was: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. 4. Because of table configuration public keys and internal keys are written within the same transaction, and validation happens just before the transaction commit, it is possible internal keys calculation will be failed with IllegalArgumentException. It is expected the only Validation exception will be thrown, but there is no way, for now, to force configuration validation on incomplete change. Let's just rethrow the correct exception and left a todo with link to IGNITE-15747/ > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify all linked properties > and the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. > 4. Because of table configuration public keys and internal keys are written > within the same transaction, and validation happens just before the > transaction commit, it is possible internal keys calculation will be failed > with IllegalArgumentException. > It is expected the only Validation exception will be thrown, but there is no > way, for now, to force configuration validation on incomplete change. > Let's just rethrow the correct exception and left a TODO with a link to > IGNITE-15747. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15414: -- Description: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. 4. Because of table configuration public keys and internal keys are written within the same transaction, and validation happens just before the transaction commit, it is possible internal keys calculation will be failed with IllegalArgumentException. It is expected the only Validation exception will be thrown, but there is no way, for now, to force configuration validation on incomplete change. Let's just rethrow the correct exception and left a todo with link to IGNITE-15747/ was: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. 4. Because of table configuration public keys and internal keys are written within the same transaction, and validation happens just before the transaction commit, it is possible internal keys calculation will be failed with IllegalArgumentException. Let's > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify all linked properties > and the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. > 4. Because of table configuration public keys and internal keys are written > within the same transaction, and validation happens just before the > transaction commit, it is possible internal keys calculation will be failed > with IllegalArgumentException. > It is expected the only Validation exception will be thrown, but there is no > way, for now, to force configuration validation on incomplete change. > Let's just rethrow the correct exception and left a todo with link to > IGNITE-15747/ -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Description:
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data isto put in cache, below error occurs in
IgniteToIgniteCdcStreamer which streams data from updated datacenter:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extensions): [^CdcFailureOnComplexKey.patch]
There are 3 test failures (total amount of test - 4):
# Cache create via DDL with not null f
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Attachment: CdcFailureOnComplexKey.html
> IgniteToIgniteCdcStreamer fails on complex keys
> ---
>
> Key: IGNITE-15769
> URL: https://issues.apache.org/jira/browse/IGNITE-15769
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Shishkov
>Priority: Major
> Attachments: CdcFailureOnComplexKey.html, CdcFailureOnComplexKey.patch
>
>
> In case if complex key is used (i.e. custom class or composite primary key in
> terms of SQL), when data isto put in cache, below error occurs in
> IgniteToIgniteCdcStreamer which streams data from updated datacenter:
> {code}
> [18:46:36,757][SEVERE][Thread-1][] Cdc error
> class org.apache.ignite.IgniteException: Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
> at java.util.ArrayList.forEach(ArrayList.java:1257)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
> at
> org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
> at
> org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
> ... 20 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
> object class [type=class
> org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
> at
> org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
> at
> org.apache.ignite.internal.p
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Description:
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data isto put in cache, below error occurs in
IgniteToIgniteCdcStreamer which streams data from updated datacenter:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extensions): [^CdcFailureOnComplexKey.patch]
There are 3 test failures (total amount of test - 4):
# Cache create via DDL with not null f
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Description:
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data is put in cache, below error occurs in
IgniteToIgniteCdcStreamer:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extensions): [^CdcFailureOnComplexKey.patch]
There are 3 test failures (total amount of test - 4):
# Cache create via DDL with not null fields in key.
# Cache create via DDL without
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Description:
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data is put in cache, below error occurs in
IgniteToIgniteCdcStreamer:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extensions): [^CdcFailureOnComplexKey.patch]
There are 3 test failures:
# Cache create via DDL with not null fields in key.
# Cache create via DDL without not null-fields in key (str
[jira] [Updated] (IGNITE-15769) IgniteToIgniteCdcStreamer fails on complex keys
[
https://issues.apache.org/jira/browse/IGNITE-15769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ilya Shishkov updated IGNITE-15769:
---
Summary: IgniteToIgniteCdcStreamer fails on complex keys (was:
IgniteToIgniteCdcConsumer fails on complex keys)
> IgniteToIgniteCdcStreamer fails on complex keys
> ---
>
> Key: IGNITE-15769
> URL: https://issues.apache.org/jira/browse/IGNITE-15769
> Project: Ignite
> Issue Type: Bug
>Reporter: Ilya Shishkov
>Priority: Major
> Attachments: CdcFailureOnComplexKey.patch
>
>
> In case if complex key is used (i.e. custom class or composite primary key in
> terms of SQL), when data is put in cache, below error occurs in
> IgniteToIgniteCdcStreamer:
> {code}
> [18:46:36,757][SEVERE][Thread-1][] Cdc error
> class org.apache.ignite.IgniteException: Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
> at
> org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
> at
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
> at java.util.ArrayList.forEach(ArrayList.java:1257)
> at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
> at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
> at
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
> at
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
> at
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
> at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
> at
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
> at
> org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
> at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
> at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: class
> org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
> Transaction has been rolled back:
> 5b8c25a7c71--0ea3-dad8--0002
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
> at
> org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
> at
> org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
> at
> org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
> at
> org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
> at
> org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
> ... 20 more
> Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
> object class [type=class
> org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
> at
> org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
> at
> org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
> at
> org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
> at
> org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
> at
> org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
> at
> org.apache.ignite.internal.p
[jira] [Updated] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15414: -- Description: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. 4. Because of table configuration public keys and internal keys are written within the same transaction, and validation happens just before the transaction commit, it is possible internal keys calculation will be failed with IllegalArgumentException. Let's was: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify all linked properties > and the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. > 4. Because of table configuration public keys and internal keys are written > within the same transaction, and validation happens just before the > transaction commit, it is possible internal keys calculation will be failed > with IllegalArgumentException. > Let's -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-15769) IgniteToIgniteCdcConsumer fails on complex keys
Ilya Shishkov created IGNITE-15769:
--
Summary: IgniteToIgniteCdcConsumer fails on complex keys
Key: IGNITE-15769
URL: https://issues.apache.org/jira/browse/IGNITE-15769
Project: Ignite
Issue Type: Bug
Reporter: Ilya Shishkov
Attachments: CdcFailureOnComplexKey.patch
In case if complex key is used (i.e. custom class or composite primary key in
terms of SQL), when data is put in cache, below error occurs in
IgniteToIgniteCdcStreamer:
{code}
[18:46:36,757][SEVERE][Thread-1][] Cdc error
class org.apache.ignite.IgniteException: Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:138)
at
org.apache.ignite.internal.cdc.WalRecordsConsumer.onRecords(WalRecordsConsumer.java:157)
at
org.apache.ignite.internal.cdc.CdcMain.consumeSegment(CdcMain.java:472)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
at
java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at java.util.stream.SortedOps$RefSortingSink.end(SortedOps.java:390)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.Sink$ChainedReference.end(Sink.java:258)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
at
java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at
java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at
java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at
java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at
org.apache.ignite.internal.cdc.CdcMain.consumeWalSegmentsUntilStopped(CdcMain.java:403)
at org.apache.ignite.internal.cdc.CdcMain.runX(CdcMain.java:285)
at org.apache.ignite.internal.cdc.CdcMain.run(CdcMain.java:229)
at java.lang.Thread.run(Thread.java:748)
Caused by: class
org.apache.ignite.internal.transactions.IgniteTxRollbackCheckedException:
Transaction has been rolled back:
5b8c25a7c71--0ea3-dad8--0002
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4385)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.putAllConflict(GridCacheAdapter.java:2606)
at
org.apache.ignite.internal.processors.cache.GridCacheProxyImpl.putAllConflict(GridCacheProxyImpl.java:565)
at
org.apache.ignite.cdc.CdcEventsApplier.applyIf(CdcEventsApplier.java:151)
at
org.apache.ignite.cdc.CdcEventsApplier.apply(CdcEventsApplier.java:126)
at
org.apache.ignite.cdc.IgniteToIgniteCdcStreamer.onEvents(IgniteToIgniteCdcStreamer.java:119)
... 20 more
Caused by: class org.apache.ignite.IgniteCheckedException: Unexpected binary
object class [type=class
org.apache.ignite.internal.processors.cache.KeyCacheObjectImpl]
at
org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.value(QueryBinaryProperty.java:128)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateProps(QueryTypeDescriptorImpl.java:615)
at
org.apache.ignite.internal.processors.query.QueryTypeDescriptorImpl.validateKeyAndValue(QueryTypeDescriptorImpl.java:587)
at
org.apache.ignite.internal.processors.query.GridQueryProcessor.validateKeyAndValue(GridQueryProcessor.java:3685)
at
org.apache.ignite.internal.processors.cache.GridCacheContext.validateKeyAndValue(GridCacheContext.java:1883)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWriteEntry(GridNearTxLocal.java:1550)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.enlistWrite(GridNearTxLocal.java:1278)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllAsync0(GridNearTxLocal.java:927)
at
org.apache.ignite.internal.processors.cache.distributed.near.GridNearTxLocal.putAllDrAsync(GridNearTxLocal.java:553)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$23.inOp(GridCacheAdapter.java:2608)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$SyncInOp.op(GridCacheAdapter.java:5641)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4366)
... 25 more
{code}
Here is a problem reproducer (should be applied to master of
ignite-extension
[jira] [Updated] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15414: -- Description: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify all linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. was: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify all linked properties > and the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrey Mashenkov updated IGNITE-15414: -- Description: 1. TableValidator code exists, but validator is never registered. Let's fix this. 2. ColumnType configuration is not validated. Let's mark all properties as immutable and add a validator for ColumnType to verify linked properties and the whole type can be built correctly. 3. Some values make no sense in some cases (e.g. precision for non-numeric types). It is ok as the will be removed once the polymorphic validator will be implemented. These values don't affect configuration correctness as they will be never used. was: Current approach of validating configuration changes by throwing SchemaModificationExceptions during analyzing configuration from within one of it's listeners has few disadvantages: * Configuration has already been stored, so it could be retrieved by other components that didn't know that it was considered invalid. * It's not possible to have different listeners for different configuration items that were triggered by one change if one of items considered to be invalid. In other word: ** Let's assume that there are two listeners one for column.nullable() and another for collumn.type(). ** Customer alters table by both changing column's nullable and type values. Let's say that new nullable value is valid and type isn't. ** column.nullable().listener() triggers first and successfully updates schema registry with given change. ** After that column.type.listener() takes it time and throws SchemaModificationException. ** It actually means that we either: *** will have partially applied schema changes, that seems to be error prone, or *** should implement schema registry rollback logic, or *** strictly use only one top level listener, like we do it now. It worth to mention that such big listeners looks messy. All in all, in order to overcome drawbacks mentioned above and some unmentioned ones it's possible to use configuration validations that prevents processing and saving an invalid configuration changes. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > 1. TableValidator code exists, but validator is never registered. Let's fix > this. > 2. ColumnType configuration is not validated. Let's mark all properties as > immutable and add a validator for ColumnType to verify linked properties and > the whole type can be built correctly. > 3. Some values make no sense in some cases (e.g. precision for non-numeric > types). It is ok as the will be removed once the polymorphic validator will > be implemented. These values don't affect configuration correctness as they > will be never used. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, new ServiceRequestContext("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContext
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Or using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ServiceRequestCo
[jira] [Updated] (IGNITE-15718) Issues in implementation of the method TableManager#table(String)
[ https://issues.apache.org/jira/browse/IGNITE-15718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-15718: - Description: The method (TableManager#table(String)), where we are waiting for a table creation by name, is not correct, because we can see an event about another table with the same name (it is possible when a node is lagging behind of other and is seeing outdated events about drop/create the table, but the name unique only in one time). Need to wait for table creation by ID, because ID is unique for each table even on infinity time. Also, after the ticket (IGNITE-15412) where Configuration API was extended of possibility to read a Metasorage value directly, has been fixed, it is time to fix method _TableManager#isTableConfigured(String)_. was: In the method (TableManager#table(String)) where we are waiting for a table creation by name is not correct, because we can to see event about another table with the same name (it is possible when a node is lagging behind of other and is seeing outdated events about drop/create table, but the name unique only in one time). Need to make waiting of table creation by ID, because ID is unique for each table even on infinity time. Also, after the ticket (IGNITE-15412) where Configuration API was extended of possibility to read a Metasorage value directly, have been fixed, it is a time to fix method _TableManager#isTableConfigured(String)_. > Issues in implementation of the method TableManager#table(String) > - > > Key: IGNITE-15718 > URL: https://issues.apache.org/jira/browse/IGNITE-15718 > Project: Ignite > Issue Type: Bug >Reporter: Vladislav Pyatkov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha4 > > > The method (TableManager#table(String)), where we are waiting for a table > creation by name, is not correct, because we can see an event about another > table with the same name (it is possible when a node is lagging behind of > other and is seeing outdated events about drop/create the table, but the name > unique only in one time). Need to wait for table creation by ID, because ID > is unique for each table even on infinity time. > Also, after the ticket (IGNITE-15412) where Configuration API was extended of > possibility to read a Metasorage value directly, has been fixed, it is time > to fix method _TableManager#isTableConfigured(String)_. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430025#comment-17430025
]
Andrey Mashenkov commented on IGNITE-13667:
---
[~korlov], Could you please review, does it look to be enough for your purposes
in Calcite?
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
> Fix For: 3.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We may need a way to keep original columns order, e.g. for SQL, where user
> expects "select * from" return columns in user-defined order in "create
> table" statement. It may be a simple relocation table which is a permutation
> of indices {{[0..n)}}, so that an internal column order for user index {{n}}
> is {{relocationTbl[n]}}. Or maybe we could just store column user-defined
> order within the column object to use it in Calcite.
> NB: the row assembler will still need to access the internal sorted order for
> proper row assembly.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15711) Implement Loza stop
[ https://issues.apache.org/jira/browse/IGNITE-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-15711: - Description: My suggestions: # Add a busy lock on all public methods of Loza # Add checks that all RAFT group (nodes and services) have been stopped before that component stop. # Implement unit tests for the manager. ## All public methods should throw an exception when the component is stopped. ## All resources are closed after the component is stopped. > Implement Loza stop > --- > > Key: IGNITE-15711 > URL: https://issues.apache.org/jira/browse/IGNITE-15711 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3 > > My suggestions: > # Add a busy lock on all public methods of Loza > # Add checks that all RAFT group (nodes and services) have been stopped > before that component stop. > # Implement unit tests for the manager. > ## All public methods should throw an exception when the component is > stopped. > ## All resources are closed after the component is stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15711) Implement Loza stop
[ https://issues.apache.org/jira/browse/IGNITE-15711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-15711: - Description: My suggestions: # Add a busy lock on all public methods of Loza # Add checks that all RAFT group (nodes and services) have been stopped before that component stop. # Implement unit tests for the manager. ## All public methods should throw an exception when the component is stopped. ## All resources are closed after the component is stopped. was: My suggestions: # Add a busy lock on all public methods of Loza # Add checks that all RAFT group (nodes and services) have been stopped before that component stop. # Implement unit tests for the manager. ## All public methods should throw an exception when the component is stopped. ## All resources are closed after the component is stopped. > Implement Loza stop > --- > > Key: IGNITE-15711 > URL: https://issues.apache.org/jira/browse/IGNITE-15711 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Priority: Major > Labels: ignite-3 > > My suggestions: > # Add a busy lock on all public methods of Loza > # Add checks that all RAFT group (nodes and services) have been stopped > before that component stop. > # Implement unit tests for the manager. > ## All public methods should throw an exception when the component is > stopped. > ## All resources are closed after the component is stopped. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15724) MetastorageManager uses incorrect predicate in order to create corresponding raft group service
[
https://issues.apache.org/jira/browse/IGNITE-15724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vyacheslav Koptilin updated IGNITE-15724:
-
Description:
Currently, when the {{MetaStorageManager}} is started, the metastorage raft
group is reduced to online nodes that satisfies to the
{{metaStorageNodesContainsLocPred}}.
{code:java}
String[] metastorageNodes =
this.locCfgMgr.configurationRegistry().getConfiguration(NodeConfiguration.KEY).metastorageNodes().value();
Predicate metaStorageNodesContainsLocPred =
clusterNode -> Arrays.asList(metastorageNodes).contains(clusterNode.name());
...
List metaStorageMembers =
clusterNetSvc.topologyService().allMembers().stream()
.filter(metaStorageNodesContainsLocPred)
.collect(Collectors.toList());
{code}
That is actually a subset of required nodes. In that case, only the first node
creates the metastorage raft group, and the second one will be rejected with
the following messages:
{noformat}
[WARNING][%ITIgnitionTest_testNodesStartWithBootstrapConfiguration_3344%JRaft-Request-Processor-27][NodeImpl]
Node ignore PreVoteRequest from
172.24.144.1:3345 as it is not in conf .
[WARNING][%ITIgnitionTest_testNodesStartWithBootstrapConfiguration_3345%JRaft-Response-Processor-1][NodeImpl]
Node received invalid
PreVoteResponse from 172.24.144.1:3344, term 1, expect=0.
{noformat}
It seems that this issue cannot be properly fixed until is IGNITE-14414 is
implemented. For now, we can just prohibit creating a metastorage group greater
than one node.
> MetastorageManager uses incorrect predicate in order to create corresponding
> raft group service
> ---
>
> Key: IGNITE-15724
> URL: https://issues.apache.org/jira/browse/IGNITE-15724
> Project: Ignite
> Issue Type: Bug
>Reporter: Vyacheslav Koptilin
>Priority: Critical
> Labels: ignite-3
> Fix For: 3.0.0-alpha4
>
>
> Currently, when the {{MetaStorageManager}} is started, the metastorage raft
> group is reduced to online nodes that satisfies to the
> {{metaStorageNodesContainsLocPred}}.
> {code:java}
> String[] metastorageNodes =
> this.locCfgMgr.configurationRegistry().getConfiguration(NodeConfiguration.KEY).metastorageNodes().value();
> Predicate metaStorageNodesContainsLocPred =
> clusterNode ->
> Arrays.asList(metastorageNodes).contains(clusterNode.name());
> ...
>
> List metaStorageMembers =
> clusterNetSvc.topologyService().allMembers().stream()
> .filter(metaStorageNodesContainsLocPred)
> .collect(Collectors.toList());
> {code}
> That is actually a subset of required nodes. In that case, only the first
> node creates the metastorage raft group, and the second one will be rejected
> with the following messages:
> {noformat}
> [WARNING][%ITIgnitionTest_testNodesStartWithBootstrapConfiguration_3344%JRaft-Request-Processor-27][NodeImpl]
> Node ignore PreVoteRequest from
> 172.24.144.1:3345 as it is not in conf [index=1, term=1], conf=172.24.144.1:3344, oldConf=]>.
> [WARNING][%ITIgnitionTest_testNodesStartWithBootstrapConfiguration_3345%JRaft-Response-Processor-1][NodeImpl]
> Node received invalid
> PreVoteResponse from 172.24.144.1:3344, term 1, expect=0.
> {noformat}
> It seems that this issue cannot be properly fixed until is IGNITE-14414 is
> implemented. For now, we can just prohibit creating a metastorage group
> greater than one node.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430016#comment-17430016 ] Yury Gerzhedovich commented on IGNITE-15414: [~amashenkov], let's actualize the description of the ticket. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Current approach of validating configuration changes by throwing > SchemaModificationExceptions during analyzing configuration from within one > of it's listeners has few disadvantages: > * Configuration has already been stored, so it could be retrieved by other > components that didn't know that it was considered invalid. > * It's not possible to have different listeners for different configuration > items that were triggered by one change if one of items considered to be > invalid. In other word: > ** Let's assume that there are two listeners one for column.nullable() and > another for collumn.type(). > ** Customer alters table by both changing column's nullable and type values. > Let's say that new nullable value is valid and type isn't. > ** column.nullable().listener() triggers first and successfully updates > schema registry with given change. > ** After that column.type.listener() takes it time and throws > SchemaModificationException. > ** It actually means that we either: > *** will have partially applied schema changes, that seems to be error > prone, or > *** should implement schema registry rollback logic, or > *** strictly use only one top level listener, like we do it now. It worth to > mention that such big listeners looks messy. > All in all, in order to overcome drawbacks mentioned above and some > unmentioned ones it's possible to use configuration validations that prevents > processing and saving an invalid configuration changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430013#comment-17430013
]
Yury Gerzhedovich commented on IGNITE-13667:
[~amashenkov], LGTM, except few comments in PR.
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
> Fix For: 3.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We may need a way to keep original columns order, e.g. for SQL, where user
> expects "select * from" return columns in user-defined order in "create
> table" statement. It may be a simple relocation table which is a permutation
> of indices {{[0..n)}}, so that an internal column order for user index {{n}}
> is {{relocationTbl[n]}}. Or maybe we could just store column user-defined
> order within the column object to use it in Calcite.
> NB: the row assembler will still need to access the internal sorted order for
> proper row assembly.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-15599) Get rid Guava in Calcite module
[ https://issues.apache.org/jira/browse/IGNITE-15599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430004#comment-17430004 ] Yury Gerzhedovich commented on IGNITE-15599: [~tledkov-gridgain], Looks good to me except Integration tests which use guava still. > Get rid Guava in Calcite module > > > Key: IGNITE-15599 > URL: https://issues.apache.org/jira/browse/IGNITE-15599 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Assignee: Taras Ledkov >Priority: Major > Labels: calcite2-required, calcite3-required, ignite-3, tech-debt > Fix For: 3.0.0-alpha4 > > > Need to remove all guava usages in the Calcite module. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov updated IGNITE-13667:
--
Description:
When a schema is defined, the key chunk columns and value chunk columns are
sorted so that fixlen columns go first and varlen columns go second, so the
sorted column order differs from the order of the user-defined columns.
We may need a way to keep original columns order, e.g. for SQL, where user
expects "select * from" return columns in user-defined order in "create table"
statement. It may be a simple relocation table which is a permutation of
indices {{[0..n)}}, so that an internal column order for user index {{n}} is
{{relocationTbl[n]}}. Or maybe we could just store column user-defined order
within the column object to use it in Calcite.
NB: the row assembler will still need to access the internal sorted order for
proper row assembly.
was:
When a schema is defined, the key chunk columns and value chunk columns are
sorted so that fixlen columns go first and varlen columns go second, so the
sorted column order differs from the order of the user-defined columns.
We may need a way to keep original columns order, e.g. for SQL, where user
expects "select * from" return columns in user-defined order in "create table"
statement. It may be a simple relocation table which is a permutation of
indices {{[0..n)}}, so that an internal column order for user index {{n}} is
{{relocationTbl[n]}}.
NB: the row assembler will still need to access the internal sorted order for
proper row assembly.
Update:
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
> Fix For: 3.0
>
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We may need a way to keep original columns order, e.g. for SQL, where user
> expects "select * from" return columns in user-defined order in "create
> table" statement. It may be a simple relocation table which is a permutation
> of indices {{[0..n)}}, so that an internal column order for user index {{n}}
> is {{relocationTbl[n]}}. Or maybe we could just store column user-defined
> order within the column object to use it in Calcite.
> NB: the row assembler will still need to access the internal sorted order for
> proper row assembly.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov updated IGNITE-13667:
--
Description:
When a schema is defined, the key chunk columns and value chunk columns are
sorted so that fixlen columns go first and varlen columns go second, so the
sorted column order differs from the order of the user-defined columns.
We may need a way to keep original columns order, e.g. for SQL, where user
expects "select * from" return columns in user-defined order in "create table"
statement. It may be a simple relocation table which is a permutation of
indices {{[0..n)}}, so that an internal column order for user index {{n}} is
{{relocationTbl[n]}}.
NB: the row assembler will still need to access the internal sorted order for
proper row assembly.
Update:
was:
When a schema is defined, the key chunk columns and value chunk columns are
sorted so that fixlen columns go first and varlen columns go second, so the
sorted column order differs from the order of the user-defined columns.
We need to add a simple relocation table which is a permutation of indices
{{[0..n)}}, so that an internal column order for user index {{n}} is
{{relocationTbl[n]}}.
NB: the tuple assembler will still need to access the internal sorted order for
proper tuple assembly.
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
> Fix For: 3.0
>
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We may need a way to keep original columns order, e.g. for SQL, where user
> expects "select * from" return columns in user-defined order in "create
> table" statement. It may be a simple relocation table which is a permutation
> of indices {{[0..n)}}, so that an internal column order for user index {{n}}
> is {{relocationTbl[n]}}.
> NB: the row assembler will still need to access the internal sorted order for
> proper row assembly.
> Update:
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov updated IGNITE-13667:
--
Fix Version/s: 3.0
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
> Fix For: 3.0
>
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We need to add a simple relocation table which is a permutation of indices
> {{[0..n)}}, so that an internal column order for user index {{n}} is
> {{relocationTbl[n]}}.
> NB: the tuple assembler will still need to access the internal sorted order
> for proper tuple assembly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15768) Change get version script in nightly release build after IGNITE-15645
[
https://issues.apache.org/jira/browse/IGNITE-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Daschinsky updated IGNITE-15768:
-
Description:
After IGNITE-15645,
[script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
should be changed
Before:
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
After
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat parent/pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
Without this change, incorrect version of nighlty build is generated - 3.0.0
instead of 2.12.0
was:
After IGNITE-15645,
[script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
should be changed
Before:
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
After
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat parent/pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
> Change get version script in nightly release build after IGNITE-15645
> -
>
> Key: IGNITE-15768
> URL: https://issues.apache.org/jira/browse/IGNITE-15768
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Daschinsky
>Assignee: Petr Ivanov
>Priority: Major
>
> After IGNITE-15645,
> [script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
> should be changed
> Before:
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
> After
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat parent/pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
> Without this change, incorrect version of nighlty build is generated - 3.0.0
> instead of 2.12.0
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-15746) Configuration: in-closure data validation
[
https://issues.apache.org/jira/browse/IGNITE-15746?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429975#comment-17429975
]
Alexander Lapin edited comment on IGNITE-15746 at 10/18/21, 12:01 PM:
--
Duplicates https://issues.apache.org/jira/browse/IGNITE-15747
was (Author: alapin):
Duplicates https://issues.apache.org/jira/browse/IGNITE-15746
> Configuration: in-closure data validation
> -
>
> Key: IGNITE-15746
> URL: https://issues.apache.org/jira/browse/IGNITE-15746
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: configuration, ignite-3
>
> h3. Problem
> Configuration validation occurs after the code executed in the change
> closure, which means that inside closure it's possible to have invalid data
> in configuration view.
> Let's take a look at the following example.
> Assuming that there's following schema:
>
> {code:java}
> @Config
> public class TableConfigurationSchema {
> /** Count of table partition replicas. */
> @Min(1)
> @Value(hasDefault = true)
> public int replicas = 1;
> }{code}
> and following change closure
> {code:java}
> tblCfg.change(ch -> {
> ch.changeReplicas(outherValueThatMightBeZero);
>
> ch.changePartitions(1000 / ch.replicas());
> });
> {code}
> we might and up with an ArithmeticException: divide by zero if
> outherValueThatMightBeZero == 0 instead of ValidationException.
> In order not to duplicate validation logic that was specified in validators,
> throw ValidationException instead of ones specified in closure, etc it'll
> useful to guarantee that the data inside Views is valid at list at some
> points.
> h3. Possible solution
> From the user experience it'll be great to validate data on every setter,
> however that seems to be impossible because of lack of terminal build()
> operation.
> Another option is to have validate() method on view and maybe value.
> On more solution, to have validate() method within some utility class instead
> of view.
> So, all-in-all, something similar to
> {code:java}
> tblCfg.change(ch -> {
> ch.changeReplicas(outherValueThatMightBeZero);
>
> ch.validate();
> // or
> SomeCfgUtils.validate(ch);
>
> ch.changePartitions(1000 / ch.replicas());
> });
> {code}
> is expected.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (IGNITE-15746) Configuration: in-closure data validation
[
https://issues.apache.org/jira/browse/IGNITE-15746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alexander Lapin resolved IGNITE-15746.
--
Resolution: Duplicate
Duplicates https://issues.apache.org/jira/browse/IGNITE-15746
> Configuration: in-closure data validation
> -
>
> Key: IGNITE-15746
> URL: https://issues.apache.org/jira/browse/IGNITE-15746
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexander Lapin
>Priority: Major
> Labels: configuration, ignite-3
>
> h3. Problem
> Configuration validation occurs after the code executed in the change
> closure, which means that inside closure it's possible to have invalid data
> in configuration view.
> Let's take a look at the following example.
> Assuming that there's following schema:
>
> {code:java}
> @Config
> public class TableConfigurationSchema {
> /** Count of table partition replicas. */
> @Min(1)
> @Value(hasDefault = true)
> public int replicas = 1;
> }{code}
> and following change closure
> {code:java}
> tblCfg.change(ch -> {
> ch.changeReplicas(outherValueThatMightBeZero);
>
> ch.changePartitions(1000 / ch.replicas());
> });
> {code}
> we might and up with an ArithmeticException: divide by zero if
> outherValueThatMightBeZero == 0 instead of ValidationException.
> In order not to duplicate validation logic that was specified in validators,
> throw ValidationException instead of ones specified in closure, etc it'll
> useful to guarantee that the data inside Views is valid at list at some
> points.
> h3. Possible solution
> From the user experience it'll be great to validate data on every setter,
> however that seems to be impossible because of lack of terminal build()
> operation.
> Another option is to have validate() method on view and maybe value.
> On more solution, to have validate() method within some utility class instead
> of view.
> So, all-in-all, something similar to
> {code:java}
> tblCfg.change(ch -> {
> ch.changeReplicas(outherValueThatMightBeZero);
>
> ch.validate();
> // or
> SomeCfgUtils.validate(ch);
>
> ch.changePartitions(1000 / ch.replicas());
> });
> {code}
> is expected.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-14776:
Summary: .NET: ClientFailoverSocket sets logger too late, resulting in null
loggers downstream (was: .NET: ClientFailoverSocket sets logger too late,
resulting in null loggers downstream.)
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream
> -
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-14776) .NET: ClientFailoverSocket sets logger too late, resulting in null loggers downstream.
[
https://issues.apache.org/jira/browse/IGNITE-14776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Tupitsyn updated IGNITE-14776:
Priority: Critical (was: Major)
> .NET: ClientFailoverSocket sets logger too late, resulting in null loggers
> downstream.
> --
>
> Key: IGNITE-14776
> URL: https://issues.apache.org/jira/browse/IGNITE-14776
> Project: Ignite
> Issue Type: Bug
> Components: clients, platforms
>Affects Versions: 2.10, 2.11
>Reporter: Robert May
>Assignee: Pavel Tupitsyn
>Priority: Critical
> Labels: .NET
> Fix For: 2.12
>
>
> Because the logger is set last inside of the
> {code:c#}ClientFailoverSocket{code} class, if there are issues with the
> {code:c#}GetIpEndpoints{code} call, an argument exception can occur when a
> debug message is logged inside of {code:c#}GetIps{code} when the ip address
> can't be parsed. In my case, this occurred with a DNS failure.
> Stack Trace:
> {code:c#}
> System.ArgumentNullException: Value cannot be null. (Parameter 'logger')
> at Apache.Ignite.Core.Impl.Common.IgniteArgumentCheck.NotNull(Object arg,
> String argName)
> at Apache.Ignite.Core.Log.LoggerExtensions.Log(ILogger logger, LogLevel
> level, Exception ex, String message)
> at Apache.Ignite.Core.Log.LoggerExtensions.Debug(ILogger logger, Exception
> ex, String message)
> at Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.GetIps(String host,
> Boolean suppressExceptions)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket.d__11.MoveNext()
> at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
> at System.Linq.Enumerable.ToList[TSource](IEnumerable`1 source)
> at
> Apache.Ignite.Core.Impl.Client.ClientFailoverSocket..ctor(IgniteClientConfiguration
> config, Marshaller marsh, TransactionsClient transactions)
> at
> Apache.Ignite.Core.Impl.Client.IgniteClient..ctor(IgniteClientConfiguration
> clientConfiguration)
> at Apache.Ignite.Core.Ignition.StartClient(IgniteClientConfiguration
> clientConfiguration)
> {code}
> Here's the constructor code:
> {code:c#}
> Debug.Assert(config != null);
> Debug.Assert(marsh != null);
> Debug.Assert(transactions != null);
> _config = config;
> _marsh = marsh;
> _transactions = transactions;
> #pragma warning disable 618 // Type or member is obsolete
> if (config.Host == null && (config.Endpoints == null ||
> config.Endpoints.Count == 0))
> {
> throw new IgniteClientException("Invalid
> IgniteClientConfiguration: Host is null, " +
> "Endpoints is null or empty.
> Nowhere to connect.");
> }
> #pragma warning restore 618
> _endPoints = GetIpEndPoints(config).ToList();
> if (_endPoints.Count == 0)
> {
> throw new IgniteClientException("Failed to resolve all
> specified hosts.");
> }
> _logger = (_config.Logger ??
> NoopLogger.Instance).GetLogger(GetType());
> ConnectDefaultSocket();
> OnFirstConnection();
> {code}
> Note how the _logger variable isn't set until the very end of the constructor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15490) Create an example(s) that can allow to demonstrate rebalancing from the end-user point of view
[ https://issues.apache.org/jira/browse/IGNITE-15490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vyacheslav Koptilin updated IGNITE-15490: - Parent: (was: IGNITE-14209) Issue Type: Improvement (was: Sub-task) > Create an example(s) that can allow to demonstrate rebalancing from the > end-user point of view > -- > > Key: IGNITE-15490 > URL: https://issues.apache.org/jira/browse/IGNITE-15490 > Project: Ignite > Issue Type: Improvement >Reporter: Vyacheslav Koptilin >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > Fix For: 3.0.0-alpha3 > > > Need to implement examples that show joining new nodes to the cluster and > rebalancing data across all the cluster nodes. The case with nodes moved > offline should be covered as well. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-15600) Cache for Calcite SQL plans in 3.0
[ https://issues.apache.org/jira/browse/IGNITE-15600?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Konstantin Orlov reassigned IGNITE-15600: - Assignee: Konstantin Orlov > Cache for Calcite SQL plans in 3.0 > -- > > Key: IGNITE-15600 > URL: https://issues.apache.org/jira/browse/IGNITE-15600 > Project: Ignite > Issue Type: Improvement > Components: sql >Reporter: Yury Gerzhedovich >Assignee: Konstantin Orlov >Priority: Major > Labels: calcite3-required, ignite-3, tech-debt > > As of now, we have Calcite SQL plan cache only for Ignite 2.x. For 3.0 need > to implement new one. It can be a simple LRU cache based on LinkedHashMap as > a first step or take from separate project, e.g. caffeine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15768) Change get version script in nightly release build after IGNITE-15645
[
https://issues.apache.org/jira/browse/IGNITE-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Daschinsky updated IGNITE-15768:
-
Description:
After IGNITE-15645,
[script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
should be changed
Before:
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
After
{code}
#!/usr/bin/env bash
set -x
igniteVersion=$(cat parent/pom.xml | \
grep -E -m 1
'[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
sed -r 's|.*>(.*)<.*|\1|' | sed -r 's|(.*)-SNAPSHOT|\1|').$(date
+%%Y%%m%%d -d "+3 hours")
echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
{code}
was:After IGNITE-15645, script should be changed
> Change get version script in nightly release build after IGNITE-15645
> -
>
> Key: IGNITE-15768
> URL: https://issues.apache.org/jira/browse/IGNITE-15768
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Daschinsky
>Priority: Major
>
> After IGNITE-15645,
> [script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
> should be changed
> Before:
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
> After
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat parent/pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-15768) Change get version script in nightly release build after IGNITE-15645
[
https://issues.apache.org/jira/browse/IGNITE-15768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Daschinsky reassigned IGNITE-15768:
Assignee: Petr Ivanov
> Change get version script in nightly release build after IGNITE-15645
> -
>
> Key: IGNITE-15768
> URL: https://issues.apache.org/jira/browse/IGNITE-15768
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Daschinsky
>Assignee: Petr Ivanov
>Priority: Major
>
> After IGNITE-15645,
> [script|https://ci.ignite.apache.org/admin/editRunType.html?id=buildType:Releases_NightlyRelease_ApacheIgniteNightlyReleasePrepare&runnerId=RUNNER_227&cameFromUrl=%2Fadmin%2FeditBuildRunners.html%3Fid%3DbuildType%253AReleases_NightlyRelease_ApacheIgniteNightlyReleasePrepare%26init%3D1&cameFromTitle=]
> should be changed
> Before:
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?' | \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
> After
> {code}
> #!/usr/bin/env bash
> set -x
> igniteVersion=$(cat parent/pom.xml | \
> grep -E -m 1
> '[0-9]+.[0-9]+.[0-9]+(-SNAPSHOT)?'| \
> sed -r 's|.*>(.*)<.*|\1|' | sed -r
> 's|(.*)-SNAPSHOT|\1|').$(date +%%Y%%m%%d -d "+3 hours")
> echo "##teamcity[setParameter name='IGNITE_VERSION' value='${igniteVersion}']"
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-15768) Change get version script in nightly release build after IGNITE-15645
Ivan Daschinsky created IGNITE-15768: Summary: Change get version script in nightly release build after IGNITE-15645 Key: IGNITE-15768 URL: https://issues.apache.org/jira/browse/IGNITE-15768 Project: Ignite Issue Type: Improvement Reporter: Ivan Daschinsky After IGNITE-15645, script should be changed -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-15414) Schema validation refactoring with configuration validators
[ https://issues.apache.org/jira/browse/IGNITE-15414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17429944#comment-17429944 ] Andrey Mashenkov commented on IGNITE-15414: --- Most of described were refactored in IGNITE-15404. I've turned on missed validators and added more tests. > Schema validation refactoring with configuration validators > --- > > Key: IGNITE-15414 > URL: https://issues.apache.org/jira/browse/IGNITE-15414 > Project: Ignite > Issue Type: Improvement >Reporter: Alexander Lapin >Assignee: Andrey Mashenkov >Priority: Major > Labels: ignite-3, tech-debt > Fix For: 3.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Current approach of validating configuration changes by throwing > SchemaModificationExceptions during analyzing configuration from within one > of it's listeners has few disadvantages: > * Configuration has already been stored, so it could be retrieved by other > components that didn't know that it was considered invalid. > * It's not possible to have different listeners for different configuration > items that were triggered by one change if one of items considered to be > invalid. In other word: > ** Let's assume that there are two listeners one for column.nullable() and > another for collumn.type(). > ** Customer alters table by both changing column's nullable and type values. > Let's say that new nullable value is valid and type isn't. > ** column.nullable().listener() triggers first and successfully updates > schema registry with given change. > ** After that column.type.listener() takes it time and throws > SchemaModificationException. > ** It actually means that we either: > *** will have partially applied schema changes, that seems to be error > prone, or > *** should implement schema registry rollback logic, or > *** strictly use only one top level listener, like we do it now. It worth to > mention that such big listeners looks messy. > All in all, in order to overcome drawbacks mentioned above and some > unmentioned ones it's possible to use configuration validations that prevents > processing and saving an invalid configuration changes. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ServiceRequestContext.builder("login", "user1").build(), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContext
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Or using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Serv
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ServiceRequestContext.builder("login", "user1").build(), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContext
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Or using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
Sample code for using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuil
[jira] [Updated] (IGNITE-15572) Ability to set custom execution context for Ignite service.
[
https://issues.apache.org/jira/browse/IGNITE-15572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pavel Pereslegin updated IGNITE-15572:
--
Description:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ServiceRequestContext.builder("login", "user1").build(), 0);
public class MyServiceImpl implements MyService {
@Override public void businessMethod() {
// Read thread local context.
ServiceRequestContext ctx = ServiceRequestContext.current();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 2. Pass Map directly and read it using special annotation.
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, Collections.singletonMap("login", "user1"), 0);
public class MyServiceImpl implements MyService {
@ServiceRequestContext
Supplier> ctxSupplier.
@Override public void businessMethod() {
Map ctx = ctxSupplier.get();
String user = ctx.attribute("login");
}
...
}
{code}
h4. 3. Or using the special ServiceContext method.
{code:java}
public class MyServiceImpl implements MyService {
private ServiceContext svcCtx;
@Override public void init(ServiceContext svcCtx) {
this.ctx = svcCtx;
}
@Override public void businessMethod() {
Map ctx = ctx.attributes();
String user = ctx.attribute("login");
}
...
}
{code}
h4. A more complex example of using "execution context".
{code:java}
// Creating service proxy invocation context with two attributes.
ServiceProxyContext ctx1 = new ServiceProxyContextBuilder("arg1",
10).put("usr", "X").build();
ServiceProxyContext ctx2 = new ServiceProxyContextBuilder("arg1",
6).put("usr", "X").build();
// Binding "execution context" to proxy invocation handler.
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx1, 0);
MyService proxy2 = ignite.services().serviceProxy("svc", MyService.class,
false, ctx2, 0);
// Invoke service methods with argument "10".
assert proxy1.multiply(2) == 10 * 2;
assert proxy1.divide(2) == 10 / 2;
// Invoke service methods with argument "6".
assert proxy2.multiply(2) == 6 * 2;
assert proxy2.divide(2) == 6 / 2;
...
{code}
Sample code for reading "execution context".
{code:java}
class MyServiceImpl implements MyService {
@LoggerResource
private IgniteLogger log;
@Override public int multiply(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"multiply", arg1, arg2));
return arg1 * arg2;
}
@Override public int divide(int arg2) {
// Getting proxy context.
ServiceProxyContext ctx = ServiceProxyContext.current();
// Reading attributes.
int arg1 = ctx.attribute("arg1");
String userId = ctx.attribute("usr");
log.info(String.format("user=%s, oper=%s, a=%d, b=%d", userId,
"divide", arg1, arg2));
return arg1 / arg2;
}
...
}
{code}
was:
In traditional microservices, we have the ability to set a custom execution
context.
For example, a REST service may obtain the session ID from the request. We can
say that each client request, in this case, has a set of explicit and implicit
parameters. One of the implicit parameters is a session identifier that can be
passed to the service using request headers.
It would be nice to have a similar feature in Ignite services.
The basic idea behind the implementation:
1. Allow the user to bind the "execution context" to the service proxy object.
2. Pass this context as an additional implicit parameter each time the user
service method is called.
h3. API proposal.
h4. 1. Using special class ServiceRequestContext (ServiceProxyContext).
{code:java}
MyService proxy1 = ignite.services().serviceProxy("svc", MyService.class,
false
[jira] [Updated] (IGNITE-13667) Add schema columns mapping from user order to system order
[
https://issues.apache.org/jira/browse/IGNITE-13667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrey Mashenkov updated IGNITE-13667:
--
Summary: Add schema columns mapping from user order to system order (was:
Add schema columns relocation table to map from user order to system order)
> Add schema columns mapping from user order to system order
> --
>
> Key: IGNITE-13667
> URL: https://issues.apache.org/jira/browse/IGNITE-13667
> Project: Ignite
> Issue Type: Improvement
>Reporter: Alexey Goncharuk
>Assignee: Andrey Mashenkov
>Priority: Minor
> Labels: iep-54, ignite-3
>
> When a schema is defined, the key chunk columns and value chunk columns are
> sorted so that fixlen columns go first and varlen columns go second, so the
> sorted column order differs from the order of the user-defined columns.
> We need to add a simple relocation table which is a permutation of indices
> {{[0..n)}}, so that an internal column order for user index {{n}} is
> {{relocationTbl[n]}}.
> NB: the tuple assembler will still need to access the internal sorted order
> for proper tuple assembly.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-15767) Need to workaround JDK bug JDK-8247750
[
https://issues.apache.org/jira/browse/IGNITE-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov reassigned IGNITE-15767:
--
Assignee: Ivan Bessonov
> Need to workaround JDK bug JDK-8247750
> --
>
> Key: IGNITE-15767
> URL: https://issues.apache.org/jira/browse/IGNITE-15767
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Assignee: Ivan Bessonov
>Priority: Major
> Fix For: 2.13
>
>
> See [https://bugs.openjdk.java.net/browse/JDK-8247750]
> ServerSocket.accept() with no timeout may throw SocketTimeoutException when
> the process receives a signal. It can cause unexpected exception in the disco
> reader:
> {noformat}
> [09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
> Failed to accept TCP connection.
> java.net.SocketTimeoutException: Accept timed out at
> java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
> at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
> at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
> at
> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
> There are mentions of this on StackOverflow and userlist, and we also see it
> when running Docker on Alpine 3.15. Speculation is that this is caused by a
> combination of a specific version of Linux kernel and environment.
> Sidenote: based on strace analysis on Alpine, it doesn't even receive any
> signals; it could be so that Alpine interrupts the syscall "as if" by a
> signal.
> The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
> accept in try-catch, and retry if getting the unexpected timeout exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-15767) Need to workaround JDK bug JDK-8247750
[
https://issues.apache.org/jira/browse/IGNITE-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-15767:
---
Description:
See [https://bugs.openjdk.java.net/browse/JDK-8247750]
ServerSocket.accept() with no timeout may throw SocketTimeoutException when the
process receives a signal. It can cause unexpected exception in the disco
reader:
{noformat}
[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi] Failed
to accept TCP connection.
java.net.SocketTimeoutException: Accept timed out at
java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
There are mentions of this on StackOverflow and userlist, and we also see it
when running Docker on Alpine 3.15. Speculation is that this is caused by a
combination of a specific version of Linux kernel and environment.
Sidenote: based on strace analysis on Alpine, it doesn't even receive any
signals; it could be so that Alpine interrupts the syscall "as if" by a signal.
The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
accept in try-catch, and retry if getting the unexpected timeout exception.
was:
See [https://bugs.openjdk.java.net/browse/JDK-8247750]
ServerSocket.accept() with no timeout may throw SocketTimeoutException when the
process receives a signal. It can cause unexpected exception in the disco
reader:
{noformat}
[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi] Failed
to accept TCP
connection.[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
Failed to accept TCP connection.java.net.SocketTimeoutException: Accept timed
out at java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
There are mentions of this on StackOverflow and userlist, and we also see it
when running Docker on Alpine 3.15. Speculation is that this is caused by a
combination of a specific version of Linux kernel and environment.
Sidenote: based on strace analysis on Alpine, it doesn't even receive any
signals; it could be so that Alpine interrupts the syscall "as if" by a signal.
The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
accept in try-catch, and retry if getting the unexpected timeout exception.
> Need to workaround JDK bug JDK-8247750
> --
>
> Key: IGNITE-15767
> URL: https://issues.apache.org/jira/browse/IGNITE-15767
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
> Fix For: 2.13
>
>
> See [https://bugs.openjdk.java.net/browse/JDK-8247750]
> ServerSocket.accept() with no timeout may throw SocketTimeoutException when
> the process receives a signal. It can cause unexpected exception in the disco
> reader:
> {noformat}
> [09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
> Failed to accept TCP connection.
> java.net.SocketTimeoutException: Accept timed out at
> java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
> at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
> at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
> at
> org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
> There are mentions of this on StackOverflow and userlist, and we also see it
> when running Docker on Alpine 3.15. Speculation i
[jira] [Updated] (IGNITE-15767) Need to workaround JDK bug JDK-8247750
[
https://issues.apache.org/jira/browse/IGNITE-15767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Bessonov updated IGNITE-15767:
---
Description:
See [https://bugs.openjdk.java.net/browse/JDK-8247750]
ServerSocket.accept() with no timeout may throw SocketTimeoutException when the
process receives a signal. It can cause unexpected exception in the disco
reader:
{noformat}
[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi] Failed
to accept TCP
connection.[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
Failed to accept TCP connection.java.net.SocketTimeoutException: Accept timed
out at java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
at
java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
There are mentions of this on StackOverflow and userlist, and we also see it
when running Docker on Alpine 3.15. Speculation is that this is caused by a
combination of a specific version of Linux kernel and environment.
Sidenote: based on strace analysis on Alpine, it doesn't even receive any
signals; it could be so that Alpine interrupts the syscall "as if" by a signal.
The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
accept in try-catch, and retry if getting the unexpected timeout exception.
was:
See [https://bugs.openjdk.java.net/browse/JDK-8247750]
ServerSocket.accept() with no timeout may throw SocketTimeoutException when the
process receives a signal. It can cause unexpected exception in the disco
reader:
{noformat}
[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi] Failed
to accept TCP
connection.[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
Failed to accept TCP connection.java.net.SocketTimeoutException: Accept timed
out at java.base/java.net.PlainSocketImpl.socketAccept(Native Method) at
java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565) at
java.base/java.net.ServerSocket.accept(ServerSocket.java:533) at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
{{}}
There are mentions of this on StackOverflow and userlist, and we also see it
when running Docker on Alpine 3.15. Speculation is that this is caused by a
combination of a specific version of Linux kernel and environment.
Sidenote: based on strace analysis on Alpine, it doesn't even receive any
signals; it could be so that Alpine interrupts the syscall "as if" by a signal.
The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
accept in try-catch, and retry if getting the unexpected timeout exception.
> Need to workaround JDK bug JDK-8247750
> --
>
> Key: IGNITE-15767
> URL: https://issues.apache.org/jira/browse/IGNITE-15767
> Project: Ignite
> Issue Type: Improvement
>Reporter: Ivan Bessonov
>Priority: Major
> Fix For: 2.13
>
>
> See [https://bugs.openjdk.java.net/browse/JDK-8247750]
> ServerSocket.accept() with no timeout may throw SocketTimeoutException when
> the process receives a signal. It can cause unexpected exception in the disco
> reader:
> {noformat}
> [09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
> Failed to accept TCP
> connection.[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
> Failed to accept TCP connection.java.net.SocketTimeoutException: Accept
> timed out at java.base/java.net.PlainSocketImpl.socketAccept(Native Method)
> at
> java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
> at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565)
> at java.base/java.net.ServerSocket.accept(ServerSocket.java:533)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
> at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
> at
> org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
>
[jira] [Created] (IGNITE-15767) Need to workaround JDK bug JDK-8247750
Ivan Bessonov created IGNITE-15767:
--
Summary: Need to workaround JDK bug JDK-8247750
Key: IGNITE-15767
URL: https://issues.apache.org/jira/browse/IGNITE-15767
Project: Ignite
Issue Type: Improvement
Reporter: Ivan Bessonov
Fix For: 2.13
See [https://bugs.openjdk.java.net/browse/JDK-8247750]
ServerSocket.accept() with no timeout may throw SocketTimeoutException when the
process receives a signal. It can cause unexpected exception in the disco
reader:
{noformat}
[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi] Failed
to accept TCP
connection.[09:52:26,301][SEVERE][tcp-disco-srvr-[:47500]-#3-#71][TcpDiscoverySpi]
Failed to accept TCP connection.java.net.SocketTimeoutException: Accept timed
out at java.base/java.net.PlainSocketImpl.socketAccept(Native Method) at
java.base/java.net.AbstractPlainSocketImpl.accept(AbstractPlainSocketImpl.java:458)
at java.base/java.net.ServerSocket.implAccept(ServerSocket.java:565) at
java.base/java.net.ServerSocket.accept(ServerSocket.java:533) at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServer.body(ServerImpl.java:6750)
at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:119)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$TcpServerThread.body(ServerImpl.java:6673)
at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:57){noformat}
{{}}
There are mentions of this on StackOverflow and userlist, and we also see it
when running Docker on Alpine 3.15. Speculation is that this is caused by a
combination of a specific version of Linux kernel and environment.
Sidenote: based on strace analysis on Alpine, it doesn't even receive any
signals; it could be so that Alpine interrupts the syscall "as if" by a signal.
The bug is not fixed in JDK 11 (surprisingly). WA is easy though - wrap the
accept in try-catch, and retry if getting the unexpected timeout exception.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
