[jira] [Commented] (IGNITE-13727) Calcite bug. Assertion on UNION with DISTINCT.
[
https://issues.apache.org/jira/browse/IGNITE-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250152#comment-17250152
]
Konstantin Orlov commented on IGNITE-13727:
---
Should be fixed by https://issues.apache.org/jira/browse/IGNITE-13817
> Calcite bug. Assertion on UNION with DISTINCT.
> --
>
> Key: IGNITE-13727
> URL: https://issues.apache.org/jira/browse/IGNITE-13727
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.9
>Reporter: Stanilovsky Evgeny
>Assignee: Konstantin Orlov
>Priority: Major
> Labels: calcite
>
> {noformat}
> SELECT distinct(name) FROM Orders UNION SELECT name from Account{noformat}
> breaks Calcite with assertion:
> {noformat}
> java.lang.AssertionError: traits=IGNITE.[].any.one-way, distributionhash[0]
> at org.apache.calcite.rel.core.Exchange.(Exchange.java:62)
> at org.apache.calcite.rel.core.Exchange.(Exchange.java:71)
> at
> org.apache.ignite.internal.processors.query.calcite.rel.IgniteTrimExchange.(IgniteTrimExchange.java:49)
> at SC.apply(Unknown Source)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJson$RelFactory.apply(RelJson.java:115)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.readRel(RelJsonReader.java:111)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.readRels(RelJsonReader.java:103)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.read(RelJsonReader.java:92)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.fromJson(RelJsonReader.java:77)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareFragment(ExecutionServiceImpl.java:522)
> at
> org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:84)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.onMessage(ExecutionServiceImpl.java:860)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.lambda$init$1(ExecutionServiceImpl.java:435)
> at
> org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.onMessageInternal(MessageServiceImpl.java:276)
> at
> org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.lambda$onMessage$0(MessageServiceImpl.java:256)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> {noformat}
> appropriate plan:
> {noformat}
> IgniteReduceAggregate(rowType=[RecordType(JavaType(class java.lang.String)
> NAME)], group=[{0}]): rowcount = 110.0, cumulative cost = 6140.0, id = 244
> IgniteExchange(distribution=[single]): rowcount = 110.0, cumulative cost =
> 6030.0, id = 243
> IgniteMapAggregate(group=[{0}]): rowcount = 110.0, cumulative cost =
> 4710.0, id = 242
> IgniteUnionAll(all=[true]): rowcount = 1100.0, cumulative cost =
> 4600.0, id = 241
> IgniteTrimExchange(distribution=[hash[0]]): rowcount = 100.0,
> cumulative cost = 2500.0, id = 240
> IgniteReduceAggregate(rowType=[RecordType(JavaType(class
> java.lang.String) NAME)], group=[{0}]): rowcount = 100.0, cumulative cost =
> 2400.0, id = 239
> IgniteExchange(distribution=[broadcast]): rowcount = 100.0,
> cumulative cost = 2300.0, id = 238
> IgniteMapAggregate(group=[{0}]): rowcount = 100.0, cumulative
> cost = 1100.0, id = 237
> IgniteTableScan(table=[[PUBLIC, ORDERS]],
> requiredColunms=[{2}]): rowcount = 1000.0, cumulative cost = 1000.0, id = 65
> IgniteTableScan(table=[[PUBLIC, ACCOUNT]], requiredColunms=[{2}]):
> rowcount = 1000.0, cumulative cost = 1000.0, id = 97
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (IGNITE-13727) Calcite bug. Assertion on UNION with DISTINCT.
[
https://issues.apache.org/jira/browse/IGNITE-13727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Konstantin Orlov reassigned IGNITE-13727:
-
Assignee: Konstantin Orlov
> Calcite bug. Assertion on UNION with DISTINCT.
> --
>
> Key: IGNITE-13727
> URL: https://issues.apache.org/jira/browse/IGNITE-13727
> Project: Ignite
> Issue Type: Bug
> Components: sql
>Affects Versions: 2.9
>Reporter: Stanilovsky Evgeny
>Assignee: Konstantin Orlov
>Priority: Major
> Labels: calcite
>
> {noformat}
> SELECT distinct(name) FROM Orders UNION SELECT name from Account{noformat}
> breaks Calcite with assertion:
> {noformat}
> java.lang.AssertionError: traits=IGNITE.[].any.one-way, distributionhash[0]
> at org.apache.calcite.rel.core.Exchange.(Exchange.java:62)
> at org.apache.calcite.rel.core.Exchange.(Exchange.java:71)
> at
> org.apache.ignite.internal.processors.query.calcite.rel.IgniteTrimExchange.(IgniteTrimExchange.java:49)
> at SC.apply(Unknown Source)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJson$RelFactory.apply(RelJson.java:115)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.readRel(RelJsonReader.java:111)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.readRels(RelJsonReader.java:103)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.read(RelJsonReader.java:92)
> at
> org.apache.ignite.internal.processors.query.calcite.externalize.RelJsonReader.fromJson(RelJsonReader.java:77)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.prepareFragment(ExecutionServiceImpl.java:522)
> at
> org.apache.ignite.internal.processors.query.calcite.prepare.QueryPlanCacheImpl.queryPlan(QueryPlanCacheImpl.java:84)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.onMessage(ExecutionServiceImpl.java:860)
> at
> org.apache.ignite.internal.processors.query.calcite.exec.ExecutionServiceImpl.lambda$init$1(ExecutionServiceImpl.java:435)
> at
> org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.onMessageInternal(MessageServiceImpl.java:276)
> at
> org.apache.ignite.internal.processors.query.calcite.message.MessageServiceImpl.lambda$onMessage$0(MessageServiceImpl.java:256)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> {noformat}
> appropriate plan:
> {noformat}
> IgniteReduceAggregate(rowType=[RecordType(JavaType(class java.lang.String)
> NAME)], group=[{0}]): rowcount = 110.0, cumulative cost = 6140.0, id = 244
> IgniteExchange(distribution=[single]): rowcount = 110.0, cumulative cost =
> 6030.0, id = 243
> IgniteMapAggregate(group=[{0}]): rowcount = 110.0, cumulative cost =
> 4710.0, id = 242
> IgniteUnionAll(all=[true]): rowcount = 1100.0, cumulative cost =
> 4600.0, id = 241
> IgniteTrimExchange(distribution=[hash[0]]): rowcount = 100.0,
> cumulative cost = 2500.0, id = 240
> IgniteReduceAggregate(rowType=[RecordType(JavaType(class
> java.lang.String) NAME)], group=[{0}]): rowcount = 100.0, cumulative cost =
> 2400.0, id = 239
> IgniteExchange(distribution=[broadcast]): rowcount = 100.0,
> cumulative cost = 2300.0, id = 238
> IgniteMapAggregate(group=[{0}]): rowcount = 100.0, cumulative
> cost = 1100.0, id = 237
> IgniteTableScan(table=[[PUBLIC, ORDERS]],
> requiredColunms=[{2}]): rowcount = 1000.0, cumulative cost = 1000.0, id = 65
> IgniteTableScan(table=[[PUBLIC, ACCOUNT]], requiredColunms=[{2}]):
> rowcount = 1000.0, cumulative cost = 1000.0, id = 97
> {noformat}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13860) Provide the tool to build cluster performance report
[ https://issues.apache.org/jira/browse/IGNITE-13860?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250146#comment-17250146 ] Amelchev Nikita commented on IGNITE-13860: -- The TC run with new test suite: https://ci.ignite.apache.org/buildConfiguration/Tests_IgniteExtensions_Build/5792230 > Provide the tool to build cluster performance report > > > Key: IGNITE-13860 > URL: https://issues.apache.org/jira/browse/IGNITE-13860 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Fix For: 2.10 > > > Provide the tool to build cluster performance report -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12666) Provide cluster performance profiling tool
[ https://issues.apache.org/jira/browse/IGNITE-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17250127#comment-17250127 ] Amelchev Nikita commented on IGNITE-12666: -- [~nizhikov] Thank you for the review! I have created the ticket for the report part: https://issues.apache.org/jira/browse/IGNITE-13860 > Provide cluster performance profiling tool > -- > > Key: IGNITE-12666 > URL: https://issues.apache.org/jira/browse/IGNITE-12666 > Project: Ignite > Issue Type: New Feature >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Critical > Labels: IEP-35, important > Fix For: 2.10 > > Attachments: Tracing benchmarks.docx, benchmark_get_put.zip > > Time Spent: 29h 20m > Remaining Estimate: 0h > > For now, Ignite has not build-in profiling tool for user's operations and > internal processes. Such a tool will be able to collect performance > statistics and create a human-readable report. It will help to analyze > workload and to tune configuration and applications. > Example of similar tools in other products: AWR > [[1]|https://docs.oracle.com/cd/E11882_01/server.112/e41573/autostat.htm#PFGRF94176] > [[2]|http://www.dba-oracle.com/t_sample_awr_report.htm] > [[3]|http://expertoracle.com/2018/02/06/performance-tuning-basics-15-awr-report-analysis/] > (Oracle) ; pgbadger [[4]|https://github.com/darold/pgbadger], pgmetrics > [[5]|https://pgmetrics.io/docs/index.html#example], powa > [[6]|https://powa.readthedocs.io/en/latest/] (PostgresSQL). Example of html > report: [powa|https://powa.readthedocs.io/en/latest/]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13861) Add logical/physical reads checks to existing tests
Amelchev Nikita created IGNITE-13861: Summary: Add logical/physical reads checks to existing tests Key: IGNITE-13861 URL: https://issues.apache.org/jira/browse/IGNITE-13861 Project: Ignite Issue Type: Sub-task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Add logical/physical reads checks to existing tests -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13860) Provide the tool to build cluster performance report
Amelchev Nikita created IGNITE-13860: Summary: Provide the tool to build cluster performance report Key: IGNITE-13860 URL: https://issues.apache.org/jira/browse/IGNITE-13860 Project: Ignite Issue Type: Sub-task Reporter: Amelchev Nikita Assignee: Amelchev Nikita Provide the tool to build cluster performance report -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12666) Provide cluster performance profiling tool
[ https://issues.apache.org/jira/browse/IGNITE-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249831#comment-17249831 ] Nikolay Izhikov commented on IGNITE-12666: -- [~NSAmelchev] Let's move PR with the report into a separate ticket to preserve one ticket one commit strategy. > Provide cluster performance profiling tool > -- > > Key: IGNITE-12666 > URL: https://issues.apache.org/jira/browse/IGNITE-12666 > Project: Ignite > Issue Type: New Feature >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Critical > Labels: IEP-35, important > Fix For: 2.10 > > Attachments: Tracing benchmarks.docx, benchmark_get_put.zip > > Time Spent: 29h 20m > Remaining Estimate: 0h > > For now, Ignite has not build-in profiling tool for user's operations and > internal processes. Such a tool will be able to collect performance > statistics and create a human-readable report. It will help to analyze > workload and to tune configuration and applications. > Example of similar tools in other products: AWR > [[1]|https://docs.oracle.com/cd/E11882_01/server.112/e41573/autostat.htm#PFGRF94176] > [[2]|http://www.dba-oracle.com/t_sample_awr_report.htm] > [[3]|http://expertoracle.com/2018/02/06/performance-tuning-basics-15-awr-report-analysis/] > (Oracle) ; pgbadger [[4]|https://github.com/darold/pgbadger], pgmetrics > [[5]|https://pgmetrics.io/docs/index.html#example], powa > [[6]|https://powa.readthedocs.io/en/latest/] (PostgresSQL). Example of html > report: [powa|https://powa.readthedocs.io/en/latest/]. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13859) .NET: Build scripts and instructions cleanup
[ https://issues.apache.org/jira/browse/IGNITE-13859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pavel Tupitsyn updated IGNITE-13859: Description: The one and only build script is *build.ps1*. * Remove *build-mono.sh* - Ignite does not work properly under Mono * Change *build.sh* to delegate to *build.ps1* same way as *build.bat* does * Update *build.ps1* to work by default on Linux/macOS: instead of looking for Mono msbuild, simply skip .NET Framework build part and print a warning along the lines of ".NET Core build succeeded. Full Ignite.NET build requires Windows and .NET Framework 4.x.". Using msbuild from Mono produces incorrect results anyway. * Update *build.ps1* to print a summary at the end, something like "Java build succeeded, .NET Framework build skipped, .NET Core build succeeded." * Update README files accordingly was: The one and only build script is *build.ps1*. * Remove *build-mono.sh* - Ignite does not work properly under Mono * Change *build.sh* to delegate to *build.ps1* same way as *build.bat* does * Update *build.ps1* to work by default on Linux/macOS: instead of looking for Mono msbuild, simply skip .NET Framework build part and print a warning along the lines of ".NET Core build succeeded. Full Ignite.NET build requires Windows and .NET Framework 4.x.". Using msbuild from Mono produces incorrect results anyway. * Update README files accordingly > .NET: Build scripts and instructions cleanup > > > Key: IGNITE-13859 > URL: https://issues.apache.org/jira/browse/IGNITE-13859 > Project: Ignite > Issue Type: Improvement > Components: platforms >Reporter: Pavel Tupitsyn >Assignee: Pavel Tupitsyn >Priority: Minor > Labels: .NET > > The one and only build script is *build.ps1*. > * Remove *build-mono.sh* - Ignite does not work properly under Mono > * Change *build.sh* to delegate to *build.ps1* same way as *build.bat* does > * Update *build.ps1* to work by default on Linux/macOS: instead of looking > for Mono msbuild, simply skip .NET Framework build part and print a warning > along the lines of ".NET Core build succeeded. Full Ignite.NET build requires > Windows and .NET Framework 4.x.". Using msbuild from Mono produces incorrect > results anyway. > * Update *build.ps1* to print a summary at the end, something like "Java > build succeeded, .NET Framework build skipped, .NET Core build succeeded." > * Update README files accordingly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13859) .NET: Build scripts and instructions cleanup
Pavel Tupitsyn created IGNITE-13859: --- Summary: .NET: Build scripts and instructions cleanup Key: IGNITE-13859 URL: https://issues.apache.org/jira/browse/IGNITE-13859 Project: Ignite Issue Type: Improvement Components: platforms Reporter: Pavel Tupitsyn Assignee: Pavel Tupitsyn The one and only build script is *build.ps1*. * Remove *build-mono.sh* - Ignite does not work properly under Mono * Change *build.sh* to delegate to *build.ps1* same way as *build.bat* does * Update *build.ps1* to work by default on Linux/macOS: instead of looking for Mono msbuild, simply skip .NET Framework build part and print a warning along the lines of ".NET Core build succeeded. Full Ignite.NET build requires Windows and .NET Framework 4.x.". Using msbuild from Mono produces incorrect results anyway. * Update README files accordingly -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12824) Interoperable Ignite.NET Dates
[
https://issues.apache.org/jira/browse/IGNITE-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249788#comment-17249788
]
Pavel Tupitsyn commented on IGNITE-12824:
-
[~nizhikov] on master branch add the following to {{BinaryDateTimeTest}}:
{code}
[Test]
public void TestDateTimeRoundtrip()
{
var cache = Ignition.GetIgnite().GetOrCreateCache(TestUtils.TestName);
var dt = DateTime.Now;
cache.Put(1, new DateTimeObj2 {Value = dt});
var dt2 = cache.Get(1).Value;
Assert.AreEqual(dt.ToUniversalTime(), dt2.ToUniversalTime());
}
{code}
Then remove the restriction from {{BinaryUtils.ToJavaDate}} and run the test.
> Interoperable Ignite.NET Dates
> --
>
> Key: IGNITE-12824
> URL: https://issues.apache.org/jira/browse/IGNITE-12824
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.8
>Reporter: Alexey Kukushkin
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: .NET, iep-54, ignite-3, sbcf
> Fix For: 3.0
>
> Attachments: IGNITE-12824-from-2.9.0.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *+The Problem+*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see [DateTime Serialization
> note|https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility].
> But then Ignite accepts only UTC dates, requiring the application developers
> to convert local dates to UTC dates and back. This task is not trivial:
> [DateTime.ToUniversalTime|https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1]
> uses calendars different from Java (and the .NET calendars are the invalid
> ones - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *+The Solution+*
> # Always write .NET dates as portable Ignite dates: get rid of the
> {{BinaryReflectiveSerializer.ForceTimestamp}} flag that currently triggers
> this behavior.
> Keep the {{ForceTimestamp}} flag if saving .NET dates as transparent objects
> seems a useful case.
> # Automatically convert Local dates to UTC and back *inside* Ignite.NET.
> Add a {{BinaryReflectiveSerializer.UtcDate flag}} to disable this
> conversion. This prevents loosing the {{DateTime.Kind}} property of UTC
> dates. {{UtcDate}} set to true implies the existing behavior: Ignite gets
> UTC dates and throws a "Date must be in UTC" exception on an attempt to write
> a Local date. The {{UtcDate}} flag is false by default.
> # Use [NodaTime|https://nodatime.org/] for UTC<->Local conversions. Noda
> time uses Java calendars making the conversion truly portable.
>
> *+The Benefits+*
> # Portable dates is a more frequent use-case than storing time zone info for
> every date in Ignite. This becomes default behavior and the developers do not
> need to always explicitly configure it.
> # Non-trivial code to make the truly portable UTC<->Local conversion is
> implemented once inside Ignite instead of having every Ignite.NET application
> implementing it.
> +*References*+
> * [Dev-List
> Discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Interoperable-Ignite-NET-Dates-td49946.html]
> * IEP-TBD
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12666) Provide cluster performance profiling tool
[
https://issues.apache.org/jira/browse/IGNITE-12666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249781#comment-17249781
]
Ignite TC Bot commented on IGNITE-12666:
{panel:title=Branch: [pull/7693/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/7693/head] Base: [master] : New Tests
(50)|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}
{color:#8b}Continuous Query 4{color} [[tests
28|https://ci.ignite.apache.org/viewLog.html?buildId=5791092]]
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testScanQuery[pageSize=100, clientType=SERVER] -
PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testDdlAndDmlQueries[pageSize=100,
clientType=SERVER] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testSqlFieldsQuery[pageSize=100,
clientType=SERVER] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testMultipleStatementsSql[pageSize=100,
clientType=SERVER] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testSqlFieldsJoinQuery[pageSize=10,
clientType=THIN_CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testSqlFieldsQuery[pageSize=10,
clientType=THIN_CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testScanQuery[pageSize=10, clientType=CLIENT] -
PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testDdlAndDmlQueries[pageSize=10,
clientType=CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testSqlFieldsQuery[pageSize=10,
clientType=CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testMultipleStatementsSql[pageSize=10,
clientType=CLIENT] - PASSED{color}
* {color:#013220}IgniteCacheQuerySelfTestSuite6:
PerformanceStatisticsQueryTest.testDdlAndDmlQueries[pageSize=10,
clientType=THIN_CLIENT] - PASSED{color}
... and 17 new tests
{color:#8b}Basic 3{color} [[tests
22|https://ci.ignite.apache.org/viewLog.html?buildId=5791067]]
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsMultipleStartTest.testStartCreateNewFile - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
ForwardReadTest.testStringForwardRead[unknownStrs=true] - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
ForwardReadTest.testStringForwardRead[unknownStrs=false] - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
StringCacheTest.testCacheTaskName - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsPropertiesTest.testFlushSize - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsSelfTest.testTransaction[clientType=CLIENT] - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsPropertiesTest.testFileMaxSize - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsPropertiesTest.testCachedStringsThreshold - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsSelfTest.testCacheOperation[clientType=SERVER] -
PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsSelfTest.testTransaction[clientType=SERVER] - PASSED{color}
* {color:#013220}IgniteBasicWithPersistenceTestSuite:
PerformanceStatisticsSelfTest.testCompute[clientType=SERVER] - PASSED{color}
... and 11 new tests
{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5791126buildTypeId=IgniteTests24Java8_RunAll]
> Provide cluster performance profiling tool
> --
>
> Key: IGNITE-12666
> URL: https://issues.apache.org/jira/browse/IGNITE-12666
> Project: Ignite
> Issue Type: New Feature
>Reporter: Amelchev Nikita
>Assignee: Amelchev Nikita
>Priority: Critical
> Labels: IEP-35, important
> Fix For: 2.10
>
> Attachments: Tracing benchmarks.docx, benchmark_get_put.zip
>
> Time Spent: 29h 10m
> Remaining Estimate: 0h
>
> For now, Ignite has not build-in profiling tool for user's operations and
> internal processes. Such a tool will be able to collect performance
> statistics and create a human-readable report. It will help to analyze
> workload and to tune configuration and applications.
> Example of similar tools in other products: AWR
> [[1]|https://docs.oracle.com/cd/E11882_01/server.112/e41573/autostat.htm#PFGRF94176]
>
[jira] [Updated] (IGNITE-13854) Add documentation for the cluster performance profiling tool
[ https://issues.apache.org/jira/browse/IGNITE-13854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-13854: - Labels: IEP-35 (was: ) > Add documentation for the cluster performance profiling tool > > > Key: IGNITE-13854 > URL: https://issues.apache.org/jira/browse/IGNITE-13854 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Labels: IEP-35 > Fix For: 2.10 > > > Add documentation for the cluster performance profiling tool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (IGNITE-13854) Add documentation for the cluster performance profiling tool
[ https://issues.apache.org/jira/browse/IGNITE-13854?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nikolay Izhikov updated IGNITE-13854: - Fix Version/s: 2.10 > Add documentation for the cluster performance profiling tool > > > Key: IGNITE-13854 > URL: https://issues.apache.org/jira/browse/IGNITE-13854 > Project: Ignite > Issue Type: Sub-task >Reporter: Amelchev Nikita >Assignee: Amelchev Nikita >Priority: Major > Fix For: 2.10 > > > Add documentation for the cluster performance profiling tool. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (IGNITE-13734) .NET Service loses returned array type information
[
https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nikolay Izhikov reassigned IGNITE-13734:
Assignee: (was: Nikolai Kulagin)
> .NET Service loses returned array type information
> --
>
> Key: IGNITE-13734
> URL: https://issues.apache.org/jira/browse/IGNITE-13734
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.9
>Reporter: Alexey Kukushkin
>Priority: Major
>
> .NET service client receives Object[] instead of strongly typed array from a
> .NET service.
> There was another already resolved similar issue IGNITE-12823 that addressed
> the problem of using arrays as parameters. The problem of using arrays as
> results still exists.
> h3. Reproducer
> A .NET service returning an array of user-defined types is deployed:
> {code:c#}
> public interface ITestService
> {
> Parameter[] TestReturnParametersArray();
> }
> public sealed class Parameter
> {
> public int Id { get; set; }
> public int[] Values { get; set; }
> }
> {code}
> A .NET client calls the service:
> {code:c#}
> Parameter[] res = svcProxy.TestReturnParametersArray()
> {code}
> The service call fails with exception:
> {code}
> System.InvalidCastException : Unable to cast object of type 'System.Object[]'
> to type 'Parameter[]'.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13734) .NET Service loses returned array type information
[
https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249777#comment-17249777
]
Pavel Tupitsyn commented on IGNITE-13734:
-
[~YAMolochkov] the problem here is similar to what we fixed on Java side in
IGNITE-12823. Losing array type information is caused by limitations of Ignite
binary format.
Replacing returned array with reflection is what we do on Java side, see
{{PlatformServices#convertArrayArgs}}. This is an easier approach, and I think
the performance hit would not be very noticeable compared to the overall
service call cost.
Alternatively, we can implement a more efficient but slightly more complex
approach:
* Pass expected result type to {{ServiceProxySerializer.ReadInvocationResult}}
* Add non-generic {{BinaryReader.ReadArray}} overload that takes element type
and uses {{Array.CreateInstance}}
> .NET Service loses returned array type information
> --
>
> Key: IGNITE-13734
> URL: https://issues.apache.org/jira/browse/IGNITE-13734
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.9
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
>
> .NET service client receives Object[] instead of strongly typed array from a
> .NET service.
> There was another already resolved similar issue IGNITE-12823 that addressed
> the problem of using arrays as parameters. The problem of using arrays as
> results still exists.
> h3. Reproducer
> A .NET service returning an array of user-defined types is deployed:
> {code:c#}
> public interface ITestService
> {
> Parameter[] TestReturnParametersArray();
> }
> public sealed class Parameter
> {
> public int Id { get; set; }
> public int[] Values { get; set; }
> }
> {code}
> A .NET client calls the service:
> {code:c#}
> Parameter[] res = svcProxy.TestReturnParametersArray()
> {code}
> The service call fails with exception:
> {code}
> System.InvalidCastException : Unable to cast object of type 'System.Object[]'
> to type 'Parameter[]'.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13858) Enabling near cache on a client node may lead to blocking eviction on server nodes
Vyacheslav Koptilin created IGNITE-13858: Summary: Enabling near cache on a client node may lead to blocking eviction on server nodes Key: IGNITE-13858 URL: https://issues.apache.org/jira/browse/IGNITE-13858 Project: Ignite Issue Type: Bug Affects Versions: 2.9, 2.8, 2.9.1 Reporter: Vyacheslav Koptilin When Native Persistence is off, creating near-cache on client node enforces (to be more accurate, near-cache entries) to maintain a list client node ids (aka readers) per-entry basis. Unfortunately, this list is not cleaned even though the eviction policy is configured on the near cache. Moreover, such entries cannot be evicted on the server-side, with corresponding pages, and this may lead to hanging cache operations. Originally discussed at user-list: http://apache-ignite-users.70518.x6.nabble.com/Apache-Ignite-Clientside-NearCache-and-Serverside-Eviction-blocking-cluster-tt34645.html -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13857) Update Ignite-3.0 java to 11 version
[ https://issues.apache.org/jira/browse/IGNITE-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249769#comment-17249769 ] Andrey N. Gura commented on IGNITE-13857: - [~kgusakov] LGTM. Thanks for contribution! Merged to main brainch. > Update Ignite-3.0 java to 11 version > > > Key: IGNITE-13857 > URL: https://issues.apache.org/jira/browse/IGNITE-13857 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > According to discussion > [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Java-11-for-Ignite-3-0-development-td50534.html] > we should update java version in ignite-3 repo to 11 version > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-12824) Interoperable Ignite.NET Dates
[
https://issues.apache.org/jira/browse/IGNITE-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249733#comment-17249733
]
Nikolay Izhikov commented on IGNITE-12824:
--
[~ptupitsyn] Can you, please, give me some examples of bug with the data
conversion?
> Interoperable Ignite.NET Dates
> --
>
> Key: IGNITE-12824
> URL: https://issues.apache.org/jira/browse/IGNITE-12824
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.8
>Reporter: Alexey Kukushkin
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: .NET, iep-54, ignite-3, sbcf
> Fix For: 3.0
>
> Attachments: IGNITE-12824-from-2.9.0.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *+The Problem+*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see [DateTime Serialization
> note|https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility].
> But then Ignite accepts only UTC dates, requiring the application developers
> to convert local dates to UTC dates and back. This task is not trivial:
> [DateTime.ToUniversalTime|https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1]
> uses calendars different from Java (and the .NET calendars are the invalid
> ones - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *+The Solution+*
> # Always write .NET dates as portable Ignite dates: get rid of the
> {{BinaryReflectiveSerializer.ForceTimestamp}} flag that currently triggers
> this behavior.
> Keep the {{ForceTimestamp}} flag if saving .NET dates as transparent objects
> seems a useful case.
> # Automatically convert Local dates to UTC and back *inside* Ignite.NET.
> Add a {{BinaryReflectiveSerializer.UtcDate flag}} to disable this
> conversion. This prevents loosing the {{DateTime.Kind}} property of UTC
> dates. {{UtcDate}} set to true implies the existing behavior: Ignite gets
> UTC dates and throws a "Date must be in UTC" exception on an attempt to write
> a Local date. The {{UtcDate}} flag is false by default.
> # Use [NodaTime|https://nodatime.org/] for UTC<->Local conversions. Noda
> time uses Java calendars making the conversion truly portable.
>
> *+The Benefits+*
> # Portable dates is a more frequent use-case than storing time zone info for
> every date in Ignite. This becomes default behavior and the developers do not
> need to always explicitly configure it.
> # Non-trivial code to make the truly portable UTC<->Local conversion is
> implemented once inside Ignite instead of having every Ignite.NET application
> implementing it.
> +*References*+
> * [Dev-List
> Discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Interoperable-Ignite-NET-Dates-td49946.html]
> * IEP-TBD
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-12824) Interoperable Ignite.NET Dates
[
https://issues.apache.org/jira/browse/IGNITE-12824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249727#comment-17249727
]
Pavel Tupitsyn commented on IGNITE-12824:
-
[~nizhikov] it is not correct to remove this restriction.
The value should be the same after it passes through Ignite (serialize ->
deserialize).
Without the restriction users would get unexpected bugs where local DateTime
becomes UTC.
> Interoperable Ignite.NET Dates
> --
>
> Key: IGNITE-12824
> URL: https://issues.apache.org/jira/browse/IGNITE-12824
> Project: Ignite
> Issue Type: Improvement
> Components: platforms
>Affects Versions: 2.8
>Reporter: Alexey Kukushkin
>Assignee: Nikolay Izhikov
>Priority: Major
> Labels: .NET, iep-54, ignite-3, sbcf
> Fix For: 3.0
>
> Attachments: IGNITE-12824-from-2.9.0.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> *+The Problem+*
> Presently .NET API writes dates as composite Ignite objects. Only .NET
> clients can read such dates: any other client (JDBC, Java, etc) does not
> understand it without custom deserialization.
>
> It is still possible to configure .NET serialization to write dates as
> Ignite dates - see [DateTime Serialization
> note|https://ignite.apache.org/docs/latest/net-specific/net-platform-interoperability#types-compatibility].
> But then Ignite accepts only UTC dates, requiring the application developers
> to convert local dates to UTC dates and back. This task is not trivial:
> [DateTime.ToUniversalTime|https://docs.microsoft.com/en-us/dotnet/api/system.datetime.touniversaltime?view=netcore-3.1]
> uses calendars different from Java (and the .NET calendars are the invalid
> ones - especially for pre-1990 dates).
>
> The motivation for the current default behavior was probably the desire to
> keep the Time Zone information: Ignite dates do not store time zones.
>
> In our experience interoperability is more important than storing time zone
> info.
>
> *+The Solution+*
> # Always write .NET dates as portable Ignite dates: get rid of the
> {{BinaryReflectiveSerializer.ForceTimestamp}} flag that currently triggers
> this behavior.
> Keep the {{ForceTimestamp}} flag if saving .NET dates as transparent objects
> seems a useful case.
> # Automatically convert Local dates to UTC and back *inside* Ignite.NET.
> Add a {{BinaryReflectiveSerializer.UtcDate flag}} to disable this
> conversion. This prevents loosing the {{DateTime.Kind}} property of UTC
> dates. {{UtcDate}} set to true implies the existing behavior: Ignite gets
> UTC dates and throws a "Date must be in UTC" exception on an attempt to write
> a Local date. The {{UtcDate}} flag is false by default.
> # Use [NodaTime|https://nodatime.org/] for UTC<->Local conversions. Noda
> time uses Java calendars making the conversion truly portable.
>
> *+The Benefits+*
> # Portable dates is a more frequent use-case than storing time zone info for
> every date in Ignite. This becomes default behavior and the developers do not
> need to always explicitly configure it.
> # Non-trivial code to make the truly portable UTC<->Local conversion is
> implemented once inside Ignite instead of having every Ignite.NET application
> implementing it.
> +*References*+
> * [Dev-List
> Discussion|http://apache-ignite-developers.2346864.n4.nabble.com/Interoperable-Ignite-NET-Dates-td49946.html]
> * IEP-TBD
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13734) .NET Service loses returned array type information
[
https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249718#comment-17249718
]
Yaroslav Molochkov edited comment on IGNITE-13734 at 12/15/20, 2:35 PM:
[~nizhikov], [~ptupitsyn],
there's one idea: is it worth the effort to modify returned array from a
reflection call? We have to create one and then move each element to the newly
created array. That's double the memory and O\(n\) complexity. And not every
client code needs that cast. And then, it's not really consistent if client
code will receive casted array and, for example, not parametrised list.
was (Author: yamolochkov):
[~nizhikov], [~ptupitsyn],
there's one idea: is it worth the effort to modify returned array from a
reflection call? We have to create one and then move each element to the newly
created array. That's double the memory and O\(n\) complexity. And not every
client code need that cast. And then, it's not really consistent if client code
will receive casted array and, for example, not parametrised list.
> .NET Service loses returned array type information
> --
>
> Key: IGNITE-13734
> URL: https://issues.apache.org/jira/browse/IGNITE-13734
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.9
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
>
> .NET service client receives Object[] instead of strongly typed array from a
> .NET service.
> There was another already resolved similar issue IGNITE-12823 that addressed
> the problem of using arrays as parameters. The problem of using arrays as
> results still exists.
> h3. Reproducer
> A .NET service returning an array of user-defined types is deployed:
> {code:c#}
> public interface ITestService
> {
> Parameter[] TestReturnParametersArray();
> }
> public sealed class Parameter
> {
> public int Id { get; set; }
> public int[] Values { get; set; }
> }
> {code}
> A .NET client calls the service:
> {code:c#}
> Parameter[] res = svcProxy.TestReturnParametersArray()
> {code}
> The service call fails with exception:
> {code}
> System.InvalidCastException : Unable to cast object of type 'System.Object[]'
> to type 'Parameter[]'.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13734) .NET Service loses returned array type information
[
https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249718#comment-17249718
]
Yaroslav Molochkov edited comment on IGNITE-13734 at 12/15/20, 2:25 PM:
[~nizhikov], [~ptupitsyn],
there's one idea: is it worth the effort to modify returned array from a
reflection call? We have to create one and then move each element to the newly
created array. That's double the memory and O\(n\) complexity. And not every
client code need that cast. And then, it's not really consistent if client code
will receive casted array and, for example, not parametrised list.
was (Author: yamolochkov):
[~nizhikov], [~ptupitsyn],
there's one idea: is it worth the effort to modify returned array from a
reflection call? We have to create one and then move each element to the newly
created array. That's double the memory and O(n) complexity. And not every
client code need that cast. And then, it's not really consistent if client code
will receive casted array and, for example, not parametrised list.
> .NET Service loses returned array type information
> --
>
> Key: IGNITE-13734
> URL: https://issues.apache.org/jira/browse/IGNITE-13734
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.9
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
>
> .NET service client receives Object[] instead of strongly typed array from a
> .NET service.
> There was another already resolved similar issue IGNITE-12823 that addressed
> the problem of using arrays as parameters. The problem of using arrays as
> results still exists.
> h3. Reproducer
> A .NET service returning an array of user-defined types is deployed:
> {code:c#}
> public interface ITestService
> {
> Parameter[] TestReturnParametersArray();
> }
> public sealed class Parameter
> {
> public int Id { get; set; }
> public int[] Values { get; set; }
> }
> {code}
> A .NET client calls the service:
> {code:c#}
> Parameter[] res = svcProxy.TestReturnParametersArray()
> {code}
> The service call fails with exception:
> {code}
> System.InvalidCastException : Unable to cast object of type 'System.Object[]'
> to type 'Parameter[]'.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13734) .NET Service loses returned array type information
[
https://issues.apache.org/jira/browse/IGNITE-13734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249718#comment-17249718
]
Yaroslav Molochkov commented on IGNITE-13734:
-
[~nizhikov], [~ptupitsyn],
there's one idea: is it worth the effort to modify returned array from a
reflection call? We have to create one and then move each element to the newly
created array. That's double the memory and O(n) complexity. And not every
client code need that cast. And then, it's not really consistent if client code
will receive casted array and, for example, not parametrised list.
> .NET Service loses returned array type information
> --
>
> Key: IGNITE-13734
> URL: https://issues.apache.org/jira/browse/IGNITE-13734
> Project: Ignite
> Issue Type: Bug
> Components: platforms
>Affects Versions: 2.9
>Reporter: Alexey Kukushkin
>Assignee: Nikolai Kulagin
>Priority: Major
>
> .NET service client receives Object[] instead of strongly typed array from a
> .NET service.
> There was another already resolved similar issue IGNITE-12823 that addressed
> the problem of using arrays as parameters. The problem of using arrays as
> results still exists.
> h3. Reproducer
> A .NET service returning an array of user-defined types is deployed:
> {code:c#}
> public interface ITestService
> {
> Parameter[] TestReturnParametersArray();
> }
> public sealed class Parameter
> {
> public int Id { get; set; }
> public int[] Values { get; set; }
> }
> {code}
> A .NET client calls the service:
> {code:c#}
> Parameter[] res = svcProxy.TestReturnParametersArray()
> {code}
> The service call fails with exception:
> {code}
> System.InvalidCastException : Unable to cast object of type 'System.Object[]'
> to type 'Parameter[]'.
> {code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13743) Defragmentation JMX API for schedule/cancel/status
[
https://issues.apache.org/jira/browse/IGNITE-13743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249708#comment-17249708
]
Ignite TC Bot commented on IGNITE-13743:
{panel:title=Branch: [pull/8496/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8496/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5789746buildTypeId=IgniteTests24Java8_RunAll]
> Defragmentation JMX API for schedule/cancel/status
> --
>
> Key: IGNITE-13743
> URL: https://issues.apache.org/jira/browse/IGNITE-13743
> Project: Ignite
> Issue Type: Sub-task
>Reporter: Ivan Bessonov
>Assignee: Semyon Danilov
>Priority: Major
> Labels: IEP-47
> Time Spent: 10m
> Remaining Estimate: 0h
>
> API for JMX bean as follows:
> *DefragmentationMXBean {*
> ** /** \{@code true} if possible to schedule given caches or false
> otherwise. */
> *boolean schedule(String... cacheName);*
>
> /** cancel the current defragmentation. */
> *boolean cancel();*
>
> /** Quantity of processed partitions. */
> *int* *processedPartitions**();*
>
> ** /** Total partitions for defragmentation */
> *int* *totalPartitions**();*
>
> /** Defragmentation start timestamp */
> *long startTime();*
>
> /** \{@code true} if defragmentation is in progress right now. */
> *boolean inProgress();*
> *}*
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (IGNITE-13857) Update Ignite-3.0 java to 11 version
[ https://issues.apache.org/jira/browse/IGNITE-13857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kirill Gusakov updated IGNITE-13857: Labels: ignite-3 (was: ) > Update Ignite-3.0 java to 11 version > > > Key: IGNITE-13857 > URL: https://issues.apache.org/jira/browse/IGNITE-13857 > Project: Ignite > Issue Type: Task >Reporter: Kirill Gusakov >Assignee: Kirill Gusakov >Priority: Major > Labels: ignite-3 > > According to discussion > [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Java-11-for-Ignite-3-0-development-td50534.html] > we should update java version in ignite-3 repo to 11 version > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13857) Update Ignite-3.0 java to 11 version
Kirill Gusakov created IGNITE-13857: --- Summary: Update Ignite-3.0 java to 11 version Key: IGNITE-13857 URL: https://issues.apache.org/jira/browse/IGNITE-13857 Project: Ignite Issue Type: Task Reporter: Kirill Gusakov Assignee: Kirill Gusakov According to discussion [http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Java-11-for-Ignite-3-0-development-td50534.html] we should update java version in ignite-3 repo to 11 version -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13785) Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for test purposes.
[ https://issues.apache.org/jira/browse/IGNITE-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249699#comment-17249699 ] Taras Ledkov edited comment on IGNITE-13785 at 12/15/20, 1:44 PM: -- Merged to [sql-calcite|https://github.com/apache/ignite/commit/c44ef65616793664f02c4782d2f800905a37] was (Author: tledkov-gridgain): Merged to [sql-calcite|https://github.com/apache/ignite/commit/23195e74cd1c4420b666ccb04886ae11c4d1bcc5] > Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for > test purposes. > --- > > Key: IGNITE-13785 > URL: https://issues.apache.org/jira/browse/IGNITE-13785 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.9 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Labels: calcite > Time Spent: 20m > Remaining Estimate: 0h > > For the test purposes we need to have a possibility to *create* and *insert* > using h2 engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13785) Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for test purposes.
[ https://issues.apache.org/jira/browse/IGNITE-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249699#comment-17249699 ] Taras Ledkov edited comment on IGNITE-13785 at 12/15/20, 1:39 PM: -- Merged to [sql-calcite|https://github.com/apache/ignite/commit/23195e74cd1c4420b666ccb04886ae11c4d1bcc5] was (Author: tledkov-gridgain): Merged to [sql-caclite|https://github.com/apache/ignite/commit/23195e74cd1c4420b666ccb04886ae11c4d1bcc5] > Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for > test purposes. > --- > > Key: IGNITE-13785 > URL: https://issues.apache.org/jira/browse/IGNITE-13785 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.9 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Labels: calcite > Time Spent: 20m > Remaining Estimate: 0h > > For the test purposes we need to have a possibility to *create* and *insert* > using h2 engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249661#comment-17249661 ] Kirill Tkalenko commented on IGNITE-13847: -- Hi [~akalashnikov], [~sk0x50] suggested you to review it too. Can you can code review? > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249661#comment-17249661 ] Kirill Tkalenko edited comment on IGNITE-13847 at 12/15/20, 12:30 PM: -- Hi [~akalashnikov], [~sk0x50] suggested you to review it too. Can you make code review? was (Author: ktkale...@gridgain.com): Hi [~akalashnikov], [~sk0x50] suggested you to review it too. Can you can code review? > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[ https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249651#comment-17249651 ] Vladislav Pyatkov commented on IGNITE-13847: LGTM > Make GridEncryptionManager#onWalSegmentRemoved async > > > Key: IGNITE-13847 > URL: https://issues.apache.org/jira/browse/IGNITE-13847 > Project: Ignite > Issue Type: Improvement > Components: persistence >Reporter: Kirill Tkalenko >Assignee: Kirill Tkalenko >Priority: Major > Labels: IEP-18 > Fix For: 2.10 > > Time Spent: 10m > Remaining Estimate: 0h > > When implementing IGNITE-13831 I was faced with deadlock. > When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive > since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after > deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved* > that wants to write to the metastore, but it will not succeed, since it will > wait for *FileWriteAheadLogManager#rollOver*. > I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous > in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13796) Update docs for the kubernetes module
[ https://issues.apache.org/jira/browse/IGNITE-13796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249652#comment-17249652 ] Maksim Timonin commented on IGNITE-13796: - [~dmagda] [~nsafonov] hi! Sorry for such long time to answer, email was gone to the SPAM folder :O I saw your comments in my PR, I'll fix them on this week. ThinClientKubernetesAddressFinder implements interface ClientAddressFinder that is a setting of ClientConfiguration only. It just helps client to find server addresses. It's not aimed to be used with IgniteConfiguration. > Update docs for the kubernetes module > - > > Key: IGNITE-13796 > URL: https://issues.apache.org/jira/browse/IGNITE-13796 > Project: Ignite > Issue Type: New Feature > Components: documentation >Affects Versions: 2.10 >Reporter: Maksim Timonin >Assignee: Nikita Safonov >Priority: Major > Time Spent: 20m > Remaining Estimate: 0h > > Docs related to the ticket IGNITE-13204. Before this change it was impossible > to rely on partition awareness feature [1] on thin clients within Kubernetes > environment. The reason was a Kubernetes cluster can scale dynamically but > thin client was able to configure only static list of addresses in moment of > start. The ticket fix it, provide opportunity to dynamically change list of > addresses known to thin clients. That ticket relates only to thin clients > started *within* Kubernetes cluster. Thin clients that are running outside of > Kubernetes cluster must be configured the same way as before (with static > list of addresses, usually it is an address of Ingress or other load > balancer), it's impossible to provide partition awareness feature for them as > there are no directly access to pods. > The ticket: > 1. Introduced class KubernetesConnectionConfiguration that represents common > settings to connect within Kubernetes cluster. > 2. There are 2 classes that use the configuration class to configure the > connection: > * TcpDiscoveryKubernetesIpFinder is responsible to configure Ignite nodes. > * ThinClientKubernetesAddressFinder is responsible to configure Thin clients > in case when partition awareness feature is enabled. > 3. Introduced ClientAddressFinder interface that provides dynamic list of > address to thin client. Generally, every user can implement this interface > and provide dynamic list of addresses in any environment. Ignite provides the > only implementation of this interface - ThinClientKubernetesAddressFinder > class, that works for Kubernetes environment. This interface should be useful > for any scaling environment. > > There was a refactoring of TcpDiscoveryKubernetesIpFinder to enable to use it > this configuration class, so some methods of the Finder are marked as > deprecated. > [1] > [https://ignite.apache.org/docs/latest/thin-clients/java-thin-client#partition-awareness] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13844) Calcite improvements. Append additional dependencies.
[ https://issues.apache.org/jira/browse/IGNITE-13844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249618#comment-17249618 ] Konstantin Orlov commented on IGNITE-13844: --- [~zstan], LGTM! [~tledkov-gridgain] , could you please help with merging? > Calcite improvements. Append additional dependencies. > - > > Key: IGNITE-13844 > URL: https://issues.apache.org/jira/browse/IGNITE-13844 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.9 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Labels: calcite > Time Spent: 10m > Remaining Estimate: 0h > > I found that calcite feature-branch started from _ignite.sh_ requires > additional dependencies: com.esri.geometry and org.javassist -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-13856) Superlinear performance of DirectByteBufferStreamImplV2.writeMessage(msg, writer)
Ilya Kazakov created IGNITE-13856:
-
Summary: Superlinear performance of
DirectByteBufferStreamImplV2.writeMessage(msg, writer)
Key: IGNITE-13856
URL: https://issues.apache.org/jira/browse/IGNITE-13856
Project: Ignite
Issue Type: Improvement
Components: binary
Affects Versions: 2.9
Reporter: Ilya Kazakov
Assignee: Ilya Kazakov
Attachments: LongStringSQL.java
{code:java}
@Override public void writeMessage(Message msg, MessageWriter writer) {
if (msg != null) {
if (buf.hasRemaining()) {
try {
writer.beforeInnerMessageWrite()
writer.setCurrentWriteClass(msg.getClass());
lastFinished = msg.writeTo(buf, writer);
}
finally {
writer.afterInnerMessageWrite(lastFinished);
}
}
}
}{code}
It is going to do multiple invocations of msg.writeTo(). If msg is
GridH2String, it will to val.getBytes() on every invocation of writeTo(),
leading to spiking of CPU and RAM usage.
We should change this module to make sure that all serialization happens only
once.
Reproducer is attached. If we increase string size in 10 times, then the
execution time increases more than 10 times.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (IGNITE-13855) Integration with micrometer.io as part of integration with spring-boot
Tanmay Ambre created IGNITE-13855:
-
Summary: Integration with micrometer.io as part of integration
with spring-boot
Key: IGNITE-13855
URL: https://issues.apache.org/jira/browse/IGNITE-13855
Project: Ignite
Issue Type: Wish
Components: general
Affects Versions: 2.9
Reporter: Tanmay Ambre
hi,
I run Ignite server nodes as Spring-boot applications. We use actuator,
micrometer and prometheus meter registries for monitoring.
I implemented a new MetricExporterSpi to push metrics as gauges in micrometer.
So that we can easily monitor the metrics using our prometheus-grafana
dashboards. However, the problem is the MetricExporterSpi never gets attached.
Reason for that is as follows:
# I use an xml file to configure ignite.
# After I do Ignition.start(configFilePath) - I perform
ignite.configuration().setMetricExporterSpi(). - however, after this call the
spiStart method of the MetricExporterSpi is not called.
# If I configure ExporterSpi in the ignite config xml file - the exporterSpi
lifecycle methods are invoked as part of Ignition.start(). But then I can't
inject the micrometer registry in my custom ExporterSpi.
# If i call the spi.start() manually - it doesnt work because the
setReadOnlyMetricRegistry is not invoked.
Any suggestions? I can use Opencensus - and then export metrics to Prometheus -
however - this would require me to start a HTTP Server inside ignite. This can
be become cumbersome when deployed in container mode (on K8S, Openshift, etc).
Code of my IgniteConfig is shown below
{quote}@Configuration
public class IgniteConfig
{
@Autowired
ApplicationConfig applicationConfig;
@Autowired
MicrometerMetricsExporterSpi mmes;
@Bean
public Ignite ignite()
{
Ignite igniteInstance =
Ignition.start(applicationConfig.getIgniteConfig().getConfigFile());
MetricExporterSpi[] metricSpis =
igniteInstance.configuration().getMetricExporterSpi();
igniteInstance.configuration().setMetricExporterSpi(mmes);
if (applicationConfig.getIgniteConfig().isAutoActivateCluster())
{
igniteInstance.cluster().state(ClusterState.ACTIVE);
}
return igniteInstance;
}
}{quote}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (IGNITE-13785) Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for test purposes.
[ https://issues.apache.org/jira/browse/IGNITE-13785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249591#comment-17249591 ] Taras Ledkov commented on IGNITE-13785: --- [~zstan], my comment: Why we force usage new query engine without any flags / conditions? > Calcite improvements. Support CREATE TABLE and INSERT through h2 engine, for > test purposes. > --- > > Key: IGNITE-13785 > URL: https://issues.apache.org/jira/browse/IGNITE-13785 > Project: Ignite > Issue Type: Improvement > Components: sql >Affects Versions: 2.9 >Reporter: Stanilovsky Evgeny >Assignee: Stanilovsky Evgeny >Priority: Major > Labels: calcite > Time Spent: 20m > Remaining Estimate: 0h > > For the test purposes we need to have a possibility to *create* and *insert* > using h2 engine. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (IGNITE-13847) Make GridEncryptionManager#onWalSegmentRemoved async
[
https://issues.apache.org/jira/browse/IGNITE-13847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17249566#comment-17249566
]
Ignite TC Bot commented on IGNITE-13847:
{panel:title=Branch: [pull/8576/head] Base: [master] : No blockers
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#D6F7C1}{panel}
{panel:title=Branch: [pull/8576/head] Base: [master] : No new tests
found!|borderStyle=dashed|borderColor=#ccc|titleBGColor=#F7D6C1}{panel}
[TeamCity *-- Run :: All*
Results|https://ci.ignite.apache.org/viewLog.html?buildId=5791020buildTypeId=IgniteTests24Java8_RunAll]
> Make GridEncryptionManager#onWalSegmentRemoved async
>
>
> Key: IGNITE-13847
> URL: https://issues.apache.org/jira/browse/IGNITE-13847
> Project: Ignite
> Issue Type: Improvement
> Components: persistence
>Reporter: Kirill Tkalenko
>Assignee: Kirill Tkalenko
>Priority: Major
> Labels: IEP-18
> Fix For: 2.10
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> When implementing IGNITE-13831 I was faced with deadlock.
> When execute *FileWriteAheadLogManager#rollOver*, begin to clean WAL archive
> since we have reached the *DataStorageConfiguration#maxWalArchiveSize*, after
> deleting a segment, execute the *GridEncryptionManager#onWalSegmentRemoved*
> that wants to write to the metastore, but it will not succeed, since it will
> wait for *FileWriteAheadLogManager#rollOver*.
> I suggest making the *GridEncryptionManager#onWalSegmentRemoved* asynchronous
> in a separate pool, for example, as a *CacheGroupPageScanner#singleExecSvc*.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
