[jira] [Created] (IGNITE-12187) [IEP-35] Move GridMetricManager and dependent classes to org.apache.ignite.internal.managers
Nikolay Izhikov created IGNITE-12187: Summary: [IEP-35] Move GridMetricManager and dependent classes to org.apache.ignite.internal.managers Key: IGNITE-12187 URL: https://issues.apache.org/jira/browse/IGNITE-12187 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Fix For: 2.8 Right now metric internal classes are in {{org.apache.ignite.internal.processors.metric}} package. Seems, we should move it to {{org.apache.ignite.internal.managers.metric}} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (IGNITE-12145) [IEP-35] Monitoring list engine
Nikolay Izhikov created IGNITE-12145: Summary: [IEP-35] Monitoring list engine Key: IGNITE-12145 URL: https://issues.apache.org/jira/browse/IGNITE-12145 Project: Ignite Issue Type: Sub-task Affects Versions: 2.7.5 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.8 The base engine for the monitoring list. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Created] (IGNITE-11987) [IEP-35] Add ability to configure metrics
Nikolay Izhikov created IGNITE-11987: Summary: [IEP-35] Add ability to configure metrics Key: IGNITE-11987 URL: https://issues.apache.org/jira/browse/IGNITE-11987 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Ignite should be able to: * Enable or disable an arbitrary subset of the metrics. User should be able to do it in runtime. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11981) [IEP-35] Create MU shortcut instead of static import of MetricUtils
Nikolay Izhikov created IGNITE-11981: Summary: [IEP-35] Create MU shortcut instead of static import of MetricUtils Key: IGNITE-11981 URL: https://issues.apache.org/jira/browse/IGNITE-11981 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov More Ignite-way of coding is the usage of short cut classes. We should use MU instead of static import of {{MetricUtils}}. -- This message was sent by Atlassian JIRA (v7.6.14#76016)
[jira] [Created] (IGNITE-11960) Throw on attemp to create existing metric
Nikolay Izhikov created IGNITE-11960: Summary: Throw on attemp to create existing metric Key: IGNITE-11960 URL: https://issues.apache.org/jira/browse/IGNITE-11960 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Currently, `MetricRegistry` returns an existing metric instance on a second creation attempt. We should add assert and throw in such cases. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11945) [IEP-35] Performance drop on cache stop
Nikolay Izhikov created IGNITE-11945: Summary: [IEP-35] Performance drop on cache stop Key: IGNITE-11945 URL: https://issues.apache.org/jira/browse/IGNITE-11945 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov `MetricRegistry` implementation drop performance on cache stops. Has to be fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11944) [IEP-35] OpencensusExporter should export Histogram metrics
Nikolay Izhikov created IGNITE-11944: Summary: [IEP-35] OpencensusExporter should export Histogram metrics Key: IGNITE-11944 URL: https://issues.apache.org/jira/browse/IGNITE-11944 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov For now, OpenCensusMetricExporter doesn't export HistogramMetric. Ignite should support export of this type of metrics. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11941) [IEP-35] Rewrite GridLocalMetrics using new framework
Nikolay Izhikov created IGNITE-11941: Summary: [IEP-35] Rewrite GridLocalMetrics using new framework Key: IGNITE-11941 URL: https://issues.apache.org/jira/browse/IGNITE-11941 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov 1. GridLocalMetrics should be moved to GridMetricManager 2. Standard JVM JMX beans should be registered as metrics on startup. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11927) [IEP-35] Add ability to disable subset of metrics
Nikolay Izhikov created IGNITE-11927: Summary: [IEP-35] Add ability to disable subset of metrics Key: IGNITE-11927 URL: https://issues.apache.org/jira/browse/IGNITE-11927 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Ignite should be able to enable or disable an arbitrary subset of the metrics. User should be able to do it in runtime. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11926) [IEP-35] Migrage GridJobMetricsProcessor
Nikolay Izhikov created IGNITE-11926: Summary: [IEP-35] Migrage GridJobMetricsProcessor Key: IGNITE-11926 URL: https://issues.apache.org/jira/browse/IGNITE-11926 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `GridJobMetricsProcessor` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11925) [IEP-35] Migrage QueryMetrics
Nikolay Izhikov created IGNITE-11925: Summary: [IEP-35] Migrage QueryMetrics Key: IGNITE-11925 URL: https://issues.apache.org/jira/browse/IGNITE-11925 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `QueryMetrics` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11923) [IEP-35] Migrate IgniteMXBean
Nikolay Izhikov created IGNITE-11923: Summary: [IEP-35] Migrate IgniteMXBean Key: IGNITE-11923 URL: https://issues.apache.org/jira/browse/IGNITE-11923 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `IgniteMXBean` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11924) [IEP-35] Migrate TransactionMetricsMxBean
Nikolay Izhikov created IGNITE-11924: Summary: [IEP-35] Migrate TransactionMetricsMxBean Key: IGNITE-11924 URL: https://issues.apache.org/jira/browse/IGNITE-11924 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `TransactionMetricsMxBean` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11922) [IEP-35] Migrate ClusterMetricsMxBean
Nikolay Izhikov created IGNITE-11922: Summary: [IEP-35] Migrate ClusterMetricsMxBean Key: IGNITE-11922 URL: https://issues.apache.org/jira/browse/IGNITE-11922 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `ClusterMetricsMxBean` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11921) [IEP-35] Migrage CacheGroupMetrics
Nikolay Izhikov created IGNITE-11921: Summary: [IEP-35] Migrage CacheGroupMetrics Key: IGNITE-11921 URL: https://issues.apache.org/jira/browse/IGNITE-11921 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov After merging of IGNITE-11848 we should migrate `CacheGroupMetricsMXBean` to the new metric framework. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11920) [IEP-35] Improve Metric API
Nikolay Izhikov created IGNITE-11920: Summary: [IEP-35] Improve Metric API Key: IGNITE-11920 URL: https://issues.apache.org/jira/browse/IGNITE-11920 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov MetricRegistry may be made safer if we explicitly extract a group of metrics for some Ignite entity(cache, service, etc.). Internally, the registry will stay the same. API proposition is: {code:java} MetricRegistry { MetricSet default(); MetricSet group(String name); } MetricSet { LongCounter counter(); void registrer(Metric m); //other methods. } {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11912) [IEP-35] Research possibility to optimize MetricRegistry
Nikolay Izhikov created IGNITE-11912: Summary: [IEP-35] Research possibility to optimize MetricRegistry Key: IGNITE-11912 URL: https://issues.apache.org/jira/browse/IGNITE-11912 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov We should research and benchmark different data structure for a MetricRegistry implementation. A basic assumption of `MetricRegistry` usage: 1. Collection of metrics almost constant during Ignite lifetime. It will be changed on cache creation(destroy) and other not frequent operations. 2. Collection of metrics will be read very frequently(each several seconds or frequently) by configured metric exporters. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11906) Scalar examples fails on TC
Nikolay Izhikov created IGNITE-11906: Summary: Scalar examples fails on TC Key: IGNITE-11906 URL: https://issues.apache.org/jira/browse/IGNITE-11906 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Scalar examples tests fails in master. https://ci.ignite.apache.org/viewLog.html?buildId=4085544=buildResultsDiv=IgniteTests24Java8_ScalaExamples -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11905) [IEP-35] Monitoring Phase 2
Nikolay Izhikov created IGNITE-11905: Summary: [IEP-35] Monitoring Phase 2 Key: IGNITE-11905 URL: https://issues.apache.org/jira/browse/IGNITE-11905 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Phase 2 should introduce: Ability to collect lists of some internal object Ignite manage. Examples of such objects: * Caches * Queries (including continuous queries) * Services * Compute tasks * Distributed Data Structures * etc... 1. Fields for each list should be discussed in separate tickets 2. Metric Exporters (optionally) can support list export. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11890) Metrics documentation
Nikolay Izhikov created IGNITE-11890: Summary: Metrics documentation Key: IGNITE-11890 URL: https://issues.apache.org/jira/browse/IGNITE-11890 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Ignite should provide full user documentation about existing metrics: * ClusterMetrics * CacheMetrics * CacheGroupMetrics * DataRegionMetrics * DataStorageMetrics * etc... -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11881) Spark Data Fram example not working
Nikolay Izhikov created IGNITE-11881: Summary: Spark Data Fram example not working Key: IGNITE-11881 URL: https://issues.apache.org/jira/browse/IGNITE-11881 Project: Ignite Issue Type: Improvement Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov # Spark Data Frames examples fail. # Spark Data Frame examples don't execute on TC. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11872) Add EmptyLineSeparator to codestyle checker
Nikolay Izhikov created IGNITE-11872: Summary: Add EmptyLineSeparator to codestyle checker Key: IGNITE-11872 URL: https://issues.apache.org/jira/browse/IGNITE-11872 Project: Ignite Issue Type: Improvement Affects Versions: 2.7 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.8 We should add following check: {code} {code} and fix all current errors -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11848) [IEP-35] Monitoring Phase 1
Nikolay Izhikov created IGNITE-11848: Summary: [IEP-35] Monitoring Phase 1 Key: IGNITE-11848 URL: https://issues.apache.org/jira/browse/IGNITE-11848 Project: Ignite Issue Type: Task Affects Versions: 2.7 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.8 Umbrella ticket for the IEP-35. Monitoring and profiling. Phase 1 should include: * NextGen monitoring subsystem implementation to manage ** metrics ** lists ** exporters * JMX, SQLView, Log exporters * Migration of existing metrics to new manager * Lists for all Ignite user API -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-11677) LOCAL cache on client node can't be created if persistence enabled
Nikolay Izhikov created IGNITE-11677: Summary: LOCAL cache on client node can't be created if persistence enabled Key: IGNITE-11677 URL: https://issues.apache.org/jira/browse/IGNITE-11677 Project: Ignite Issue Type: Task Components: cache Affects Versions: 2.7 Reporter: Nikolay Izhikov Reproducer: {code:java} /** */ public class LocalCacheWithPersistenceEnabledTest extends GridCommonAbstractTest { /** */ private boolean client = false; /** {@inheritDoc} */ @Override protected IgniteConfiguration getConfiguration(String igniteInstanceName) throws Exception { IgniteConfiguration cfg = super.getConfiguration(igniteInstanceName); cfg.setClientMode(client); cfg.setDataStorageConfiguration(new DataStorageConfiguration() .setDataRegionConfigurations( new DataRegionConfiguration() .setName("data-region") .setPersistenceEnabled(true))); return cfg; } /** @throws Exception If failed. */ @Test public void testLocalCacheOnClientNodeWithLazyAllocation() throws Exception { client = false; IgniteEx srv = startGrid(0); srv.cluster().active(true); awaitPartitionMapExchange(); client = true; IgniteEx clnt = startGrid(1); IgniteCache cache = clnt.createCache(new CacheConfiguration("my-cache") .setCacheMode(CacheMode.LOCAL) .setDataRegionName("data-region")); cache.put(1, "test"); assertEquals(cache.get(1), "test"); } @Before public void before() throws Exception { cleanPersistenceDir(); } } {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-10325) Spark Data Frame - Thin Client
Nikolay Izhikov created IGNITE-10325: Summary: Spark Data Frame - Thin Client Key: IGNITE-10325 URL: https://issues.apache.org/jira/browse/IGNITE-10325 Project: Ignite Issue Type: Improvement Components: spark Affects Versions: 2.6 Reporter: Nikolay Izhikov For now, client node required to connect to Ignite cluster from Spark. We need to add ability to use Thin Client protocol for Spark integration. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9817) Update documentation and examples for Spark SQL Table Specification
Nikolay Izhikov created IGNITE-9817: --- Summary: Update documentation and examples for Spark SQL Table Specification Key: IGNITE-9817 URL: https://issues.apache.org/jira/browse/IGNITE-9817 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov We should update documentation and examples according to the results of IGNITE-9228. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9815) [TC Bot] Muted tests shouldn't considered as blocker
Nikolay Izhikov created IGNITE-9815: --- Summary: [TC Bot] Muted tests shouldn't considered as blocker Key: IGNITE-9815 URL: https://issues.apache.org/jira/browse/IGNITE-9815 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Currently, TC bot doesn't analyze stack trace of test fail [1]. If some test failed with the link to Ignite ticket it shouldn't be considered as a blocker. Link with examples of such behaviour. https://issues.apache.org/jira/browse/IGNITE-9272?focusedCommentId=16641617=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16641617 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9737) Ignite WatchDog service should be configurable
Nikolay Izhikov created IGNITE-9737: --- Summary: Ignite WatchDog service should be configurable Key: IGNITE-9737 URL: https://issues.apache.org/jira/browse/IGNITE-9737 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Fix For: 2.7 At the moment, there is no way to disable Ignite WatchDog service from config or JVM option. In any corner case or bug in that feature Ignite can become fully unusable due to unpredictable shutdown. We should provide a way to enable/disable this feature from config or from JVM option. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9681) Wrong GIT config in Team City release archive
Nikolay Izhikov created IGNITE-9681: --- Summary: Wrong GIT config in Team City release archive Key: IGNITE-9681 URL: https://issues.apache.org/jira/browse/IGNITE-9681 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Nikolay Izhikov Fix For: 2.7 Release archive created by "[Prepare Vote #1] Java &.Net & C++ (Complete assembly)" [1] contains wrong .git/config. It includes local Team City path and doesn't work properly on release manager local environment. Example of config file(lfs section is Team City specifi): {noformat} [core] repositoryformatversion = 0 filemode = false bare = false logallrefupdates = true sparseCheckout = true [remote "origin"] url = https://git-wip-us.apache.org/repos/asf/ignite fetch = +refs/heads/*:refs/remotes/origin/* [lfs] storage = /data/teamcity/system/git/git-E4D58B67.git/lfs [branch "master"] remote = origin merge = refs/heads/master {noformat} [1] https://ci.ignite.apache.org/viewType.html?buildTypeId=ApacheIgniteReleaseJava8_PrepareVote -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9678) [TC Bot] Trigger Run All for github comment
Nikolay Izhikov created IGNITE-9678: --- Summary: [TC Bot] Trigger Run All for github comment Key: IGNITE-9678 URL: https://issues.apache.org/jira/browse/IGNITE-9678 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Team City bot should work in the following way: It should be able to: 1. Execute "Run all" suite for specific PR when the author of PR makes a comment "@mtcga.bot Run Tests!" in GitHub comments. 2. Send a comment with "Run All" results both: to a Jira ticket and GitHub comment. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9665) RPM build doesn't work on lates Ubuntu linux
Nikolay Izhikov created IGNITE-9665: --- Summary: RPM build doesn't work on lates Ubuntu linux Key: IGNITE-9665 URL: https://issues.apache.org/jira/browse/IGNITE-9665 Project: Ignite Issue Type: Bug Affects Versions: 2.6 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.7 Release script fails to build RPM package on lates Ubuntu Linux Error message is: {{noformat}} Checking for unpackaged file(s): /usr/lib/rpm/check-files /tmp/tmp.ROYlZNaiI4/BUILDROOT/apache-ignite-2.7.0-1.x86_64 error: Installed (but unpackaged) file(s) found: /usr/share/doc/apache-ignite-2.7.0/bin/ignitevisorcmd.sh /usr/share/doc/apache-ignite-2.7.0/examples/README.txt /usr/share/doc/apache-ignite-2.7.0/examples/config/example-cache.xml /usr/share/doc/apache-ignite-2.7.0/examples/config/example-data-regions.xml {{noformat}} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9593) Spart Optimization fails to optimize statements
Nikolay Izhikov created IGNITE-9593: --- Summary: Spart Optimization fails to optimize statements Key: IGNITE-9593 URL: https://issues.apache.org/jira/browse/IGNITE-9593 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.6 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.7 Attachments: Spark_optimization_bugs_reproducer.patch In some cases, {{IgniteOptimization}} fails to optimize spark query. Reproducer attached. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9403) TDE - Phase-1. Thin clients
Nikolay Izhikov created IGNITE-9403: --- Summary: TDE - Phase-1. Thin clients Key: IGNITE-9403 URL: https://issues.apache.org/jira/browse/IGNITE-9403 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov We should provide support for a new {{encryptionEnabled}} flag in cache configuration for all thin clients: - .Net - Java - NodeJs - Python - PHP - C++(for now it doesn't have support of {{CacheConfiguration}}) Backward compatibility should be preserved. A contributor can take [commit](https://github.com/apache/ignite/commit/baab0a6ddd3973fb8fca6ecb0a7d841d7d3a72be) as an example -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9229) Integration with Spark Data Frame. Add support for a ARRAY data type
Nikolay Izhikov created IGNITE-9229: --- Summary: Integration with Spark Data Frame. Add support for a ARRAY data type Key: IGNITE-9229 URL: https://issues.apache.org/jira/browse/IGNITE-9229 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.6 Reporter: Nikolay Izhikov Fix For: 2.7 Currently, integration with Spark Data Frames doesn't support ARRAY data type. We need to add support for it -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9172) IgniteEx.cachex doesn't return existing cache
Nikolay Izhikov created IGNITE-9172: --- Summary: IgniteEx.cachex doesn't return existing cache Key: IGNITE-9172 URL: https://issues.apache.org/jira/browse/IGNITE-9172 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov {{IgniteEx.cachex}} method doesn't return an existing cache from a client node. Reproducer: {code:java} public class ClientNodeTest extends GridCommonAbstractTest { /** {@inheritDoc} */ @Override protected IgniteConfiguration getConfiguration(String name) throws Exception { IgniteConfiguration cfg = super.getConfiguration(name); cfg.setClientMode(name.equals("client")); if (name.equals("client") || name.equals(GRID_0)) cfg.setCacheConfiguration(new CacheConfiguration("cache-1")); else cfg.setCacheConfiguration(new CacheConfiguration("cache-2")); return cfg; } public void testClient() throws Exception { cleanPersistenceDir(); IgniteEx grid0 = startGrid(GRID_0); IgniteEx grid1 = startGrid(GRID_1); IgniteEx client = startGrid("client"); grid0.cluster().active(true); awaitPartitionMapExchange(); boolean cachex1 = client.cachex("cache-1") != null; boolean cachex2 = client.cachex("cache-2") != null; boolean cache1 = client.cache("cache-2") != null; boolean cache2 = client.cache("cache-2") != null; System.out.println("cachex1 = " + cachex1); System.out.println("cachex2 = " + cachex2); System.out.println("cache1 = " + cache1); System.out.println("cache2 = " + cache2); assertTrue(cachex1); assertTrue(cachex2); assertTrue(cache1); assertTrue(cache2); } } {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-9063) Wrong Data Frame parition order
Nikolay Izhikov created IGNITE-9063: --- Summary: Wrong Data Frame parition order Key: IGNITE-9063 URL: https://issues.apache.org/jira/browse/IGNITE-9063 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.6 Reporter: Nikolay Izhikov Fix For: 2.7 If cluster size is greater that 2 Spark Data Frame tests failed with exception. {noformat} java.lang.IllegalArgumentException: requirement failed: partitions(0).partition == 3, but it should equal 0 at scala.Predef$.require(Predef.scala:224) at org.apache.spark.rdd.RDD$$anonfun$partitions$2$$anonfun$apply$3.apply(RDD.scala:255) at org.apache.spark.rdd.RDD$$anonfun$partitions$2$$anonfun$apply$3.apply(RDD.scala:254) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:254) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:251) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:251) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:251) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:251) at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:253) at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:251) at scala.Option.getOrElse(Option.scala:121) at org.apache.spark.rdd.RDD.partitions(RDD.scala:251) at org.apache.spark.SparkContext.runJob(SparkContext.scala:2092) at org.apache.spark.rdd.RDD.count(RDD.scala:1162) at org.apache.ignite.spark.IgniteSQLDataFrameSpec$$anonfun$1$$anonfun$apply$mcV$sp$11.apply$mcV$sp(IgniteSQLDataFrameSpec.scala:145) {noformat} https://github.com/stuartmacd/ignite/commit/c9e7294c71de9e7b2bddfae671605a71260b80b3 - reproducer and fix. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8889) Spark examples doesn't work
Nikolay Izhikov created IGNITE-8889: --- Summary: Spark examples doesn't work Key: IGNITE-8889 URL: https://issues.apache.org/jira/browse/IGNITE-8889 Project: Ignite Issue Type: Bug Affects Versions: 2.5 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.6 https://ci.ignite.apache.org/viewLog.html?buildId=1427270=IgniteTests24Java8_Examples=buildLog=err -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8871) TDE - Phase-1. Documentation
Nikolay Izhikov created IGNITE-8871: --- Summary: TDE - Phase-1. Documentation Key: IGNITE-8871 URL: https://issues.apache.org/jira/browse/IGNITE-8871 Project: Ignite Issue Type: Sub-task Affects Versions: 2.5 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.7 TDE feature should be documented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8870) TDE - Phase-1. Examples
Nikolay Izhikov created IGNITE-8870: --- Summary: TDE - Phase-1. Examples Key: IGNITE-8870 URL: https://issues.apache.org/jira/browse/IGNITE-8870 Project: Ignite Issue Type: Sub-task Affects Versions: 2.5 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.7 Examples should be provided for TDE implementations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8868) IgniteOptimizationDateFuncSpec not included in IgniteDataFrameSuite
Nikolay Izhikov created IGNITE-8868: --- Summary: IgniteOptimizationDateFuncSpec not included in IgniteDataFrameSuite Key: IGNITE-8868 URL: https://issues.apache.org/jira/browse/IGNITE-8868 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov All Data Frame specs should be included in IgniteDataFrameSuite -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8516) Not null constraint doesn't checked
Nikolay Izhikov created IGNITE-8516: --- Summary: Not null constraint doesn't checked Key: IGNITE-8516 URL: https://issues.apache.org/jira/browse/IGNITE-8516 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.4 Reporter: Nikolay Izhikov Fix For: 2.6 User are able to insert null into the not null column through cache-api. Reproducer: {code:java} package org.apache.ignite.internal.processors.sql; public class IgniteNotNullBug extends GridCommonAbstractTest { @Override protected void beforeTestsStarted() throws Exception { startGrid(0); Set nn = new HashSet<>(); nn.add("address"); jcache(grid(0), cacheConfiguration(new QueryEntity(Organization.class.getName(), Address.class.getName()) .addQueryField("address", "java.lang.String", "address") .setNotNullFields(nn)), "ORG_ADDRESS"); } public void testPutNullField() throws Exception { Mapentries = new HashMap<>(); entries.put(new Organization("1"), new Address(null)); //entries.put(new Organization("2"), new Address("Some address")); IgniteCache cache = jcache(0, "ORG_ADDRESS"); cache.putAll(entries); System.out.println("cache.getConfiguration(CacheConfiguration) = " + cache.getConfiguration(CacheConfiguration.class).getQueryEntities()); List objects = execSql("SELECT address FROM ORG_ADDRESS.ADDRESS"); assert ((List)objects.get(0)).get(0) == null; } protected CacheConfiguration cacheConfiguration(QueryEntity qryEntity) { CacheConfiguration cache = defaultCacheConfiguration(); cache.setCacheMode(CacheMode.PARTITIONED); cache.setAtomicityMode(CacheAtomicityMode.ATOMIC); cache.setBackups(1); cache.setWriteSynchronizationMode(FULL_SYNC); cache.setQueryEntities(Collections.singletonList(qryEntity)); return cache; } private List execSql(String sql, Object... args) { SqlFieldsQuery qry = new SqlFieldsQuery(sql) .setArgs(args); return grid(0).context().query().querySqlFields(qry, true).getAll(); } private static class Organization implements Serializable { private final String name; private Organization(String name) { this.name = name; } } private static class Address implements Serializable { private final String address; private Address(String address) { this.address = address; } } } {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8485) TDE - Phase-1
Nikolay Izhikov created IGNITE-8485: --- Summary: TDE - Phase-1 Key: IGNITE-8485 URL: https://issues.apache.org/jira/browse/IGNITE-8485 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.6 Basic support for a Transparent Data Encryption should be implemented: 1. Usage of standard JKS, Java Security. 2. Persistent Data Encryption/Decryption. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8447) .Net support of DataStreamer.perThreadBufferSize
Nikolay Izhikov created IGNITE-8447: --- Summary: .Net support of DataStreamer.perThreadBufferSize Key: IGNITE-8447 URL: https://issues.apache.org/jira/browse/IGNITE-8447 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.6 Need to add support of DataStreamer#perThreadBuffer property. It was added in IGNITE-6699. Related failed test - DataStreamerTest.TestBufferSize https://ci.ignite.apache.org/project.html?projectId=IgniteTests24Java8=-7699185391938208048=%3Cdefault%3E=testDetails -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8331) SQL: Add Decimal precision and scale constraint
Nikolay Izhikov created IGNITE-8331: --- Summary: SQL: Add Decimal precision and scale constraint Key: IGNITE-8331 URL: https://issues.apache.org/jira/browse/IGNITE-8331 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov We should support DECIMAL(X, Y) syntax. Currently, we ignore it. It affects semantics. E.g., one can insert a DECIMAL with greater precision into a cache/table without any problems. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8265) TDE - MEK replacement
Nikolay Izhikov created IGNITE-8265: --- Summary: TDE - MEK replacement Key: IGNITE-8265 URL: https://issues.apache.org/jira/browse/IGNITE-8265 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov If MEK is lost or stolen while the cluster is alive, TDE should provide way to replace(regenerate) MEK. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8264) TDE - Node join enhanecements
Nikolay Izhikov created IGNITE-8264: --- Summary: TDE - Node join enhanecements Key: IGNITE-8264 URL: https://issues.apache.org/jira/browse/IGNITE-8264 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov All nodes that are joined to the cluster with TDE enabled should be configured to get TDE working. They need access to MEK and CEK's. We should extend node join mechanism to support TDE. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8263) TDE - Encryption/Decription of pages
Nikolay Izhikov created IGNITE-8263: --- Summary: TDE - Encryption/Decription of pages Key: IGNITE-8263 URL: https://issues.apache.org/jira/browse/IGNITE-8263 Project: Ignite Issue Type: Sub-task Affects Versions: 2.4 Reporter: Nikolay Izhikov Fix For: 2.6 When data for an encrypted cache are written to the persistence store. Data page should be encrypted through configured encryption provider. * Encryption/decryption should be implemented -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8262) TDE - MEK and CEK processing
Nikolay Izhikov created IGNITE-8262: --- Summary: TDE - MEK and CEK processing Key: IGNITE-8262 URL: https://issues.apache.org/jira/browse/IGNITE-8262 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov To get TDE working we should implement managing MEK and CEK's * MEK should be loaded from configured KeyStore * CEK's should be stored in some internal data storage and be encrypted with MEK. * Cluster shouldn't get activated before MEK are loaded. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8261) TDE - Configuration
Nikolay Izhikov created IGNITE-8261: --- Summary: TDE - Configuration Key: IGNITE-8261 URL: https://issues.apache.org/jira/browse/IGNITE-8261 Project: Ignite Issue Type: Sub-task Affects Versions: 2.4 Reporter: Nikolay Izhikov Fix For: 2.6 Ignite configuration should be extended to support all TDE specific configuration parameters: * KeyStore configuration. * New option for encrypted caches. * Default KeyStore implementation should use JDK provided KeyStore - https://docs.oracle.com/javase/8/docs/api/java/security/KeyStore.html. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-8260) Transparend data encryption
Nikolay Izhikov created IGNITE-8260: --- Summary: Transparend data encryption Key: IGNITE-8260 URL: https://issues.apache.org/jira/browse/IGNITE-8260 Project: Ignite Issue Type: New Feature Affects Versions: 2.4 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.6 TDE feature should allow to user to protect data stored in the persistence storage with some cypher algorithm. Design details described in [IEP-18|https://cwiki.apache.org/confluence/display/IGNITE/IEP-18%3A+Transparent+Data+Encryption]. When this task will be done production ready TDE implementation should be available for Ignite. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7862) Update flatten-plugin to 1.0.1 version
Nikolay Izhikov created IGNITE-7862: --- Summary: Update flatten-plugin to 1.0.1 version Key: IGNITE-7862 URL: https://issues.apache.org/jira/browse/IGNITE-7862 Project: Ignite Issue Type: Improvement Components: general Affects Versions: 2.4 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 With flatten-plugin version {{1.0.0-beta-3}} we has to enlist all transitive dependencies. dev-list discussion - http://apache-ignite-developers.2346864.n4.nabble.com/Maven-Issues-with-flatten-plugin-td27537.html Solution is to update plugin to {{1.0.1}} version -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7863) Cleanup spark and spark_2.10 dependencies
Nikolay Izhikov created IGNITE-7863: --- Summary: Cleanup spark and spark_2.10 dependencies Key: IGNITE-7863 URL: https://issues.apache.org/jira/browse/IGNITE-7863 Project: Ignite Issue Type: Sub-task Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov After solving issue with flatten plugin it possible to cleanup dependency list in spark and spark_2.10 modules. Transitive dependencies should be resolved using standart maven mechanism. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7835) Scala tests appeared in wrong configurations
Nikolay Izhikov created IGNITE-7835: --- Summary: Scala tests appeared in wrong configurations Key: IGNITE-7835 URL: https://issues.apache.org/jira/browse/IGNITE-7835 Project: Ignite Issue Type: Bug Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 Scala tests are executed in wrong configurations. This happens because scalatest-maven-plugin doesn't properly handle {{-Dtest}} settings. http://apache-ignite-developers.2346864.n4.nabble.com/TeamCity-Ignite-RDD-tests-tp27129p27343.html -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7816) DataFrame examples: wrong folder, hidden on TeamCity
Nikolay Izhikov created IGNITE-7816: --- Summary: DataFrame examples: wrong folder, hidden on TeamCity Key: IGNITE-7816 URL: https://issues.apache.org/jira/browse/IGNITE-7816 Project: Ignite Issue Type: Sub-task Components: documentation Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.4 1. Spark DataFrame examples are in {{examples/src/main/spark/org/apache/ignite/examples/spark}} folder. Should be moved to {{examples/src/main/scala/org/apache/ignite/scalar/examples/spark}}. 2. Spark DataFrame examples not tested on Team City under Ignite Examples. https://issues.apache.org/jira/browse/IGNITE-7655?focusedCommentId=16377875=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16377875 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7756) Streamer fails if IgniteUuid is indexed
Nikolay Izhikov created IGNITE-7756: --- Summary: Streamer fails if IgniteUuid is indexed Key: IGNITE-7756 URL: https://issues.apache.org/jira/browse/IGNITE-7756 Project: Ignite Issue Type: Bug Components: streaming Affects Versions: 2.3 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 IgniteDataStreamer are failed to put data to the cache if IgniteUuid is IndexedType. Spark tests in IGNITE-7227 are failed because of this issue. Reproducer: {code:java} public void testStreamer() throws Exception { Ignite client = grid("client"); CacheConfiguration ccfg = new CacheConfiguration("UUID_CACHE"); ccfg.setIndexedTypes(IgniteUuid.class, String.class); client.createCache(ccfg); try(IgniteDataStreamercache = client.dataStreamer("UUID_CACHE")) { for(Integer i=0; i<2; i++) cache.addData(IgniteUuid.randomUuid(), i.toString()); } } {code} Exception stack trace: {noformat} [23:43:35] (err) Failed to execute compound future reducer: GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=0, done=false, cancelled=false, err=null, futs=[true, true]][23:43:35] (err) Failed to execute compound future reducer: GridCompoundFuture [rdc=null, initFlag=1, lsnrCalls=0, done=false, cancelled=false, err=null, futs=[true, true]]class org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=57961924-82ec-4d56-81eb-1a4109a0] at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$Buffer.onResponse(DataStreamerImpl.java:1900) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$3.onMessage(DataStreamerImpl.java:344) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1554) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1182) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:125) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1089) at org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:499) at java.lang.Thread.run(Thread.java:748) Caused by: class org.apache.ignite.IgniteException: Failed to set initial value for cache entry at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2135) at org.apache.ignite.internal.processors.datastreamer.DataStreamerUpdateJob.call(DataStreamerUpdateJob.java:140) at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.localUpdate(DataStreamProcessor.java:397) at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.processRequest(DataStreamProcessor.java:302) at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor.access$000(DataStreamProcessor.java:59) at org.apache.ignite.internal.processors.datastreamer.DataStreamProcessor$1.onMessage(DataStreamProcessor.java:89) ... 6 more Caused by: class org.apache.ignite.IgniteCheckedException: Failed to update index, incorrect key class [expCls=org.apache.ignite.lang.IgniteUuid, actualCls=org.apache.ignite.internal.binary.BinaryObjectImpl] at org.apache.ignite.internal.processors.query.GridQueryProcessor.typeByValue(GridQueryProcessor.java:1954) at org.apache.ignite.internal.processors.query.GridQueryProcessor.store(GridQueryProcessor.java:1877) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.store(GridCacheQueryManager.java:403) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.finishUpdate(IgniteCacheOffheapManagerImpl.java:1343) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl$CacheDataStoreImpl.invoke(IgniteCacheOffheapManagerImpl.java:1207) at org.apache.ignite.internal.processors.cache.IgniteCacheOffheapManagerImpl.invoke(IgniteCacheOffheapManagerImpl.java:345) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.storeValue(GridCacheMapEntry.java:3527) at org.apache.ignite.internal.processors.cache.GridCacheMapEntry.initialValue(GridCacheMapEntry.java:2735) at org.apache.ignite.internal.processors.datastreamer.DataStreamerImpl$IsolatedUpdater.receive(DataStreamerImpl.java:2113) ... 11 more class org.apache.ignite.IgniteCheckedException: DataStreamer request failed [node=57961924-82ec-4d56-81eb-1a4109a0] at
[jira] [Created] (IGNITE-7727) IgniteRDDSpec. Failing tests
Nikolay Izhikov created IGNITE-7727: --- Summary: IgniteRDDSpec. Failing tests Key: IGNITE-7727 URL: https://issues.apache.org/jira/browse/IGNITE-7727 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.4 Reporter: Nikolay Izhikov Fix For: 2.5 Two spark tests are broken. Need to fix it. 1. IgniteRDDSpec.IgniteRDD should successfully store data to ignite using saveValues 2. IgniteRDDSpec.IgniteRDD should successfully store data to ignite using saveValues with inline transformation -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7691) Provide info about DECIMAL column scale and precision
Nikolay Izhikov created IGNITE-7691: --- Summary: Provide info about DECIMAL column scale and precision Key: IGNITE-7691 URL: https://issues.apache.org/jira/browse/IGNITE-7691 Project: Ignite Issue Type: Improvement Components: sql Affects Versions: 2.4 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 Currently, it impossible to obtain scale and precision of DECIMAL column from sql table metadata. Ignite should provide those type of meta information. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7683) ContinuousQueryWithTransformer needs to be documented
Nikolay Izhikov created IGNITE-7683: --- Summary: ContinuousQueryWithTransformer needs to be documented Key: IGNITE-7683 URL: https://issues.apache.org/jira/browse/IGNITE-7683 Project: Ignite Issue Type: Improvement Components: documentation Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 New API - ContinuousQueryWithTransformer should be documented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7655) Spark Data Frames: saving data frames into Ignite needs to be documented
Nikolay Izhikov created IGNITE-7655: --- Summary: Spark Data Frames: saving data frames into Ignite needs to be documented Key: IGNITE-7655 URL: https://issues.apache.org/jira/browse/IGNITE-7655 Project: Ignite Issue Type: Bug Components: spark Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.4 Once IGNITE-7337 is ready for merge. This new feature of Ignite needs to be documented. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7652) ContinuousQueryWithTransformer needs to be documented
Nikolay Izhikov created IGNITE-7652: --- Summary: ContinuousQueryWithTransformer needs to be documented Key: IGNITE-7652 URL: https://issues.apache.org/jira/browse/IGNITE-7652 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.5 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.5 As long as IGNITE-425 merged to master ContinuousQueryWithTransformer API needs to be documented. Example has to be provided as well. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (IGNITE-7136) Add JMX metric for indexing to .Net
Nikolay Izhikov created IGNITE-7136: --- Summary: Add JMX metric for indexing to .Net Key: IGNITE-7136 URL: https://issues.apache.org/jira/browse/IGNITE-7136 Project: Ignite Issue Type: Sub-task Components: platforms Reporter: Nikolay Izhikov Assignee: Pavel Tupitsyn New JMX metric implemented in IGNITE-6903. We need to add support for this metric to .Net -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7077) Spark Data Frame Support. Convert complete query to Ignite SQL
Nikolay Izhikov created IGNITE-7077: --- Summary: Spark Data Frame Support. Convert complete query to Ignite SQL Key: IGNITE-7077 URL: https://issues.apache.org/jira/browse/IGNITE-7077 Project: Ignite Issue Type: Task Components: spark Affects Versions: 2.3 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.4 Basic support of Spark Data Frame for Ignite implemented in IGNITE-3084. We need to implement custom spark strategy that can convert whole Spark SQL query to Ignite SQL Query if query consists of only Ignite tables. The strategy does nothing if spark query includes not only Ignite tables. Memsql implementation can be taken as an example - https://github.com/memsql/memsql-spark-connector -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-7042) Test written in scala doesn't executed on TC
Nikolay Izhikov created IGNITE-7042: --- Summary: Test written in scala doesn't executed on TC Key: IGNITE-7042 URL: https://issues.apache.org/jira/browse/IGNITE-7042 Project: Ignite Issue Type: Bug Components: spark Affects Versions: 2.3 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Priority: Minor Fix For: 2.4 Tests from module `spark` and `spark_2.10` written in scala don't executes on TC. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6312) Link to unexisted URL on ExecutorService page
Nikolay Izhikov created IGNITE-6312: --- Summary: Link to unexisted URL on ExecutorService page Key: IGNITE-6312 URL: https://issues.apache.org/jira/browse/IGNITE-6312 Project: Ignite Issue Type: Bug Components: documentation Reporter: Nikolay Izhikov Priority: Minor Link "IgniteCompute" forward from [ExecutorService|https://apacheignite.readme.io/docs/executor-service] to 404 page. https://apacheignite.readme.io/docs/executor-service -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6301) CacheConfiguration.indexedTypes never initialized and can be removed
Nikolay Izhikov created IGNITE-6301: --- Summary: CacheConfiguration.indexedTypes never initialized and can be removed Key: IGNITE-6301 URL: https://issues.apache.org/jira/browse/IGNITE-6301 Project: Ignite Issue Type: Bug Components: sql Affects Versions: 2.1 Reporter: Nikolay Izhikov Priority: Trivial CacheConfiguration#indexedTypes never initialized {code:java} public class CacheConfigurationextends MutableConfiguration { //... /** */ private transient Class[] indexedTypes; // public CacheConfiguration setIndexedTypes(Class... indexedTypes) { if (F.isEmpty(indexedTypes)) return this; int len = indexedTypes.length; if (len == 0) return this; A.ensure((len & 1) == 0, "Number of indexed types is expected to be even. Refer to method javadoc for details."); if (this.indexedTypes != null) throw new CacheException("Indexed types can be set only once."); Class[] newIndexedTypes = new Class[len]; // other method body fill qryEntities list. {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-6221) ContinuousQuery. Local listener notified if filter throws exception
Nikolay Izhikov created IGNITE-6221: --- Summary: ContinuousQuery. Local listener notified if filter throws exception Key: IGNITE-6221 URL: https://issues.apache.org/jira/browse/IGNITE-6221 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.1 Reporter: Nikolay Izhikov Priority: Minor Fix For: 2.2 Local listener of continuous query receives event if filter throw exception from `evaluate`. Steps to reproduce the bug: 1. Run continuous query with remote filter. 2. Throw exception from filter. Current behavior: 3. Local listener notified. Expected behavior: 3. Local listener doesn't notify. [Mail-list discussion|http://apache-ignite-developers.2346864.n4.nabble.com/ContinuousQueryWithTransformer-implementation-questions-2-td21418.html] Filter description from [jcache Javadoc|https://static.javadoc.io/javax.cache/cache-api/1.0.0/javax/cache/event/CacheEntryEventFilter.html#evaluate(javax.cache.event.CacheEntryEvent)]: {noformat} Returns: true if the evaluation passes, otherwise false. The effect of returning true is that listener will be invoked {noformat} Test to reproduce error: {code:java} package org.apache.ignite.internal.processors.cache.query.continuous; import java.io.Serializable; import javax.cache.Cache; import javax.cache.event.CacheEntryEvent; import javax.cache.event.CacheEntryUpdatedListener; import org.apache.ignite.IgniteCache; import org.apache.ignite.cache.CacheEntryEventSerializableFilter; import org.apache.ignite.cache.CacheMode; import org.apache.ignite.cache.query.ContinuousQuery; import org.apache.ignite.cache.query.QueryCursor; public class GridCacheContinuousQueryFilterExceptionTest extends GridCacheContinuousQueryAbstractSelfTest implements Serializable { /** * @throws Exception If failed. */ public void testListenerAfterFilterException() throws Exception { IgniteCachecache = grid(0).cache(DEFAULT_CACHE_NAME); ContinuousQuery qry = new ContinuousQuery<>(); qry.setLocalListener(new CacheEntryUpdatedListener () { @Override public void onUpdated(Iterable evts) { fail("Listener shouldn't be called"); } }); qry.setRemoteFilter(new CacheEntryEventSerializableFilter () { @Override public boolean evaluate(CacheEntryEvent evt) { throw new RuntimeException("Test error."); } }); try (QueryCursor > ignored = cache.query(qry)) { for (int i = 0; i < 100; i++) cache.put(i, i); } } @Override protected CacheMode cacheMode() { return CacheMode.REPLICATED; } @Override protected int gridCount() { return 1; } } {code} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5866) JettyRestProcessorUnsignedSelfTest and JettyRestProcessorSignedSelfTest fails on master
Nikolay Izhikov created IGNITE-5866: --- Summary: JettyRestProcessorUnsignedSelfTest and JettyRestProcessorSignedSelfTest fails on master Key: IGNITE-5866 URL: https://issues.apache.org/jira/browse/IGNITE-5866 Project: Ignite Issue Type: Bug Components: clients Affects Versions: 2.1 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Fix For: 2.2 JettyRestProcessorSignedSelfTest and JettyRestProcessorUnsignedSelfTest fails on master when run whole test class. If run single method both tests succeed. testMetadataLocal: {noformat} junit.framework.AssertionFailedError: expected:<5> but was:<6> at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at org.apache.ignite.internal.processors.rest.JettyRestProcessorAbstractSelfTest.testMetadataLocal(JettyRestProcessorAbstractSelfTest.java:1127) {noformat} testMetadataRemote {noformat} junit.framework.AssertionFailedError: expected:<6> but was:<7> at junit.framework.Assert.fail(Assert.java:57) at junit.framework.Assert.failNotEquals(Assert.java:329) at junit.framework.Assert.assertEquals(Assert.java:78) at junit.framework.Assert.assertEquals(Assert.java:234) at junit.framework.Assert.assertEquals(Assert.java:241) at junit.framework.TestCase.assertEquals(TestCase.java:409) at org.apache.ignite.internal.processors.rest.JettyRestProcessorAbstractSelfTest.testMetadataRemote(JettyRestProcessorAbstractSelfTest.java:1174) {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5659) BinaryMarshaller loads class with local classloader even if it excluded in Configuration
Nikolay Izhikov created IGNITE-5659: --- Summary: BinaryMarshaller loads class with local classloader even if it excluded in Configuration Key: IGNITE-5659 URL: https://issues.apache.org/jira/browse/IGNITE-5659 Project: Ignite Issue Type: Bug Components: cache Affects Versions: 2.0 Reporter: Nikolay Izhikov Priority: Critical Run Ignite with configuration: Two nodes: server and client # BinaryMarshaller # PeerClassLoadingEnabled = true # PeerClassLoadingLocalClassPathExclude = org.apache.ignite.test.* Do following actions: # put some Employee data to the server node # run ScanQuery from client node Error on step 2: {noformat} Caused by: java.lang.ClassCastException: org.apache.ignite.test.ignite2190.Employee cannot be cast to org.apache.ignite.test.ignite2190.Employee at org.apache.ignite.test.ignite2190.EmployeePredicate.apply(EmployeePredicate.java:8) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$PeekValueExpiryAwareIterator.advance(GridCacheQueryManager.java:3000) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager$PeekValueExpiryAwareIterator.onHasNext(GridCacheQueryManager.java:2923) at org.apache.ignite.internal.util.GridCloseableIteratorAdapter.hasNextX(GridCloseableIteratorAdapter.java:53) at org.apache.ignite.internal.util.lang.GridIteratorAdapter.hasNext(GridIteratorAdapter.java:45) at org.apache.ignite.internal.processors.cache.query.GridCacheQueryManager.runQuery(GridCacheQueryManager.java:1188) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager.processQueryRequest(GridCacheDistributedQueryManager.java:231) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:109) at org.apache.ignite.internal.processors.cache.query.GridCacheDistributedQueryManager$2.apply(GridCacheDistributedQueryManager.java:107) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1020) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:541) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:359) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:296) at org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:104) at org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:286) at org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1556) at org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1184) at org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126) at org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1097) ... 3 more {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (IGNITE-5321) DiscoverySpi#setDataExchange returns concrete implementation
Nikolay Izhikov created IGNITE-5321: --- Summary: DiscoverySpi#setDataExchange returns concrete implementation Key: IGNITE-5321 URL: https://issues.apache.org/jira/browse/IGNITE-5321 Project: Ignite Issue Type: Bug Affects Versions: 2.0 Reporter: Nikolay Izhikov Assignee: Nikolay Izhikov Priority: Trivial Fix For: 2.1 Two method of DiscoverySpi interface returns concrete implementation instead of interface. {code:java} public interface DiscoverySpi extends IgniteSpi { TcpDiscoverySpi setDataExchange(DiscoverySpiDataExchange var1); TcpDiscoverySpi setMetricsProvider(DiscoveryMetricsProvider var1); ... } {code} -- This message was sent by Atlassian JIRA (v6.3.15#6346)