Error in running wordcount hadoop example in ignite

hi i want to execute wordcount example of hadoop in apache ignite. i have used apache-ignite-hadoop-2.6.0-bin for execute map reduce tasks. my default-config.xml in apache-ignite-hadoop-2.6.0-bin/config folder is just as bellow: http://www.springframework.org/schema/beans;

?????? Ignite Data Streamer Hung after a period

Thank for your reply. 1. Yes, I have persistence. 2. I think the cache store is not the bottleneck, because the skipStore is enabled when loading data. IgniteDataStreamer streamer = ignite.dataStreamer(IgniteCacheKey.DATA_POINT_NEW.getCode()); streamer.skipStore(true);

Re: same cache cannot update twice in one transaction

Hi Ilya, Since I'm using Cassandra as data store, it raises the following exceptions once MVCC is enabled - class org.apache.ignite.IgniteCheckedException: Grid configuration parameter invalid: readThrough cannot be used with TRANSACTIONAL_SNAPSHOT atomicity mode at

Re: What is the correct way of keeping ignite alive in a console app

Hi, Your application exits because Ignite node is started in try-with-resources block. So, the node is stopped upon leaving try block. If you write simply public static void main(String[] args) { Ignition.start(igniteConfiguration); } application will continue running after main method

Re: introspection of ignite data

Thank you Jeff 972.900.1561 Scott Cote From: Jeff Zemerick Sent: Wednesday, February 27, 2019 10:30 AM To: user@ignite.apache.org Subject: Re: introspection of ignite data Scott, I have used: CacheConfiguration config =

Re: Access a cache loaded by DataStreamer with SQL

OK, was able to get it working. Apparently the cache name has to be PUBLIC and it will create a table based on the object definition that I have. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

What is the correct way of keeping ignite alive in a console app

Hello What is the best way to keep my app running after i have initialized my ignite instance? public static void main(String[] args) { try (Ignite ignite = Ignition.start(igniteConfiguration)) { } // How to wait here in a correct way to make ignite not exit the

Re: introspection of ignite data

Scott, I have used: CacheConfiguration config = cache.getConfiguration(CacheConfiguration.class); Collection entities = config.getQueryEntities(); for(QueryEntity e : entities) { System.out.println("Table: " + e.getTableName()); } I'm new to Ignite so there's a chance it might not be the best

Re: ignite.net custom sql functions

Hello! > I have found several examples in java code for registering the custom function but nothing showing how to do it in the xml config file. // Preparing a cache configuration.CacheConfiguration cfg = new CacheConfiguration(); // Registering the class that contains custom SQL

Re: Access a cache loaded by DataStreamer with SQL

I have looked at the docs and it is not clear to me how to set-up a queryable table that can be loaded with a datastreamer. If I want to use the datastreamer and still have it queryable from a third party tool like Tableau what would I need to set-up. I tried one thing and it showed up in the

Re: ignite.net custom sql functions

Hi Wayne, You can write a service in Java that registers custom SQL function. Then call that service from .NET. Classpath is provided in IgniteConfiguration.JvmClasspath. See https://ptupitsyn.github.io/Ignite-Plugin/ to get some idea on how to combine Java-based Ignite stuff with Ignite.NET.

Re: ignite.net custom sql functions

Thanks Ilya So i need a little guidance here if you don't mind i am just throwing together a quick test as a precursor to a new project. I have a .net ignite service wrapper var cfg = new IgniteConfiguration() { SpringConfigUrl = "/ignitecfg.xml",

Re: On Multiple Endpoints Mode of JDBC Driver

Hello! I doubt that you will have speed-up here compared to just using JDBC (possibly with randomized endpoints list). Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 14:52, 李玉珏@163 <18624049...@163.com>: > The main consideration is that using JDBC interface, the existing code >

Re: Access a cache loaded by DataStreamer with SQL

Hello! Please refer to this page: https://apacheignite.readme.io/docs/indexes In short, you can use CREATE TABLE or GetOrCreateCache indexedTypes+annotations or QueryEntities. All of those are compatible with DataStreamer as long as type name and field/column names match. By default caches do

Re: Ignite Data streamer optimization

Hello! Why do you have nodeParallelOperations of 1? Do you really have issues with default configuration? Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 08:59, ashishb888 : > Sure. But in my case I can not do so. Any other options for single threads? > > > > -- > Sent from:

Re: Ignite Data Streamer Hung after a period

Hello! It's hard to say. Do you have persistence? Are you sure that cache store is not the bottleneck? I would start with gathering thread dumps from whole cluster when in stuck state. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 15:06, Justin Ji : > Dmitry - > > I also encountered

Re: H2 SQL query optimiser strategy in Ignite

Hello! There's 'enforceJoinOrder' connection setting which will prevent reordering of joins, so if you manually write query to have optimal join ordering then it would perform well. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 13:39, joseheitor : > Thanks, Ilya. > > The reason that I

Re: ignite.net custom sql functions

Hello! They are not supported yet. However, you can still configure your caches to use Java custom functions and use them even if the rest of your project is .Net. I assume that functions are usually self-contained. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 12:13, wt : > good day >

Re: Performance degradation in case of high volumes

Hello! It is possible that your primary key would not fit into index and walking index tree will put unnecessary strain on persistent page memory. On 2.7 there will be a warning about that in log. Please try to tune IGNITE_MAX_INDEX_PAYLOAD_SIZE system property since it's the only available

Re: [Backup/Restore] Can we use copy data file from IGNITE_HOME/work/db/nodexxxxx/ to other place if Ignite process is running ?

Hello! You can deactivate the cluster, copy data and then reactivate cluster back. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 15:39, James Wang 王升平 (edvance CN) < james.w...@edvancesecurity.com>: > Hi Ilya, > > > > If so, could you please provide some advice on how to backup ignite

RE: [Backup/Restore] Can we use copy data file from IGNITE_HOME/work/db/nodexxxxx/ to other place if Ignite process is running ?

Hi Ilya, If so, could you please provide some advice on how to backup ignite data OUT to disk. What do you mean about 3rd party extensions ? Our use case is: we don't have other RDBMS as 3 third party persistent, we all rely on IGNITE. I know GridGain have enterprise features, but we don’t

Re: Ignite Data Streamer Hung after a period

Dmitry - I also encountered this problem. I used both persistence and indexing, when I loaded 20 million records, the loading speed became much slower than before, but the CPU of the ignite server is low.

Re: On Multiple Endpoints Mode of JDBC Driver

The main consideration is that using JDBC interface, the existing code modification workload is small. 在 2019/2/27 下午5:31, Stephen Darlington 写道: If you’re already using Ignite-specific APIs (IgniteCallable), why not use the other Ignite-native APIs for reading/writing/processing data? That

Re: H2 SQL query optimiser strategy in Ignite

Thanks, Ilya. The reason that I am trying to better understand the rationale is that I had a multi-join query that was performing badly on Ignite compared to PostgreSQL. When comparing the query plans from the two systems, I discovered that the Postgres query-planner had re-ordered the joins in

Re: [Backup/Restore] Can we use copy data file from IGNITE_HOME/work/db/nodexxxxx/ to other place if Ignite process is running ?

Hello! No, you cannot do this (without using 3rd party extensions, anyway). Regard, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 11:00, James Wang 王升平 (edvance CN) < james.w...@edvancesecurity.com>: > Hi Support, > > > > I am thinking of backup / restore approach. > > > > Can we copy data file

Re: Memory tuning [set sysctl -w vm.extra_free_kbytes]

Hello! You can probably try and ignore this setting. I haven't heard about case where it would make a difference. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 12:29, James Wang 王升平 (edvance CN) < james.w...@edvancesecurity.com>: > Hi Support, > > > > The doc mentions to set sysctl -w

Re: H2 SQL query optimiser strategy in Ignite

Hello! As far as my understanding goes, H2 planner is sometimes fed constants instead of actual statistics. So that it is able to plan queries for general cases in absense of precise statistics. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 13:13, joseheitor : > Hi, > > H2 uses a

Re: web-console not displaying the latest version of ignite

Thanks for the reply. I am able to see the proper web-console version i.e 2.7. I took the old ignite version frontend as u said. Since I was working on both ignite versions 2.5 and 2.7 , i missed something while compiling the web-console. regards Krupa -- Sent from:

H2 SQL query optimiser strategy in Ignite

Hi, H2 uses a cost-based query optimisation strategy for the query-planner. But in the case of Ignite, I suspect that this cannot be leveraged because the client node does not have access to data statistics (distribution, size, etc) throughout the various client nodes... So how is the query plan

Re: On Multiple Endpoints Mode of JDBC Driver

If you’re already using Ignite-specific APIs (IgniteCallable), why not use the other Ignite-native APIs for reading/writing/processing data? That way you can use affinity functions for load balancing where it makes sense and Ignite’s normal load balancing processing for general compute tasks.

Memory tuning [set sysctl -w vm.extra_free_kbytes]

Hi Support, The doc mentions to set sysctl -w vm.extra_free_kbytes=124 for CentOS. But I find this is only related to CentOS 6. Please advise which shall we set for CentOS-7 Thank you. Best Regards, James Wang M/WeChat: +86 135 1215 1134 This message contains information that is deemed

Re: Do client nodes also have to define the cache configuration?

Thank you very much -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Do client nodes also have to define the cache configuration?

Hello! No, it is not necessary. But you can supply additional configurations of caches which will be started when client joins. Regards, -- Ilya Kasnacheev ср, 27 февр. 2019 г. в 12:03, PBSLogitek : > Hello > > I have a simple question. I have a server a node and some clients > connecting >

Do client nodes also have to define the cache configuration?

Hello I have a simple question. I have a server a node and some clients connecting to it. In my server xml configuration i have defined all caches. Now my question is do i have to define those caches in my client xml config to or is it not necessary? Will i get an error when i try to access a

Re: pre-load data (Apache ignite native persistence store or Cassandra) into two partitioned cache tables

Thanks for reply. I got the most working thanks to the example (https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/starschema/CacheStarSchemaExample.java) provided by Ignite. Here is my sql for POC (Cassandra DDL scripts) create table

[Backup/Restore] Can we use copy data file from IGNITE_HOME/work/db/nodexxxxx/ to other place if Ignite process is running ?

Hi Support, I am thinking of backup / restore approach. Can we copy data file from IGNITE_HOME/work/db/nodex/ to other place if Ignite process is running ? If we restore the copied data, will be file be crashed if ignite process was running when this file was being backup. Thank you,