Re: Why does CacheBasedDataSet destroy the cache it is given

Dear Courtney Robinson, please write if you have any cases to update helper cache with paritions to better understand the situation. How and when are you going to clear this helper caches if the alternative version (as you suggested in the first email) of CacheBased Dataset will be provided?

Re: Why does CacheBasedDataSet destroy the cache it is given

Dear Courtney Robinson, let's discuss here the possible behaviour of this CacheBased Dataset closing. When designed this feature we think, that the all training parts and stuff should be deleted from Caches ad model should be serialized or exported somwhere. What is your use-case& Could you

Re: Xgboost inference on regression task.

Dear akorensh, thank you for the support, please, next time provide the correct link to Apache Ignite documentation, not to GridGain site (there could be outdated documentation). The actual documentation is here for release 2.8.0 https://apacheignite.readme.io/docs/machine-learning

Re: Xgboost inference on regression task.

Hi, askelejboelle! Could you please share the model, snippet of data (if it's not secret) and full code sample to reproduce problem. It looks like a bug and I'll try to fix it. -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite ML - Weka integration?

Hi! It's a good question and note about integration. We have no this integration on roadmap, but I agree, that we have no good 'workbench environment'. But should it be a kind of destop workbench or something else? I don't know. I don't any examples of integration with the Weka for another

Re: Machine Learning questions

SPOILER: I need to say that the release 2.8 will be published after New Year and all answers will be related to the new release. If we talk to 2.8 release (the last update of ML functionality in master and release branch) +++ I assume that I would start by extracting features from my JSON

Re: Ignite AI learning resources

Hi, we are going to release new AI next 2 months with new really cool ML and TensorFlow integration. No documentation and papers released and written yet, but I hope to publish them next 2-3 months too. Also I could recommend to learn this code tutorial

Re: Apache Spark and Apache Ignite

Hi, Wellington I'll be happy to help you if you give me more information of your goal. I have a few questions, could you answer please? 1. What's your main goal? To solve optimization task on the data in Ignite or in Spark? 2. What's the average size of initial population? 3. Did you run these

Re: Support User defined aggregate function ?

Currently, they are not supported, but you could ask about it on dev-list and maybe somebody implement it. Or propose PR with UDAF support Also use-cases for UDAF usages will be helpful -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: 2.8 release date?

I couldn't find any information about release plans on dev-list. Also I don't know what release number will be in the new release: 2.8 or 3.0 for example. Ilya, could you share some links about that? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: IGNITE-3180 : please add support for median, stddev, var in Ignite SQL

Are you ready to use them as a part of Ignite ML functions (part of stat or math package?) or you need in SQL only? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: DatasetFactory.createSimpleDataset with a cache

Ok, will wait your feedback, also we are going to add example to master "how to work with SQL cache from ML module" -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: DatasetFactory.createSimpleDataset with a cache

Hi, what kind of ML algorithms are you going to use? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Ignite and spark for deep learning

Currently we have no integration with DL4j in ignite ML and have no such plans in roadmap. The TF is the best platform for the Deep Learning now and we spend time on integration with TF first of all. Maybe, in future, direct integration between DL4j and ignite will be added. But it costst a

Re: Distributed Training in tensorflow

Yes, I agree with your conclusion. I have no benchmarks, of course, but it seems that no speedup in DL4j on Spark on Ignite RDD -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Distributed Training in tensorflow

Dear Mehdi Sey First of all, we should have running Ignite cluster with a dataset loaded into caches. NOTE: This dataset could be reached via "from tensorflow.contrib.ignite import IgniteDataset" in your Jupiter Notebook. In the second, we shouldn't forget about tf.device("...") call The

Re: Ignite and spark for deep learning

Dear Mehdi Sey Yes, both platforms are used for in-memory computing, but they have different APIs and history of feature creation and different ways of integration with famous DL frameworks (like DL4j and TensorFlow). >From my point of view, you have no speed up in Ignite + Spark + DL4j

Re: JDK 11 support

I haven't any troubles with running Ignite 2.6 with JDK 8 and Ignite 2.7 with JDK 8,9. But a few weeks ago it [Ignite 2.6/Ignite 2.7) doesn't compile with JDK 11 (Oracle). -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Loading data from Spark Cluster to Ignite Cache to perform Ignite ML

Hi, Igniters! I am looking for a possibility to load data from Spark RDD or DataFrame to Ignite cache with next declaration IgniteCache dataCache to perform Ignite ML algorithms. As I understand the current mechanism of Ignite-Spark integration helps to store RDD/DF from Spark in Ignite to

Re: deep learning over apache ignite

I will try to answer these questions 1) We can using ignite as cache storage and preparing data for processing on dl4j? - yes you can, it's main case, to load data to ignite caches correctly 2) Job on spark can spawn hirarchy job for accelerate execution. - Ignite can compute piece pf code in

Re: HELLO WORLD GA EXAMPLE

Sorry for the long delay, I missed your answer Please, play with examples located here https://github.com/apache/ignite/tree/master/examples/src/main/java/org/apache/ignite/examples/ml For example, you could get

Re: deep learning over apache ignite

No, currently we doesn't work on integration bw DL4j and Ignite The first step can be getting data from IgniteCache and wrapping to tensor and putting to local instance of DL4j. But currently I could help with code review only not with the example building. First of all, we should understand

Re: deep learning over apache ignite

Hi, I'm a contributor of Ignite ML too and I'd like dl4j library too and have experience with that. I have a few experiments with integration but the main problem is tensor algebra implementation and effective Ignite cluster/caches usage. Could you provide a simple example of your draft ideas on

Re: Run Spark with Ignite Shared RDD on Large Volume of Data

I could make only simple advise: try Ignite KMeans clusterization over the data directly stored in Ignite Please, have look to example KMeans If you

Re: [ANNOUNCE] Apache Ignite 2.7.0 Released

Congratulate all Ignite users and especially the responsible person: @Nikolay Izhikov -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: When will Apache Ignite support Java 11?

A lot of answers here, but I'll put my 5 cents here. I'm going to test different Ignite modules like SQL, Core, ML, Streaming with Liberica JDK, Amazon Coretto and another JDKs based on OpenJDK personally. If I found any critical things, I post them tp the user or dev list. -- Sent from:

[ML][Community] Slides (eng) and video (eng) about Ignite ML module from JFuture conference

:) Please use the direct link to the video <https://www.youtube.com/watch?v=DmoMBsiHxf8> . Slides could be found here <https://speakerdeck.com/zaleslaw/nuances-of-machine-learning-with-ignite-ml> . The code of example run in this video could be found in our ML tutorial: Read Titanic da

Re: HELLO WORLD GA EXAMPLE

Dear @AlphaMufasaOmega could you please run another ML examples like KNNRegression example or LogisticRegression in ml-examples. Could you post the result here or in separate thread? Will it be the same problem or not? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Alternatives for XML configuration file(s)?

Of course, you can. I use Java Configuration on my project with Ignite and it works without any problems. I hope, that Ignite Core developers will add a few examples with Java config -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Blog post "Introduction to the Apache(R) Ignite™ community structure"

[For discussion] It's interesting to know more about possible conflicts: when a few persons are involved in contribution as a commiters, for example, and they have different opinions about next steps in roadmap implementation or about certain feature. How to measure correctly their weights? One of

Re: Heap size

Sometimes, for blobs and images with size around 100-1000 mbs I'm using G1 with increased heap region size via -XX:G1HeapRegionSize Also, if you have nodes with 8-24 cores, try to use CMS, it gives small second GC pause during "remark" phase when you don't change the graph of objects very often

Re: NPE exception in KMeansTrainer

Hi, try to play with current KMeans from master I hope this bug was detected correctly and fixed in https://issues.apache.org/jira/browse/IGNITE-9393 Could you post any results of your experiments here? -- Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: NPE exception in KMeansTrainer

Thank you, I think I've found this bug (or related to this) here https://issues.apache.org/jira/browse/IGNITE-9239 It will be delivered in 2.7 (Currently it's in master branch). To be sure 100% that the bug is closed, could @DocDVZ provide an approach of cache populating? I mean this cache