Re: Clarity on Ignite filesytem

2019-04-01 Thread Vladimir Ozerov
Hi Maros, Over the years of IGFS existence we came to conclusion that it doesn't fit to typical Ignite use cases well. Currently development of this component is frozen. Most likely it will be removed in nearest future. Hence, no plans to improve it or extend with new APIs. On Mon, Apr 1, 2019

Re: SQL Query plan confusion

2018-12-07 Thread Vladimir Ozerov
Hi Jose, I am a bit lost in two provided explains. Both "godd" and "bad" contain "k = 'trans.cust.first_name'" condition. Could you please confirm that they are correct? Specifically, I cannot understand why this condition is present in "good" explain. On Tue, Nov 27, 2018 at 12:06 PM joseheitor

Re: Difference between replicated and local cache mode regarding time execution of query

2018-09-07 Thread Vladimir Ozerov
Hi Moti, Could you please attach execution plans for both LOCAL and REPLICATED cases? On Wed, Sep 5, 2018 at 6:13 PM ilya.kasnacheev wrote: > Hello! > > Unfortunately I'm not aware why you see such a big difference in this case. > It should be comparable. Maybe SQL people will chime in? > >

Re: Ignite SQL Queries not getting all data back in ignite 2.4 and 2.6

2018-08-16 Thread Vladimir Ozerov
out that Ignite has nightly builds, so you can try > them instead of doing your own build as well. > > https://ignite.apache.org/download.cgi#nightly-builds > > D. > > On Thu, Aug 16, 2018 at 1:38 AM, Vladimir Ozerov > wrote: > >> Hi, >> >> There were

Re: Ignite SQL Queries not getting all data back in ignite 2.4 and 2.6

2018-08-16 Thread Vladimir Ozerov
Hi, There were a lot of changes in the product since 2.3 which may affect it. Most important change was baseline topology, as already mentioned. I am aware of a case when incorrect result might be returned [1], which is already fixed in *master*. Not sure if this is the same issue, but you may

Re: Ignite running on JDK10?

2018-08-15 Thread Vladimir Ozerov
HI, Please try adding add these flags to JVM startup arguments. On Fri, Aug 10, 2018 at 5:31 PM KJQ wrote: > As a note, I downgraded all of the Docker containers to use JDK 9 (9.0.4) > and > I still get the same problem running the SpringBoot 2 application. Running > in my IDE a test case

Re: Delete SQL is failing with IN clause for a table which has composite key

2018-04-12 Thread Vladimir Ozerov
Hi, The problem is already fixed [1]. Fix will be available in AI 2.5. [1] https://issues.apache.org/jira/browse/IGNITE-8147 On Wed, Apr 11, 2018 at 9:21 AM, Naveen wrote: > Hi > > These are the queries I have used, which I was getting the error every time > I

Re: SQL Transactional Support for Commit and rollback operation.

2018-04-11 Thread Vladimir Ozerov
Hi, Transactional SQL is extremely huge multi-man-year effort. We hope to have it in the product in Q3. We are working on it in normal (not "ASAP") mode, as our ultimate goal is great product. On Wed, Apr 11, 2018 at 2:32 PM, joseheitor wrote: > Yes - please clarify? >

Re: Upgrade from 2.1.0 to 2.4.0 resulting in error within transaction block

2018-04-05 Thread Vladimir Ozerov
at 12:21 PM, Yakov Zhdanov <yzhda...@apache.org> wrote: > Cross posting to dev. > > Vladimir Ozerov, can you please take a look at NPE from query processor > (see below - GridQueryProcessor.typeByValue(GridQueryProcessor.java:1901) > )? > > --Yakov > > 2018-0

Re: Affinity Key column to be always part of the Primary Key

2018-03-20 Thread Vladimir Ozerov
Because without AFFINITY KEY option we do not know order of fields within composite PK which is very important for index creation. вт, 20 марта 2018 г. в 19:58, Dmitriy Setrakyan <dsetrak...@apache.org>: > On Tue, Mar 20, 2018 at 2:09 PM, Vladimir Ozerov <voze...@gridgain.

Re: Affinity Key column to be always part of the Primary Key

2018-03-20 Thread Vladimir Ozerov
Internally Ignite is key-value storage. It use key to derive partition it belongs to. By default the whole key is used. Alternatively you can use @AffinityKey annotation in cache API or "affinityKey" option in CREATE TABLE to specify *part of the key* to be used for affinity calculation. Affinity

Re: query on BinaryObject index and table

2018-02-14 Thread Vladimir Ozerov
He Rajesh, Method CacheConfiguration.setIndexedTypes() should only be used for classes with SQL annotations. Since you operate on binary objects, you should use CacheConfiguration.setQueryEntity(), and define QueryEntity with all necessary fields. Also there is a property QueryEntity.tableName

Re: Cannot insert data into table using JDBC

2018-01-17 Thread Vladimir Ozerov
Hi Michael, The issue is almost fixed. The fix will be available as a part of Apache Ignite 2.4 release. On Tue, Dec 19, 2017 at 1:48 PM, Michael Jay <841519...@qq.com> wrote: > Hi,has it been solved or is it a bug? I just met with the same > problem.When I > set "streaming=false" ,it worked,

Re: BinaryObjectImpl.deserializeValue with specific ClassLoader

2018-01-15 Thread Vladimir Ozerov
The ticket is on the radar, but not in immediate plans. The problem might sounds simple at first glance, but we already spent considerable time on implementation and review, because we heavily rely on classes caching, and a lot of internal BinaryMarshaller infrastructure should be reworked to

Re: insert so slow via JDBC thin driver

2018-01-15 Thread Vladimir Ozerov
Streaming mode for thin JDBC driver is expected in Apache Ignite 2.5. Meanwhile you can load data using thick driver which support streaming, and then switch to thin driver for normal operations it you prefer it over thick one. On Fri, Dec 15, 2017 at 9:09 PM, Denis Magda

Re: How can I get Ignite security plugin to work with JDBC thin client?

2017-11-09 Thread Vladimir Ozerov
Hi Caleb, This appears to be a problem with our query execution engine, rather than with thin JDBC driver. I created a ticket to fix it [1]. [1] https://issues.apache.org/jira/browse/IGNITE-6856 On Tue, Oct 31, 2017 at 4:49 PM, Andrey Mashenkov < andrey.mashen...@gmail.com> wrote: > Caleb, > >

Re: Ignite 2.0.0 GridUnsafe unmonitor

2017-10-31 Thread Vladimir Ozerov
Guys, Printing a warning in this case is really strange idea. First, how would explain it in case of OPTIMISTIC/SERIALIZABLE transactions where deadlocks are impossible? Second, what would you do in case tow sorted maps are passed one by one in a transaction? User still may have a deadlock.

Re: Query performance against table with/out backup

2017-10-27 Thread Vladimir Ozerov
Hi, We know one serious source of slowdown when backups are enabled. See [1] and [2]. It will be fixed in 2.4. Vladimir. [1] https://issues.apache.org/jira/browse/IGNITE-6624 [2] https://issues.apache.org/jira/browse/IGNITE-6626 On Thu, Oct 19, 2017 at 9:29 PM, blackfield

Re: work around for problem where ignite query does not include objects added into Cache from within a transaction

2017-09-18 Thread Vladimir Ozerov
Hi, Looks like you need the feature which is simply not implement in Ignite at the moment. SELECTs are not transactional and there is not easy way to make not-yet-committed updates visible to them. Solution 1 doesn't work as there are not "temporal" caches in Ignite. Not-yet-committed updates are

Re: GROUP_CONCAT function is unsupported

2017-09-08 Thread Vladimir Ozerov
Hi, This function is not supported yet. No immediate plans to fix it as there are lots of tasks with higher priority and impact. But will definitely support it at some point. On Wed, Sep 6, 2017 at 11:38 AM, mhetea wrote: > Hello, > I saw that the ticket was not fixed.

Re: New API docs look for Ignite.NET

2017-09-06 Thread Vladimir Ozerov
Looks good to me. Clean layout, fast build - do we need anything else? On Tue, Sep 5, 2017 at 12:06 PM, Oleg Ostanin wrote: > Great news, thanks a lot! > > On Tue, Sep 5, 2017 at 11:47 AM, Pavel Tupitsyn > wrote: > >> DocFX takes around 30 seconds

Re: SQL query is slow

2017-09-04 Thread Vladimir Ozerov
Hi Mihaela, Index is not used in your case because you specify function-based condition. Usually this is resolved by adding functional index, but Ignite doesn't support it at the moment unfortunately. Is it possible to "materialize" the condition "POSITION ('Z',manufacturerCode)>0" as additional

Re: using hdfs as secondary file system

2017-06-14 Thread Vladimir Ozerov
Hi Antonio, Is it possible to attach logs from server nodes? -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/using-hdfs-as-secondary-file-system-tp13580p13696.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: odd query plan with joins?

2017-06-14 Thread Vladimir Ozerov
Hi, "1=1" is normal thing in the plan provided that index is used (see "/* PropertyCache.PROPERTY_CITY_ID_IDX: CITY_ID = 59053 */"). Is it possible to attach a reproducer? Vladimir. On Wed, Jun 14, 2017 at 5:10 AM, djardine wrote: > Hi, > I'm attempting to run a query that

Re: How to properly close igfs file after appending it

2017-06-09 Thread Vladimir Ozerov
Is it possible to provide a reproducer? On Fri, Jun 9, 2017 at 9:11 AM, ishan-jain wrote: > Ivan please ellaborate > > > > -- > View this message in context: http://apache-ignite-users. > 70518.x6.nabble.com/How-to-properly-close-igfs-file-after-appending-it- >

Re: Memory Leak or my Error?

2017-06-08 Thread Vladimir Ozerov
> > > > *Please consider your environmental responsibility before printing this > e-mail* > > This communication (including any attachments) may contain privileged or > confidential information of FlexValley SRL and is intended for a specific > individual. If you are not the

Re: Is it necessary to configure setting HBase which use Secondary File System?

2017-06-08 Thread Vladimir Ozerov
Hi Takashi, "igfs://" prefix should be used in your application code, in those places where data is accessed. It is illegal to change "hbase.wal.dir" property, as it breaks HBase internals. On Thu, Jun 8, 2017 at 6:30 AM, Takashi Sasaki wrote: > Hello, > > I used igfs://

Re: igfs-meta behavior when node restarts

2017-06-08 Thread Vladimir Ozerov
Hi, Is it possible to create isolated reproducer which will show gradual increase of failed operations over time? On Wed, Jun 7, 2017 at 6:39 AM, joewang wrote: > Would it be helpful if I uploaded logs collected from the cluster? I can > point to the time when the behavior

Re: Memory Leak or my Error?

2017-06-06 Thread Vladimir Ozerov
g them in > any manner, and you are hereby notified that any disclosure, copying, or > distribution of this communication, or the taking of any action based on > it, is strictly prohibited. > > > > *Da:* Vladimir Ozerov [mailto:voze...@gridgain.com] > *Inviato:* martedì 6 giugno 2017 15:02 &

Re: Memory Leak or my Error?

2017-06-06 Thread Vladimir Ozerov
Hi Devis, Can you show GC roots of these Session objects? On Tue, Jun 6, 2017 at 3:52 PM, Devis Balsemin < devis.balse...@flexvalley.com> wrote: > Hi, > > I’m using Ignite 1.7. > > I have a function nativeSQL that is called in many point of my programs. > > But after 2/3days I receive my OOM

Re: igfs-meta behavior when node restarts

2017-06-06 Thread Vladimir Ozerov
Hi, Ignite performs delete in a "soft" fashion: 1) When "remove" command is executed, we propagate it to the secondary file system; 2) For IGFS meta cache, we do not remove all records immediately, but rather execute a single "move" operation and move removed tree to a hidden "TRASH" folder. 3)

Re: Adding new fields to QueryEntity

2017-05-21 Thread Vladimir Ozerov
Denis, I think the question not about new indexes, but about new fields. The answer is - you must change QueryEntity manually still. сб, 20 мая 2017 г. в 16:25, Denis Magda : > Hi, > > You need to use CREATE index command in runtime to achieve that: >

Re: [ANNOUNCE] Apache Ignite 2.0.0 Released

2017-05-06 Thread Vladimir Ozerov
Paolo, Yes, there were braking API changes. Please refer to Migration Guide [1]. [1] https://cwiki.apache.org/confluence/display/IGNITE/Apache+Ignite+2.0+Migration+Guide On Sat, May 6, 2017 at 11:58 AM, Paolo Di Tommaso wrote: > Congrats people! > > The new ML Grid

Re: Integrate IGFS and Impala

2017-03-26 Thread Vladimir Ozerov
It looks like Impala expect some specific file system types. We can do nothing with it. IGFS will never extend "org.apache.hadoop.hdfs. DistributedFileSystem" because it is completely different implementation. Question should be addressed to Impala developers. On Sun, Mar 26, 2017 at 9:19 PM,

Re: Remote SPI with the same name is not configured

2017-03-02 Thread Vladimir Ozerov
Hi, Looks like some of your caches have *swapEnabled *property set to *true*. Please try setting *FileSwapSpaceSpi *explicitly in configuration of your nodes. Vladimir. On Thu, Mar 2, 2017 at 3:00 PM, ght230 wrote: > I start an Ingnite Server node first, Sometimes when I start

Re: @SqlQueryField on composite primary key

2017-02-16 Thread Vladimir Ozerov
zaid, In this case "id" column will be created. If you query this attribute, PersonPK will be returned. Sergi, This is in fact widely-used pattern which I saw many times in production systems. It allows user to have self-contained object. Otherwise one will have to introduce third value object

Re: ignite as Hadoop's Map Reduce engine - how it works?

2017-02-13 Thread Vladimir Ozerov
Hi, Our Hadoop Accelerator has its own shuffle algorithm. When jobs request arrives, we assign mappers and reducers to the most appropriate nodes in terms of data locality and available resources. Shuffle itself adds K-V pairs of local reducer to sorted collection right away. K-V pairs of

Re: IGFS Questions

2017-01-26 Thread Vladimir Ozerov
Hi, 1. Durability depends on IGFS mode. In PRIMARY there is no durability, In other modes IGFS will propagate writes to underlying file system (e.g. to HDFS). 2. Files stored in IGFS are always partitioned. You can specify number of backups and/or REPLICATED mode in data cache configuration. 3.

Re: Ignite cluster will be jammed for long-running queries

2017-01-25 Thread Vladimir Ozerov
IGNITE-4105 is a must ticket for Apache Ignite 2.0. On Wed, Jan 25, 2017 at 8:58 AM, ght230 wrote: > Yes, some queries will be long running, they are unavoidable. > > Query slow is acceptable, but the cluster jammed is a bit of trouble. > > I will be grateful if IGNITE-4105 can

Re: Abandon of the non-throwing version of C++ client API

2017-01-23 Thread Vladimir Ozerov
e of error-code methods. Why have you > changed you mind proposing to make the throwing version default one? Any > new tendency in C++ community? > > — > Denis > > On Jan 23, 2017, at 2:56 AM, Vladimir Ozerov <voze...@gridgain.com> wrote: > > +1 to Igor's idea

Re: How IGFS keep sync with HDFS?

2016-12-02 Thread Vladimir Ozerov
Hi, IGFS always propagate updates to HDFS immediately (or with slight delay in case of writes in DUAL_ASYNC mode). It doesn't remove data from memory after flushing it to HDFS. You can try configuring *org.apache.ignite.cache.eviction.igfs.IgfsPerBlockLruEvictionPolicy *to evict some data blocks

Re: BinaryObject pros/cons

2016-10-30 Thread Vladimir Ozerov
; < valentin.kuliche...@gmail.com> написал: > Vova, > > Why do we need to write zeros and nulls in the first place? What's the > value of having them in the byte array? > > -Val > > On Fri, Oct 28, 2016 at 1:18 AM, Vladimir Ozerov <voze...@gridgain.com> > wrote: &g

Re: BinaryObject pros/cons

2016-10-28 Thread Vladimir Ozerov
rect message marshaling and it > reduced overall traffic by around 30%. > > -Val > > > On Thu, Oct 27, 2016 at 2:21 PM, Vladimir Ozerov <voze...@gridgain.com> > wrote: > >> Hi, >> >> I am not very concerned with null fields overhead, because usually it

Re: BinaryObject pros/cons

2016-10-27 Thread Vladimir Ozerov
Hi, I am not very concerned with null fields overhead, because usually it won't be significant. However, there is a problem with zeros. User object might have lots of int/long zeros, this is not uncommon. And each zero will consume 4-8 additional bytes. We probably will implement special

Re: Does Apache Ignite use any proprietary api?

2016-10-19 Thread Vladimir Ozerov
Hi, Thank you for pointing this out. I asked Spring folks to clarify what do they mean. Vladimir. On Wed, Oct 19, 2016 at 10:11 AM, edwardk wrote: > Hi, > > Does Apache Ignite use any proprietary api within it. > > It seems the folks at Spring believe Apache Ignite uses

Re: MapReduce with Apache-Ignite

2016-10-15 Thread Vladimir Ozerov
Hi, Currently mapper output is stored in-memory only. We are working on spilling algorithm at the moment which will flush part of it to the disk according to some pre-configured thresholds. As per job logs, they are not supported at the moment by Ignite in the same way as it is done for Hadoop.

Re: MapReduce with Apache-Ignite

2016-09-29 Thread Vladimir Ozerov
Hi, 1. Mapper output should be written to the same place as if it job was run through native Apache Hadoop engine. Apache Ignite Hadoop Accelerator can work without IGFS at all. 2. Currently you will not see jobs in Resource Manager because they are executed through separate engine. We will

Re: Hive on Ignited- Yarn

2016-09-27 Thread Vladimir Ozerov
Hi, First of all we need to ensure that Ignite is really used in your case. Please advise which Hadoop deployment you use and where mapred-site.xml and core-site.xml files you mentioned are located? Vladimir. -- View this message in context:

Re: Is ignite installation required on all nodes of hadoop cluster

2016-09-27 Thread Vladimir Ozerov
Hi, Could you please provide more information on your use case? By default we recommend to have a single Ignite node started on every HDFS data node so that we can take advantage of data locality and minimize network traffic. But this is not strict requirement and final cluster deployment may

Re: Is ignite supported on Mapr File system (mfs)

2016-09-27 Thread Vladimir Ozerov
Hi, MapR support will be available in Apache Ignite 1.8. Vladimir. -- View this message in context: http://apache-ignite-users.70518.x6.nabble.com/Is-ignite-supported-on-Mapr-File-system-mfs-tp7914p7962.html Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Affinity and Topology versions

2016-09-26 Thread Vladimir Ozerov
Hi, Yes, Ignite guarantees that all nodes participating in transactions operate on the same topology version. I explained the problem with assumed race condition in relevant thread. Vladimir. -- View this message in context:

Re: Race condition with partition affinity mapping

2016-09-26 Thread Vladimir Ozerov
Hi. Looks like you observe "late affinity assignment" feature which is aimed to achieve better overall cluster performance during rebalance. Please see IgniteConfiguration.isLateAffinityAssignment() method for more information. Vladimir. -- View this message in context:

Re: IgniteException: No FileSystem for scheme: hdfs

2016-08-15 Thread Vladimir Ozerov
Hi Vikram, As you delegate to HDFS, you should have all relevant HDFS classes in classpath. Please make sure that you added all JAR files from the following directories to your startup script: /opt/cloudera/parcels/CDH-5.8.0-1.cdh5.8.0.p0.42/lib/hadoop/

Re: Cannot get the data in Tableau

2016-08-12 Thread Vladimir Ozerov
Hi Austin, Most probably Igor Sapego will be able to assist you. As far as I know he is unavailable during this week, but I hope he will be able to answer you in the beginning of the next week. Vladimir. On Tue, Aug 9, 2016 at 2:34 PM, austin solomon wrote: > Hi,

Re: MR jobs NOT starting in HIVE over IGFS

2016-08-12 Thread Vladimir Ozerov
Hi, Could you please subscribe to the user list so that we see your messages not only on the forum, but in the mailing lists as well? Instructions can be found here: http://ignite.apache.org/community/resources.html#mail-lists As per your problem, we need additional information. In particular:

Re: 1.7.0 release on mvn central?

2016-08-08 Thread Vladimir Ozerov
Hi, Yes, it was just released. E.g.: https://repository.apache.org/content/repositories/releases/org/apache/ignite/ignite-core/ It will take some for the release to appear on Maven repo search sites. Vladimir. On Mon, Aug 8, 2016 at 5:46 AM, barrettbnr wrote: > Hello

Re: question about ignite benchmark test

2016-08-04 Thread Vladimir Ozerov
Hi Kevin, This is the number of keys involved into PUT or GET operation. If there is 1 key, then *IgniteCache.get() *or *IgniteCache.put() *operation was used. If there are more keys, then we benchmarked *IgniteCache.getAll() *or *IgniteCache.putAll() *operations. On Thu, Aug 4, 2016 at 11:08

Re: Problem about Ignite Hadoop Accelerator MRv2 with CDH5.5.2 and kerberos

2016-07-28 Thread Vladimir Ozerov
Hi Mao, Ignite has special file system factory for Kerberized environments - *org.apache.ignite.hadoop.fs.KerberosHadoopFileSystemFactory*. Please try using it to resolve your problem. Should you have any questions please provide your current XML configuration and we will try to help you.

Re: 答复: ignite Connection refused exception

2016-07-21 Thread Vladimir Ozerov
Hi Kevin, The last error message suggests that it was a problem when trying to establish communication between two nodes on the same machine using shared memory mechanics. It is not clear now what is the reason for this, but I would try to disable shared memory and see if it helps. In XML file

Re: igfs.withAsync is still synchronous in most file operations?

2016-07-21 Thread Vladimir Ozerov
Hi Nicolae, This is not very easy question. First, "withAsync()" was introduced to IGFS mainly to support task execution (methods "execute(...)"). For now it is pretty clear that these methods are of very little use because there are much more convenient frameworks to achieve the same goals -

Re: How to use rest api to put an object into cache?

2016-06-28 Thread Vladimir Ozerov
Hi Kevin, Yes, currently REST protocol interpret everything as String. At this moment you can use *ConnectorMessageInterceptor *interface. Once you implement and configure it, you will start receiving callback for all keys and values passed back and forth. So you can encode you object as a String

Re: Can I get a better performance on BinaryMarshaller?

2016-06-24 Thread Vladimir Ozerov
> > I got it, thank you for your fast reply. There are some third party jar > files without source code, so we can't implementing the Binarylizable > interface directly. > > Thanks again. > > Lin. > > > > -- Original -- > *From: * &

Re: Where to watch the cache informations in IGFS when i use SecondryFileSystem

2016-06-15 Thread Vladimir Ozerov
Hi, Currently you can only access IGFS programatically, using *IgniteFileSystem.metrics() *method. If this is not an option for you, please attach the following: 1) Ignite configuration 2) Content of *core-site.xml* used for job execution. 3) Command line you use to start the job. Vladimir. On

Re: ignite in-memory sql query performance issue

2016-06-07 Thread Vladimir Ozerov
index, the time used to retrieve the query > reduced to 16ms. > > > > In order to speed up the sql queries, do I need to set all the possible > group indexes ? > > > > Best regards, > > Kevin > > > > *发件人:* Vladimir Ozerov [mailto:voze...@gridgain.com]

Re: ignite in-memory sql query performance issue

2016-06-06 Thread Vladimir Ozerov
Hi Kevin, Could you please provide the source code of SelectedClass and estimate number of entries in the cache? As Vladislav mentioned, most probably this is a matter of setting indexes on relevant fields. If you provide the source code, we will be able to give you exact example on how to do

Re: Data Streamer

2016-05-23 Thread Vladimir Ozerov
Hi Alexander, Please make sure that you flush data streamer before checking the "sum" value: fileStream.mapToInt(Integer::parseInt).forEach(line->streamer.addData(line,1L)); streamer.flush(); Vladimir. On Mon, May 23, 2016 at 10:35 AM, Александр Савинов wrote: > > Hello.

Re: Problem with getting started on Windows 7

2016-05-06 Thread Vladimir Ozerov
Hi Alexey, Please try unpacking Ignite to some directory other than "Program Files", so that there are no spaces in the path. Vladimir. On Fri, May 6, 2016 at 4:35 PM, Alexey wrote: > Hi, I am trying to start Ignite on Windows using cygwin. > I downloaded Apache Ignite as zip,

Re: Error starting c++ client node using 1.6

2016-05-04 Thread Vladimir Ozerov
Hi Murthy, The stack trace you provided shows that you still use old unpatched version. This could be observed by the line "at org.apache.ignite.Ignition.start(Ignition.java:322)". Patched version do not perform this call any more and goes directly to the "IgnitionEx.start()". You can see the fix

Re: Ignite WRite Behind

2016-05-04 Thread Vladimir Ozerov
Hi, writeAll() method is only executed for entries located in the same cache. As you have only 1 entry per cache in transaction, this is expected behavior that writeAll() is not called. Normally, you should make your logic dependent on whether write() or writeAll() is called, because it is up to

Re: Number of partitions of IgniteRDD

2016-04-29 Thread Vladimir Ozerov
method, but it has been overridden in > IgniteRDD. > > Regards, > Vij > > > On Friday, April 29, 2016 5:25 PM, Vladimir Ozerov <voze...@gridgain.com> > wrote: > > > Hi Vij, > > I am not quite uderstand where does method "getNumPartitions" came from. >

Re: Affinitykey is not working

2016-04-29 Thread Vladimir Ozerov
Hi, Could you please explain how do you detect a node to which key is mapped? Do you use Affinity API? Vladimir. On Fri, Apr 29, 2016 at 11:48 AM, nikhilknk wrote: > I used the below ToleranceCacheKey as the key . I want to keep all the > keys > whose marketSectorId is

Re: Issue with Java 8 datatype LocalDate while using IgniteRDD

2016-04-29 Thread Vladimir Ozerov
Hi Vij, Do you see any exception or some other kind of error? Please provide more error description. Vladimir. On Fri, Apr 29, 2016 at 2:47 PM, vijayendra bhati wrote: > Hi Guys, > > I am trying to store a object which contains object of type LocalDate > datatype of

Re: Ignite Installation with Spark under CDH

2016-04-29 Thread Vladimir Ozerov
Hi Michael, Ok, so it looks like the process didn't have enough heap. Thank you for your inputs about CDH configuration. We will improve our documentation based on this. Vladimir On Thu, Apr 28, 2016 at 5:15 PM, mdolgonos wrote: > Vladimir, > > I fixed this by

Re: Ignite Installation with Spark under CDH

2016-04-28 Thread Vladimir Ozerov
Hi Michael, Did you have a chance to analyze heap dump to understand what caused OOME? As per *IgniteConfiguration*, it is made non-serialziable intentionally, because we do not expect it to be passed over the wire. Could you please provide a stack trace where you see it is being serialized?

Re: Ignite Installation with Spark under CDH

2016-04-27 Thread Vladimir Ozerov
Hi Michael, I meant to echo to the console SPARK_CLASSPATH variable which you created in *spark-env.sh* following recommendations from Ignite docs. We need to ensure that JCache jar is included into it. BTW, do you see Ignite jars in " java.class.path" property? Vladimir. On Wed, Apr 27, 2016

Re: Error running nodes in .net and c++

2016-04-27 Thread Vladimir Ozerov
t 11:50 AM, Vladimir Ozerov <voze...@gridgain.com> wrote: > Hi Murthy, > > Please provide configs you used to start clients and servers. > > Vladimir. > > On Wed, Apr 27, 2016 at 5:45 AM, Murthy Kakarlamudi <ksa...@gmail.com> > wrote: > >> Can someone

Re: ignite nodes close exception when using cache -clear -c=@c7

2016-04-27 Thread Vladimir Ozerov
Hi Kevin, Looks like the topology got broken for some reason. Could you please attach logs from all nodes so that I can investigate it deeper? Vladimir. On Wed, Apr 27, 2016 at 1:46 PM, Zhengqingzheng wrote: > Hi there, > > When I tried to clear one specific cache,

Re: ODBC Driver?

2016-04-27 Thread Vladimir Ozerov
Hi Vij, It is not merged to master yet. We think this will happen in the nearest days. Vladimir. On Wed, Apr 27, 2016 at 3:34 PM, vijayendra bhati wrote: > If I build Ignite from nightly build , would I will be getting working > ODBC driver with it ? > I need to

Re: Ignite cache data size problem.

2016-04-27 Thread Vladimir Ozerov
. Because I assume > that each node can load different data simultaneously. Clearly, it is my > misunderstanding about increasing number of nodes. > > > > Best regards, > > Kevin > > > > *发件人:* Vladimir Ozerov [mailto:voze...@gridgain.com] > *发送时间:* 2016年4月27

Re: Gar file gives me java.lang.ClassNotFoundException

2016-04-27 Thread Vladimir Ozerov
Hi, Please provide the full stack trace and (if possible) code to reproduce the problem. Vladimir. On Wed, Apr 27, 2016 at 9:26 AM, mortias wrote: > Hi all, > > I'm trying to use a gar file to deploy my project however I get a > java.lang.ClassNotFoundException > > In

Re: Affinity Collocation

2016-04-27 Thread Vladimir Ozerov
Hi, There should not be any problems with config like this because all Organization entries will be located on all nodes in the cluster. Vladimir. On Wed, Apr 27, 2016 at 9:47 AM, Kamal C wrote: > What do you mean by cache configuration? > > If I go with the below

Re: Caused by: class org.apache.ignite.binary.BinaryInvalidTypeException:

2016-04-27 Thread Vladimir Ozerov
Hi. Yes, normally computations are performed on both primary and backup nodes. Vladimir. On Wed, Apr 27, 2016 at 10:19 AM, kcheng.mvp wrote: > Saying cache mode is "PARTITIONED", in this case both primary node and > backup > nodes would execute the same piece of code,

Re: Ignite Installation with Spark under CDH

2016-04-27 Thread Vladimir Ozerov
Hi Michael, Could you please print resulting value of the SPARK_CLASSPATH variable? Vladimir. On Tue, Apr 26, 2016 at 8:55 PM, mdolgonos wrote: > Vladimir, > I verified that the cache jar is in the Cloudera jars directory. All the > cache packages are also included

Re: Error running nodes in .net and c++

2016-04-26 Thread Vladimir Ozerov
ory.java:96) > at > org.apache.ignite.internal.processors.cache.GridCacheProcessor.createCache(GridCacheProcessor.java:1243) > at > org.apache.ignite.internal.processors.cache.GridCacheProcessor.prepareCacheStart(GridCacheProcessor.java:1638) > at > org.apache.ign

Re: Ignite cache data size problem.

2016-04-26 Thread Vladimir Ozerov
be clear, when you say log file, do you mean files located at > work/log/*? > > I use IgntieCache.loadCache(null, “select * from table”) to load all the > data. > > > > Best regards, > > Kevin > > > > *发件人:* Vladimir Ozerov [mailto:voze...@gridgain.com]

Re: Ignite cache data size problem.

2016-04-26 Thread Vladimir Ozerov
(314560K), > 0.0100806 secs] 315418K->35807K(1013632K), 0.0102764 secs] [Times: > user=0.00 sys=0.00, real=0.00 secs] > > 2016-04-26T18:56:33.184+0800: 4579.506: [GC (Allocation Failure) > 2016-04-26T18:56:33.184+0800: 4579.507: [DefNew: 281290K->1674K(314560K), > 0.011

Re: Error running nodes in .net and c++

2016-04-26 Thread Vladimir Ozerov
t;> >> Also Satya, it is possible for you to prepare small reproducible example >> that we could debug? >> >> >> On Mon, Apr 25, 2016 at 9:39 PM, Vladimir Ozerov <voze...@gridgain.com> >> wrote: >> >>> Alexey Kuznetsov, >&

Re: Ignite Installation with Spark under CDH

2016-04-26 Thread Vladimir Ozerov
Hi, This is pretty hard to say what is the root cause, especially in complex deployments like CDH. Most probably you JAR is packaged incorrectly because your application is able to load Ignite classes, but cannot load jcache API. Could you try to simply put cache-api-1.0.0.jar to all places and

Re: Detecting a node leaving the cluster

2016-04-26 Thread Vladimir Ozerov
into separate thread or thread pool. Vladimir. On Mon, Apr 25, 2016 at 2:54 PM, Ralph Goers <ralph.go...@dslextreme.com> wrote: > Great, thanks! > > Is listening for that the way you would implement what I am trying to do? > > Ralph > > On Apr 25, 2016, at 4:22 AM, Vladimir O

Re: Not able to save data in Ignite datagrid using JavaIgniteRDD

2016-04-25 Thread Vladimir Ozerov
Hi Vij, I am a bit confused - do you still have any problems? Because one message earlier you mentioned that finally it worked. Is any assistance still needed? Vladimir. On Thu, Apr 21, 2016 at 7:32 AM, vijayendra bhati wrote: > Hi Alexei, > > Here it is, the file

Re: Error running nodes in .net and c++

2016-04-25 Thread Vladimir Ozerov
Alexey Kuznetsov, Provided you have more expertise with POJO store, could you please advise what could cause this exception? Seems that POJO store expects some injection, which doesn't happen. Are there any specific requirements here? C++ node starts as a regular node and also use Spring.

Re: IgniteSemaphore problem

2016-04-25 Thread Vladimir Ozerov
Hi, There was several issues associated with distributed semaphore in 1.5.0 release. As far as I know most of the will be fixed in 1.6.0 release. Can you try building from current development master and see if the problem is still there? Vladimir. On Wed, Apr 20, 2016 at 1:00 PM, swoky

Re: Getting a null pointer exception when trying to bulk load a cache using LoadCache

2016-04-25 Thread Vladimir Ozerov
Hi, Could you please clarify what is ROCCache and ROCCacheConfiguration? And how many nodes do you have in topology? Vladimir. On Thu, Apr 21, 2016 at 10:34 AM, abhishek94 wrote: > 1 client node and 1 remote server node, have added the Pojo jar and MYSQl > jar >

Re: Ignite Installation with Spark under CDH

2016-04-25 Thread Vladimir Ozerov
Hi, Could you please clarify the exact problem? Do you see any exception? Vladimir. On Fri, Apr 22, 2016 at 4:57 PM, mdolgonos wrote: > I'm trying to install and integrate Ignite with Spark under CDH by > following > the recommendation at >

Re: Detecting a node leaving the cluster

2016-04-25 Thread Vladimir Ozerov
Ralph, EVT_NODE_LEFT and EVT_NODE_FAILED occur on local node. They essentially mean "I saw that remote node went down". Vladimir. On Sat, Apr 23, 2016 at 5:48 PM, Ralph Goers wrote: > Some more information that may be of help. > > Each user of a client application

Re: java.lang.OutOfMemoryError in normal Ignite usage

2016-04-25 Thread Vladimir Ozerov
Hi, Most probably, you just have insufficient heap, as queries require some heap space during execution. Probably you should give your node more heap and monitor it for some. If all is fine, you will see a saw-like pattern - when memory is allocated and then release. If there is a leak, you will

Re: Compute Grid API in C++/Scala

2016-04-25 Thread Vladimir Ozerov
Hi Arthi, I think C++ Compute API will be ready somewhere around 1.7 version. Exact timeline is not know for now, but I think this is a matter of several months. Vladimir. On Mon, Apr 25, 2016 at 8:47 AM, arthi wrote: > Thanks Igor & Val. > > Is there a

Re: Ignite cache data size problem.

2016-04-22 Thread Vladimir Ozerov
Hi, It looks like you have relatively small entries (somewhere about 60-70 bytes per key-value pair). Ignite also has some intrinsic overhead, which could be more than actual data in this case. However, I surely would not expect that it will not fit into 80GB. Could you please share your key and

Re: Performance Issue - Threads blocking

2016-04-22 Thread Vladimir Ozerov
Hi, Could you please explain why do you think that the thread is blocked? I see it is in a RUNNABLE state. Vladimir. On Fri, Apr 22, 2016 at 2:41 AM, ccanning wrote: > We seem to be having some serious performance issues after adding Apache > Ignite Local cache to our

Re: Map-reduce proceesing

2016-04-20 Thread Vladimir Ozerov
Hi, If you broadcast the job and want to iterate over cache inside it, then please make sure that you iterate only over local entries (e.g. IgniteCache.localEntries(), ScanQuery.setLocal(true), etc.). Otherwise your jobs will duplicate work and performance will suffer. Also please note that

  1   2   >