Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[Evgenii Zhuravlev]

Unfortunately, there is more than one file and it will look unreadable
here. All code can be accessed on github via the link I've provided earlier.

Evgenii

2018-01-30 7:10 GMT+03:00 Amir Akhmedov :

> You need to register a bean as follow
>
>  class="org.apache.ignite.transactions.spring.SpringTransactionManager">
> 
> 
> 
>
>
> Please check the link a sent you earlier it has tests with all required 
> configurations. It can serve as a good example.
>
>

Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[Amir Akhmedov]

You need to register a bean as follow







Please check the link a sent you earlier it has tests with all
required configurations. It can serve as a good example.

Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[rizal123]

Hi Amir,

How to registered Ignite's Spring Transaction Manager?
Would you mind share your code...

Thanks for your help Amir



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[rizal123]

Hi Evgenii ,

Yes i have try that. Without Spring Transaction.

private int sequenceManual(String seqName) {
int seq = 0;
Query query;
try {
em.getTransaction().begin();
query = this.em.createNativeQuery("UPDATE SEQUENCE SET SEQ_COUNT =
SEQ_COUNT + " + INCREMENT + " WHERE SEQ_NAME = '" +seqName+"'");
seq = (Integer) query.executeUpdate();
em.getTransaction().commit();

And also get an error:
java.lang.IllegalStateException: Not allowed to create transaction on shared
EntityManager - use Spring transactions or EJB CMT instead

Would you share your code here Evgenii ?
Thanks for your help...



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Apache Ignite & unixODBC and truncating text

[Rick Alexander]

Hey Igor or anyone who knows,

Looking to see when this will be released as well.

You mention 2.4 but the task says 2.5 release.

Please help! My project is at a stand-still...

On Mon, Jan 15, 2018 at 9:33 AM, bagsiur 
wrote:

> ok,
>
> I ask becouse I wont to be sure. Fix will be included in Ignite 2.4 or 2.5?
>
> On the ticket: https://issues.apache.org/jira/browse/IGNITE-7362 in
> details
> is write that fix this bug is planning for 2.5 version...
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Index question.

[Amir Akhmedov]

Ok, then it makes sense.
You cannot pass null into setIndexedTypes(). But if you don't put any
@QuerySqlField annotation in value class or declare fields/indexes through
Java API then no columns/indexes will be created in SQL storage.

On Mon, Jan 29, 2018 at 3:59 PM, Mikael  wrote:

> Thanks, the reason I need SQL is that my key is not a primitive, it's a
> class made of 2 int and one String and I need to query on parts of the key
> and not the entire key, like select all keys where one of the integers is
> equal to 55 for example.
>
> Mikael
>
> Den 2018-01-29 kl. 19:05, skrev Amir Akhmedov:
>
> Hi Mikael,
>
> 1. This is just a warning informing you that Ignite's object serialization
> will differ from yours Externalizable implementation. By default Ignite
> will serialize all fields in object and if you want to customize then you
> need implement Binarylizable interface or set custom serializer as stated
> in warning message.
> Even if you did not specify any @QuerySqlField in your object Ignite
> stores the whole serialized object in SQL table under _val field for
> internal usage.
> The open question is why do you need SQL if you are using only key based
> search? You can make exactly the same using Java Cache API.
>
> 2. You can leave Externalizable implementation in the class, it won't hurt.
>
> 3. Please check bullet #1, if you don't want indexes then you don't need
> create them.
>
> Thanks,
> Amir
>
>
>


-- 
Sincerely Yours Amir Akhmedov

Re: ClassCastException: DataPageIO cannot be cast to BPlusIO

[Arseny Kovalchuk]

Thanks Ilya, I'll try mentioned argument.

​
Arseny Kovalchuk

Senior Software Engineer at Synesis
skype: arseny.kovalchuk
mobile: +375 (29) 666-16-16
​LinkedIn Profile ​

On 29 January 2018 at 18:58, Ilya Kasnacheev 
wrote:

> Hello!
>
> I believe the root cause is this part of log:
>
> Caused by: java.nio.channels.ClosedByInterruptException
>   at 
> java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
>   at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:746)
>   at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.read(RandomAccessFileIO.java:62)
>   at 
> org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:322)
>   ... 60 more
>
>
> Here, a thread tht was using Persistence file channel got interrupted and
> it became dysfunctional after that. Then there's a multitude of problems
> caused by this.
>
> This should not generally happen under normal load, however if you're
> seeing such errors, consider setting the following Java argument for Ignite:
> -DIGNITE_USE_ASYNC_FILE_IO_FACTORY=true
>
> Please try to re-run with this setting, see if you still have any issues
>
> Regards,
>
> --
> Ilya Kasnacheev
>
> 2018-01-29 11:06 GMT+03:00 Arseny Kovalchuk :
>
>> Hi guys.
>>
>> We've got 2.3 with persistence enabled. 5 server nodes, 3 client query
>> nodes, 1 writer client node (it gets events from Kafka and puts them to
>> Ignite with key-val API). All client and server nodes are Pods in
>> Kubernetes.
>>
>> Cluster's worked well for a wile and then has stuck with the error which
>> sounded as
>>
>> Suppressed: org.apache.ignite.IgniteCheckedException: Failed to update
>> keys on primary node.
>> Caused by: java.lang.ClassCastException: org.apache.ignite.internal.pro
>> cessors.cache.persistence.tree.io.DataPageIO cannot be cast to
>> org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO
>>
>> At this stage the cluster doesn't accept data. Please take a look at
>> attached logs.
>>
>> client-log.txt - log of the client, which tries to put data to cluster
>> log-instance-N.txt - log of each server node. See ignite-instance-1.txt
>> for the error on server node.
>>
>> Any known issues about that?
>>
>> Sincerely,
>>
>> ​
>> Arseny Kovalchuk
>>
>> Senior Software Engineer at Synesis
>> skype: arseny.kovalchuk
>> mobile: +375 (29) 666-16-16 <+375%2029%20666-16-16>
>> ​LinkedIn Profile ​
>>
>
>

Re: Index question.

[Mikael]

Thanks, the reason I need SQL is that my key is not a primitive, it's a 
class made of 2 int and one String and I need to query on parts of the 
key and not the entire key, like select all keys where one of the 
integers is equal to 55 for example.


Mikael


Den 2018-01-29 kl. 19:05, skrev Amir Akhmedov:

Hi Mikael,

1. This is just a warning informing you that Ignite's object 
serialization will differ from yours Externalizable implementation. By 
default Ignite will serialize all fields in object and if you want to 
customize then you need implement Binarylizable interface or set 
custom serializer as stated in warning message.
Even if you did not specify any @QuerySqlField in your object Ignite 
stores the whole serialized object in SQL table under _val field for 
internal usage.
The open question is why do you need SQL if you are using only key 
based search? You can make exactly the same using Java Cache API.


2. You can leave Externalizable implementation in the class, it won't 
hurt.


3. Please check bullet #1, if you don't want indexes then you don't 
need create them.


Thanks,
Amir

Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[Amir Akhmedov]

Also, please make sure you registered Ignite's Spring Transaction Manager.
As an example for Ignite and Spring Transaction you can check
https://github.com/apache/ignite/tree/master/modules/spring/src/test/java/org/apache/ignite/transactions/spring

Thanks,
Amir

On Mon, Jan 29, 2018 at 9:38 AM, ezhuravlev 
wrote:

> Hi,
>
> I think now you need to start and commit transaction manually in your code:
> em.getTransaction().begin();
> em.getTransaction().commit();
>
> At least I've tried it some time ago and it worked. Please check this
> example on github:
> https://github.com/ezhuravl/ignite-eclipselink-example
>
> Evgenii
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Sincerely Yours Amir Akhmedov

Re: Index question.

[Amir Akhmedov]

Hi Mikael,

1. This is just a warning informing you that Ignite's object serialization
will differ from yours Externalizable implementation. By default Ignite
will serialize all fields in object and if you want to customize then you
need implement Binarylizable interface or set custom serializer as stated
in warning message.
Even if you did not specify any @QuerySqlField in your object Ignite stores
the whole serialized object in SQL table under _val field for internal
usage.
The open question is why do you need SQL if you are using only key based
search? You can make exactly the same using Java Cache API.

2. You can leave Externalizable implementation in the class, it won't hurt.

3. Please check bullet #1, if you don't want indexes then you don't need
create them.

Thanks,
Amir

RE: default location for ignite file db

[Rajesh Kishore]

Perfect Stan

On 29 Jan 2018 7:23 p.m., "Stanislav Lukyanov" 
wrote:

> Make sure you have persistence enabled as described at
> https://apacheignite.readme.io/docs/distributed-persistent-store.
>
> Also, make sure you don’t already have a custom setting for the
> StoragePath, WalPath or WalArchivePath.
>
>
>
> Making an environment variable visible to your process is OS-specific. On
> Windows you need to specify it in the System Properties dialog,
>
> and on Unix you need to use `export` command (you often put the `export`
> in your .bashrc). You should consult your system docs
>
> on doing so.
>
>
>
> If you don’t set IGNITE_HOME Ignite will try to find the installation
> based on your current directory and class path,
>
> and if it fails then a temporary directory directory will be created in
> the standard system location (e.g. /tmp on Linux)
>
> to store Ignite’s DB and other files such as logs.
>
>
>
> Stan
>
>
>
> *From: *Rajesh Kishore 
> *Sent: *29 января 2018 г. 15:18
> *To: *user@ignite.apache.org
> *Subject: *Re: default location for ignite file db
>
>
>
> Thanks Stan,
>
> I just extracted the zip , and I am starting the ignite server in embedded
> mode. I dont see DB files getting created at the ${IGNITE_HOME}/work/db.
> More specifically how embedded mode server would get to know about the env
> variable
> r ${IGNITE_HOME} , as in my classpath of application I am using only
> required jars of ignite
>
> -Rajesh
>
>
>
> On Mon, Jan 29, 2018 at 5:41 PM, Stanislav Lukyanov <
> stanlukya...@gmail.com> wrote:
>
> It will be under ${IGNITE_HOME}/work/db.
>
> Please refer to https://apacheignite.readme.io/docs/distributed-
> persistent-store which covers this topic in details.
>
>
>
> Stan
>
>
>
> *From: *Rajesh Kishore 
> *Sent: *29 января 2018 г. 15:04
> *To: *user@ignite.apache.org
> *Subject: *default location for ignite file db
>
>
>
> Hi All.
>
> How do we determine the default location for ignite file based DB
>
> Thanks in advance
>
> -Rajesh
>
>
>
>
>
>
>

Re: cache configuration without Key

[Rajesh Kishore]

Thanks Ilya it answers my question

On 29 Jan 2018 3:34 p.m., "ilya.kasnacheev" 
wrote:

> Hello Rajesh!
>
> In Ignite, cache (and SQL table) maps not to one object but to two - key
> and
> value.
>
> Keys are unique per cache per cluster. They act as primary key in SQL and
> as, well, keys in cache API.
>
> So yes, you need to specify key type (or compound object) too.
>
> Regards,
> Ilya.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: horizontal scaling

[Evgenii Zhuravlev]

Well, then it's not a horizontal scaling at all. I don't understand how do
you want to achieve more throughput while all your nodes will compete for
the same physical resources(CPU, network, IO).

If you want horizontal scaling, you need to run all nodes on separate
physical machines.

Evgenii

2018-01-29 19:03 GMT+03:00 Rajarshi Pain :

> Hi  Evgenii,
>
> Yes for now we are running all the nodes on same physical server.
>
> Thanks,
> Raj
>
> On Mon, Jan 29, 2018 at 7:03 PM, ezhuravlev 
> wrote:
>
>> Hi,
>>
>> > Right now we are running 4 nodes (same physical system)
>>
>> Do you mean that you run 4 nodes on the same physical machine and after
>> this, you add 2 additional nodes on the same physical machine?
>>
>> Evgenii
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>
>
>
> --
> Regards,
> Rajarshi Pain
>

Re: horizontal scaling

[Rajarshi Pain]

Hi  Evgenii,

Yes for now we are running all the nodes on same physical server.

Thanks,
Raj

On Mon, Jan 29, 2018 at 7:03 PM, ezhuravlev 
wrote:

> Hi,
>
> > Right now we are running 4 nodes (same physical system)
>
> Do you mean that you run 4 nodes on the same physical machine and after
> this, you add 2 additional nodes on the same physical machine?
>
> Evgenii
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Regards,
Rajarshi Pain

Re: ClassCastException: DataPageIO cannot be cast to BPlusIO

[Ilya Kasnacheev]

Hello!

I believe the root cause is this part of log:

Caused by: java.nio.channels.ClosedByInterruptException
at 
java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterruptibleChannel.java:202)
at sun.nio.ch.FileChannelImpl.readInternal(FileChannelImpl.java:746)
at sun.nio.ch.FileChannelImpl.read(FileChannelImpl.java:727)
at 
org.apache.ignite.internal.processors.cache.persistence.file.RandomAccessFileIO.read(RandomAccessFileIO.java:62)
at 
org.apache.ignite.internal.processors.cache.persistence.file.FilePageStore.read(FilePageStore.java:322)
... 60 more


Here, a thread tht was using Persistence file channel got interrupted and
it became dysfunctional after that. Then there's a multitude of problems
caused by this.

This should not generally happen under normal load, however if you're
seeing such errors, consider setting the following Java argument for Ignite:
-DIGNITE_USE_ASYNC_FILE_IO_FACTORY=true

Please try to re-run with this setting, see if you still have any issues

Regards,

-- 
Ilya Kasnacheev

2018-01-29 11:06 GMT+03:00 Arseny Kovalchuk :

> Hi guys.
>
> We've got 2.3 with persistence enabled. 5 server nodes, 3 client query
> nodes, 1 writer client node (it gets events from Kafka and puts them to
> Ignite with key-val API). All client and server nodes are Pods in
> Kubernetes.
>
> Cluster's worked well for a wile and then has stuck with the error which
> sounded as
>
> Suppressed: org.apache.ignite.IgniteCheckedException: Failed to update
> keys on primary node.
> Caused by: java.lang.ClassCastException: org.apache.ignite.internal.
> processors.cache.persistence.tree.io.DataPageIO cannot be cast to
> org.apache.ignite.internal.processors.cache.persistence.tree.io.BPlusIO
>
> At this stage the cluster doesn't accept data. Please take a look at
> attached logs.
>
> client-log.txt - log of the client, which tries to put data to cluster
> log-instance-N.txt - log of each server node. See ignite-instance-1.txt
> for the error on server node.
>
> Any known issues about that?
>
> Sincerely,
>
> ​
> Arseny Kovalchuk
>
> Senior Software Engineer at Synesis
> skype: arseny.kovalchuk
> mobile: +375 (29) 666-16-16
> ​LinkedIn Profile ​
>

Re: No transaction is currently active || Not allowed to create transaction on shared EntityManager - use Spring transactions or EJB CMT instead

[ezhuravlev]

Hi,

I think now you need to start and commit transaction manually in your code:
em.getTransaction().begin();
em.getTransaction().commit();

At least I've tried it some time ago and it worked. Please check this
example on github:
https://github.com/ezhuravl/ignite-eclipselink-example

Evgenii



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: default location for ignite file db

[Stanislav Lukyanov]

Make sure you have persistence enabled as described at 
https://apacheignite.readme.io/docs/distributed-persistent-store.
Also, make sure you don’t already have a custom setting for the StoragePath, 
WalPath or WalArchivePath.

Making an environment variable visible to your process is OS-specific. On 
Windows you need to specify it in the System Properties dialog,
and on Unix you need to use `export` command (you often put the `export` in 
your .bashrc). You should consult your system docs
on doing so. 

If you don’t set IGNITE_HOME Ignite will try to find the installation based on 
your current directory and class path,
and if it fails then a temporary directory directory will be created in the 
standard system location (e.g. /tmp on Linux)
to store Ignite’s DB and other files such as logs.

Stan 

From: Rajesh Kishore
Sent: 29 января 2018 г. 15:18
To: user@ignite.apache.org
Subject: Re: default location for ignite file db

Thanks Stan,

I just extracted the zip , and I am starting the ignite server in embedded 
mode. I dont see DB files getting created at the ${IGNITE_HOME}/work/db. More 
specifically how embedded mode server would get to know about the env variable 
r ${IGNITE_HOME} , as in my classpath of application I am using only required 
jars of ignite
-Rajesh

On Mon, Jan 29, 2018 at 5:41 PM, Stanislav Lukyanov  
wrote:
It will be under ${IGNITE_HOME}/work/db.
Please refer to 
https://apacheignite.readme.io/docs/distributed-persistent-store which covers 
this topic in details.
 
Stan
 
From: Rajesh Kishore
Sent: 29 января 2018 г. 15:04
To: user@ignite.apache.org
Subject: default location for ignite file db
 
Hi All.
How do we determine the default location for ignite file based DB
Thanks in advance
-Rajesh
 

Re: horizontal scaling

[ezhuravlev]

Hi,

> Right now we are running 4 nodes (same physical system)

Do you mean that you run 4 nodes on the same physical machine and after
this, you add 2 additional nodes on the same physical machine?

Evgenii



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: default location for ignite file db

[Mikael]

You can set it in your XML file or Ingnite configuration from Java.

    
    class="org.apache.ignite.configuration.DataStorageConfiguration">

  
  

  
  

  
  
    
    


Den 2018-01-29 kl. 13:18, skrev Rajesh Kishore:

Thanks Stan,

I just extracted the zip , and I am starting the ignite server in 
embedded mode. I dont see DB files getting created at the 
${IGNITE_HOME}/work/db. More specifically how embedded mode server 
would get to know about the env variable
r ${IGNITE_HOME} , as in my classpath of application I am using only 
required jars of ignite


-Rajesh

On Mon, Jan 29, 2018 at 5:41 PM, Stanislav Lukyanov 
> wrote:


It will be under ${IGNITE_HOME}/work/db.

Please refer to
https://apacheignite.readme.io/docs/distributed-persistent-store

which covers this topic in details.

Stan

*From: *Rajesh Kishore 
*Sent: *29 января 2018 г. 15:04
*To: *user@ignite.apache.org 
*Subject: *default location for ignite file db

Hi All.

How do we determine the default location for ignite file based DB

Thanks in advance

-Rajesh

Re: default location for ignite file db

[Rajesh Kishore]

Thanks Stan,

I just extracted the zip , and I am starting the ignite server in embedded
mode. I dont see DB files getting created at the ${IGNITE_HOME}/work/db.
More specifically how embedded mode server would get to know about the env
variable
r ${IGNITE_HOME} , as in my classpath of application I am using only
required jars of ignite

-Rajesh

On Mon, Jan 29, 2018 at 5:41 PM, Stanislav Lukyanov 
wrote:

> It will be under ${IGNITE_HOME}/work/db.
>
> Please refer to https://apacheignite.readme.io/docs/distributed-
> persistent-store which covers this topic in details.
>
>
>
> Stan
>
>
>
> *From: *Rajesh Kishore 
> *Sent: *29 января 2018 г. 15:04
> *To: *user@ignite.apache.org
> *Subject: *default location for ignite file db
>
>
>
> Hi All.
>
> How do we determine the default location for ignite file based DB
>
> Thanks in advance
>
> -Rajesh
>
>
>

RE: default location for ignite file db

[Stanislav Lukyanov]

It will be under ${IGNITE_HOME}/work/db.
Please refer to 
https://apacheignite.readme.io/docs/distributed-persistent-store which covers 
this topic in details.

Stan

From: Rajesh Kishore
Sent: 29 января 2018 г. 15:04
To: user@ignite.apache.org
Subject: default location for ignite file db

Hi All.
How do we determine the default location for ignite file based DB
Thanks in advance
-Rajesh

Re: starting Ignite server in embedded mode

[Rajesh Kishore]

Thanks it perfectly answers my query

Thanks,
Rajesh

On Mon, Jan 29, 2018 at 3:31 PM, Stanislav Lukyanov 
wrote:

> Hi,
>
>
>
> You can start Ignite in the same JVM as your application by using
> Ignition.start() method. You can start multiple Ignite instances,
>
> just create different IgniteConfiguration for them with different instance
> names.
>
>
>
> IgniteConfiguration serverConfig = new IgniteConfiguration()
>
> .setIgniteInstanceName("my-server")
>
> .setDiscoverySpi(new TcpDiscoverySpi()
>
> .setIpFinder(
>
> new TcpDiscoveryVmIpFinder()
>
> .setAddresses(Collections.singleton("
> 127.0.0.1:47500..47502"))
>
> ));
>
> Ignite server = Ignition.start(serverConfig);
>
>
>
> IgniteConfiguration clientConfig = new IgniteConfiguration()
>
> .setIgniteInstanceName("my-client")
>
> .setDiscoverySpi(new TcpDiscoverySpi()
>
> .setIpFinder(
>
> new TcpDiscoveryVmIpFinder()
>
> .setAddresses(Collections.singleton("
> 127.0.0.1:47500..47502"))
>
> ))
>
> .setClientMode(true);
>
> Ignite client = Ignition.start(clientConfig);
>
>
>
> // ... client and server run together here ...
>
>
>
> // don't forget to close ignite instances when your application
> finishes, or JVM won't be able to exit
>
> client.close();
>
> server.close();
>
>
>
> However, you generally shouldn't need two instances of ignite in the same
> application. If you want your application to natively use Ignite and
>
> have server running alongside it, you can just make API calls on the
> server instance, no need to create an additional client for that.
>
>
>
> Stan
>
>
>
> *From: *Rajesh Kishore 
> *Sent: *29 января 2018 г. 6:26
> *To: *user@ignite.apache.org
> *Subject: *starting Ignite server in embedded mode
>
>
>
> Hi All,
>
> We have requirement to manage Ignite server and client by the same
> application in embedded mode. Whats the way forward?
>
> Thanks,
>
> Rajesh
>
>
>

default location for ignite file db

[Rajesh Kishore]

Hi All.

How do we determine the default location for ignite file based DB

Thanks in advance
-Rajesh

RE: Question about data distribution

[svonn]

Yes - so I've been merely observing some 'unlucky' distribution if it's
solely chance-based. 
Since the real data will consist of thousands of measurements it should be
fine as it is, thanks!

Best regards
svonn



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Index question.

[Mikael]

Question 1:

I am using a class that have readObject/writeObject methods so I get 
this warning:


2018-01-24T16:15:49,248 [exchange-worker-#42] WARN : Some classes in 
query configuration cannot be written in binary format because they 
either implement Externalizable interface or have writeObject/readObject 
methods. Instances of these classes will be deserialized in order to 
build indexes. Please ensure that all nodes have these classes in 
classpath. To enable binary serialization either implement Binarylizable 
interface or set explicit serializer using 
BinaryTypeConfiguration.setSerializer() method: [class 
org.test.common.RtuProperties]


This class is used for the value in the cache, I set it with the 
setIndexedTypes( keyclass, valueclass) because I use it in SQL queries, 
but I have no @QuerySqlField annotations in that class and all SQL 
queries are done using the key only and not the value, will this cause 
any performance problems on indexing ? even if I have not specified any 
index on the value class will Ignite have to deserialize the objects 
when it builds the index data anyway ?


Question 2:

If so, is it ok to add the Binarylizable methods and interface and still 
keep the Externalizable implementation in the class?


Question 3:

If I don't want any indexed data on for example the value in a cache, 
but it is a primitive like a String for example, when I call 
setIndexedTypes( k, v) it will create an index on the key and value, the 
javadoc say nothing about using null for the value class for example, 
would this be ok ? if I have a huge number of entries I would think the 
performance would suffer if it has to update an index that is never used 
and it waste memory also.


Mikael

Re: Ignite query performance

[Andrey Mashenkov]

Hi,

1. Please, check if index is used for field you use in order by clause. You
can check this with EXPLAIN command [1].
If you see wrong index is used, then try hint [2] ignite for proper index.

2. FYI: there is limit and offset in query. For partitioned cache, Ignite
will fetch 200 (limit + offset) from each remote nodes and then merge
results and then apply limits.
So, actually to return 100 entries, Ignite have to fetch 200 entries per
node.

3. IN clause is used in query which may be inefficient due to lack of
underlying H2 optimizer.
Try to replace it with JOIN [3].

4. Also, conditions on Date\Time values can bring a slowdown (seems, due to
lack of H2).
It is recommended to use Long value (e.g. unix timestamp) instead of dates
if possible. It looks ok for historical date\time values.
Of cource, for time in future it is a bad idea as timezones can be changed
later.


[1]
https://apacheignite.readme.io/docs/sql-performance-and-debugging#section-using-explain-statement
[2]
https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-index-hints
[3]
https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-sql-performance-and-usability-considerations



On Sun, Jan 28, 2018 at 3:59 PM, suds123  wrote:

> I'm evaluating apache ignite for a use case very similar to the setup
> described at -
> https://dzone.com/articles/apache-ignite-how-to-read-
> data-from-persistent-sto
>
> 1. Instead of Person I'm using Transaction for the proof-of-concept, so I
> have followed the above link to create 3 source files - Transaction.java,
> TransactionStore.java and TransactionStoreExample.java.
>
> 2. I'm loading cache from an Oracle database. The Transaction table has 2
> million rows, each row having 30 fields and average row size is 300 bytes.
>
> 3. This is my query in the TransactionStoreExample.java
>
> QueryCursor> cursor = cache.query(new SqlFieldsQuery("select id,
> transactionAmount from Transaction where groupCode = ' ' and scheme = ' '
> and transactionDate > '' and transactionDate < '' and
> mid in ( )  order by transactionDate asc limit 100 offset 100"));
>
> 4. groupCode, scheme, transactionDate, and mid are indexed in model
> Transaction.java
>
> 5. My infrastructure is a single VM on Google Cloud (4 CPUs and 20 GB RAM)
> and as per dzone example I start a server instance with spring config file
> and a client instance where I run the TransactionStoreExample class.
>
> 6. I'm timing the query by printing timestamps before and after
> cursor.getAll().
>
> 7. For 2 million records I get a 1770 millis response time.
>
> My questions are -
>  - is this response timing expected for given setup?
>  - can this be improved?
>  - is there another way to capture query timings instead of printing
> timestamps before and after cursor.getAll()?
>
> Thanks
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>



-- 
Best regards,
Andrey V. Mashenkov

Re: cache configuration without Key

[ilya.kasnacheev]

Hello Rajesh!

In Ignite, cache (and SQL table) maps not to one object but to two - key and
value.

Keys are unique per cache per cluster. They act as primary key in SQL and
as, well, keys in cache API.

So yes, you need to specify key type (or compound object) too.

Regards,
Ilya.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: starting Ignite server in embedded mode

[Stanislav Lukyanov]

Hi,

You can start Ignite in the same JVM as your application by using 
Ignition.start() method. You can start multiple Ignite instances,
just create different IgniteConfiguration for them with different instance 
names.

IgniteConfiguration serverConfig = new IgniteConfiguration()
.setIgniteInstanceName("my-server")
.setDiscoverySpi(new TcpDiscoverySpi()
.setIpFinder(
new TcpDiscoveryVmIpFinder()

.setAddresses(Collections.singleton("127.0.0.1:47500..47502"))
));
Ignite server = Ignition.start(serverConfig);

IgniteConfiguration clientConfig = new IgniteConfiguration()
.setIgniteInstanceName("my-client")
.setDiscoverySpi(new TcpDiscoverySpi()
.setIpFinder(
new TcpDiscoveryVmIpFinder()

.setAddresses(Collections.singleton("127.0.0.1:47500..47502"))
))
.setClientMode(true);
Ignite client = Ignition.start(clientConfig);

// ... client and server run together here ...

// don't forget to close ignite instances when your application 
finishes, or JVM won't be able to exit
client.close();
server.close();

However, you generally shouldn't need two instances of ignite in the same 
application. If you want your application to natively use Ignite and
have server running alongside it, you can just make API calls on the server 
instance, no need to create an additional client for that.

Stan

From: Rajesh Kishore
Sent: 29 января 2018 г. 6:26
To: user@ignite.apache.org
Subject: starting Ignite server in embedded mode

Hi All,
We have requirement to manage Ignite server and client by the same application 
in embedded mode. Whats the way forward? 
Thanks,
Rajesh

Re: ScanQuery with predicate always returns empty result.

[Evgenii Zhuravlev]

Then you have one more mistake in your code.
Here is the example, which using Scan query with predicate:
https://github.com/apache/ignite/blob/master/examples/src/main/java/org/apache/ignite/examples/datagrid/CacheQueryExample.java

If you want the community to help you, please share full reproducer.

Evgenii

2018-01-29 11:48 GMT+03:00 Thomas Isaksen :

> Actually it didn’t help to get the cache with the correct types:
>
>
>
> Ignition.*setClientMode*(true);
> Ignite ignite = Ignition.*start*("/config/gatekeeper-token-config.xml");
> ignite.active(true);
>
> IgniteCache skipCache = ignite.cache("skipCache").
> withKeepBinary();
> ScanQuery scan2 = new ScanQuery<>((key, value) -> {
> *LOG*.debug("Key: {} - Value: {}", () -> key, () -> value);
> return true;
> });
> List> result2 =
> skipCache.query(scan2).getAll();
> *LOG*.debug("result2: {}", () -> result2);
>
>
>
> Now gives all rows in cache but lambda is ignored.
>
>
>
> 2018-01-29 09:43:22,274 DEBUG [no.toyota.gatekeeper.test.ApplicationTest]
> result2: [{1=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=846274760, hash=2107026056, EXPRESSION=/gatekeeper/credentials]},
> {2=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=1928312257, hash=677512603, EXPRESSION=/swagger]},
> {3=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=1092619788, hash=-422574611, EXPRESSION=/gatekeeper/test]},
> {4=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=1337835760, hash=-57658554, EXPRESSION=/foo/index.html]},
> {5=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=1807911555, hash=-340982938, EXPRESSION=/rainbow.zul]},
> {6=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3
> [idHash=324507700, hash=-341590176, EXPRESSION=/swagger.json]}]
>
>
>
> The same happens for static class as predicate but now the result is empty.
>
>
>
> static class Predicate implements IgniteBiPredicate
> {
> @Override
> public boolean apply(Long aLong, BinaryObject binaryObject)
> {
> *LOG*.debug("apply {} {}", aLong, binaryObject);
> return true;
> }
> }
>
>
>
> ScanQuery scan3 = new ScanQuery<>(new Predicate());
> List> result3 =
> skipCache.query(scan3).getAll();
> *LOG*.debug("result3 : {}", () -> result3);
>
>
>
> 2018-01-29 09:43:22,331 DEBUG [no.toyota.gatekeeper.test.ApplicationTest]
> result3 : []
>
>
>
>
>
> *From:* Thomas Isaksen [mailto:thomas.isak...@sysco.no]
> *Sent:* fredag 26. januar 2018 21.12
> *To:* user@ignite.apache.org; user@ignite.apache.org
> *Subject:* Re: ScanQuery with predicate always returns empty result.
>
>
>
> Ah! I make silly mistakes too often. I will give it a try. I will also
> make a separate class instead of an inner class. Thanks!
>
>
>
> --
>
> Thomas Isaksen
> --
>
> *From:* ezhuravlev 
> *Sent:* Friday, January 26, 2018 3:48:56 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: ScanQuery with predicate always returns empty result.
>
>
>
> Hi, in case when you use:
> ScanQuery scan = new ScanQuery<>((key, value) -> {
>
> System.out.println(key + “ = “ + value);
>
> return true;
>
> })
> I think the problem is that you created ScanQuery while
> your cache is IgniteCache. Scan query should be  String> too.
>
> In case of Inner class, it's trying to serialize your outer class, but it
> faces some problems in this process. Here you can create separate class
> instead of inner, it will help
>
> Evgenii
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: One problem about Cluster Configuration(cfg)

[Andrey Mashenkov]

Hi,

I've tried to run test several times with code\configs attached on my
notebook and still can't reproduce.
Please try to upgrade to the latest version of Ignite, if you think there
is a bug in Ignite.
I can't reproduce the issue neither on 1.9, nor 2.1 nor 2.3 versions.


On Mon, Jan 29, 2018 at 8:54 AM,  wrote:

> Dear Andrey,
>
>
>
> I'm attaching my version of One node(shell script, command: ./bin/ignite.sh
> config/example-cache.xml)
>
>
>
> and the other node (maven java and pom.xml, command: mvn compile exec:java
> -Dexec.mainClass=com.itri.ignite.IgniteTrigger).
>
>
>
> Would you be available to reproduce my problem via these attachments?
>
>
>
> In addition, I would appreciate it if you could offer me your version.
>
>
>
> Thanks
>
>
>
> Rick
>
>
>
> *From:* Andrey Mashenkov [mailto:andrey.mashen...@gmail.com]
> *Sent:* Friday, January 26, 2018 6:55 PM
> *To:* user@ignite.apache.org
> *Subject:* Re: One problem about Cluster Configuration(cfg)
>
>
>
> Rick,
>
>
>
> I can't reproduce the issue. Query"=select+*+from+String" works fine
> for me.
>
>
>
> On Fri, Jan 26, 2018 at 12:57 PM,  wrote:
>
> Hello Andrey,
>
>
>
> 1.  I have check the setting of cache is Partitioned and run two
> nodes again. These program codes are respectively as follows:
>
> One node(shell script)===
> ===
>
> 
>
> 
>
> 
>
> 
>
>   
>
> *  *
>
>   
>
> 
>
> java.lang.String
>
> java.lang.String
>
> 
>
>   
>
> 
>
> 
>
> 
>
> 
> ===
>
> The other node(maven java)=
> =
>
> *cacheConf**.setIndexedTypes(String.**class**, String.**class**)*;
>
>
>
> cacheConf.setCacheMode(CacheMode.*PARTITIONED*);
>
>
>
> *IgniteCache* cache = *igniteVar**.getOrCreateCache(**cacheConf**)*;
>
> 
> ===
>
> 2.  Yes, I can see the information of 2 servers* after I close the
> other node* in the following text.
>
> I thinks that both of the One node and the other node are linked, *but I
> can not put or get data into the oneCache.*
>
> 
> ===
>
> [26-01-2018 17:23:46][INFO 
> ][disco-event-worker-#28%null%][GridDiscoveryManager]
> Topology snapshot [ver=2, *servers=2, *clients=0, CPUs=4, heap=4.5GB]
>
>
>
> [26-Jan-2018 17:23:46][WARN 
> ][disco-event-worker-#28%null%][GridDiscoveryManager]
> Node FAILED: TcpDiscoveryNode [id=abe48607-8b7a-4413-9711-c7a14ecbd1a5,
> addrs=[0:0:0:0:0:0:0:1%lo, 127.0.0.1, 127.0.0.1], sockAddrs=[ubuntu/
> 127.0.0.1:47501, /0:0:0:0:0:0:0:1%lo:47501, /127.0.0.1:47501],
> discPort=47501, order=2, intOrder=2, lastExchangeTime=1516958573637,
> loc=false, ver=1.9.0#20170302-sha1:a8169d0a, isClient=false]
>
> 
> ===
>
> In addition, the following ports are created on localhost, as:
>
> *One node(shell script)*
>
> java 12266 root   TCP *:40447 (LISTEN)
>
> java 12266 root   TCP *:49187 (LISTEN)
>
> java 12266 root   TCP *:46451 (LISTEN)
>
> *java 12266 root  TCP *:47100 (LISTEN)*
>
> *java 12266 root  TCP *:11211 (LISTEN)*
>
> *java 12266 root  TCP *:8080 (LISTEN)*
>
> *java 12266 root  TCP *:47500 (LISTEN)*
>
> *The other node(maven java)*
>
> *java 12349 root  TCP *:47101 (LISTEN)*
>
> *java 12349 root  TCP *:11212 (LISTEN)*
>
> *java 12349 root  TCP *:8081 (LISTEN)*
>
> *java 12349 root  TCP *:47501 (LISTEN)*
>
>
>
> *However, I can not to print data from the oneCache **in the console of
> the maven java project.*
>
>
>
> 3.  I am not sure if url:http://127.0.0.1:8080/
> ignite?cmd=qryfldexe=100&
> *cacheName=oneCache=select+*+from+String*
>
> meet your command “SELECT _val FROM ” to show all data in oneCache via
> restful api or not.
>
>
>
> Rick
>
>
>
> *From:* Andrey Mashenkov [mailto:andrey.mashen...@gmail.com]
> *Sent:* Friday, January 26, 2018 4:52 PM
>
>
> *To:* user@ignite.apache.org
> *Subject:* Re: One problem about Cluster Configuration(cfg)
>
>
>
> Rick,
>
>
>
> 1. As you are able to put entry to cache, you should see cache.get()
> result in console.
>
> Please, check cache is not Local, but Partitioned or Replicated.
>
>
>
> 2. Also check the topology version message. You should see 2 servers in it.
>
>
>
> 3. Try to select value explicitly via _val field. E.g. "SELECT _val FROM
> "
>

Re: setNodeFilter throwing a CacheException

[dkarachentsev]

Hi Shravya,

To understand what's going on in your cluster I need full logs from all
nodes. Please, share all files, if it's possible.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Question about data distribution

[Stanislav Lukyanov]

AFAIU you simultaneously have about 1 million entries which correspond to 5-10 
groups (measurements).
Is that correct?
If so, that might be the reason of the distribution that you see. Even though 
you have a lot of entries,
you only have a few affinity mapped groups. Ignite tries keep all entries of 
the same measurement together,
and each measurement has ~50% chance to end up on the first or the second node. 
For 5 measurements, it results
in a ~3% chance that all data will be stored on a single node.
To have a better distribution among the cluster, try to restructure your data 
to have more distinct values
of the affinity mapped IDs. For example, you could split each measurement in, 
say, 1k or 10k batches, assign each
batch an ID and make that batch ID @AffinityKeyMapped instead of the 
measurementId.

Thanks,
Stan

From: svonn
Sent: 27 января 2018 г. 15:46
To: user@ignite.apache.org
Subject: RE: Question about data distribution

Hi!

My class for my keys looks like this.

private String deviceId;

@AffinityKeyMapped
private long measurementId;

private long timestamp;

public IgniteKey(String deviceId, long measurementId, long timestamp) {
this.deviceId = deviceId;
this.measurementId = measurementId;
this.timestamp = timestamp;

}


One device can have multiple measurements, but any calculation only requires
other entries from the same measurement as of now, thus only the
measurementId should be relevant.

One measurement contains 100k - 200k entries in one stream, and 500-1000 in
the other stream. Both streams use the same class for keys.

Whenever a new measurementId arrives I'm doing some output on the node it's
being processed on - I've had following case:
Measurement 1 (short M1) -> node1
M2 -> node1
M3 -> node2
M4 -> node1
M5 -> node1
M6 -> node1

I expected that even M2 will already be placed on node2 - however,
performance wise, I don't think either node is close to it's limit, I'm not
sure if that also relevant.
Due to the 5min expiry policy I can end up with one node having ~1 million
cache entries while the other one has 0.

- svonn





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: ScanQuery with predicate always returns empty result.

[Thomas Isaksen]

Actually it didn't help to get the cache with the correct types:

Ignition.setClientMode(true);
Ignite ignite = Ignition.start("/config/gatekeeper-token-config.xml");
ignite.active(true);

IgniteCache skipCache = 
ignite.cache("skipCache").withKeepBinary();
ScanQuery scan2 = new ScanQuery<>((key, value) -> {
LOG.debug("Key: {} - Value: {}", () -> key, () -> value);
return true;
});
List> result2 = skipCache.query(scan2).getAll();
LOG.debug("result2: {}", () -> result2);

Now gives all rows in cache but lambda is ignored.

2018-01-29 09:43:22,274 DEBUG [no.toyota.gatekeeper.test.ApplicationTest] 
result2: [{1=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 
[idHash=846274760, hash=2107026056, EXPRESSION=/gatekeeper/credentials]}, 
{2=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 [idHash=1928312257, 
hash=677512603, EXPRESSION=/swagger]}, 
{3=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 [idHash=1092619788, 
hash=-422574611, EXPRESSION=/gatekeeper/test]}, 
{4=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 [idHash=1337835760, 
hash=-57658554, EXPRESSION=/foo/index.html]}, 
{5=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 [idHash=1807911555, 
hash=-340982938, EXPRESSION=/rainbow.zul]}, 
{6=SQL_PUBLIC_SKIP_bd79b460_e9c9_425d_8ad5_9ebe2d32c6b3 [idHash=324507700, 
hash=-341590176, EXPRESSION=/swagger.json]}]

The same happens for static class as predicate but now the result is empty.

static class Predicate implements IgniteBiPredicate
{
@Override
public boolean apply(Long aLong, BinaryObject binaryObject)
{
LOG.debug("apply {} {}", aLong, binaryObject);
return true;
}
}

ScanQuery scan3 = new ScanQuery<>(new Predicate());
List> result3 = skipCache.query(scan3).getAll();
LOG.debug("result3 : {}", () -> result3);

2018-01-29 09:43:22,331 DEBUG [no.toyota.gatekeeper.test.ApplicationTest] 
result3 : []


From: Thomas Isaksen [mailto:thomas.isak...@sysco.no]
Sent: fredag 26. januar 2018 21.12
To: user@ignite.apache.org; user@ignite.apache.org
Subject: Re: ScanQuery with predicate always returns empty result.

Ah! I make silly mistakes too often. I will give it a try. I will also make a 
separate class instead of an inner class. Thanks!

--
Thomas Isaksen

From: ezhuravlev >
Sent: Friday, January 26, 2018 3:48:56 PM
To: user@ignite.apache.org
Subject: Re: ScanQuery with predicate always returns empty result.

Hi, in case when you use:
ScanQuery scan = new ScanQuery<>((key, value) -> {

System.out.println(key + " = " + value);

return true;

})
I think the problem is that you created ScanQuery while
your cache is IgniteCache. Scan query should be  too.

In case of Inner class, it's trying to serialize your outer class, but it
faces some problems in this process. Here you can create separate class
instead of inner, it will help

Evgenii



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/