Re: YCSB Benchmark

[Ivan Artiukhov]

Hi!

What results are you using for comparison? Please provide URL. There are
plenty of factors which affects performance. Hardware, number of server and
client nodes, Ignite memory mode, replication factor (number of backups),
number of benchmarking threads, data amount (record count) -- just to
mention a few.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Performance of SQL query by partial primary key

[Ray]

To answer my own question here, basically the index created on PK now is
useless according to this ticket.
https://issues.apache.org/jira/browse/IGNITE-8386

Ignite will perform a whole table scan when trying to execute the SQL query
I posted above unless an index identical to PK is created manually.

Please correct me if understand it wrong.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Failed to get page IO instance (page content is corrupted) after onenode failed when trying to reboot.

[Ray]

Hi Stan,

Thanks for the reply.

10.29.42.49 is a client node trying to connect to cluster to write data.
Don't know why the log shows it's not a client node.

Anyway, can you help take a look why restart 2-4 didn't work?
In restart 2-4 theres no server trying to join cluster as server nodes.

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to start an Ignite cluster with fixed number of servers?

[Ray]

Let's say I want to start an Ignite cluster of three server nodes with fixed
IP address and prevent other servers joining cluster as Ignite server nodes,
how can I do that?






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Error while loading data to cache

[Roman Guseinov]

Hi,

The error is clear, you try to cast BinaryObject to Customer type. It can
happen if you use CacheJdbcPojoStoreFactory for example. In this case, you
can use BinaryObject.field:

int id = ((BinaryObject)value).field("id");
return id == 11;

According to your filter, cachestore will fetch all data from 3rd party
store and put into cache only entries with id == 11. Is it what you
need?

Best Regards,
Roman



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Do I need CacheStoreFactory class and hibernate configuration on Remote Node?

[Вячеслав Коптилин]

Hello,

> Do I need CacheStoreFactory class and hibernate configuration on Remote
Node?
Yes, CacheStore classes must be in the classpath on all nodes (including
the client nodes)
For example, you can put them to '$IGNITE_HOME/libs' folder (a subfolder
can be created for convenience as well).

> Note that: I have tried peer class loading false and true, but nothing
changed.
Long story short, 'CacheStore' does not support zero deployment feature (it
is about Ignite Compute, in general)
Please take a look at this page:
https://apacheignite.readme.io/docs/zero-deployment

Thanks,
S.


пн, 17 сент. 2018 г. в 21:57, monstereo :

> Simply hibernate example:
>
> One node contains HibernateCacheStore and other configuration
> (hibernate.cfg.xml vs...)
> I have created that node and read database and write to cache (It works as
> it is) (cache name is "hibernateCache")
>
> Now I have created another node which has default configuration. Then in
> the
> main app just contains::
>
> public static void printCache(IgniteCache igniteCache){
> System.out.println("Print the all caches");
> Iterator> cacheIterator =
> igniteCache.iterator();
> while (cacheIterator.hasNext()){
> Cache.Entry cacheEntry = cacheIterator.next();
> System.out.println("Key: " + cacheEntry.getKey() + ", value:" +
> cacheEntry.getValue());
> }
> }
>
> public static void main...(){
>Ignite node = Ignition.start(defaultxmlpath)
> IgniteCache cache = node.getOrCreateCache("hibernateCache");
> System.out.println(cache.size());
> printCache(cache);
> }
>
> even if I work on local mode(and server nodes become 2 when I run the other
> node), I can not see the cache size on the screen and does not print the
> cache. It just waiting!!!
>
>
> Note that: I have tried peer class loading false and true, but nothing
> changed.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Do I need CacheStoreFactory class and hibernate configuration on Remote Node?

[monstereo]

Simply hibernate example:

One node contains HibernateCacheStore and other configuration
(hibernate.cfg.xml vs...)
I have created that node and read database and write to cache (It works as
it is) (cache name is "hibernateCache")

Now I have created another node which has default configuration. Then in the
main app just contains::

public static void printCache(IgniteCache igniteCache){
System.out.println("Print the all caches");
Iterator> cacheIterator =
igniteCache.iterator();
while (cacheIterator.hasNext()){
Cache.Entry cacheEntry = cacheIterator.next();
System.out.println("Key: " + cacheEntry.getKey() + ", value:" +
cacheEntry.getValue());
}
}

public static void main...(){
   Ignite node = Ignition.start(defaultxmlpath)
IgniteCache cache = node.getOrCreateCache("hibernateCache");
System.out.println(cache.size());
printCache(cache);
}

even if I work on local mode(and server nodes become 2 when I run the other
node), I can not see the cache size on the screen and does not print the
cache. It just waiting!!!


Note that: I have tried peer class loading false and true, but nothing
changed.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: WARN TcpDiscoverySpi:135 - Timed out waiting for message to be read(most probably....)

[monstereo]

I think you are right, when I killed some jar process on the systems, then it
does not give errors



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Ignite Cluster: Cache Misses happening for existing keys

[HEWA WIDANA GAMAGE, SUBASH]

With your idea of that race condition, we decided to lock the get, and put for 
a given key on app level, and keep using the JCache API. So far for 250 tps 
with 3 nodes, everything looks good.

Thank you very much!

From: Вячеслав Коптилин [mailto:slava.kopti...@gmail.com]
Sent: Thursday, September 13, 2018 9:01 AM
To: user@ignite.apache.org
Subject: Re: Ignite Cluster: Cache Misses happening for existing keys

This email is from an external source - exercise caution regarding links and 
attachments. Please visit cybersecurity.fmr.com/phishing/ to learn more.

Hi,

You can try with using EntryProcessor. Please take a look at the following 
example:

public class EntryUpdater implements EntryProcessor {
@Override
public String process(MutableEntry entry, Object... 
arguments) throws EntryProcessorException {
if (!entry.exists()) {
// Cache does not contain a mapping for the given key.

String newValue = getNewValue();

entry.setValue(newValue);
}
else {
// The entry is already exists, just return the existing value.
}

return entry.getValue();
}
}

private static String processKey(String key) {
return cache.invoke(key, new EntryUpdater());
}

[1] 
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#invoke-K-org.apache.ignite.cache.CacheEntryProcessor-java.lang.Object...-

Thanks,

S.

чт, 13 сент. 2018 г. в 15:06, HEWA WIDANA GAMAGE, SUBASH 
mailto:subash.hewawidanagam...@fmr.com>>:
Thank you for the prompt response.

I agree there’s a race condition here. But the problem of the suggestion is, 
“getNewValue()” will get called even for cache hits. This getNewValue 
eventually calling some backend legacy API and the caching layer is setup to 
reduce the load for it.

From: Вячеслав Коптилин 
[mailto:slava.kopti...@gmail.com]
Sent: Thursday, September 13, 2018 4:08 AM
To: user@ignite.apache.org
Subject: Re: Ignite Cluster: Cache Misses happening for existing keys

This email is from an external source - exercise caution regarding links and 
attachments. Please visit 
cybersecurity.fmr.com/phishing/ to 
learn more.

Hello,

Your code contains the following obvious race:

String value = cache.get(key);
if (value == null) {
cache.put(key, getNewValue());
LOG.info("Cache put key={} ", key);
}

It's easy to imagine that two threads managed to execute get() method and got 
null value before put() operation is called.

Please try to modify the code as follows:

if (cache.putIfAbsent(key, getNewValue())) {

LOG.info("Cache put key={} ", key);

}

else {

LOG.info("Cache hit key={}", key);

}



[1] 
https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/IgniteCache.html#putIfAbsent-K-V-



Thanks,

S.

чт, 13 сент. 2018 г. в 6:29, HEWA WIDANA GAMAGE, SUBASH 
mailto:subash.hewawidanagam...@fmr.com>>:
Hi all,
We’re observing this in a 3 node server cluster(3 separate JVMs, in 3 separate 
VMs within same datacenter, node to node latency is 2-3 milliseconds within the 
network).

As with following code is wrapped inside an http API, and that API is getting 
called by a 5000 users ramping up(60seconds) forever, hitting 3 nodes in a 
round robin manner.

With this, for same cache key, I could see more than one “Cache put key=” log 
appearing with in 15mins window(actually I am getting this duplicate put logs 
after 2-3mins of the load test)..

For the SAME cache key, there cannot be more than one put within 15mins. Based 
on cache size, it’s well below eviction size, and since it’s well within the 
expiry window, looks to me some timing issue when replicating the cache between 
the nodes.

Time between same key cache put logs is about 8-12 seconds usually.  Am I doing 
something wrong here ? Any way we can make a cache.put operation synchronously 
complete only upon a full node replication (not quite sure whether it helps 
though)?

Version: Ignite 1.9

Code to create the cache

IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setDiscoverySpi(getDiscoverySpi()); // static ip list on tcp discovery
cfg.setClientMode(false);
cfg.setIncludeEventTypes(EventType.EVT_NODE_SEGMENTED,EventType.EVT_NODE_FAILED);
Ignite ignite = Ignition.start(cfg);

ignite.events().localListen(event -> {
LOG.info("Cache event received: {} ", event);
return true;},EventType.EVT_NODE_SEGMENTED, EventType.EVT_NODE_FAILED);


CacheConfiguration cc  = new CacheConfiguration<>();
cc.setName("mycache1");
cc.setExpiryPolicyFactory(CreatedExpiryPolicy.factoryOf(new 
Duration(TimeUnit.MINUTES, 15)));
cc.setCacheMode(CacheMode.REPLICATED);
  LruEvictionPolicy evictionPolicy = new LruEvictionPolicy();
  evictionPolicy.setMaxMemorySize(500 * 1024 * 1024);
cc.setEvictionPolicy(evictionPolicy);

IgniteCache cache = 

Error while loading data to cache

[Skollur]

I am trying to load data to cache using below code and seeing an error.


ignite.cache("CustomerCache").loadCache(new IgniteBiPredicate()
{
@Override
public boolean apply(Object key, Object value) {
if(((Customer) value).getId()==11);
return true;
}
});

ERROR:-

Caused by: java.lang.ClassCastException:
org.apache.ignite.internal.binary.BinaryObjectImpl cannot be cast to
com.idb.cache.model.Customer
at
com.idb.cache.load.LoadIdbIgniteCaches$1.apply(LoadIdbIgniteCaches.java:36)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.loadEntry(GridDhtCacheAdapter.java:639)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Unreasonable segmentation in Kubernetes on one node reboot

[Maxim.Pudov]

Hi, 
Can you share the full logs from all nodes? Looks like everything should
work just fine.

Maxim.




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

RE: Failed to get page IO instance (page content is corrupted) after onenode failed when trying to reboot.

[Stanislav Lukyanov]

I see a node in the topology flacking up and down every minute in the 
restart1.log:
 TcpDiscoveryNode [id=d6e52510-3380-4258-8a8e-798640b1786c, addrs=[10.29.42.49, 
127.0.0.1], sockAddrs=[/10.29.42.49:47500, /127.0.0.1:47500], discPort=47500, 
order=596, intOrder=302, lastExchangeTime=1537154393454, loc=false, 
ver=2.6.0#20180710-sha1:669feacc, isClient=false]

In other logs I also see different node IDs.

Try finding what is the node on 10.29.42.49 that connects to your cluster.

In the last attempt you’re getting a page corruption which is not supposed to 
happen in any case,
but perhaps this is some bug that’s already fixed in the latest versions.

Stan

From: Ray
Sent: 17 сентября 2018 г. 13:10
To: user@ignite.apache.org
Subject: Failed to get page IO instance (page content is corrupted) after 
onenode failed when trying to reboot.

I have a three nodes Ignite 2.6.0 cluster with native persistence enabled.
Here's the config

http://www.springframework.org/schema/beans;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="
   http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans.xsd;>































node1:49500
node2:49500
node3:49500














One node failed when trying to process a long running sql, the detailed log
can be found in attachment with filename nodefail.log.

The other two nodes may have some new data coming in when one node in failed
state.
When I try to reboot this server after several hours, first it got stuck for
one hour with "Failed to wait for partition map exchange" exception.
The detailed log can be found in attachment with filename restart1.log.

So I try to reboot this server again but got
"org.apache.ignite.spi.IgniteSpiException: Node with set up BaselineTopology
is not allowed to join cluster without one:" exception.
The detailed log can be found in attachment with filename restart2.log.

So I try to reboot the whole cluster by starting failed node first, but I
got "Unable to await partitions release latch within timeout:" exception
when two other servers are started.
The detailed log can be found in attachment with filename restart3.log.

So I try to reboot this server again, but I got "Failed to get page IO
instance (page content is corrupted)" exception.
The detailed log can be found in attachment with filename restart4.log.

>From this point on, the cluster is in non-recoverable state.
Please advice me how to avoid this situation and how to recover data.
The log of failed node is in log.zip.
The other two files are logs for two good nodes.

log.zip   
log-goodNode.zip
  
log-GoodNode2.zip
  






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Message grid failure due to userVersion setting

[Dave Harvey]

I probably did not explain this clearly.  When sending a message from
server to client using the message grid, from a context unrelated to any
client call, the server, as you would expect uses its installed libraries,
and userVersion 0.For some reason, when the client receives this
message, it require that the user version match it's current user version.

The use case is we have a stable set of libraries on the server, and the
server wants to send a topic based message to the client, using only the
type "String".   Unrelated to this, the client is using the compute grid,
where P2P is used, but that is interfering with basic functionality.
This,  IGNITE-7905  ,
and the paucity  of  results when I google for "ignite userVersion"  makes
it clear that shooting down classes in CONTINUOUS mode with userVersion is
not completely thought through.  We certainly never want to set a
userVersion on the servers.

The documentation for P2P says:
"

   1. Ignite will check if class is available on local classpath (i.e. if
   it was loaded at system startup), and if it was, it will be returned. No
   class loading from a peer node will take place in this case."

Clearly, java.lang.String is on the local class path.So it seems like a
user version mismatch should not be a reason to reject a class that is on
the local classpath.

On Mon, Sep 17, 2018 at 11:01 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> I think that Ignite cannot unload old version of code, unless it is loaded
> with something like URI deployment module.
> Version checking is there but server can't get rid of old code if it's on
> classpath.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 17 сент. 2018 г. в 16:47, Dave Harvey :
>
>> We have a client that uses the compute grid and message grid, as well as
>> the discovery API.  It communicates with a server plugin.   The cluster is
>> configured for CONTINUOUS peer class loading.  In order to force the proper
>> code to be loaded for the compute tasks, we change the user version, e.g.,
>> to 2.
>>
>> If the server sends the client a message on the message grid, using
>> java.lang.string, the client fails because the user version sent for
>> java.lang.string is 0, but the client insists on 2.
>>
>> How is this supposed to work?   Our expectation was that the message grid
>> should not be affected by peer class loading settings.
>>
>>
>>
>>
>> *Disclaimer*
>>
>> The information contained in this communication from the sender is
>> confidential. It is intended solely for use by the recipient and others
>> authorized to receive it. If you are not the recipient, you are hereby
>> notified that any disclosure, copying, distribution or taking action in
>> relation of the contents of this information is strictly prohibited and may
>> be unlawful.
>>
>> This email has been scanned for viruses and malware, and may have been
>> automatically archived by *Mimecast Ltd*, an innovator in Software as a
>> Service (SaaS) for business. Providing a *safer* and *more useful* place
>> for your human generated data. Specializing in; Security, archiving and
>> compliance. To find out more Click Here
>> .
>>
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

Re: Loading Cache

[Skollur]

Tried below code, but code is not complaining in IDE..Keep seeing error
message asking to change the method loadcache() to method localloadCache().
When changed to method localloadcache(), asking to change method to
loadCache().


ignite.cache("CustomerCache").loadCache(new
IgniteBiPredicate(){
@Override
public boolean apply(Object key, Customer 
customer) {
return 
customer.field("dwTimestamp").after(fromDate) &&
customer.field("dwTimestamp").after(toDate);
}}, null);



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: WARN TcpDiscoverySpi:135 - Timed out waiting for message to be read(most probably....)

[Evgenii Zhuravlev]

It's possible that you have some application that listens to 47504 port and
doesn't read anything from the socket. I'd recommend checking which
application bound to this port.

Evgenii

пн, 17 сент. 2018 г. в 17:02, monstereo :

> Most the time, it happens when starting the first ignite node
>
> I can not share logs at this time.
>  But simply it prints that warning forever.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Using compute().call from a Service has P2P class loading feature?

[ezhuravlev]

Hi,

Are you sure that you use proper service? I mean from the client node. I've
just checked this case and everything works fine.

Evgenii



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: SQL query and Indexes architecture

[Ilya Kasnacheev]

Hello!

I recommend starting with H2TreeIndex class. Maybe dropping mails on
developer list with precise questions.

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 16:25, eugene miretsky :

> Thanks!
>
> I am curious about the process of loading data from Ignite to H2 on the
> fly, as H2 creating indexes but storing them in Ignite. Can you point me to
> some JIRAs that discuss it, or which part of the code is responsible for
> that?
>
> On Mon, Sep 17, 2018 at 9:18 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> 1. 1. H2 executes the query, during which it has to load rows from
>> tables, and Ignite does the row loading part. Then Ignite will collect
>> query results on all nodes and aggregate them on a single node.
>> 1. 2. Index is created by H2, but it is stored in Ignite pages (?).
>> 2. Maybe you're right, I have to admit I'm unfamiliar with precise
>> details here.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 17 сент. 2018 г. в 16:02, eugene miretsky > >:
>>
>>> Thanks!
>>>
>>>
>>>1.
>>>1.  "Ignite feeds H2 rows that it asks for, and H2 creates indexes
>>>   on them and executes queries on them." - what exactly do you mean by 
>>> that?
>>>   Do you mean that all parts of a query that use indexes are executed 
>>> by H2,
>>>   then the actual data is retrieved from Ignite pages, and the final
>>>   (non-indexed) parts of the query executed by Ignite?
>>>   2.  What happens when I create an index on a new column? Is the
>>>   index created in Ignite (and stored in Ignite pages?), or is it 
>>> created in
>>>   H2?
>>>2.  The reason I was asking about AFFINITY_KEY, _key_PK and
>>>_key_PK_hash indexed is that in this   code
>>>
>>> 
>>>  it
>>>looks like they are created in H2
>>>
>>>
>>>
>>> On Mon, Sep 17, 2018 at 8:36 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 1. H2 does not store data but, as far as my understanding goes, it
 created SQL indexes from data. Ignite feeds H2 rows that it asks for, and
 H2 creates indexes on them and executes queries on them.
 2. Ignite always has special index on your key (since it's a key-value
 storage it can always find tuple by key). Ignite is also aware of key's
 hash code, and affinity key value always maps to one partition of data (of
 1024 by default). Those are not H2 indexes and they're mostly used on
 planning stage. E.g. you can map query to one node if affinity key is
 present in the request.
 3. Data is brought onto the heap to read any fields from row. GROUP BY
 will hold its tuples on heap. Ignite has configurable index inlining where
 you can avoid reading objects from heap just to access indexed fields.
 4. With GROUP BY, lazy evaluation will not help you much. It will still
 have to hold all data on heap at some point. Lazy evaluation mostly helps
 with "SELECT * FROM table" type queries which provide very large and boring
 result set.

 Hope this helps.
 --
 Ilya Kasnacheev


 пт, 14 сент. 2018 г. в 17:39, eugene miretsky <
 eugene.miret...@gmail.com>:

> Hello,
>
> Trying to understand how exactly SQL queries are executed in Ignite. A
> few questions
>
>
>1. To what extent is H2 used? Does it store the data? Does it
>create the indexes? Is it used only for generating execution plans? I
>believe that all the data used to be stored in H2, but with the new 
> durable
>memory architecture, I believe that's no longer the case.
>2. Which indexes are used? Ignite creates  B+ tree indexes and
>stores them in Index pages, but I also see AFFINITY_KEY, _key_PK and
>_key_PK_hash indexes created in H2.
>3. When is data brought onto the heap? I am assuming that groupby
>and aggregate require all the matching queries to first be copied from
>off-heap to heap
>4. How does lazy evaluation work? For example, for group_by, does
>it bring batches of matching records with the same group_by key onto 
> the
>heap?
>
> I am not necessarily looking for the exact answers, but rather pointer
> in the right direction (documentation, code, jiras)
>
> Cheers,
> Eugene
>

Re: Message grid failure due to userVersion setting

[Ilya Kasnacheev]

Hello!

I think that Ignite cannot unload old version of code, unless it is loaded
with something like URI deployment module.
Version checking is there but server can't get rid of old code if it's on
classpath.

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 16:47, Dave Harvey :

> We have a client that uses the compute grid and message grid, as well as
> the discovery API.  It communicates with a server plugin.   The cluster is
> configured for CONTINUOUS peer class loading.  In order to force the proper
> code to be loaded for the compute tasks, we change the user version, e.g.,
> to 2.
>
> If the server sends the client a message on the message grid, using
> java.lang.string, the client fails because the user version sent for
> java.lang.string is 0, but the client insists on 2.
>
> How is this supposed to work?   Our expectation was that the message grid
> should not be affected by peer class loading settings.
>
>
>
>
> *Disclaimer*
>
> The information contained in this communication from the sender is
> confidential. It is intended solely for use by the recipient and others
> authorized to receive it. If you are not the recipient, you are hereby
> notified that any disclosure, copying, distribution or taking action in
> relation of the contents of this information is strictly prohibited and may
> be unlawful.
>
> This email has been scanned for viruses and malware, and may have been
> automatically archived by *Mimecast Ltd*, an innovator in Software as a
> Service (SaaS) for business. Providing a *safer* and *more useful* place
> for your human generated data. Specializing in; Security, archiving and
> compliance. To find out more Click Here
> .
>

RE: IGNITE-8386 question (composite pKeys)

[Stanislav Lukyanov]

Hi,

The thing is that the PK index is currently created roughly as
CREATE INDEX T(_key)
and not
CREATE INDEX T(customer_id, date).

You can’t use the _key column in the WHERE clause directly, so the query 
optimizer can’t use the index.

After the IGNITE-8386 is fixed the index will be created as a multi-column 
index, and will behave the way you expect (e.g. it will be used instead of the 
affinity key index).

Stan

From: eugene miretsky
Sent: 12 сентября 2018 г. 23:45
To: user@ignite.apache.org
Subject: IGNITE-8386 question (composite pKeys)

Hi, 

A question regarding 
https://issues.apache.org/jira/browse/IGNITE-8386?focusedCommentId=16511394=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16511394

It states that a pkey index with a  compoise pKey is "effectively useless". 
Could you please explain why is that? We have a pKey that we are using as an 
index. 

Also, on our pKey is (customer_id, date) and affinity column is customer_id. I 
have noticed that most queries use AFFINITY_KEY index. Looking at the source 
code, AFFINITY_KEY index should not even be created since the first field of 
the pKey  is the affinity key. Any idea what may be happening? 

Cheers,
Eugene

Re: WARN TcpDiscoverySpi:135 - Timed out waiting for message to be read(most probably....)

[monstereo]

Most the time, it happens when starting the first ignite node

I can not share logs at this time.
 But simply it prints that warning forever.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Message grid failure due to userVersion setting

[Dave Harvey]

We have a client that uses the compute grid and message grid, as well as
the discovery API.  It communicates with a server plugin.   The cluster is
configured for CONTINUOUS peer class loading.  In order to force the proper
code to be loaded for the compute tasks, we change the user version, e.g.,
to 2.

If the server sends the client a message on the message grid, using
java.lang.string, the client fails because the user version sent for
java.lang.string is 0, but the client insists on 2.

How is this supposed to work?   Our expectation was that the message grid
should not be affected by peer class loading settings.

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and compliance. To find 
out more visit the Mimecast website.

YCSB Benchmark

[Shay Alon]

Hi,

I have followed the instructions written in the README page here:

https://github.com/brianfra...


I used the following configuration:
IgniteClient (not IgniteSqlClient).
JAVA 8
Linux
RAM 500G
72 CPU
threads 2
3 backups
1M entries
And run workloads a,b and c and got the following results:
workloada: [OVERALL], Throughput(ops/sec), 323.51989647363314
workloadb: OVERALL], Throughput(ops/sec), 385.2080123266564
workloadc: [OVERALL], Throughput(ops/sec), 375.7985719654265

This results are very low compare to the results that are published in this
page.
Can you please direct me? What am I doing wrong?

Thanks

Re: Handling split brain with Zookeeper and persistence

[Pavel Kovalenko]

Hello Eugene,

1) Split brain resolver takes into account only server nodes (not client).
No difference between in-memory only or with persistence.
2) It's no necessary to immediately remove a node from baseline topology
after split-brain. If you lost backup factor for some partitions (All
partition owner primary/backup are in shut down part of a cluster) and
explicitly change shrink baseline topology, your affinity for all
partitions will be changed and you will lose the data contained in shut
down nodes.
When split-brain have resolved you may up shutting-down nodes and join them
to existing baseline topology, after that outdated data will re-balanced.
In the conclusion, you should carefully operate with baseline and do not
change it without necessary.

сб, 15 сент. 2018 г. в 23:58, Gaurav Bajaj :

> Hello,
>
> For second question above, you can have listener to listen AFTER_NODE_STOP
> event and take action as per your logic (in your case changing BLT).
>
> Regards,
> Gaurav
>
>
> On 14-Sep-2018 9:33 PM, "eugene miretsky" 
> wrote:
>
> Hi,
>
> What are best practices for handling split brain with persistence?
>
> 1) Does Zookeeper split brain resolver consider all nodes as the same
> (client, memory only, persistent). Ideally, we want to shut down persistent
> nodes only as last resort.
> 2) If a persistent node is shut down, we need to remove it from baseline
> topology. Are there events we can subscribe to?
>
> Cheers,
> Eugene
>
>
>

Re: SQL query and Indexes architecture

[eugene miretsky]

Thanks!

I am curious about the process of loading data from Ignite to H2 on the
fly, as H2 creating indexes but storing them in Ignite. Can you point me to
some JIRAs that discuss it, or which part of the code is responsible for
that?

On Mon, Sep 17, 2018 at 9:18 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> 1. 1. H2 executes the query, during which it has to load rows from tables,
> and Ignite does the row loading part. Then Ignite will collect query
> results on all nodes and aggregate them on a single node.
> 1. 2. Index is created by H2, but it is stored in Ignite pages (?).
> 2. Maybe you're right, I have to admit I'm unfamiliar with precise details
> here.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 17 сент. 2018 г. в 16:02, eugene miretsky :
>
>> Thanks!
>>
>>
>>1.
>>1.  "Ignite feeds H2 rows that it asks for, and H2 creates indexes on
>>   them and executes queries on them." - what exactly do you mean by 
>> that? Do
>>   you mean that all parts of a query that use indexes are executed by H2,
>>   then the actual data is retrieved from Ignite pages, and the final
>>   (non-indexed) parts of the query executed by Ignite?
>>   2.  What happens when I create an index on a new column? Is the
>>   index created in Ignite (and stored in Ignite pages?), or is it 
>> created in
>>   H2?
>>2.  The reason I was asking about AFFINITY_KEY, _key_PK and
>>_key_PK_hash indexed is that in this   code
>>
>> 
>>  it
>>looks like they are created in H2
>>
>>
>>
>> On Mon, Sep 17, 2018 at 8:36 AM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> 1. H2 does not store data but, as far as my understanding goes, it
>>> created SQL indexes from data. Ignite feeds H2 rows that it asks for, and
>>> H2 creates indexes on them and executes queries on them.
>>> 2. Ignite always has special index on your key (since it's a key-value
>>> storage it can always find tuple by key). Ignite is also aware of key's
>>> hash code, and affinity key value always maps to one partition of data (of
>>> 1024 by default). Those are not H2 indexes and they're mostly used on
>>> planning stage. E.g. you can map query to one node if affinity key is
>>> present in the request.
>>> 3. Data is brought onto the heap to read any fields from row. GROUP BY
>>> will hold its tuples on heap. Ignite has configurable index inlining where
>>> you can avoid reading objects from heap just to access indexed fields.
>>> 4. With GROUP BY, lazy evaluation will not help you much. It will still
>>> have to hold all data on heap at some point. Lazy evaluation mostly helps
>>> with "SELECT * FROM table" type queries which provide very large and boring
>>> result set.
>>>
>>> Hope this helps.
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 14 сент. 2018 г. в 17:39, eugene miretsky >> >:
>>>
 Hello,

 Trying to understand how exactly SQL queries are executed in Ignite. A
 few questions


1. To what extent is H2 used? Does it store the data? Does it
create the indexes? Is it used only for generating execution plans? I
believe that all the data used to be stored in H2, but with the new 
 durable
memory architecture, I believe that's no longer the case.
2. Which indexes are used? Ignite creates  B+ tree indexes and
stores them in Index pages, but I also see AFFINITY_KEY, _key_PK and
_key_PK_hash indexes created in H2.
3. When is data brought onto the heap? I am assuming that groupby
and aggregate require all the matching queries to first be copied from
off-heap to heap
4. How does lazy evaluation work? For example, for group_by, does
it bring batches of matching records with the same group_by key onto the
heap?

 I am not necessarily looking for the exact answers, but rather pointer
 in the right direction (documentation, code, jiras)

 Cheers,
 Eugene

>>>

Re: SQL query and Indexes architecture

[Ilya Kasnacheev]

Hello!

1. 1. H2 executes the query, during which it has to load rows from tables,
and Ignite does the row loading part. Then Ignite will collect query
results on all nodes and aggregate them on a single node.
1. 2. Index is created by H2, but it is stored in Ignite pages (?).
2. Maybe you're right, I have to admit I'm unfamiliar with precise details
here.

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 16:02, eugene miretsky :

> Thanks!
>
>
>1.
>1.  "Ignite feeds H2 rows that it asks for, and H2 creates indexes on
>   them and executes queries on them." - what exactly do you mean by that? 
> Do
>   you mean that all parts of a query that use indexes are executed by H2,
>   then the actual data is retrieved from Ignite pages, and the final
>   (non-indexed) parts of the query executed by Ignite?
>   2.  What happens when I create an index on a new column? Is the
>   index created in Ignite (and stored in Ignite pages?), or is it created 
> in
>   H2?
>2.  The reason I was asking about AFFINITY_KEY, _key_PK and
>_key_PK_hash indexed is that in this   code
>
> 
>  it
>looks like they are created in H2
>
>
>
> On Mon, Sep 17, 2018 at 8:36 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> 1. H2 does not store data but, as far as my understanding goes, it
>> created SQL indexes from data. Ignite feeds H2 rows that it asks for, and
>> H2 creates indexes on them and executes queries on them.
>> 2. Ignite always has special index on your key (since it's a key-value
>> storage it can always find tuple by key). Ignite is also aware of key's
>> hash code, and affinity key value always maps to one partition of data (of
>> 1024 by default). Those are not H2 indexes and they're mostly used on
>> planning stage. E.g. you can map query to one node if affinity key is
>> present in the request.
>> 3. Data is brought onto the heap to read any fields from row. GROUP BY
>> will hold its tuples on heap. Ignite has configurable index inlining where
>> you can avoid reading objects from heap just to access indexed fields.
>> 4. With GROUP BY, lazy evaluation will not help you much. It will still
>> have to hold all data on heap at some point. Lazy evaluation mostly helps
>> with "SELECT * FROM table" type queries which provide very large and boring
>> result set.
>>
>> Hope this helps.
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 14 сент. 2018 г. в 17:39, eugene miretsky > >:
>>
>>> Hello,
>>>
>>> Trying to understand how exactly SQL queries are executed in Ignite. A
>>> few questions
>>>
>>>
>>>1. To what extent is H2 used? Does it store the data? Does it create
>>>the indexes? Is it used only for generating execution plans? I believe 
>>> that
>>>all the data used to be stored in H2, but with the new durable memory
>>>architecture, I believe that's no longer the case.
>>>2. Which indexes are used? Ignite creates  B+ tree indexes and
>>>stores them in Index pages, but I also see AFFINITY_KEY, _key_PK and
>>>_key_PK_hash indexes created in H2.
>>>3. When is data brought onto the heap? I am assuming that groupby
>>>and aggregate require all the matching queries to first be copied from
>>>off-heap to heap
>>>4. How does lazy evaluation work? For example, for group_by, does it
>>>bring batches of matching records with the same group_by key onto the 
>>> heap?
>>>
>>> I am not necessarily looking for the exact answers, but rather pointer
>>> in the right direction (documentation, code, jiras)
>>>
>>> Cheers,
>>> Eugene
>>>
>>

Re: Query 3x slower with index

[Ilya Kasnacheev]

Hello!

Why don't you diff the results of those two queries, tell us what the
difference is?

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 16:08, eugene miretsky :

> Hello,
>
> Just wanted to see if anybody had time to look into this.
>
> Cheers,
> Eugene
>
> On Wed, Sep 12, 2018 at 6:29 PM eugene miretsky 
> wrote:
>
>> Thanks!
>>
>> Tried joining with an inlined table instead of IN as per the second
>> suggestion, and it didn't quite work.
>>
>> Query1:
>>
>>- Select COUNT(*) FROM( Select customer_id from GATABLE3  use Index(
>>) where category_id in (9005, 175930, 175930, 175940,175945,101450, 6453)
>>group by customer_id having SUM(product_views_app) > 2 OR
>>SUM(product_clicks_app) > 1 )
>>- exec time = 17s
>>- *Result: 3105868*
>>- Same exec time if using AFFINITY_KEY index or "_key_PK_hash or
>>customer_id index
>>- Using an index on category_id increases the query time 33s
>>
>> Query2:
>>
>>- Select COUNT(*) FROM( Select customer_id from GATABLE3 ga  use
>>index (PUBLIC."_key_PK") inner join table(category_id int = (9005, 175930,
>>175930, 175940,175945,101450, 6453)) cats on cats.category_id =
>>ga.category_id   group by customer_id having SUM(product_views_app) > 2 OR
>>SUM(product_clicks_app) > 1 )
>>- exec time = 38s
>>- *Result: 3113921*
>>- Same exec time if using AFFINITY_KEY index or "_key_PK_hash or
>>customer_id index or category_id index
>>- Using an index on category_id doesnt change the run time
>>
>> Query plans are attached.
>>
>> 3 questions:
>>
>>1. Why is the result differnt for the 2 queries - this is quite
>>concerning.
>>2. Why is the 2nd query taking longer
>>3. Why  category_id index doesn't work in case of query 2.
>>
>>
>> On Wed, Sep 5, 2018 at 8:31 AM Ilya Kasnacheev 
>> wrote:
>>
>>> Hello!
>>>
>>> I don't think that we're able to use index with IN () clauses. Please
>>> convert it into OR clauses.
>>>
>>> Please see
>>> https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-sql-performance-and-usability-considerations
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пн, 3 сент. 2018 г. в 12:46, Andrey Mashenkov <
>>> andrey.mashen...@gmail.com>:
>>>
 Hi

 Actually, first query uses index on affinity key which looks more
 efficient than index on category_id column.
 The first query can process groups one by one and stream partial
 results from map phase to reduce phase as it use sorted index lookup,
 while second query should process full dataset on map phase before pass
 it for reducing.

 Try to use composite index (customer_id, category_id).

 Also, SqlQueryFields.setCollocated(true) flag can help Ignite to build
 more efficient plan when group by on collocated column is used.

 On Sun, Sep 2, 2018 at 2:02 AM eugene miretsky <
 eugene.miret...@gmail.com> wrote:

> Hello,
>
> Schema:
>
>-
>
>PUBLIC.GATABLE2.CUSTOMER_ID
>
>PUBLIC.GATABLE2.DT
>
>PUBLIC.GATABLE2.CATEGORY_ID
>
>PUBLIC.GATABLE2.VERTICAL_ID
>
>PUBLIC.GATABLE2.SERVICE
>
>PUBLIC.GATABLE2.PRODUCT_VIEWS_APP
>
>PUBLIC.GATABLE2.PRODUCT_CLICKS_APP
>
>PUBLIC.GATABLE2.PRODUCT_VIEWS_WEB
>
>PUBLIC.GATABLE2.PRODUCT_CLICKS_WEB
>
>PUBLIC.GATABLE2.PDP_SESSIONS_APP
>
>PUBLIC.GATABLE2.PDP_SESSIONS_WEB
>- pkey = customer_id,dt
>- affinityKey = customer
>
> Query:
>
>- select COUNT(*) FROM( Select customer_id from GATABLE2 where
>category_id in (175925, 101450, 9005, 175930, 175930, 
> 175940,175945,101450,
>6453) group by customer_id having SUM(product_views_app) > 2 OR
>SUM(product_clicks_app) > 1 )
>
> The table has 600M rows.
> At first, the query took 1m, when we added an index on category_id the
> query started taking 3m.
>
> The SQL execution plan for both queries is attached.
>
> We are using a single x1.16xlarge insntace with query parallelism set
> to 32
>
> Cheers,
> Eugene
>
>

 --
 Best regards,
 Andrey V. Mashenkov

>>>

Re: Backup failover with persistence

[Ilya Kasnacheev]

Hello!

1. A thing called "late affinity assignment" will happen.
2. Before "late affinity assignment" happens Node A is not primary. Once it
happens, Node A is primary.

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 16:07, eugene miretsky :

> Thanks Ilya,
>
>
>1. "So all nodes will know when node A begins hosting that partition
>as primary" - how is that consensus achieved? Will it result in partition
>map exchange and new topology version?
>2. What I actually meant is that it is impossible to know when Node A
>is fully caught up to node B unless you stop all the writes to Node B while
>node A is catching up. So how does Ignite know that it is safe to set A to
>primary again?
>
>
> On Mon, Sep 17, 2018 at 8:48 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Apache Ignite is NOT "eventually consistent" if you ask that. Apache
>> Ignite is strongly consistent. It has discovery ring (or discovery star
>> with Zk) which allows messages to be sent and acknowledged by all nodes.
>>
>> So all nodes will know when node A begins hosting that partition as
>> primary.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 17 сент. 2018 г. в 15:45, eugene miretsky > >:
>>
>>> How is "finish syncing" defined? Since it is a distributed system that
>>> is no way to guarantee that node A is 100% caught up to node B. In Kafka
>>> there is a replica.lag.time.max.ms settings, is there something
>>> similar in Ignite?
>>>
>>>
>>>
>>> On Mon, Sep 17, 2018 at 8:37 AM Ilya Kasnacheev <
>>> ilya.kasnach...@gmail.com> wrote:
>>>
 Hello!

 Node A will have two choices: either drop partition completely and
 re-download it from B, or replicate recent changes on it. Either one will
 be choosed internally.
 Node A will only become primary again when it finishes syncing that
 partition.

 Regards,
 --
 Ilya Kasnacheev


 пт, 14 сент. 2018 г. в 22:23, eugene miretsky <
 eugene.miret...@gmail.com>:

> What is the process when a node goes down and then restarts?
>
> Say backups = 1. We have node A that is primary for some key, and node
> B that is back up.
>
> Node A goes down and then restarts after 5 min. What are the steps?
> 1) Node A is servicing all traffic for key X
> 2) Node A goes down
> 3) Node B starts serving all traffic for key X (I guess the clients
> detect the failover and start calling node B )
> 4) Node A comes back up
> 5) WAL replication is initiated
>
> What happens next? When does node A become the primary again? How are
> in-flight updates happen?
>
>

Re: Query 3x slower with index

[eugene miretsky]

Hello,

Just wanted to see if anybody had time to look into this.

Cheers,
Eugene

On Wed, Sep 12, 2018 at 6:29 PM eugene miretsky 
wrote:

> Thanks!
>
> Tried joining with an inlined table instead of IN as per the second
> suggestion, and it didn't quite work.
>
> Query1:
>
>- Select COUNT(*) FROM( Select customer_id from GATABLE3  use Index( )
>where category_id in (9005, 175930, 175930, 175940,175945,101450, 6453)
>group by customer_id having SUM(product_views_app) > 2 OR
>SUM(product_clicks_app) > 1 )
>- exec time = 17s
>- *Result: 3105868*
>- Same exec time if using AFFINITY_KEY index or "_key_PK_hash or
>customer_id index
>- Using an index on category_id increases the query time 33s
>
> Query2:
>
>- Select COUNT(*) FROM( Select customer_id from GATABLE3 ga  use index
>(PUBLIC."_key_PK") inner join table(category_id int = (9005, 175930,
>175930, 175940,175945,101450, 6453)) cats on cats.category_id =
>ga.category_id   group by customer_id having SUM(product_views_app) > 2 OR
>SUM(product_clicks_app) > 1 )
>- exec time = 38s
>- *Result: 3113921*
>- Same exec time if using AFFINITY_KEY index or "_key_PK_hash or
>customer_id index or category_id index
>- Using an index on category_id doesnt change the run time
>
> Query plans are attached.
>
> 3 questions:
>
>1. Why is the result differnt for the 2 queries - this is quite
>concerning.
>2. Why is the 2nd query taking longer
>3. Why  category_id index doesn't work in case of query 2.
>
>
> On Wed, Sep 5, 2018 at 8:31 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> I don't think that we're able to use index with IN () clauses. Please
>> convert it into OR clauses.
>>
>> Please see
>> https://apacheignite-sql.readme.io/docs/performance-and-debugging#section-sql-performance-and-usability-considerations
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пн, 3 сент. 2018 г. в 12:46, Andrey Mashenkov > >:
>>
>>> Hi
>>>
>>> Actually, first query uses index on affinity key which looks more
>>> efficient than index on category_id column.
>>> The first query can process groups one by one and stream partial results
>>> from map phase to reduce phase as it use sorted index lookup,
>>> while second query should process full dataset on map phase before pass
>>> it for reducing.
>>>
>>> Try to use composite index (customer_id, category_id).
>>>
>>> Also, SqlQueryFields.setCollocated(true) flag can help Ignite to build
>>> more efficient plan when group by on collocated column is used.
>>>
>>> On Sun, Sep 2, 2018 at 2:02 AM eugene miretsky <
>>> eugene.miret...@gmail.com> wrote:
>>>
 Hello,

 Schema:

-

PUBLIC.GATABLE2.CUSTOMER_ID

PUBLIC.GATABLE2.DT

PUBLIC.GATABLE2.CATEGORY_ID

PUBLIC.GATABLE2.VERTICAL_ID

PUBLIC.GATABLE2.SERVICE

PUBLIC.GATABLE2.PRODUCT_VIEWS_APP

PUBLIC.GATABLE2.PRODUCT_CLICKS_APP

PUBLIC.GATABLE2.PRODUCT_VIEWS_WEB

PUBLIC.GATABLE2.PRODUCT_CLICKS_WEB

PUBLIC.GATABLE2.PDP_SESSIONS_APP

PUBLIC.GATABLE2.PDP_SESSIONS_WEB
- pkey = customer_id,dt
- affinityKey = customer

 Query:

- select COUNT(*) FROM( Select customer_id from GATABLE2 where
category_id in (175925, 101450, 9005, 175930, 175930, 
 175940,175945,101450,
6453) group by customer_id having SUM(product_views_app) > 2 OR
SUM(product_clicks_app) > 1 )

 The table has 600M rows.
 At first, the query took 1m, when we added an index on category_id the
 query started taking 3m.

 The SQL execution plan for both queries is attached.

 We are using a single x1.16xlarge insntace with query parallelism set
 to 32

 Cheers,
 Eugene


>>>
>>> --
>>> Best regards,
>>> Andrey V. Mashenkov
>>>
>>

Re: Backup failover with persistence

[eugene miretsky]

Thanks Ilya,


   1. "So all nodes will know when node A begins hosting that partition as
   primary" - how is that consensus achieved? Will it result in partition map
   exchange and new topology version?
   2. What I actually meant is that it is impossible to know when Node A is
   fully caught up to node B unless you stop all the writes to Node B while
   node A is catching up. So how does Ignite know that it is safe to set A to
   primary again?


On Mon, Sep 17, 2018 at 8:48 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> Apache Ignite is NOT "eventually consistent" if you ask that. Apache
> Ignite is strongly consistent. It has discovery ring (or discovery star
> with Zk) which allows messages to be sent and acknowledged by all nodes.
>
> So all nodes will know when node A begins hosting that partition as
> primary.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пн, 17 сент. 2018 г. в 15:45, eugene miretsky :
>
>> How is "finish syncing" defined? Since it is a distributed system that is
>> no way to guarantee that node A is 100% caught up to node B. In Kafka there
>> is a replica.lag.time.max.ms settings, is there something similar in
>> Ignite?
>>
>>
>>
>> On Mon, Sep 17, 2018 at 8:37 AM Ilya Kasnacheev <
>> ilya.kasnach...@gmail.com> wrote:
>>
>>> Hello!
>>>
>>> Node A will have two choices: either drop partition completely and
>>> re-download it from B, or replicate recent changes on it. Either one will
>>> be choosed internally.
>>> Node A will only become primary again when it finishes syncing that
>>> partition.
>>>
>>> Regards,
>>> --
>>> Ilya Kasnacheev
>>>
>>>
>>> пт, 14 сент. 2018 г. в 22:23, eugene miretsky >> >:
>>>
 What is the process when a node goes down and then restarts?

 Say backups = 1. We have node A that is primary for some key, and node
 B that is back up.

 Node A goes down and then restarts after 5 min. What are the steps?
 1) Node A is servicing all traffic for key X
 2) Node A goes down
 3) Node B starts serving all traffic for key X (I guess the clients
 detect the failover and start calling node B )
 4) Node A comes back up
 5) WAL replication is initiated

 What happens next? When does node A become the primary again? How are
 in-flight updates happen?

Re: SQL query and Indexes architecture

[eugene miretsky]

Thanks!


   1.
   1.  "Ignite feeds H2 rows that it asks for, and H2 creates indexes on
  them and executes queries on them." - what exactly do you mean
by that? Do
  you mean that all parts of a query that use indexes are executed by H2,
  then the actual data is retrieved from Ignite pages, and the final
  (non-indexed) parts of the query executed by Ignite?
  2.  What happens when I create an index on a new column? Is the index
  created in Ignite (and stored in Ignite pages?), or is it created in H2?
   2.  The reason I was asking about AFFINITY_KEY, _key_PK and _key_PK_hash
   indexed is that in this   code
   

it
   looks like they are created in H2



On Mon, Sep 17, 2018 at 8:36 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> 1. H2 does not store data but, as far as my understanding goes, it created
> SQL indexes from data. Ignite feeds H2 rows that it asks for, and H2
> creates indexes on them and executes queries on them.
> 2. Ignite always has special index on your key (since it's a key-value
> storage it can always find tuple by key). Ignite is also aware of key's
> hash code, and affinity key value always maps to one partition of data (of
> 1024 by default). Those are not H2 indexes and they're mostly used on
> planning stage. E.g. you can map query to one node if affinity key is
> present in the request.
> 3. Data is brought onto the heap to read any fields from row. GROUP BY
> will hold its tuples on heap. Ignite has configurable index inlining where
> you can avoid reading objects from heap just to access indexed fields.
> 4. With GROUP BY, lazy evaluation will not help you much. It will still
> have to hold all data on heap at some point. Lazy evaluation mostly helps
> with "SELECT * FROM table" type queries which provide very large and boring
> result set.
>
> Hope this helps.
> --
> Ilya Kasnacheev
>
>
> пт, 14 сент. 2018 г. в 17:39, eugene miretsky :
>
>> Hello,
>>
>> Trying to understand how exactly SQL queries are executed in Ignite. A
>> few questions
>>
>>
>>1. To what extent is H2 used? Does it store the data? Does it create
>>the indexes? Is it used only for generating execution plans? I believe 
>> that
>>all the data used to be stored in H2, but with the new durable memory
>>architecture, I believe that's no longer the case.
>>2. Which indexes are used? Ignite creates  B+ tree indexes and stores
>>them in Index pages, but I also see AFFINITY_KEY, _key_PK and _key_PK_hash
>>indexes created in H2.
>>3. When is data brought onto the heap? I am assuming that groupby and
>>aggregate require all the matching queries to first be copied from 
>> off-heap
>>to heap
>>4. How does lazy evaluation work? For example, for group_by, does it
>>bring batches of matching records with the same group_by key onto the 
>> heap?
>>
>> I am not necessarily looking for the exact answers, but rather pointer in
>> the right direction (documentation, code, jiras)
>>
>> Cheers,
>> Eugene
>>
>

Re: Backup failover with persistence

[Ilya Kasnacheev]

Hello!

Apache Ignite is NOT "eventually consistent" if you ask that. Apache Ignite
is strongly consistent. It has discovery ring (or discovery star with Zk)
which allows messages to be sent and acknowledged by all nodes.

So all nodes will know when node A begins hosting that partition as primary.

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 15:45, eugene miretsky :

> How is "finish syncing" defined? Since it is a distributed system that is
> no way to guarantee that node A is 100% caught up to node B. In Kafka there
> is a replica.lag.time.max.ms settings, is there something similar in
> Ignite?
>
>
>
> On Mon, Sep 17, 2018 at 8:37 AM Ilya Kasnacheev 
> wrote:
>
>> Hello!
>>
>> Node A will have two choices: either drop partition completely and
>> re-download it from B, or replicate recent changes on it. Either one will
>> be choosed internally.
>> Node A will only become primary again when it finishes syncing that
>> partition.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> пт, 14 сент. 2018 г. в 22:23, eugene miretsky > >:
>>
>>> What is the process when a node goes down and then restarts?
>>>
>>> Say backups = 1. We have node A that is primary for some key, and node B
>>> that is back up.
>>>
>>> Node A goes down and then restarts after 5 min. What are the steps?
>>> 1) Node A is servicing all traffic for key X
>>> 2) Node A goes down
>>> 3) Node B starts serving all traffic for key X (I guess the clients
>>> detect the failover and start calling node B )
>>> 4) Node A comes back up
>>> 5) WAL replication is initiated
>>>
>>> What happens next? When does node A become the primary again? How are
>>> in-flight updates happen?
>>>
>>>

Re: Huge Query order by PK causes OOMKilled

[Ilya Kasnacheev]

Hello!

Have you tried setting lazy=true for this query? Ignite has couple of ways
to do SQL, but each of those allows you to set lazy=true.

If this does not help, please try to add explicit SQL index on the field
that you're doing ORDER BY on.

Regards,
-- 
Ilya Kasnacheev


пт, 14 сент. 2018 г. в 23:49, Renato Melo :

> I am running on Kubertes three nodes. 32 caches are replicated across
> these 3 nodes.
>
> Currently each node is Limited to 10 cpus and 30 Gigabits of memory.
> Caches are persisted. Max heap memory is set to 18g.
>
> Before running the query, with all nodes up and connect, I ran top and I
> see that Ignite Java process is using 11 GB of RAM (RS) and 18 GB (Virt)
>
> When I run a query in on table that has 135 records and ordering the
> data by the primary key (ORDER BY PK), the memory of one nodes is consumed
> until the node crashes.
>
> The data is need to CDC (Change-data-capture) thus the select must return
> all rows order by PK.
>
> During the query execution I got this logs in the node that failed:
>
> [16:51:27,258][WARNING][client-connector-#1437][IgniteH2Indexing] Query
> execution is too long [time=9638 ms, sql='SELECT
>
> Then:
>
> [20:24:39,552][INFO][grid-timeout-worker-#199][IgniteKernal]
> Metrics for local node (to disable set 'metricsLogFrequency' to 0)
> ^-- Node [id=ea6f72d6, uptime=03:19:00.981]
> ^-- H/N/C [hosts=3, nodes=3, CPUs=288]
> ^-- CPU [cur=0.47%, avg=0.03%, GC=0%]
> ^-- PageMemory [pages=1163734]
> ^-- Heap [used=3530MB, free=80.84%, comm=5120MB]
> ^-- Non heap [used=74MB, free=-1%, comm=75MB]
> ^-- Outbound messages queue [size=0]
> ^-- Public thread pool [active=0, idle=0, qSize=0]
> ^-- System thread pool [active=0, idle=6, qSize=0]
> [20:24:43,859][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 1443 milliseconds.
> [20:24:54,859][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 1179 milliseconds.
> [20:25:01,181][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 979 milliseconds.
> [20:25:03,897][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 508 milliseconds.
> [20:25:17,503][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 1337 milliseconds.
> [20:25:21,520][WARNING][jvm-pause-detector-worker][] Possible too long JVM
> pause: 2661 milliseconds.
> #
> # A fatal error has been detected by the Java Runtime Environment:
> #
> #  SIGSEGV (0xb) at pc=0x7ff5463758e5, pid=373, tid=0x7fe02700
> #
> # JRE version: OpenJDK Runtime Environment (8.0_171-b11) (build
> 1.8.0_171-8u171-b11-1~deb9u1-b11)
> # Java VM: OpenJDK 64-Bit Server VM (25.171-b11 mixed mode linux-amd64
> compressed oops)
> # Problematic frame:
> # J 7881 C2
> org.apache.ignite.internal.binary.BinaryObjectImpl.fieldByOrder(I)Ljava/lang/Object;
> (766 bytes) @ 0x7ff5463758e5 [0x7ff546374e80+0xa65]
> #
> # Core dump written. Default location: /opt/ignite/core or core.373
> #
> # An error report file with more information is saved as:
> # /opt/ignite/hs_err_pid373.log
> #
> # If you would like to submit a bug report, please visit:
> #   http://bugreport.java.com/bugreport/crash.jsp
> #
> /opt/ignite/apache-ignite-fabric-2.6.0-bin/bin/ignite.sh: line 181:   373
> Aborted (core dumped) "$JAVA" ${JVM_OPTS} ${QUIET}
> "${RESTART_SUCCESS_OPT}" ${JMX_MON} -DIGNITE_HOME="${IGNITE_HOME}"
> -DIGNITE_PROG_NAME="$0" ${JVM_XOPTS} -cp "${CP}" ${MAIN_CLASS} "${CONFIG}"
>
> I tried different configurations. I would appreciate any help.
>
>
>

Re: Backup failover with persistence

[eugene miretsky]

How is "finish syncing" defined? Since it is a distributed system that is
no way to guarantee that node A is 100% caught up to node B. In Kafka there
is a replica.lag.time.max.ms settings, is there something similar in
Ignite?



On Mon, Sep 17, 2018 at 8:37 AM Ilya Kasnacheev 
wrote:

> Hello!
>
> Node A will have two choices: either drop partition completely and
> re-download it from B, or replicate recent changes on it. Either one will
> be choosed internally.
> Node A will only become primary again when it finishes syncing that
> partition.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> пт, 14 сент. 2018 г. в 22:23, eugene miretsky :
>
>> What is the process when a node goes down and then restarts?
>>
>> Say backups = 1. We have node A that is primary for some key, and node B
>> that is back up.
>>
>> Node A goes down and then restarts after 5 min. What are the steps?
>> 1) Node A is servicing all traffic for key X
>> 2) Node A goes down
>> 3) Node B starts serving all traffic for key X (I guess the clients
>> detect the failover and start calling node B )
>> 4) Node A comes back up
>> 5) WAL replication is initiated
>>
>> What happens next? When does node A become the primary again? How are
>> in-flight updates happen?
>>
>>

Re: rmdbs integration readthrough - Ownership flag not set for binary property. Have you set 'keyFields' property of QueryEntity in programmatic or XML configuration?

[Ilya Kasnacheev]

Hello!

You have "fields" list in your QueryEntity, but not "keyFields". I suggest
you add them.

Regards,
-- 
Ilya Kasnacheev


сб, 15 сент. 2018 г. в 18:50, wt :

> i have setup rmdbs integration with read-through
>
> if i query an empty cache it should read-through from the underlying system
> but that isn't happening i get the following error
>
>
> [16:43:35,351][SEVERE][client-connector-#56%StoxxDataLake%][JdbcRequestHandler]
> Failed to get parameters metadata [reqId=0, req=JdbcMetaPrimaryKeysRequest
> [schemaName=IndexcompositionCache, tblName=INDEXCOMPOSITION]]
> java.lang.IllegalStateException: Ownership flag not set for binary
> property.
> Have you set 'keyFields' property of QueryEntity in programmatic or XML
> configuration?
> at
>
> org.apache.ignite.internal.processors.query.property.QueryBinaryProperty.key(QueryBinaryProperty.java:269)
> at
>
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.getPrimaryKeys(JdbcRequestHandler.java:940)
> at
>
> org.apache.ignite.internal.processors.odbc.jdbc.JdbcRequestHandler.handle(JdbcRequestHandler.java:233)
> at
>
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:160)
> at
>
> org.apache.ignite.internal.processors.odbc.ClientListenerNioListener.onMessage(ClientListenerNioListener.java:44)
> at
>
> org.apache.ignite.internal.util.nio.GridNioFilterChain$TailFilter.onMessageReceived(GridNioFilterChain.java:279)
> at
>
> org.apache.ignite.internal.util.nio.GridNioFilterAdapter.proceedMessageReceived(GridNioFilterAdapter.java:109)
> at
>
> org.apache.ignite.internal.util.nio.GridNioAsyncNotifyFilter$3.body(GridNioAsyncNotifyFilter.java:97)
> at
> org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
> at
>
> org.apache.ignite.internal.util.worker.GridWorkerPool$1.run(GridWorkerPool.java:70)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
>
>
> here is my configuration file
> StoxxDataLake-server.xml
> <
> http://apache-ignite-users.70518.x6.nabble.com/file/t1892/StoxxDataLake-server.xml>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Java maven Application

[Ilya Kasnacheev]

Hello!

You need to add ignite-core-VERSION.jar to classpath to be able to use
Ignite JDBC driver.
If it's not enough, provide driver class as
`org.apache.ignite.IgniteJdbcThinDriver'

Regards,
-- 
Ilya Kasnacheev


пн, 17 сент. 2018 г. в 10:17, Malashree :

> how to connect to the ignite database using java maven web application
> using jdbc driver.
>
> For this i am getting error that  No suitable driver found for
> jdbc:ignite:thin://127.0.0.1/
> Please help me resolving these issues.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Backup failover with persistence

[Ilya Kasnacheev]

Hello!

Node A will have two choices: either drop partition completely and
re-download it from B, or replicate recent changes on it. Either one will
be choosed internally.
Node A will only become primary again when it finishes syncing that
partition.

Regards,
-- 
Ilya Kasnacheev


пт, 14 сент. 2018 г. в 22:23, eugene miretsky :

> What is the process when a node goes down and then restarts?
>
> Say backups = 1. We have node A that is primary for some key, and node B
> that is back up.
>
> Node A goes down and then restarts after 5 min. What are the steps?
> 1) Node A is servicing all traffic for key X
> 2) Node A goes down
> 3) Node B starts serving all traffic for key X (I guess the clients detect
> the failover and start calling node B )
> 4) Node A comes back up
> 5) WAL replication is initiated
>
> What happens next? When does node A become the primary again? How are
> in-flight updates happen?
>
>

Re: SQL query and Indexes architecture

[Ilya Kasnacheev]

Hello!

1. H2 does not store data but, as far as my understanding goes, it created
SQL indexes from data. Ignite feeds H2 rows that it asks for, and H2
creates indexes on them and executes queries on them.
2. Ignite always has special index on your key (since it's a key-value
storage it can always find tuple by key). Ignite is also aware of key's
hash code, and affinity key value always maps to one partition of data (of
1024 by default). Those are not H2 indexes and they're mostly used on
planning stage. E.g. you can map query to one node if affinity key is
present in the request.
3. Data is brought onto the heap to read any fields from row. GROUP BY will
hold its tuples on heap. Ignite has configurable index inlining where you
can avoid reading objects from heap just to access indexed fields.
4. With GROUP BY, lazy evaluation will not help you much. It will still
have to hold all data on heap at some point. Lazy evaluation mostly helps
with "SELECT * FROM table" type queries which provide very large and boring
result set.

Hope this helps.
-- 
Ilya Kasnacheev


пт, 14 сент. 2018 г. в 17:39, eugene miretsky :

> Hello,
>
> Trying to understand how exactly SQL queries are executed in Ignite. A few
> questions
>
>
>1. To what extent is H2 used? Does it store the data? Does it create
>the indexes? Is it used only for generating execution plans? I believe that
>all the data used to be stored in H2, but with the new durable memory
>architecture, I believe that's no longer the case.
>2. Which indexes are used? Ignite creates  B+ tree indexes and stores
>them in Index pages, but I also see AFFINITY_KEY, _key_PK and _key_PK_hash
>indexes created in H2.
>3. When is data brought onto the heap? I am assuming that groupby and
>aggregate require all the matching queries to first be copied from off-heap
>to heap
>4. How does lazy evaluation work? For example, for group_by, does it
>bring batches of matching records with the same group_by key onto the heap?
>
> I am not necessarily looking for the exact answers, but rather pointer in
> the right direction (documentation, code, jiras)
>
> Cheers,
> Eugene
>

Re: WARN TcpDiscoverySpi:135 - Timed out waiting for message to be read(most probably....)

[ezhuravlev]

Hi,

Do you have connectivity between nodes? Have you checked that all nodes are
reachable and there are no long GC pauses?

Can you share the full logs from all nodes?

Evgenii



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: multiple tables in same cache - is it possible

[Ilya Kasnacheev]

Hello!

Yes, this is possible, for example, by specifying multiple key-value type
pairs in setIndexedTypes().

For every pair, a table of VALUETYPECLASSNAME will be created.

The requirements will be that key and value types should not intersect
between those tables.

Regards,
-- 
Ilya Kasnacheev


пт, 14 сент. 2018 г. в 22:32, wt :

> i have read that the preferred way is to have a cache per table. Is it
> however possible to have multiple tables in a single cache to make the
> administration of that cache simpler. what are the sort of problems that
> would surface if this were to be done?
>
> thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: .NET java thread count keeps growing

[Ilya Kasnacheev]

Hello!

You seem to have 3000 threads of form "Thread-", which is not what
Ignite usually uses, and they're all empty.
This is mysterious so I urge you to share the sample reproducer (or,
barring that, code snippet) that leads to such behavior.

By the way, do you observe any errors in the log?

Regards,
-- 
Ilya Kasnacheev


вс, 16 сент. 2018 г. в 4:09, Alew :

> Hi!
>
> I have growing java thread count. The application crashes with OOME. The
> application is very simple, only reading values in a cache.
>
> Any suggestion to debug this issue?
>
> Ignite 2.4
>
>
>

Failed to get page IO instance (page content is corrupted) after one node failed when trying to reboot.

[Ray]

I have a three nodes Ignite 2.6.0 cluster with native persistence enabled.
Here's the config

http://www.springframework.org/schema/beans;
   xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance;
   xsi:schemaLocation="
   http://www.springframework.org/schema/beans
   http://www.springframework.org/schema/beans/spring-beans.xsd;>































node1:49500
node2:49500
node3:49500














One node failed when trying to process a long running sql, the detailed log
can be found in attachment with filename nodefail.log.

The other two nodes may have some new data coming in when one node in failed
state.
When I try to reboot this server after several hours, first it got stuck for
one hour with "Failed to wait for partition map exchange" exception.
The detailed log can be found in attachment with filename restart1.log.

So I try to reboot this server again but got
"org.apache.ignite.spi.IgniteSpiException: Node with set up BaselineTopology
is not allowed to join cluster without one:" exception.
The detailed log can be found in attachment with filename restart2.log.

So I try to reboot the whole cluster by starting failed node first, but I
got "Unable to await partitions release latch within timeout:" exception
when two other servers are started.
The detailed log can be found in attachment with filename restart3.log.

So I try to reboot this server again, but I got "Failed to get page IO
instance (page content is corrupted)" exception.
The detailed log can be found in attachment with filename restart4.log.

>From this point on, the cluster is in non-recoverable state.
Please advice me how to avoid this situation and how to recover data.
The log of failed node is in log.zip.
The other two files are logs for two good nodes.

log.zip   
log-goodNode.zip
  
log-GoodNode2.zip
  






--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

WARN TcpDiscoverySpi:135 - Timed out waiting for message to be read(most probably....)

[monstereo]

2018-09-17 10:51:00 WARN  TcpDiscoverySpi:135 - Timed out waiting for message
to be read (most probably, the reason is long GC pauses on remote node)
[curTimeout=1, rmtAddr=/127.0.0.1:47504, rmtPort=47504]

this warning is printing out on the screen, and my ignite node can not
start.
How do i solve this ?

Here is the discovery spi from my configuration.








127.0.0.1:47500..47509









--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to write Trigger Script

[Evgenii Zhuravlev]

Hi,

There are continuous queries in Ignite fir such purpose:
https://apacheignite.readme.io/docs/continuous-queries

Evgenii

пн, 17 сент. 2018 г. в 9:22, Malashree :

> How to write Trigger Script for Apache Ignite which is similar to MYSQL
> Trigger Script.
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: How to connect to the Rest API

[Evgenii Zhuravlev]

Hi,

Here is a doc:
https://apacheignite.readme.io/docs/rest-api#section-getting-started

Evgenii

пн, 17 сент. 2018 г. в 10:04, Malashree :

> How to connect to the Rest API using Ignite Database.
>
> http://localhost:8080/ignite?cmd=version
> This command is not working,i am getting error like
> HTTP Status 404-/ignite
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

How to connect to the Rest API

[Malashree]

How to connect to the Rest API using Ignite Database.

http://localhost:8080/ignite?cmd=version
This command is not working,i am getting error like
HTTP Status 404-/ignite



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to write Trigger Script

[Malashree]

How to write Trigger Script for Apache Ignite which is similar to MYSQL
Trigger Script.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/