How to check if key exists in DataStreamer buffer so that it can be flushed?

[the_palakkaran]

Hi,

I have a data streamer to load data into a cache. While loading I might need
to update value of a particular key in cache, so I need to check if it is
already there in the streamer buffer. If so, either I need to update value
against that key in the buffer or I need to flush the data in the streamer
and then update. Is there a way to do this?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: ignte cluster hang with GridCachePartitionExchangeManager

[wangsan]

I can reproduce the bug,  above log is the server(first) node print when I
stop other nodes .



import java.io.IOException;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.concurrent.CompletableFuture;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import java.util.stream.IntStream;

import org.apache.ignite.Ignite;
import org.apache.ignite.IgniteCache;
import org.apache.ignite.Ignition;
import org.apache.ignite.cache.CacheMode;
import org.apache.ignite.cluster.ClusterNode;
import org.apache.ignite.configuration.CacheConfiguration;
import org.apache.ignite.configuration.IgniteConfiguration;
import org.apache.ignite.events.DiscoveryEvent;
import org.apache.ignite.events.EventType;
import org.apache.ignite.internal.managers.discovery.IgniteDiscoverySpi;
import org.apache.ignite.logger.slf4j.Slf4jLogger;
import org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi;
import
org.apache.ignite.spi.discovery.tcp.ipfinder.multicast.TcpDiscoveryMulticastIpFinder;
import org.junit.Test;

import com.google.common.collect.Lists;

/**
 * @author wangsan
 * @date 2018/08/23
 */
public class IgniteMultiThreadOnOffTest {
private static final String CACHE_DATA_REGION_PERSISTENCE =
"PERSISTENCE_REGION";
private static final String stringCacheOneName = "StringCacheOneName";

@Test
public void testFirstServer() throws Exception {
IgniteConfiguration cfg = new IgniteConfiguration();
cfg.setClientMode(false);
cfg.setDiscoverySpi(tcpDiscoverySpi());
cfg.setConsistentId("server_test_1");
cfg.setIgniteInstanceName("server_test_1");
cfg.setGridLogger(new Slf4jLogger());
Ignite ignite = Ignition.start(cfg);

CacheConfiguration cacheOneConfiguration = new
CacheConfiguration<>(stringCacheOneName);
cacheOneConfiguration.setCacheMode(CacheMode.REPLICATED);
IgniteCache cacheOne =
ignite.getOrCreateCache(cacheOneConfiguration);

ignite.events().localListen(event -> {
System.err.println("get event " + event.name() + " " + event);
if (event instanceof DiscoveryEvent) {
ClusterNode clusterNode = ((DiscoveryEvent)
event).eventNode();

String item = "event_" + clusterNode.consistentId();
System.err.println("--  oldest node process the message
: " + item);

switch (event.type()) {
case EventType.EVT_NODE_JOINED:
cacheOne.put(item, item);
//   
System.err.println("--- do add async" + item);
//   
ForkJoinPool.commonPool().execute(() -> cacheOne.put(item, item));
break;
case EventType.EVT_NODE_FAILED:
case EventType.EVT_NODE_LEFT:
// will block
cacheOne.remove(item);
//   
System.err.println("--- do remove async " + item);
//   
ForkJoinPool.commonPool().execute(() -> cacheOne.remove(item));
break;
default:
System.err.println("ignore discovery event:" +
event);
break;
}
return true;
} else {
return false;
}
}, EventType.EVTS_DISCOVERY);

while (true) {
TimeUnit.SECONDS.sleep(5);

System.err.println(" " + cacheOne.size());
   
System.err.println(ignite.cacheNames().stream().collect(Collectors.joining(",")));
}
}

@Test
public void testMultiServer() throws IOException {
ExecutorService executorService = Executors.newCachedThreadPool();
long start = System.currentTimeMillis();
CompletableFuture> allIgnite =
IntStream.range(0, 5)
.mapToObj(i -> CompletableFuture.supplyAsync(() ->
testServer(i), executorService))
.map(f -> f.thenApply(i -> Lists.newArrayList(i)))
.reduce((x, y) -> x.thenCombine(y, (a, b) -> {
a.addAll(b);
return a;
})).get();

allIgnite.thenAccept(list -> {
System.err.println("start use time ms " +
(System.currentTimeMillis() - start));
list.forEach(n -> System.err.println("n.id " +
n.cluster().localNode().consistentId()));
try {
TimeUnit.SECONDS.sleep(20);
} catch (InterruptedException e) {
e.printStackTrace();
}
System.err.println("going to close ignite,see first server ");
list.forEach(n -> n.close());
}).join();
}

private Ignite testServer(int index) {
String name = 

Re: Affinity key in SQL execution

[Prasad Bhalerao]

3. I did not understand your answer in point 3.

I must join by userId to get the correct data.
Are you saying that (c1.affinityId = c2.affinityId) condition should be
included in on clause?

If I rewrite SQL as follows, will it work?

Select * from cache1 c1 join cache2 c2 on ( c1.userId = c2.userId and
and c1.affinityId
= c2.affinityId)
Where c1.affinityId = ?

Can you please explain how ignite breaks and parse and executes this SQL
internally?
 I will also try to spend some time to debug into ingite code to understand
this.


My colocation strategy as follows:
I have around 20 caches. These caches contains subscription's data.
I have around 5000 subscriptions.

Affinity key for all 20 caches in "subscriptionId".  This is done to make
sure that  data in all 20 caches which belongs to same the subscriptionId
will landup on same node.
Although this leaves my cluster unbalanced as data is not distributed
equally.

On Wed, Aug 29, 2018, 3:21 AM vkulichenko 
wrote:

> Prasad,
>
> 1. Yes, you will always see it in the execution plan because there are
> always two stages. Of course, if only one server node participates in the
> execution, the reduce stage is effectively no-op.
>
> 2. Yes, you need to put the annotation to make sure you can access it from
> queries. I would recommend you to treat it in the same way as any other SQL
> field.
>
> 3. The issue here is that you do not join by affinity key, but by user ID.
> This is a non-collocated join, so it can't be executed on a single node in
> the first place. I believe you should revisit your collocation strategy
> here.
>
> 4. The fact that it exists does not mean it would be used. I think you
> should fix the point #3 above before diving into further optimizations.
>
> -Val
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Can Ignite.getOrCreateCache(CacheConfiguration) return null ?

[HEWA WIDANA GAMAGE, SUBASH]

Hi all,
Is there any possibility for this to happen ? We're using Ignite 1.9.0

Following is the code we use to obtain the cache. And we call this line for 
every cache operation(unintentionally), but wanted to know if following line 
can return a null cache instance under any circumstance.

Cache cache = 
getOrCreateCache(CACHE_NAME,CreatedExpiryPolicy.factoryOf(new 
Duration(TimeUnit.SECONDS, 300)));

Re: Affinity key in SQL execution

[Prasad Bhalerao]

3. I did not understand your answer in point 3.

My colocation strategy as follows:
I have around 20 caches. These caches contains subscription's data.
I have around 5000 subscriptions.

Affinity key for all 20 caches in "subscriptionId".  This is done to make
sure that  data in all 20 caches which belongs to same the subscriptionId
will landup on same node.
Although this leaves my cluster unbalanced as data is not distributed
equally.



On Wed, Aug 29, 2018, 3:21 AM vkulichenko 
wrote:

> Prasad,
>
> 1. Yes, you will always see it in the execution plan because there are
> always two stages. Of course, if only one server node participates in the
> execution, the reduce stage is effectively no-op.
>
> 2. Yes, you need to put the annotation to make sure you can access it from
> queries. I would recommend you to treat it in the same way as any other SQL
> field.
>
> 3. The issue here is that you do not join by affinity key, but by user ID.
> This is a non-collocated join, so it can't be executed on a single node in
> the first place. I believe you should revisit your collocation strategy
> here.
>
> 4. The fact that it exists does not mean it would be used. I think you
> should fix the point #3 above before diving into further optimizations.
>
> -Val
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

keep old cache value than new value

[luqmanahmad]

Hi,

Is there any feature in ignite through which I can decide whether to keep
the old value or new value in a cache? We have few cases when the update
comes from the persistence store we want do not want to update the cache
value and want to keep the old one.

Thanks,
Luqman



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: unsubscribe

[vkulichenko]

Ignite community is helpful even to those who leaves :)



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Affinity key in SQL execution

[vkulichenko]

Prasad,

1. Yes, you will always see it in the execution plan because there are
always two stages. Of course, if only one server node participates in the
execution, the reduce stage is effectively no-op.

2. Yes, you need to put the annotation to make sure you can access it from
queries. I would recommend you to treat it in the same way as any other SQL
field.

3. The issue here is that you do not join by affinity key, but by user ID.
This is a non-collocated join, so it can't be executed on a single node in
the first place. I believe you should revisit your collocation strategy
here.

4. The fact that it exists does not mean it would be used. I think you
should fix the point #3 above before diving into further optimizations.

-Val



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Unable to create an index

[bobjoe]

Index query:

CREATE INDEX IF NOT EXISTS DA_Something_Idx 
on
DeviceAvailability(somethingUuid,blahId,blah2Id,blahDate,blah2Date,blahType,blah3Id);



Error in logs (ive removed/fudged some values for privacy reasons):

[15:57:19,973][SEVERE][parallel-idx-worker-Table-1-#55][SchemaIndexCacheVisitorImpl$AsyncWorker]
 
 Error during parallel index create/rebuild.
class org.apache.ignite.IgniteException: Runtime failure on row:
Row@2e91644e[ key: TableKey [idHash=1287041829, hash=-1957195699, REDACTED],
val: Table [idHash=411576018, hash=-1250215456, blahId=16, blah3Id=38690,
UPDATED=Sat Jan 06 02:29:20 PST 2018, somethingUuid=null, blahDate=Fri Oct
20 22:30:00 PDT 2017], ver: GridCacheVersion [topVer=145849619,
order=1534471084063, nodeOrder=2] ][ REDACTED ]
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.doPut(BPlusTree.java:2119)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putx(BPlusTree.java:2066)
at
org.apache.ignite.internal.processors.query.h2.database.H2TreeIndex.putx(H2TreeIndex.java:247)
at
org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$3.apply(IgniteH2Indexing.java:786)
at
org.apache.ignite.internal.processors.cache.GridCacheMapEntry.updateIndex(GridCacheMapEntry.java:3762)
at
org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processKey(SchemaIndexCacheVisitorImpl.java:243)
at
org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartition(SchemaIndexCacheVisitorImpl.java:206)
at
org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.processPartitions(SchemaIndexCacheVisitorImpl.java:165)
at
org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl.access$100(SchemaIndexCacheVisitorImpl.java:50)
at
org.apache.ignite.internal.processors.query.schema.SchemaIndexCacheVisitorImpl$AsyncWorker.body(SchemaIndexCacheVisitorImpl.java:316)
at
org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:110)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.IllegalStateException: Item not found: 2
at
org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.findIndirectItemIndex(AbstractDataPageIO.java:341)
at
org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.getDataOffset(AbstractDataPageIO.java:450)
at
org.apache.ignite.internal.processors.cache.persistence.tree.io.AbstractDataPageIO.readPayload(AbstractDataPageIO.java:492)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:150)
at
org.apache.ignite.internal.processors.cache.persistence.CacheDataRowAdapter.initFromLink(CacheDataRowAdapter.java:102)
at
org.apache.ignite.internal.processors.query.h2.database.H2RowFactory.getRow(H2RowFactory.java:61)
at
org.apache.ignite.internal.processors.query.h2.database.H2Tree.createRowFromLink(H2Tree.java:149)
at
org.apache.ignite.internal.processors.query.h2.database.io.H2ExtrasLeafIO.getLookupRow(H2ExtrasLeafIO.java:126)
at
org.apache.ignite.internal.processors.query.h2.database.io.H2ExtrasLeafIO.getLookupRow(H2ExtrasLeafIO.java:36)
at
org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:167)
at
org.apache.ignite.internal.processors.query.h2.database.H2Tree.getRow(H2Tree.java:46)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.getRow(BPlusTree.java:4482)
at
org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:244)
at
org.apache.ignite.internal.processors.query.h2.database.H2Tree.compare(H2Tree.java:46)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.compare(BPlusTree.java:4469)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.findInsertionPoint(BPlusTree.java:4389)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.access$1500(BPlusTree.java:83)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$Search.run0(BPlusTree.java:278)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4816)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree$GetPageHandler.run(BPlusTree.java:4801)
at
org.apache.ignite.internal.processors.cache.persistence.tree.util.PageHandler.readPage(PageHandler.java:158)
at
org.apache.ignite.internal.processors.cache.persistence.DataStructure.read(DataStructure.java:332)
at
org.apache.ignite.internal.processors.cache.persistence.tree.BPlusTree.putDown(BPlusTree.java:2336)
at

Re: unsubscribe

[Rick Alexander]

Found it ironic that Murthy received support in his unsubscribe request. :D

On Tue, Aug 28, 2018 at 3:50 PM Murthy Kakarlamudi  wrote:

> Thanks Alexey.
>
> On Tue, Aug 28, 2018 at 8:08 AM Alexey Goncharuk <
> alexey.goncha...@gmail.com> wrote:
>
>> Hi Murthy,
>>
>> You should use user-unsubscr...@ignite.apache.org in order to
>> unsubscribe from the list.
>>
>> Cheers,
>> Alexey
>>
>> вт, 28 авг. 2018 г. в 5:37, Murthy Kakarlamudi :
>>
>>>
>>>

Re: unsubscribe

[Murthy Kakarlamudi]

Thanks Alexey.

On Tue, Aug 28, 2018 at 8:08 AM Alexey Goncharuk 
wrote:

> Hi Murthy,
>
> You should use user-unsubscr...@ignite.apache.org in order to unsubscribe
> from the list.
>
> Cheers,
> Alexey
>
> вт, 28 авг. 2018 г. в 5:37, Murthy Kakarlamudi :
>
>>
>>

BinaryMarshaller (micro) Benchmark

[steve.hostettler]

Hello,

I am puzzled by a micro-benchmark I made and that you can find here: 
https://github.com/hostettler/IgnitePerfTest.git
  

To reproduce the below results, just run:

$ mvn clean install
$ cd target
$ java -Xmx512m -Xmx512m -XX:+UseG1GC -jar benchmarks.jar


This yields the following result:
# Run complete. Total time: 00:13:39
Benchmark  Mode  Cnt   ScoreError 
Units
MHBenchmark.igniteRead  avgt  100  1800.052 ± 49.343  ns/op
MHBenchmark.igniteReadKeepBinary  avgt  100  1865.829 ± 45.450  ns/op

So the keepBinary is little bit slower then the standard get.



Here is the question: this very simple test basically compares getting a key
as Binary and directly.

My assumption (that I wanted to validate) was that the binary marshaller and
the withKeepBinary would usually be faster than unmarshalling the whole
object.

This micro-benchmarks proves otherwise. I do not doubt that I am missing
something but I cannot understand what I did wrong.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ICacheEntryProcessor throws BinaryObjectException when WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC

[crenique]

Hi,  We are testing WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC mode to compare
performance to PRIMARY_SYNC.These same code works fine in PRIMARY_SYNC, but
it throws BinaryObjectException on FULL_ASYNC  mode.  Would you please
recommend any way to fix this issue ?Thanks!*Environment: * ignite 2.6 jdk 8 
Windows C# ignite.net*Cache creation: *CREATE TABLE IF NOT EXISTS
UserDataCache(CacheKey VARCHAR,AffinityKey LONG,LastUpdated
TIMESTAMP,Data VARBINARY(MAX),PRIMARY KEY (CacheKey,
AffinityKey))WITH ""BACKUPS=2, TEMPLATE=UserDataCache,
WRITE_SYNCHRONIZATION_MODE=FULL_ASYNC, AFFINITYKEY=AffinityKey,
VALUE_TYPE=UserData""*Cache EntryProcessor for data
upsert*[Serializable]public class UserDataUpsertEntryProcessor :
ICacheEntryProcessor{public int
Process(IMutableCacheEntry entry, UserData addVal)   
{if (entry.Exists){var updateVal = entry.Value; 
  
updateVal.LastUpdated = DateTime.UtcNow;updateVal.Data =
addVal.Data;entry.Value = updateVal;return 1;   
}else{entry.Value = addVal;return 0;
   
}}}*call Invoke*await cache.Invoke(key, new
StateDataUpsertEntryProcessor(), userData);*BinaryObjectException on
FULL_ASYNC
mode*\Ignite\modules\platforms\dotnet\Apache.Ignite.Core\Impl\Binary\BinaryReader.cs,
Ln 537BinaryObjectException: Invalid data on deserialization. Expected:
'System.Int32' But was: null



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Drop table taking more time for execution

[Ilya Kasnacheev]

Hello!

This use case doesn't sound very real-time, so I have to repeat my
question, why is it considered a problem by you?
-- 
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 19:49, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Hi Ilya..
> We load data from various data sources to ignite as tables and run data
> join queries on ignite and sends the response back.
>
> Next time if there is change in the data from the data source in terms of
> addition of new column or delete column...Then we drop the table in ignite
> and recreate it.
>
> Thanks,
> Sriveena
> 
> From: Ilya Kasnacheev 
> Sent: 28 August 2018 20:21:20
> To: user@ignite.apache.org
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> First of all, what is the problem that you are facing? Why DROP TABLE
> taking 15s while being ran infrequently is a cause for concern?
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com >>:
> Thanks Ilya for the quick reply..
>
> We have to drop the table in few cases where the table columns and
> datatypes changes ...
> I guess alter table to add/ drop columns may take same longer time.
>
> Any other alternative to drop table?
>
> Regards,
> Sriveena
> 
> From: Ilya Kasnacheev  ilya.kasnach...@gmail.com>>
> Sent: 28 August 2018 20:04:09
> To: user@ignite.apache.org
> Subject: Re: Drop table taking more time for execution
>
> Hello!
>
> As far as my understanding goes, DROP TABLE will usually destroy a cache,
> and that will require partition map exchange, which in turn will wait for
> all operations to finish. With increase of load and cluster size 15s does
> not seem excessive.
>
> The suggestion is to avoid dropping/creating tables often, and especially
> as the course of normal operation.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com >>:
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>

Re: Drop table taking more time for execution

[Sriveena Mattaparthi]

Hi Ilya..
We load data from various data sources to ignite as tables and run data join 
queries on ignite and sends the response back.

Next time if there is change in the data from the data source in terms of 
addition of new column or delete column...Then we drop the table in ignite and 
recreate it.

Thanks,
Sriveena

From: Ilya Kasnacheev 
Sent: 28 August 2018 20:21:20
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution

Hello!

First of all, what is the problem that you are facing? Why DROP TABLE taking 
15s while being ran infrequently is a cause for concern?

Regards,
--
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Thanks Ilya for the quick reply..

We have to drop the table in few cases where the table columns and datatypes 
changes ...
I guess alter table to add/ drop columns may take same longer time.

Any other alternative to drop table?

Regards,
Sriveena

From: Ilya Kasnacheev 
mailto:ilya.kasnach...@gmail.com>>
Sent: 28 August 2018 20:04:09
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution

Hello!

As far as my understanding goes, DROP TABLE will usually destroy a cache, and 
that will require partition map exchange, which in turn will wait for all 
operations to finish. With increase of load and cluster size 15s does not seem 
excessive.

The suggestion is to avoid dropping/creating tables often, and especially as 
the course of normal operation.

Regards,
--
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi,

Execution of drop table command is taking 15 sec on client node.Any suggestions 
to spewed up the execution.

Thanks & Regards,
Sriveena “Confidentiality Notice: The contents of this email message and any 
attachments are intended solely for the addressee(s) and may contain 
confidential and/or privileged information and may be legally protected from 
disclosure. If you are not the intended recipient of this message or their 
agent, or if this message has been addressed to you in error, please 
immediately alert the sender by reply email and then delete this message and 
any attachments. If you are not the intended recipient, you are hereby notified 
that any use, dissemination, copying, or storage of this message or its 
attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments 
are intended solely for the addressee(s) and may contain confidential and/or 
privileged information and may be legally protected from disclosure. If you are 
not the intended recipient of this message or their agent, or if this message 
has been addressed to you in error, please immediately alert the sender by 
reply email and then delete this message and any attachments. If you are not 
the intended recipient, you are hereby notified that any use, dissemination, 
copying, or storage of this message or its attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments 
are intended solely for the addressee(s) and may contain confidential and/or 
privileged information and may be legally protected from disclosure. If you are 
not the intended recipient of this message or their agent, or if this message 
has been addressed to you in error, please immediately alert the sender by 
reply email and then delete this message and any attachments. If you are not 
the intended recipient, you are hereby notified that any use, dissemination, 
copying, or storage of this message or its attachments is strictly prohibited.”

Re: Data Compresion in Ignite

[Ilya Kasnacheev]

Hello!

1. https://apacheignite.readme.io/docs/memory-architecture
2. No.
3. Roughly the same. Disk has more meta-data that is needed to swap data in
and out, there's WAL, etc... But pages are the same.

Regards,

-- 
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 18:35, ApacheUser :

> Hi Team,
>
> We are using persistent storage . Could you please answer the following.
>
> 1. What is the data format(Binary?) .
> 2. Is it compressed on Disc and in Memory?
> 3. Is the Data format in Memory on Disc same?
>
> Thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: What happens during network failure?

[Lokesh Sharma]

I mainly want to know whether the detached need would join the cluster
automatically after the network is back or not?

On Tue, Aug 28, 2018, 8:09 PM Lokesh Sharma 
wrote:

> What is the expected behaviour of Ignite when a node is unreachable from
> other nodes of the cluster due to *network failure *(and not due to node
> failure), and after a while the node is again accessible from the cluster?
> In other words, one node gets dropped from the cluster due to network
> failure and then again joins the cluster but it does not reboots.
>

Data Compresion in Ignite

[ApacheUser]

Hi Team,

We are using persistent storage . Could you please answer the following. 

1. What is the data format(Binary?) .
2. Is it compressed on Disc and in Memory?
3. Is the Data format in Memory on Disc same?

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: C++ affinity key set up

[ilya.kasnacheev]

Hello!

I have just ran your code with two nodes, and I could not observe any case
where two Key()s will be on different nodes with same id but different flag.
They're always on the same node Can you please elaborate how you got that
result?

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Difference between replicated and local cache mode regarding time execution of query

[ilya.kasnacheev]

Hello!

Can you try setting replicatedOnly(true) hint on SqlFieldsQuery, or
;replicatedOnly=true parameter for Ignite Thin JDBC Driver? This will
prevent query to be distributed between nodes.

Regards,



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Drop table taking more time for execution

[Ilya Kasnacheev]

Hello!

First of all, what is the problem that you are facing? Why DROP TABLE
taking 15s while being ran infrequently is a cause for concern?

Regards,
-- 
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 17:42, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Thanks Ilya for the quick reply..
>
> We have to drop the table in few cases where the table columns and
> datatypes changes ...
> I guess alter table to add/ drop columns may take same longer time.
>
> Any other alternative to drop table?
>
> Regards,
> Sriveena
> --
> *From:* Ilya Kasnacheev 
> *Sent:* 28 August 2018 20:04:09
> *To:* user@ignite.apache.org
> *Subject:* Re: Drop table taking more time for execution
>
> Hello!
>
> As far as my understanding goes, DROP TABLE will usually destroy a cache,
> and that will require partition map exchange, which in turn will wait for
> all operations to finish. With increase of load and cluster size 15s does
> not seem excessive.
>
> The suggestion is to avoid dropping/creating tables often, and especially
> as the course of normal operation.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
> sriveena.mattapar...@ekaplus.com>:
>
> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>
> “Confidentiality Notice: The contents of this email message and any
> attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>

Re: Drop table taking more time for execution

[Sriveena Mattaparthi]

Thanks Ilya for the quick reply..

We have to drop the table in few cases where the table columns and datatypes 
changes ...
I guess alter table to add/ drop columns may take same longer time.

Any other alternative to drop table?

Regards,
Sriveena

From: Ilya Kasnacheev 
Sent: 28 August 2018 20:04:09
To: user@ignite.apache.org
Subject: Re: Drop table taking more time for execution

Hello!

As far as my understanding goes, DROP TABLE will usually destroy a cache, and 
that will require partition map exchange, which in turn will wait for all 
operations to finish. With increase of load and cluster size 15s does not seem 
excessive.

The suggestion is to avoid dropping/creating tables often, and especially as 
the course of normal operation.

Regards,
--
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi 
mailto:sriveena.mattapar...@ekaplus.com>>:
Hi,

Execution of drop table command is taking 15 sec on client node.Any suggestions 
to spewed up the execution.

Thanks & Regards,
Sriveena “Confidentiality Notice: The contents of this email message and any 
attachments are intended solely for the addressee(s) and may contain 
confidential and/or privileged information and may be legally protected from 
disclosure. If you are not the intended recipient of this message or their 
agent, or if this message has been addressed to you in error, please 
immediately alert the sender by reply email and then delete this message and 
any attachments. If you are not the intended recipient, you are hereby notified 
that any use, dissemination, copying, or storage of this message or its 
attachments is strictly prohibited.”
“Confidentiality Notice: The contents of this email message and any attachments 
are intended solely for the addressee(s) and may contain confidential and/or 
privileged information and may be legally protected from disclosure. If you are 
not the intended recipient of this message or their agent, or if this message 
has been addressed to you in error, please immediately alert the sender by 
reply email and then delete this message and any attachments. If you are not 
the intended recipient, you are hereby notified that any use, dissemination, 
copying, or storage of this message or its attachments is strictly prohibited.”

What happens during network failure?

[Lokesh Sharma]

What is the expected behaviour of Ignite when a node is unreachable from
other nodes of the cluster due to *network failure *(and not due to node
failure), and after a while the node is again accessible from the cluster?
In other words, one node gets dropped from the cluster due to network
failure and then again joins the cluster but it does not reboots.

Re: Cache Configuration Templates

[Dave Harvey]

I did a suggested edit adding the Spring configuration of templates.The
rest of the current semantics seem a bit odd, so I was somewhat at a loss
as to what to write.

The wildcard approach means that I have to know the structure of the cache
name a priori.   Seems like there should be a Java API that is equivalent
to CREATE TABLE with allows a cache to be created from an arbitrary
template name, as well as a way to retrieve a copy of the
CacheConfiguration from a template name, so that it can subsequently be
enhanced.  I would assume that Ignite.cache(templateName) would fail if
the template name has a "*" in it, and instantiate a cache otherwise.

On Mon, Aug 27, 2018 at 7:31 AM, Ilya Kasnacheev 
wrote:

> Hello!
>
> Unfortunately, cache templates are not documented that good. AFAIK they
> were mostly implemented to be able to reference to complex cache
> configurations with CREATE TABLE.
>
> As far as my understanding goes, caches from cacheConfigurations are
> actually started when grid starts.
>
> 1) I think that only the first one will be used to create a cache. Even if
> you join a node with distinct cacheConfigurations to the cluster, and it
> already has some caches started, those will just be re-used (by name).
> 2) Yes, you can have a cacheConfiguration with name "prefix*", which will
> lead to a) not starting this cache on grid start, and b) when you start a
> cache "prefix1" it will use configuration from template. There's a test for
> it named IgniteCacheConfigurationTemplateTest in code base.
> 3) Nothing will happen, it will return early if such cache already exists.
> 4) Yes.
> 5) Good question, I will have to check that. Still I won't rely on that
> and just always have this configuration around.
> 6) See above about the '*'.
>
> Regards,
> --
> Ilya Kasnacheev
>
>
> сб, 25 авг. 2018 г. в 0:55, Dave Harvey :
>
>> I found what I've read in this area confusing, and here is my current
>> understanding.
>>
>> When creating an IgniteConfiguration in Java or XML, I can specify the
>> property cacheConfiguration, which is an array of CacheConfigurations.
>> This causes Ignite to preserve these configurations, but this will not
>> cause Ignite to create a cache. If I call Ignite.getOrCreateCache(string),
>> if there is an existing cache, I will get that, otherwise a new cache will
>> be created using that configuration.
>>
>> It seems like creating a cache with a configuration will add to this
>> list, because Ignite.configuration.getCacheConfiguration() returns all
>> caches.
>>
>> I can later call Ignite.addCacheConfiguration(). This will add a
>> template to that list.
>>
>> Questions:
>> 1)  what happens if there are entries with duplicate names on
>> IgniteConfiguration.setCacheConfiguration()   when this is used to
>> create a grid?
>> 2) There was mention in one e-mail talking about a convention where
>> templates have "*" in their name?
>> 3) What happens if addCacheConfiguration() tries to add a duplicate name?
>> 4) Is a template simply a cache that not fully instantiated?
>> 5) What about template persistence?   Are they persisted if they specify
>> a region that is persistent?
>> 6) My use case is that I want to create caches based some default for
>> the cluster, so in Java I would like to construct the new configuration
>> from the a template of a known name.   So far, I can only see that I can
>> call Ignite.configuration.getCacheConfiguration() and then search the
>> array for a matching name.   Is there a better way?
>>
>>
>> *Disclaimer*
>>
>> The information contained in this communication from the sender is
>> confidential. It is intended solely for use by the recipient and others
>> authorized to receive it. If you are not the recipient, you are hereby
>> notified that any disclosure, copying, distribution or taking action in
>> relation of the contents of this information is strictly prohibited and may
>> be unlawful.
>>
>> This email has been scanned for viruses and malware, and may have been
>> automatically archived by *Mimecast Ltd*, an innovator in Software as a
>> Service (SaaS) for business. Providing a *safer* and *more useful* place
>> for your human generated data. Specializing in; Security, archiving and
>> compliance. To find out more Click Here
>> .
>>
>

Disclaimer

The information contained in this communication from the sender is 
confidential. It is intended solely for use by the recipient and others 
authorized to receive it. If you are not the recipient, you are hereby notified 
that any disclosure, copying, distribution or taking action in relation of the 
contents of this information is strictly prohibited and may be unlawful.

This email has been scanned for viruses and malware, and may have been 
automatically archived by Mimecast Ltd, an innovator in Software as a Service 
(SaaS) for business. Providing a safer and more useful place for your human 
generated data. Specializing in; Security, archiving and 

Re: Drop table taking more time for execution

[Ilya Kasnacheev]

Hello!

As far as my understanding goes, DROP TABLE will usually destroy a cache,
and that will require partition map exchange, which in turn will wait for
all operations to finish. With increase of load and cluster size 15s does
not seem excessive.

The suggestion is to avoid dropping/creating tables often, and especially
as the course of normal operation.

Regards,
-- 
Ilya Kasnacheev


вт, 28 авг. 2018 г. в 17:24, Sriveena Mattaparthi <
sriveena.mattapar...@ekaplus.com>:

> Hi,
>
> Execution of drop table command is taking 15 sec on client node.Any
> suggestions to spewed up the execution.
>
> Thanks & Regards,
> Sriveena “Confidentiality Notice: The contents of this email message and
> any attachments are intended solely for the addressee(s) and may contain
> confidential and/or privileged information and may be legally protected
> from disclosure. If you are not the intended recipient of this message or
> their agent, or if this message has been addressed to you in error, please
> immediately alert the sender by reply email and then delete this message
> and any attachments. If you are not the intended recipient, you are hereby
> notified that any use, dissemination, copying, or storage of this message
> or its attachments is strictly prohibited.”
>

Re: ignte cluster hang with GridCachePartitionExchangeManager

[Ilya Kasnacheev]

Hello!

Please check that there are no problems with connectivity in your cluster,
i.e. that all nodes can open communication and discovery connections to all
other nodes.

>From what I observe in the log, there are massive problems with cluster
stability:
23:48:44.624 [tcp-disco-sock-reader-#48%test01%] DEBUG
o.a.i.s.d.tcp.TcpDiscoverySpi  - Message has been added to queue:
TcpDiscoveryNodeLeftMessage [super=TcpDiscoveryAbstractMessage
[sndNodeId=230e516f-6c12-4391-b902-822afc6f7bc4,
id=12c221c7561-710fac06-6272-42f7-a8a8-8f0861c36c63, verifierNodeId=null,
topVer=0, pendingIdx=0, failedNodes=null, isClient=false]]
23:48:44.631 [tcp-disco-msg-worker-#2%test01%] DEBUG
o.a.i.s.d.tcp.TcpDiscoverySpi  - Removed node from topology:
TcpDiscoveryNode [id=230e516f-6c12-4391-b902-822afc6f7bc4,
addrs=[0:0:0:0:0:0:0:1, 127.0.0.1, 192.168.10.103],
sockAddrs=[/0:0:0:0:0:0:0:1:8315, /127.0.0.1:8315, /192.168.10.103:8315],
discPort=8315, order=70, intOrder=38, lastExchangeTime=1535384891465,
loc=false, ver=2.5.0#20180524-sha1:86e110c7, isClient=false]
23:48:44.637 [tcp-disco-msg-worker-#2%test01%] DEBUG
o.a.i.s.d.tcp.TcpDiscoverySpi  - Discarding node left message since sender
node is not in topology: TcpDiscoveryNodeLeftMessage
[super=TcpDiscoveryAbstractMessage
[sndNodeId=230e516f-6c12-4391-b902-822afc6f7bc4,
id=62c221c7561-c261b07c-9495-4058-889b-bd484be10477, verifierNodeId=null,
topVer=0, pendingIdx=0, failedNodes=null, isClient=false]]
and then finally
23:48:44.738 [grid-nio-worker-tcp-comm-1-#26%test01%] DEBUG
o.a.i.s.c.tcp.TcpCommunicationSpi  - Remote client closed connection:
GridSelectorNioSessionImpl [worker=DirectNioClientWorker
[super=AbstractNioClientWorker [idx=1, bytesRcvd=28387, bytesSent=563858,
bytesRcvd0=0, bytesSent0=0, select=true, super=GridWorker
[name=grid-nio-worker-tcp-comm-1, igniteInstanceName=test01,
finished=false, hashCode=686001483, interrupted=false,
runner=grid-nio-worker-tcp-comm-1-#26%test01%]]],
writeBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
readBuf=java.nio.DirectByteBuffer[pos=0 lim=32768 cap=32768],
inRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=28,
sentCnt=28, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
 [id=c261b07c-9495-4058-889b-bd484be10477, addrs=[0:0:0:0:0:0:0:1,
127.0.0.1, 192.168.10.103], sockAddrs=[/0:0:0:0:0:0:0:1:8312, /
127.0.0.1:8312, /192.168.10.103:8312], discPort=8312, order=68,
intOrder=36, lastExchangeTime=1535384891444, loc=false,
ver=2.5.0#20180524-sha1:86e110c7, isClient=false], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
outRecovery=GridNioRecoveryDescriptor [acked=0, resendCnt=0, rcvCnt=28,
sentCnt=28, reserved=true, lastAck=0, nodeLeft=false, node=TcpDiscoveryNode
[id=c261b07c-9495-4058-889b-bd484be10477, addrs=[0:0:0:0:0:0:0:1,
127.0.0.1, 192.168.10.103], sockAddrs=[/0:0:0:0:0:0:0:1:8312, /
127.0.0.1:8312, /192.168.10.103:8312], discPort=8312, order=68,
intOrder=36, lastExchangeTime=1535384891444, loc=false,
ver=2.5.0#20180524-sha1:86e110c7, isClient=false], connected=true,
connectCnt=0, queueLimit=4096, reserveCnt=1, pairedConnections=false],
super=GridNioSessionImpl [locAddr=/0:0:0:0:0:0:0:1:8410,
rmtAddr=/0:0:0:0:0:0:0:1:64102, createTime=1535384893005, closeTime=0,
bytesSent=116175, bytesRcvd=10585, bytesSent0=0, bytesRcvd0=0,
sndSchedTime=1535384893005, lastSndTime=1535384920458,
lastRcvTime=1535384920458, readsPaused=false,
filterChain=FilterChain[filters=[GridNioCodecFilter
[parser=org.apache.ignite.internal.util.nio.GridDirectParser@47fbc56,
directMode=true], GridConnectionBytesVerifyFilter], accepted=true]]
23:48:44.904 [tcp-disco-sock-reader-#48%test01%] ERROR
o.a.i.s.d.tcp.TcpDiscoverySpi  - Caught exception on message read
[sock=Socket[addr=/0:0:0:0:0:0:0:1,port=64098,localport=8310],
locNodeId=016f5c35-ac7d-4391-9142-1a7aea1c3378,
rmtNodeId=230e516f-6c12-4391-b902-822afc6f7bc4]
org.apache.ignite.IgniteCheckedException: Failed to deserialize object with
given class loader: sun.misc.Launcher$AppClassLoader@18b4aac2
at
org.apache.ignite.marshaller.jdk.JdkMarshaller.unmarshal0(JdkMarshaller.java:147)
at
org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:94)
at
org.apache.ignite.internal.util.IgniteUtils.unmarshal(IgniteUtils.java:9907)
at
org.apache.ignite.spi.discovery.tcp.ServerImpl$SocketReader.body(ServerImpl.java:5981)
at
org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
Caused by: java.io.EOFException: null
at
java.io.ObjectInputStream$PeekInputStream.readFully(ObjectInputStream.java:2638)
at
java.io.ObjectInputStream$BlockDataInputStream.readShort(ObjectInputStream.java:3113)
at
java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:853)
at java.io.ObjectInputStream.(ObjectInputStream.java:349)
at

Drop table taking more time for execution

[Sriveena Mattaparthi]

Hi,

Execution of drop table command is taking 15 sec on client node.Any suggestions 
to spewed up the execution.

Thanks & Regards,
Sriveena "Confidentiality Notice: The contents of this email message and any 
attachments are intended solely for the addressee(s) and may contain 
confidential and/or privileged information and may be legally protected from 
disclosure. If you are not the intended recipient of this message or their 
agent, or if this message has been addressed to you in error, please 
immediately alert the sender by reply email and then delete this message and 
any attachments. If you are not the intended recipient, you are hereby notified 
that any use, dissemination, copying, or storage of this message or its 
attachments is strictly prohibited."

Simulate Read Only Caches

[steve.hostettler]

Hello,

I do have a bunch of caches that I would like to have replicated but to keep
them "read only" after a certain point. It is a fairly standard use case.
There are master (exchange rates) that are fixed once and for all for a
given (set of processes). Once loaded there is no reason to bother with
locking and transactionality.


I looked at the implementation and there are quite a number of gates and
checks that are in place. I wonder how to work around these. 

I there a way to simulate this? Maybe there are even more things that we can
temporary disable to speed up common read only use cases.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: apache ignite cassandra persistentStore for enum fields

[Ilya Kasnacheev]

Hello!

I think that VARCHAR option is only available when object is passed to
cache store in deserialized (POJO) form. If it is passed in serialized
(BinaryObject) form then it will always store ordinal.
And, for some reason, after
https://issues.apache.org/jira/browse/IGNITE-4949 for CacheJdbcPojoStore
serialized form is always used.

+ Dmitriy

Why was this change made? Is

if (fieldVal instanceof Enum) {
Enum val = (Enum)fieldVal;

fieldVal = NUMERIC_TYPES.contains(field.getDatabaseFieldType()) ?
val.ordinal() : val.name();
}
block from CacheAbstractJdbcStore still useful after this change?

Regards,
-- 

Ilya Kasnacheev


вт, 28 авг. 2018 г. в 11:57, michal23849 :

> Hi Ilya,
>
> Thank you for your advice. It does work now when I put my class as the
> javaFieldType. However one thing still does bother me.
>
> Previously you advised that Ignite will determine itself if the enum is
> ORDINAL or a STRING and store accordingly - either Ordinal or the Name.
>
> So I have accordingly mapped 2 fields - one as Integer and one as String.
>
> 
>   
>   
>   
>   
>   
>   
> 
>
>
> 
>   
>   
>   
>   
>   
>   
> 
>
> Here is my SQLServer Table:
>
> CREATE TABLE [dbo].[CCache1](
> [ric] [varchar](255) NOT NULL,
> [state] [int] null,
> [type] [varchar](255) null
> )
>
> The problem is that no matter if I map the ENUM to Integer or String - it
> always stores it as Integer, even in Varchar column. Please advise.
>
> Please see the SQLServer results after storing data in Ignite:
> select top 100 ric, state, type from [CCache1]
> ric state   type
> .CSUS0457   1   20
>
>
>
> Thanks,
> Michal
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: How many threads does ignite.executorService() have.

[akurbanov]

Hi,

Executor service may be described as a distributed pool which executes it's
tasks on any cluster node or any node from a given subset of nodes. If you
feel any necessity to configure this thread pool, you may set up public
thread pool size as described here:
https://apacheignite.readme.io/docs/thread-pools#section-public-pool





--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How many threads does ignite.executorService() have.

[Вячеслав Коптилин]

Hello,

Usually, the size of a thread pool is max(8, total number of CPU's).
The details about configuration particular thread pools can be found here:
https://apacheignite.readme.io/docs/thread-pools

Thanks,
S.

вт, 28 авг. 2018 г. в 15:49, begineer :

> Hi Guys,
> I have one simple query about ignite executor service.
> I know java executor service accepts the no of threads to be run, but there
> is no option to pass no of threads to ignite executor service.
>
> So how does it manage threads to be started.
>
> many thanks
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Reaching Ignite's maximum throughput

[Ilya Kasnacheev]

Hello!

I think that centralized databases will have more optimizations for cases
when serializable transactions don't really ever affect each other, but as
you will increase amount of work in transaction, making them eventually
touching each other's data, I expect the performance will be rapidly
approaching the same number for traditional RDBMS.

However, I think we can have more insight if you post your benchmark as a
small reproducer project.

Regards,
-- 
Ilya Kasnacheev


пн, 27 авг. 2018 г. в 20:30, Kia Rahmani :

> Thanks, Ilya for your message.
> I am aware of the cost of enforcing Serializability. What I am more
> concerned about is if I am achieving Ignite's best or not? Specifically,
> since I am not performing any replication and I have only one server right
> now, I should be getting a performance somewhat close a centralized
> database
> which I assume should be much much much more than 600 txns/sec
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Question on DiscoveryCustomEvent

[Alex Plehanov]

Hello,

StopRoutineDiscoveryMessage is sent when a continuous query is stopped
(when query cursor is closed) to unregister listener on all nodes.
DiscoveryCustomEvent is an internal API event, what for you listening to
this type of events?

вт, 28 авг. 2018 г. в 12:43, Sashika :

> Hi,
>
> In one of our code segments, we evaluate a condition to check whether the
> event received by local listener is an instance of DiscoveryCustomEvent.
> Then if it is true we get the customMessage from it and check whether the
> message is an instance of StopRoutineDiscoveryMessage. Our Ignition.start()
> will start in client mode which will connect to an Ignite cluster.
>
> My question is what is the condition that we receive a
> StopRoutineDiscoveryMessage?. Can't find any more information online apart
> from the source code of StopRoutineDiscoveryMessage.java which does not
> have any description given.
>
> Any help would be greatly appreciated
>

How many threads does ignite.executorService() have.

[begineer]

Hi Guys,
I have one simple query about ignite executor service.
I know java executor service accepts the no of threads to be run, but there
is no option to pass no of threads to ignite executor service. 

So how does it manage threads to be started.

many thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

ignite multi thread stop bug

[wangsan wang]

In my test case:see the attachment.I can reproduce the bug

   1.

   testStartServer

   2.

   testMultiClient

   3.

   close testMultiClient

   4.

   then testStartServer will Failed to wait for partition map exchang



I need update cache in DiscoveryEvent. how to fix this?


IgniteMultiThreadTest.java
Description: Binary data

Re: unsubscribe

[Alexey Goncharuk]

Hi Murthy,

You should use user-unsubscr...@ignite.apache.org in order to unsubscribe
from the list.

Cheers,
Alexey

вт, 28 авг. 2018 г. в 5:37, Murthy Kakarlamudi :

>
>

Data migration using persistent Queues

[ipsxd]

Let's imagine the following scenario. We have a queue (distributed in a
cluster and partitioned) with a persistent storage. We add Person ( name :
String ) to it. Then, because of business requirements we will add another
field let's say address which is also o String. What options do we have to
do data migration to support new model and be backwards compatible ?

Possible options that I found :

1. Create a new queue and migrate data programatically - the application
will use the new queue.
2. Store data in another format let's say JSON and provide backwards
compatibility at code level.
3. Is possible to use inheritance, but then we will have versioning in model
which is not desirable.

I think this problem arises also in the context of in memory queues.

Do you have other idea ?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: custom plugin question - jdbc client

[Taras Ledkov]

Hi,

You are absolutely right.
In case you use thin JDBC driver (recommended):
1. You have to define SSLContext factory for client connector for Ignite 
node [1]


2. And setup SSL socket factory for Ignite thin JDBC driver [2]

If you are going to use JDBCv2 driver please keep in mind that the 
JDBCv2 driver starts
the Ignite client node to connect to Ignite cluster and  read the 
documentation [3]


[1] 
org.apache.ignite.configuration.ClientConnectorConfiguration#setSslContextFactory 
(https://ignite.apache.org/releases/latest/javadoc/org/apache/ignite/configuration/ClientConnectorConfiguration.html)
[2] See `sslFactory` property: 
https://apacheignite-sql.readme.io/docs/jdbc-driver#jdbc-thin-driver

[3] https://apacheignite-sql.readme.io/docs/jdbc-client-driver

On 28.08.2018 12:39, wt wrote:

i have finally managed to get a plugin working for a white list on ignite
2.6. I am now going to start working on an authorization for users
connecting to the cluster.

How can i get clients pass through a kerberos ticket to the cluster? I think
i need to override the authorization context class but that would mean that
i need to do it both on the server and the clients for odbc\jdbc etc.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/


--
Taras Ledkov
Mail-To: tled...@gridgain.com

Question on DiscoveryCustomEvent

[Sashika]

Hi,

In one of our code segments, we evaluate a condition to check whether the
event received by local listener is an instance of DiscoveryCustomEvent.
Then if it is true we get the customMessage from it and check whether the
message is an instance of StopRoutineDiscoveryMessage. Our Ignition.start()
will start in client mode which will connect to an Ignite cluster.

My question is what is the condition that we receive a
StopRoutineDiscoveryMessage?. Can't find any more information online apart
from the source code of StopRoutineDiscoveryMessage.java which does not
have any description given.

Any help would be greatly appreciated

Difference between replicated and local cache mode regarding time execution of query

[misraeli]

Hi,
I have this query (used Explain on original query):
SELECT
COUNT(DISTINCT _1__Z3.IP) AS __X0
FROM (
SELECT
ENDPOINT__Z0___KEY AS IP,
ENDPOINT__Z0__ELEMENTID AS TARGET,
CASEWHEN((ENDPOINT__Z0__STATUS IS NOT 'RULES_PENDING_ADD'),
ENDPOINT__Z0__STATUS, CASEWHEN((RULES__Z2.DEPLOYEDCOUNT > 0),
CASEWHEN((RULES__Z2.FAILEDCOUNT > 0), 'DEPLOYMENT_PARTIALLY_FAILED',
'RULES_DEPLOYED_OK'), CASEWHEN((RULES__Z2.FAILEDCOUNT > 0),
'RULES_DEPLOY_FAILED', 'RULES_PENDING'))) AS STATUS,
ENDPOINT__Z0__ENDPOINTTYPE AS EPTYPE,
ENDPOINT__Z0__COMBINATIONID AS COMBINATION
FROM (
SELECT
__C0_0 AS ENDPOINT__Z0___KEY,
__C0_1 AS ENDPOINT__Z0__COMBINATIONID,
__C0_2 AS ENDPOINT__Z0__ENDPOINTTYPE,
__C0_3 AS ENDPOINT__Z0__ELEMENTID,
__C0_4 AS ENDPOINT__Z0__STATUS
FROM PUBLIC.__T0
ORDER BY 1
) __Z5
LEFT OUTER JOIN (
SELECT
__C1_0 AS DESTIP,
SUM(__C1_1) AS DEPLOYEDCOUNT,
SUM(__C1_2) AS FAILEDCOUNT
FROM PUBLIC.__T1
GROUP BY __C1_0
ORDER BY 1
) RULES__Z2
ON ENDPOINT__Z0___KEY = RULES__Z2.DESTIP
ORDER BY 1
) _1__Z3
/* SELECT
ENDPOINT__Z0___KEY AS IP,
ENDPOINT__Z0__ELEMENTID AS TARGET,
CASEWHEN((ENDPOINT__Z0__STATUS IS NOT 'RULES_PENDING_ADD'),
ENDPOINT__Z0__STATUS, CASEWHEN((RULES__Z2.DEPLOYEDCOUNT > 0),
CASEWHEN((RULES__Z2.FAILEDCOUNT > 0), 'DEPLOYMENT_PARTIALLY_FAILED',
'RULES_DEPLOYED_OK'), CASEWHEN((RULES__Z2.FAILEDCOUNT > 0),
'RULES_DEPLOY_FAILED', 'RULES_PENDING'))) AS STATUS,
ENDPOINT__Z0__ENDPOINTTYPE AS EPTYPE,
ENDPOINT__Z0__COMBINATIONID AS COMBINATION
FROM (
SELECT
__C0_0 AS ENDPOINT__Z0___KEY,
__C0_1 AS ENDPOINT__Z0__COMBINATIONID,
__C0_2 AS ENDPOINT__Z0__ENDPOINTTYPE,
__C0_3 AS ENDPOINT__Z0__ELEMENTID,
__C0_4 AS ENDPOINT__Z0__STATUS
FROM PUBLIC.__T0
ORDER BY 1
) __Z5
/++ SELECT
__C0_0 AS ENDPOINT__Z0___KEY,
__C0_1 AS ENDPOINT__Z0__COMBINATIONID,
__C0_2 AS ENDPOINT__Z0__ENDPOINTTYPE,
__C0_3 AS ENDPOINT__Z0__ELEMENTID,
__C0_4 AS ENDPOINT__Z0__STATUS
FROM PUBLIC.__T0
/++ PUBLIC."merge_sorted" ++/
ORDER BY 1
/++ index sorted ++/
 ++/
LEFT OUTER JOIN (
SELECT
__C1_0 AS DESTIP,
SUM(__C1_1) AS DEPLOYEDCOUNT,
SUM(__C1_2) AS FAILEDCOUNT
FROM PUBLIC.__T1
GROUP BY __C1_0
ORDER BY 1
) RULES__Z2
/++ SELECT
__C1_0 AS DESTIP,
SUM(__C1_1) AS DEPLOYEDCOUNT,
SUM(__C1_2) AS FAILEDCOUNT
FROM PUBLIC.__T1
/++ PUBLIC."merge_scan": __C1_0 IS ?1 ++/
WHERE __C1_0 IS ?1
GROUP BY __C1_0
ORDER BY 1: DESTIP = ENDPOINT__Z0___KEY
 ++/
ON ENDPOINT__Z0___KEY = RULES__Z2.DESTIP
ORDER BY 1
 */
/* WHERE _1__Z3.IP IS NOT NULL
*/
LEFT OUTER JOIN (
SELECT
__C2_0 AS PRIORITY__Z4___KEY
FROM PUBLIC.__T2
ORDER BY 1
) __Z6
/* SELECT
__C2_0 AS PRIORITY__Z4___KEY
FROM PUBLIC.__T2
/++ PUBLIC."merge_sorted": __C2_0 IS ?1 ++/
WHERE __C2_0 IS ?1
ORDER BY 1
/++ index sorted ++/: PRIORITY__Z4___KEY = _1__Z3.IP
 */
ON _1__Z3.IP = PRIORITY__Z4___KEY
WHERE _1__Z3.IP IS NOT NULL


The ignite topology snapshot is the following(I am running ignite 2.6):
 /Topology snapshot [ver=1, servers=1, clients=0, CPUs=8, offheap=3.1GB,
heap=2.0GB]/

Sizes if the tables:
/PUBLIC.__T0 is ~30k,
PUBLIC.__T1 is ~100k
PUBLIC.__T2 is 0/

I defined the tables to be in cache mode replicated with the following:
//

If I run the query with that configuration it takes ~1m 30s to get the
result.

But if I change the cache mode to LOCAL it will take <1s to get the result.

Is this accepted behavior? because, as I understand from the documentation
if I want to read using sql query it should work the same with REPLICATED as
LOCAL.

In the document it says ( from
https://apacheignite-sql.readme.io/docs/how-ignite-sql-works):
"First, if a query is executed against a fully REPLICATED data then Ignite
will send it to a single cluster node and run it over the local data there."

And I have only 1 node currently so it should run it over the local data on
this node.

Also, if I had more than one node , shouldn't the running be the same?

Thanks,
Moti




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

custom plugin question - jdbc client

[wt]

i have finally managed to get a plugin working for a white list on ignite
2.6. I am now going to start working on an authorization for users
connecting to the cluster. 

How can i get clients pass through a kerberos ticket to the cluster? I think
i need to override the authorization context class but that would mean that
i need to do it both on the server and the clients for odbc\jdbc etc. 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: apache ignite cassandra persistentStore for enum fields

[michal23849]

Hi Ilya,

Thank you for your advice. It does work now when I put my class as the
javaFieldType. However one thing still does bother me. 

Previously you advised that Ignite will determine itself if the enum is
ORDINAL or a STRING and store accordingly - either Ordinal or the Name.

So I have accordingly mapped 2 fields - one as Integer and one as String.


  
  
  
  
  
  
 



  
  
  
  
  
  
 

Here is my SQLServer Table:

CREATE TABLE [dbo].[CCache1](
[ric] [varchar](255) NOT NULL,
[state] [int] null,
[type] [varchar](255) null
)

The problem is that no matter if I map the ENUM to Integer or String - it
always stores it as Integer, even in Varchar column. Please advise.

Please see the SQLServer results after storing data in Ignite:
select top 100 ric, state, type from [CCache1]
ric state   type
.CSUS0457   1   20



Thanks,
Michal



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to run a job every 5 seconds in Ignite

[Павлухин Иван]

Hi Lokesh,

You could try out extended cron syntax implemented by Ignite [1].

[1]
https://apacheignite.readme.io/docs/cron-based-scheduling#section-syntax-extension


2018-08-28 10:51 GMT+03:00 Lokesh Sharma :

> Is it possible to run the job every few seconds? As far as I know, cron
> API doesn't support scheduling in seconds.
>
> On Tue, Aug 28, 2018 at 11:27 AM Lokesh Sharma 
> wrote:
>
>> This is what I was looking for. Many thanks!
>>
>> On Mon, Aug 27, 2018 at 3:01 PM Evgenii Zhuravlev <
>> e.zhuravlev...@gmail.com> wrote:
>>
>>> Hi Lokesh,
>>>
>>> I'd suggest to start Ignite service, which will guarantee
>>> failover-safety for you: https://apacheignite.readme.
>>> io/docs/service-grid. Just choose cluster-singleton to make sure that
>>> you will have 1 instance of Service in cluster. Inside this service you can
>>> use ignite-scheduler: https://apacheignite.readme.io/docs/
>>> cron-based-scheduling, it has cron API.
>>>
>>> Evgenii
>>>
>>> пн, 27 авг. 2018 г. в 9:16, Lokesh Sharma :
>>>
 I'm using Ignite with Spring Boot. Is there a way to run a job every 5
 seconds on exactly one node of the cluster (which could be any node)?

 Thank You

>>>


-- 
Best regards,
Ivan Pavlukhin

Re: How to run a job every 5 seconds in Ignite

[Lokesh Sharma]

Is it possible to run the job every few seconds? As far as I know, cron API
doesn't support scheduling in seconds.

On Tue, Aug 28, 2018 at 11:27 AM Lokesh Sharma 
wrote:

> This is what I was looking for. Many thanks!
>
> On Mon, Aug 27, 2018 at 3:01 PM Evgenii Zhuravlev <
> e.zhuravlev...@gmail.com> wrote:
>
>> Hi Lokesh,
>>
>> I'd suggest to start Ignite service, which will guarantee failover-safety
>> for you: https://apacheignite.readme.io/docs/service-grid. Just choose
>> cluster-singleton to make sure that you will have 1 instance of Service in
>> cluster. Inside this service you can use ignite-scheduler:
>> https://apacheignite.readme.io/docs/cron-based-scheduling, it has
>> cron API.
>>
>> Evgenii
>>
>> пн, 27 авг. 2018 г. в 9:16, Lokesh Sharma :
>>
>>> I'm using Ignite with Spring Boot. Is there a way to run a job every 5
>>> seconds on exactly one node of the cluster (which could be any node)?
>>>
>>> Thank You
>>>
>>