Re: “Failed to communicate with Ignite cluster" error when using JDBC Thin driver

[gunman524]

We are using mybatis to manage connection, is that matters?

BTW, I check the code in JdbcThinTcpIo method sendRequest 
 
JdbcResponse sendRequest(JdbcRequest req) throws IOException {
int cap = guessCapacity(req);

BinaryWriterExImpl writer = new BinaryWriterExImpl(null, new
BinaryHeapOutputStream(cap), null, null);

req.writeBinary(writer);

send(writer.array());

BinaryReaderExImpl reader = new BinaryReaderExImpl(null, new
BinaryHeapInputStream(read()), null, null, false);

JdbcResponse res = new JdbcResponse();

res.readBinary(reader);

return res;
}

After send() execute,  run read() immediately. Is that might introduce
synchronized issue?  The msg not come back from server when read it from
client out? As in our test env, the insert action has big chance to success
after several times retry. 



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Page eviction mode is ignored

[Denis Magda]

Hi Sergey,

> But... page eviction doesn't work for "inMemory" memory region... Why?

If Ignite persistence is enabled, then the page-based evictions have no effect 
because the oldest pages will be purged from memory automatically.

Updated the text of the warning to makes things clearer.

> Does Ignite supports a mix (persistenceEnabled or not) of DataRegion 
> configurations?

Yes, you can have the persistence enabled for region A and disabled for region 
B.

—
Denis


> On Dec 22, 2017, at 1:13 PM, Sergey Sergeev  wrote:
> 
> Hi, 
> 
> I'm confused. After the successful launch of the Ignite cluster version 
> 2.3.0, I saw the WARN message: 
> WARN  GridCacheDatabaseSharedManager - Page eviction mode for [inMemory] 
> memory region is ignored because Ignite Native Persistence is enabled
> 
> Ignite spring config see below:
> 
> ...
> 
> 
> 
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
> 
> 
> 
> 
> 
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
> 
> 
> 
> 
> 
> 
>  class="org.apache.ignite.configuration.DataRegionConfiguration">
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> ...
> 
> 
> 
> But... page eviction doesn't work for "inMemory" memory region... Why?
> Does Ignite supports a mix (persistenceEnabled or not) of DataRegion 
> configurations?
> 
> -- 
> Best regards,
> Sergey S. Sergeev
> exit(0);

Page eviction mode is ignored

[Sergey Sergeev]

Hi,

I'm confused. After the successful launch of the Ignite cluster version
2.3.0, I saw the WARN message:
WARN  GridCacheDatabaseSharedManager - Page eviction mode for [inMemory]
memory region is ignored because Ignite Native Persistence is enabled

Ignite spring config see below:

...


































...



But... page eviction doesn't work for "inMemory" memory region... Why?
Does Ignite supports a mix (persistenceEnabled or not) of DataRegion
configurations?

-- 
Best regards,
Sergey S. Sergeev
exit(0);

Re: loadAll and removeAll from cache with custom store

[slava.koptilin]

Hi Matt,

I've tried the following code based on your configuration and it works.

public static void main(String[] args) throws Exception {
Ignite ignite = Ignition.start(createIgniteConfiguration());
IgniteCache cache1 =
ignite.getOrCreateCache(createULConfiguration("test-cache-1"));

// upload 10_000 keys
for (int i = 0; i < 10_000; ++i)
cache1.put(i, i);

// remove all keys
cache1.removeAll();

Thread.sleep(10_000);

ignite.close();
}

private static CacheConfiguration createULConfiguration(String name) {
CacheConfiguration cacheConfig = new
CacheConfiguration<>();
cacheConfig.setName(name);
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setBackups(2);

   
cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);

cacheConfig.setWriteBehindEnabled(true);
cacheConfig.setWriteBehindBatchSize(512);
cacheConfig.setWriteBehindFlushSize(10240);
cacheConfig.setWriteBehindFlushFrequency(5_000);

   
cacheConfig.setCacheStoreFactory(FactoryBuilder.factoryOf(CacheStoreExample.class));

cacheConfig.setReadThrough(true);
cacheConfig.setWriteThrough(true);

return cacheConfig;
}

public static class CacheStoreExample extends CacheStoreAdapter {
private Map store = new HashMap();

@Override public Object load(Object key) throws CacheLoaderException
{
return store.get(key);
}
@Override public void write(Cache.Entry entry) throws
CacheWriterException {
store.put(entry.getKey(), entry.getValue());
}
@Override public void delete(Object key) throws CacheWriterException
{
store.remove(key);
}
@Override public void deleteAll(Collection keys) {
System.out.println("CacheStoreExample::deleteAll(), keys.size="
+ keys.size() + " Thread=" + Thread.currentThread().getName());
for (Object k : keys)
store.remove(k);
}
}

The output is following
---
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=512
Thread=flusher-0-#44%test-grid%
CacheStoreExample::deleteAll(), keys.size=272
Thread=flusher-0-#44%test-grid%
[18:16:53] Ignite node stopped OK [name=test-grid, uptime=00:00:05:083]


So, all 10_000 keys were removed.

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

How to Load Data from DataBase to Ignite which have native persistence enabled

[siva]

Hi,

I am very new to ignite,I am working around the examples.


I have used cache store factory to load the data from sql server database to
ignite which is configured with native persistence 

This is my code:

public String getData(String query,  String key) throws Exception
{
String value = "";
Ignite ignite = Ignition.ignite();
String CACHE_NAME = key+"_schemas";
IgniteCache cache= 
ignite.cache(CACHE_NAME);
if(cache==null||cache.size()==0)
{

IgniteCache schemasCache =
createCacheForSchema(key,CacheAtomicityMode.ATOMIC);
schemasCache.loadCache(null);

}
IgniteQueryService service = new IgniteQueryService();
String result = service.query(CACHE_NAME,query);
return result;  
}

private IgniteCache createCacheForSchema(String key,
CacheAtomicityMode atomic)  throws Exception {

Ignite ignite = Ignition.ignite();
String CACHE_NAME = key+"_schemas";
CacheConfiguration cacheCfg = new 
CacheConfiguration(CACHE_NAME);

cacheCfg.setReadThrough(true);
cacheCfg.setWriteThrough(true);

cacheCfg.setCacheMode(CacheMode.PARTITIONED); // Default.
cacheCfg.setIndexedTypes(String.class, EntitySchemas.class);
cacheCfg.setAtomicityMode(atomic);

// get sql connection
String connectionString = getSqlConnection(key, atomic);
// Configure store
cacheCfg.setCacheStoreFactory(new
DataCacheStoreFactory(connectionString));


cacheCfg.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);

return ignite.getOrCreateCache(cacheCfg);
}


Here are my questions:
1. Is this right way to do?
  Because I am using ignite with native persistence enabled but I am using
third party database to load data from database..?
2. What will be the performance issue because it will write in both file
system by ignite native persistence and into sql database by third party
database.? 
 and so

Thanks



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: loadAll and removeAll from cache with custom store

[matt]

Hi,

The size in my store is about 9k. I do have loadAll() working now, so once
that completes, the cache has 9k items as well.

For deleteAll(), I had it work once where it called my adapter multiple
times with all of the expected keys. But then restarting my app and trying
again (after loading new data) it didn't work at all. Do I need to loadAll()
into my Ignite cache before I can delete everything from the backend store?

Here's my configuration:

String name = "foo";

CacheConfiguration cacheConfig = new
CacheConfiguration<>();
cacheConfig.setName(name);
cacheConfig.setCacheMode(CacheMode.PARTITIONED);
cacheConfig.setBackups(2);
   
cacheConfig.setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC);
cacheConfig.setAtomicityMode(CacheAtomicityMode.ATOMIC);

cacheConfig.setWriteBehindEnabled(true);
cacheConfig.setWriteBehindBatchSize(512);
cacheConfig.setWriteBehindFlushSize(10240);
cacheConfig.setWriteBehindFlushFrequency(5_000);

cacheConfig.setCacheStoreFactory(new
MyCacheStoreAdapter.MyCacheStoreFactory(name));
cacheConfig.setReadThrough(true);
cacheConfig.setWriteThrough(true);



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: When do we get this error - Unknown pair [platformId=0, typeId=1078091073]]

[dkarachentsev]

Hi,

Looks like not on all nodes exist your classes. Please check if all classes
that you're using in cache are available on all nodes.

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: loadAll and removeAll from cache with custom store

[slava.koptilin]

Hi Matt,

I think you can use IgniteCache.loadCache() [1] in order to load all
key-value pairs into the cache before running any job.

> For removeAll, I'm seeing that my cache store adapter only ever gets
> called once (deleteAll) but the keys only ever have 1000 keys
I've tried this use case and it works as expected.
Could you please share cache configuration and clarify the size of your
cache before IgniteCache.size(..)?

[1] https://apacheignite.readme.io/docs/data-loading#ignitecacheloadcache

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: How to do 'stream processing' and more questions of a Ignite newbie

[svonn]

Hi,

Thanks alot!
I managed to finally get my interpolation up an running.

Most of my questions were caused by confusing due to weird peer class
loading errors. 
The errors I encountered while having peer class loading enabled apparently
were caused by classes not getting redeployed on the ignite nodes. Hence, I
kept runnining into the very same errors despite editing the code. From what
I've read at 
https://apacheignite.readme.io/docs/deployment-modes
it should work with the standard 'shared' mode, so I'm not quite sure why it
didn't update the classes. 
I'll do some further testing with 'continuous mode', as of now I need to
restart my nodes to redeploy the class.

For now, my only question left is following thing:

Here's a method from my class:

Ignition.setClientMode(true);
Ignite ignite = Ignition.ignite();
IgniteCache accCache =
Ignition.ignite().cache("AccelerationPoint").withKeepBinary();

(1)double interpolationProgress = ((double)
(accPoint.getKey().field("timestamp") -
prevGps.field("timestamp"))) / ((double)
(curGps.field("timestamp") - prevGps.field("timestamp")));
double interpolatedLat = prevGps.field("lat") +
interpolationProgress * (curGps.field("lat") -
prevGps.field("lat"));
double interpolatedLon = prevGps.field("lon") +
interpolationProgress * (curGps.field("lon") -
prevGps.field("lon"));
 
(2)   System.out.println("Previous lat: " + prevGps.field("lat")
+ " ; Interpolated lat: " + interpolatedLat + " ; Next lat: " +
curGps.field("lat"));


(3)accCache.withKeepBinary().invoke(
accPoint.getKey(), new CacheEntryProcessor() {
public Object process(MutableEntry entry,
  Object... objects) throws
EntryProcessorException {
// Create builder from the old value.
BinaryObjectBuilder bldr =
entry.getValue().toBuilder();

//Update the field in the builder.
bldr.setField("lat", interpolatedLat);
bldr.setField("lon", interpolatedLon);

// Set new value to the entry.
entry.setValue(bldr.build());

return null;
}
});

I'm still trying to grasp how Ignite 'decides' to run the code.
For (3) I know that it's run directly on the ignite node. (2) Is printed in
my IDE, thus it has to be executed on the client node.
Where is (2) being calculated? If its on the client node, whats the best way
to transfer that processing to the nodes? If it's on the nodes already, how
does it 'know' where to do it?

Best regards and thanks alot for the help!
Svonn




--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Is it possible to import existing mysql database from file in console?

[dkarachentsev]

Hi,

There are few options:
1) You need to have backups to survive node loss. [1]
2) You may enable persistence to survive grid restart and store more data
that available in memory. [2]
3) Checkout nohup command [3]

[1] https://apacheignite.readme.io/docs/primary-and-backup-copies
[2] https://apacheignite.readme.io/docs/distributed-persistent-store
[3] http://linux.101hacks.com/unix/nohup-command/

Thanks!
-Dmitry



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

When do we get this error - Unknown pair [platformId=0, typeId=1078091073]]

[Naveen]

Hi 

Am using 2.3

I have created cache called Account, was trying to access data thru rest API
with the below URL

http://10.144.114.115:8080/ignite?cmd=get=A10001=Account

This is the response I get 

{"successStatus":1,"sessionToken":null,"error":"Failed to handle request:
[req=CACHE_GET, err=Unknown pair [platformId=0,
typeId=1078091073]]","response":null}

Here is the error log

[16:24:26,844][SEVERE][sys-stripe-6-#7%IgnitePOC%][GridRestProcessor] Failed
to handle request: CACHE_GET
class org.apache.ignite.IgniteCheckedException: Unknown pair [platformId=0,
typeId=1078091073]
at
org.apache.ignite.internal.util.IgniteUtils.cast(IgniteUtils.java:7252)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.resolve(GridFutureAdapter.java:259)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:171)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:140)
at
org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$2.applyx(GridCacheCommandHandler.java:329)
at
org.apache.ignite.internal.processors.rest.handlers.cache.GridCacheCommandHandler$2.applyx(GridCacheCommandHandler.java:325)
at
org.apache.ignite.internal.util.lang.IgniteClosureX.apply(IgniteClosureX.java:38)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:70)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:30)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:383)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:347)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:335)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:495)
at
org.apache.ignite.internal.processors.cache.GridCacheFutureAdapter.onDone(GridCacheFutureAdapter.java:55)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:474)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.onDone(GridPartitionedSingleGetFuture.java:740)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:462)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.setResult(GridPartitionedSingleGetFuture.java:655)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridPartitionedSingleGetFuture.onResult(GridPartitionedSingleGetFuture.java:506)
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtCacheAdapter.processNearSingleGetResponse(GridDhtCacheAdapter.java:348)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache.access$1400(GridDhtAtomicCache.java:129)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$15.apply(GridDhtAtomicCache.java:421)
at
org.apache.ignite.internal.processors.cache.distributed.dht.atomic.GridDhtAtomicCache$15.apply(GridDhtAtomicCache.java:416)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.processMessage(GridCacheIoManager.java:1060)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.onMessage0(GridCacheIoManager.java:579)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:378)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.handleMessage(GridCacheIoManager.java:304)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager.access$100(GridCacheIoManager.java:99)
at
org.apache.ignite.internal.processors.cache.GridCacheIoManager$1.onMessage(GridCacheIoManager.java:293)
at
org.apache.ignite.internal.managers.communication.GridIoManager.invokeListener(GridIoManager.java:1555)
at
org.apache.ignite.internal.managers.communication.GridIoManager.processRegularMessage0(GridIoManager.java:1183)
at
org.apache.ignite.internal.managers.communication.GridIoManager.access$4200(GridIoManager.java:126)
at
org.apache.ignite.internal.managers.communication.GridIoManager$9.run(GridIoManager.java:1090)
at
org.apache.ignite.internal.util.StripedExecutor$Stripe.run(StripedExecutor.java:505)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.ClassNotFoundException: Unknown pair [platformId=0,
typeId=1078091073]
at
org.apache.ignite.internal.MarshallerContextImpl.getClassName(MarshallerContextImpl.java:376)
at

Re: How to connect apache ignite node by static ip address

[manatee]

Mission complete. I created VPN server on my hetzner server and connect my
client node to vpn.



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/