What's the difference between ClientCache#removeAll() and ClientCache#clear()

[kcheng.mvp]

Here is the code snippet from 
org.apache.ignite.client.ClientCache

what's the difference between these two methods?

*
/**
 * Clears the contents of the cache.
 */
public void clear() throws ClientException;



/**
 * Removes all of the mappings from this cache.
 * 
 * The order that the individual entries are removed is undefined.
 */
public void removeAll() throws ClientException;

*







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: waiting for partition map exchange questions

[Scott Feldstein]

Hi Andrei, have you had a chance to check out these logs?

Thanks

On Sun, Apr 14, 2019 at 17:18 scottmf  wrote:

> ignite.tgz
> 
> (attached file)
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Ignite DataStreamer Memory Problems

[kellan]

Here is a reproducible example of the DataStreamer memory leak:
https://github.com/kellanburket/ignite-leak

I've also added a public image to DockerHub: miraco/ignite:leak

This can be run on a machine with at least 22GB of memory available to
Docker and probably 50GB of storage between WAL and persistent storage, just
to be safe.
I'm following the guidelines here:
https://apacheignite.readme.io/docs/durable-memory-tuning#section-share-ram

10GB of Durable Memory
4GB of Heap

with a 22GB memory limit in Docker that adds up to about 63% of overall RAM

Now run this container: (You adjust the cpus as needed. I'm using AWS r4.4xl
nodes with 16 cores running Amazon Linux):

docker run -v $LOCAL_STORAGE:$CONTAINER_STORAGE -v $LOCAL_WAL:$CONTAINER_WAL
-m 22G --cpus=12 --memory-swappiness 0 --name ignite.leak -d
miraco/ignite:leak

I would expect memory usage to stabilize somewhere around 18-19GB (4GB Heap
+ 10GB Durable + 640M WAL + 2GB Checkpoint Buffer + 1-2GB jdk overhead), but
instead usage per docker stats rises to the container limit forcing an OOM
kill. Feel free to increase the memory limit above 22GB. Results should be
the same though it make take longer to get there.

Now this is interesting. If I replace the cache value type, which is
Array[Byte] with a Long and run it again, memory usage eventually stabilizes
at around 19-20GB:

docker run -v $LOCAL_STORAGE:$CONTAINER_STORAGE -v $LOCAL_WAL:$CONTAINER_WAL
-e VALUE_TYPE=ValueLong -m 22G --cpus=12 --memory-swappiness 0 --name
ignite.leak -d miraco/ignite:leak

Is there something I'm missing here, or is this a bug?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: What happens when a client gets disconnected

[MattNohelty]

According to some of our historical metrics, the blocking looks to have been
approximately a minute but the granularity of that monitoring is not super
precise so I don't have an exact time.  I can try to go back to our logs and
see if I can determine a more accurate period of time.

How does near caching come into play here?  If near caching is enabled for
these caches, which should have been fully populated so I'd expect a cache
hit pretty much ever time, would you expect the client to ever go back out
to the server nodes?  Is there a straight forward way to determine if a
cache lookup hit the near cache or if it had to go out to the server nodes?



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failover in thin client mode

[Ken Cheng]

Thank your information very much!

Thanks,
Ken Cheng


On Fri, Apr 26, 2019 at 12:32 AM Igor Sapego  wrote:

> The point of "Best effort" is that we do not guarantee every requested
> to be routed to the optimal node, we only apply our best effort to make
> this happen.
>
> No, cluster will NOT return a address list to the client upon handshake
> because.
>
> 1. It may be not secure to connect to nodes, which user did not specified
> in the connection list.
> 2. It is not always possible, as client may be in a different network than
> server nodes, so cluster nodes do not necessary know IPs of nodes, which
> are valid for the client.
>
> Best Regards,
> Igor
>
>
> On Thu, Apr 25, 2019 at 7:25 PM Ken Cheng  wrote:
>
>> I am still on the design phrase of go thin client. once it's ready I will
>> share the link. thanks.
>> Thanks,
>> Ken Cheng
>>
>>
>> On Thu, Apr 25, 2019 at 10:02 PM Igor Sapego  wrote:
>>
>>> Hello,
>>>
>>> Are you developing a Go thin client? Can you share a link?
>>>
>>> There is already one go client I'm aware of [1], is yours related
>>> somehow?
>>>
>>> As for your question, thin client is given list of IPs and ports by a
>>> user. During the start,
>>> or in background thin client tries to establish connection to all of
>>> them. Upon connection,
>>> during handshake, it find outs node ID and associates it with given
>>> IP:port pair. Thin client
>>> should not try to establish connection with any address, which is not
>>> given by a user.
>>>
>>> Does it helps?
>>>
>>> [1] - https://github.com/amsokol/ignite-go-client
>>>
>>> Best Regards,
>>> Igor
>>>
>>>
>>> On Thu, Apr 25, 2019 at 12:17 PM kcheng.mvp 
>>> wrote:
>>>
 Thank your information very much!



 --
 Sent from: http://apache-ignite-users.70518.x6.nabble.com/

>>>

Re: Failover in thin client mode

[Igor Sapego]

The point of "Best effort" is that we do not guarantee every requested
to be routed to the optimal node, we only apply our best effort to make
this happen.

No, cluster will NOT return a address list to the client upon handshake
because.

1. It may be not secure to connect to nodes, which user did not specified
in the connection list.
2. It is not always possible, as client may be in a different network than
server nodes, so cluster nodes do not necessary know IPs of nodes, which
are valid for the client.

Best Regards,
Igor


On Thu, Apr 25, 2019 at 7:25 PM Ken Cheng  wrote:

> I am still on the design phrase of go thin client. once it's ready I will
> share the link. thanks.
> Thanks,
> Ken Cheng
>
>
> On Thu, Apr 25, 2019 at 10:02 PM Igor Sapego  wrote:
>
>> Hello,
>>
>> Are you developing a Go thin client? Can you share a link?
>>
>> There is already one go client I'm aware of [1], is yours related somehow?
>>
>> As for your question, thin client is given list of IPs and ports by a
>> user. During the start,
>> or in background thin client tries to establish connection to all of
>> them. Upon connection,
>> during handshake, it find outs node ID and associates it with given
>> IP:port pair. Thin client
>> should not try to establish connection with any address, which is not
>> given by a user.
>>
>> Does it helps?
>>
>> [1] - https://github.com/amsokol/ignite-go-client
>>
>> Best Regards,
>> Igor
>>
>>
>> On Thu, Apr 25, 2019 at 12:17 PM kcheng.mvp  wrote:
>>
>>> Thank your information very much!
>>>
>>>
>>>
>>> --
>>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>>
>>

Re: Failover in thin client mode

[Ken Cheng]

I am still on the design phrase of go thin client. once it's ready I will
share the link. thanks.
Thanks,
Ken Cheng


On Thu, Apr 25, 2019 at 10:02 PM Igor Sapego  wrote:

> Hello,
>
> Are you developing a Go thin client? Can you share a link?
>
> There is already one go client I'm aware of [1], is yours related somehow?
>
> As for your question, thin client is given list of IPs and ports by a
> user. During the start,
> or in background thin client tries to establish connection to all of them.
> Upon connection,
> during handshake, it find outs node ID and associates it with given
> IP:port pair. Thin client
> should not try to establish connection with any address, which is not
> given by a user.
>
> Does it helps?
>
> [1] - https://github.com/amsokol/ignite-go-client
>
> Best Regards,
> Igor
>
>
> On Thu, Apr 25, 2019 at 12:17 PM kcheng.mvp  wrote:
>
>> Thank your information very much!
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

Re: Failover in thin client mode

[kcheng.mvp]

Not sure my understanding is correct or not.
 There is the case when a new node join a cluster, if it's just as your said
then some key can not get directly from *the new added node*

I suppose it would work as below.

1: 5 node cluster(a,b,c,d,e)
2: even I just set two nodes/endpoints in the client configuration. upon the
first successfully hand-shark, the response should return all the nodes with
IP in the cluster. 
3: clients try to connect other server node.


if the client just connect to the specified nodes/endpoints then for the
other un-specified nodes. it still need 
an additional net hop.







--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failover in thin client mode

[Ken Cheng]

with  IEP-23

there is a change


   1. Tries to establish connection to hosts, which is not yet connected.
   This is required as changes of the topology may be caused by the new node
   joining the cluster.



Thanks,
Ken Cheng


On Thu, Apr 25, 2019 at 10:02 PM Igor Sapego  wrote:

> Hello,
>
> Are you developing a Go thin client? Can you share a link?
>
> There is already one go client I'm aware of [1], is yours related somehow?
>
> As for your question, thin client is given list of IPs and ports by a
> user. During the start,
> or in background thin client tries to establish connection to all of them.
> Upon connection,
> during handshake, it find outs node ID and associates it with given
> IP:port pair. Thin client
> should not try to establish connection with any address, which is not
> given by a user.
>
> Does it helps?
>
> [1] - https://github.com/amsokol/ignite-go-client
>
> Best Regards,
> Igor
>
>
> On Thu, Apr 25, 2019 at 12:17 PM kcheng.mvp  wrote:
>
>> Thank your information very much!
>>
>>
>>
>> --
>> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>>
>

data loss during rebalancing when using invokeAsync + EntryProcessor

[kimec.ethome.sk]

Greetings,

we've been chasing a weird issue in a two node cluster for few days now. 
We have a spring boot application bundled with an ignite server node.


We use invokeAsync on TRANSACTIONAL PARTITIONED cache with 1 backup. We 
assume that each node in the two node cluster has a copy of the other 
node's data. In a way, this mimics REPLICATED cache configuration.  Our 
business logic is written within an EntryProcessor. The "business code" 
in the EntryProcessor is idempotent and arguments to the processor are 
fixed. At the end of the "invokeAsync" call, i.e. when IgniteFuture is 
resolved, we return a value returned from the EntryProcessor via REST to 
the caller of our API.


The problem occurres when one of the two nodes is restarted (triggering 
re-balancing) and we simultaneously receive a call to our REST API 
launching a businesses computation in EntryProcessor.
The code in EntryProcessor properly computes a new value that we want to 
store in the cache. No exception is thrown so we leak it out the REST 
caller as a return value, but when rebalancing finishes, the value is 
not in the cache anymore.
Yet the caller "saw" and stored the value we returned from our 
EntryProcessor.


We did experiment with various cache settings but the problem simply 
persists. In fact we initially used REPLICATED cache configuration but 
the behavior was pretty much the same.


We have currently settled on a rather extreme configuration, but the 
data is still lost during rebalancing from time to time. We are using 
Ignite 2.6 and gatling for REST load testing.

The load on the REST api and consequently on Ignite is not very high.

setAtomicityMode(CacheAtomicityMode.TRANSACTIONAL)
setCacheMode(CacheMode.PARTITIONED)
setBackups(1)
setWriteSynchronizationMode(CacheWriteSynchronizationMode.FULL_SYNC)
setRebalanceMode(CacheRebalanceMode.SYNC)
setPartitionLossPolicy(PartitionLossPolicy.READ_WRITE_SAFE)
setAffinity(new RendezvousAffinityFunction().setPartitions(2))

I would appreciate any pointers what may be wrong with our setup/config.

Thank you.

Kamil

Re: [EXTERNAL] Re: Replace or Put after PutAsync causes Ignite to hang

[Ilya Kasnacheev]

Hello!

Yes, I can reproduce this scenario.

I attribute it to async/await, where there is a .Net callback which is
invoked from cache operation and which calls cache operations, and this
leads to attempts of scheduling striped pool operations to same stripe as
cuirrently occupied.

Pavel, what do you think of that? Stack trace below:

"sys-stripe-0-#1" #12 prio=5 os_prio=0 tid=0x55cecf9e2800 nid=0x3358
waiting on condition [0x7f25b85c7000]
   java.lang.Thread.State: WAITING (parking)
at sun.misc.Unsafe.park(Native Method)
at java.util.concurrent.locks.LockSupport.park(LockSupport.java:304)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get0(GridFutureAdapter.java:178)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.get(GridFutureAdapter.java:141)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2470)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$22.op(GridCacheAdapter.java:2468)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.syncOp(GridCacheAdapter.java:4233)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put0(GridCacheAdapter.java:2468)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.put(GridCacheAdapter.java:2449)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter.replace(GridCacheAdapter.java:2896)
at
org.apache.ignite.internal.processors.cache.IgniteCacheProxyImpl.replace(IgniteCacheProxyImpl.java:1294)
at
org.apache.ignite.internal.processors.cache.GatewayProtectedCacheProxy.replace(GatewayProtectedCacheProxy.java:1012)
at
org.apache.ignite.internal.processors.platform.cache.PlatformCache.processInStreamOutLong(PlatformCache.java:483)
at
org.apache.ignite.internal.processors.platform.PlatformTargetProxyImpl.inStreamOutLong(PlatformTargetProxyImpl.java:67)
at
org.apache.ignite.internal.processors.platform.callback.PlatformCallbackUtils.inLongOutLong(Native
Method)
at
org.apache.ignite.internal.processors.platform.callback.PlatformCallbackGateway.futureNullResult(PlatformCallbackGateway.java:643)
at
org.apache.ignite.internal.processors.platform.utils.PlatformFutureUtils$1.apply(PlatformFutureUtils.java:208)
at
org.apache.ignite.internal.processors.platform.utils.PlatformFutureUtils$1.apply(PlatformFutureUtils.java:189)
at
org.apache.ignite.internal.processors.platform.utils.PlatformFutureUtils$FutureListenable$1.apply(PlatformFutureUtils.java:382)
at
org.apache.ignite.internal.processors.platform.utils.PlatformFutureUtils$FutureListenable$1.apply(PlatformFutureUtils.java:377)
at
org.apache.ignite.internal.util.future.IgniteFutureImpl$InternalFutureListener.apply(IgniteFutureImpl.java:215)
at
org.apache.ignite.internal.util.future.IgniteFutureImpl$InternalFutureListener.apply(IgniteFutureImpl.java:179)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.applyCallback(GridFutureChainListener.java:78)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:70)
at
org.apache.ignite.internal.util.future.GridFutureChainListener.apply(GridFutureChainListener.java:30)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblock(GridFutureAdapter.java:349)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.unblockAll(GridFutureAdapter.java:337)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:497)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:476)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.onDone(GridFutureAdapter.java:453)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$AsyncOpRetryFuture$1.apply(GridCacheAdapter.java:5022)
at
org.apache.ignite.internal.processors.cache.GridCacheAdapter$AsyncOpRetryFuture$1.apply(GridCacheAdapter.java:5017)
at
org.apache.ignite.internal.util.future.GridFutureAdapter.notifyListener(GridFutureAdapter.java:385)
at
org.apache.ignite.internal.

Re: Failover in thin client mode

[Igor Sapego]

Hello,

Are you developing a Go thin client? Can you share a link?

There is already one go client I'm aware of [1], is yours related somehow?

As for your question, thin client is given list of IPs and ports by a user.
During the start,
or in background thin client tries to establish connection to all of them.
Upon connection,
during handshake, it find outs node ID and associates it with given IP:port
pair. Thin client
should not try to establish connection with any address, which is not given
by a user.

Does it helps?

[1] - https://github.com/amsokol/ignite-go-client

Best Regards,
Igor


On Thu, Apr 25, 2019 at 12:17 PM kcheng.mvp  wrote:

> Thank your information very much!
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: Node joining didn't trigger Affinity recalculation problem

[relax ken]

Hi Denis,

I just found an issue with the ec2 tcp discovery. If I have one node
running in elb and launch a new node in elb, they will join the same
cluster properly. However, if I launch two nodes separately first without
joining elb and then add them into the same elb after they are fully
running, they will find each other by ip finder but won't join the cluster.
Any ideas?

Thanks

On Sat, Apr 20, 2019 at 7:01 AM Denis Magda  wrote:

> No problem, thanks for taking the time and sharing the issue. We'll update
> the docs. Good luck with your Ignite deployment!
>
> -
> Denis
>
>
> On Fri, Apr 19, 2019 at 10:32 PM relax ken  wrote:
>
>> Great! Thanks Denis! that's the issue. After I configured it in security
>> group, it works
>>
>> On Sat, Apr 20, 2019 at 6:03 AM Denis Magda  wrote:
>>
>>> By default, each node will be connecting to port 47500 and the next 100
>>> ports if nobody responds on 45700, 457001, etc.
>>>
>>> Please check that the port is opened. Alternatively, you can always
>>> manage the port with TcpDiscoverySpi.setLocalPort [1]. The ports range can
>>> be adjusted as well. [2]
>>>
>>> [1]
>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L771
>>> [2]
>>> https://github.com/apache/ignite/blob/master/modules/core/src/main/java/org/apache/ignite/spi/discovery/tcp/TcpDiscoverySpi.java#L782
>>>
>>> -
>>> Denis
>>>
>>>
>>> On Fri, Apr 19, 2019 at 9:38 PM relax ken  wrote:
>>>
 I don't think both nodes joined the cluster. That is the problem. I
 don't know what it's wrong and why they are not in the cluster. They are in
 the same subnet,. They can talk to each other. They found each other's IP.

 What port does ignite use for cluster joining? It may be security group
 issue. Will ignite throw exception or log warning if they can't join the
 cluster?

 Thanks

 On Sat, 20 Apr 2019, 05:50 Denis Magda,  wrote:

> Hi,
>
> Could you confirm that both nodes on AWS joined the cluster? Data will
> be rebalanced between the nodes once you see 2 server nodes in your 
> cluster
> topology. Might be some networking issue.
>
> -
> Denis
>
>
> On Tue, Apr 16, 2019 at 9:53 AM relax ken  wrote:
>
>> Hi,
>>
>> I wrote an aws ELB based tcp discovery ip finder. When I launched two
>> ec2 nodes in the same target group, the ip finder found node IPs 
>> correctly.
>> However, affinity recalculation wasn't triggered. Basically nothing
>> happened. also no exception. If I ran two nodes locally with multicast,
>> affinity recalculation worked properly. I just wonder what else I need
>> to do to make the custom IP finder works?
>>
>> Here is the overrided method in IPFinder. Basically it just is a
>> slightly modified version of TcpDiscoveryElbIpFinder
>>
>> @Override public Collection getRegisteredAddresses() 
>> throws IgniteSpiException {
>> initClients();
>>
>> logger.debug("elb host discovery");
>> List privateIPs = 
>> awsService.getHealthyPrivateIPsFromTarget(targetGroupArn);
>> List addrs = new ArrayList<>();
>>
>> for (String privateIP : privateIPs) {
>> logger.debug("tcp discovery. found host "+privateIP);
>> addrs.add(new InetSocketAddress(privateIP, 0));
>> }
>>
>> return addrs;
>> }
>>
>> @Override public void registerAddresses(Collection 
>> addrs) throws IgniteSpiException {
>> //No-op, ELB will take care of registration.
>> }
>>
>> @Override public void unregisterAddresses(Collection 
>> addrs) throws IgniteSpiException {
>> // No-op, ELB will take care of this process.
>> }
>>
>> And here is the ignite config setting
>>
>> TcpDiscoverySpi spi = new TcpDiscoverySpi();
>>
>> spi.setIpFinder(getTcpELBIpFinder(elbDiscoveryConfig));
>>
>> IgniteConfiguration igniteConfig = new IgniteConfiguration();
>> igniteConfig.setMetricsLogFrequency(0);
>> igniteConfig.setDiscoverySpi(spi);
>> DataStorageConfiguration storageCfg = new DataStorageConfiguration();
>> storageCfg.getDefaultDataRegionConfiguration().setPersistenceEnabled(false);
>> igniteConfig.setDataStorageConfiguration(storageCfg);
>>
>> Any ideas?
>>
>>
>> Thanks
>>
>>
>> Ken
>>
>>

RE: [EXTERNAL] Re: Replace or Put after PutAsync causes Ignite to hang

[Barney Pippin]

Hi, is anyone else seeing this or able to reproduce? 

Thanks!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: What happens when a client gets disconnected

[Ilya Kasnacheev]

Hello!

"threads on all the other clients block for a period of time" - how long is
this period of time?

It definitely makes sense to try more recent version of Ignite.

The thread dump that you have shown should be only waiting for all data
nodes, which usually are server nodes, so it's not obvious how it is
related to client leaving.

Regards,
-- 
Ilya Kasnacheev


вт, 23 апр. 2019 г. в 20:50, Matt Nohelty :

> What period of time are you asking about?  We deploy fairly regularly so
> our application servers (i.e. the Ignite clients) get restarted at least
> weekly which will trigger a disconnect and reconnect event for each.  We
> have not noticed any issues during our regular release process but in this
> case we are shutting down the Ignite clients gracefully with Ignite#close.
> However, it's also possible that something bad happens on an application
> servers causing it to crash.  This is the scenario where we've seen
> blocking across the cluster.  We'd obviously like our application servers
> to be as independent of one another as possible and it's problematic if an
> issue on one server is allowed to ripple across all of them.
>
> I should have mentioned it in my initial post but we are currently using
> version 2.4.  I received the following response on my Stack Overflow post:
> "When topology changes, partition map exchange is triggered internally. It
> blocks all operations on the cluster. Also in old versions ongoing
> rebalancing was cancelled. But in the latest versions client
> connection/disconnection doesn't affect some processes like this. So, it's
> worth trying the most fresh release"
>
> This comment also mentions PME so it sounds like you both are referencing
> the same behavior.  However, this comment also states that client
> connect/disconnect events do not trigger  PME in the more recent versions
> of Ignite.  Can anyone confirm that this is true, and if so, which version
> was this change made in?
>
> Thank you very much for the help.
>
> On Tue, Apr 23, 2019 at 10:00 AM Ilya Kasnacheev <
> ilya.kasnach...@gmail.com> wrote:
>
>> Hello!
>>
>> What's the period of time?
>>
>> When client disconnects, topology will change, which will trigger waiting
>> for PME, which will delay all further operations until PME is finished.
>>
>> Avoid having short-lived clients.
>>
>> Regards,
>> --
>> Ilya Kasnacheev
>>
>>
>> вт, 23 апр. 2019 г. в 03:40, Matt Nohelty :
>>
>>> I already posted this question to stack overflow here
>>> https://stackoverflow.com/questions/55801760/what-happens-in-apache-ignite-when-a-client-gets-disconnected
>>> but this mailing list is probably more appropriate.
>>>
>>> We use Apache Ignite for caching and are seeing some unexpected behavior
>>> across all of the clients of cluster when one of the clients fails. The
>>> Ignite cluster itself has three servers and there are approximately 12
>>> servers connecting to that cluster as clients. The cluster has persistence
>>> disabled and many of the caches have near caching enabled.
>>>
>>> What we are seeing is that when one of the clients fail (out of memory,
>>> high CPU, network connectivity, etc.), threads on all the other clients
>>> block for a period of time. During these times, the Ignite servers
>>> themselves seem fine but I see things like the following in the logs:
>>>
>>> Topology snapshot [ver=123, servers=3, clients=11, CPUs=XXX, 
>>> offheap=XX.XGB, heap=XXX.GB]Topology snapshot [ver=124, servers=3, 
>>> clients=10, CPUs=XXX, offheap=XX.XGB, heap=XXX.GB]
>>>
>>> The topology itself is clearly changing when a client
>>> connects/disconnects but is there anything happening internally inside the
>>> cluster that could cause blocking on other clients? I would expect
>>> re-balancing of data when a server disconnects but not a client.
>>>
>>> From a thread dump, I see many threads stuck in the following state:
>>>
>>> java.lang.Thread.State: TIMED_WAITING (parking)
>>> at sun.misc.Unsafe.park(Native Method)- parking to wait for  
>>> <0x00078a86ff18> (a java.util.concurrent.CountDownLatch$Sync)
>>> at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
>>> at 
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1037)
>>> at 
>>> java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
>>> at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
>>> at org.apache.ignite.internal.util.IgniteUtils.await(IgniteUtils.java:7452)
>>> at 
>>> org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.awaitAllReplies(GridReduceQueryExecutor.java:1056)
>>> at 
>>> org.apache.ignite.internal.processors.query.h2.twostep.GridReduceQueryExecutor.query(GridReduceQueryExecutor.java:733)
>>> at 
>>> org.apache.ignite.internal.processors.query.h2.IgniteH2Indexing$8.iterator(IgniteH2Indexing.java:1339)
>>> at 
>>> org.apache.ignite.internal.processors.cache

Re: Failover in thin client mode

[kcheng.mvp]

Thank your information very much!



--
Sent from: http://apache-ignite-users.70518.x6.nabble.com/

Re: Failover in thin client mode

[Evgenii Zhuravlev]

Hi,

I believe that this paragraph describes what you want to see:
Updating partition mapping

It is important for client to keep partition mapping updated. To ensure
this, the following changes are proposed:

   1. Changes in Standard Message Response header. It is proposed to add
   flag to every response, that shows whether the Affinity Topology Version of
   the cluster has changed since the last request from the client.
   2. If the Affinity Topology Version of the cluster has changed since the
   last request from the client, new Affinity Topology Version is also
   included in the response.

Once client detects changes in the Affinity Topology, it does the following:

   1. Updates *distributionMap *and *partitionMap* (preferably
   asynchronously) for all active caches. It also may be done "on demand" - on
   the first call to the Cache instance;
   2. Tries to establish connection to hosts, which is not yet connected.
   This is required as changes of the topology may be caused by the new node
   joining the cluster.


More information is available on the wiki page.

Evgenii


чт, 25 апр. 2019 г. в 06:48, kcheng.mvp :

> From the document
>
> https://apacheignite.readme.io/docs/java-thin-client-initialization-and-configuration
>
> knows that ignite **thin** client mode failover is achieved by specified
> multiple hosts(endpoints).
>
>
>
> But I know there is a new feature called  IEP-23: Best Effort Affinity for
> thin clients
> <
> https://cwiki.apache.org/confluence/display/IGNITE/IEP-23%3A+Best+Effort+Affinity+for+thin+clients>
>
>
> my understanding is that this new feature by sending  the cache commands to
> the primary node of the key to improve the performance.
>
> saying there is a cluster with 4 server nodes(a,b,c,d)
>
> in the old fashion(without IEP-23), the service is available if there is at
> least **One** server node is reachable.(for example: configure node-a,
> node-b)
>
> but the same case with IEP-23, how does the client knows the remained
> server
> node in the cluster(ip address and port)? from java ignite thin client I
> know a **Node ID** introduced, not I can not find the corresponding IP,
> Port. from the document
>
> https://apacheignite.readme.io/docs/java-thin-client-initialization-and-configuration
> i can't find information about this.
>
> As right now I am trying to implement an ignite go thin client, that's why
> I
> ask the detail information.
>
>
>
>
>
>
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>

Re: critical thread has been detected

[Evgenii Zhuravlev]

Could you share logs from client nodes? I believe we can find more
useful information there. Also, could you share a full log file, not a
console output? Or, alternatively, you can run Ignite with
-DIGNITE_QUIET=false to enable INFO logging for console output.

Evgenii

чт, 25 апр. 2019 г. в 00:19, siva :

> Sorry, Now i have uploaded file .You can check now
>
> igniteerror.txt
> 
>
>
>
>
> --
> Sent from: http://apache-ignite-users.70518.x6.nabble.com/
>