RE: Grid Events and oldValue versus newValue

[gordon.reid@ninemilefinancial]

Sure, thanks Andrey. Please see attached. I have removed some sections which 
aren't relevant such as the service configuration. I have also attached my main 
class.

Thanks,
Gordon.



From: Andrey Mashenkov [via Apache Ignite Users] 
[mailto:ml-node+s70518n10620...@n6.nabble.com]
Sent: Tuesday, 14 February 2017 4:07 PM
To: Gordon Reid (Nine Mile) 
Subject: Re: Grid Events and oldValue versus newValue

Hi Gorgon,

I can't reproduce your issue with default cache configuration on different 
Ignite versions.
Would you please provide grid and cache configuration?

On Tue, Feb 14, 2017 at 1:27 AM, 
gordon.reid@ninemilefinancial <[hidden 
email]> wrote:
Thanks for your reply Andrew.

Actually I did not set copyOnRead manually. Based on your suggestion, I have
now tried setting it manually to true, and then also to false. This does not
seem to change the behavior in my case. I still am not able to see the
previous state of the record on the oldValue property of the event.





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Grid-Events-and-oldValue-versus-newValue-tp10577p10611.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.



If you reply to this email, your message will be added to the discussion below:
http://apache-ignite-users.70518.x6.nabble.com/Grid-Events-and-oldValue-versus-newValue-tp10577p10620.html
To unsubscribe from Grid Events and oldValue versus newValue, click 
here.
NAML


cache_reduced.xml (7K) 

ServiceGrid.java (5K) 





--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Grid-Events-and-oldValue-versus-newValue-tp10577p10625.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: persist periodically

[shawn.du]

exactly.  Thank you, Andrey!






ThanksShawn





On 02/14/2017 13:54，Andrey Mashenkov wrote： 


Hi Shawn,Look at CacheConfiguration.setWriteBehindEnabled [1] and related methods.Does it meets your needs?[1] http://apacheignite.gridgain.org/v1.8/docs/persistent-store#write-behind-cachingOn Tue, Feb 14, 2017 at 8:40 AM, shawn.du  wrote:

 





Hi,ignite support persist data periodically?see blow use case:we want to implement storm state management using ignite.we maintain storm stateful object as caches in ignite.for storm used processing message in very low latency, so the stateful object maybe changed very fast.if ignite persist each change into storage, I think it will both decrease ignite and storm performance.how to make ignite persist periodically? like every seconds.

 



 
ThanksShawn

Re: persist periodically

[Andrey Mashenkov]

Hi Shawn,

Look at CacheConfiguration.setWriteBehindEnabled [1] and related methods.
Does it meets your needs?

[1]
http://apacheignite.gridgain.org/v1.8/docs/persistent-store#write-behind-caching

On Tue, Feb 14, 2017 at 8:40 AM, shawn.du  wrote:

> Hi,
>
> ignite support persist data periodically?
>
> see blow use case:
> we want to implement storm state management using ignite.
> we maintain storm stateful object as caches in ignite.
> for storm used processing message in very low latency, so the stateful
> object maybe changed very fast.
> if ignite persist each change into storage, I think it will both decrease
> ignite and storm performance.
> how to make ignite persist periodically? like every seconds.
>
> Thanks
> Shawn
>
>

persist periodically

[shawn.du]

Hi,ignite support persist data periodically?see blow use case:we want to implement storm state management using ignite.we maintain storm stateful object as caches in ignite.for storm used processing message in very low latency, so the stateful object maybe changed very fast.if ignite persist each change into storage, I think it will both decrease ignite and storm performance.how to make ignite persist periodically? like every seconds.






ThanksShawn

Re: Grid Events and oldValue versus newValue

[Andrey Mashenkov]

Hi Gorgon,

I can't reproduce your issue with default cache configuration on different
Ignite versions.
Would you please provide grid and cache configuration?

On Tue, Feb 14, 2017 at 1:27 AM, gordon.reid@ninemilefinancial <
gordon.r...@ninemilefinancial.com> wrote:

> Thanks for your reply Andrew.
>
> Actually I did not set copyOnRead manually. Based on your suggestion, I
> have
> now tried setting it manually to true, and then also to false. This does
> not
> seem to change the behavior in my case. I still am not able to see the
> previous state of the record on the oldValue property of the event.
>
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Grid-Events-and-oldValue-versus-
> newValue-tp10577p10611.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Equal Distribution of data among Ignite instances

[vkulichenko]

Rishi,

The problem is that you created a partition strategy in which number of
partitions is equal to number of nodes. Affinity functions are designed to
work when number of partitions is much bigger then number of nodes, so you
will have to implement new function, or change the partition strategy (I
would recommend the latter). It's always better to logically separate the
data (i.e. if there is no reason for some pieces of data to be on the same
node, do not force them to do so). This is much more scalable.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Equal-Distribution-of-data-among-Ignite-instances-tp10602p10619.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: simple file based persistent store support TTL

[shawn.du]

Thanks Val, I forgot to enable them. I just got example code and run them.  Now it works.






ThanksShawn





On 02/14/2017 11:40，vkulichenko wrote： 


Hi Shawn,

Did you enable the event (IgniteConfiguration.setIncludeEventTypes(..))?

-Val



--
View this message in context: http://apache-ignite-users.70518.x6.nabble.com/simple-file-based-persistent-store-support-TTL-tp10355p10616.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: loadCache with ExpiryPolicy

[vkulichenko]

Shawn,

I don't think this will be possible in loadCache(). Can you switch to
loading with IgniteDataStreamer? In this case you will be able to create
your own receiver that will save entries with correct expiration.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/loadCache-with-ExpiryPolicy-tp10544p10617.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Register CacheEvent Remote Listener

[vkulichenko]

Hi,

System events are global, so you can't subscribe for a particular cache, but
can filter events by CacheEvent.cacheName(). Also if you're interested in
data update notifications, consider using continuous queries [1]. Events are
designed more for auditing purposes.

[1] https://apacheignite.readme.io/docs/continuous-queries

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Register-CacheEvent-Remote-Listener-tp10588p10615.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Questions on Ignite Clients

[vkulichenko]

Hi,

Please properly subscribe to the mailing list so that the community can
receive email notifications for your messages. To subscribe, send empty
email to user-subscr...@ignite.apache.org and follow simple instructions in
the reply.


abdul shareef wrote
> The client node is also taking time to establish connection to server. We
> tend to open/close connection after using it. Do you intend to have open
> connection between client and server all the time?

Exactly. Ignite client is designed to be started once per application
(usually on startup) and then used from everywhere in this application. It's
thread safe, so can be used by multiple threads concurrently. Indeed, Ignite
client node takes more time to start and connect than thin clients in other
products, but generally this provides much better performance afterwards.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Questions-on-Ignite-Clients-tp10304p10614.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite for fine-grained policy/quota control

[vkulichenko]

Evans,

If you need to atomically update multiple counters, then you indeed need
locks or transaction. If both are applicable, I would recommend using
transactions.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-for-fine-grained-policy-quota-control-tp10563p10613.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Can continuousAsyncQuery guarantee event to be processed in order?

[vkulichenko]

Well, the ordering is actually defined only key level. I.e., if you update
the same key twice, you will get notifications in proper order so that the
last one corresponds to the current value in cache. Ordering between
different keys does not exist as in general case they can arrive from
different nodes.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Can-continuousAsyncQuery-guarantee-event-to-be-processed-in-order-tp10521p10610.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

RE: Effective size limit for cache items in Ignite

[Raymond Wilson]

Hi Pavel,



Thanks for the clarification.



Just curious: If CopyOnRead has no effect in .Net, why is it exposed in the
.Net API?



It sounds like if my code was Java, rather than c#, setting CopyOnRead
result in an object reference being passed to the client with no
transmission of data and no deserialization overhead, even if not using
Binary Mode, is that correct? This suggests a pretty significant
performance penalty if you’re using the .Net API and suggests ‘heavy-lift’
aspects of a solution would be much more performant if implemented in Java.
Is that correct?



My reason for asking is that I have workflows that might access 10’s of
thousands of cached items where each cached item could be several Kb in
size and where responses are returned in 200-300 of milliseconds. Some
quick finger in the air tests indicate the deserialization overhead alone
by using a .Net client (at least for those workflows) is much longer than
that.



Thanks,

Raymond.



*From:* Pavel Tupitsyn [mailto:ptupit...@apache.org]
*Sent:* Tuesday, February 14, 2017 2:16 AM
*To:* user@ignite.apache.org
*Subject:* Re: Effective size limit for cache items in Ignite



Hi Raymond,



CopyOnRead setting has no effect in .NET code.

It exists for cases when there are both Java and .NET nodes in a cluster.



In Ignite.NET each cache.Get call causes two things:

1) Copy a piece of memory from Java (serialized data)

2) Deserialize the data into user type instance.



You can avoid deserialization by using Binary Mode:

https://apacheignite-net.readme.io/docs/binary-mode



Pavel



On Mon, Feb 13, 2017 at 6:38 AM, Raymond Wilson 
wrote:

I found this code snippet in the Ignite v1.8 source:



This appears to be the core place where the value of CopyOnRead has an
effect, though I don’t understand the context of the other logical
conditions around it; it seems that CopyOnRead is dependent on other
configuration state before it will be honoured. I’m still digging to locate
the detailed help around this setting to understand these other conditions.

/** {@inheritDoc} */

@Override public CacheObjectContext contextForCache(CacheConfiguration
ccfg) throws IgniteCheckedException {

assert ccfg != null;



CacheMemoryMode memMode = ccfg.getMemoryMode();



boolean storeVal = !ccfg.isCopyOnRead() || (!isBinaryEnabled(ccfg)
&&

(GridQueryProcessor.isEnabled(ccfg) ||
ctx.config().isPeerClassLoadingEnabled()));



CacheObjectContext res = new CacheObjectContext(ctx,

ccfg.getName(),

ccfg.getAffinityMapper() != null ? ccfg.getAffinityMapper() :
new GridCacheDefaultAffinityKeyMapper(),

ccfg.isCopyOnRead() && memMode != OFFHEAP_VALUES,

storeVal,

ctx.config().isPeerClassLoadingEnabled() &&
!isBinaryEnabled(ccfg));



ctx.resource().injectGeneric(res.defaultAffMapper());



return res;

}



Thanks,

Raymond.



*From:* Raymond Wilson [mailto:raymond_wil...@trimble.com]
*Sent:* Monday, February 13, 2017 2:43 PM
*To:* user@ignite.apache.org
*Subject:* RE: Effective size limit for cache items in Ignite



Ah, I found the CopyOnRead flag in the cache configuration.



Unfortunately, it seems to have the same behaviour regardless of the
setting for this flag.



If I create an example like the below, it seems that querying the same
element from the cache many times takes about the same amount of time in
both cases. Visual Studio also reports large numbers of GC episodes while
it cleans up the large freed MyCacheClass instances. Is this flag only
applicable to Java contexts? I did also try setting KeepBinaryInStore to
true, though there was no noticeable difference.



[Serializable]

public class MyCacheClass

{

public String name = String.Empty;

private byte[] localData = null;



public MyCacheClass(String _name)

{

name = _name;

localData = new byte[400];

}

}



class Program

{

static void Main(string[] args)

{

IIgnite ignite = Ignition.Start();



// Add a cache to Ignite

ICache cache = ignite.CreateCache

(new CacheConfiguration()

{

Name = "TestCache",

CopyOnRead = false,

KeepBinaryInStore = true

});



// Add a cache item

cache.Put("First", new MyCacheClass("FirstItem"));



// query back the cache items

for (int i = 0; i < 3; i++)

{

MyCacheClass first = cache.Get("First");

}

  }





*From:* Raymond Wilson [mailto:raymond_wil...@trimble.com
]
*Sent:* Monday, February 13, 2017 11:35 AM
*To:* user@ignite.apache.org
*Subject:* Effective size limit for cache 

Re: Reduce Movement of data in cluster while data loading in PARTITIONED cache

[vkulichenko]

If you're using streamer, there is no movement between nodes, only from the
node that streams and data nodes. It's already properly maps the data to
nodes, so partition aware approach is obviously redundant here.

Comparing local and distributed cache doesn't make much sense. If the
streamer is on the same node where data is, it will definitely be much
faster, but that limits you to a single node.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Reduce-Movement-of-data-in-cluster-while-data-loading-in-PARTITIONED-cache-tp10570p10608.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Help needed

[vkulichenko]

Hi Ranjit,

Not sure I understood. The main problem with executors is that they are
controlled by Spark and they are created per application. So you can't share
the data stored in embedded mode and it's not really safe to store it there.
This can be useful only for some simple tests/demos, but not for real apps.
Let me know if I'm missing something in your use case.

-Val



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Help-needed-tp10540p10607.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Distributed job scheduling

[Denis Magda]

Hi guys,

There is a relevant discussion on the dev list to introduce such a feature.

Sergei, I would recommend you join the conversation and share your thoughts and 
use case:
http://apache-ignite-developers.2346864.n4.nabble.com/Tasks-Scheduling-and-Chaining-td14293.html
 


—
Denis

> On Feb 13, 2017, at 5:10 AM, dkarachentsev  wrote:
> 
> Hi Sergei,
> 
> Unfortunately IgniteScheduler doesn't provide such guarantees. You will have
> to write your own implementation based on top of compute tasks and
> ComputeTaskSession, or use external job submission from client.
> 
> Thanks.
> 
> -Dmitry.
> 
> 
> 
> --
> View this message in context: 
> http://apache-ignite-users.70518.x6.nabble.com/Distributed-job-scheduling-tp10566p10597.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Reduce Movement of data in cluster while data loading in PARTITIONED cache

[rishi007bansod]

But in my case I am loading data from kafka and not from database, so i am
loading data using ignite data streamer. What can I do for partition aware
data loading in this case(for ignite data streamer)?



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Reduce-Movement-of-data-in-cluster-while-data-loading-in-PARTITIONED-cache-tp10570p10603.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Equal Distribution of data among Ignite instances

[rishi007bansod]

I have 9 ignite server instances I0, I1,..., I8 which has cache in
PARTITIONED mode, in which I am loading data in parallel from partitions P0,
P1.P8 in kafka. Here partition P0, P1P8 contains number of entries
which can be uniquely identified by field  seq_no, also I am using part_ID
for collocating entries from one partition to one instance only. So, i have
defined key as,

class key()
{
int seq_no;
@AffinityKeyMapped
int part_ID; //for collocating entries from one partition to one
instance only
} 

So, I am trying to achieve one to one mapping between cache entries in
ignite instances and partitions e.g. I0->P0, I1->P1, ...,I8->P8. But in
my case mapping I am getting is 

I0-> NULL(No Entries), 
I1-> P5, 
I2-> NULL,
I3-> P7,
I4-> P2, P6
I5-> P1
I6-> P8
I7-> P0, P4
I8-> P3

So, Affinity part is achieved here i.e. entries with same partition ID gets
cached on same ignite instance. But, data is not equally distributed among
ignite instances i.e. I4 and I7 holds 2 partitions' data whereas I0 and I2
does not contain any data. So how can we achieve this equal distribution of
data? 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Equal-Distribution-of-data-among-Ignite-instances-tp10602.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Pessimistic TXN did not release lock on a key, all subsequent txns failed

[bintisepaha]

Hi Andrey,

We are using ignite 1.7.0.
I am attaching the thread dumps. As far as we can tell the grid was working
fine, no threads were hanging. Only subsequent updates to this key were
hung.

The topology was stable. And the client sends an object to be saved to the
server and the transaction is actually only on the server side. 
prodThreadDump20170208.zip

  

Please let us know if you see something useful.

Thanks,
Binti




--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Pessimistic-TXN-did-not-release-lock-on-a-key-all-subsequent-txns-failed-tp10536p10601.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Grid Events and oldValue versus newValue

[Andrey Mashenkov]

Hi Gordon,

It looks like you turn off CacheConfguration.copyOnRead that is true by
default [1].
In this case you update cached value directly bypassing Ignite.

[1]
http://apacheignite.gridgain.org/docs/performance-tips#do-not-copy-value-on-read


On Mon, Feb 13, 2017 at 2:04 AM, gordon.reid@ninemilefinancial <
gordon.r...@ninemilefinancial.com> wrote:

> When updating entities in my cache, the oldValue=newValue in the event,
> unless I do a put with the original object instance. This is of course not
> always possible.
>
> Here is a simplified example.
>
> IgnitePredicate locLsnr = evt -> {
> // do something
> };
> ignite.events().localListen(locLsnr,EventType.EVT_CACHE_OBJECT_PUT);
> IgniteCache cache = ignite.getOrCreateCache("MyCache");
>
> Trade trade1 = new Trade();
> trade1.setId(1);
> trade1.setSize(10);
> cache.put(new TradeKey(trade.getId()), trade1);
> // event is generated
> //evt.oldValue is null, no problem
> trade1.setSize(20);
> cache.put(new TradeKey(trade.getId()), trade1);
> // event is generated
> // evt.oldValue().getSize() is 10, evt.newValue().getSize() is 20, this is
> GOOD
>
> But, if I retrieve the trade from the cache again before updating (from
> another part of the application for example), I am not able to see what the
> old value was in the event. Old value will just show the new value.
>
> IgnitePredicate locLsnr = evt -> {
> // do something
> };
> ignite.events().localListen(locLsnr,EventType.EVT_CACHE_OBJECT_PUT);
> IgniteCache cache = ignite.getOrCreateCache("MyCache");
>
> Trade trade1 = new Trade();
> trade1.setId(1);
> trade1.setSize(10);
> cache.put(new TradeKey(trade.getId()), trade1);
> // event is generated
> //evt.oldValue is null, no problem
> trade1 = cache.get(new TradeKey(1)); // or could be a query or any search
> on
> the cache
> trade1.setSize(20);
> cache.put(new TradeKey(trade.getId()),trade1);
> // event is generated
> // evt.oldValue().getSize() is 20, evt.newValue().getSize() is 20, this is
> BAD
>
> Any advice? Thanks.
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Grid-Events-and-oldValue-versus-newValue-tp10577.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>



-- 
Best regards,
Andrey V. Mashenkov

Re: Pessimistic TXN did not release lock on a key, all subsequent txns failed

[Andrey Mashenkov]

Hi Binti,

It looks like smth happed while transaction was rollingback and some locks
was not released properly.
What Ignite version do you use?

Is it happened on stable topology or not?
Did you see hanging threads?
Are there any clients on topology that was involved in transaction?

Please provide more information, logs and thread-dumps if possible.

On Fri, Feb 10, 2017 at 1:36 AM, bintisepaha  wrote:

> Guys,
>
> We use pessimistic txns in ignite and they have been working great for
> months now, however yesterday we ran into an issue, where on multiple
> updates to the same key (although in interval of minutes), many updates
> went
> through, but at some point, we started getting TransactionTimeoutException
> because the key was locked. After that any update for the same key failed
> with the same error.
>
> We checked the logs for the last successful update and all looks good.
> Something happened that although the txn was completed, they key was still
> not released. We had to actually restart the cluster post business hours to
> resolve the issue.
>
> 1) Is there a way to avoid this?
> 2) Can we monitor objects via visor or jconsole to see if some keys are
> locked for a prolonged period of time?
> 3) Is there a way to free up the lock remotely or by running some salvaging
> code?
>
> Please advise, your help is much appreciated.
>
> Thanks,
> Binti
>
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Pessimistic-TXN-did-not-release-lock-on-a-key-all-
> subsequent-txns-failed-tp10536.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: Effective size limit for cache items in Ignite

[Pavel Tupitsyn]

Hi Raymond,

CopyOnRead setting has no effect in .NET code.
It exists for cases when there are both Java and .NET nodes in a cluster.

In Ignite.NET each cache.Get call causes two things:
1) Copy a piece of memory from Java (serialized data)
2) Deserialize the data into user type instance.

You can avoid deserialization by using Binary Mode:
https://apacheignite-net.readme.io/docs/binary-mode

Pavel

On Mon, Feb 13, 2017 at 6:38 AM, Raymond Wilson 
wrote:

> I found this code snippet in the Ignite v1.8 source:
>
>
>
> This appears to be the core place where the value of CopyOnRead has an
> effect, though I don’t understand the context of the other logical
> conditions around it; it seems that CopyOnRead is dependent on other
> configuration state before it will be honoured. I’m still digging to locate
> the detailed help around this setting to understand these other conditions.
>
> /** {@inheritDoc} */
>
> @Override public CacheObjectContext contextForCache(CacheConfiguration
> ccfg) throws IgniteCheckedException {
>
> assert ccfg != null;
>
>
>
> CacheMemoryMode memMode = ccfg.getMemoryMode();
>
>
>
> boolean storeVal = !ccfg.isCopyOnRead() || (!isBinaryEnabled(ccfg)
> &&
>
> (GridQueryProcessor.isEnabled(ccfg) || ctx.config().
> isPeerClassLoadingEnabled()));
>
>
>
> CacheObjectContext res = new CacheObjectContext(ctx,
>
> ccfg.getName(),
>
> ccfg.getAffinityMapper() != null ? ccfg.getAffinityMapper() :
> new GridCacheDefaultAffinityKeyMapper(),
>
> ccfg.isCopyOnRead() && memMode != OFFHEAP_VALUES,
>
> storeVal,
>
> ctx.config().isPeerClassLoadingEnabled() &&
> !isBinaryEnabled(ccfg));
>
>
>
> ctx.resource().injectGeneric(res.defaultAffMapper());
>
>
>
> return res;
>
> }
>
>
>
> Thanks,
>
> Raymond.
>
>
>
> *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com]
> *Sent:* Monday, February 13, 2017 2:43 PM
> *To:* user@ignite.apache.org
> *Subject:* RE: Effective size limit for cache items in Ignite
>
>
>
> Ah, I found the CopyOnRead flag in the cache configuration.
>
>
>
> Unfortunately, it seems to have the same behaviour regardless of the
> setting for this flag.
>
>
>
> If I create an example like the below, it seems that querying the same
> element from the cache many times takes about the same amount of time in
> both cases. Visual Studio also reports large numbers of GC episodes while
> it cleans up the large freed MyCacheClass instances. Is this flag only
> applicable to Java contexts? I did also try setting KeepBinaryInStore to
> true, though there was no noticeable difference.
>
>
>
> [Serializable]
>
> public class MyCacheClass
>
> {
>
> public String name = String.Empty;
>
> private byte[] localData = null;
>
>
>
> public MyCacheClass(String _name)
>
> {
>
> name = _name;
>
> localData = new byte[400];
>
> }
>
> }
>
>
>
> class Program
>
> {
>
> static void Main(string[] args)
>
> {
>
> IIgnite ignite = Ignition.Start();
>
>
>
> // Add a cache to Ignite
>
> ICache cache = ignite.CreateCache MyCacheClass>
>
> (new CacheConfiguration()
>
> {
>
> Name = "TestCache",
>
> CopyOnRead = false,
>
> KeepBinaryInStore = true
>
> });
>
>
>
> // Add a cache item
>
> cache.Put("First", new MyCacheClass("FirstItem"));
>
>
>
> // query back the cache items
>
> for (int i = 0; i < 3; i++)
>
> {
>
> MyCacheClass first = cache.Get("First");
>
> }
>
>   }
>
>
>
>
>
> *From:* Raymond Wilson [mailto:raymond_wil...@trimble.com
> ]
> *Sent:* Monday, February 13, 2017 11:35 AM
> *To:* user@ignite.apache.org
> *Subject:* Effective size limit for cache items in Ignite
>
>
>
> Hi,
>
>
>
> What is the practical size limit for items in an Ignite cache?
>
>
>
> I suspect the answer is something “As large as the memory you have to hold
> it”, but my question is more aimed at the practicality of large items in a
> cache due to the overhead of pulling copies of the items out of the cache
> in response to a Cache.Get() request.
>
>
>
> For instance, let’s say I had cache items in the 64Kb size range, and had
> requests that commonly refer to those cache items to perform some work on
> them in response to a request. Will each Cache.Get() request require an
> extraction and repackaging of the cache item prior to handing it back to
> the caller as a new (copied) version of that cache item, or is there a way
> for just a reference to the cache item to be returned to the caller?
>
>
>
> I understand there is a way to designate the information in a cache as
> just blobs of 

Re: Distributed job scheduling

[dkarachentsev]

Hi Sergei,

Unfortunately IgniteScheduler doesn't provide such guarantees. You will have
to write your own implementation based on top of compute tasks and
ComputeTaskSession, or use external job submission from client.

Thanks.

-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Distributed-job-scheduling-tp10566p10597.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Ignite client connections hanged

[dkarachentsev]

Hi Raja,

Could you please attach logs for updated and not updated client, and thread
dump for hanged client?

- Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/Ignite-client-connections-hanged-tp10561p10595.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: Register CacheEvent Remote Listener

[Andrey Gura]

Yes, it's possible. Moreover, Ignite doesn't support event listening
only for particular cache (except of possibility to filter out cache
events by cache name).

Just register your listener and you will receive events for all caches:

ignite.events().remoteListen(lsnr, filter, types);

On Mon, Feb 13, 2017 at 6:57 AM, shawn.du  wrote:
> Hi,
>
> I want to only register listeners for cacheEvent, but I don't want to
> specify the particular cache.
> it is possible?
>
> like this code:
> ignite.events().remoteListen(null, new IgnitePredicate()
> {
>
> }
>
> Thanks
> Shawn
>

Re: Help needed

[Ranjit Sahu]

Hi Val,

Actually the idea was not to build something dedicated like the standalone
cluster for ignite.
Do you foresee any issues if i run the ignite inside the executor node.
So when i start my spark job, the ignite cluster will be kind of the
executor nodes and we will be querying again from the executor nodes the
cache.


Thanks,
Ranjit

On Sat, Feb 11, 2017 at 4:13 AM, vkulichenko 
wrote:

> Ranjit,
>
> I would recommend you to start a standalone Ignite cluster, separate from
> Spark. You may also need to specify clientMode=true in the configuration
> provided to IgniteContext, to make sure that nodes on executors are
> clients,
> not servers.
>
> -Val
>
>
>
> --
> View this message in context: http://apache-ignite-users.
> 70518.x6.nabble.com/Help-needed-tp10540p10558.html
> Sent from the Apache Ignite Users mailing list archive at Nabble.com.
>

Re: ignite as Hadoop's Map Reduce engine - how it works?

[Vladimir Ozerov]

Hi,

Our Hadoop Accelerator has its own shuffle algorithm. When jobs request
arrives, we assign mappers and reducers to the most appropriate nodes in
terms of data locality and available resources. 

Shuffle itself adds K-V pairs of local reducer to sorted collection right
away. K-V pairs of remote reducers are packed into batches and sent to them,
then added to sorted collection.

Internally we share job state between nodes with help of cache, this is why
you may see occasional transactions. 



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/ignite-as-Hadoop-s-Map-Reduce-engine-how-it-works-tp10551p10592.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.