Re: Improving Ignite window processing support

[Murthy Kakarlamudi]

Thanks for thinking about this feature. Due to lack of this feature, in our
application we are currently planning to have some Timer based classes
query the cache, perform aggregations and send the events to all the
listeners.
Having this feature built in into Ignite certainly helps.

Satya.
On Mar 25, 2016 10:38 PM, "Roman Shtykh"  wrote:

> Igniters,
> I was thinking about improving Ignite window processing support so that
> queries for windowed data is not user-initiated (which can have timing
> issues) but rather event-driven, similar to ContinuousQuery but being able
> to fire size-based or time-based entry windows to the listening client.
> Some thougths on implementation:1. Guarantee all window events go to the
> same partition (user will need to specify it as a windowed cache). Maybe it
> can be rather implemented by extending IgniteQueue?2. Set a trigger on
> cache, which will listen to eviction (in case of size-based windowed cache)
> or expiration (time-based cache) events of the cache and fire entries.
> It has to be exposed to the user by some API, very roughly something
> likeIgniteStream is = IgniteStream.on(cache).with(windowingType,
> filterPredidate).aggregate(aggFunc)is.run() // to continuously return
> windowed results
> where windowingType is time/size/session-based windows, aggFunc is
> sum/min/max/etc. or user-specified.
> It can be an experimental feature and I think it is a useful API to
> enforce our stream processing, but I would like to know your opinion. Do
> you think such API is needed? I have a limited knowledge of Ignite
> internals -- any other ideas on the implementation?
> For your reference,
> https://flink.apache.org/news/2015/12/04/Introducing-windows.html is a
> good introduction on windows processing.
> -Roman
>
> P.S. Other things like operator/event/ingress time but has to be
> considered for the implementation are omitted.
>

Re: Stupide question about Queue ?

[Murthy Kakarlamudi]

What are the possibilities for swap spaces? Can we persist the cache/queue
content to file system based on the eviction policy?

Thanks,
Murthy.

On Thu, Jan 14, 2016 at 5:23 PM, Valentin Kulichenko <
valentin.kuliche...@gmail.com> wrote:

> Agree with Denis. Sounds like this can be easily achieved by enabling swap
> layer for queue cache. So we just need to put this on configuration.
>
> -Val
>
> On Thu, Jan 14, 2016 at 5:44 AM, Denis Magda  wrote:
>
>> Hi,
>>
>> This seems to be an interesting functionality. As I understand you're
>> referring to [1].
>>
>> Personally I think that it makes sense to support the same in Ignite.
>> However I would just extend CollectionConfiguration adding ability to set
>> an eviction policy and enable swap on demand. When eviction policy faces
>> with a limit entries will be evicted to swap space.
>> Don't think that we need to support other kind of storages here.
>>
>> Igniters, do you have other thoughts on this?
>>
>> [1] http://docs.hazelcast.org/docs/3.5/manual/html/queue-persistence.html
>>
>> --
>> Denis
>>
>>
>> On 1/14/2016 1:41 PM, Yann BLAZART wrote:
>>
>> Hello,
>>
>>
>>
>> In hazelcast we can persist Queues in Database, this  help me for example
>> when consumer are down to no have OOM exception.
>>
>>
>>
>> But I don’t see this possibility in Ignite docs ?
>>
>>
>>
>> This message and any attachments (the "message") is
>> intended solely for the intended addressees and is confidential.
>> If you receive this message in error,or are not the intended
>> recipient(s),
>> please delete it and any copies from your systems and immediately notify
>> the sender. Any unauthorized view, use that does not comply with its
>> purpose,
>> dissemination or disclosure, either whole or partial, is prohibited.
>> Since the internet
>> cannot guarantee the integrity of this message which may not be reliable,
>> BNP PARIBAS
>> (and its subsidiaries) shall not be liable for the message if modified,
>> changed or falsified.
>> Do not print this message unless it is necessary,consider the environment.
>>
>>
>> --
>>
>> Ce message et toutes les pieces jointes (ci-apres le "message")
>> sont etablis a l'intention exclusive de ses destinataires et sont
>> confidentiels.
>> Si vous recevez ce message par erreur ou s'il ne vous est pas destine,
>> merci de le detruire ainsi que toute copie de votre systeme et d'en
>> avertir
>> immediatement l'expediteur. Toute lecture non autorisee, toute
>> utilisation de
>> ce message qui n'est pas conforme a sa destination, toute diffusion ou
>> toute
>> publication, totale ou partielle, est interdite. L'Internet ne permettant
>> pas d'assurer
>> l'integrite de ce message electronique susceptible d'alteration, BNP
>> Paribas
>> (et ses filiales) decline(nt) toute responsabilite au titre de ce message
>> dans l'hypothese
>> ou il aurait ete modifie, deforme ou falsifie.
>> N'imprimez ce message que si necessaire, pensez a l'environnement.
>>
>>
>>
>