Re: Improving Ignite window processing support
Thanks for thinking about this feature. Due to lack of this feature, in our application we are currently planning to have some Timer based classes query the cache, perform aggregations and send the events to all the listeners. Having this feature built in into Ignite certainly helps. Satya. On Mar 25, 2016 10:38 PM, "Roman Shtykh" wrote: > Igniters, > I was thinking about improving Ignite window processing support so that > queries for windowed data is not user-initiated (which can have timing > issues) but rather event-driven, similar to ContinuousQuery but being able > to fire size-based or time-based entry windows to the listening client. > Some thougths on implementation:1. Guarantee all window events go to the > same partition (user will need to specify it as a windowed cache). Maybe it > can be rather implemented by extending IgniteQueue?2. Set a trigger on > cache, which will listen to eviction (in case of size-based windowed cache) > or expiration (time-based cache) events of the cache and fire entries. > It has to be exposed to the user by some API, very roughly something > likeIgniteStream is = IgniteStream.on(cache).with(windowingType, > filterPredidate).aggregate(aggFunc)is.run() // to continuously return > windowed results > where windowingType is time/size/session-based windows, aggFunc is > sum/min/max/etc. or user-specified. > It can be an experimental feature and I think it is a useful API to > enforce our stream processing, but I would like to know your opinion. Do > you think such API is needed? I have a limited knowledge of Ignite > internals -- any other ideas on the implementation? > For your reference, > https://flink.apache.org/news/2015/12/04/Introducing-windows.html is a > good introduction on windows processing. > -Roman > > P.S. Other things like operator/event/ingress time but has to be > considered for the implementation are omitted. >
Re: Stupide question about Queue ?
What are the possibilities for swap spaces? Can we persist the cache/queue content to file system based on the eviction policy? Thanks, Murthy. On Thu, Jan 14, 2016 at 5:23 PM, Valentin Kulichenko < valentin.kuliche...@gmail.com> wrote: > Agree with Denis. Sounds like this can be easily achieved by enabling swap > layer for queue cache. So we just need to put this on configuration. > > -Val > > On Thu, Jan 14, 2016 at 5:44 AM, Denis Magda wrote: > >> Hi, >> >> This seems to be an interesting functionality. As I understand you're >> referring to [1]. >> >> Personally I think that it makes sense to support the same in Ignite. >> However I would just extend CollectionConfiguration adding ability to set >> an eviction policy and enable swap on demand. When eviction policy faces >> with a limit entries will be evicted to swap space. >> Don't think that we need to support other kind of storages here. >> >> Igniters, do you have other thoughts on this? >> >> [1] http://docs.hazelcast.org/docs/3.5/manual/html/queue-persistence.html >> >> -- >> Denis >> >> >> On 1/14/2016 1:41 PM, Yann BLAZART wrote: >> >> Hello, >> >> >> >> In hazelcast we can persist Queues in Database, this help me for example >> when consumer are down to no have OOM exception. >> >> >> >> But I don’t see this possibility in Ignite docs ? >> >> >> >> This message and any attachments (the "message") is >> intended solely for the intended addressees and is confidential. >> If you receive this message in error,or are not the intended >> recipient(s), >> please delete it and any copies from your systems and immediately notify >> the sender. Any unauthorized view, use that does not comply with its >> purpose, >> dissemination or disclosure, either whole or partial, is prohibited. >> Since the internet >> cannot guarantee the integrity of this message which may not be reliable, >> BNP PARIBAS >> (and its subsidiaries) shall not be liable for the message if modified, >> changed or falsified. >> Do not print this message unless it is necessary,consider the environment. >> >> >> -- >> >> Ce message et toutes les pieces jointes (ci-apres le "message") >> sont etablis a l'intention exclusive de ses destinataires et sont >> confidentiels. >> Si vous recevez ce message par erreur ou s'il ne vous est pas destine, >> merci de le detruire ainsi que toute copie de votre systeme et d'en >> avertir >> immediatement l'expediteur. Toute lecture non autorisee, toute >> utilisation de >> ce message qui n'est pas conforme a sa destination, toute diffusion ou >> toute >> publication, totale ou partielle, est interdite. L'Internet ne permettant >> pas d'assurer >> l'integrite de ce message electronique susceptible d'alteration, BNP >> Paribas >> (et ses filiales) decline(nt) toute responsabilite au titre de ce message >> dans l'hypothese >> ou il aurait ete modifie, deforme ou falsifie. >> N'imprimez ce message que si necessaire, pensez a l'environnement. >> >> >> >
Re: Hello!
That helps a lot. Thanks much for the information. Murthy. On Sep 8, 2015 7:59 AM, "Sergi Vladykin" wrote: > Guys, > > The approach I'll suggest you will be quite common for fairly any code base > you will work on. > > 1. Get familiar with current features, usage scenarios and respective > public APIs of the product. > > You can look at http://ignite.apache.org/features.html and > http://ignite.apache.org/usecases.html > More extensive docs are here https://apacheignite.readme.io/docs > Usage examples are here > https://github.com/apache/ignite/tree/master/examples > > 2. Setup project from Github or Apache git repo. With Github it is easier > to work in my experience. > Links are here http://ignite.apache.org/community/resources.html#git > > Apache Ignite is just a usual Maven project. It is preferable to use IDEA > from JetBrains https://www.jetbrains.com/idea/ > (the project is complex, has many Maven profiles, I'm not sure if other > IDEs will work out for you). > > 3. Edit and run examples under Java debugger to see what happens internally > in Ignite. > > 4. When you have some understanding of how things work, you can take one of > Jira issues to work on > here http://ignite.apache.org/community/contribute.html#contribute > > or directly from Jira > > https://issues.apache.org/jira/browse/IGNITE-1226?jql=project%20%3D%20IGNITE%20AND%20component%20in%20%28newbie%29 > > Have fun! > > Sergi > > > > > 2015-09-08 14:25 GMT+03:00 Murthy Kakarlamudi : > > > Absolutely, that makes sense. As a newbie, I am trying to understand > where > > I should start from to understand the framework. Probably anyone in here > > can share that perspective, to get a handle of the different classes and > > their purpose. > > > > Thanks, > > Murthy. > > > > On Tue, Sep 8, 2015 at 7:18 AM, chandresh pancholi < > > chandreshpancholi...@gmail.com> wrote: > > > > > Murthy/Sudhanshu, > > > IMO Please use ignite dev list for further discussion or communication > > your > > > discussion might help new contributors to early ramp up. > > > > > > Thanks > > > > > > On Tue, Sep 8, 2015 at 4:37 PM, Murthy Kakarlamudi > > > wrote: > > > > > > > Hello Sudhanshu, > > > > My name is Murthy. I also joined Ignite Dev group recently and am > > > > trying to understand it. Probably we can share our understandings > > before > > > we > > > > can contribute to issues. My id is ksa...@gmail.com. Please share > your > > > id > > > > too. > > > > > > > > Thanks, > > > > Murthy. > > > > > > > > On Mon, Sep 7, 2015 at 5:06 AM, Sudhanshu < > sudhanshu97gu...@gmail.com> > > > > wrote: > > > > > > > > > Hello Developers, > > > > > > > > > > I am Sudhanshu Gupta, software engineer working in Bangalore, > India. > > > > > > > > > > I am willing to contribute to the project. I have forked and cloned > > the > > > > > repo in my local. I want to get start on it. It would be great if > > > anyone > > > > > from the community can help to understand the codebase and let me > fix > > > > some > > > > > issues. > > > > > > > > > > My Jira id: sudhanshu-gupta > > > > > > > > > > -- > > > > > Regards, > > > > > Sudhanshu > > > > > > > > > > > > > > > > > > > > > -- > > > Chandresh Pancholi > > > Senior Software Engineer > > > Flipkart.com > > > Email-id:chandresh.panch...@flipkart.com > > > Contact:08951803660 > > > > > >
Re: Hello!
Absolutely, that makes sense. As a newbie, I am trying to understand where I should start from to understand the framework. Probably anyone in here can share that perspective, to get a handle of the different classes and their purpose. Thanks, Murthy. On Tue, Sep 8, 2015 at 7:18 AM, chandresh pancholi < chandreshpancholi...@gmail.com> wrote: > Murthy/Sudhanshu, > IMO Please use ignite dev list for further discussion or communication your > discussion might help new contributors to early ramp up. > > Thanks > > On Tue, Sep 8, 2015 at 4:37 PM, Murthy Kakarlamudi > wrote: > > > Hello Sudhanshu, > > My name is Murthy. I also joined Ignite Dev group recently and am > > trying to understand it. Probably we can share our understandings before > we > > can contribute to issues. My id is ksa...@gmail.com. Please share your > id > > too. > > > > Thanks, > > Murthy. > > > > On Mon, Sep 7, 2015 at 5:06 AM, Sudhanshu > > wrote: > > > > > Hello Developers, > > > > > > I am Sudhanshu Gupta, software engineer working in Bangalore, India. > > > > > > I am willing to contribute to the project. I have forked and cloned the > > > repo in my local. I want to get start on it. It would be great if > anyone > > > from the community can help to understand the codebase and let me fix > > some > > > issues. > > > > > > My Jira id: sudhanshu-gupta > > > > > > -- > > > Regards, > > > Sudhanshu > > > > > > > > > -- > Chandresh Pancholi > Senior Software Engineer > Flipkart.com > Email-id:chandresh.panch...@flipkart.com > Contact:08951803660 >
Re: Hello!
Hello Sudhanshu, My name is Murthy. I also joined Ignite Dev group recently and am trying to understand it. Probably we can share our understandings before we can contribute to issues. My id is ksa...@gmail.com. Please share your id too. Thanks, Murthy. On Mon, Sep 7, 2015 at 5:06 AM, Sudhanshu wrote: > Hello Developers, > > I am Sudhanshu Gupta, software engineer working in Bangalore, India. > > I am willing to contribute to the project. I have forked and cloned the > repo in my local. I want to get start on it. It would be great if anyone > from the community can help to understand the codebase and let me fix some > issues. > > My Jira id: sudhanshu-gupta > > -- > Regards, > Sudhanshu >
