Or you could hire me on to your team as I specialise in this area. i.e. high performance ingest and stream processing. To give you an idea the platform I am the chief architect of consumes in the north region of 2 billion events per day.
> On 6 Mar 2017, at 16:35, Michael Stolz <[email protected]> wrote: > > Of course if you're REALLY in need of speed you can write your own custom > implementations of toData and fromData for the DataSerializable Interface. > > I haven't seen anyone need that much speed in a long time though. > > > > -- > > Mike Stolz > Principal Engineer - Gemfire Product Manager > Mobile: 631-835-4771 <tel:(631)%20835-4771> > > On Mar 3, 2017 11:16 PM, "Real Wes" <[email protected] > <mailto:[email protected]>> wrote: > Amit, > > > > John and Mike’s advice about tradeoffs is worth heeding. You’ll find that > your speed is probably just fine with putAll but if you just have to have NOS > in your tank, you might consider - since you’re inside a function - do the > putAll from the function into your region but change the region scope to > distributed-no-ack. See: > <>https://geode.apache.org/docs/guide/developing/distributed_regions/choosing_level_of_dist.html > > <https://geode.apache.org/docs/guide/developing/distributed_regions/choosing_level_of_dist.html> > > > Wes > > > > From: Amit Pandey [mailto:[email protected] > <mailto:[email protected]>] > Sent: Friday, March 3, 2017 12:26 PM > To: [email protected] <mailto:[email protected]> > Subject: Re: fastest way to bulk insert in geode > > > > Hey John , > > > > Thanks I am planning to use Spring XD. But my current usecase is that I am > aggregating and doing some computes in a Function and then I want to populate > it with the values I have a map , is region.putAll the fastest? > > > > Regards > > > > On Fri, Mar 3, 2017 at 10:52 PM, John Blum <[email protected] > <mailto:[email protected]>> wrote: > > You might consider using the Snapshot service > <http://gemfire90.docs.pivotal.io/geode/managing/cache_snapshots/chapter_overview.html> > [1] if you previously had data in a Region of another Cluster (for instance). > > > > If the data is coming externally, then Spring XD > <http://projects.spring.io/spring-xd/> [2] is a great tool for moving > (streaming) data from a source > <http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sources> > [3] to a sink > <http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sinks> > [4]. It also allows you to perform all manners of > transformations/conversions, trigger events, and so and so forth. > > > > -j > > > > > > [1] > http://gemfire90.docs.pivotal.io/geode/managing/cache_snapshots/chapter_overview.html > > <http://gemfire90.docs.pivotal.io/geode/managing/cache_snapshots/chapter_overview.html> > [2] http://projects.spring.io/spring-xd/ > <http://projects.spring.io/spring-xd/> > [3] > http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sources > <http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sources> > [4] http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sinks > <http://docs.spring.io/spring-xd/docs/1.3.1.RELEASE/reference/html/#sinks> > > > > > On Fri, Mar 3, 2017 at 9:13 AM, Amit Pandey <[email protected] > <mailto:[email protected]>> wrote: > > Hey Guys, > > > > Whats the fastest way to do bulk insert in a region? > > > > I am using region.putAll , is there any alternative/faster API? > > > > regards > > > > > > > -- > > -John > > john.blum10101 (skype) > > >
