Re: How do you serve the data computed by Samza?

the right things", so > that even newcomers to Samza can have an easy way to ingest data into their > serving system. > > My 2 cents. > > -- > > Felix GV > Data Infrastructure Engineer > Distributed Data Systems > LinkedIn > > f...@linkedin.com > linkedin.

RE: How do you serve the data computed by Samza?

__ From: Roger Hoover [roger.hoo...@gmail.com] Sent: Thursday, April 02, 2015 3:24 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? Is it because the Kafka partitioning might not be the same as the storage partitioning? So that a slow storage shard will pr

Re: How do you serve the data computed by Samza?

; scenarios, but does not support very high throughput of ingestion like >> > > Kafka can provide. By co-locating the pulling process (i.e.: Kafka >> > > consumer) with the data serving node, it makes it a bit more >> affordable >> > to >> > > do single

Re: How do you serve the data computed by Samza?

whatever point in the past it needs (i.e.: rewind) > without > > > affecting the other replicas. Tracking offsets across many replicas in > > the > > > push model is a bit more annoying, though still doable, of course. > > > > > > -- > > > >

RE: How do you serve the data computed by Samza?

cture Engineer Distributed Data Systems LinkedIn f...@linkedin.com linkedin.com/in/felixgv From: Chinmay Soman [chinmay.cere...@gmail.com] Sent: Thursday, April 02, 2015 2:54 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza?

Re: How do you serve the data computed by Samza?

the past it needs (i.e.: rewind) without > > affecting the other replicas. Tracking offsets across many replicas in > the > > push model is a bit more annoying, though still doable, of course. > > > > -- > > > > Felix GV > > Data Infrastructure Engineer >

Re: How do you serve the data computed by Samza?

stems > LinkedIn > > f...@linkedin.com > linkedin.com/in/felixgv > > ____________ > From: Roger Hoover [roger.hoo...@gmail.com] > Sent: Tuesday, March 31, 2015 8:57 PM > To: dev@samza.apache.org > Subject: Re: How do you serve the data comput

RE: How do you serve the data computed by Samza?

Sent: Wednesday, April 01, 2015 12:49 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? Felix, Since the webservices call is not maintained in a single session, how do we tie up the incoming request to the enriched kv pair? For example, lets take a use case a

Re: How do you serve the data computed by Samza?

Infrastructure Engineer > Distributed Data Systems > LinkedIn > > f...@linkedin.com > linkedin.com/in/felixgv > > > From: Shekar Tippur [ctip...@gmail.com] > Sent: Wednesday, April 01, 2015 6:54 AM > To: dev@samza.apache.org &g

RE: How do you serve the data computed by Samza?

lixgv From: Shekar Tippur [ctip...@gmail.com] Sent: Wednesday, April 01, 2015 6:54 AM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? I am still not fully sure how this would pan out. Since at each stage, producer sends an event and has no control over the con

RE: How do you serve the data computed by Samza?

ed Data Systems LinkedIn f...@linkedin.com linkedin.com/in/felixgv From: Roger Hoover [roger.hoo...@gmail.com] Sent: Tuesday, March 31, 2015 8:57 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? Ah, thanks for the gr

Re: How do you serve the data computed by Samza?

if RocksDB was used in both >> Samza and the serving system, it may benefit from being tuned >> differently (i.e.: one for write throughput and the other for read >> performance). >> >> >> -- >> >> Felix GV >> Data Infrastructure Enginee

Re: How do you serve the data computed by Samza?

nkedin.com linkedin.com/in/felixgv From: Harlan Iverson [har...@pubnub.com] Sent: Saturday, March 28, 2015 5:30 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? Felix/Shekar, Given that Samza itself uses RocksDB to create a ma

Re: How do you serve the data computed by Samza?

Ah, thanks for the great explanation. Any particular reason that the job(s) you described should not be Samza jobs? We're started experimenting with such jobs for Druid and Elasticsearch. For Elasticsearch, the Samza job containers join the Elasticsearch cluster as transport nodes and use the Jav

RE: How do you serve the data computed by Samza?

Thanks for your reply Roger! Very insightful (: > 6. If there was a highly-optimized and reliable way of ingesting > partitioned streams quickly into your online serving system, would that > help you leverage Samza more effectively? >> 6. Can you elaborate please? Sure. The feature set I have in

RE: How do you serve the data computed by Samza?

How do you serve the data computed by Samza? Felix/Shekar, Given that Samza itself uses RocksDB to create a materialized view of a partition from earliest to latest offset, I imagine that to be a good choice to begin evaluating. Two parts: 1. A method recommended in this article/v

Re: How do you serve the data computed by Samza?

Hi Felix, 1,3. We're experimenting with both Druid and Elasticsearch for this. We're using Samza to enrich user activity and system performance events then index them in Druid +/or Elasticsearch depending on the use case. 2. These are internal BI/Operations applications 4. We're still getting up

Re: How do you serve the data computed by Samza?

Felix/Shekar, Given that Samza itself uses RocksDB to create a materialized view of a partition from earliest to latest offset, I imagine that to be a good choice to begin evaluating. Two parts: 1. A method recommended in this article/video

Re: How do you serve the data computed by Samza?

Felix/Jordan, 1 - 2 is exactly what I was looking for as well. I want to expose webservices call to Kafka/samza. As there is no concept of a session, I was wondering how to send back enriched data to the web services request. Or am I way off on this? Meaning, is this a completely wrong use case to

RE: How do you serve the data computed by Samza?

015 12:42 PM To: dev@samza.apache.org Subject: Re: How do you serve the data computed by Samza? Felix, Here are my thoughts below 1 - 2) I think so far a majority of samza applications are internal so far. However I've developed a Samza Publisher for PubNub that would allow you to send data from process

Re: How do you serve the data computed by Samza?

Felix, Here are my thoughts below 1 - 2) I think so far a majority of samza applications are internal so far. However I've developed a Samza Publisher for PubNub that would allow you to send data from process or window out over a Data Stream Network. Right now it looks something like this: (.send

How do you serve the data computed by Samza?

Hi Samza devs, users and enthusiasts, I've kept an eye on the Samza project for a while and I think it's super cool! I hope it continues to mature and expand as it seems very promising (: One thing I've been wondering for a while is: how do people serve the data they computed on Samza? More spe