Re: [PROPOSAL] Connectors for memcache and Couchbase

than < > sraghunat...@etouch.net> wrote: > > > Yes, that is correct ! > > > > Regards, > > Seshadri > > > > -Original Message- > > From: Eugene Kirpichov [mailto:kirpic...@google.com.INVALID] > > Sent: Thursday, July 20, 2017 5:05 PM >

Re: [PROPOSAL] Connectors for memcache and Couchbase

r a memcache IO in Apache Beam. > > > > Consistency / Snapshot - > > - > > This makes perfect sense for a full read / unbounded source, not sure > > if it suits a key-based read. Key-based read will simply return the > > value from memca

Re: [PROPOSAL] Connectors for memcache and Couchbase

return the value from > memcache store transparently at a given point of time. > > Please let me know your comments, I plan to start developing this once we > have a consensus. > > Regards, > Seshadri > > -Original Message- > From: Ismaël Mejía [mailto:ieme...@gm

RE: [PROPOSAL] Connectors for memcache and Couchbase

rt developing this once we have a consensus. Regards, Seshadri -Original Message- From: Ismaël Mejía [mailto:ieme...@gmail.com] Sent: Tuesday, July 11, 2017 9:54 AM To: dev@beam.apache.org Subject: Re: [PROPOSAL] Connectors for memcache and Couchbase Hello again, Thanks Lukasz for the de

Re: [PROPOSAL] Connectors for memcache and Couchbase

Hello again, Thanks Lukasz for the details. We will take a look and discuss with the others on how to achieve this. We hadn’t considered the case of a full scan Read (as Eugene mentions) so now your comments about the snapshot make more sense, however I am still wondering if the snapshot is worth

Re: [PROPOSAL] Connectors for memcache and Couchbase

I think Madhusudan's proposal does not involve reading the whole contents of the memcached cluster - it's applied to a PCollection of keys. So I'd suggest to call it MemcachedIO.lookup() rather than MemcachedIO.read(). And it will not involve the questions of splitting - however, it *will*

Re: [PROPOSAL] Connectors for memcache and Couchbase

Splitting on slabs should allow you to split more finely grained then per server since each server itself maintains this information. If you take a look at the memcached protocol, you can see that lru_crawler supports a metadump command which will enumerate all the key for a set of given slabs or

Re: [PROPOSAL] Connectors for memcache and Couchbase

For the source: Do you plan to support enumerating all the keys via cachedump / lru_crawler metadump / ...? If there is an option which doesn't require enumerating the keys, how will splitting be done (no splitting / splitting on slab ids / ...)? Can the cache be read while its still being

[PROPOSAL] Connectors for memcache and Couchbase

Hi, We are proposing to build connectors for memcache first and then use it for Couchbase. The connector for memcache will be build as a IOTransform and then it can be used for other memcache implementations including Couchbase. 1. As Source input will be a key(String / byte[]), output will