Ya, that is exactly what I had in mind. A non data storage cluster member. Specifically to act as additional read/computation asset.
-Alexander Sicular @siculars On Apr 5, 2012, at 4:12 PM, John E. Vincent wrote: > ElasticSearch has this feature as well. > > When using the Java transport client (as opposed to the HTTP/JSON > interface), you actually become a cluster member as a non-data node. > You can also add any number of nodes to the cluster as non-data nodes > that then participate in general processing. > > http://www.elasticsearch.org/guide/reference/modules/node.html > http://www.elasticsearch.org/guide/reference/java-api/client.html > > On Thu, Apr 5, 2012 at 3:52 PM, Alexander Sicular <[email protected]> wrote: >> I think that being able to have a native erlang interface to a riak cluster >> would be a "Good Thing." I can imagine some compute applications that could >> benefit from having access to data at the speed of network but not actually >> be a member of the storage pool. >> >> Just watched Rich Hickey discussing similar concepts employed in his new >> system, Datomic, here, http://www.infoq.com/interviews/hickey-datomic . >> Worth the 30min. >> >> -Alexander Sicular >> >> @siculars >> >> On Apr 5, 2012, at 3:30 PM, Ryan Zezeski wrote: >> >> Mike, >> >> The native Erlang client was never designed to be used in this fashion, as >> you are finding out first hand. You are correct in that some operations >> assume they are running on the Riak VM, e.g. mapred_dynamic_inputs_stream. >> >> Normally I would say not to establish a Erlang dist connection to the Riak >> cluster but given you need a work around for the performance issue it seems >> to be your only good choice ATM. You can use `rpc:call` to invoke the >> mapred queries. >> >> -Z >> >> On Wed, Apr 4, 2012 at 2:40 PM, Michael Radford <[email protected]> wrote: >>> >>> In case anyone else is wondering, the answer is no, the native client >>> isn't fully usable from a non-Riak node. >>> >>> I started a hidden node and connected to one of my Riak nodes using >>> riak:client_connect. The returned client is able to successfully >>> execute Client:get(Bucket, Key) (and probably other simple things like >>> put/delete), but it fails on mapred with a {modfun, ...} input. >>> >>> >>> The problem is that the native client tries to run the input function >>> on the local node, rather than the node the client points to, which of >>> course won't work unless the local node is participating in the Riak >>> cluster. >>> >>> Mike >>> >>> On Tue, Apr 3, 2012 at 9:43 AM, Michael Radford <[email protected]> wrote: >>>> Is it OK to use Riak's native erlang client from an erlang node that's >>>> connected to a Riak cluster via normal erlang distribution >>>> (net_adm:ping etc.), but isn't participating in the Riak cluster? I'm >>>> wondering if e.g. Riak makes any assumptions about nodes listed in >>>> nodes(), or the nodes of processes running client code, that would >>>> break if a node isn't running Riak. >>>> >>>> I'm trying to figure out a fallback strategy in case I can't resolve >>>> performance issues I'm seeing with the protocol buffers client (see >>>> other thread about slow searches). The scariest option would be to >>>> run my application on the same nodes as Riak, which seems like it >>>> would complicate operations too much. Or if >>>> connected-but-not-participating nodes aren't a good idea, maybe my >>>> application nodes could each start a slave node with -hidden, which >>>> would then connect to Riak and act as a proxy? >>>> >>>> Mike >>> >>> _______________________________________________ >>> riak-users mailing list >>> [email protected] >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> >> >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com >> _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
