ElasticSearch has this feature as well. When using the Java transport client (as opposed to the HTTP/JSON interface), you actually become a cluster member as a non-data node. You can also add any number of nodes to the cluster as non-data nodes that then participate in general processing.
http://www.elasticsearch.org/guide/reference/modules/node.html http://www.elasticsearch.org/guide/reference/java-api/client.html On Thu, Apr 5, 2012 at 3:52 PM, Alexander Sicular <[email protected]> wrote: > I think that being able to have a native erlang interface to a riak cluster > would be a "Good Thing." I can imagine some compute applications that could > benefit from having access to data at the speed of network but not actually > be a member of the storage pool. > > Just watched Rich Hickey discussing similar concepts employed in his new > system, Datomic, here, http://www.infoq.com/interviews/hickey-datomic . > Worth the 30min. > > -Alexander Sicular > > @siculars > > On Apr 5, 2012, at 3:30 PM, Ryan Zezeski wrote: > > Mike, > > The native Erlang client was never designed to be used in this fashion, as > you are finding out first hand. You are correct in that some operations > assume they are running on the Riak VM, e.g. mapred_dynamic_inputs_stream. > > Normally I would say not to establish a Erlang dist connection to the Riak > cluster but given you need a work around for the performance issue it seems > to be your only good choice ATM. You can use `rpc:call` to invoke the > mapred queries. > > -Z > > On Wed, Apr 4, 2012 at 2:40 PM, Michael Radford <[email protected]> wrote: >> >> In case anyone else is wondering, the answer is no, the native client >> isn't fully usable from a non-Riak node. >> >> I started a hidden node and connected to one of my Riak nodes using >> riak:client_connect. The returned client is able to successfully >> execute Client:get(Bucket, Key) (and probably other simple things like >> put/delete), but it fails on mapred with a {modfun, ...} input. >> >> >> The problem is that the native client tries to run the input function >> on the local node, rather than the node the client points to, which of >> course won't work unless the local node is participating in the Riak >> cluster. >> >> Mike >> >> On Tue, Apr 3, 2012 at 9:43 AM, Michael Radford <[email protected]> wrote: >> > Is it OK to use Riak's native erlang client from an erlang node that's >> > connected to a Riak cluster via normal erlang distribution >> > (net_adm:ping etc.), but isn't participating in the Riak cluster? I'm >> > wondering if e.g. Riak makes any assumptions about nodes listed in >> > nodes(), or the nodes of processes running client code, that would >> > break if a node isn't running Riak. >> > >> > I'm trying to figure out a fallback strategy in case I can't resolve >> > performance issues I'm seeing with the protocol buffers client (see >> > other thread about slow searches). The scariest option would be to >> > run my application on the same nodes as Riak, which seems like it >> > would complicate operations too much. Or if >> > connected-but-not-participating nodes aren't a good idea, maybe my >> > application nodes could each start a slave node with -hidden, which >> > would then connect to Riak and act as a proxy? >> > >> > Mike >> >> _______________________________________________ >> riak-users mailing list >> [email protected] >> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > > > > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com > _______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
