Sean,
I'll fork and submit a pull request sometime soon. Will things continue
to be configured with binaries, or will it switch to atoms at some point?
Also, Is there a way to configure specific buckets in the config, or to
specify default bucket properties for a particular bucket type in the config?
I see there is default_bucket_props, and you can have bucket specific
properties, but what I would want is some way to have a custom bucket type
which would require the hash function be different. This doesn't seem like
bucket specific config (ie, config passed to start/2), but instead feels like
per bucket type default config. I think I can work around it by specifyng
it at the same time I specify the custom backend. Like
curl -X PUT -H "content-type: application/json" -d
'{"props":{"backend":"my_backend"},"chash_keyfun":{"mod":"my_backend","fun":"my_hash"}}'
http://127.0.0.1:8098/riak/mybucket
but this feels risky, as it would be easy for someone to miss.
I'd prefer being able to set that in default bucket props for a particular
backend. That seems cleaner, and less error-prone.
Anyway, if it doesn't work in this way, any reason it couldn't?
-Anthony
On Mon, Jan 24, 2011 at 08:57:26AM -0500, Sean Cribbs wrote:
> Anthony,
>
> Thanks for noticing that documentation error. If you would, please file an
> issue on the wiki's issue tracker: https://github.com/basho/riak_wiki/issues
> and we'll get it corrected. If you're feeling enterprising, you can also
> fork, fix and send a pull-request for the page.
>
> Sean Cribbs <[email protected]>
> Developer Advocate
> Basho Technologies, Inc.
> http://basho.com/
>
> On Jan 24, 2011, at 2:43 AM, Anthony Molinaro wrote:
>
> > Hi Sean,
> >
> > Thanks, that seems to work. Should the wiki be changed? Currently
> > this page
> >
> > http://wiki.basho.com/Configuration-Files.html
> >
> > has
> >
> > multi_backend: list of backends to provide
> > Format of each backend specification is {BackendName, BackendModule,
> > BackendConfig}, where BackendName is any atom, BackendModule is the name of
> > the Erlang module implementing the backend (the same values you would
> > provide as "storage_backend" settings), and BackendConfig is
> > a parameter that will be passed to the "start/2" function of the backend
> > module.
> >
> > and it sounds like BackendName has to be a binary.
> >
> > Also, it has
> >
> > Specify the backend to use for a bucket with
> > riak_client:set_bucket(BucketName,[{backend, BackendName}] in Erlang
> >
> > but I'm a little unclear how you would invoke this call, do you attach
> > with riak attach, then run the command there? Because when I try that
> > I get
> >
> > ** exception error: undefined function riak_client:set_bucket/2
> >
> > Seems there might be a riak_client:set_bucket/3, so that documentation may
> > be out of date as well.
> >
> > Thanks,
> >
> > -Anthony
> >
> > On Sun, Jan 23, 2011 at 02:50:32PM -0500, Sean Cribbs wrote:
> >> Anthony,
> >>
> >> This is something I discovered a while back - define your backend names as
> >> binaries and you'll be able to set them properly from the REST interface.
> >> Example:
> >>
> >> {storage_backend, riak_kv_multi_backend},
> >> {multi_backend_default, <<"bitcask">>},
> >> {multi_backend,
> >> [ {<<"bitcask">>, riak_kv_bitcask_backend,
> >> [{data_root, "/var/lib/riak/bitcask"}]},
> >> {<<"dets">>, riak_kv_dets_backend,
> >> [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]},
> >> {<<"ets">>, riak_kv_ets_backend, []},
> >> {<<"fs">>, riak_kv_fs_backend,
> >> [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]},
> >> {<<"cache">>, riak_kv_cache_backend,
> >> [ {riak_kv_cache_backend_memory, 100},
> >> {riak_kv_cache_backend_ttl, 600},
> >> {riak_kv_cache_backend_max_ttl, 3600}
> >> ]},
> >> {<<"my_backend">>, my_backend, []}
> >> ]},
> >>
> >> Then you can set it like so
> >>
> >> curl -X PUT -H "content-type: application/json" -d
> >> '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/mybucket
> >>
> >> Also note that only allow_mult and n_val are supported bucket properties
> >> from the PB interface (something we'll be fixing soon).
> >>
> >> Sean Cribbs <[email protected]>
> >> Developer Advocate
> >> Basho Technologies, Inc.
> >> http://basho.com/
> >>
> >> On Jan 23, 2011, at 12:37 PM, Anthony Molinaro wrote:
> >>
> >>>
> >>> Hi,
> >>>
> >>> So I wanted to play around with creating a custom backend, and using
> >>> the multi backend, but I am having problems getting anything to work.
> >>>
> >>> Here's what I tried so far
> >>>
> >>> in app.config
> >>>
> >>> {storage_backend, riak_kv_multi_backend},
> >>> {multi_backend_default, bitcask},
> >>> {multi_backend,
> >>> [ {bitcask, riak_kv_bitcask_backend,
> >>> [{data_root, "/var/lib/riak/bitcask"}]},
> >>> {dets, riak_kv_dets_backend,
> >>> [{riak_kv_dets_backend_root, "/var/lib/riak/dets"}]},
> >>> {ets, riak_kv_ets_backend, []},
> >>> {fs, riak_kv_fs_backend,
> >>> [{riak_kv_fs_backend_root, "/var/lib/riak/fs"}]},
> >>> {cache, riak_kv_cache_backend,
> >>> [ {riak_kv_cache_backend_memory, 100},
> >>> {riak_kv_cache_backend_ttl, 600},
> >>> {riak_kv_cache_backend_max_ttl, 3600}
> >>> ]},
> >>> {my_backend, my_backend, []}
> >>> ]},
> >>>
> >>> Then I restart, open a shell and do the following
> >>>
> >>> 1> {ok, Pid} = riakc_pb_socket:start_link("127.0.0.1", 8087).
> >>> {ok,<0.68.0>}
> >>> 2> riakc_pb_socket:set_bucket(Pid, <<"b">>, [{backend, my_backend}]).
> >>> ok
> >>> 3> riakc_pb_socket:get_bucket(Pid,<<"b">>).
> >>> {ok,[{n_val,3},{allow_mult,false}]}
> >>>
> >>> So I didn't see my backend there, thus tried the REST API
> >>>
> >>>> curl 'http://127.0.0.1:8098/riak/b' ; echo
> >>> {"props":{"name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}}
> >>>
> >>> It's not there either. So I try to set it with REST
> >>>
> >>>> curl -X PUT -H "Content-Type: application/json" -d
> >>>> '{"props":{"backend":"my_backend"}}' http://127.0.0.1:8098/riak/b
> >>>
> >>> Which works, in that now I have
> >>>
> >>>> curl 'http://127.0.0.1:8098/riak/b' ;
> >>>> echo{"props":{"backend":"my_backend","name":"b","n_val":3,"allow_mult":false,"last_write_wins":false,"precommit":[],"postcommit":[],"chash_keyfun":{"mod":"riak_core_util","fun":"chash_std_keyfun"},"linkfun":{"mod":"riak_kv_wm_link_walker","fun":"mapreduce_linkfun"},"old_vclock":86400,"young_vclock":20,"big_vclock":50,"small_vclock":10,"r":"quorum","w":"quorum","dw":"quorum","rw":"quorum"}}
> >>>
> >>> However, in the shell I still get
> >>>
> >>> 4> riakc_pb_socket:get_bucket(Pid,<<"b">>).
> >>> {ok,[{n_val,3},{allow_mult,false}]}
> >>>
> >>> Also, if I attempt to put something I get
> >>>
> >>> 5> riakc_pb_socket:put(Pid, riakc_obj:new (<<"b">>, <<"c">>, <<"d">>)).
> >>>
> >>> =ERROR REPORT==== 19-Jan-2011::02:21:04 ===
> >>> ** Generic server <0.68.0> terminating
> >>> ** Last message in was {req_timeout,#Ref<0.0.0.186>}
> >>> ** When Server state == {state,"127.0.0.1",8087,false,false,undefined,
> >>> undefined,
> >>> {[],[]},
> >>> 1,[],infinity,100}
> >>> ** Reason for termination ==
> >>> ** disconnected
> >>> ** exception exit: disconnected
> >>>
> >>> =CRASH REPORT==== 19-Jan-2011::02:21:04 ===
> >>> crasher:
> >>> initial call: riakc_pb_socket:init/1
> >>> pid: <0.68.0>
> >>> registered_name: []
> >>> exception exit: disconnected
> >>> in function gen_server:terminate/6
> >>> ancestors: [<0.65.0>]
> >>> messages: [{tcp,#Port<0.816>,
> >>> [0|<<10,7,116,105,109,101,111,117,116,16,1>>]}]
> >>> links: [<0.65.0>]
> >>> dictionary: []
> >>> trap_exit: false
> >>> status: running
> >>> heap_size: 987
> >>> stack_size: 24
> >>> reductions: 1541
> >>> neighbours:
> >>> neighbour: [{pid,<0.65.0>},
> >>> {registered_name,[]},
> >>> {initial_call,{erlang,apply,2}},
> >>> {current_function,{gen,do_call,4}},
> >>> {ancestors,[]},
> >>> {messages,[{#Ref<0.0.0.185>,{error,timeout}}]},
> >>> {links,[<0.25.0>,<0.68.0>]},
> >>> {dictionary,[]},
> >>> {trap_exit,false},
> >>> {status,runnable},
> >>> {heap_size,1597},
> >>> {stack_size,38},
> >>> {reductions,17388}]
> >>>
> >>> And the attached sasl log which has many errors. So I'm trying to figure
> >>> out
> >>> whether the problem is configuration, or a bug or what, and wondering if
> >>> any
> >>> one else has gotten a custom backend, or multi backend to work?
> >>>
> >>> I'm using 0.14 on Centos 5, using the basho RPM.
> >>>
> >>> Thanks,
> >>>
> >>> -Anthony
> >>>
> >>> --
> >>> ------------------------------------------------------------------------
> >>> Anthony Molinaro <[email protected]>
> >>> <sasl-error.log>_______________________________________________
> >>> riak-users mailing list
> >>> [email protected]
> >>> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >>
> >
> > --
> > ------------------------------------------------------------------------
> > Anthony Molinaro <[email protected]>
>
--
------------------------------------------------------------------------
Anthony Molinaro <[email protected]>
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
