Hi Tom,
Here is an example that runs from the Riak console and dumps all keys of a
bucket to a file and shows how stream_list_keys can be used:
-module(keylister).
-export([list_keys_to_file/2]).
-define(TIMEOUT, 10000000).
%% @spec list_keys_to_file(binary(), string()) ->
%% ok | {error, term()}
list_keys_to_file(Bucket, File) when is_binary(Bucket) andalso is_list(File) ->
{ok, C} = riak:local_client(),
case file:open(File, [write]) of
{ok, IoDev} ->
C:stream_list_keys(Bucket, ?TIMEOUT),
write_keys_to_file(IoDev);
{error, Reason} ->
{error, Reason}
end.
write_keys_to_file(IoDev) ->
receive
{_, {keys,Keys}} ->
Output = [[K, <<"\n">>] || K <- Keys],
file:write(IoDev, Output),
write_keys_to_file(IoDev);
{_, From, {keys,Keys}} ->
riak_kv_keys_fsm:ack_keys(From),
Output = [[K, <<"\n">>] || K <- Keys],
file:write(IoDev, Output),
write_keys_to_file(IoDev);
{_, done} ->
file:close(IoDev);
M ->
{error, unexpected_message, M}
end.
Although there are times when this is useful to use, it is NOT recommended for
production use as it has to traverse ALL keys stored in the cluster, see:
http://docs.basho.com/riak/latest/references/apis/http/HTTP-List-Keys/
Best regards,
Christian
On 16 Apr 2013, at 15:49, tom kelly <[email protected]> wrote:
> Hi List,
> I'm experimenting with riak 1.3.1 and I've started a cluster with two nodes,
> connected from a dev node and wrote & read a few test keys and everything was
> looking good. I was in the process of writing a few functions we'll need in
> our system, one of which needs to cycle through all the keys at startup,
> stream_list_keys looked like it would do the job but I've just noticed this
> strange behavior:
>
> ([email protected])41>
> ([email protected])41> DB:stream_list_keys(<<"test">>).
> {ok,2583625}
> ([email protected])42> DB:stream_list_keys(<<"test">>).
> {ok,98542259}
> ([email protected])43> DB:stream_list_keys(<<"test">>).
> {ok,83513611}
> ([email protected])44> DB:stream_list_keys(<<"test">>).
> {ok,112529022}
> ([email protected])45> DB:stream_list_keys(<<"test">>).
> {ok,66267591}
> ([email protected])46> DB:stream_list_keys(<<"test">>).
> {ok,90620806}
> ([email protected])47> DB:stream_list_keys(<<"test">>).
> {ok,24108838}
> ([email protected])48> DB:stream_list_keys(<<"test">>).
> {ok,17013899}
> ([email protected])49> DB:stream_list_keys(<<"test">>).
> {ok,48399864}
> ([email protected])50> flush().
> Shell got {2583625,{<5272.17627.0>,#Ref<5272.0.0.95523>},{keys,[<<1>>]}}
> Shell got {98542259,{<5272.17656.0>,#Ref<5272.0.0.95733>},{keys,[<<1>>]}}
> Shell got {83513611,{<5272.17670.0>,#Ref<5272.0.0.95901>},{keys,[<<1>>]}}
> Shell got {112529022,{<5272.17683.0>,#Ref<5272.0.0.96064>},{keys,[]}}
> Shell got {66267591,{<5272.17698.0>,#Ref<5272.0.0.96224>},{keys,[]}}
> Shell got {90620806,{<5272.17708.0>,#Ref<5272.0.0.96375>},{keys,[<<1>>]}}
> Shell got {24108838,{<5272.17717.0>,#Ref<5272.0.0.96482>},{keys,[<<1>>]}}
> Shell got {17013899,{<5272.17731.0>,#Ref<5272.0.0.96638>},{keys,[<<1>>]}}
> Shell got {48399864,{<5272.17736.0>,#Ref<5272.0.0.96767>},{keys,[]}}
> ok
>
> The key <<1>> was written over an hour ago and these successive calls to
> stream_list_keys are ~2 seconds apart, The key isn't always in the first keys
> message returned and there's only ever one message returned per call. From
> the comments in the source I expected all the keys to be returned followed by
> the done message.
> Am I misunderstanding or misusing anything here? Any pointers welcomed.
> Thanks,
> //TTom.
>
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
