Re: riak_dt
I just pushed an updated 'riak-dt' branch to the riak repo on github.
Just pull the branch and build and you should have a working riak_dt.
Let me know if you have any issues
Cheers
Russell
On 26 Jan 2013, at 10:54, Petter Egesund petter.eges...@boklink.no wrote:
Great, thanks - I will stay tuned :-)
Petter
Fra: Russell Brown [russell.br...@me.com]
Sendt: 25. januar 2013 19:52
Til: Petter Egesund
Kopi: riak-users List
Emne: Re: riak_dt
Hi Petter,
Hi
I would like to use Riak, but counters is a must in this project. They are
so handy :-)
I tried to build the riak-dt, but the build-system seems broken. I get this
when i do make devrel:
ERROR: generate failed while processing /Users/petter/erlang/riak/rel:
{'EXIT',{{badmatch,{error,luke: : Missing application directory.}},
[{rebar_reltool,generate,2,[]},
{rebar_core,run_modules,4,[]},
{rebar_core,execute,5,[]},
{rebar_core,process_dir0,6,[]},
{rebar_core,process_dir,4,[]},
{rebar_core,process_commands,2,[]},
{rebar,main,1,[]},
{escript,run,2,[{file,escript.erl},{line,741}]}]}}
Any clues?
I think the riak-dt branch of riak has got a bit stale and needs a rebase.
I'll do that today and post back when I have it working.
The riak-dt does also seem a little bit forgotten - no commits in a couple
of months. Is it still planned?
Riak_dt, or something like it _is_ still planned. We're working out how best
to integrate CRDTs in general (and counters in particular) into Riak_kv. In
the meantime I'll fix up that experimental branch.
Cheers
Russell
Cheers,
Petter
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
big cache vs. many partitions and replicas placement
Hi, we want to save plenty of mails in riak which are in average 14k big. Our cluster will become between 10 to 20 nodes so the question is: what is better, having more partitions but each has less cache or is it the other way around so having less partitions with more cache size? Also with an n_val of 3 it is always said, that there is no guarantee for having this on 3 physical nodes but also that it should be distributed evenly. What does it mean? Is there any calculation I can do to get a feeling how often 3 replicas are only on 2 nodes? Or can you say: with 6 nodes and an n_val of 3 it should be the case that almost 99.9 percent is distributed over different physical nodes? Cheers, Simon ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
blocking get call?
Does riak have a blocking get call? I'm thinking of something that works like this... if a GET call is made for an object that doesn't exist, the call blocks until the object is created. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: blocking get call?
I should have added that I want to avoid busy waiting. If the object doesn't exist, some other process will be working to populate it, and I don't want to keep on polling the database to see if the other process is done. Moreover, the process that wants the result doesn't have a direct connection to the process that will be creating the result, so I'm wondering if I can use riak to facilitate them rendezvousing with each other. (Imaging, for instance, that the requesting process is a web generated AJAX request, and the process creating the result is a backend server process.) On Mon, Jan 28, 2013 at 10:50 AM, Alexander Sicular sicul...@gmail.comwrote: No. but try something with a head request in your application. @siculars http://siculars.posterous.com Sent from my rotary phone. On Jan 28, 2013 11:47 AM, E R pc88m...@gmail.com wrote: Does riak have a blocking get call? I'm thinking of something that works like this... if a GET call is made for an object that doesn't exist, the call blocks until the object is created. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: blocking get call?
That would defy the purpose of distributed and high availability; the short answer is no, you will have to implement at the application level a blocking policy for the keys, assuming you know your key before hand /(think if it of your primary key if it was an SQL DB)/, in Riak last write wins, unless you are have siblings enabled which you will have to resolve conflicts later, take a look at the following presentations, it might give you ideas; they are not necessary related to your subject, but will tell you why is not possible: http://vimeo.com/43903960 http://www.infoq.com/presentations/CRDT Hope that helps, Guido. On 28/01/13 16:46, E R wrote: Does riak have a blocking get call? I'm thinking of something that works like this... if a GET call is made for an object that doesn't exist, the call blocks until the object is created. ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: big cache vs. many partitions and replicas placement
Hi Simon, By cache size, I am assuming you are referring to the leveldb internal cache. Is this correct? The arguably most important parameter in your configuration is the ring size. Defining an appropriate ring size is very important as it can't change later on and determines how far you can scale. It is recommended that you allow for at about 10 partitions per physical node (http://docs.basho.com/riak/1.3.0/references/appendices/Cluster-Capacity-Planning/#Ring-Size-Number-of-Partitions), which means that if you are preparing for a cluster of around 20 nodes, you should probably use 256 partitions. At the moment we do not recommend tuning the eleveldb cache_size at all, so it would be best to leave it at the default value. With an n_val of 3, Riak will store every record on 3 consecutive vnodes in the ring. The case where multiple copies may end up on less than 3 different physical servers primarily applies to cases where there are less than 4 nodes in the cluster as nodes will repeat around the ring wrap around point.This is one of the reasons we recommend using at least 5 nodes in the cluster. Best regards, Christian On 28 Jan 2013, at 15:47, Effenberg, Simon seffenb...@team.mobile.de wrote: Hi, we want to save plenty of mails in riak which are in average 14k big. Our cluster will become between 10 to 20 nodes so the question is: what is better, having more partitions but each has less cache or is it the other way around so having less partitions with more cache size? Also with an n_val of 3 it is always said, that there is no guarantee for having this on 3 physical nodes but also that it should be distributed evenly. What does it mean? Is there any calculation I can do to get a feeling how often 3 replicas are only on 2 nodes? Or can you say: with 6 nodes and an n_val of 3 it should be the case that almost 99.9 percent is distributed over different physical nodes? Cheers, Simon ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Restore Fails!
Hi Jorge,
My first guess was that there was an issue with backing up data from
1.0.x to 1.2.x but we've tested that and it works as expected.
Let's see if we can get to the bottom of this ...
Have you done any other backups that were successful? Any chance you
could test backing up new data from your 1.0 cluster to 1.2 to see if
you can reproduce the error?
Mark
twitter.com/pharkmillups
On Sun, Jan 27, 2013 at 7:47 AM, Jorge Garrido
jorge.garr...@morelosoft.com wrote:
Hi,
I create a file backup with command backup in riak version 1.0.2
When I try restore command with this file in the riak version 1.2.1, show me
this error:
$ sh dev1/bin/riak-admin restore dev1@127.0.0.1 riak /root/rbackup-102.backup
Restoring from '/root/rbackup-102.backup' to cluster to which
'dev1@127.0.0.1' belongs.
{init terminating in
do_boot,{function_clause,[{riak_kv_backup,traverse_backup,[{{continuation,0.41.0,934256632,[]},[],658},#Funriak_kv_backup.2.8487,332440],[{file,src/riak_kv_backup.erl},{line,132}]},{riak_kv_backup,restore,2,[{file,src/riak_kv_backup.erl},{line,125}]},{erl_eval,do_apply,6,[{file,erl_eval.erl},{line,572}]},{init,start_it,1,[]},{init,start_em,1,[]}]}}
Any ideas?
Thanks
Jorge Garrido
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: testing Riak (incomplete doc)
Thank you. It works after this: ./kerl install r15b01 ~/erlang/r15b01 . ~/erlang/r15b01/activate When I run the tests in the master branch, I got: Failed: 24. Skipped: 0. Passed: 221. I wanted to contribute, but how do I know that I have not broken something if the tests normally fail ? Cheers, Andrey On Sun, Jan 27, 2013 at 10:14 PM, Jared Morrow ja...@basho.com wrote: You don't appear to have erlang in your path that's why the escript is failing. -Jared On Jan 27, 2013, at 1:16 PM, Andrey Somov public.so...@gmail.com wrote: Hi all, I have just tried to run the tests for riak_kv as described here: https://github.com/basho/riak Both 'make eunit' and 'rebar' fail: ~/projects/riak_kv$ make eunit make: *** No rule to make target `eunit'. Stop. ~/projects/riak_kv$ ./rebar skip_deps=true eunit /usr/bin/env: escript: No such file or directory What should I do to run the tests ? Should the documentation be updated to add the dependencies? Cheers, Andrey ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Is JSON number stored in binary format?
I apologize if question is answered on archives, but it is not easy to search for this. Suppose I store following JSON array, PUT /riak/mybucket/mykey [1359401530, 1359401531, 1359401532, 1359401533, 1359401534, 1359401535, 1359401536] Is number 1359401530 stored as array of bytes ['1', '3'.] in 10 bytes, or as binary value 0101000101101101001000111010, in 4 bytes? Many thanks, Boris ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Riak startup failing.
Kevin: First things first. What version of Riak are you running? The `riak-admin cluster` commands are available as of Riak 1.2.0. Next, if you have already renamed all of the nodes, you will need to use the `reip` method to modify all of the ring files on each of the nodes. If you are able, it's easier to roll back the changes to the configuration and get the nodes starting again. Once they are started and running, here's a writeup of the scheme you would use to do the rolling configuration changes: https://gist.github.com/4566736 Thanks, Charlie Voiselle On Dec 29, 2012, at 1:02 PM, Kevin Burton rkevinbur...@charter.net wrote: Here is what I found out (I am still stuck) https://github.com/basho/basho_docs/issues/28 The step says to rename or remove the data/ring directory. My data directory is /var/lib/riak. If I rename or remove that directory then no commands work. I have assumed it meant to rename or remove just the ring directory which is for me /var/lib/riak/ring. When the node is restarted the ring directory is recreated. When I issue riak-admin down (I have to specify the node name since it is required) I get errors/warnings: ~$ sudo riak-admin down r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Node is not running! I am not sure what it means to join the existing cluster on the first node. Is this riak-admin cluster join? For the first node I get an error ~$ sudo riak-admin cluster join r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Join failed. Try again in a few moments. There is no ‘force-replace’ argument to riak-admin and that command seems to be deprecated. There is a force-replace for riak-admin cluster but it takes two nodes as arguments which I am not sure what to put there. What should I use? These steps seem to get me further but still there are gaps (at least for me). ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
RE: Riak startup failing.
I am not sure which version you are asking for. Here is a snippet from the riak-admin status command: riak_control_version : 1.2.0 cluster_info_version : 1.2.2 riak_api_version : 1.2.0 riak_search_version : 1.2.1 merge_index_version : 1.2.1 riak_kv_version : 1.2.1 riak_pipe_version : 1.2.1 riak_core_version : 1.2.1 lager_version : 1.2.0 From: Charlie Voiselle [mailto:cvoise...@basho.com] Sent: Monday, January 28, 2013 2:06 PM To: Kevin Burton Cc: riak-users@lists.basho.com Subject: Re: Riak startup failing. Kevin: First things first. What version of Riak are you running? The `riak-admin cluster` commands are available as of Riak 1.2.0. Next, if you have already renamed all of the nodes, you will need to use the `reip` method to modify all of the ring files on each of the nodes. If you are able, it's easier to roll back the changes to the configuration and get the nodes starting again. Once they are started and running, here's a writeup of the scheme you would use to do the rolling configuration changes: https://gist.github.com/4566736 Thanks, Charlie Voiselle On Dec 29, 2012, at 1:02 PM, Kevin Burton rkevinbur...@charter.net wrote: Here is what I found out (I am still stuck) https://github.com/basho/basho_docs/issues/28 https://github.com/basho/basho_docs/issues/28 The step says to rename or remove the data/ring directory. My data directory is /var/lib/riak. If I rename or remove that directory then no commands work. I have assumed it meant to rename or remove just the ring directory which is for me /var/lib/riak/ring. When the node is restarted the ring directory is recreated. When I issue riak-admin down (I have to specify the node name since it is required) I get errors/warnings: ~$ sudo riak-admin down mailto:r...@xxx.xxx.xxx.xxx r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Node is not running! I am not sure what it means to join the existing cluster on the first node. Is this riak-admin cluster join? For the first node I get an error ~$ sudo riak-admin cluster join mailto:r...@xxx.xxx.xxx.xxx r...@xxx.xxx.xxx.xxx Attempting to restart script through sudo -H -u riak Join failed. Try again in a few moments. There is no 'force-replace' argument to riak-admin and that command seems to be deprecated. There is a force-replace for riak-admin cluster but it takes two nodes as arguments which I am not sure what to put there. What should I use? These steps seem to get me further but still there are gaps (at least for me). ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Is JSON number stored in binary format?
Riak is oblivious of your data's format, so it will store the literal JSON. If you need more compact formats, you should marshal it yourself (and probably send it as a different media type) before storing. On Mon, Jan 28, 2013 at 1:35 PM, Boris Solovyov boris.solov...@gmail.com wrote: I apologize if question is answered on archives, but it is not easy to search for this. Suppose I store following JSON array, PUT /riak/mybucket/mykey [1359401530, 1359401531, 1359401532, 1359401533, 1359401534, 1359401535, 1359401536] Is number 1359401530 stored as array of bytes ['1', '3'.] in 10 bytes, or as binary value 0101000101101101001000111010, in 4 bytes? Many thanks, Boris ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com -- Sean Cribbs s...@basho.com Software Engineer Basho Technologies, Inc. http://basho.com/ ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Re: Is JSON number stored in binary format?
I developed a custom IntRangeCollection class in Java for that, we store list of IDs (That come from a DB sequence), like Sean said; you will have to do it yourself, I spent a couple of days looking for something like that and ended up implementing it myself: http://stackoverflow.com/questions/5744572/is-there-a-java-library-that-will-create-a-number-range-from-a-list-of-numbers/12371432#12371432 Hope that helps, Guido. On 28/01/13 20:43, Sean Cribbs wrote: Riak is oblivious of your data's format, so it will store the literal JSON. If you need more compact formats, you should marshal it yourself (and probably send it as a different media type) before storing. On Mon, Jan 28, 2013 at 1:35 PM, Boris Solovyov boris.solov...@gmail.com wrote: I apologize if question is answered on archives, but it is not easy to search for this. Suppose I store following JSON array, PUT /riak/mybucket/mykey [1359401530, 1359401531, 1359401532, 1359401533, 1359401534, 1359401535, 1359401536] Is number 1359401530 stored as array of bytes ['1', '3'.] in 10 bytes, or as binary value 0101000101101101001000111010, in 4 bytes? Many thanks, Boris ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
Riak Recap for January 15 - 28
Evening, Morning, Afternoon to All - Sorry for the delay in Recaps. It has been a hectic few weeks. We've got all sorts of goodness in here. Enjoy, and thanks for being a part of Riak. Also, just a heads up, but clear your calendar for the 2nd week of May, especially if you're on the East Coast of the US (or plan to be)... Lastly, I'll be San Diego tomorrow night (for this - http://www.eventbrite.com/event/5174472988/) and LA the day after for a few hours. If anyone wants to chat Riak, let me know and I'll try to make some time. Mark twitter.com/pharkmillups Riak Recap for January 15 - 28 === 1) Basho is sponsoring the ErlangDC event that's happening February 9th. You should go. * Details and tickets here --- http://erlangdc.com/ 2) myfinder released some new Perl code for testing Riak. * Code here --- https://github.com/myfinder/p5-test-riak 3) Matt Snyder wrote a blog post about 2i vs. Key Filters and is looking for feedback on which query method is faster for his use case. * Read here --- http://www.cranialpulse.com/2013/01/riak-whats-faster-2i-or-key-filters.html 4) The team at Klarna just released some Riak-flavored code. * Klarna Riak Client (which wraps the Riak Erlang Client) --- https://github.com/klarna/krc * Riak implementations of MeshUp interfaces ---https://github.com/klarna/rimu 5) Christopher Biscardi wrote his second blog post about Riak. This one is all about Consistent Hashing. * Read here --- http://www.christopherbiscardi.com/2013/01/16/consistant-hash-routing-in-riak-core/ 6) The 451Group is doing a survey on database usage. If you're using (or testing) Riak, don't be scared to help us out. We appreciate it. * Survey here --- bit.ly/451db13 7) Myles Megyesi from 8th Light wrote a quick post about some JavaScript issues they ran into when using large numbers with MapReduce. * Read here --- http://blog.8thlight.com/myles-megyesi/2013/01/18/riak-mapreduce.html 8) Theo Bot's Catalyst Riak Model just hit 0.4. * Code here on CPAN--- http://search.cpan.org/~nltbo/Catalyst-Model-Riak-0.04/ 9) We added a few Riak use cases to the Basho Blog that are well worth the read if you're interested in Riak being used in production. * Mad Mimi --- http://basho.com/blog/technical/2013/01/21/Riak-Mad-Mimi/ * Alert Logic --- http://basho.com/blog/technical/2013/01/24/Alert-Logic-Riak/ * Copious --- http://basho.com/blog/technical/2013/01/28/Copious-Uses-Riak/ 10) A few Riak meetups on the horizon. * Riak Portland (OR) is happening tonight --- http://www.meetup.com/Portland-Riak/events/95049492/ * The first Riak Munich meetup --- http://www.meetup.com/Riak-Munchen/events/100799492/ * Riak London is tomorrow night --- http://www.meetup.com/riak-london/events/94949812/ * The next Riak NYC was just announced --- http://www.meetup.com/NYC-Riak-Meetup/events/101793192/ 11) The crew at OmnitTI wrote a post about how they support Viggle. Riak gets a nice mention. * Read here --- http://omniti.com/remembers/2013/omniti-provides-high-performance-support-for-viggle-television-loyalty-program 12) Basho is sponsoring flatMap(Oslo) this coming May in Norway. * Details here --- http://2013.flatmap.no/about.html 13) We added a bunch of functions for manipulating user metadata and secondary indexes to the Riak Erlang Client (courtesy of newly-minted Basho Hacker Christian Dahlqvist). * Code here --- https://github.com/basho/riak-erlang-client/pull/81 14) Basho Hacker Marcel Neuhausler added support to Yokozuna for storing and retrieving schemas via http * Commit here --- https://github.com/rzezeski/yokozuna/commit/7cb90fd2dba7cfc3241384a09bc6c097472924f3 15) Nathan Aschbacher (also now a Basho dev) bumped Nodiak up to version 0.1.1. * Code here --- https://github.com/nathanaschbacher/nodiak 16) A WIP branch for Protocol Buffers support in riak-js just landed. * Take it for a spin here --- https://github.com/mostlyserious/riak-js/pull/138 17) Riak is now available on the Engine Yard cloud. And they're giving away 500 free hours on their platform to try it out. (Congrats to Ines, Tasha, and the EY crew for shipping this.) * Details here --- http://basho.com/blog/technical/2013/01/25/Riak-at-Engine-Yard/ 18) Chlu wrote a Riak cache backend for TYPO3 Flow * Code here --- https://github.com/chlu/TYPO3.Riak 19) Some Q A. * --- http://stackoverflow.com/questions/14531708/how-can-i-delete-a-riak-bucket-in-python * --- http://stackoverflow.com/questions/14555602/riak-not-starting * --- http://stackoverflow.com/questions/14464847/writing-date-objects-to-riak-with-python-and-flask-with-overly-complex-jsonific ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
byte array in entity
Hello guys, hope you are well. If Riak stores data in JSON format, is it possible to use byte[] as property in your Entity class? Best, Mikhail ___ riak-users mailing list riak-users@lists.basho.com http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
