Re: Docker support

I would discourage running Riak in Docker. If you use Docker in bridge mode, then it becomes fairly difficult to deal with networking across machines. If you run it in host mode, you run into issues with epmd in the host network namespace. There are some workarounds to this, like using third party

Re: How to cold (re)boot a cluster with already existing node data

Two suggestions: 1. Use Riak-EE, and have two rings. When you do an update, copy over one ring to the other side after you do a "cold reboot" 2. Use the Riak Mesos Framework. Mesos is like K8s, but it has stateful storage primitives. (Link: https://github.com/basho-labs/riak-mesos) On Mon, Jun 6,

Re: Riak Cluster Behavior - Clarification

What do you mean it's not returning? It's returning stale data? Or it's erroring? On Tue, May 24, 2016 at 7:34 AM, Vikram Lalit wrote: > Hi - I'd appreciate if someone can opine on the below behavior of Riak that > I am observing... is that expected, or something wrong in

Re: Questions about installing Stanchion for Riak CS

I never got around to it, but it should be pretty easy to glue Consul to Stanchion to HaProxy. As far as gluing HAProxy to Consule -- that should be pretty easy: https://github.com/hashicorp/consul-template. And Stanchion with Consul lock: https://www.consul.io/docs/commands/lock.html. It

Re: Riak install on instance with multiple OTP's

How did you install OTP18? When dealing with Erlang, and multiple installs of it, I might suggest using kerl (https://github.com/kerl/kerl). It's an excellent tool for dealing with the problem. On Sat, May 21, 2016 at 12:01 PM, Robert Latko wrote: > Hi all, > > Quick

Re: Riak DT refresh

Is the plan to keep using riak_dt_vclock? If so, I might contribute some optimizations for large numbers of actor entries (1000s). On Thu, Apr 28, 2016 at 12:55 AM, Russell Brown wrote: > Hi, > Riak DT[1] is in need of some love. I know that some of you on this list >

Re: Riak DT refresh

We're using riak_dt in anger in our product. We are already using it with rebar3, and Erlang 18.3 through some super messy patches. I would love to see a register that takes the logical clock, and timestamp for resolution, rather than just a straightup timestamp. My biggest ask though is

Re: Riak_ensemble implementation

; protocols? > > Actually, my target is so simple, I just need to be able to put some > key-value by > executing the multi-paxos. > > On Tue, Mar 8, 2016 at 11:39 PM, Sargun Dhillon <sar...@sargun.me> wrote: >> >> If you want to learn to use riak_ensemble the l

Re: Riak_ensemble implementation

If you want to learn to use riak_ensemble the library, the documentation that Joe put together (and others) https://github.com/basho/riak_ensemble/blob/develop/doc/Readme.md There's real world usage of the code here: -https://github.com/basho/riak_kv/blob/develop/src/riak_kv_ensembles.erl

Re: Riak_ensemble implementation

Can you tell us a little bit about the application? It might be easier to use Riak kv rather than riak ensemble directly. Sent from my iPhone On Mar 8, 2016, at 06:54, Agung Laksono wrote: Hi Basho developer, I've seen the video that you guys present about

Re: ring_creation_size & ring_partition_number error joining 2.1.3 node to 2.1.1 cluster

They're effectively equal Sent from my iPhone On Mar 6, 2016, at 17:32, Robert Latko wrote: Hi all, Found the problem/solution. With the first cluster, I set it up with a ring size of 64. Prior to loading data. I stopped the node(s), changed the ring size to 256, then

Re: Continuous HTTP POSTs to Riak

So, if you pass back the vclock while pr=pw=quorum, and sloppy_quorum = false you should get RYOW consistency. On Thu, Mar 3, 2016 at 6:25 PM, Christopher Mancini wrote: > If you don't need strong consistency for all Riak requests, just certain > ones, then explore the use of

Re: Reliability quastion

Given the tunables that Riak has, I would say that its reliability in terms of consistency (as in ACID) are unparalleled. It offers a variety of weak consistency options from eventual consistency, to strong eventual consistency, to strong consistency*. In terms of fault-tolerance, it is

Re: error on object retrieval: Cannot send another request while one is in progress

if there is a failure, and leader is re-elected, the epoch of the ensemble changes, so you may have to refetch the object to get a new causal context in order to perform another write. If this occurs, you should be able to heal by doing another read, modify, write cycle. On Wed, Sep 23, 2015 at

Re: Running out of disk space

Are you using bitcask, or LevelDB? What version of Riak are you using? Bitcask will lazily merge files in the background in order to reclaim space. This is pretty aggressive, and it should show up pretty quickly. On the other hand, LevelDB deletes files at compaction time. Compactions aren't quite

Re: distributed basho bench

So, the way it should work is pretty simple: Run the command a la: ./basho_bench -N nodeA@1 0.0.1.123 -C basho_bench_cookie (It's key that the IP address be the external IP, and not the internal IP of the box, a la loopback) In addition, nodeB must have the same version of

Re: Problem with Riak not thinking hostname is fully-qualified

Put a period at the end. On Wed, Aug 12, 2015 at 8:52 PM, Toby Corkindale t...@dryft.net wrote: I'm trying to set up another cluster, and I'm hitting problems with Riak complaining that ** System running to use fully qualified hostnames ** ** Hostname db04 is illegal ** However, as far as I

Re: How to run bash bench distributed

When you start basho_bench, you must start Erlang in distributed mode, which means you must set a node name, and a cookie. So, it would look something like the following: ./basho_bench -N foo@127.0.0.1 -C basho_bench examples/riakc_pb_distributed.config Ensure that in

Re: Riak 1.4 vs riak 1.2

Are you using LevelDB? If so, it'd be impossible, and even unfair for me to summarize all the work that's been done by MVM on LevelDB as part of 1.4, and 2.0: https://github.com/basho/leveldb/wiki On Tue, Feb 10, 2015 at 6:15 PM, siva Ram sivaram.pasupul...@gmail.com wrote: In terms of

Re: Strong consistent buckets ready after starting a cluster.

I really don't recommend doing this. Really, the cluster shouldn't be changing often enough where the ensemble status is changing all that often. You can do this by exploiting internal APIs that are accessible over distributed Erlang in riak_ensemble. Though, connecting to the cluster over

Re: Adding nodes to cluster

is completed? Thanks a lot for the other tips, you've been very helpful! Best regards, Edgar On 24 January 2015 at 21:09, Sargun Dhillon sar...@sargun.me wrote: Several things: 1) If you have data at rest that doesn't change, make sure you have AAE, and it's ran before your cluster is manipulated

Re: Adding nodes to cluster

Several things: 1) If you have data at rest that doesn't change, make sure you have AAE, and it's ran before your cluster is manipulated. Given that you're running at 85% space, I would be a little worried to turn it on, because you might run out of disk space. You can also pretty reasonably put

Re: erlang compile

So, you need to set add_paths as a runtime option, once you compile the Erlang program into beam files (perhaps with a .app) so that Riak can load your code. Have you already compiled your program with the Erlang compiler that comes with your version of Riak? Which version of Riak are you using?

Re: Wiping Riak 2.0 data

Assuming your storage engine is bitcask: 1. Turn off all your Riak nodes 2. rm -rf --verbose /var/lib/riak/bitcask/*/* 3. rm -rf --verbose /var/lib/riak/anti_entropy/*/* 4. Turn Riak back on This will preserve your buckets and bucket types in in cluster metadata. You can also automate that, but

Re: Random timeouts on Riak

Several things: 1) I recommend you have a 5-node cluster: http://basho.com/why-your-riak-cluster-should-have-at-least-five-nodes/ 2) What version of Riak are you using? 3) What backend(s) are you using? 4) What's the size of your keyspace? 5) Are you actively rewriting keys, or writing keys to the

Re: Random timeouts on Riak

is the CPU utilisation in the cluster when you're experience these timeouts? Can you spot anything peculiar in your server's $ dmesg outputs? Any I/O errors there? Regards, Ciprian On Mon, Dec 29, 2014 at 1:55 PM, Sargun Dhillon sar...@sargun.me wrote: Several things: 1) I recommend you

Re: problems with starting nodes

Can you post the files in the log directory on a github gist, and run the command ./dev1/bin/riak console? In addition, run Riak as a non-root user, with the max files limit bumped up as high as you can set it. On Wed, Dec 24, 2014 at 10:47 PM, Ildar Alishev ildaralis...@gmail.com wrote: Hello

Re: Weird RIAK behavior

= false). Regards. On Mon, Dec 22, 2014 at 9:17 PM, Sargun Dhillon sar...@sargun.me wrote: What versions of Riak are you using? And are these CRDT sets? Sent from my iPhone On Dec 22, 2014, at 16:04, Claudio Cesar Sanchez Tejeda demoncc...@gmail.com wrote: I'm a sysadmin and I managing

Re: Weird RIAK behavior

What versions of Riak are you using? And are these CRDT sets? Sent from my iPhone On Dec 22, 2014, at 16:04, Claudio Cesar Sanchez Tejeda demoncc...@gmail.com wrote: I'm a sysadmin and I managing 5 cluster of RIAK: - two of them are LXC containers on the same physical machine (3 nodes

Re: Riak errors after node left the cluster

So, from my understanding, one of your servers was being replaced, so you did a leave from the cluster, and it failed to commit, and then another node failed, resulting in 3/4ish or 3/5ish of the ring being up? Did you down the failed node, or remove it from the cluster? What's the current status

Re: Crypto with R16B02-basho5

How did you install R17? kerl? If so, it silently doesn't install crypto if you don't have libssl-dev installed, see here: https://github.com/evax/kerl/issues/13 On Wed, Nov 5, 2014 at 12:28 AM, Abd El-Fattah Mahran abdoo.mah...@gmail.com wrote: Hi, I downloaded .deb package with R17 and it

Re: Creating a new object immediately after deleting one with the same key

Distributed deletion, and garbage collection is a really hard problem. Riak has a couple different ways to do it talked about here: http://docs.basho.com/riak/latest/ops/advanced/deletion/ The default mechanism tombstones the key, and then waits 3 seconds after the write has reached stable state

Re: Connecting to a single host vs balancing requests

When you do read / modify / writes, are you also planning on sending the relevant read through one node only? In that case, your update latency might suffer if the egress queues of your designated node get backed up on writes, waiting for a very low cost read query. You're more likely to get

Re: Riak for low latency/high write volumes

So, I've been doing some testing lately. Riak sounds like it'll meet your usecases with some caveats. LevelDB on SSDs will handle change often fairly well, as long as your disks have enough throughput, and CPUs are large enough to handle the compaction. If you're in GCE, I recommend using

Re: Riak for low latency/high write volumes

So, from my understanding, there are two types of data that you have. Family 1: -Constantly changing -Representable by CRDTs -Can handle eventual consistency Family 2: -Rarely changing -Need immediate consistency, and linearizability With both key families, you care about tail latency.

Re: replacing node results in error with diag

So, I don't have a ton of experience with Riaknostic, but taking a casual glance at the source code, it appears that Riaknostic caches some node-local data about the ring (see: https://github.com/basho/riaknostic/blob/2.0.0/src/riaknostic_node.erl#L192-L208). You should be able to unset this by

Re: unable to delete certain keys in a bucket

So, if I'm interpreting your message correctly, you're executing the following steps, with the following steps, in order? 1. Key delete, X - success 2. Key list - yields a set, which contains X 3. Key fetch, X - 404 / missing I'd first ask why you're doing key listing. That's an anti-pattern.

Re: If I go to RICON 2014

Although, I'm not a Basho engineer, I work for a partner. I've deployed Riak, and Riak-CS in production services. I promise you don't have to stay in your corner if you want to talk about deploying, and using Riak. You'll find me at the conference. There tends to be some split of industry types,

Re: Logical Export/Cluster Migration

If you upgrade to 2.0, then there is a ring-resize feature built right in. That might be a better approach to take. On Tue, Sep 2, 2014 at 10:07 AM, Mark Rechler mrech...@bitly.com wrote: Hello All, What would be the best tool for moving data from one riak cluster to another? To give some

Re: ID generation techniques

I have questions for your question. 1. What are you using your keys for? Do they get passed around in to clients in Javascript? This is important because Javascript only reliably implements IEEE 754 floating point, which is limited to 53 bits of precision. 2. What backend are you using? In

Re: ID generation techniques

. 3. Coming from RDBMS background, we are used to seeing incremental Ids. K-sort is not a hard requirement but nice to have. On 8/20/14, 11:03 AM, Sargun Dhillon sar...@sargun.me wrote: I have questions for your question. 1. What are you using your keys for? Do they get passed around

Re: riak 2.0 - java client 'unavailable' exception

Is the requirement for having AAE enabled now removed for strong consistency? On Mon, Jul 28, 2014 at 4:55 PM, Joseph Blomstedt j...@basho.com wrote: This means the consistency sub-system is not enabled/active. You can verify this with the output of `riak-admin ensemble-status`. To enable

Re: Cluster security

You really should have some level of IP filtering to prevent people from connecting directly to your BEAM / EPM instances, but even if they do have the ability to make a TCP/IP connection, they have to know the distributed Erlang cookie in order to connect. More on this:

Consistent Riak (riak_ensemble) without Anti-entropy?

So, I noticed that if I don't have anti-entropy on, and I enable strongly consistent Riak, it doesn't work. Specifically, what happens is that riak_kv_ensembles sets up the ensembles, but the riak_ensemble_peer never gets past to all_sync state. It appears that this is because the

Re: Consistent Riak (riak_ensemble) without Anti-entropy?

exchange to occur, as on a real cluster, this might take a while. On Wed, May 28, 2014 at 3:24 AM, Sargun Dhillon sar...@sargun.me wrote: So, I noticed that if I don't have anti-entropy on, and I enable strongly consistent Riak, it doesn't work. Specifically, what happens is that riak_kv_ensembles

Re: RIAK best practices for storage

How much deeper does your tree go? What's the average number of children a node has? What is your query pattern (fetch a parent, and all of its children?)? On Fri, Apr 11, 2014 at 10:13 AM, Sapre, Meghna A meghna.a.sa...@intel.com wrote: Hi all, Most of my data is in parent-child format

Re: Randomizing Bitcask merges

If you can afford the disk space, and potential latency overheads, you can make your bitcask merges less frequent. You can find a window when your databases are going to be in low-usage, and have each node have a separate window for bitcask merges, which (A) will latency-level across your cluster,

Re: Custom hashing (partitioning) functions

://github.com/basho/riak_kv/issues On Mon, Jan 20, 2014 at 3:08 PM, Sargun Dhillon sar...@sargun.me wrote: So, I don't know how many people are aware of if, but Riak supports custom hashing (partitioning) functions. It's exposed as a bucket property (chash_keyfun), in which you can deploy your own

Re: Riak Search and Yokozuna Backup Strategy

Not to fork the thread too far from the topic being discussed, but is there any possibility of opening up the API used for multidatacenter replication? Specifically, the fullsync API? I imagine the code inside riak_repl can also be used for an external node to connect and get a full dump of a

Custom hashing (partitioning) functions

So, I don't know how many people are aware of if, but Riak supports custom hashing (partitioning) functions. It's exposed as a bucket property (chash_keyfun), in which you can deploy your own code to hash keys to ensure data locality to specific vnodes. This can come in handy when doing custom