On 02/14/2013 12:19 PM, Erik Søe Sørensen wrote:
Those 18.4MB/s is utilized bandwidth, I presume?
Yes. What I find odd is that riak is writing max 6-700KB/sec on disk.
And how about the net between client and Riak?
They're on the same machine: the Java client and the riak process (one
node).
Further questions:
I've already answered, but it's ok to refresh again:
- Which backend are you using - LevelDB?
Bitcask, many writer threads each with it's own riakClient instance
configured to use all nodes in cluster.
(If LevelDB: how does it look over time - is it faster at first, but
slowing down?)
- And how many writes do you do in parallel?
If you're only writing from one thread, your problem is probably not
bandwidth, but latency.
Regards,
On 14-02-2013 10:21, Bogdan Flueras wrote:
What I have is:
Network bandwidth between nodes: 18.4 MB/sec
Network cards: 1.000Mbps full duplex
Riak process: IO: avg 600 K/s for disk write, 7% RAM used, 17% CPU used
ing. Bogdan Flueras
On Thu, Feb 14, 2013 at 10:17 AM, Erik Søe Sørensen <[email protected]
<mailto:[email protected]>> wrote:
Just to rule something out: what is the capacity of the network
between the cluster and the client - and among the cluster nodes?
-------- Oprindelig meddelelse --------
Fra: Shuhao <[email protected] <mailto:[email protected]>>
Dato:
Til: [email protected] <mailto:[email protected]>
Emne: Re: Tune Riak for fast inserts - populate DB
Hi,
I've had similar experiences with riak. I used the Riak Python client
with HTTP and PBC (even rolled my own client! That only had a
~10% speed
up though, and it was HTTP only). I've also examined that the
Riak host
machine's CPU is only at around 70% when I insert. Other NoSQL
dbs uses
about 100% when I just continuous insert. Some DBs even have bulk
insert.
Maybe this is an issue with Riak itself... I don't really know.
Maybe a
Riak dev could chip in on the conversation
Cheers,
Shuhao
On 13-02-13 02:37 AM, Bogdan Flueras wrote:
> Hello all,
> I've got a 5 node cluster with Riak 1.2.1, all machines are
multicore,
> with min 4GB RAM.
>
> I want to insert something like 50 million records in Riak with
the java
> client (Protobuf used) with default settings. I've tried also
with HTTP
> protocol and set w = 1 but got some problems.
>
> However the process is very slow: it doesn't write more than
6GB/ hour or
> aprox. 280 KB/second.
> To have all my data filled in, it would take aprox 2 days !!
>
> What can I do to have the data filled into Riak ASAP?
> How should I configure the cluster ? (vm.args/ app.config) I
don't care so
> much about consistency at this point.
>
> Thank you,
> ing. Bogdan Flueras
>
--
Mobile: + 45 26 36 17 55 | Skype: eriksoesorensen | Twitter: @eriksoe
Trifork A/S | Margrethepladsen 4 | DK-8000 Aarhus C |
www.trifork.com <http://www.trifork.com/>
--
ing. Bogdan Flueras
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
