Re: Updating Riak values in batch

2016-06-14 Thread Luke Bakken 
Hi Ricardo,

I don't have examples right at hand, here is a link to our advanced
map/reduce examples:

http://docs.basho.com/riak/kv/2.1.4/developing/app-guide/advanced-mapreduce/

Our own Joe Caswell provided this example: http://stackoverflow.com/a/23633491

If you run into issues after following our examples, feel free to
share what you've done via a gist and we can continue to help out.

--
Luke Bakken
Engineer
lbak...@basho.com

On Tue, Jun 14, 2016 at 10:51 AM, Ricardo Mayerhofer
 wrote:
> Hi Luke,
> Thanks for your reply! Can I use map/reduce operation to update documents?
> Do you have any link with an example?
>
> On Tue, Jun 14, 2016 at 2:44 PM, Luke Bakken  wrote:
>>
>> Hi Ricardo,
>>
>> If you know your keys in advance, you can fetch the keys in parallel,
>> update them, and write them back to Riak in parallel.
>>
>> Other options include map/reduce jobs that iterate over all the keys,
>> but keep in mind that any key listing operation will be resource
>> intensive.
>> --
>> Luke Bakken
>> Engineer
>> lbak...@basho.com
>>
>>
>> On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
>>  wrote:
>> > Hi all,
>> > I've a large dataset in Riak (about 20 million keys), storing JSON
>> > documents. I'd like to update those documents to remove a JSON
>> > attribute.
>> > What's the best way to approach this problem in Riak?
>> >
>> > Thanks.
>> >
>> > --
>> > Ricardo Mayerhofer
>> >
>> > ___
>> > riak-users mailing list
>> > riak-users@lists.basho.com
>> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>> >
>
>
>
>
> --
> Ricardo Mayerhofer

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: get and put operations are slow on Riak KV cluster

2016-06-14 Thread Luke Bakken 
Hello Abhinav,

Have you followed all of our suggested tuning in this document?

http://docs.basho.com/riak/kv/2.1.4/using/performance/

Specifically, these settings may help a lot in your environment:

http://docs.basho.com/riak/kv/2.1.4/using/performance/#optional-i-o-settings


--
Luke Bakken
Engineer
lbak...@basho.com


On Mon, Jun 13, 2016 at 12:00 AM, Abhinav Tripathi
 wrote:
> Hi,
> I am seeing quite slow get and put operations on our Riak KV cluster. We are
> new to Riak and have tried to follow the instructions in the documentation.
> It would be great if you could take a look and let us know what are we doing
> wrong.
>
> We are using Riak as a distributed cache for our application. The 95th
> percentile (and above) get operations are >10ms. Similarly, 99th and above
> percentile for put operations is getting close to 10ms. Ideally, we want the
> 99% gets to stay around 1ms and 99% puts to stay <5ms.
>
> I have attached the configurations for all 3 nodes in our cluster. Also,
> attached are recent console.log files. I have attached a recent one-minute
> stats file as well from which the below values are.
>
> GETs
> "node_get_fsm_time_100": 71429,
> "node_get_fsm_time_95": 16551,
> "node_get_fsm_time_99": 27513,
> "node_get_fsm_time_mean": 4393,
> "node_get_fsm_time_median": 989
>
> PUTs
> "node_put_fsm_time_100": 10785,
> "node_put_fsm_time_95": 2679,
> "node_put_fsm_time_99": 7800,
> "node_put_fsm_time_mean": 1752,
> "node_put_fsm_time_median": 1608
>
> I have recorded many one-minute stats for our cluster. Most of them for the
> past couple of days show high values like these.
>
> All three Riak nodes are 4-core, 16GB RAM, 128 GB SSD machines ... basically
> m3.xlarge AWS instances.
>
> Also, I can see many such lines in the console.log,
>
> 2016-06-13 00:03:32.545 [info] <0.23701.3269> Merged
> {["/apps/riak/lib/bitcask/50239118783249787813251666124688006726811648/75.bitcask.data","/apps/riak/lib/bitcask/50239118783249787813251666124688006726811648/76.bitcask.data","/apps/riak/lib/bitcask/50239118783249787813251666124688006726811648/77.bitcask.data","/apps/riak/lib/bitcask/50239118783249787813251666124688006726811648/78.bitcask.data"],[]}
> in 596.530484 seconds.
>
> These merge operations are usually taking close to 600 seconds. Could that
> be a problem as well?
>
> Thanks,
> Abhinav.
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Updating Riak values in batch

2016-06-14 Thread Ricardo Mayerhofer 
Hi Luke,
Thanks for your reply! Can I use map/reduce operation to update documents?
Do you have any link with an example?

On Tue, Jun 14, 2016 at 2:44 PM, Luke Bakken  wrote:

> Hi Ricardo,
>
> If you know your keys in advance, you can fetch the keys in parallel,
> update them, and write them back to Riak in parallel.
>
> Other options include map/reduce jobs that iterate over all the keys,
> but keep in mind that any key listing operation will be resource
> intensive.
> --
> Luke Bakken
> Engineer
> lbak...@basho.com
>
>
> On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
>  wrote:
> > Hi all,
> > I've a large dataset in Riak (about 20 million keys), storing JSON
> > documents. I'd like to update those documents to remove a JSON attribute.
> > What's the best way to approach this problem in Riak?
> >
> > Thanks.
> >
> > --
> > Ricardo Mayerhofer
> >
> > ___
> > riak-users mailing list
> > riak-users@lists.basho.com
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
> >
>



-- 
Ricardo Mayerhofer
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Updating Riak values in batch

2016-06-14 Thread Luke Bakken 
Hi Ricardo,

If you know your keys in advance, you can fetch the keys in parallel,
update them, and write them back to Riak in parallel.

Other options include map/reduce jobs that iterate over all the keys,
but keep in mind that any key listing operation will be resource
intensive.
--
Luke Bakken
Engineer
lbak...@basho.com


On Mon, Jun 13, 2016 at 11:06 AM, Ricardo Mayerhofer
 wrote:
> Hi all,
> I've a large dataset in Riak (about 20 million keys), storing JSON
> documents. I'd like to update those documents to remove a JSON attribute.
> What's the best way to approach this problem in Riak?
>
> Thanks.
>
> --
> Ricardo Mayerhofer
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com