Re: How to set HTTP Last-Modified header from HTTP API

[Sean Cribbs]

The problem is not the Python client per se, but that Riak will always
overwrite this field when you store the object. Although that property is
necessarily writeable from within the client, it has no meaning when
storing the object.


On Fri, Mar 14, 2014 at 3:38 PM, Jeff Peck  wrote:

> Using the python client, it is possible to set an object's "Last-Modified"
> date with obj.last_modified. However, I cannot figure out how to do it via
> curl. I have looked carefully through the riak client source code, but did
> not find anywhere where the Last-Modified header is even passed.
>
> Here is a simple example to show what I mean. Note the Last-Modified
> header being passed:
>
> $ curl -XPUT http://localhost:8098/riak/wcc_dev/test_key -H
> 'Content-Type: text/plain' -H 'Last-Modified: Tue, 19 Feb 2013 04:20:28
> GMT' -d test
>
> And now check out the Last-Modified header in the response:
>
> $ curl -vs http://localhost:8098/riak/wcc_dev/test_key; echo
> * About to connect() to localhost port 8098
> *   Trying 127.0.0.1... connected
> * Connected to localhost (127.0.0.1) port 8098
> > GET /riak/wcc_dev/test_key HTTP/1.1
> > User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5
> OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> > Host: localhost:8098
> > Accept: */*
> >
> < HTTP/1.1 200 OK
> < X-Riak-Vclock: a85hYGBgzGDKBVIclrxzrgWJzlqSwZTInMfK8OTx9DN8WQA=
> < Vary: Accept-Encoding
> < Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
> < Link: ; rel="up"
> < Last-Modified: Fri, 14 Mar 2014 20:34:44 GMT
> < ETag: "3CegcRlc8Is7WtXxosnOEW"
> < Date: Fri, 14 Mar 2014 20:36:13 GMT
> < Content-Type: text/plain
> < Content-Length: 4
> Connection #0 to host localhost left intact
> * Closing connection #0
> test
>
> I have tried passing it as  "Date" too, to no avail.
>
> It must be possible, because the Python API seems to be able to do it,
> right?
>
> Thanks for any help with this.
>
> - Jeff
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>



-- 
Sean Cribbs 
Software Engineer
Basho Technologies, Inc.
http://basho.com/
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

How to set HTTP Last-Modified header from HTTP API

[Jeff Peck]

Using the python client, it is possible to set an object's "Last-Modified" date 
with obj.last_modified. However, I cannot figure out how to do it via curl. I 
have looked carefully through the riak client source code, but did not find 
anywhere where the Last-Modified header is even passed.

Here is a simple example to show what I mean. Note the Last-Modified header 
being passed:

$ curl -XPUT http://localhost:8098/riak/wcc_dev/test_key -H 'Content-Type: 
text/plain' -H 'Last-Modified: Tue, 19 Feb 2013 04:20:28 GMT' -d test

And now check out the Last-Modified header in the response:

$ curl -vs http://localhost:8098/riak/wcc_dev/test_key; echo
* About to connect() to localhost port 8098
*   Trying 127.0.0.1... connected
* Connected to localhost (127.0.0.1) port 8098
> GET /riak/wcc_dev/test_key HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 
> OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: localhost:8098
> Accept: */*
> 
< HTTP/1.1 200 OK
< X-Riak-Vclock: a85hYGBgzGDKBVIclrxzrgWJzlqSwZTInMfK8OTx9DN8WQA=
< Vary: Accept-Encoding
< Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
< Link: ; rel="up"
< Last-Modified: Fri, 14 Mar 2014 20:34:44 GMT
< ETag: "3CegcRlc8Is7WtXxosnOEW"
< Date: Fri, 14 Mar 2014 20:36:13 GMT
< Content-Type: text/plain
< Content-Length: 4
Connection #0 to host localhost left intact
* Closing connection #0
test

I have tried passing it as  "Date" too, to no avail.

It must be possible, because the Python API seems to be able to do it, right?

Thanks for any help with this.

- Jeff
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Format bug on Riak Search PBC Response

[Sean Cribbs]

I would not be concerned that Solr or Riak Search will ever match
4,294,967,296 (2^32) results. uint32 is plenty large for this use.


On Fri, Mar 14, 2014 at 2:45 AM, EmiNarcissus  wrote:

> It was a old bug but I get around it with http request instead. Still want
> to confirm.
>
> message RpbSearchQueryResp {
>   repeated RpbSearchDoc docs  = 1; // Result documents
>   optional floatmax_score = 2; // Maximum score
>   optional uint32   num_found = 3; // Number of results
> }
>
> num_found should better be uint64, uint32 may restrict the number can be
> found by the query.
>
> --
> Best Regards
> Tim Lee
>
>
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>



-- 
Sean Cribbs 
Software Engineer
Basho Technologies, Inc.
http://basho.com/
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Cleaning up bucket after basho_bench run

[István]

Hi,

I am trying to clean up some of the test data that was inserted by
basho_bench. The first approach to use curl and streaming the keys
fails like this:

# curl -XGET -i http://127.0.0.1:8098/buckets/test/keys?keys=stream
HTTP/1.1 200 OK
Vary: Accept-Encoding
Transfer-Encoding: chunked
Server: MochiWeb/1.1 WebMachine/1.10.0 (never breaks eye contact)
Date: Fri, 14 Mar 2014 16:59:08 GMT
Content-Type: application/json

curl: (18) transfer closed with outstanding read data remaining

When I am trying to the same thing with MapReduce it fails like this:

curl -X POST "http://localhost:8098/mapred"; -H "Content-Type:
application/json" -d '{
"inputs": "test",
"query": [
{
"map": {
"language": "javascript",
"source": "function(riakObject) { return [riakObject.key]; }"
}
}
]
}'

Error:

{"phase":0,"error":"bad_utf8_character_code","input":"{ok,{r_object,<<\"test\">>,<<0,116,71,0>>,[{r_content,{dict,3,16,16,8,80,48,{[],[],[],[],[],[],[],[],[],[],[],[],[],[],[],[]},{{[],[],[],[],[],[],[],[],[],[],[[<<\"X-Riak-VTag\">>,71,81,80,81,87,76,105,54,113,120,97,116,114,106,51,86,72,53,67,50,82]],[[<<\"index\">>]],[],[[<<\"X-Riak-Last-Modified\">>|{1391,27501,255280}]],[],[]}}},<<75,191,51,171,193,113,206,163,24,68,247,188,84,72,5,72,179,195,99,44,202,122,136,31,250,94,166,5,160,199,182,137,40,6,253,115,100,4,34,67,64,10,25,210,58,23,104,97,228,...>>}],...},...}"}

I am wondering how else could I just get a list of keys in that
bucket. The ultimate goal is to be able to delete them all.

Thank you in advance,
Istvan

-- 
the sun shines for all

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Question about link-walk results returned by erlang_pb_client

[Alex Moore]

Hi fxmy,

> The return value is just Bucket/Key/link-tag pairs, without ObjectValue or 
> other metadata.
> {ok,[{0,[[<<"people">>,<<"timoreilly">>,<<"friend">>]]},
> {1,[[<<"people">>,<<"dhh">>,<<"friend">>]]}]}
> 
> Is this intended or not?

This is intended.  You are running a map reduce job with one “link” stage, 
which does the link walking for you.   
Link stages only return a list of key/value/relation tuples.

> If so, what's the best way to get these ObjectValues through one single pass 
> of link-walking?

If you want to grab everything in one shot you would have to feed the link 
stage’s output into a map stage to grab the actual objects:

{ok, RiakObj} = riakc_pb_socket:mapred(Pid,[{<<"people">>, <<"timoreilly">>}],
[{link, <<"people">>, <<"friend">>, false},
 {link, <<"people">>, <<"friend">>, false}, 
 {map, {modfun, riak_kv_mapreduce, map_identity}, none, true}]).

This should give you the entire object for the results of the last link phase, 
namely Dave Thomas’s.

I should warn you thought that while link walking and map reduce let you do 
things like this in one shot, you should be cautious about using them in 
production since a bad query can kill performance.  

Thanks,
Alex

On Mar 14, 2014, at 6:49 AM, fxmy wang  wrote:

> 
> Hi, list,
> 
> This should be a trivial question and I think I'm definitely missing 
> something( and feeling stupid :\).
> 
> So when I am doing a chained link-walking through HTTP interface like 
> this(copied from link walking docs):
> curl -v localhost:8091/riak/people/davethomas/_,friend,1/_,friend,_/
>  
> The output is quite verbose, including Bucket/Key/Value etc.etc.
> --JCgqdOHsL4BdXPCb0cuQDnLTxOH
> Content-Type: multipart/mixed; boundary=LpfqXc9urbAJJNFH7aGGPBiAtnX
> 
> --LpfqXc9urbAJJNFH7aGGPBiAtnX
> X-Riak-Vclock: a85hYGBgzGDKBVIcc+04TgWFOj/NYEpkzGNlyNCadoYvCwA=
> Location: /riak/people/timoreilly
> Content-Type: text/plain
> Link: ; riaktag="friend", ; rel="up"
> Etag: 3DmGNeyDj2hUlLR2UhJvMr
> Last-Modified: Thu, 13 Mar 2014 13:11:04 GMT
> 
> I am an excellent public speaker.
> --LpfqXc9urbAJJNFH7aGGPBiAtnX--
> 
> --JCgqdOHsL4BdXPCb0cuQDnLTxOH
> Content-Type: multipart/mixed; boundary=IcBLyeIFObvJlJGyXuhTty5cRSs
> 
> --IcBLyeIFObvJlJGyXuhTty5cRSs
> X-Riak-Vclock: a85hYGBgzGDKBVIcR4M2cgeFOkdkMCUy5rEyzNSadoYvCwA=
> Location: /riak/people/dhh
> Content-Type: text/plain
> Link: ; rel="up"
> Etag: 4qbA2ZufXNgzFRb8PlSLUO
> Last-Modified: Thu, 13 Mar 2014 13:11:53 GMT
> 
> I drive a Zonda.
> --IcBLyeIFObvJlJGyXuhTty5cRSs--
> 
> --JCgqdOHsL4BdXPCb0cuQDnLTxOH--
> 
> 
> But when I retried it through the erlang_pb_client
> riakc_pb_socket:mapred(Pid,[{<<"people">>, <<"davethomas">>}],[{link, 
> <<"people">>, <<"friend">>, true},{link, <<"people">>, <<"friend">>, true}]).
>  
> The return value is just Bucket/Key/link-tag pairs, without ObjectValue or 
> other metadata.
> {ok,[{0,[[<<"people">>,<<"timoreilly">>,<<"friend">>]]},
> {1,[[<<"people">>,<<"dhh">>,<<"friend">>]]}]}
> 
> Is this intended or not?
> If so, what's the best way to get these ObjectValues through one single pass 
> of link-walking?
> 
> 
> Cheers,
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Riak CS : "405 (Method Not Allowed)" when creating a bucket

[Alex Moore]

Hi Mikhail,

> Look like a single point of failure.
> In current version situation is same?

Yes, this is a possible single point of failure, but Stanchion is only needed 
for the creation of buckets and user accounts. Object access is unaffected if 
Stanchion isn’t running.
If stanchion did die, the system would run at partial feature availability 
until you get it running again.  If the node irrecoverably died, you could just 
point all the RiakCS instances to a new stanchion node.

One of Riak 2.0’s features is strong consistency, so we plan on using that in 
the future instead of stanchion. For Riak 1.4.8 (current version), stanchion is 
required. 

Thanks,
Alex


On Mar 14, 2014, at 12:44 AM, ten  wrote:

> Hello,
> 
> "Bucket creation is one of the operations handled by 'stanchion'.  So,
> make sure you've install stanchion installed on only one node and that
> all riak-cs nodes are pointing at that one stanchion node."
> 
> Look like a single point of failure.
> In current version situation is same?
> 
> Regards,
> Mikhail.
> 
> ___
> riak-users mailing list
> riak-users@lists.basho.com
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Re: Partitions placement

[Ciprian Manea]

Hi Daniel,

"A Little Riak Book" covers the logic behind partition allocation in an
overly simplified way.

Riak will distribute partitions to vnodes in a pseudo-random fashion,
resulting in allocations like you described. These allocations are less
optimal when the number of riak nodes are small, hence we (strongly)
recommend 5+ nodes for production use.

Storing 3 data copies in 3 different servers sounds trivial to do, but not
that easy to scale up once the numbers of servers grows. To cope with
scalability, Riak introduces an "overlay". Data is first placed in
"partitions" (always a power of 2) which are then distributed to different
server nodes. As powers of 2 are not divisible by 3, this approach has a
problem at lower scale: some nodes will hold a few extra partitions (which
were not intended to be stored there).

If you know you are not going to need n_val greater then 3 in your buckets,
one way to hint this to Riak and get a better distribution of partitions to
nodes is to configure [0] target_n_val to 3.


[0]
http://docs.basho.com/riak/latest/ops/advanced/configs/configuration-files/


Regards,
Ciprian


On Fri, Mar 14, 2014 at 12:09 AM, Daniel Iwan  wrote:

> Below is an output of my Riak cluster. 3 physical nodes. Ring size 128.
> As far as I can tell when Riak installed fresh it is always place
> partitions
> in the same way on a ring as long as number of vnodes and servers is the
> same.
>
> All presentations including "A Little Riak Book' show pretty picture of
> ring
> and nodes claiming partitions in a  sequential fashion. That's clearly not
> a
> case.
> Output below shows that node2 is picked as favourite, which means replicas
> of certain keys will definitely be on the same hardware. Partitions are
> split 44 + 42 + 42. Why not 43+43+42?
>
> Another thing, why the algorithm selects nodes in 'random' non-sequential
> fashion? When the cluster gets created and nodes 2 & 3 are joined to node
> 1,
> it's a clear situation. Partitions are empty so vnodes could be assigned in
> a way so there's no consecutive partitions on the same hw.
> My issue is that in my case if node2 goes down and I'm storing some data
> with N=2 I will definitely not be able access certain keys and more
> surprisingly all 2i will no longer work for the buckets with N=2 due to
> {error,insufficient_vnodes_available}. That is all 2i's for those buckets.
>
> I understand that when new nodes are attached Riak tries to avoid
> reshuffling everything and just moves certain partitions, and at that point
> you may end up with copies on the same physical nodes. But even then Riak
> should make best effort and try not to put consecutive partitions on the
> same server. If it has to move it anyway it could as well put it on any
> other machine but the one that holds partition with preceding and following
> index.
> I also understand Riak does not guarantee that replicas are on distinct
> servers (why? it should, at least for N=2 and N=3 if possible)
>
> I appreciate minimum recommended setup is 5 nodes and I should be storing
> with N=3 minimum.
> But I just find it confusing when presentations show something that is not
> even remotely close to reality.
>
> Just to be clear I have nothing against Riak, I think it's great though bit
> disappointing that there are no stronger conditions about replica placement
> here.
>
> I'm probably missing something and simplifying too much. Any clarification
> appreciated.
>
> Daniel
>
>
> riak@10.173.240.1)2>
> (riak@10.173.240.1)2> {ok, Ring} = riak_core_ring_manager:get_my_ring().
> {ok,
>  {chstate_v2,'riak@10.173.240.1',
>   [{'riak@10.173.240.1',{303,63561952927}},
>{'riak@10.173.240.2',{31,63561952907}},
>{'riak@10.173.240.3',{25,63561952907}}],
>   {128,
>[{0,'riak@10.173.240.1'},
> {11417981541647679048466287755595961091061972992,
>  'riak@10.173.240.2'},
> {22835963083295358096932575511191922182123945984,
>  'riak@10.173.240.2'},
> {34253944624943037145398863266787883273185918976,
>  'riak@10.173.240.3'},
> {45671926166590716193865151022383844364247891968,
>  'riak@10.173.240.1'},
> {57089907708238395242331438777979805455309864960,
>  'riak@10.173.240.2'},
> {68507889249886074290797726533575766546371837952,
>  'riak@10.173.240.2'},
> {79925870791533753339264014289171727637433810944,
>  'riak@10.173.240.3'},
> {91343852333181432387730302044767688728495783936,
>  'riak@10.173.240.1'},
> {102761833874829111436196589800363649819557756928,
>  'riak@10.173.240.2'},
> {114179815416476790484662877555959610910619729920,
>  'riak@10.173.240.2'},
> {12559779695812446953312916531172001681702912,
>  'riak@10.173.240.3'},
> {137015778499772148581595453067151533092743675904,
>  'riak@10.173.240.1'},
> {148433760041419827630061740822747494183805648896,
>  'riak@10.173.240.2'},
> {159851741583067506678528028578343455274867621888,
>  'riak@10.173.240.2'},
> {1712697231

Question about link-walk results returned by erlang_pb_client

[fxmy wang]

Hi, list,

This should be a trivial question and I think I'm definitely missing
something( and feeling stupid :\).

So when I am doing a chained link-walking through HTTP interface like
this(copied
from link walking
docs
):

> curl -v localhost:8091/riak/people/davethomas/_,friend,1/_,friend,_/


The output is quite verbose, including Bucket/Key/Value etc.etc.

> --JCgqdOHsL4BdXPCb0cuQDnLTxOH
>
Content-Type: multipart/mixed; boundary=LpfqXc9urbAJJNFH7aGGPBiAtnX
>

> --LpfqXc9urbAJJNFH7aGGPBiAtnX
>
X-Riak-Vclock: a85hYGBgzGDKBVIcc+04TgWFOj/NYEpkzGNlyNCadoYvCwA=
>
Location: /riak/people/timoreilly
>
Content-Type: text/plain
>
Link: ; riaktag="friend", ; rel="up"
>
Etag: 3DmGNeyDj2hUlLR2UhJvMr
>
Last-Modified: Thu, 13 Mar 2014 13:11:04 GMT
>

> I am an excellent public speaker.
>
--LpfqXc9urbAJJNFH7aGGPBiAtnX--
>

> --JCgqdOHsL4BdXPCb0cuQDnLTxOH
>
Content-Type: multipart/mixed; boundary=IcBLyeIFObvJlJGyXuhTty5cRSs
>

> --IcBLyeIFObvJlJGyXuhTty5cRSs
>
X-Riak-Vclock: a85hYGBgzGDKBVIcR4M2cgeFOkdkMCUy5rEyzNSadoYvCwA=
>
Location: /riak/people/dhh
>
Content-Type: text/plain
>
Link: ; rel="up"
>
Etag: 4qbA2ZufXNgzFRb8PlSLUO
>
Last-Modified: Thu, 13 Mar 2014 13:11:53 GMT
>

> I drive a Zonda.
>
--IcBLyeIFObvJlJGyXuhTty5cRSs--
>

> --JCgqdOHsL4BdXPCb0cuQDnLTxOH--
>


But when I retried it through the erlang_pb_client

> riakc_pb_socket:mapred(Pid,[{<<"people">>, <<"davethomas">>}],[{link,
> <<"people">>, <<"friend">>, true},{link, <<"people">>, <<"friend">>,
> true}]).


The return value is just Bucket/Key/link-tag pairs, without ObjectValue or
other metadata.

> {ok,[{0,[[<<"people">>,<<"timoreilly">>,<<"friend">>]]},

{1,[[<<"people">>,<<"dhh">>,<<"friend">>]]}]}
>

Is this intended or not?
If so, what's the best way to get these ObjectValues through one single
pass of link-walking?


Cheers,
___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com

Format bug on Riak Search PBC Response

[EmiNarcissus]

It was a old bug but I get around it with http request instead. Still want to 
confirm.

message RpbSearchQueryResp {
  repeated RpbSearchDoc docs      = 1; // Result documents
  optional float        max_score = 2; // Maximum score
  optional uint32       num_found = 3; // Number of results
}

num_found should better be uint64, uint32 may restrict the number can be found 
by the query.

--  
Best Regards  
Tim Lee


___
riak-users mailing list
riak-users@lists.basho.com
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com