On 23 Feb 2012, at 14:37, Jeremiah Peschka wrote:
> Out of interest, to those of us who are rusty on their shell scripting, could
> you share the aforementioned script?
find png_test -name "*.png" -print0 | xargs -0 -P 4 -I file curl -X PUT
localhost:8098/riak/file --data-binary @file -H"Content-Type: image/png"
I'm a shell scripting n00b so there is probably a much better way. In this case
png_test is a directory I created with 1000*10k pngs. So the bucket name is the
directory name 'png_test'. Only works for a flat dir, it was just something to
check the times that Eric mentioned, please don't think it is like a working
script or anything.
Cheers
Russell
> ---
> Jeremiah Peschka
> Managing Director, Brent Ozar PLF, LLC
>
>
> On Thu, Feb 23, 2012 at 2:02 AM, Russell Brown <[email protected]> wrote:
> Hi Eric,
>
> On 23 Feb 2012, at 08:45, Eric Fong wrote:
>
> > Hi
> >
> > Help. I want to store few thousands of small png (each of them will just be
> > 10K) into riak.
> > But I find that is really slow (1 to 2 second for one png).
> >
> > Currently, I use a bash script to find and loop the directory
> > And curl PUT the png onto riak.
> >
> > Do we have a bulk upload or will it be faster if I use erlang or or driver?
> > I need to upload 1000 small png within one minute. Can it be done?
>
> I imagine it is possible. I just managed to do it in 10seconds on my laptop,
> single node with n=3 and r=2 using xargs and curl to put in parallel.
>
> How many nodes do you have? What is your n-val?
>
> cheers
>
> Russell
>
> >
> > Currently, I am on riak 1.01 and levelDB backend.
> >
> > Thanks a lot!
> > --
> > Best Regards,
> > Eric Fong
> > _______________________________________________
> > riak-users mailing list
> > [email protected]
> > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
>
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
>
> _______________________________________________
> riak-users mailing list
> [email protected]
> http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________
riak-users mailing list
[email protected]
http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
