Fisher, In the cases where you see file size discrepancies, are you able to retrieve those files intact from riak? Having a node in the cluster with an unstable network connection should not cause data loss because replicas should be saved to other nodes in your cluster, but it may cause some issues with being able to properly access those files via luwak. The chances of data being lost when a riak node goes down is dependent on what you mean by the node going down. In most cases, the replicas will take care of it, but in the case that multiple machines in your cluster fail, you should be familiar with the sync strategies for the backend you are using. For example, you can read about the bitcask sync strategies here: http://wiki.basho.com/Bitcask.html.
Kelly On Dec 5, 2011, at 12:56 AM, vuleetu wrote: > Hi, all > > I am working on a storage system which receives data from the client and > stores it to luwak. The server also records the file size of that file which > is uploaded successfully to the database, but i found it is not the same size > as the data in luwak even after several days, so that is not a consistency > problem. > > The following is the main process: > > 1) check if file exists already > luwak_file:exists() > > 2) if it doesn't exist, will use luwak_file:create to create one, or else > will use luwak_file:get to get that file handle, > > 3) use the last offset which is recorded in database, and the create put > stream > luwak_put_stream:start(Riak, RiakFile, Offset, ?STREAM_PUT_TTL) > > 4) receive data from client, and write to put stream created above > gen_tcp: recv() > luwak_put_stream:send(PutStream, Binary) > > Is there any possibility that data is lost because one of the riak node's > network is not stable? Because i found when i call luwak_put_stream:put(), it > just send message contain the data to one process, so i dont know if data is > lost or sent successfully. Even if I call flush, it is still a request > message to that process. Is there any function to tell me if the write was > successful or not? What about if that riak node goes down in the middle of > writing data to luwak, what would happen, will it cause the data to be lost? > > > Thanks, > Fisher > _______________________________________________ > riak-users mailing list > [email protected] > http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
_______________________________________________ riak-users mailing list [email protected] http://lists.basho.com/mailman/listinfo/riak-users_lists.basho.com
