With regard to assigning files to chunks, I suggest we start by using an
algorithm similar to that we use in DHT.
hash=$(cat $filename | md5sum) # convert from hex to decimal?
chunk=$((hash % number_of_chunks))
if [ x"$chunk" = x"$my_chunk_id" ]; then
bash $filename # ...and so on
fi
This is completely automatic, robust as the test set or directory structure
changes (or as the number of workers changes), and should give us an
approximately equal distribution among chunks.
_______________________________________________
Gluster-devel mailing list
[email protected]
http://www.gluster.org/mailman/listinfo/gluster-devel
