We've talked about having pvfs2-client pull out duplicates (or the
kernel module) in the cases where one of those chooses to break a
readdir into multiple operations, but we haven't spent much time
investigating where the replication is actually happening in order to
accomplish this.
Other solutions could include locking the directory (not going to
happen), restarting the ls entirely if the directory changes during
the read (would cause starvation), improving ls to remove duplicates
on its own (probably realistic for pvfs2-ls, unlikely to get accepted
by GNU tools group for stock ls), and reordering directory entries
returned so that most recently changing entries are returned last
(high overhead on server to sort, lots of coding probably).
Any other ideas?
Yeah I guess my proposed changes wouldn't help in this case. Berkeley
db has the notion of a secondary (read-only) database based on the
primary, where keys in the secondary are based on the primary's data
and some function you provide. So it might be possible to create a
secondary database for iterating based on update time instead of
alphabetically based on component name. I'm not sure how efficient
that would be though...we might just be pushing the sorting problem to
the db layer. Also, we would have to start storing update times in
keyval dirent entries.
I started thinking about some more possible ideas, but I realized after
looking closer at the code that I don't actually see why duplicates
would occur in the first place with the algorithm that is being used :)
I apologize if this has been discussed a few times already, but could
we walk through it one more time?
I know that the request protocol uses a token (integer based) to keep
track of position. However, the pcache converts this into a particular
key based on where the last iteration left off. This key contains the
handle as well as the alphanumeric name of the entry.
Trove then does a c_get on that key with the DB_SET flag, which should
put the cursor at the proper position. If the entry has been deleted
(which is not happening in my case- I am only creating files), then it
retries the c_get with the DB_SET_RANGE flag which should set the cursor
at the next position. "next" in this case is defined by the comparison
function, PINT_trove_dbpf_keyval_compare().
The keyval_comare() function sorts the keys based on handle value, then
key length, then stncmp of the key name.
This means that essentially we are indexing off of the name of the entry
rather than a position in the database.
So how could inserting a new entry between readdir requests cause a
duplicate? The old entry that is stored in the pcache should still be
valid. If the newly inserted entry comes after it (according to the
keyval_comare() sort order), then we should see it as we continue
iterating. If the new entry comes before it, then it should not show up
(we don't back up in the directory listing). It doesn't seem like there
should be any combination that causes it to show up twice.
Is c_get() not traversing the db in the order defined by the
keyval_comare() function?
The only other danger that I see is that if the pcache_lookup() fails,
the code falls back to stepping linearly through the db to the token
position which I could imagine might have ordering implications.
However, I am only talking to the server from a single client, so I
don't see why it would ever miss the pcache lookup.
I just want to confirm that there is actually an algorithm problem here
rather than just a bug in the code somewhere.
-Phil
_______________________________________________
Pvfs2-developers mailing list
Pvfs2-developers@beowulf-underground.org
http://www.beowulf-underground.org/mailman/listinfo/pvfs2-developers
