On Jun 11, 2008, at 4:51 AM, Josh Nisly wrote:
Attached is a patch that implements the make_file_dict as described
above. Let me know what you think.
The patch for cmodule.c looks about right.
For rpath.py I have a concern: From earlier testing, I found try/
except to be about 3-4x slower than if/else, with (almost) no penalty
if the except branch is not taken. So, although your patch would cause
(almost) no penalty for non-Windows, there is a slow down for Windows.
Maybe the solution is to bind the setdata function when rdiff-backup
is loaded or connections are made (based on os.name) -- an extra round-
trip to the remote inside the function is certainly a no-no...
Looking in the CVS logs, it looks like cmodule.c was introduced six
years ago as an "up to 3x speed improvement". It would be interesting
to see if that is still the case today. If it isn't, then, heck, maybe
we can just scrap C.make_file_dict(). I think the old way breaks down
like this:
- os.lstat()
- tuple result before network transmit <-- only necessary for Python
2.2 compatibility
- network transmit (pickle / unpickle)
- build dictionary from result (in Python)
whereas the current method is:
- stat()
- build dictionary from result (in C)
- network transmit (pickle / unpickle)
Andrew
PS -- I also did some more testing regarding the try/except penalty:
=== foo.py ===
#!/usr/bin/python
import os
for i in range(1,5000000):
try:
bar = os.foo()
except AttributeError:
print i
=== foo2.py ===
#!/usr/bin/python
import os
for i in range(1,5000000):
print i
=============
$ time ./foo.py > /dev/null
real 0m17.575s
user 0m17.370s
sys 0m0.138s
$ time ./foo2.py > /dev/null
real 0m3.740s
user 0m3.620s
sys 0m0.101s
_______________________________________________
rdiff-backup-users mailing list at rdiff-backup-users@nongnu.org
http://lists.nongnu.org/mailman/listinfo/rdiff-backup-users
Wiki URL: http://rdiff-backup.solutionsfirst.com.au/index.php/RdiffBackupWiki
