I mentioned before that I'm writing a system to watch the filesystem for
changes. I'm finishing that up and I have a few questions about
concurrency issues.
1) How do I determine what a safe maximum number of threads is?
2) I think that (fold-files) is the best way to walk a directory tree,
since it will process the files in constant space without building a giant
list of files the way (find-files) would. Is there a better option? For
this specific purpose all I care about is getting the subdirectories --
processing the actual files happens elsewhere -- so if there was a way to
say "
3) If you have any comments on the following they would be appreciated:
First, the spec:
- Monitor a directory tree for any changes, ensure that the DB accurately
records what's on the disk.
- The tree probably isn't more than 5-10 directories deep, but it could be
arbitrarily more.
- There are potentially (tens of) millions of files in the tree.
- Any given directory probably only has a few thousand files at most.
I monitor a directory with this:
(define change (sync (filesystem-change-evt "/foo")))
...do something with 'change'...
but that only monitors /foo, not the subdirectories of /foo. To monitor
the rest of the tree I will need to walk down from /foo creating new
monitor events for each subdirectory.
I believe that:
- When I create the 'monitor this' event, I will need to wrap the sync in a
thread since (sync) is synchronous.
- The result of the sync will be the event itself, so I can then do
whatever it takes and then plug it back into sync to keep watching.
- It's okay to have each monitoring thread do its own DB processing as
opposed to having it instruct a different thread (probably via a channel)
to deal with it.
Thoughts?
--
You received this message because you are subscribed to the Google Groups
"Racket Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to racket-users+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
