Re: [Haskell-cafe] Filesystem questions
Hello Andrew, Friday, October 12, 2007, 9:21:07 PM, you wrote: I notice that getDirectoryContents appears to return its results in alphabetical order. Is this behaviour actually guaranteed? on NTFS filesystem, files are stored in directory alphabetically sorted. on FAT disks the order may be arbitrary -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Yitzchak Gale wrote: How about a built-in function that represents a directory tree as a lazy Data.Tree? Please no. The last thing haskell needs is more dangerous semantically broken non-referentially-transparent lazy IO structures. Jules ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
I wrote:
...a tool for recursing through directories...
How about a built-in function that represents a directory tree
as a lazy Data.Tree?
Bryan O'Sullivan wrote:
See System.FilePath.Find in
http://hackage.haskell.org/cgi-bin/hackage-scripts/package/FileManip-0.2
Not a very good idea. Representing a directory structure as a tree
makes people think they can manipulate it as if it were a tree, which
leads to all kinds of nasty bugs when the real world bleeds through the
holes in the abstraction.
You are right. Well, I didn't say what I meant by a
lazy Data.Tree :).
Your library is very nice. But - it suffers from the
same problem. You use unsafe IO operations to build
a lazy IO list, and we all know what grief that can
lead to.
Especially in this application, using that type of laziness
is really unsafe. Any user must be explicitly aware of
potential side effect complications, such as directory
structure shifting under your feet as you work, possibly
induced by your own traversal if you are not careful.
(In my opinion, the documention for Python's os.walk
http://docs.python.org/lib/os-file-dir.html#l2h-2715
gives a nice, clear description of what must be
avoided during such a traversal.)
So I think that what is needed here is something like
the following. Based on my experience, this would be
a really convenient API:
data TraversalDirection = TopDown | BottomUp
data Directory = Directory {
dirPath :: FilePath,
fileNames :: [FilePath],
subdirNames :: [FilePath]}
-- List all directories in the tree rooted at the given path
traverseDirectories :: MonadIO m =
TraversalDirection - FilePath - ListT m Directory
-- List all non-directory files in the tree rooted at the given path
traverseFiles :: MonadIO m =
TraversalDirection - FilePath - ListT m FilePath
traverseFiles = join . fmap (liftList . fileNames) . traverseDirectories
-- List the paths to all subdirectories in the tree rooted at the given path
traverseSubdirs :: MonadIO m =
TraversalDirection - FilePath - ListT m FilePath
traverseSubdirs = join . fmap (liftList . subdirNames) . traverseDirectories
Here it is critical that ListT be taken from
http://haskell.org/haskellwiki/ListT_done_right
and not the broken implementation that comes with mtl.
(Finally fixed in 6.8? Please?)
You could plug the above into your machinery for
recursion predicates and all the other nice stuff.
-Yitz
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
I wrote: How about a built-in function that represents a directory tree as a lazy Data.Tree? Jules Bean wrote: Please no. The last thing haskell needs is more dangerous semantically broken non-referentially-transparent lazy IO structures. Agreed. I would definitely not want it to be a dangerous semantically broken non-referentially-transparent lazy IO structure. -Yitz ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
I wrote: ...a tool for recursing through directories... How about a built-in function that represents a directory tree as a lazy Data.Tree? Bryan O'Sullivan wrote: See System.FilePath.Find in http://hackage.haskell.org/cgi-bin/hackage-scripts/package/FileManip-0.2 -- List all directories in the tree rooted at the given path traverseDirectories :: MonadIO m = TraversalDirection - FilePath - ListT m Directory You could plug the above into your machinery for recursion predicates and all the other nice stuff. Or - getting back to the lazy Data.Tree idea - we could define TreeT, analgous to ListT: newtype TreeT m a = NodeT (m (a, TreeT (ListT m) a)) and give it a nice Monad instance. Then you can prune and filter trees in a natural way, inside the monad. -Yitz ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Hi Yitzchak, On Sun, Oct 14, 2007 at 11:33:38AM +0200, Yitzchak Gale wrote: Here it is critical that ListT be taken from http://haskell.org/haskellwiki/ListT_done_right and not the broken implementation that comes with mtl. (Finally fixed in 6.8? Please?) mtl is not part of GHC (although it is currently included in some distributions as an extra lib). If you want to propose that mtl switches to a different definition of ListT then please see http://www.haskell.org/haskellwiki/Library_submissions Thanks Ian ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Yitzchak Gale wrote: Your library is very nice. But - it suffers from the same problem. You use unsafe IO operations to build a lazy IO list, and we all know what grief that can lead to. This is little different from the approach taken by Python's os.walk, which lazily yields the contents of a directory tree as it traverses it. I'm a little unclear on why one appears good in your eyes, while the other is not, beyond perhaps the depth/breadth knob and differences in amount of documentation. Maybe you could expand on that a bit? b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Hi Bryan, You wrote: This is little different from the approach taken by Python's os.walk, which lazily yields the contents of a directory tree as it traverses it. I'm a little unclear on why one appears good in your eyes, while the other is not, beyond perhaps the depth/breadth knob and differences in amount of documentation. Maybe you could expand on that a bit? Well firstly, I never said that your approach is not good. I was very pleased to see your work on this, and I was inspired by it. I do think that it is much better to provide IO laziness using monad transformers (or whatever) rather than unsafe IO. The basic difference is that with unsafe IO, you relinquish control over the order in which things happen, and leave it to the whims of the compiler/runtime. Besides the fact that it can always lead to bugs (viz. the constant problems people have with readFile and getContents), in this case the exact order of the side effects happens to be a critical and delicate issue, so you really don't want to give up that control here. My second proposal introduces another deviation from your approach in that it presents a full tree view rather than a pre-flattened tree view. I think that is more natural and more flexible - just my own taste. Regards, Yitz ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Yitzchak Gale wrote: I do think that it is much better to provide IO laziness using monad transformers (or whatever) rather than unsafe IO. That's fair enough. I think it would be great if you were to turn your ideas into a library and provide a few examples of its use. b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
On Sun, 14 Oct 2007, Bryan O'Sullivan wrote: Yitzchak Gale wrote: I do think that it is much better to provide IO laziness using monad transformers (or whatever) rather than unsafe IO. That's fair enough. I think it would be great if you were to turn your ideas into a library and provide a few examples of its use. I'm also interested in a more structured approach to lazy reading of data. I used unsafeInterleaveIO several times (including readFile) and got several problems including too many open files. Problems, that are certainly solved easily if you have more control over the IO laziness. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Andrew Coppin wrote: Is there a way to get rid of . and .. in the results? Brandon S. Allbery wrote: Manual filtering is always required, whether C, Perl, Haskell, etc. I dunno, maybe python filters them for you or something. Correct, Python filters them out. This is clearly the correct behavior. That is what is needed in the vast majority of cases, and it is still reasonably easy to deal with the unusual cases. It is too bad that Haskell is among the many languages that get this wrong. Python also has os.walk, a very convenient functional (sort of) tool for recursing through directories. (It sounds trivial, but it is not, there are enough annoying details that this function saves huge amounts of time.) Very embarrassing that Haskell is missing this. How about a built-in function that represents a directory tree as a lazy Data.Tree? -Yitz ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
On Sat, 13 Oct 2007, Yitzchak Gale wrote: Andrew Coppin wrote: Is there a way to get rid of . and .. in the results? Brandon S. Allbery wrote: Manual filtering is always required, whether C, Perl, Haskell, etc. I dunno, maybe python filters them for you or something. Correct, Python filters them out. This is clearly the correct behavior. That is what is needed in the vast majority of cases, and it is still reasonably easy to deal with the unusual cases. It is too bad that Haskell is among the many languages that get this wrong. me too Python also has os.walk, a very convenient functional (sort of) tool for recursing through directories. (It sounds trivial, but it is not, there are enough annoying details that this function saves huge amounts of time.) Very embarrassing that Haskell is missing this. Maybe it is already in one of the Haskell for scripting packages? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
On Sat, Oct 13, 2007 at 23:27:13 +0200, Yitzchak Gale wrote: Andrew Coppin wrote: Is there a way to get rid of . and .. in the results? Brandon S. Allbery wrote: Manual filtering is always required, whether C, Perl, Haskell, etc. I dunno, maybe python filters them for you or something. Correct, Python filters them out. This is clearly the correct behavior. That is what is needed in the vast majority of cases, and it is still reasonably easy to deal with the unusual cases. It is too bad that Haskell is among the many languages that get this wrong. Python also has os.walk, a very convenient functional (sort of) tool for recursing through directories. (It sounds trivial, but it is not, there are enough annoying details that this function saves huge amounts of time.) Very embarrassing that Haskell is missing this. How about a built-in function that represents a directory tree as a lazy Data.Tree? http://therning.org/magnus/index.php?tag=haskellpaged=3 Not really what you're looking for, but hopefully it's a good place to start. /M -- Magnus Therning (OpenPGP: 0xAB4DFBA4) magnus＠therning．org Jabber: magnus．therning＠gmail．com http://therning.org/magnus signature.asc Description: Digital signature ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
Yitzchak Gale wrote: Python also has os.walk, a very convenient functional (sort of) tool for recursing through directories. (It sounds trivial, but it is not, there are enough annoying details that this function saves huge amounts of time.) Very embarrassing that Haskell is missing this. See System.FilePath.Find in http://hackage.haskell.org/cgi-bin/hackage-scripts/package/FileManip-0.2 How about a built-in function that represents a directory tree as a lazy Data.Tree? Not a very good idea. Representing a directory structure as a tree makes people think they can manipulate it as if it were a tree, which leads to all kinds of nasty bugs when the real world bleeds through the holes in the abstraction. b ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
All these questions are actually Windows-centric; the answers are different on Unix. On Oct 12, 2007, at 13:21 , Andrew Coppin wrote: I notice that getDirectoryContents appears to return its results in alphabetical order. Is this behaviour actually guaranteed? There is no guarantee, however on Windows this is done by invoking Win32 directory globbing routines which happen to return things in alpha order. (On Unix it might be alpha order if the underlying filesystem is a btree, but there are no guarantees at all except perhaps that '.' and '..' come first.) Related: Is there a way to get rid of . and .. in the results? (Obviously this causes directory recusion to malfunction.) I can of course manually filter them out, but it's annoying. Manual filtering is always required, whether C, Perl, Haskell, etc. I dunno, maybe python filters them for you or something. Every language I've ever used makes you deal. doesDirectoryExist C: doesDirectoryExist C:\ both yield False? (This frequently trips up programs that try to check that a destination folder exists.) Ask Microsoft. Drive top level handling has been weird since MS-DOS 2.0. -- brandon s. allbery [solaris,freebsd,perl,pugs,haskell] [EMAIL PROTECTED] system administrator [openafs,heimdal,too many hats] [EMAIL PROTECTED] electrical and computer engineering, carnegie mellon universityKF8NH ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Filesystem questions
On Fri, 12 Oct 2007 18:21:07 +0100, you wrote: I notice that getDirectoryContents appears to return its results in alphabetical order. Is this behaviour actually guaranteed? This is a Windows thing. All of the NT-based operating systems list files in alphabetical order by default. You see the same thing if you use the DIR command from a command-line prompt. Steve Schafer Fenestra Technologies Corp. http://www.fenestra.com/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Filesystem questions
I notice that getDirectoryContents appears to return its results in alphabetical order. Is this behaviour actually guaranteed? Related: Is there a way to get rid of . and .. in the results? (Obviously this causes directory recusion to malfunction.) I can of course manually filter them out, but it's annoying. Finally, why do doesDirectoryExist C: doesDirectoryExist C:\ both yield False? (This frequently trips up programs that try to check that a destination folder exists.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
