Mostly due to having to deal with a bunch of FS stuff in the world of
npm, I've written a few utilities that you might find useful in the
context of "doing stuff with bunches of files".
If your goal really is to walk over a file system and match filenames
against patterns then node-glob is probably what you need. It's 100%
js now, and fully matches the behavior of glob(3).
I'm not sure about network drops and reconnects, but graceful-fs will
handle EMFILE errors. I'd be tempted to call write errors an error,
and just abort.
If you want to do a deep copy, and preserve file attributes as well as
contents, then fstream has a lot of helpful stuff to do that. (It's
the file system streaming util that node-tar is built on.) In fact,
filewalker looks an awful lot like fstream, except that fstream is
designed to be able to do
fstream.Reader("/some/dir").pipe(fstream.Writer("/some/other/dir")).
By necessity (since tarballs need to support them) fstream also
handles symlinks, hard links, and so on. I extended it to
fstream-ignore to provide a DirReader class that will skip over any
globs mentioned in a .ignore file. Then I extended that to
fstream-npm, to provide a DirReader class that encapsulates all the
"what goes in a package" logic for npm's use.
My goal is eventually to not even cache tarballs on the filesystem,
and instead just write the tar stream directly to the upload when you
publish. To support that, fstream is 100% pause-strict; if you
.pause() a readable stream, it will not emit any data until calling
resume(), or it's a bug.
On Mon, Apr 16, 2012 at 00:34, Oliver Leics <[email protected]> wrote:
> On Mon, Apr 16, 2012 at 8:21 AM, Vitaly Puzrin <[email protected]> wrote:
>> According to my experience, most use cases require just file scan with
>> pattern-matching. Big flexibility causes code complicity on user side.
>> Here is example, that we did for our project
>> https://github.com/nodeca/fs-tools . Not as flexible as your one, but very
>> simple in use.
>
> filewalker is less about flexibility, filewalker is more about reliability.
>
> Say, I have to copy a large amount of files. I really want to copy
> them in a way that guarantees me that those files are in fact copied,
> no matter how often a networkdrive drops and reconnects, no matter how
> much open files the system can handle, etc pp.
>
> https://github.com/oleics/node-filewalker
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to [email protected]
> To unsubscribe from this group, send email to
> [email protected]
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
