Here is a different idea that might also work:
1. Initiate breadth-first-search retrieval of IFileTree data
2. While retrieving data in a stream, allow to send a
blacklist of folders to the remote as evaluate by a
local programmatic filter
3. As soon as the remote receives the blacklist, it
prunes the BFS tree that it's currently working on
by the root nodes received.
Sounds complex, but would allow for the most efficient
evaluation of complex filters like "return folders under
Team support only" even on a remote file system.
Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of
Oberhuber, Martin
Sent: Tuesday, October 21, 2008 12:07 AM
To: E4 developer list
Subject: RE: [eclipse-incubator-e4-dev] [resources] EFS
IFileTree
The advantage of the Directory Stream approach is that clients
can
see / work on intermediate results before the traversal is
complete,
and they can cancel the Stream retrieval if the intermediate
results don't match their expectations.
I agree that programmatic approaches like visitors or arbitrary
coded filters are problematic when the receiver of the query
is remote (Java does support things like remote object
execution,
but not all server-sides support Java...and local evaluation of
the filter might not always be a good fallback). LDAP filters
might be an interesting thing to explore, I haven't heard about
these before but anything Standard seems like a good thing
to look at.
Here is a different idea that might also work:
* Initiate
Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of John
Arthorne
Sent: Monday, October 20, 2008 9:12 PM
To: E4 developer list
Subject: RE: [eclipse-incubator-e4-dev] [resources] EFS
IFileTree
IFileTree doesn't have a depth because it was really
intended to optimize the DEPTH_INFINITE case. For DEPTH_ZERO or
DEPTH_ONE, it doesn't provide much advantage. I agree though that it
could be interesting to support different scoping to pick up smaller
subtrees, or to have some termination condition. The purpose was really
to cut down on round-trips, so I'm not sure you'd get the same effect
with a visitor or directory stream approach. Perhaps filters could be
passed in a serialized form (like an LDAP filter), so that it can be
interpreted on the machine where the file system lives, to avoid passing
unnecessary data over the wire.
John
"Oberhuber, Martin" <[EMAIL PROTECTED]>
Sent by: [EMAIL PROTECTED]
10/20/2008 12:49 PM
Please respond to
E4 developer list <[email protected]>
To
"E4 developer list" <[email protected]>
cc
Subject
RE: [eclipse-incubator-e4-dev] [resources] EFS IFileTree
Hi John,
One disadvantage of the IFileTree today is that the file
tree to be
returned must terminate at some point, and not the
slightest
amount of information is returned if it does not
terminate.
This makes it unusable for file systems that can be
endless
(like the Internet), or even very very large with the
user only
being interested in parts of the file tree. Even
refreshLocal
performs a breadth-first-search and can be limited by
the
depth to visit. fetchFileTree() does not support that.
One option for improving this situation could be if the
fetchFileTree() method would not return a snapshot like
the IFileTree, but rather return a Stream of IFileStore
/ IFileInfo
objects that the client can cancel at any time. A
breadth-first-search algorithm for returning these nodes
might
be preferred, though I'd also see some potential for
allowing
arbitrary ordering of returned nodes with some concept
of
allowing a node to be "incomplete" e.g. a folder node
with
not all children returned yet.
In JSR 203, the Path#newDirectoryStream() [1]
<http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Path.html#ne
wDirectoryStream(java.nio.file.DirectoryStream.Filter)> and
Files#walkFileTree() [2]
<http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Files.html#w
alkFileTree(java.nio.file.Path,%20java.nio.file.FileVisitor)> methods
go in this direction.
[1]
http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Path.html#new
DirectoryStream(java.nio.file.DirectoryStream.Filter
<http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Path.html#ne
wDirectoryStream(java.nio.file.DirectoryStream.Filter> )
[2]
http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Files.html#wa
lkFileTree(java.nio.file.Path,%20java.nio.file.FileVisitor
<http://openjdk.java.net/projects/nio/javadoc/java/nio/file/Files.html#w
alkFileTree(java.nio.file.Path,%20java.nio.file.FileVisitor> )
Cheers,
--
Martin Oberhuber, Senior Member of Technical Staff, Wind
River
Target Management Project Lead, DSDP PMC Member
http://www.eclipse.org/dsdp/tm
<http://www.eclipse.org/dsdp/tm>
________________________________
From: [EMAIL PROTECTED]
[mailto:[EMAIL PROTECTED] On Behalf Of John
Arthorne
Sent: Friday, October 17, 2008 7:40 PM
To: E4 developer list
Subject: [eclipse-incubator-e4-dev] [resources] EFS
IFileTree
I forgot to mention this during the meeting, so I just
wanted to mention a little know EFS interface called IFileTree. The idea
of this interface was to allow for batched interaction with an entire
file sub-tree. This prevents the large number of round-trips needed when
a client needs to walk over an entire subtree of a slow/remote file
system. It allows you to get a snapshot of an entire remote tree state
in a single round-trip. In some experiments we did back in the day, this
dramatically sped up certain operations like refreshLocal over a high
latency remote tree. I mention it only because it's probably an
under-exploited concept that could perhaps be expanded upon to improve
performance in remote resource scenarios. I could imagine expanding the
idea to allow a client to queue up a whole batch of file changes, that
could be fired off in a single round-trip to the remote system for
processing.
John_______________________________________________
eclipse-incubator-e4-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
_______________________________________________
eclipse-incubator-e4-dev mailing list
[email protected]
https://dev.eclipse.org/mailman/listinfo/eclipse-incubator-e4-dev
