Re: [9fans] de-duplication/archival software

2010-10-15 Thread Bakul Shah
On Thu, 14 Oct 2010 22:35:57 PDT Tharaneedharan Vilwanathan vdhar...@gmail.com wrote: hi, about a year back, someone mentioned about a tool or utility for archiving/storage like venti. it has de-duplication and uses hashing mechanism. i think it runs on linux mainly. can someone give me

Re: [9fans] de-duplication/archival software

2010-10-15 Thread Tharaneedharan Vilwanathan
hi bakul, thanks for the quick response. i did see the thread on zfs but what i remember is a tool rather than a file system. i even remember green letter over black background webpage that talked about this tool (manpage?). i will try for a little more time and then probably give up. regards

Re: [9fans] πp

2010-10-15 Thread Sape Mullender
Are we talking about πP or 9P? ΠP doesn't have Twalk. It has open, which combines clone, walk, and open from 9P. Before you start jumping up and down and telling me that you can't open without revieing the qids from the walk (to check them for mount points), let me tell you that we are also

Re: [9fans] de-duplication/archival software

2010-10-15 Thread Thomas Östreicher
On Fri, Oct 15, 2010 at 7:35 AM, Tharaneedharan Vilwanathan vdhar...@gmail.com wrote: hi, about a year back, someone mentioned about a tool or utility for archiving/storage like venti. it has de-duplication and uses hashing mechanism. i think it runs on linux mainly. can someone give me some

Re: [9fans] πp

2010-10-15 Thread hiro
Is it Pi-Ro, or do you actually call it Pipi?

Re: [9fans] πp

2010-10-15 Thread Jacob Todd
Eh, what's Πp?

Re: [9fans] πp

2010-10-15 Thread Iruatã Souza
2010/10/15 Jacob Todd jaketodd...@gmail.com: Eh,  what's Πp? check http://iwp9.org/slides/osprey.pdf

Re: [9fans] πp

2010-10-15 Thread Sape Mullender
We pronouce it pie pee Is it Pi-Ro, or do you actually call it Pipi?

Re: [9fans] πp

2010-10-15 Thread erik quanstrom
isn't tag field for this intended? [...] so this means to me that a client can send some T-messages and then (or concurrently) wait the R-messages. in inferno from mount(1) and styxmon(8) i deduced that this case is also considered. it's true that most of the servers i seen until now

Re: [9fans] πp

2010-10-15 Thread Russ Cox
A simpler way to add pipelining to 9P is to define that multiple outstanding messages with the same tag are allowed and that the server must process and respond to messages with a given tag in the order it receives them. This only requires changes to servers that are actually multithreaded. All

Re: [9fans] Status of 9vx on Windows?

2010-10-15 Thread Russ Cox
On Thu, Oct 14, 2010 at 3:16 PM, Skip Tavakkolian skip.tavakkol...@gmail.com wrote: Has anyone attempted to get it running on Windows? Andre Gunther had a look during GSoC a few years back. It is a pain. Windows does not trigger the exception handler if it thinks the segment registers are

Re: [9fans] de-duplication/archival software

2010-10-15 Thread Enrico Weigelt
* Tharaneedharan Vilwanathan vdhar...@gmail.com wrote: hi, about a year back, someone mentioned about a tool or utility for archiving/storage like venti. it has de-duplication and uses hashing mechanism. i think it runs on linux mainly. can someone give me some details on this if you

Re: [9fans] πp

2010-10-15 Thread David Leimbach
2010/10/14 Latchesar Ionkov lu...@ionkov.net It can't be dealt with the current protocol. It doesn't guarantee that Topen will be executed once Twalk is done. So can get Rerrors even if Twalk succeeds. It can be dealt with if the scheduling of the pipeline is done properly. You just have to

Re: [9fans] πp

2010-10-15 Thread David Leimbach
2010/10/15 Sape Mullender s...@plan9.bell-labs.com Are we talking about πP or 9P? It's about both. I was just curious about how 9P was deficient in terms of pipelining. It might not be optimal for all cases of pipelining, but the protocol seems to support it in certain cases just fine. ΠP

Re: [9fans] πp

2010-10-15 Thread Francisco J Ballesteros
It's not just that you can stream requests or not. If you have caches in the path to the server, you'd like to batch together (or stream or whatever you'd like to call that) requests so that if a client is reading a file and a single rpc suffices, the cache, in the worst case, knows that it has to

Re: [9fans] πp

2010-10-15 Thread Latchesar Ionkov
I already implemented that behavior in go9p on the plane back from Seattle. I need to test it a bit before I check it in the repository. The good thing about it is that it doesn't break any existing clients or servers. How is it going to be exposed to the clients is another issue. I am still not

Re: [9fans] πp

2010-10-15 Thread cinap_lenrek
i wonder if making 9p work better over high latency connections is even the right answer to the problem. the real problem is that the data your program wants to work on in miles away from you and transfering it all will suck. would it not be cool to have a way to teleport/migrate your process to

Re: [9fans] πp

2010-10-15 Thread Lyndon Nerenberg
i wonder if making 9p work better over high latency connections is even the right answer to the problem. the real problem is that the data your program wants to work on in miles away from you and transfering it all will suck. would it not be cool to have a way to teleport/migrate your process

Re: [9fans] πp

2010-10-15 Thread Latchesar Ionkov
What if the data your process needs is located on more than one server? Play ping-pong? Thanks, Lucho 2010/10/15 cinap_len...@gmx.de: i wonder if making 9p work better over high latency connections is even the right answer to the problem.  the real problem is that the data your program

Re: [9fans] πp

2010-10-15 Thread cinap_lenrek
fork! -- cinap ---BeginMessage--- What if the data your process needs is located on more than one server? Play ping-pong? Thanks, Lucho 2010/10/15 cinap_len...@gmx.de: i wonder if making 9p work better over high latency connections is even the right answer to the problem.  the real

Re: [9fans] πp

2010-10-15 Thread Latchesar Ionkov
And how is fork going to help when the forked processes need to exchange the data over the same high-latency link? 2010/10/15 cinap_len...@gmx.de: fork! -- cinap -- Forwarded message -- From: Latchesar Ionkov lu...@ionkov.net To: Fans of the OS Plan 9 from Bell Labs

Re: [9fans] πp

2010-10-15 Thread ron minnich
2010/10/15 cinap_len...@gmx.de: i wonder if making 9p work better over high latency connections is even the right answer to the problem. The reason I care is that the link from a CPU node to a file server on blue gene is high latency. It might as well be cross-country, it's so bad.   would it

Re: [9fans] amd64 port

2010-10-15 Thread Eric Van Hensbergen
On Fri, Oct 15, 2010 at 12:29 AM, Tharaneedharan Vilwanathan vdhar...@gmail.com wrote: hi, i think we could stress on a specific architecture (and aim to provide basic and improved support) based on these criteria: - mainline architecture - cheap and affordable both for companies and

Re: [9fans] πp

2010-10-15 Thread cinap_lenrek
if it doesnt help, you apply the mechanism to the wrong problem :) or the mechanism is not that usefull as i thought... thanks ron for your comment! i was just hoping to get some responses from the osprey dudes as they had it on ther slides :) -- cinap ---BeginMessage--- And how is fork going

Re: [9fans] πp

2010-10-15 Thread Latchesar Ionkov
If the mechanism cannot be applied to every (even wrong) problem, then it still doesn't solve the file I/O over high-latency links issue that we started with first. 2010/10/15 cinap_len...@gmx.de: if it doesnt help, you apply the mechanism to the wrong problem :) or the mechanism is not that

Re: [9fans] πp

2010-10-15 Thread Charles Forsyth
The transition from a stream to the packet-oriented file IO protocol has never been comfortable. `RPC-oriented' might be more accurate than `packet-oriented', given the way that streams are implemented.

Re: [9fans] πp

2010-10-15 Thread Nemo
back to the 80s, or was it 70s? many people have implemented migration. the first i remember is swaping (yes, not paging) out a process, then swapping it back into a different machine. iirc, it might be the sprite unix, not quite sure. the point is, the migrated process still needs all the

Re: [9fans] πp

2010-10-15 Thread ron minnich
2010/10/15 Charles Forsyth fors...@terzarima.net: The transition from a stream to the packet-oriented file IO protocol has never been comfortable. `RPC-oriented' might be more accurate than `packet-oriented', given the way that streams are implemented. Correct. ron

Re: [9fans] πp

2010-10-15 Thread ron minnich
2010/10/15 Nemo nemo.m...@gmail.com: So, when I hear migration, I just tend to see what happens after it has been implemented and faces the spaghethy phase. And even if you get that right, it may not work well on hardware. We saw cases with linux migration, while migrating from one x86 to

Re: [9fans] πp

2010-10-15 Thread Bakul Shah
On Fri, 15 Oct 2010 18:19:50 +0200 cinap_len...@gmx.de wrote: i wonder if making 9p work better over high latency connections is even the right answer to the problem. the real problem is that the data your program wants to work on in miles away from you and transfering it all will suck.

Re: [9fans] de-duplication/archival software

2010-10-15 Thread Tharaneedharan Vilwanathan
hi thomas, yes, this is the one. thanks a lot! i couldnt find it because my search keywords didnt match. regards dharani On Fri, Oct 15, 2010 at 3:20 AM, Thomas Östreicher thomas.oestreic...@gmail.com wrote: On Fri, Oct 15, 2010 at 7:35 AM, Tharaneedharan Vilwanathan vdhar...@gmail.com wrote:

Re: [9fans] πp

2010-10-15 Thread David Leimbach
2010/10/15 cinap_len...@gmx.de i wonder if making 9p work better over high latency connections is even the right answer to the problem. the real problem is that the data your program wants to work on in miles away from you and transfering it all will suck. would it not be cool to have a way

Re: [9fans] πp

2010-10-15 Thread erik quanstrom
And even if you get that right, it may not work well on hardware. We saw cases with linux migration, while migrating from one x86 to another, where valid FP values would cause the target to get an FP trap. Made no sense, but it's what happened, because the two x86's were different

Re: [9fans] de-duplication/archival software

2010-10-15 Thread Nathaniel W Filardo
On Fri, Oct 15, 2010 at 01:49:58PM +0200, Enrico Weigelt wrote: * Tharaneedharan Vilwanathan vdhar...@gmail.com wrote: hi, about a year back, someone mentioned about a tool or utility for archiving/storage like venti. it has de-duplication and uses hashing mechanism. i think it runs on

Re: [9fans] πp

2010-10-15 Thread erik quanstrom
On Fri Oct 15 12:33:19 EDT 2010, lu...@ionkov.net wrote: What if the data your process needs is located on more than one server? Play ping-pong? one either plays ping pong with the process or data. one could imagine cases where the former case makes sense. - erik

Re: [9fans] πp

2010-10-15 Thread Latchesar Ionkov
There are definitely cases when moving the code instead of the data makes sense. But that discussion is mostly unrelated to the one on how to make the file I/O work better over high-latency links. 2010/10/15 erik quanstrom quans...@quanstro.net: On Fri Oct 15 12:33:19 EDT 2010, lu...@ionkov.net

[9fans] seattle?

2010-10-15 Thread Bruce Ellis
if anyone is still in seattle my google redirect is 513-3BRUCEE

[9fans] IWP9 Attendees List

2010-10-15 Thread Skip Tavakkolian
9fans, I am making the IWP9 attendees list and am missing one name; it is the 9fan who recorded the audio of all the sessions. Could you send me email with your name? Thanks, -Skip

Re: [9fans] πp

2010-10-15 Thread dorin bumbu
2010/10/15 erik quanstrom quans...@quanstro.net: isn't tag field for this intended? [...] so this means to me that a client can send some T-messages and then (or concurrently) wait the R-messages. in inferno from mount(1) and styxmon(8) i deduced that this case is also considered. it's

Re: [9fans] πp

2010-10-15 Thread Julius Schmidt
Perhaps I'm getting this all wrong, but to me this seems like an interesting idea, especially if you consider the impact of being near the files on some classically considered computationally stressy tasks like compiling (esp. with kencc). So moving the code near the data definitely seems worth

Re: [9fans] πp

2010-10-15 Thread David Leimbach
CPUs have big caches to move the code closer to the data (well a copy of the data anyway). Closeness in general is good, the question is what to move and how :-) Dave 2010/10/15 Julius Schmidt a...@phicode.de Perhaps I'm getting this all wrong, but to me this seems like an interesting idea,

Re: [9fans] πp

2010-10-15 Thread John Floren
It is probably worth trying. However, it wouldn't make copying a file from sources any faster, or help a Blue Gene node do a snapshot any quicker. John 2010/10/15 Julius Schmidt a...@phicode.de: Perhaps I'm getting this all wrong, but to me this seems like an interesting idea, especially if

Re: [9fans] πp

2010-10-15 Thread erik quanstrom
On Fri Oct 15 17:14:18 EDT 2010, bumbudo...@gmail.com wrote: 2010/10/15 erik quanstrom quans...@quanstro.net: isn't tag field for this intended? [...] so this means to me that a client can send some T-messages and then (or concurrently) wait the R-messages. in inferno from mount(1)

Re: [9fans] πp

2010-10-15 Thread dorin bumbu
2010/10/16 erik quanstrom quans...@quanstro.net: On Fri Oct 15 17:14:18 EDT 2010, bumbudo...@gmail.com wrote: 2010/10/15 erik quanstrom quans...@quanstro.net: isn't tag field for this intended? [...] so this means to me that a client can send some T-messages and then (or

Re: [9fans] πp

2010-10-15 Thread cinap_lenrek
when you are in kiev, video streams YOU! -- cinap ---BeginMessage--- 2010/10/16 erik quanstrom quans...@quanstro.net: On Fri Oct 15 17:14:18 EDT 2010, bumbudo...@gmail.com wrote: 2010/10/15 erik quanstrom quans...@quanstro.net: isn't tag field for this intended? [...] so this means to

Re: [9fans] πp

2010-10-15 Thread dorin bumbu
ok, let's say that i want to read a 2 gb file :) 2010/10/16 cinap_len...@gmx.de: when you are in kiev, video streams YOU! -- cinap -- Forwarded message -- From: dorin bumbu bumbudo...@gmail.com To: Fans of the OS Plan 9 from Bell Labs 9fans@9fans.net Date: Sat, 16 Oct