Re: [tahoe-dev] Advertised invalid node port

2010-06-15 Thread Brian Warner
On 6/15/10 2:48 PM, slush wrote: > Update: > > After few hours and many restarts of both nodes I finally see correct > port on slush2. I was confused a little because I expect that new port > will be advertised immediately after slush1 restart and reconnect to > node slush2. Hm, it sounds like th

Re: [tahoe-dev] servers-of-happiness default of 7 prevents first-time installation from working "out of the box"

2010-06-15 Thread Brian Warner
> At 2010-06-15 17:14 (-0600), Zooko O'Whielacronx wrote: > >> One possible solution to this would be to lower the default >> servers-of-happiness from 7 to 1. This would require us to also lower >> the default number of shares needed from 3 to 1, because the current >> code won't let you have a

Re: [tahoe-dev] Advertised invalid node port

2010-06-16 Thread Brian Warner
On 6/16/10 9:31 AM, David-Sarah Hopwood wrote: > Zooko wrote: >> If you >> aren't serving as a web gateway then you are not going to act as a >> Tahoe-LAFS storage client -- only as a storage server (or upload >> helper or introducer), so you shouldn't elicit connections from >> storage servers. >

Re: [tahoe-dev] Advertised invalid node port

2010-06-16 Thread Brian Warner
On 6/15/10 8:52 PM, Zooko O'Whielacronx wrote: > > Wait, what? You wanted a node which wasn't a server to receive > connections from other nodes that weren't servers? Or you wanted a > node which wasn't a server to receive connections from servers and for > some reason the servers weren't opening c

Re: [tahoe-dev] servers-of-happiness default of 7 prevents first-time installation from working "out of the box"

2010-06-16 Thread Brian Warner
>>> At 2010-06-15 17:14 (-0600), Zooko O'Whielacronx wrote: >>> One possible solution to this would be to lower the default servers-of-happiness from 7 to 1. This would require us to also lower the default number of shares needed from 3 to 1, because the current code won't let

Re: [tahoe-dev] Advertised invalid node port

2010-06-16 Thread Brian Warner
On 6/16/10 2:12 PM, slush wrote: > Hi Brian, > > Did you consider something like NAT punch [1]? > > And did you consider uPnP? Briefly. Those are some of our oldest enhancement-request tickets: STUNT: http://tahoe-lafs.org/trac/tahoe-lafs/ticket/50 UPnP: http://tahoe-lafs.org/trac/tahoe-lafs

Re: [tahoe-dev] note about hash-based digital signatures

2010-06-23 Thread Brian Warner
On 6/23/10 7:18 AM, David-Sarah Hopwood wrote: > David-Sarah Hopwood wrote: > > Ah, but it will work for a multi-layer Merkle tree scheme, such as > GMSS: if keys are generated deterministically from a seed, then the > signatures certifying keys at upper layers are also deterministic, so > there's

Re: [tahoe-dev] where has all my disk space gone

2010-06-26 Thread Brian Warner
On 6/25/10 11:59 PM, Randy Bush wrote: > > i am down 20g since reboot. /private/var/vm content has been constant. > > rmac.psg.com:/Users/randy> du -s .tahoe > 52 .tahoe Weird! Tahoe only touches the disk underneath it's base directory (which is ~/.tahoe by default), so if du doesn't show

Re: [tahoe-dev] what do we want from decentralized introduction?

2010-07-14 Thread Brian Warner
On 7/9/10 11:18 AM, Zooko O'Whielacronx wrote: I have some questions about how decentralized (gossip-based) introduction is supposed to work. Faruq (and everyone who cares about decentralized introduction!) please tell me if my assumptions are wrong. I'd discourage a change that causes tahoe.c

Re: [tahoe-dev] Sighting reports [correction]

2010-07-14 Thread Brian Warner
On 7/13/10 5:45 PM, Kyle Markley wrote: When a storage server accepts responsibility for a share during peer selection, it makes a placeholder file of the same size as it was asked to store. This is wrong -- the storage server writes a small placeholder file, not a placeholder file of the same

Re: [tahoe-dev] Perf-related architecture question

2010-07-21 Thread Brian Warner
On 7/20/10 11:01 PM, Kyle Markley wrote: KM> I am running a helper, and see that while the helper is fetching KM> ciphertext that the storage nodes see essentially no activity. Makes KM> sense. But what I don't understand is why it takes so long to fetch KM> the ciphertext. The client-to-helper

Re: [tahoe-dev] Perf-related architecture question

2010-07-22 Thread Brian Warner
On 7/21/10 11:47 PM, Zooko O'Whielacronx wrote: On Thu, Jul 22, 2010 at 12:38 AM, Kyle Markley wrote: KM>> This was exciting to read. The encrypt+encode/transfer ping-pong KM>> guarantees that we will either be using CPU, or network, but not KM>> both simultaneously, leading to low utilizatio

Re: [tahoe-dev] privacy bug with google chart api

2010-07-24 Thread Brian Warner
On 7/24/10 1:04 PM, Greg Troxel wrote: > > I was very surprised to see requests to > > http://chart.apis.google.com/chart > > From the web UI. It seems that while one can argue that sending the > hash of a serverid is not tha big a deal, it still reveals the IP > address of a tahoe client to g

Re: [tahoe-dev] Perf-related architecture question

2010-07-24 Thread Brian Warner
On 7/23/10 7:18 PM, Kyle Markley wrote: > > I'll vary these parameters: > > 1) pipeline_size (in WriteBucketProxy.__init__) > - What values would be interesting, and in what step size? > 2) Segment size (ticket #809) (in class Client?) > - I'll try each power of two from 32KiB to 4MiB. > 3) A

Re: [tahoe-dev] privacy bug with google chart api

2010-07-25 Thread Brian Warner
On 7/24/10 2:12 PM, Terrell Russell wrote: > Also, consider flot for standalone javascript graph building. > > http://code.google.com/p/flot/ Hey, this is great! I think this will do exactly what I need. thanks! -Brian ___ tahoe-dev mailing list taho

Re: [tahoe-dev] File seeks

2010-07-25 Thread Brian Warner
On 7/25/10 5:09 PM, slush wrote: > > is there any way how to download, say, second half of large file > without downloading first half? Is that even teoritically possible in > current implementation (can I decode file from any position)? Yes and no. We carefully designed the file-encoding scheme

Re: [tahoe-dev] Tahoe benchmarking data

2010-07-25 Thread Brian Warner
Wow, what awesome data! On 7/25/10 1:02 PM, Kyle Markley wrote: > > For large files, tuning the parameters seems not to make a lot of > difference. Increasing the pipeline_size helps, but quickly levels > off, and changes in the segment_size then don't seem to matter either. > > For small files

Re: [tahoe-dev] share discovery and scalability

2010-07-25 Thread Brian Warner
On 7/22/10 10:22 PM, Ravi Pinjala wrote: > Hi all, > > So I have a few questions about how Tahoe scales. First, how well would > it currently handle cases where there are thousands (or millions?) of > clients? I remember reading in the documentation that a new client tries > to connect to every no

Re: [tahoe-dev] different uids, and multiple introducer/client/server nodes on one box

2010-07-25 Thread Brian Warner
On 7/23/10 1:52 PM, Greg Troxel wrote: > > Run the introducer node as a service uid, like tahoe.tahoe, similar > to how one runs servers as an unprivileged user. Yeah, good idea. > Run a server node as a service uid, perhaps the same as above, on > many machines. This would mean introduc

Re: [tahoe-dev] Tahoe benchmarking data

2010-07-25 Thread Brian Warner
On 7/25/10 8:10 PM, Kyle Markley wrote: > Brian, > > Yeah, I think we're approximately saturating the network during the large > file transfers. But for the small files, both network and CPU load are > very low (under 10%). Yup. I suspect that your large files are running into python's performan

Re: [tahoe-dev] report of an unsuccessful assault on our fortress

2010-07-26 Thread Brian Warner
>> For example, when serving HTML files from the WUI, serve them from a >> different origin (hostname:different-port, >> ip-instead-of-hostname:same-port, >> ip-instead-of-hostname:different-port, et c.). > > This would be a promising solution to our problem, if we could > persuade the browser tha

Re: [tahoe-dev] IRC isn't a reliable way to learn about Tahoe-LAFS

2010-07-26 Thread Brian Warner
On 7/26/10 12:32 PM, Zooko O'Whielacronx wrote: > Anyway, was wrong that repair works on immutable files and > directories given a verify cap to the thing—instead you require a > read-cap to the thing. Repair also works on mutable files and > directories given a write-cap to the thing (which mean

Re: [tahoe-dev] documentation project diagramming thought

2010-07-26 Thread Brian Warner
On 7/26/10 1:49 PM, Raoul Duke wrote: > > i consider myself not completely useless at > drawing/graphing/informatics etc. even though i'm not a graphic > designer by profession. anybody in the sf bay area who'd think this > sort of project would be fun to work on, on and off? Sure! I'm in SF.. I'

Re: [tahoe-dev] report of an unsuccessful assault on our fortress

2010-07-26 Thread Brian Warner
> The unguessable caps make the attack payload trickier than the usual > trivial-pwnage payload, but not impossible. Yeah, it means that the attacker cannot acquire authority (the ability to read or write a tahoe file) by merely guessing at a URL: they have to steal one from a tab which already k

Re: [tahoe-dev] Tahoe benchmarking data

2010-07-26 Thread Brian Warner
On 7/26/10 2:29 PM, Chris Palmer wrote: > Brian Warner writes: > >> Yup. I suspect that your large files are running into python's performance >> limits: the best way to speed those up will be to move our transport to >> something with less overhead (signed HTT

[tahoe-dev] new-downloader pushed!

2010-08-04 Thread Brian Warner
I just finished pushing the new-downloader (#798) code into trunk. This represents about 6 months of persistent effort: it's a great relief to finally get it published. I'll start working on the bugs that Francois found (#1154, #1155) tomorrow, hopefully I can get them fixed in a day or two. Plea

Re: [tahoe-dev] Introducer request period

2010-08-04 Thread Brian Warner
On 7/29/10 11:04 PM, eurekafag wrote: > I've noticed that NAT'ed nodes require a lot of time to connect to a > new node with external IP. Something between 20 mins and an hour. > Where can I set introducer request period to make it faster? Node with > external IP can't connect to one behind the NAT

Re: [tahoe-dev] Upload misunderstanding or bug?

2010-08-04 Thread Brian Warner
On 7/28/10 5:30 AM, Greg Troxel wrote: >> >> I needed a workaround, so I thought of one. Since I wanted to use >> 2/4/4, shares.happy == shares.total, and I had exactly 4 storage >> nodes in my grid, so I reasoned that I should expect every storage >> node to get exactly one share. My workaround w

Re: [tahoe-dev] IPv6 support?

2010-08-05 Thread Brian Warner
>>> I wasn’t sure, so I thought I’d bring it up – does Tahoe support >>> IPv6 yet? If not, where is it on the roadmap? >> >> Not yet. There's a feature request filled in trac #867 and it has >> been previously discussed on this mailing-list, see this post¹. I updated #867 with a pointer to a new f

Re: [tahoe-dev] same origin

2010-08-05 Thread Brian Warner
On 7/29/10 12:50 AM, James A. Donald wrote: > Some time ago, someone proposed a local service that would map all > domains of the form *.tahoe-stuff to the same network address, thereby > allowing every web page to have a separate origin, thus preventing > common origin attacks, but this would cre

[tahoe-dev] hashlib-vs-pycryptopp benchmarks

2010-08-09 Thread Brian Warner
A few weeks ago we were discussing whether to stick with the pycryptopp/Crypto++ version of certain hash functions, or if we should switch to Python's standard-library's hashlib versions. On many (most?) platforms, hashlib uses libopenssl's functions. Out of curiosity, I ran some quick benchmarks.

Re: [tahoe-dev] Tahoe-LAFS v1.8.0 potentially delayed by performance issue

2010-08-13 Thread Brian Warner
On 8/13/10 12:03 PM, Wayne Scott wrote: > Does it matter what version the machine in the cluster are running? > > -Wayne Nope. None of the server-side code changed from 1.7 to 1.8. The expected speedups of the new-downloader code in 1.8 are on small files (<100KB): specifically 1.8 should have m

Re: [tahoe-dev] [tahoe-lafs] #1170: does new-downloader perform badly for certain situations (such as today's Test Grid)?

2010-08-14 Thread Brian Warner
On 8/14/10 6:15 AM, Greg Troxel wrote: > >> attachment:1.8.0c2-r4698-run-106-down-0.html and every request-block >> in it (for three different shares) went to the same server -- >> nszizgf5 -- which was the first server to respond to the DYHB >> (barely) and which happened to be the only serve

Re: [tahoe-dev] New logo revision (was Re: About Us page)

2010-08-21 Thread Brian Warner
> * If you render it, make sure you have the Futura font or the layout > will be more or less off depending on font metrics. Hm, is there a slightly more interesting-looking font than Futura that we could use for this? I want to avoid being a font snob, but maybe something more.. thicker? ro

Re: [tahoe-dev] Benchmarks - 1.7.1 vs. 1.8.0c3

2010-09-06 Thread Brian Warner
On 9/5/10 10:00 PM, Kyle Markley wrote: > I was surprised by the low CPU utilization even when all the nodes > were local. The storage nodes typically ran about 2-3% utilization and > the client node meandered around 15-30%. With everything local (and no > network to blame) I expected high utiliza

[tahoe-dev] plugin architecture for tahoe?

2010-09-06 Thread Brian Warner
So, I've been wondering if we should build some sort of plug-in mechanism into Tahoe to make it easier to ship non-core features without adding to the dependency load of the core system. I think the webapi is a pretty core feature, and basic HTTP comes with Twisted for free. Having a "welcome pag

Re: [tahoe-dev] FUSE/SFTP (was: plugin architecture for tahoe?)

2010-09-08 Thread Brian Warner
On 9/7/10 2:05 PM, Greg Troxel wrote: > > wrt. sftp, I'd like to see a fuse (high-level api) client. but i > haven't the spare time to hack on it. Incidentally, I originally put more energy into SFTP than into a direct FUSE frontend because there's a good FUSE-to-SFTP layer for all the platforms

Re: [tahoe-dev] Benchmarks - 1.7.1 vs. 1.8.0c3

2010-09-08 Thread Brian Warner
On 9/7/10 6:08 AM, Kyle Markley wrote: > Zooko, > >> Oh say it ain't so! Upgrading to 1.8.0c3 from 1.7.1 will make your >> large file downloads go about half as fast as they used to? > > Don't despair -- my result might be true only for low-latency grids! We > already have lots of data showing

Re: [tahoe-dev] 1.8.0 progress report

2010-09-08 Thread Brian Warner
On 9/7/10 6:29 PM, Greg Troxel wrote: > > I built 1.8.0c3 on netbsd-5/i386 (via an updated pkgsrc entry not yet > committed) and it seems to work. 'tahoe check --raw' feels faster than > the previous beta and 1.7.1. > > tahoe manifest seems to work reasonably quickly. Hrm, if your directories

Re: [tahoe-dev] proposal for an HTTP-based storage protocol

2010-09-28 Thread Brian Warner
>>> Do we actually need server-side verification of data? We already let >>> clients upload whatever they want to servers, as long as it's >>> properly formatted as a share. >> >> Yes, but they can't upload something that looks like a share of file >> A but actually contains some other content (un

[tahoe-dev] Tahoe-LAFS source now available on GitHub

2010-10-15 Thread Brian Warner
For about 18 months, I've been running a personal bidirectional git<->darcs bridge, and doing all of my own Tahoe development in git. Last night I pushed a copy of this git repository up to GitHub: http://github.com/warner/tahoe-lafs The "master" branch in that repo will exactly follow our cano

[tahoe-dev] making Tahoe nicer to hack on

2010-10-17 Thread Brian Warner
A little while ago, someone at Mozilla did a short presentation on "Firefox Papercuts", based upon looking at hundreds of vague complaints thrown out to the surprisingly-caring-but-still-not-a-support-channel winds of "#firefox" on Twitter (in particular, things that were annoying enough to compla

Re: [tahoe-dev] Authentication issues in Tahoe

2010-10-18 Thread Brian Warner
On 10/18/10 9:25 AM, Ravi Pinjala wrote: > Yes, exactly. The possession of the file cap *is* the authorization. > (As far as whether there are any plans to implement a more traditional > authentication method, like username/password: I don't think so, but > I'll defer to the much-more-experienced p

Re: [tahoe-dev] Against DARCS - using the patch cache

2010-10-18 Thread Brian Warner
> On 2010-10-18 3:32 PM, Brian Warner wrote: >> - darcs is slow, hard to publish branches, hard to use local >> branches, Zooko just pointed me at the following tidbit: http://darcs.net/manual/Best_practices.html#SECTION0053 to turn on the global p

Re: [tahoe-dev] making Tahoe nicer to hack on

2010-10-18 Thread Brian Warner
On 10/18/10 12:00 AM, Ravi Pinjala wrote: > > I'd even suggest that what you're suggesting for downloading the > dependencies is too much. The standard for *nix systems (in other > words, what users expect) is for the package to fail-fast at build > time if anything's missing, and let the package

Re: [tahoe-dev] Installation "hangs" at Trac

2010-10-19 Thread Brian Warner
On 10/19/10 7:15 PM, Siddhartha Kasivajhula wrote: > Reading http://pypi.python.org/simple/foolscap/ > Reading http://foolscap.lothar.com/trac > > > ...and then it just gets stuck here. It's been at this step for about 30 > minutes now with no developments. I went to foolscap.lothar.com >

Re: [tahoe-dev] Downloading file range

2010-10-20 Thread Brian Warner
On 10/20/10 5:53 AM, slush wrote: > Hi, > > I found that TahoeLAFS solves support for "Range" header with > downloading whole file from grid and then returning a subset of original > file. Is that because downloading just of small piece of file is > impossible in Tahoe by it's design or just becau

Re: [tahoe-dev] Debugging memory leaks with guppy

2010-10-22 Thread Brian Warner
On 10/21/10 5:13 PM, Francois Deppierraz wrote: > Then, I deleted about 1000 very small files from a single directory > and had a look at how many objects of each type were created. Wow, this is awesome. > 3: a3 [-] 1 allmydata.util.dictutil.DictOfSets: 0x675a0c0 > 4: a4 -- [-] 1 dic

Re: [tahoe-dev] Debugging memory leaks with guppy

2010-10-26 Thread Brian Warner
rc = hp.heap()[0].bysize[0].byid[0].rp[5].theone verinfos = set([verinfo for (verinfo,shnum) in rc.cache.keys()]) len(verinfos) > 1 > > So there's only one 'verinfo' value there. The size of the cache repr() > is indeed pretty big. Huh, that knocks out my theory that ResponseC

Re: [tahoe-dev] Help adjusting buildbot after upgrade?

2010-11-10 Thread Brian Warner
On 11/10/10 8:05 AM, Kyle Markley wrote: > On Wed, 10 Nov 2010 07:53:46 -0500, Greg Troxel wrote: >> I think you have a good point about symlinks. Probably you should file >> a ticket so this doesn't get lost. It isn't immediately obvious how >> they should work, especially because tahoe does no

Re: [tahoe-dev] Real-world Tahoe-LAFS grid deployment

2010-11-15 Thread Brian Warner
On 11/14/10 5:48 PM, Nathan Eisenberg wrote: > > No - no RAID - same disk approach as if you were running 24 nodes - > except the node is configured to use /mnt/disk1, /mnt/disk2, > /mnt/disk3, etc, instead of having a node process for /mnt/disk1, > /mnt/disk2, /mnt/disk3, etc. > > In this way, a

Re: [tahoe-dev] Real-world Tahoe-LAFS grid deployment

2010-11-16 Thread Brian Warner
On 11/15/10 4:00 PM, Nathan Eisenberg wrote: >> But you want to see more than one share per machine, right? One >> machine, one tahoe node, but up to 24 shares land there, yeah? > > Nope - one machine, one tahoe node, and one share. At least in grids > with >10 servers, this makes a lot of sense

Re: [tahoe-dev] Tahoe-LAFS v1.8 planning / Administrivia / Big Picture

2010-11-16 Thread Brian Warner
On 11/15/10 3:02 AM, Francois Deppierraz wrote: > > 1-of-3 58 > 2-of-3 63 > 3-of-3 58 > 10-of-30 210 > 20-of-60 394 Hm. Just before 1.8.0, I was using the JS/Protovis -based download timeline visualization tools (which didn't get landed) to investigate the overhead of large k (i.e.

Re: [tahoe-dev] Issue with Unplugging One Node from private grid

2010-11-24 Thread Brian Warner
On 11/24/10 3:32 PM, Greg Troxel wrote: > > Bostonian writes: > >> I am experiencing an issue with unplugging one node from my private >> grid. Here is my testbed: >> >> 1) 5 storage nodes >> 2) N=5, K=3 >> 3) two clients running on my laptops >> >> It has been working fine for normal operations

Re: [tahoe-dev] Collaboration

2010-11-28 Thread Brian Warner
On 11/28/10 1:15 AM, Ted Rolle Jr. wrote: > Alice writes a book; Bob typesets it for her. Can they collaborate on > this book, or is the book only available to the uploader? Sure they can collaborate. If she uploads the book as a mutable file, or if she uploads it as an immutable file but inside

Re: [tahoe-dev] Issue with Unplugging One Node from private grid

2010-11-29 Thread Brian Warner
On 11/29/10 11:41 AM, Bostonian wrote: > Based on these tests, here are my guesses: > > i) when stopping tahoe, it pro-actively notifies introducer about its >leave. > > ii) when unplugging the cable, introducer does not know that one node > leaves. As a result, it still tries to connect

Re: [tahoe-dev] Help uploading when file exists but needs repair

2010-11-30 Thread Brian Warner
On 11/30/10 10:06 PM, Kyle Markley wrote: > > I have a fun problem and need some advice. Yay fun problems! :) > I'm having trouble uploading an immutable file (via "tahoe backup") > because the file already exists on the grid, but is unhealthy > (UploadUnhappinessError). Huh? Shouldn't the new

Re: [tahoe-dev] Help uploading when file exists but needs repair

2010-12-01 Thread Brian Warner
On 11/30/10 10:58 PM, Kyle Markley wrote: > Brian et al, > >> Huh? Shouldn't the new upload just put new shares in place? I know >> our uploader isn't particularly clever in the face of existing shares >> (it will put multiple shares on one server, and in general not >> achieve the ideal diversity

Re: [tahoe-dev] Help uploading when file exists but needs repair

2010-12-04 Thread Brian Warner
On 12/4/10 12:42 AM, Kyle Markley wrote: > > Thanks for looking at this, everyone. I'm happy that the problem is > already known, but sad there isn't a fix yet. I'm going to work around > this by recreating my grid and this time keeping expiration turned > off. I expect that will prevent these sce

Re: [tahoe-dev] Can Tahoe works well in this scenario?

2010-12-05 Thread Brian Warner
On 12/5/10 2:47 AM, Shu Lin wrote: > Putting all nodes behind a NAT with a strict firewall and no manual > config just isn't going to work. > > I don't think Tahoe has any restrict on deployment environment. Having a > server accessed through a public IP doesn't mean I can't put the serve

Re: [tahoe-dev] Automatic rebalancing

2010-12-05 Thread Brian Warner
On 12/5/10 3:52 PM, James A. Donald wrote: > > A centralized coordinator is single point of failure and an additional > configuration issue. If everyone runs the same algorithm, they will > mostly agree without need for a central coordinator - though there > will never be 100% agreement. If the sy

Re: [tahoe-dev] Can Tahoe works well in this scenario?

2010-12-05 Thread Brian Warner
On 12/5/10 7:02 PM, Greg Troxel wrote: > > There is bulk data, and thus we need TCP-friendly congestion control. > So moving away from TCP requires reinventing it. Excellent point. uTP has a lot of congestion-management code in it, but is designed to always yield to TCP, so it'd be good for backg

Re: [tahoe-dev] Can Tahoe works well in this scenario?

2010-12-05 Thread Brian Warner
On 12/5/10 7:21 PM, Greg Troxel wrote: > > Brian Warner writes: > >> Oh, using socks as a *listening* proxy, instead of a connecting proxy? >> Interesting, I'd completely forgotten about that feature. > > My hidden agenda is tor hidden service support :-)

[tahoe-dev] Accounting, 2010 edition

2010-12-19 Thread Brian Warner
It's December, which means it's time to talk about Accounting again[1]. Each time we cycle around this topic, we chip away at the complexity, prune back some of the loftier goals, haggle for a couple of weeks, then throw up our hands and go back to our day jobs for another couple months. This ti

Re: [tahoe-dev] Accounting, 2010 edition

2010-12-20 Thread Brian Warner
On 12/20/10 9:38 AM, Shawn Willden wrote: > On Mon, Dec 20, 2010 at 10:35 AM, Ravi Pinjala > wrote: > > One thing I can't figure out: how do you handle losing your caps > for a share? In that case, it'd still count against your storage > limits, but you would

Re: [tahoe-dev] Accounting, 2010 edition

2010-12-20 Thread Brian Warner
On 12/20/10 7:53 AM, Greg Troxel wrote: > > As a concrete situation, it would be nice to target a friendnet grid > where we aren't particularly worried about malicious attempts to > cheat, but we do want to be able to observe how much space is person > is taking up. > > That means we can leave ou

Re: [tahoe-dev] Accounting, 2010 edition

2010-12-20 Thread Brian Warner
On 12/20/10 3:42 PM, Greg Troxel wrote: > > Even if I could do a 'tahoe show-usage' and get something that is > > blocks pubkey > > and then a 'tahoe remove-all offending-pubkey' as a server admin that > would be a great start. Yeah, that should be part of Phase 1. I've been thinking of a web

Re: [tahoe-dev] Accounting, 2010 edition

2010-12-21 Thread Brian Warner
On 12/21/10 5:56 AM, Greg Troxel wrote: > > and avoid reinventing the trust management wheel > > Your comment made me realize more crisply that the real property I want > From pgp is to be able to manage keys via pgp and then easily insert > them into tahoe. I really do mean "manage via and inse

Re: [tahoe-dev] Accounting, 2010 edition

2010-12-21 Thread Brian Warner
On 12/20/10 7:01 PM, David-Sarah Hopwood wrote: > On 2010-12-20 22:16, Brian Warner wrote: >> The other API would be used in a similar way, right after you build a >> manifest, but it would mean "immediately cancel all my leases on >> shares that weren't in the man

Re: [tahoe-dev] uploading unhappiness error messages

2011-01-04 Thread Brian Warner
On 1/4/11 9:36 AM, Greg Troxel wrote: > > If the server status page had information about which servers had > refused to take shares (and the smallest size that was refused) > recently, that would help people figure things out. Right now the only > way to tell that servers are full is to get upl

Re: [tahoe-dev] UnhappinessError during renew

2011-01-05 Thread Brian Warner
On 1/5/11 6:26 AM, slush wrote: > Hello, > > after few weeks I checked logs of my storage repairs and found, that > process is permanently throwing UnhappinessError. That means I probably > lost some of my data, right? Is here some way how to fix it / skip error > and let repairer to renew other f

[tahoe-dev] new foolscap version requirement

2011-01-05 Thread Brian Warner
> Author: david-sarah > Date: Thu Dec 30 22:00:39 2010 -0800 > > Update foolscap version requirement to 0.6.0, to address > http://foolscap.lothar.com/trac/ticket/167 > > - "foolscap[secure_connections] >= 0.5.1", > + # foolscap < 0.6 is incompatible

Re: [tahoe-dev] FAQ? - What happens if I loose the Tahoe-LAFS gateway machine? production ready?

2011-01-05 Thread Brian Warner
On 1/5/11 12:56 PM, Carsten Krüger wrote: > maybe it's a FAQ but I didn't find an answer on the website. Excellent questions! Yeah, we should definitely add these to the FAQ. > What happens if I loose the gateway? Did the informations on the > storage servers are sufficent to "rebuild" the hole

Re: [tahoe-dev] FAQ? - What happens if I loose the Tahoe-LAFS gateway machine? production ready?

2011-01-05 Thread Brian Warner
On 1/5/11 2:26 PM, Carsten Krüger wrote: > >> What matters most is the filecap, dircap, or "rootcap" under which >> you stored your data. You must retain access to that string. > > This is only a small amount of data that never changes? Right. Think of it like a URL that points to a whole site f

Re: [tahoe-dev] new foolscap version requirement

2011-01-05 Thread Brian Warner
On 1/5/11 5:15 PM, David-Sarah Hopwood wrote: > > Unfortunately, there is no way that I know of to declare a conditional > dependency on 'foolscap[secure_connections] >= 0.6.0' if the version > of Twisted we are using is 10.2. Yup. I'm just venting :). > 'setup.py build' should download a foolsc

[tahoe-dev] upcoming 1.8.2 release: freeze in one week!

2011-01-09 Thread Brian Warner
Greetings Tahoe-LAFS developers! I've agreed to be the release manager for the upcoming 1.8.2 . This will be a pretty small release: the main goal is to fix the somewhat embarassing incompatibility with the recent Twisted-10.2 release (ticket #1286). We're also aiming to get in a few other small f

Re: [tahoe-dev] How do lease renewal and repair work?

2011-01-11 Thread Brian Warner
On 1/11/11 7:19 AM, Shawn Willden wrote: > Specifically, what I'm wondering what happens if a client running a > deep-check --repair --add-lease tries to add a lease on an existing > share and the storage server refuses the lease renewal? That part of the code needs some work, on both sides. At p

Re: [tahoe-dev] Capability of a file could be changed in deep-copy??

2011-01-16 Thread Brian Warner
On 1/16/11 8:39 PM, Shawn Willden wrote: > Removing the image folder doesn't help, either, because even without the > directory node, Bob could have saved the caps of the files themselves. > The only way for Alice to make them inaccessible to Bob is to wait > until expiration removes the shares o

[tahoe-dev] the release train is rolling!

2011-01-17 Thread Brian Warner
The weekend Tahoe Review Party is rolling along, cranking out work on the upcoming 1.8.2 release. The target code freeze is tomorrow night (monday, say around 11pm PST). We're down to 12 tickets left to resolve, all of which are getting really close: packaging: #585 bbfreeze #668 easy_inst

Re: [tahoe-dev] small file

2011-01-17 Thread Brian Warner
On 1/17/11 11:37 AM, medhioub wrote: > Hello, > > Any news about the management of small files with Tahoe. > I give it a try few months ago (and i like it so much) but unfortunately > with bad performance and few problems using small files. We replaced the immutable-file downloader in version 1.8

Re: [tahoe-dev] backup, revision control

2011-01-17 Thread Brian Warner
On 1/16/11 4:53 AM, Greg Troxel wrote: > > Command line tools for tahoe are less functional than WUI, so it's > too tempting to use the WUI, which means firefox/etc. handles caps, > which is obviously unsafe. Getting to the point where I don't want > to use the WUI beyond seeing server sta

Re: [tahoe-dev] Restricting storage server network interface

2011-01-20 Thread Brian Warner
On 1/19/11 4:53 PM, Jody Harris wrote: > I think so. > > open .tahoe/tahoe.cfg and look at the tublocation line. > > Take out the IP:Ports you don't want to advertise. In particular, make tub.location look like "host:port,host:port" for whatever you want to advertise. If tub.location is missing,

[tahoe-dev] release progress: almost frozen, release pushed back one week

2011-01-21 Thread Brian Warner
According to the 1.8.2 Milestone[1], we've got just two bugs left open. #668 is probably code-complete, but it sounds like it's waiting for a manual test before we can consider it closed. #585 is waiting one more patch to deal with .tac files. As soon as we close those two bugs, hopefully within t

[tahoe-dev] Tree frozen! Begin your (testing) engines!

2011-01-23 Thread Brian Warner
Ok, the last expected code changes have been committed, so I'm officially closing the tree until we get 1.8.2 out the door. All checkins must be approved by the release manager (me). I'm currently aware of three remaining issues that block or appear to block the release: #668: work is complete,

[tahoe-dev] _darcs-centric setup.py (was: Tree frozen! Begin your (testing) engines!)

2011-01-24 Thread Brian Warner
> On Mon, Jan 24, 2011 at 9:27 AM, Jimmy Tang wrote: >> >> I just pulled the latest update from the git mirror of tahoe >> (git://github.com/warner/tahoe-lafs.git) >> >> Some of the tests fail >> >> [FAIL]: allmydata.test.test_client.Basic.test_versions >> [FAIL]: allmydata.test.test_runner.BinTah

[tahoe-dev] #467 static-server-selection UI (was: web "control panel", static server selection UI)

2011-01-24 Thread Brian Warner
On 1/24/11 1:49 PM, Shawn Willden wrote: > On Mon, Jan 24, 2011 at 2:23 PM, Greg Troxel > wrote: > GT> A key property is that with some churn things still work. Once GT> you add 'required', that breaks. > SW> I don't see a problem with that at all. If you don't wa

Re: [tahoe-dev] Uploading huge files

2011-01-25 Thread Brian Warner
On 1/25/11 12:40 AM, Michael Coppola wrote: > Hey devs, > > I seem to be having lots of trouble uploading large (as in, 8gb) files > to my Tahoe-LAFS network through the FTP interface. I set Filezilla to > its highest timeout, seconds, and it still hasn't finished > processing the file by the

Re: [tahoe-dev] web "control panel"

2011-01-25 Thread Brian Warner
On 1/24/11 7:43 PM, Chris Palmer wrote: > To avoid the $SECRET-in-URL leaking problem, put $SECRET in a hidden > form field that is sent to the server in POST requests to update the > configuration, rather than in a leakable URL. (Secrets don't belong in > names, no matter how much you want them to

Re: [tahoe-dev] #467 static-server-selection UI

2011-01-25 Thread Brian Warner
On 1/24/11 8:36 PM, Chris Palmer wrote: > Brian Warner writes: > >> Our assumption is that server reliability (and the user's opinion of >> it) is a fuzzy concept that the Tahoe client node cannot figure out >> on its own. > > As usual, I must disagree. :) Awe

Re: [tahoe-dev] web "control panel"

2011-01-25 Thread Brian Warner
>> Also, how does the "standard solution" deal with GETs? > > You can put the secret parameter in the URL query string, thus > defeating the porpoise. > > More to the point, GETs are supposed to be idempotent and safe. Updating > your server's configuration does not fall into that category. Use o

[tahoe-dev] release progress: b1 tarball available

2011-01-25 Thread Brian Warner
I've just pushed the 1.8.2b1 tag for the first beta snapshot, including an updated NEWS file but not yet updating the release notes. We're having some problems with the automatic tarball builder (#1335), but I've just manually copied a .tar.bz2 and .zip up to the web site: http://tahoe-lafs.org/s

Re: [tahoe-dev] release progress: b1 tarball available

2011-01-26 Thread Brian Warner
On 1/26/11 1:22 AM, Jimmy Tang wrote: > > I've just did my usual build and test, i prefer to run tahoe from > where i built it from usually, anyway here are the notes... archlinux:green OS-X 10.6.6: green winxp sp3 py27 mingw: green Excellent! Thanks for the report! -Brian

Re: [tahoe-dev] #467 static-server-selection UI

2011-01-27 Thread Brian Warner
>> Now, we have a situtation that normal users, expert users, and >> implementers can all understand. All the words have their colloquial >> meanings: status, green, yellow, red, 1, 5, overkill, cost, >> reliability. > > I *really* like that. It's like how the Dropbox UI works: as long as the > ic

Re: [tahoe-dev] what's the plan for "tahoe.exe"?

2011-01-27 Thread Brian Warner
On 1/27/11 11:10 AM, Zooko O'Whielacronx wrote: > Therefore there are now two good ways for Windows users to run > Tahoe-LAFS. Just so that I'm clear, are those two good ways the following?: 1: (quickstart): download+unpack .zip, 'python setup.py build', ./bin/tahoe 2: download .exe, ./tah

Re: [tahoe-dev] 0.8.2b1 - new/wrong dependencies

2011-01-29 Thread Brian Warner
On 1/29/11 12:16 PM, Greg Troxel wrote: > > David-Sarah Hopwood writes: > >> [make check, test documentation] > > Thanks - I've hooked in make test to pkgsrc, which amusingly enough lets > users run 'make test' because 'make check' in a pkg checks packaging things. > >>> That then got some fai

[tahoe-dev] Tahoe-LAFS-1.8.2 Released!

2011-01-30 Thread Brian Warner
. Thank you very much to the team of "hackers in the public interest" who make Tahoe-LAFS possible. Brian Warner on behalf of the Tahoe-LAFS team January 30, 2011 San Francisco, California, USA [1] http://tahoe-lafs.org/trac/tahoe/browser/relnotes.txt?rev=4865 [2] http://tahoe-lafs.org/

[tahoe-dev] tree thawed for post-1.8.2 development

2011-01-31 Thread Brian Warner
I haven't heard any panic-stricken messages about 1.8.2 being completely broken, so I don't think we need to worry about an immediate 1.8.3. Thus, I'm un-freezing the tree: new work can land once more. If something goes wonky, I'm sure we can figure something out. So what's on the plate for the n

Re: [tahoe-dev] Tahoe-LAFS is widely misunderstood

2011-02-02 Thread Brian Warner
On 2/1/11 5:36 PM, Greg Troxel wrote: > > After looking at your Octavio page, I think your simplifications > relative to tahoe can be divided into several groups > > no erasure coding; just replication. This gives up some performance, > but seems like a reasonable tradeoff for many. Note tha

Re: [tahoe-dev] dealing with NAT

2011-02-03 Thread Brian Warner
On 2/2/11 8:44 AM, Greg Troxel wrote: > > Most of the nodes on VolunteerGrid2 are set up behind routers with > port-forwarding to the nodes. With the exception of one 6+ year old > router, it is working we haven't experienced any problems. (The > router in question is scheduled for replace

Re: [tahoe-dev] problems running some of the tests on archlinux after an update

2011-02-03 Thread Brian Warner
On 2/3/11 1:24 PM, Jimmy Tang wrote: > > File > "/home/jtang/tmp/tahoe-lafs/src/allmydata-tahoe-1.8.2/src/allmydata/__init__.py", > line 301, in cross_check_pkg_resources_versus_import > % (imp_ver, name, imp_loc, pr_ver, pr_loc, e.__class__.name, e)) > AttributeError: type object 'exceptio

  1   2   3   4   5   >