Re: Distributed Debian Distribution Development
Hi. Le mercredi 01 septembre 2010 à 17:40 +0100, Luke Kenneth Casson Leighton a écrit : > i leave you with this: the idea of the "Freedom Box" caused quite a > stir at debconf2010, but i honestly doubt that, without any experience > of getting *yourselves* off of the client-server paradigm, there will > be any success in getting *the average person* off of the > client-server paradigm. so i believe that by converting the debian > development process and infrastructure to peer-to-peer distributed > infrastructure *first*, and by creating aggregation and migration > infrastructure for *yourselves*, the people who do that will be in a > much stronger position to help convert other types of internet > services to a distributed architecture. and, will likely have written > a good proportion of the necessary infrastructure along the way. Regarding distributed development tools / forges, I'd suggest to also have a look at developments happening on Savane, which will be rewritten in a way which seems to be inspired by Moglen's vision also : http://2010.rmll.info/A-new-Savane.html (slides 10+) A video recording of the talk will be available soon. Best regards, -- Olivier BERGER http://www-public.it-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8 Ingénieur Recherche - Dept INF Institut TELECOM, SudParis (http://www.it-sudparis.eu/), Evry (France) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1283438322.3825.129.ca...@inf-8657.int-evry.fr
Distributed bugtracker - Was: Re: Distributed Debian Distribution Development
Hi. Le mercredi 01 septembre 2010 à 17:40 +0100, Luke Kenneth Casson Leighton a écrit : > > * again: ikiwiki or something similar could well be used as the basis > for a distributed bugtracker. > I'd suggest you have a look at the SD tool and the incoming debbugs support for it developped by Christine Spang. See for instance a report I've made at [0]. I'm not sure the distribution involved with SD involves basic replication as in P2P systems like bittorrent, but there are strong links. [0] : http://www-public.it-sudparis.eu/~berger_o/weblog/2010/08/12/great-talk-distributed-bug-tracking-for-debian-with-sd-at-debconf/ Best regards, -- Olivier BERGER http://www-public.it-sudparis.eu/~berger_o/ - OpenPGP-Id: 2048R/5819D7E8 Ingénieur Recherche - Dept INF Institut TELECOM, SudParis (http://www.it-sudparis.eu/), Evry (France) -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/1283437047.3825.107.ca...@inf-8657.int-evry.fr
Re: Distributed Debian Distribution Development
On Thu, Sep 2, 2010 at 1:44 PM, Oscar Morante wrote: > Have you seen this project [1]? It looks like they have been already > thinking about the git+bittorrent idea. > > [1] http://code.google.com/p/gittorrent/ yes. it's effectively shelved. the name "gittorrent" was abandoned and the name "mirrorsync" selected, because the people working on it decided that bittorrent was an inappropriate protocol to use. i got them some slashdot coverage. but, ultimately, i disagreed with them that bittorrent isn't an appropriate protocol, so that's why i did the thingy and proved that it's an effective transfer / distribution mechanism. thingy. haven't even picked a name for it! :) suggestions welcome. i want to see how far i can get away with leaving the bittorrent protocol _entirely_ unchanged, by virtue of doing everything through this "vfs layer" jobbie. only if it becomes absolutely absolutely necessary, _then_ start making changes. good reasons to make changes: a) the 256k chunk size becomes blindingly obviously completely unacceptable. for example: cameron dale did a study of the .deb archives and found that a very large percentage are under 32k in size. this was the primary reason why he abandoned the bittorrent protocol (baby? bathwater?? *sigh*...) even after modifying it to be able to negotiate chunk sizes and he designed apt-p2p instead. b) ISPs start doing packet-filtering (fuckers) so it becomes necessary to change headers, port numbers, permanently enable encryption at a fundamental level such that deep packet inspection becomes impossible, e.g. move to SSL and so on. c) digital signing of individual commits becomes necessary, and... somehow (i don't know how yet) it *hand-waving* becomes necessary to integrate the GPG signature verification on git "packs" at the bittorrent-protocol-level. haven't got to that bit, yet - the project's only been going for 4 days! sam vilain solved this by simply creating refs with the signers' 32-bit GPG fingerprint, but he didn't get as far as actually _checking_ it :) l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktikkyhzu_ws7ke3f-ntppdtxy9z6-pgn0pf7m...@mail.gmail.com
Re: Distributed Debian Distribution Development
Have you seen this project [1]? It looks like they have been already thinking about the git+bittorrent idea. [1] http://code.google.com/p/gittorrent/ -- Oscar Morante "Self-education is, I firmly believe, the only kind of education there is." -- Isaac Asimov. signature.asc Description: Digital signature
Re: Distributed Debian Distribution Development
On Wed, Sep 1, 2010 at 6:47 PM, Yaroslav Halchenko wrote: > Just - Wow... thanks! > > Hopefully digesting of this tasty post would not cause too much of farting ;-) :) > seems might be worth adding (if I am not missing the point), then the > concept of "derivatives" would then converge finally to a more > digestible, more manageable, and thus more robust mechanism of > branches... ? ahh, you're still using "git": i'll be doing nothing more fancy than creating a "git-remote-helper" which will add a protocol gitp2p:// or gpp:// or something like that. so you'll still need to understand the concept of branches and branching: it's just that you'd be able to grab them from peers rather than being reliant on a central server. imagine a situation where you meet a fellow debian developer(s) and you both(all) happen to be on holiday or just... somewhere where connectivity is patchy. i envisage a situation where both (or all) publish their own trackers, and they can simply share whatever bits of git repositories they (individually) happened to be "most interested in, personally", with everyone else. for example, one person happens to have an OCD-fueled interest in keeping up-to-date with every single mailing list, whilst another happens to be interested in some obscure piece of software. everybody collaborates, does loads of packages, GPG signs them, commits them to each others' git repositories, they all say bye bye nice meeting you, get on planes or goats and the VERY first person who happens to have internet access does a "git push", wham, everyones' packages are uploaded/available to the rest of the world. including those people who shared them originally, but they have the git commit refs already _in_ their database, so when _they_ get online as well, they automatically become "seeds" as part of the wider git-bittorrent network for those packages. so... yeah. it's a little... radical, but actually nothing more than a mind-shift rather than any actual significant coding. l. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktintf4xjugend0slotu341=bfaxsdrcwo=1wq...@mail.gmail.com
Re: Distributed Debian Distribution Development
I'm glad this was cross posted otherwise I would've missed it. Even if there are technical hurdles it's an exciting idea and I'm looking forward to reading the devel mailing list for follow-ups. The point about eating your own dog food is well made i thought, though whether there is any interest in moving this forward duing a squeeze freeze will be interesting to see. Hopefully this also makes it into the debian news to give it some more reach for discussion. Personally though, as much as I find the idea interesting, I won't be having a torrent application running all the time to assist with the project. Nor would I want to get mail through torrent, or a wiki... I guess us "users" will have a better idea how such things can look once diaspora is alpha released in October - November or so. -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/aanlktimkfn2hn+gzaixn8nyqebz0mvr4s9jzlapus...@mail.gmail.com
Re: Distributed Debian Distribution Development
In , Luke Kenneth Casson Leighton wrote: >around three years ago i wrote an article recommending that debian >move step-by-step towards distributed peer-to-peer infrastructure, >thus reducing the reliance on server infrastructure, thus potentially >allowing sponsorship funds and resources to be retargetted to other >areas which would improve the debian distribution. > >mostly that was words, not actual code, and, before getting all upset >at how little progress has been made (cameron dale's fantastic apt-p2p >work literally being the only exception ) i decided a few days ago to >do something about that situation, so that i wouldn't come across as >being a complete spongeing whining knob. > >my contribution is to prove that a combination of git and bittorrent >is actually possible: >http://gitorious.org/python-libbittorrent/pybtlib > >(anyone wishing to help / contribute, do contact me: i'll happily give >you access to the repository if you can spot the irony of this >statement) > >it's turned out to be much, much simpler than i thought, by the simple >expedient of turning git commits into a "virtual filesystem" where the >git-pack-objects are stored as files, named by their commit reference. > hooking into the bittornado client's "file close" operation is enough >to fire off a test for whether the pack object is valid (check its >signature at the beginning, and the SHA-1 signature at the end), and >if it is, to run a "git unpack" operation. Packs are allowed to have information from multiple related commits and are not required to have all the objects referenced by a single commit. So, right there I'm not sure exactly sure how your virtual file system is structured. It seems to me like if packs are the file(s) in your torrent(s) then you may end up transferring much, much data redundantly. I've not sure it is possible to naively combine the bittorrent protocol with Git, since you really want to advertise individual objects, but you need to transfer in the form of a pack/bundle to avoid intra-object redundancies. It's also too "expensive" to track single objects, but also rather "expensive" to switch clients over to a new torrent. >what are the implications, and why is combining git with bittorrent a >big hairy deal? > >* alioth, a single server, could run git torrent trackers, and the >load on the server greatly reduced. and it no longer becomes a >single-point-of-failure > >* joey hess's excellent "ikiwiki", which can be configured to use git, >could be used as the basis for a distributed wiki. > >* again: ikiwiki or something similar could well be used as the basis >for a distributed bugtracker. While wikis and bug trackers have some ground in common, the information stored by a bug tracker needs to be much more structured to enable the types of queries the release team and others do. Does ikiwiki support the features we are used to on the Debian BTS? >* mailing list archives could be stored in a git repository, and, >instead of being mailed to tens of thousands of users, a slow >migration to simply... sharing the email messages via bittorrent can >be performed. this would *significantly* reduce the load on the mail >servers, which i understand to be ... a leeetle bit stressed. Having the archives available via bittorrent sounds fine. However, if subscribing or receiving mails in "real-time" requires special tools, the idea won't work. The mailing list community is so large partly because of the simplicity of the tools required to sent message to the lists and receive messages from the list; SMTP, NNTP, and POP/IMAP are venerated protocols supported on some of the least expensive devices you'll find. -- Boyd Stephen Smith Jr. ,= ,-_-. =. b...@iguanasuicide.net ((_/)o o(\_)) ICQ: 514984 YM/AIM: DaTwinkDaddy `-'(. .)`-' http://iguanasuicide.net/\_/ signature.asc Description: This is a digitally signed message part.
Re: Distributed Debian Distribution Development
Just - Wow... thanks! Hopefully digesting of this tasty post would not cause too much of farting ;-) seems might be worth adding (if I am not missing the point), then the concept of "derivatives" would then converge finally to a more digestible, more manageable, and thus more robust mechanism of branches... ? On Wed, 01 Sep 2010, Luke Kenneth Casson Leighton wrote: > ghandi said "be the change you want to see in the world". this > translates, roughly, to "eat your own dogfood". > so - something to think about. -- .-. =-- /v\ = Keep in touch// \\ (yoh@|www.)onerussian.com Yaroslav Halchenko /( )\ ICQ#: 60653192 Linux User^^-^^[17] signature.asc Description: Digital signature
