[Monotone-devel] Version Control System Review
I found this an interesting read. It's a fairly unbiased (maybe) review of most modern version control systems, and even some benchmarks in the part 3... http://ldn.linuxfoundation.org/article/dvcs-round-one-system-rule-them-all-part-1 http://ldn.linuxfoundation.org/article/dvcs-roundup-one-system-rule-them-all-part-2 http://ldn.linuxfoundation.org/article/dvcs-round-one-system-rule-them-all-part-3 I'm sad to see monotone do so badly no the benchmark page, it's still wickedly awesome. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
[Monotone-devel] list branches on server?
Is there a way to list the branches in a database on a server, without downloading the whole database? That would be useful in deciding whether to download the whole database, or just part of it. I guess if the server is running ViewMTN, that would give this info? -- -- Stephe ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 09:44 -0400, Stephen Leake wrote: Is there a way to list the branches in a database on a server, without downloading the whole database? Not yet, that happens after we move to ssl transport and enable 'automate stdio' over the network. That would be useful in deciding whether to download the whole database, or just part of it. I guess if the server is running ViewMTN, that would give this info? Yes. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
Stephen Leake schrieb: Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 09:44 -0400, Stephen Leake wrote: Is there a way to list the branches in a database on a server, without downloading the whole database? Not yet, that happens after we move to ssl transport and enable 'automate stdio' over the network. That makes sense. How is that going? I have some time to spend; can I help in some way? One of the currently unresolved issues I can think of here is the security model which has to be applied. For now the `automate` interface has no security model at all, thus you can do everything on the database as soon as you get access to a running instance. Maybe we should get an idea of how to manage security here first? Thomas. -- GPG-Key 0x160D1092 | tommyd3...@jabber.ccc.de | http://thomaskeller.biz Please note that according to the EU law on data retention, information on every electronic information exchange might be retained for a period of six months or longer: http://www.vorratsdatenspeicherung.de/?lang=en signature.asc Description: OpenPGP digital signature ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 19:13 +0200, Thomas Keller wrote: Stephen Leake schrieb: Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 09:44 -0400, Stephen Leake wrote: Is there a way to list the branches in a database on a server, without downloading the whole database? Not yet, that happens after we move to ssl transport and enable 'automate stdio' over the network. That makes sense. How is that going? I have some time to spend; can I help in some way? One of the currently unresolved issues I can think of here is the security model which has to be applied. For now the `automate` interface has no security model at all, thus you can do everything on the database as soon as you get access to a running instance. Maybe we should get an idea of how to manage security here first? Use people's monotone keys as (self-signed) client certificates (assuming it doesn't make sense for there not to be a way to retrieve the key info from the ssl layer), and add a lua hook get_automate_command_permitted(command_name, key_info) that's checked when anyone tries to an automate command from the network. So everything would be controlled by lua hooks, based on the key the client uses. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 12:31 -0400, Stephen Leake wrote: Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 09:44 -0400, Stephen Leake wrote: Is there a way to list the branches in a database on a server, without downloading the whole database? Not yet, that happens after we move to ssl transport and enable 'automate stdio' over the network. That makes sense. How is that going? I have some time to spend; can I help in some way? I think dscherger is looking into using boost::asio (net.venge.monotone.asio), which includes ssl support (I think including client certificates, which we need) but would take us back to linking against boost libraries again. I have a branch net.venge.monotone.tbrownaw.serve_automate that makes 'automate stdio' available on an alternate network port. There might be something in it that could be cannibalized, but we likely want everything served over one port so it probably wouldn't be useful directly. The 'hello' sequence would need to be changed a bit to allow for picking a particular service (netsync, 'automate stdio', possibly others). The parts dealing with authentication could go away, since that would be replaced by the ssl layer. Moving to ssl/tls would break things like usher, so we need to make sure that whatever library we use supports Server Name Indication[1] (don't know if asio does this). I suppose the thing to do would be to first migrate to asio (or some other library) with the current netsync, and then once that's working turn on the ssl parts and strip out the now-redundant hmac and authentication stuff and check against the ssl client certificate instead. So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? [1] http://en.wikipedia.org/wiki/Server_Name_Indication I'd also like to get 'mtn sync file:' working on Win32; last I checked, that meant replacing netxx with boost::asio. Is that affected by the ssl transport change? My understanding is that they would go together very nicely. Which has a better chance of getting to main first? It sounds like they might be arriving together. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 19:13 +0200, Thomas Keller wrote: Stephen Leake schrieb: Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 09:44 -0400, Stephen Leake wrote: Is there a way to list the branches in a database on a server, without downloading the whole database? Not yet, that happens after we move to ssl transport and enable 'automate stdio' over the network. That makes sense. How is that going? I have some time to spend; can I help in some way? One of the currently unresolved issues I can think of here is the security model which has to be applied. For now the `automate` interface has no security model at all, thus you can do everything on the database as soon as you get access to a running instance. Maybe we should get an idea of how to manage security here first? Use people's monotone keys as (self-signed) client certificates (assuming it doesn't make sense for there not to be a way to retrieve the key info from the ssl layer), and add a lua hook get_automate_command_permitted(command_name, key_info) that's checked when anyone tries to an automate command from the network. So everything would be controlled by lua hooks, based on the key the client uses. And the default hook returns false, so all permissions have to be explicitly added? And that hook has to run on the server, not the client. We could leave it up to the hook to get the key_info, assuming there are Lua functions to do that. It could alternately just use the ssl login name, or something else. But I guess the key_info is the best form of user identification? On the other hand, ssl already has its own access control. Do we really need another layer? I guess we do; if there are several monotone databases on a machine, they might want different permissions, while ssl just grants access to the machine. Where do we write down these design decisions? -- -- Stephe ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
Timothy Brownawell tbrow...@prjek.net writes: On Sat, 2009-08-22 at 12:31 -0400, Stephen Leake wrote: Timothy Brownawell tbrow...@prjek.net writes: I suppose the thing to do would be to first migrate to asio (or some other library) with the current netsync, and then once that's working turn on the ssl parts and strip out the now-redundant hmac and authentication stuff and check against the ssl client certificate instead. Ok. Building that branch on Debian dies in netsync.cc on: #include netxx/address.h #include netxx/peer.h #include netxx/probe.h #include netxx/socket.h #include netxx/sockopt.h #include netxx/stream.h #include netxx/streamserver.h #include netxx/timeout.h #include netxx_pipe.hh so apparently we need to edit netsync to use asio. So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? Do we have a list of such problems? Maybe we can just assume boost got better :). configure says mtn requires asio 1.2; Debian 5.0 (lenny stable) has only asio 1.1.1. sid unstable has asio 1.4.1; that might be promoted to stable before we get done with this :). -- -- Stephe ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 14:40 -0400, Stephen Leake wrote: Timothy Brownawell tbrow...@prjek.net writes: So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? Do we have a list of such problems? Maybe we can just assume boost got better :). The two that come to mind are * different (and therefore annoying) build system * version skew wrt libstdc++, eg boost and monotone have different ideas of what exactly an std::string looks like configure says mtn requires asio 1.2; Debian 5.0 (lenny stable) has only asio 1.1.1. sid unstable has asio 1.4.1; that might be promoted to stable before we get done with this :). Heh. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 14:01 -0400, Stephen Leake wrote:
Timothy Brownawell tbrow...@prjek.net writes:
On Sat, 2009-08-22 at 19:13 +0200, Thomas Keller wrote:
Maybe we should get an idea of how to manage security here first?
Use people's monotone keys as (self-signed) client certificates
(assuming it doesn't make sense for there not to be a way to retrieve
the key info from the ssl layer), and add a lua hook
get_automate_command_permitted(command_name, key_info) that's checked
when anyone tries to an automate command from the network. So everything
would be controlled by lua hooks, based on the key the client uses.
And the default hook returns false, so all permissions have to be
explicitly added?
And that hook has to run on the server, not the client.
Yes.
We could leave it up to the hook to get the key_info, assuming there
are Lua functions to do that. It could alternately just use the ssl
login name, or something else.
But I guess the key_info is the best form of user identification?
On the other hand, ssl already has its own access control. Do we
really need another layer?
I didn't think ssl had login names, just certificates (keys). Are you
thinking ssh here, or are there more features available than I thought?
I guess we do; if there are several monotone databases on a machine,
they might want different permissions, while ssl just grants access to
the machine.
We do need something more detailed than connected vs not connected,
for example like what get_{read,write}_permitted() currently allow.
Where do we write down these design decisions?
I guess it would need a wiki (but how can it be a wiki without an edit
button?) page. I don't see anything there now, besides a brief mention
on the FutureCryptography page.
___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 12:59 PM, Timothy Brownawelltbrow...@prjek.net wrote: So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? Do we have a list of such problems? Maybe we can just assume boost got better :). The two that come to mind are * different (and therefore annoying) build system * version skew wrt libstdc++, eg boost and monotone have different ideas of what exactly an std::string looks like I suppose I should pop back in at this point, since I started the asio branch, and admit that I got stuck. In addition to the above problems, asio has what is IMO a serious design flaw: its I/O channel objects are statically typed. Since we wish to treat sockets, pipes, and whatever-fds-0-and-1-are as interchangeable, this requires a large hairy wrapper around all asio interfaces, which I started but got bogged down on. I'm also not sure asio's Windows support is good enough for us. I've been looking at libevent instead, but it has its own problems, e.g. not handling the creation of a network socket. It's written in C, though, so there's no question of ABI skew, and it uses a normal build system. zw ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
Zack Weinberg za...@panix.com writes: On Sat, Aug 22, 2009 at 12:59 PM, Timothy Brownawelltbrow...@prjek.net wrote: So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? Do we have a list of such problems? Maybe we can just assume boost got better :). The two that come to mind are * different (and therefore annoying) build system * version skew wrt libstdc++, eg boost and monotone have different ideas of what exactly an std::string looks like I suppose I should pop back in at this point, since I started the asio branch, and admit that I got stuck. Can you add an entry in the mtn wiki BranchStatuses page (http://www.monotone.ca/wiki/BranchStatuses/)? In addition to the above problems, asio has what is IMO a serious design flaw: its I/O channel objects are statically typed. Since we wish to treat sockets, pipes, and whatever-fds-0-and-1-are as interchangeable, this requires a large hairy wrapper around all asio interfaces, which I started but got bogged down on. I was just reading thru the asio docs, and wondering where is the base IO object type. I gather the answer is there isn't one! So I agree, that's a design flaw. I'm also not sure asio's Windows support is good enough for us. I'll see if I can build a small example that tests the essentials. I've been looking at libevent instead, but it has its own problems, e.g. not handling the creation of a network socket. It's written in C, though, so there's no question of ABI skew, and it uses a normal build system. That sounds good; I'll see if I can work up a Windows example for that as well. -- -- Stephe ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
Timothy Brownawell tbrow...@prjek.net writes:
On Sat, 2009-08-22 at 14:01 -0400, Stephen Leake wrote:
Timothy Brownawell tbrow...@prjek.net writes:
On the other hand, ssl already has its own access control. Do we
really need another layer?
I didn't think ssl had login names, just certificates (keys). Are you
thinking ssh here, or are there more features available than I thought?
You are right, I'm thinking ssh. I have not used ssl directly.
I guess we do; if there are several monotone databases on a machine,
they might want different permissions, while ssl just grants access to
the machine.
We do need something more detailed than connected vs not connected,
for example like what get_{read,write}_permitted() currently allow.
Where do we write down these design decisions?
I guess it would need a wiki (but how can it be a wiki without an edit
button?) page. I don't see anything there now, besides a brief mention
on the FutureCryptography page.
There's http://www.monotone.ca/wiki/NetsyncTodo/ . We could just add
stuff at the beginning of that.
Or create a new wiki/Netsync2.0 (or 3.0? what version is the current netsync?)
--
-- Stephe
___
Monotone-devel mailing list
Monotone-devel@nongnu.org
http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 11:49 AM, Timothy Brownawell tbrow...@prjek.netwrote: I think dscherger is looking into using boost::asio (net.venge.monotone.asio), which includes ssl support (I think including client certificates, which we need) but would take us back to linking against boost libraries again. I have been looking at this a bit, largely staring at netsync.cc to try and get a better idea of what it's doing though. Note that the net.venge.monotone.asio branch that zack started a while ago does not use boost::asio, but the standalone variant that does not require linking against the boost libraries as far as I can tell. It does seem to need -lpthread on linux though as asio apparently uses threads internally to simulate certain asynchronous operations. See http://blog.think-async.com/2008/05/boostasio-vs-asio.html for more details on the non-boost asio variant. So the question is, what needs to be done on the asio branch? And how can we mitigate the problems people have with linking against boost? Pretty much everything at this point, as far as I can tell. I haven't found the asio docs to be all that great when trying to actually think in detail about how it will fit in to what netsync is doing, but on the surface at least asio seems fairly reasonable. I'd also like to get 'mtn sync file:' working on Win32; last I checked, that meant replacing netxx with boost::asio. Is that affected by the ssl transport change? Another thought on this that I've had floating around for a while is that perhaps rather than starting a second process and running netsync over stdio we could have two separate database instances open and sync between them from within a single process. I haven't looked at this in any detail at all so it might just be a crazy idea, but I think it would avoid all of the windows related network issues. Maybe some of the refactoring that zack and markus did a while ago relating to app_state, options and database arguments in various api's would make the idea of having two open database objects less crazy? Cheers, Derek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 12:40 PM, Stephen Leake stephen_le...@stephe-leake.org wrote: Building that branch on Debian dies in netsync.cc on: #include netxx/address.h #include netxx/peer.h #include netxx/probe.h #include netxx/socket.h #include netxx/sockopt.h #include netxx/stream.h #include netxx/streamserver.h #include netxx/timeout.h #include netxx_pipe.hh so apparently we need to edit netsync to use asio. Yeah, that seems to be the current state of things. ;) configure says mtn requires asio 1.2; Debian 5.0 (lenny stable) has only asio 1.1.1. sid unstable has asio 1.4.1; that might be promoted to stable before we get done with this :). I wondered about this as well. I happen to have boost 1.35.0 installed which has boost::asio version 1.0.0 and I wondered whether there was anything specific in asio 1.2 that we need? In an ideal world it might be nice to be able to use either boost::asio or asio for the cases where someone has one or the other (or both I guess) installed. Cheers, Derek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 1:59 PM, Timothy Brownawell tbrow...@prjek.netwrote: The two that come to mind are * different (and therefore annoying) build system 100% agree, however with plain asio we don't need to pull in boost and its sucky build system so this shouldn't be an issue. * version skew wrt libstdc++, eg boost and monotone have different ideas of what exactly an std::string looks like Fantastic. Can you elaborate on this? I wonder how it's even possible when boost is built with the same libstdc++ as monotone on my machine? Cheers, Derek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, 2009-08-22 at 21:10 -0600, Derek Scherger wrote: On Sat, Aug 22, 2009 at 1:59 PM, Timothy Brownawell tbrow...@prjek.net wrote: * version skew wrt libstdc++, eg boost and monotone have different ideas of what exactly an std::string looks like Fantastic. Can you elaborate on this? I wonder how it's even possible when boost is built with the same libstdc++ as monotone on my machine? Say your distribution is on gcc 3.4 so that's what boost is compiled with, and the binary on our site was compiled with 4.0. Or even if you compile it yourself, but you've installed a later version of gcc than the version that your distro is currently using. This might not currently be an issue with gcc being in the middle of 4.x, I don't know what their rule on incompatible changes is. ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 3:45 PM, Zack Weinberg za...@panix.com wrote: I suppose I should pop back in at this point, since I started the asio branch, and admit that I got stuck. In addition to the above problems, asio has what is IMO a serious design flaw: its I/O channel objects are statically typed. Since we wish to treat sockets, pipes, and whatever-fds-0-and-1-are as interchangeable, this requires a large hairy wrapper around all asio interfaces, which I started but got bogged down on. I'm also not sure asio's Windows support is good enough for us. I've been looking at libevent instead, but it has its own problems, e.g. not handling the creation of a network socket. It's written in C, though, so there's no question of ABI skew, and it uses a normal build system. My impression is that libevent doesn't give us anything in the way of ssl help, while asio does do provide some support and uses openssl under the covers. I vaguely recall that there were/are licensing issues with using openssl directly but maybe that was back in the days of bundling various library sources or something? Cheers, Derek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 9:03 PM, Derek Schergerde...@echologic.com wrote: On Sat, Aug 22, 2009 at 3:45 PM, Zack Weinberg za...@panix.com wrote: My impression is that libevent doesn't give us anything in the way of ssl help, while asio does do provide some support and uses openssl under the covers. I vaguely recall that there were/are licensing issues with using openssl directly but maybe that was back in the days of bundling various library sources or something? I wasn't aware that SSL was on the table, to be honest :) Libevent does not have any SSL implementation. But shouldn't that be done by, or on top of, Botan rather than at a lower level? zw ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 8:02 PM, Derek Schergerde...@echologic.com wrote: I have been looking at this a bit, largely staring at netsync.cc to try and get a better idea of what it's doing though. Note that the net.venge.monotone.asio branch that zack started a while ago does not use boost::asio, but the standalone variant that does not require linking against the boost libraries as far as I can tell. It doesn't need Boost.System, but it does still depend on a few pieces of boost that we're not currently using, notably boost::date_time::posix_time, bleah. It does seem to need -lpthread on linux though as asio apparently uses threads internally to simulate certain asynchronous operations. Yeah, there's not much of an alternative there unless you want to implement your own DNS resolver, which isn't a good idea. [Linux does have getaddrinfo_a, but it's not portable, it may still use threads under the hood, and it reports completion with *signals*. Gag.] Another thought on this that I've had floating around for a while is that perhaps rather than starting a second process and running netsync over stdio we could have two separate database instances open and sync between them from within a single process. I haven't looked at this in any detail at all so it might just be a crazy idea, but I think it would avoid all of the windows related network issues. Maybe some of the refactoring that zack and markus did a while ago relating to app_state, options and database arguments in various api's would make the idea of having two open database objects less crazy? That was one of my goals, in fact. We may not be 100% of the way there yet though. zw ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] Merging branch to allow duplicate key names, have certs use key hash
In message 1250815028.4392.1.ca...@localhost on Thu, 20 Aug 2009 19:37:08 -0500, Timothy Brownawell tbrow...@prjek.net said: tbrownaw On Fri, 2009-08-14 at 05:04 +, Timothy Brownawell wrote: tbrownaw I think branch net.venge.monotone.keys-by-hash is ready now. tbrownaw tbrownaw The central change is that certs contain a key hash tbrownaw instead of a key name, to get rid of the problem with key tbrownaw collisions. tbrownaw tbrownaw This does require a netsync flag day, because certs on the tbrownaw wire contain a key hash instead of a key name now (just tbrownaw like certs in the db). tbrownaw tbrownaw This is merged now. Cool! I wonder, what do you propose for a flag day for our repository? (and lets not forget to announce it clearly on http://monotone.ca/ and to make sure there's ample time for people to adapt, rebuild, yada yada yada) Cheers, Richard -- Richard Levitte rich...@levitte.org http://richard.levitte.org/ Life is a tremendous celebration - and I'm invited! -- from a friend's blog, translated from Swedish ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
Re: [Monotone-devel] list branches on server?
On Sat, Aug 22, 2009 at 11:15 PM, Zack Weinberg za...@panix.com wrote: I wasn't aware that SSL was on the table, to be honest :) Libevent Yeah, what got me thinking about asio was thinking about nuskool and wondering what we might do to support https, particularly on the client side. We could probably get away with not supporting https on the server side by fronting with apache or something. does not have any SSL implementation. But shouldn't that be done by, or on top of, Botan rather than at a lower level? Very possibly yes. I haven't looked at how botan might help us here, if at all. Jack, can you comment? Cheers, Derek ___ Monotone-devel mailing list Monotone-devel@nongnu.org http://lists.nongnu.org/mailman/listinfo/monotone-devel
