Re: [distcc] load management

2004-05-02 Thread Dan Kegel
[EMAIL PROTECTED] wrote: I've been toying with the following idea, which combines load balancing with fsh-like connection caching. Let's call it lbfsh. I think that would be pretty handy, even if we end up improving the built in balancer. In particular it would let people shift any old task onto

Re: [distcc] load management

2004-05-02 Thread Dan Kegel
Dan Kegel wrote: Hey, come to think of it, maybe ssh-agent should be the local daemon! On second thought, it's bad to add anything to ssh-agent, as it would make it harder to prove that it's secure. But something just like ssh-agent, but for connection caching etc., is probably apropos. -- My

Re: [distcc] load management

2004-05-02 Thread Martin Pool
On 2 May 2004, Dan Kegel [EMAIL PROTECTED] wrote: [EMAIL PROTECTED] wrote: I think the improvement in friendliness makes up for any nervousness. That sounds like the way Microsoft thought when they were designing ActiveX! There's no call for that sort of language. :-) I think what you

Re: [distcc] load management

2004-05-02 Thread Martin Pool
On 3 May 2004, Martin Pool [EMAIL PROTECTED] wrote: http://mah.everybody.org/docs/ssh That reminds me: security measures that are hard to use have the effect of lowering security. There are probably a significant number of people who use password authentication because ssh-agent and co can

Re: [distcc] load management

2004-05-02 Thread Dan Kegel
Martin Pool wrote: That sounds like the way Microsoft thought when they were designing ActiveX! There's no call for that sort of language. :-) Yeah, sorry, I knew I was hitting below the belt! I think what you have is just timidness, rather than well-founded concern. Probably. As long as the

Re: [distcc] load management

2004-05-01 Thread mbp
On 29 Apr 2004, Daniel Kegel [EMAIL PROTECTED] wrote: Here's a scheme that does what you're suggesting in a slightly different way. I've been toying with the following idea, which combines load balancing with fsh-like connection caching. Let's call it lbfsh. I think that would be pretty

Re: [distcc] load management

2004-04-29 Thread Justin Randall
On Tuesday 27 April 2004 06:04 pm, Jake McGuire wrote: I am planning to start working on adding load management enhancements to distcc for use at our company, but wanted to make sure that no one was almost done fixing the problem and also make sure that I didn't spend a bunch of time making my

Re: [distcc] load management

2004-04-29 Thread Daniel Kegel
Martin Pool wrote: Please look at the notes on this in the TODO and protocol-3.txt documents in the source distribution. Unfortunately, protocol-3.txt isn't in the current release, and you don't have a web gateway set up so we can look at the arch repository yet. Is this a clever ploy to get us to

Re: [distcc] load management

2004-04-29 Thread Daniel Kegel
Justin Randall wrote: I am planning to start working on adding load management enhancements ... 3) on startup, distcc determines appropriate server to send request to a) poll all servers in host list - wait until all respond or 0.1 seconds, whichever comes first b) multiply connection count

Re: [distcc] load management

2004-04-29 Thread Martin Pool
On 29 Apr 2004, Daniel Kegel [EMAIL PROTECTED] wrote: Martin Pool wrote: Please look at the notes on this in the TODO and protocol-3.txt documents in the source distribution. Unfortunately, protocol-3.txt isn't in the current release, and you don't have a web gateway set up so we can look

Re: [distcc] load management

2004-04-29 Thread Martin Pool
On 29 Apr 2004, Daniel Kegel [EMAIL PROTECTED] wrote: Martin Pool wrote: Please look at the notes on this in the TODO and protocol-3.txt documents in the source distribution. Unfortunately, protocol-3.txt isn't in the current release, and you don't have a web gateway set up so we can look

Re: [distcc] load management

2004-04-28 Thread Dan Kegel
Martin Pool wrote: I don't think that's needed. In fact, it's probably better if the local server is connected to via a unix domain socket. That's slightly faster and more secure. Also, it lets us do tricky things like passing an open socket from the local server to the distcc program, so the

Re: [distcc] load management

2004-04-28 Thread Martin Pool
On 27 Apr 2004, Dan Kegel [EMAIL PROTECTED] wrote: If fd passing is used, the speed difference might be noticable, since it would completely eliminate one copy and a lot of task switching. Right. But unless there is some reason for traffic to pass through a local daemon you can just skip it

Re: [distcc] load management

2004-04-28 Thread Dan Kegel
Martin Pool wrote: For the particular case of talking to distccd, we could arrange for distccd to keep listening on the socket after a job is complete, which avoids the need to run fshd. We just need something on the client to hold open the fd and pass it to distcc client processes as they need

Re: [distcc] load management

2004-04-28 Thread Martin Pool
On 28 Apr 2004, Dan Kegel [EMAIL PROTECTED] wrote: And then you might want to turn the c preprocessor into a library, to get rid of one more fork+exec. Why not just go all out and announce build.google.com, which builds *anything* in half a second? :) On the other hand, my dual 650 MHz

Re: [distcc] load management

2004-04-28 Thread Dan Kegel
Martin Pool wrote: On 28 Apr 2004, Dan Kegel [EMAIL PROTECTED] wrote: And then you might want to turn the c preprocessor into a library, to get rid of one more fork+exec. Why not just go all out and announce build.google.com, which builds *anything* in half a second? :) Heh. I think that

Re: [distcc] load management

2004-04-28 Thread Martin Pool
Please look at the notes on this in the TODO and protocol-3.txt documents in the source distribution. -- Martin signature.asc Description: Digital signature __ distcc mailing listhttp://distcc.samba.org/ To unsubscribe or change options:

Re: [distcc] load management

2004-04-27 Thread Daniel Kegel
Jake McGuire wrote: I am planning to start working on adding load management enhancements to distcc for use at our company, but wanted to make sure that no one was almost done fixing the problem and also make sure that I didn't spend a bunch of time making my changes and have them rejected. I'm

Re: [distcc] load management

2004-04-27 Thread Wayne Davison
On Tue, Apr 27, 2004 at 04:04:39PM -0700, Jake McGuire wrote: I am planning to start working on adding load management enhancements to distcc for use at our company Here's an idea I had been kicking around for awhile but never got to. All systems that want to make distcc requests need at least

Re: [distcc] load management

2004-04-27 Thread Daniel Kegel
Wayne Davison wrote: Here's an idea I had been kicking around for awhile but never got to. All systems that want to make distcc requests need at least one local server process. This server would be responsible for talking with the other servers that were around (configured in some manner),

Re: [distcc] load management

2004-04-27 Thread Daniel Kegel
Jake McGuire wrote: I don't think blasting all N servers with a UDP packet on each distcc run is a good idea. It depends on the size of your preprocessed file. The one preprocessed file that I looked at from our build was 1.5 megabytes; compared to a thousand TCP packets, plus acks, plus

Re: [distcc] load management

2004-04-27 Thread Martin Pool
On 27 Apr 2004, Daniel Kegel [EMAIL PROTECTED] wrote: I don't think that's needed. In fact, it's probably better if the local server is connected to via a unix domain socket. That's slightly faster and more secure. Also, it lets us do tricky things like passing an open socket from the local

Re: [distcc] load management

2004-04-27 Thread Martin Pool
On 27 Apr 2004, Wayne Davison [EMAIL PROTECTED] wrote: On Tue, Apr 27, 2004 at 04:04:39PM -0700, Jake McGuire wrote: I am planning to start working on adding load management enhancements to distcc for use at our company Here's an idea I had been kicking around for awhile but never got to.

Re: [distcc] load management

2004-04-27 Thread Martin Pool
On 28 Apr 2004, Martin Pool [EMAIL PROTECTED] wrote: (I've been waiting ten years for a reason to use fd passing!) Oh, here's a good place to use it: write a faster and more reliable ssh connection-caching program than fsh. You need an automatically started local daemon, which opens

Re: [distcc] load management

2004-04-27 Thread Martin Pool
On 27 Apr 2004, Jake McGuire [EMAIL PROTECTED] wrote: We're running into issues at my company, possibly related to large numbers of users relative to available CPU slots and the one-second backoff timer. Can you expand on this please? Do you think the client is not finding spare CPU slots