Re: [Dnsmasq-discuss] RFC: allowing arbitrarily long options

2018-01-19 Thread Neil Jerram 
Thanks Simon!

On Thu, Jan 18, 2018 at 10:50 PM Simon Kelley 
wrote:

> Patch applied.
>
>
> Cheers,
>
> Simon.
>
> On 18/01/18 14:15, Neil Jerram wrote:
> > On Sun, Jan 14, 2018 at 10:35 PM Simon Kelley  > > wrote:
> >
> > On 14/01/18 18:14, Neil Jerram wrote:
> > > Thanks for looking at this, Simon.  Some thoughts below...
> > >
> > > On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley
> > 
> > > >>
> > wrote:
> > >
> > > I'm in favour, in theory at least, of removing arbitrary
> limits.
> > > Experience has shown that no matter how big, someone,
> > somewhere, will
> > > always find them. The code originally used a fixed buffer
> > which happened
> > > to be unused at that point, to reduce the memory footprint.
> Whilst
> > > dnsmasq is still intended to be "small", small is a relative
> > thing, and
> > > absolutely, rather bigger than it was 15 years ago, so
> > allocating a big
> > > enough buffer is fine.
> > >
> > > In this case, though, as you hint, it's likely that shell
> > limits will
> > > also be a problem. Even eliminating that by using
> > configuration files,
> > > you still have very long lines, which is ugly.
> > >
> > > Can't we solve this problem by allowing repeated interface
> > names, so
> > >
> > > --bridge-interface=eth1,alias-1,alias-2
> > >
> > > becomes identical to
> > >
> > > --bridge-interface=eth1,alias-1
> > > --bridge-interface=eth1,alias-2
> > >
> > > the patch to implement that is probably smaller than your
> > offering.
> > >
> > >
> > > It looks like a nice alternative, but note that it doesn't help at
> all
> > > with any shell limit.  (In fact it would hit any such limit
> > sooner.)  So
> > > I think this is a matter of what you think is more elegant for
> dnsmasq
> > > itself; particularly in the configuration file context where shell
> > > limits don't apply.
> > >
> > >
> > >
> > >
> > > Maybe we should do both?
> > >
> > >
> > > In principle I'm happy to code up and test multiple solutions
> > here; it's
> > > not a large amount of work in any case.  So please do let me know
> what
> > > you would prefer.
> >
> > I'll take both, for preference.
> >
> > Actually - no. You test the long-options patch and I'll take that.
> I'll
> > do the split-into-multiple lines one.
> >
> >
> > I have (or in fact my colleague has) successfully tested this now (with
> > a real OpenStack rig, 90 cirros VMs at the same time on a GCE
> > n1-standard-64 compute node), with no further changes to the patch that
> > I attached before.
> >
> > So I believe the patch is good - please would you consider it for
> merging?
> >
> > Many thanks - Neil
> >
> >
> >
> > Cheers,
> >
> > Simon.
> >
> > >
> > > Best wishes - Neil
> > >
> > >
> > >
> > >
> > > On 07/01/18 14:25, Neil Jerram wrote:
> > > > Calico [1] with OpenStack
> > > >
> > (https://docs.projectcalico.org/v2.6/getting-started/openstack/)
> uses
> > > > dnsmasq with a very long --bridge-interface option:
> > > >
> > > >
> > >
> >
>   --bridge-interface=,,,...,
> > > >
> > > > where each occurrence of "," occupies 15
> > > characters, and
> > > > there can in principle be as many s as you
> > can have VMs
> > > > on a single OpenStack compute host.  Currently an option arg
> is
> > > limited
> > > > in dnsmasq to 1025 chars overall, which only allows for 67
> > > > s, which is not necessarily enough, on a
> powerful
> > > compute
> > > > host.
> > > >
> > > > So I wonder what folk would think about reallocating as
> > necessary to
> > > > allow an option arg to be arbitrarily long?  (Or at least,
> > as long as
> > > > getopt and the containing shell will allow.)  For reference
> I've
> > > > attached a patch that I think would implement that - but I
> > haven't yet
> > > > been able to test it at all, so please don't merge it yet!
> > > >
> > > > Thanks in advance for your thoughts!
> > > >   Neil
> > > >
> > > >
> > > >
> > > > ___
> > > > Dnsmasq-discuss mailing list
> > > > Dnsmasq-discuss@lists.thekelleys.org.uk
> > 
> > >  > >

Re: [Dnsmasq-discuss] RFC: allowing arbitrarily long options

2018-01-18 Thread Simon Kelley 
Patch applied.


Cheers,

Simon.

On 18/01/18 14:15, Neil Jerram wrote:
> On Sun, Jan 14, 2018 at 10:35 PM Simon Kelley  > wrote:
> 
> On 14/01/18 18:14, Neil Jerram wrote:
> > Thanks for looking at this, Simon.  Some thoughts below...
> >  
> > On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley
> 
> > >>
> wrote:
> >
> >     I'm in favour, in theory at least, of removing arbitrary limits.
> >     Experience has shown that no matter how big, someone,
> somewhere, will
> >     always find them. The code originally used a fixed buffer
> which happened
> >     to be unused at that point, to reduce the memory footprint. Whilst
> >     dnsmasq is still intended to be "small", small is a relative
> thing, and
> >     absolutely, rather bigger than it was 15 years ago, so
> allocating a big
> >     enough buffer is fine.
> >
> >     In this case, though, as you hint, it's likely that shell
> limits will
> >     also be a problem. Even eliminating that by using
> configuration files,
> >     you still have very long lines, which is ugly.
> >
> >     Can't we solve this problem by allowing repeated interface
> names, so
> >
> >     --bridge-interface=eth1,alias-1,alias-2
> >
> >     becomes identical to
> >
> >     --bridge-interface=eth1,alias-1
> >     --bridge-interface=eth1,alias-2
> >
> >     the patch to implement that is probably smaller than your
> offering.
> >
> >
> > It looks like a nice alternative, but note that it doesn't help at all
> > with any shell limit.  (In fact it would hit any such limit
> sooner.)  So
> > I think this is a matter of what you think is more elegant for dnsmasq
> > itself; particularly in the configuration file context where shell
> > limits don't apply.
> >  
> >
> >
> >
> >     Maybe we should do both?
> >
> >
> > In principle I'm happy to code up and test multiple solutions
> here; it's
> > not a large amount of work in any case.  So please do let me know what
> > you would prefer.
> 
> I'll take both, for preference.
> 
> Actually - no. You test the long-options patch and I'll take that. I'll
> do the split-into-multiple lines one.
> 
> 
> I have (or in fact my colleague has) successfully tested this now (with
> a real OpenStack rig, 90 cirros VMs at the same time on a GCE
> n1-standard-64 compute node), with no further changes to the patch that
> I attached before.
> 
> So I believe the patch is good - please would you consider it for merging?
> 
> Many thanks - Neil
> 
> 
> 
> Cheers,
> 
> Simon.
> 
> >
> > Best wishes - Neil
> >  
> >
> >
> >
> >     On 07/01/18 14:25, Neil Jerram wrote:
> >     > Calico [1] with OpenStack
> >     >
> (https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses
> >     > dnsmasq with a very long --bridge-interface option:
> >     >
> >     >
> >   
>  --bridge-interface=,,,...,
> >     >
> >     > where each occurrence of "," occupies 15
> >     characters, and
> >     > there can in principle be as many s as you
> can have VMs
> >     > on a single OpenStack compute host.  Currently an option arg is
> >     limited
> >     > in dnsmasq to 1025 chars overall, which only allows for 67
> >     > s, which is not necessarily enough, on a powerful
> >     compute
> >     > host.
> >     >
> >     > So I wonder what folk would think about reallocating as
> necessary to
> >     > allow an option arg to be arbitrarily long?  (Or at least,
> as long as
> >     > getopt and the containing shell will allow.)  For reference I've
> >     > attached a patch that I think would implement that - but I
> haven't yet
> >     > been able to test it at all, so please don't merge it yet!
> >     >
> >     > Thanks in advance for your thoughts!
> >     >       Neil
> >     >
> >     >
> >     >
> >     > ___
> >     > Dnsmasq-discuss mailing list
> >     > Dnsmasq-discuss@lists.thekelleys.org.uk
> 
> >      >
> >     > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> >     >
> >
> >
> >     ___
> >     Dnsmasq-discuss mailing list
> >     Dnsmasq-discuss@lists.thekelleys.org.uk
> 
> >     

Re: [Dnsmasq-discuss] RFC: allowing arbitrarily long options

2018-01-18 Thread Neil Jerram 
On Sun, Jan 14, 2018 at 10:35 PM Simon Kelley 
wrote:

> On 14/01/18 18:14, Neil Jerram wrote:
> > Thanks for looking at this, Simon.  Some thoughts below...
> >
> > On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley  > > wrote:
> >
> > I'm in favour, in theory at least, of removing arbitrary limits.
> > Experience has shown that no matter how big, someone, somewhere, will
> > always find them. The code originally used a fixed buffer which
> happened
> > to be unused at that point, to reduce the memory footprint. Whilst
> > dnsmasq is still intended to be "small", small is a relative thing,
> and
> > absolutely, rather bigger than it was 15 years ago, so allocating a
> big
> > enough buffer is fine.
> >
> > In this case, though, as you hint, it's likely that shell limits will
> > also be a problem. Even eliminating that by using configuration
> files,
> > you still have very long lines, which is ugly.
> >
> > Can't we solve this problem by allowing repeated interface names, so
> >
> > --bridge-interface=eth1,alias-1,alias-2
> >
> > becomes identical to
> >
> > --bridge-interface=eth1,alias-1
> > --bridge-interface=eth1,alias-2
> >
> > the patch to implement that is probably smaller than your offering.
> >
> >
> > It looks like a nice alternative, but note that it doesn't help at all
> > with any shell limit.  (In fact it would hit any such limit sooner.)  So
> > I think this is a matter of what you think is more elegant for dnsmasq
> > itself; particularly in the configuration file context where shell
> > limits don't apply.
> >
> >
> >
> >
> > Maybe we should do both?
> >
> >
> > In principle I'm happy to code up and test multiple solutions here; it's
> > not a large amount of work in any case.  So please do let me know what
> > you would prefer.
>
> I'll take both, for preference.
>
> Actually - no. You test the long-options patch and I'll take that. I'll
> do the split-into-multiple lines one.
>

I have (or in fact my colleague has) successfully tested this now (with a
real OpenStack rig, 90 cirros VMs at the same time on a GCE n1-standard-64
compute node), with no further changes to the patch that I attached before.

So I believe the patch is good - please would you consider it for merging?

Many thanks - Neil


>
> Cheers,
>
> Simon.
>
> >
> > Best wishes - Neil
> >
> >
> >
> >
> > On 07/01/18 14:25, Neil Jerram wrote:
> > > Calico [1] with OpenStack
> > > (https://docs.projectcalico.org/v2.6/getting-started/openstack/)
> uses
> > > dnsmasq with a very long --bridge-interface option:
> > >
> > >
> >
>  --bridge-interface=,,,...,
> > >
> > > where each occurrence of "," occupies 15
> > characters, and
> > > there can in principle be as many s as you can have
> VMs
> > > on a single OpenStack compute host.  Currently an option arg is
> > limited
> > > in dnsmasq to 1025 chars overall, which only allows for 67
> > > s, which is not necessarily enough, on a powerful
> > compute
> > > host.
> > >
> > > So I wonder what folk would think about reallocating as necessary
> to
> > > allow an option arg to be arbitrarily long?  (Or at least, as long
> as
> > > getopt and the containing shell will allow.)  For reference I've
> > > attached a patch that I think would implement that - but I haven't
> yet
> > > been able to test it at all, so please don't merge it yet!
> > >
> > > Thanks in advance for your thoughts!
> > >   Neil
> > >
> > >
> > >
> > > ___
> > > Dnsmasq-discuss mailing list
> > > Dnsmasq-discuss@lists.thekelleys.org.uk
> > 
> > > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> > >
> >
> >
> > ___
> > Dnsmasq-discuss mailing list
> > Dnsmasq-discuss@lists.thekelleys.org.uk
> > 
> > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> >
>
>
___
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Re: [Dnsmasq-discuss] RFC: allowing arbitrarily long options

2018-01-14 Thread Simon Kelley 
On 14/01/18 18:14, Neil Jerram wrote:
> Thanks for looking at this, Simon.  Some thoughts below...
>  
> On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley  > wrote:
> 
> I'm in favour, in theory at least, of removing arbitrary limits.
> Experience has shown that no matter how big, someone, somewhere, will
> always find them. The code originally used a fixed buffer which happened
> to be unused at that point, to reduce the memory footprint. Whilst
> dnsmasq is still intended to be "small", small is a relative thing, and
> absolutely, rather bigger than it was 15 years ago, so allocating a big
> enough buffer is fine.
> 
> In this case, though, as you hint, it's likely that shell limits will
> also be a problem. Even eliminating that by using configuration files,
> you still have very long lines, which is ugly.
> 
> Can't we solve this problem by allowing repeated interface names, so
> 
> --bridge-interface=eth1,alias-1,alias-2
> 
> becomes identical to
> 
> --bridge-interface=eth1,alias-1
> --bridge-interface=eth1,alias-2
> 
> the patch to implement that is probably smaller than your offering.
> 
> 
> It looks like a nice alternative, but note that it doesn't help at all
> with any shell limit.  (In fact it would hit any such limit sooner.)  So
> I think this is a matter of what you think is more elegant for dnsmasq
> itself; particularly in the configuration file context where shell
> limits don't apply.
>  
> 
> 
> 
> Maybe we should do both?
> 
> 
> In principle I'm happy to code up and test multiple solutions here; it's
> not a large amount of work in any case.  So please do let me know what
> you would prefer.

I'll take both, for preference.

Actually - no. You test the long-options patch and I'll take that. I'll
do the split-into-multiple lines one.


Cheers,

Simon.

> 
> Best wishes - Neil
>  
> 
> 
> 
> On 07/01/18 14:25, Neil Jerram wrote:
> > Calico [1] with OpenStack
> > (https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses
> > dnsmasq with a very long --bridge-interface option:
> >
> >
> --bridge-interface=,,,...,
> >
> > where each occurrence of "," occupies 15
> characters, and
> > there can in principle be as many s as you can have VMs
> > on a single OpenStack compute host.  Currently an option arg is
> limited
> > in dnsmasq to 1025 chars overall, which only allows for 67
> > s, which is not necessarily enough, on a powerful
> compute
> > host.
> >
> > So I wonder what folk would think about reallocating as necessary to
> > allow an option arg to be arbitrarily long?  (Or at least, as long as
> > getopt and the containing shell will allow.)  For reference I've
> > attached a patch that I think would implement that - but I haven't yet
> > been able to test it at all, so please don't merge it yet!
> >
> > Thanks in advance for your thoughts!
> >       Neil
> >
> >
> >
> > ___
> > Dnsmasq-discuss mailing list
> > Dnsmasq-discuss@lists.thekelleys.org.uk
> 
> > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> >
> 
> 
> ___
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss@lists.thekelleys.org.uk
> 
> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> 


___
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss

Re: [Dnsmasq-discuss] RFC: allowing arbitrarily long options

2018-01-14 Thread Neil Jerram 
Thanks for looking at this, Simon.  Some thoughts below...

On Sat, Jan 13, 2018 at 5:34 PM Simon Kelley 
wrote:

> I'm in favour, in theory at least, of removing arbitrary limits.
> Experience has shown that no matter how big, someone, somewhere, will
> always find them. The code originally used a fixed buffer which happened
> to be unused at that point, to reduce the memory footprint. Whilst
> dnsmasq is still intended to be "small", small is a relative thing, and
> absolutely, rather bigger than it was 15 years ago, so allocating a big
> enough buffer is fine.
>
> In this case, though, as you hint, it's likely that shell limits will
> also be a problem. Even eliminating that by using configuration files,
> you still have very long lines, which is ugly.
>
> Can't we solve this problem by allowing repeated interface names, so
>
> --bridge-interface=eth1,alias-1,alias-2
>
> becomes identical to
>
> --bridge-interface=eth1,alias-1
> --bridge-interface=eth1,alias-2
>
> the patch to implement that is probably smaller than your offering.
>

It looks like a nice alternative, but note that it doesn't help at all with
any shell limit.  (In fact it would hit any such limit sooner.)  So I think
this is a matter of what you think is more elegant for dnsmasq itself;
particularly in the configuration file context where shell limits don't
apply.


>
>
> Maybe we should do both?
>

In principle I'm happy to code up and test multiple solutions here; it's
not a large amount of work in any case.  So please do let me know what you
would prefer.

Best wishes - Neil


>
>
> On 07/01/18 14:25, Neil Jerram wrote:
> > Calico [1] with OpenStack
> > (https://docs.projectcalico.org/v2.6/getting-started/openstack/) uses
> > dnsmasq with a very long --bridge-interface option:
> >
> > --bridge-interface=,,,...,
> >
> > where each occurrence of "," occupies 15 characters, and
> > there can in principle be as many s as you can have VMs
> > on a single OpenStack compute host.  Currently an option arg is limited
> > in dnsmasq to 1025 chars overall, which only allows for 67
> > s, which is not necessarily enough, on a powerful compute
> > host.
> >
> > So I wonder what folk would think about reallocating as necessary to
> > allow an option arg to be arbitrarily long?  (Or at least, as long as
> > getopt and the containing shell will allow.)  For reference I've
> > attached a patch that I think would implement that - but I haven't yet
> > been able to test it at all, so please don't merge it yet!
> >
> > Thanks in advance for your thoughts!
> >   Neil
> >
> >
> >
> > ___
> > Dnsmasq-discuss mailing list
> > Dnsmasq-discuss@lists.thekelleys.org.uk
> > http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
> >
>
>
> ___
> Dnsmasq-discuss mailing list
> Dnsmasq-discuss@lists.thekelleys.org.uk
> http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss
>
___
Dnsmasq-discuss mailing list
Dnsmasq-discuss@lists.thekelleys.org.uk
http://lists.thekelleys.org.uk/mailman/listinfo/dnsmasq-discuss