Re: check successful reload using master cli

2020-09-16 Thread William Lallemand
On Tue, Sep 15, 2020 at 01:24:04PM -0300, Joao Morais wrote:
> 
> 
> > Em 15 de set de 2020, à(s) 12:36, William Lallemand 
> >  escreveu:
> > 
> > Oh right... the space in "[was: ]" is troublesome for cutting the string,
> > we must remove it.
> 
> It's not a problem at all when using chunks of fixed size, even if columns 
> differ between them, and the lay out ([was: ...]) remains the same.

Sure, but the spaces are only alignment, it could change in the future,
the real separator are the spaces, and the fields are not supposed to
contain a space or to be empty.

So the parsing should be as easy with awk or cut.
> 
> > Regarding the size of the fields, 16 bytes should be enough but there is
> > the expection of the version field that can be much larger if a dev
> > version with the commit ID in the version is used.
> 
Sorry, I wasn't clear here, I meant if you use a dev version you could
have something like this:

2.0-dev7-0124c9-7

> If changing the spec, please provide also a way to distinguish the
> haproxy version via master cli which is not in the same `show proc`
> command, something like a `show info` or `version`, or at most in a
> fixed position at the start of `show proc`, so I can use it to version
> the output parser.
> 

I only intend to remove the space in the brackets for 2.3, if you want a
parser which works in any case, just consider that the brackets work
like quotes and allow spaces in this field.

I'll add a "show info" on the master CLI, that will be useful.


-- 
William Lallemand



Re: check successful reload using master cli

2020-09-16 Thread William Lallemand
On Tue, Sep 15, 2020 at 06:43:11PM +0100, Jonathan Matthews wrote:
> On Tue, 15 Sep 2020 at 16:42, Tim Düsterhus  wrote:
> 
> > Why not use the Tab (0x09) as separator and make the output a proper TSV?
> 
> 
> I’m only 2% joking when I point out that ASCII already has single-byte
> inter-field and inter-record delimiters defined, just waiting to be used
> ... :-)
> 
> https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/
> 
> http://html-codes.info/ascii/standard/What-is-the-ASCII-value-of-record%20separator_30
> 

The problem with this is that it's not human readable since this is a
non-printable character. I prefer to keep it simple.

-- 
William Lallemand



Re: check successful reload using master cli

2020-09-16 Thread William Lallemand
On Tue, Sep 15, 2020 at 05:39:18PM +0200, Tim Düsterhus wrote:
> William,
> 
> Am 15.09.20 um 17:36 schrieb William Lallemand:
> > Oh right... the space in "[was: ]" is troublesome for cutting the string,
> > we must remove it.
> > 
> 
> Why not use the Tab (0x09) as separator and make the output a proper TSV?

Because tabs don't align correctly if the head column is bigger than the
data column so it's less human readable, we wanted to have an output
which is both human readable and parsable.

-- 
William Lallemand



Re: check successful reload using master cli

2020-09-15 Thread Jonathan Matthews
On Tue, 15 Sep 2020 at 16:42, Tim Düsterhus  wrote:

> Why not use the Tab (0x09) as separator and make the output a proper TSV?


I’m only 2% joking when I point out that ASCII already has single-byte
inter-field and inter-record delimiters defined, just waiting to be used
... :-)

https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/

http://html-codes.info/ascii/standard/What-is-the-ASCII-value-of-record%20separator_30


-- 
Jonathan Matthews
https://jpluscplusm.com


Re: check successful reload using master cli

2020-09-15 Thread Joao Morais



> Em 15 de set de 2020, à(s) 12:36, William Lallemand  
> escreveu:
> 
> Oh right... the space in "[was: ]" is troublesome for cutting the string,
> we must remove it.

It's not a problem at all when using chunks of fixed size, even if columns 
differ between them, and the lay out ([was: ...]) remains the same.

> Regarding the size of the fields, 16 bytes should be enough but there is
> the expection of the version field that can be much larger if a dev
> version with the commit ID in the version is used.

If changing the spec, please provide also a way to distinguish the haproxy 
version via master cli which is not in the same `show proc` command, something 
like a `show info` or `version`, or at most in a fixed position at the start of 
`show proc`, so I can use it to version the output parser.

~jm




Re: check successful reload using master cli

2020-09-15 Thread Tim Düsterhus
William,

Am 15.09.20 um 17:36 schrieb William Lallemand:
> Oh right... the space in "[was: ]" is troublesome for cutting the string,
> we must remove it.
> 

Why not use the Tab (0x09) as separator and make the output a proper TSV?

Best regards
Tim Düsterhus



Re: check successful reload using master cli

2020-09-15 Thread William Lallemand
On Tue, Sep 15, 2020 at 11:52:15AM -0300, Joao Morais wrote:
> 
> 
> > Em 14 de set de 2020, à(s) 19:14, William Lallemand 
> >  escreveu:
> > 
> > Hello,
> > 
> > On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote:
> >> Hello list, I'm working on an automation around haproxy process
> >> lifecycle in master-worker mode. It's working nice but I'm not
> >> confident that all premisses I used are correct. Please provide some
> >> guidance if I did any wrong assumption, RTFM link is welcome as usual.
> >> 
> > 
> > The documentation about the master CLI is probably light about how it's
> > implemented so I'll try to be as clear as possible.
> > 
> Cristal clear, thank you so much. Just a few more doubts below.
> 
> 
> >> First of all I figured out that master cli will "connection refused"
> >> after a reload cmd during the start of the new worker. I'm using a
> >> unix socket but I assume the behaviour would be the same using
> >> ip+port.
> > 
> > Indeed, during a reload all connections to the master CLI are closed
> > because the master process is executed again. There is currently no
> > mecanism to recover a previous connection from the previous process.
> > That's the same with the unix socket or an IP socket.
> > 
> Is there any chance to be fast enough to connect and receive a response to a 
> `show proc` command before the master started the reload process? Sleep 1ms 
> maybe? Or when master finishes the `reload` response it’ll immediately 
> shutdown its listener?
> 

Once it receive the 'reload' it closes the master CLI immediately, it
will be really difficult to be fast enough to do a 'show proc'.

> >> I also observed that the `show proc` output isn't that
> >> machine-friendly, anyway it's not a big deal, some regex does the job
> >> and I'm assuming the lay out is immutable. Maybe I missed some
> >> optional json output in the doc?
> >> 
> > It is supposed to be splitable easily with cut, but if that does not
> > work this is a clearly a bug. 
> 
> So may I assume fields have 16 bytes and relative pid is represented
> as "[was: <0-9...>]", otherwise we have a bug?
> 

Oh right... the space in "[was: ]" is troublesome for cutting the string,
we must remove it.

Regarding the size of the fields, 16 bytes should be enough but there is
the expection of the version field that can be much larger if a dev
version with the commit ID in the version is used.

-- 
William Lallemand



Re: check successful reload using master cli

2020-09-15 Thread Joao Morais



> Em 14 de set de 2020, à(s) 19:14, William Lallemand  
> escreveu:
> 
> Hello,
> 
> On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote:
>> Hello list, I'm working on an automation around haproxy process
>> lifecycle in master-worker mode. It's working nice but I'm not
>> confident that all premisses I used are correct. Please provide some
>> guidance if I did any wrong assumption, RTFM link is welcome as usual.
>> 
> 
> The documentation about the master CLI is probably light about how it's
> implemented so I'll try to be as clear as possible.
> 
Cristal clear, thank you so much. Just a few more doubts below.


>> First of all I figured out that master cli will "connection refused"
>> after a reload cmd during the start of the new worker. I'm using a
>> unix socket but I assume the behaviour would be the same using
>> ip+port.
> 
> Indeed, during a reload all connections to the master CLI are closed
> because the master process is executed again. There is currently no
> mecanism to recover a previous connection from the previous process.
> That's the same with the unix socket or an IP socket.
> 
Is there any chance to be fast enough to connect and receive a response to a 
`show proc` command before the master started the reload process? Sleep 1ms 
maybe? Or when master finishes the `reload` response it’ll immediately shutdown 
its listener?


>> I also observed that the `show proc` output isn't that
>> machine-friendly, anyway it's not a big deal, some regex does the job
>> and I'm assuming the lay out is immutable. Maybe I missed some
>> optional json output in the doc?
>> 
> It is supposed to be splitable easily with cut, but if that does not
> work this is a clearly a bug. 

So may I assume fields have 16 bytes and relative pid is represented as "[was: 
<0-9...>]", otherwise we have a bug?

~jm




Re: check successful reload using master cli

2020-09-14 Thread William Lallemand
Hello,

On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote:
> Hello list, I'm working on an automation around haproxy process
> lifecycle in master-worker mode. It's working nice but I'm not
> confident that all premisses I used are correct. Please provide some
> guidance if I did any wrong assumption, RTFM link is welcome as usual.
> 

The documentation about the master CLI is probably light about how it's
implemented so I'll try to be as clear as possible.

> First of all I figured out that master cli will "connection refused"
> after a reload cmd during the start of the new worker. I'm using a
> unix socket but I assume the behaviour would be the same using
> ip+port.

Indeed, during a reload all connections to the master CLI are closed
because the master process is executed again. There is currently no
mecanism to recover a previous connection from the previous process.
That's the same with the unix socket or an IP socket.

> I'm also using a successful connection as the meaning of a
> finished reload process, failed or not. Please let me know if there is
> a way to be notified, I'm retrying to connect within a for-loop and a
> growing sleep between attempts.

Unfortunately there is no way to be notified, that was part of the next
things I wanted to implement in the master process. But you are right,
once you can connect, the reload is finished.

> Another assumption I made is that if I don't have any worker in the `#
> workers` list, it means that the last reload failed. I've observed
> that as soon as the master cli accepts connections again, the new
> worker is already there provided that the reload didn't fail.

That's correct. You can also check the number of reloads the worker had.
If it reloaded correctly this number is 0 for the new workers.

> I also observed that the `show proc` output isn't that
> machine-friendly, anyway it's not a big deal, some regex does the job
> and I'm assuming the lay out is immutable. Maybe I missed some
> optional json output in the doc?
> 
It is supposed to be splitable easily with cut, but if that does not
work this is a clearly a bug. 

-- 
William Lallemand



check successful reload using master cli

2020-09-14 Thread Joao Morais


Hello list, I'm working on an automation around haproxy process lifecycle in 
master-worker mode. It's working nice but I'm not confident that all premisses 
I used are correct. Please provide some guidance if I did any wrong assumption, 
RTFM link is welcome as usual.

First of all I figured out that master cli will "connection refused" after a 
reload cmd during the start of the new worker. I'm using a unix socket but I 
assume the behaviour would be the same using ip+port. I'm also using a 
successful connection as the meaning of a finished reload process, failed or 
not. Please let me know if there is a way to be notified, I'm retrying to 
connect within a for-loop and a growing sleep between attempts.

Another assumption I made is that if I don't have any worker in the `# workers` 
list, it means that the last reload failed. I've observed that as soon as the 
master cli accepts connections again, the new worker is already there provided 
that the reload didn't fail.

I also observed that the `show proc` output isn't that machine-friendly, anyway 
it's not a big deal, some regex does the job and I'm assuming the lay out is 
immutable. Maybe I missed some optional json output in the doc?

Thanks in advance, ~jm