Re: check successful reload using master cli
On Tue, Sep 15, 2020 at 01:24:04PM -0300, Joao Morais wrote: > > > > Em 15 de set de 2020, à(s) 12:36, William Lallemand > > escreveu: > > > > Oh right... the space in "[was: ]" is troublesome for cutting the string, > > we must remove it. > > It's not a problem at all when using chunks of fixed size, even if columns > differ between them, and the lay out ([was: ...]) remains the same. Sure, but the spaces are only alignment, it could change in the future, the real separator are the spaces, and the fields are not supposed to contain a space or to be empty. So the parsing should be as easy with awk or cut. > > > Regarding the size of the fields, 16 bytes should be enough but there is > > the expection of the version field that can be much larger if a dev > > version with the commit ID in the version is used. > Sorry, I wasn't clear here, I meant if you use a dev version you could have something like this: 2.0-dev7-0124c9-7 > If changing the spec, please provide also a way to distinguish the > haproxy version via master cli which is not in the same `show proc` > command, something like a `show info` or `version`, or at most in a > fixed position at the start of `show proc`, so I can use it to version > the output parser. > I only intend to remove the space in the brackets for 2.3, if you want a parser which works in any case, just consider that the brackets work like quotes and allow spaces in this field. I'll add a "show info" on the master CLI, that will be useful. -- William Lallemand
Re: check successful reload using master cli
On Tue, Sep 15, 2020 at 06:43:11PM +0100, Jonathan Matthews wrote: > On Tue, 15 Sep 2020 at 16:42, Tim Düsterhus wrote: > > > Why not use the Tab (0x09) as separator and make the output a proper TSV? > > > I’m only 2% joking when I point out that ASCII already has single-byte > inter-field and inter-record delimiters defined, just waiting to be used > ... :-) > > https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/ > > http://html-codes.info/ascii/standard/What-is-the-ASCII-value-of-record%20separator_30 > The problem with this is that it's not human readable since this is a non-printable character. I prefer to keep it simple. -- William Lallemand
Re: check successful reload using master cli
On Tue, Sep 15, 2020 at 05:39:18PM +0200, Tim Düsterhus wrote: > William, > > Am 15.09.20 um 17:36 schrieb William Lallemand: > > Oh right... the space in "[was: ]" is troublesome for cutting the string, > > we must remove it. > > > > Why not use the Tab (0x09) as separator and make the output a proper TSV? Because tabs don't align correctly if the head column is bigger than the data column so it's less human readable, we wanted to have an output which is both human readable and parsable. -- William Lallemand
Re: check successful reload using master cli
On Tue, 15 Sep 2020 at 16:42, Tim Düsterhus wrote: > Why not use the Tab (0x09) as separator and make the output a proper TSV? I’m only 2% joking when I point out that ASCII already has single-byte inter-field and inter-record delimiters defined, just waiting to be used ... :-) https://ronaldduncan.wordpress.com/2009/10/31/text-file-formats-ascii-delimited-text-not-csv-or-tab-delimited-text/ http://html-codes.info/ascii/standard/What-is-the-ASCII-value-of-record%20separator_30 -- Jonathan Matthews https://jpluscplusm.com
Re: check successful reload using master cli
> Em 15 de set de 2020, à(s) 12:36, William Lallemand > escreveu: > > Oh right... the space in "[was: ]" is troublesome for cutting the string, > we must remove it. It's not a problem at all when using chunks of fixed size, even if columns differ between them, and the lay out ([was: ...]) remains the same. > Regarding the size of the fields, 16 bytes should be enough but there is > the expection of the version field that can be much larger if a dev > version with the commit ID in the version is used. If changing the spec, please provide also a way to distinguish the haproxy version via master cli which is not in the same `show proc` command, something like a `show info` or `version`, or at most in a fixed position at the start of `show proc`, so I can use it to version the output parser. ~jm
Re: check successful reload using master cli
William, Am 15.09.20 um 17:36 schrieb William Lallemand: > Oh right... the space in "[was: ]" is troublesome for cutting the string, > we must remove it. > Why not use the Tab (0x09) as separator and make the output a proper TSV? Best regards Tim Düsterhus
Re: check successful reload using master cli
On Tue, Sep 15, 2020 at 11:52:15AM -0300, Joao Morais wrote: > > > > Em 14 de set de 2020, à(s) 19:14, William Lallemand > > escreveu: > > > > Hello, > > > > On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote: > >> Hello list, I'm working on an automation around haproxy process > >> lifecycle in master-worker mode. It's working nice but I'm not > >> confident that all premisses I used are correct. Please provide some > >> guidance if I did any wrong assumption, RTFM link is welcome as usual. > >> > > > > The documentation about the master CLI is probably light about how it's > > implemented so I'll try to be as clear as possible. > > > Cristal clear, thank you so much. Just a few more doubts below. > > > >> First of all I figured out that master cli will "connection refused" > >> after a reload cmd during the start of the new worker. I'm using a > >> unix socket but I assume the behaviour would be the same using > >> ip+port. > > > > Indeed, during a reload all connections to the master CLI are closed > > because the master process is executed again. There is currently no > > mecanism to recover a previous connection from the previous process. > > That's the same with the unix socket or an IP socket. > > > Is there any chance to be fast enough to connect and receive a response to a > `show proc` command before the master started the reload process? Sleep 1ms > maybe? Or when master finishes the `reload` response it’ll immediately > shutdown its listener? > Once it receive the 'reload' it closes the master CLI immediately, it will be really difficult to be fast enough to do a 'show proc'. > >> I also observed that the `show proc` output isn't that > >> machine-friendly, anyway it's not a big deal, some regex does the job > >> and I'm assuming the lay out is immutable. Maybe I missed some > >> optional json output in the doc? > >> > > It is supposed to be splitable easily with cut, but if that does not > > work this is a clearly a bug. > > So may I assume fields have 16 bytes and relative pid is represented > as "[was: <0-9...>]", otherwise we have a bug? > Oh right... the space in "[was: ]" is troublesome for cutting the string, we must remove it. Regarding the size of the fields, 16 bytes should be enough but there is the expection of the version field that can be much larger if a dev version with the commit ID in the version is used. -- William Lallemand
Re: check successful reload using master cli
> Em 14 de set de 2020, à(s) 19:14, William Lallemand > escreveu: > > Hello, > > On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote: >> Hello list, I'm working on an automation around haproxy process >> lifecycle in master-worker mode. It's working nice but I'm not >> confident that all premisses I used are correct. Please provide some >> guidance if I did any wrong assumption, RTFM link is welcome as usual. >> > > The documentation about the master CLI is probably light about how it's > implemented so I'll try to be as clear as possible. > Cristal clear, thank you so much. Just a few more doubts below. >> First of all I figured out that master cli will "connection refused" >> after a reload cmd during the start of the new worker. I'm using a >> unix socket but I assume the behaviour would be the same using >> ip+port. > > Indeed, during a reload all connections to the master CLI are closed > because the master process is executed again. There is currently no > mecanism to recover a previous connection from the previous process. > That's the same with the unix socket or an IP socket. > Is there any chance to be fast enough to connect and receive a response to a `show proc` command before the master started the reload process? Sleep 1ms maybe? Or when master finishes the `reload` response it’ll immediately shutdown its listener? >> I also observed that the `show proc` output isn't that >> machine-friendly, anyway it's not a big deal, some regex does the job >> and I'm assuming the lay out is immutable. Maybe I missed some >> optional json output in the doc? >> > It is supposed to be splitable easily with cut, but if that does not > work this is a clearly a bug. So may I assume fields have 16 bytes and relative pid is represented as "[was: <0-9...>]", otherwise we have a bug? ~jm
Re: check successful reload using master cli
Hello, On Mon, Sep 14, 2020 at 12:09:21PM -0300, Joao Morais wrote: > Hello list, I'm working on an automation around haproxy process > lifecycle in master-worker mode. It's working nice but I'm not > confident that all premisses I used are correct. Please provide some > guidance if I did any wrong assumption, RTFM link is welcome as usual. > The documentation about the master CLI is probably light about how it's implemented so I'll try to be as clear as possible. > First of all I figured out that master cli will "connection refused" > after a reload cmd during the start of the new worker. I'm using a > unix socket but I assume the behaviour would be the same using > ip+port. Indeed, during a reload all connections to the master CLI are closed because the master process is executed again. There is currently no mecanism to recover a previous connection from the previous process. That's the same with the unix socket or an IP socket. > I'm also using a successful connection as the meaning of a > finished reload process, failed or not. Please let me know if there is > a way to be notified, I'm retrying to connect within a for-loop and a > growing sleep between attempts. Unfortunately there is no way to be notified, that was part of the next things I wanted to implement in the master process. But you are right, once you can connect, the reload is finished. > Another assumption I made is that if I don't have any worker in the `# > workers` list, it means that the last reload failed. I've observed > that as soon as the master cli accepts connections again, the new > worker is already there provided that the reload didn't fail. That's correct. You can also check the number of reloads the worker had. If it reloaded correctly this number is 0 for the new workers. > I also observed that the `show proc` output isn't that > machine-friendly, anyway it's not a big deal, some regex does the job > and I'm assuming the lay out is immutable. Maybe I missed some > optional json output in the doc? > It is supposed to be splitable easily with cut, but if that does not work this is a clearly a bug. -- William Lallemand
