Re: RFC uuid for log-format

2019-09-12 Thread Schimweg, Luca
Hey Willy,

thanks so much!

Best regards
Luca

On 13.09.19, 04:48, "Willy Tarreau"  wrote:

Hi Luca,

On Wed, Sep 11, 2019 at 10:31:27AM +, Schimweg, Luca wrote:
> Hey Tim,
> 
> Thanks, I missed the empty line. Fixed it
> Yeah, the email is correct. There is no github.com account associated with
> this email, so I'll use my private email to commit, which has a github.com
> account.

I've now merged your patch but I have applied a few minor adaptations:
  - there were space warnings in "git am" due to trailing spaces at the
end of a few lines (these ones appear in red when doing "git diff"
when you enable color mode, so that you can easily spot them)

  - I noticed an indentation issue in smp_fetch_uuid() which used 4
spaces everywhere while smp_check_uuid() was fine.

  - I got a warning related to %llx being used while the argument was
uint64_t (I know, both are the same but gcc likely assumes that there
are some platforms where it's not the case). Thus I added a long long
explicit cast and took this opportunity to add "ll" at the end of the
mask, for 32-bit platforms.

  - I replaced the check (args[0].data.sint == 0) in smp_fetch_uuid()
with !args[0].type to validate there is no arg which is more
accurate.

I've seen your comment about smp_check_uuid() not being called when
there is no argument. I tested and that's indeed the case. I don't
remember if this was the intended behavior or not, that's something
we should recheck to make sure either no sample fetch relies on this
or if it's a bug (and probably we should ensure it's always called,
and document it as such).

I was also thinking that the random generation could be centralized
as a generic function to produce random bytes. I'll possibly have a
look at this some time later.

Thanks,
Willy




Re: RFC uuid for log-format

2019-09-12 Thread Willy Tarreau
Hi Luca,

On Wed, Sep 11, 2019 at 10:31:27AM +, Schimweg, Luca wrote:
> Hey Tim,
> 
> Thanks, I missed the empty line. Fixed it
> Yeah, the email is correct. There is no github.com account associated with
> this email, so I'll use my private email to commit, which has a github.com
> account.

I've now merged your patch but I have applied a few minor adaptations:
  - there were space warnings in "git am" due to trailing spaces at the
end of a few lines (these ones appear in red when doing "git diff"
when you enable color mode, so that you can easily spot them)

  - I noticed an indentation issue in smp_fetch_uuid() which used 4
spaces everywhere while smp_check_uuid() was fine.

  - I got a warning related to %llx being used while the argument was
uint64_t (I know, both are the same but gcc likely assumes that there
are some platforms where it's not the case). Thus I added a long long
explicit cast and took this opportunity to add "ll" at the end of the
mask, for 32-bit platforms.

  - I replaced the check (args[0].data.sint == 0) in smp_fetch_uuid()
with !args[0].type to validate there is no arg which is more
accurate.

I've seen your comment about smp_check_uuid() not being called when
there is no argument. I tested and that's indeed the case. I don't
remember if this was the intended behavior or not, that's something
we should recheck to make sure either no sample fetch relies on this
or if it's a bug (and probably we should ensure it's always called,
and document it as such).

I was also thinking that the random generation could be centralized
as a generic function to produce random bytes. I'll possibly have a
look at this some time later.

Thanks,
Willy



Re: RFC uuid for log-format

2019-09-11 Thread Schimweg, Luca
Hey Tim,

Thanks, I missed the empty line. Fixed it
Yeah, the email is correct. There is no github.com account associated with this 
email, so I'll use my private email to commit, which has a github.com account.

Best regards
Luca


On 11.09.19, 12:26, "Tim Düsterhus"  wrote:

Luca,

Am 11.09.19 um 09:40 schrieb Schimweg, Luca:
> thanks for the feedback. I included the changes in the updated patch.
> 

Something went wrong with the commit message:

> Subject: [PATCH] MINOR: sample: Add UUID-fetch Adds the fetch uuid(int). 
New
>  feature, but could be backported

Did you make sure to leave an empty line?


Another thing I notice is that the author and email in the attached
patch does not match the email address you are sending from. You might
want to fix that if it's work done for your employer.

See

https://www.git-tower.com/learn/git/faq/change-author-name-email#chapter_using+--amend+for+the+very+last+commit
for an explanation.

Best regards
Tim Düsterhus




0001-MINOR-sample-Add-UUID-fetch.patch
Description: 0001-MINOR-sample-Add-UUID-fetch.patch


Re: RFC uuid for log-format

2019-09-11 Thread Tim Düsterhus
Luca,

Am 11.09.19 um 09:40 schrieb Schimweg, Luca:
> thanks for the feedback. I included the changes in the updated patch.
> 

Something went wrong with the commit message:

> Subject: [PATCH] MINOR: sample: Add UUID-fetch Adds the fetch uuid(int). New
>  feature, but could be backported

Did you make sure to leave an empty line?


Another thing I notice is that the author and email in the attached
patch does not match the email address you are sending from. You might
want to fix that if it's work done for your employer.

See
https://www.git-tower.com/learn/git/faq/change-author-name-email#chapter_using+--amend+for+the+very+last+commit
for an explanation.

Best regards
Tim Düsterhus



Re: RFC uuid for log-format

2019-09-11 Thread Schimweg, Luca
Hey,

thanks for the feedback. I included the changes in the updated patch.

Best regards
Luca

On 11.09.19, 08:52, "Willy Tarreau"  wrote:

On Tue, Sep 10, 2019 at 03:56:43PM +0200, Tim Düsterhus wrote:
> Luca,
> 
> Am 10.09.19 um 15:48 schrieb Schimweg, Luca:
> > sorry, didn't know about these guidelines. I packed all changes into 
one commit in the patch file in the attachments. 
> 
> One more thing: Do not leave out the body of the commit message. See the
> paragraph starting line 562 of the CONTRIBUTING file:
> 
> 
https://github.com/haproxy/haproxy/blob/e058f7359f3822beb8552f77a6d439cb053edb3f/CONTRIBUTING#L562-L567
> 
> > I included your changes, you are right, the code looks much better now.
> 
> I spotted a little typo:
> 
> ((rnd[1] >> 16u) % 0xFFF) | 0x4000,  // highest 4 bits indicate the uuid
> version
> 
> It should read `&` instead of `%` there.
> 
> > I'm open for any feedback.
> 
> I suggest to wait for Willy before sending an updated patch. I guess he
> has some more feedback.

Thank to you both for the patch and the review. I agree with everything
Tim said. I have oe other minor comment which is to add an argument check
function to verify that the argument, if present, is 4, and default it to
4 when absent. Have a look at sample_conv_protobuf_check() and
smp_check_concat() for example. Otherwise I'm fine with the patch.

Cheers,
Willy




0001-MINOR-sample-Add-UUID-fetch.patch
Description: 0001-MINOR-sample-Add-UUID-fetch.patch


Re: RFC uuid for log-format

2019-09-11 Thread Willy Tarreau
On Tue, Sep 10, 2019 at 03:56:43PM +0200, Tim Düsterhus wrote:
> Luca,
> 
> Am 10.09.19 um 15:48 schrieb Schimweg, Luca:
> > sorry, didn't know about these guidelines. I packed all changes into one 
> > commit in the patch file in the attachments. 
> 
> One more thing: Do not leave out the body of the commit message. See the
> paragraph starting line 562 of the CONTRIBUTING file:
> 
> https://github.com/haproxy/haproxy/blob/e058f7359f3822beb8552f77a6d439cb053edb3f/CONTRIBUTING#L562-L567
> 
> > I included your changes, you are right, the code looks much better now.
> 
> I spotted a little typo:
> 
> ((rnd[1] >> 16u) % 0xFFF) | 0x4000,  // highest 4 bits indicate the uuid
> version
> 
> It should read `&` instead of `%` there.
> 
> > I'm open for any feedback.
> 
> I suggest to wait for Willy before sending an updated patch. I guess he
> has some more feedback.

Thank to you both for the patch and the review. I agree with everything
Tim said. I have oe other minor comment which is to add an argument check
function to verify that the argument, if present, is 4, and default it to
4 when absent. Have a look at sample_conv_protobuf_check() and
smp_check_concat() for example. Otherwise I'm fine with the patch.

Cheers,
Willy



Re: RFC uuid for log-format

2019-09-10 Thread Tim Düsterhus
Luca,

Am 10.09.19 um 15:48 schrieb Schimweg, Luca:
> sorry, didn't know about these guidelines. I packed all changes into one 
> commit in the patch file in the attachments. 

One more thing: Do not leave out the body of the commit message. See the
paragraph starting line 562 of the CONTRIBUTING file:

https://github.com/haproxy/haproxy/blob/e058f7359f3822beb8552f77a6d439cb053edb3f/CONTRIBUTING#L562-L567

> I included your changes, you are right, the code looks much better now.

I spotted a little typo:

((rnd[1] >> 16u) % 0xFFF) | 0x4000,  // highest 4 bits indicate the uuid
version

It should read `&` instead of `%` there.

> I'm open for any feedback.

I suggest to wait for Willy before sending an updated patch. I guess he
has some more feedback.

Best regards
Tim Düsterhus



Re: RFC uuid for log-format

2019-09-10 Thread Schimweg, Luca
Hey,

sorry, didn't know about these guidelines. I packed all changes into one commit 
in the patch file in the attachments. 
I included your changes, you are right, the code looks much better now.

I'm open for any feedback.

Best regards,
Luca

On 10.09.19, 14:25, "Tim Düsterhus"  wrote:

Luca,

(not an HAProxy maintainer)

please have a look at the CONTRIBUTING file regarding the commit message
format: https://github.com/haproxy/haproxy/blob/master/CONTRIBUTING

You are missing the severity indicator and the affected subsystem. It
probably should be `MINOR: sample:`.

> diff --git a/doc/configuration.txt b/doc/configuration.txt
> index ba5a8b360..e22557134 100644
> --- a/doc/configuration.txt
> +++ b/doc/configuration.txt
> @@ -14391,6 +14391,11 @@ thread : integer
>the function, between 0 and (global.nbthread-1). This is useful for 
logging
>and debugging purposes.
>  
> +uuid([]) : string
> +  Returns a random UUID, following the RFC4122 standard. If the version 
is not
   ^^
I'd remove the random here. UUID v3 while not currently supported can
hardly be called "random". UUID in itself already implies "unique".

> +  specified, a UUID version 4 (fully random) is returned.
> +  Currently, only version 4 is supported.
> +
>  var() : undefined
>Returns a variable with the stored type. If the variable is not set, 
the
>sample fetch fails. The name of the variable starts with an indication
> diff --git a/src/sample.c b/src/sample.c
> index 6327b6b08..6caa440b7 100644
> --- a/src/sample.c
> +++ b/src/sample.c
> @@ -3196,6 +3196,47 @@ static int smp_fetch_const_meth(const struct arg 
*args, struct sample *smp, cons
>   return 1;
>  }
>  
> +// Generate a RFC4122 UUID (default is v4 = fully random)
> +static int
> +smp_fetch_uuid(const struct arg *args, struct sample *smp, const char 
*kw, void *private)
> +{
> +if (!args[0].data.sint || args[0].data.sint == 4) {
> +uint32_t rnd[4] = { 0, 0, 0, 0 };
> +uint64_t last = 0;
> +int byte = 0;
> +uint8_t bits = 0;
> +unsigned int rand_max_bits = my_flsl(RAND_MAX);
> +
> +while (byte < 4) {
> +while (bits < 32) {
> +last |= (uint64_t)random() << bits;
> +bits += rand_max_bits;
> +}
> +rnd[byte++] = last;
> +last >>= 32u;
> +bits  -= 32;
> +}
> +
> +chunk_printf(, "%8.8x-%4.4x-4%3.3x-%4.4x-%12.12llx",
 ^
Instead of hardcoding the 4 here (it's easy to miss within all the
digits for the format specifiers) I suggest to instead "force" the bits
to match the version within the random values using bit operations.

> + rnd[0], // first 32 bits
> + rnd[1] % 65536, // getting fist 16 bits
 ^ 
Hexadecimal notation and a bit mask would capture the intent better:
rand[1] & 0x. The comment is redundant then.

Typo in "first".

> + (rnd[1] >> 16u) % 4096,  // getting bits 17 - 28 
(12 bits), leaving out bits 29 - 32

With the suggestion regarding the version number this turns into:

((rand[1] >> 16u) & 0xFFF) | 0x4000 // the highest 4 bits indicate the
UUID version

> + (rnd[2] % 16384) + 32768,  // getting first 14 
bits, setting the first two bits to 1 and 0...

I'd do (rnd[2] & 0x3FFF) | 0x8000. // the highest 2 bits indicate the
UUID variant (10)

> + (*(((uint64_t*) rnd) + 1) >> 14u) % 281474976710656 
// getting 48 bits (last 18 of rnd[2] + first 30 of rnd[3], 281474976710656 = 
2^48)

Instead of casting this to a 64 bit value I guess it's easier to
understand if you add the remaining 48 bits in two steps, the remaining
18 of the 3rd byte and the 30 bits of the 4th byte.

> +);
> +
> +smp->data.type = SMP_T_STR;
> +smp->flags = SMP_F_VOL_TEST | SMP_F_MAY_CHANGE;;
 ^^
Duplicate semicolon.

> +smp->data.u.str = trash;
> +
> +return 1;
> +}
> +
> +// more implementations of other uuid formats possible here
> +return 0;
> +
> +}
> +
>  /* Note: must not be declared  as its list will be overwritten.
>   * Note: fetches that may return multiple types must be declared as the 
lowest
>   * common denominator, the type that can be casted into all other ones. 
For
> @@ -3214,6 +3255,7 @@ static struct sample_fetch_kw_list smp_kws = {ILH, {
>   { 

Re: RFC uuid for log-format

2019-09-10 Thread Tim Düsterhus
Luca,

(not an HAProxy maintainer)

please have a look at the CONTRIBUTING file regarding the commit message
format: https://github.com/haproxy/haproxy/blob/master/CONTRIBUTING

You are missing the severity indicator and the affected subsystem. It
probably should be `MINOR: sample:`.

> diff --git a/doc/configuration.txt b/doc/configuration.txt
> index ba5a8b360..e22557134 100644
> --- a/doc/configuration.txt
> +++ b/doc/configuration.txt
> @@ -14391,6 +14391,11 @@ thread : integer
>the function, between 0 and (global.nbthread-1). This is useful for logging
>and debugging purposes.
>  
> +uuid([]) : string
> +  Returns a random UUID, following the RFC4122 standard. If the version is 
> not
   ^^
I'd remove the random here. UUID v3 while not currently supported can
hardly be called "random". UUID in itself already implies "unique".

> +  specified, a UUID version 4 (fully random) is returned.
> +  Currently, only version 4 is supported.
> +
>  var() : undefined
>Returns a variable with the stored type. If the variable is not set, the
>sample fetch fails. The name of the variable starts with an indication
> diff --git a/src/sample.c b/src/sample.c
> index 6327b6b08..6caa440b7 100644
> --- a/src/sample.c
> +++ b/src/sample.c
> @@ -3196,6 +3196,47 @@ static int smp_fetch_const_meth(const struct arg 
> *args, struct sample *smp, cons
>   return 1;
>  }
>  
> +// Generate a RFC4122 UUID (default is v4 = fully random)
> +static int
> +smp_fetch_uuid(const struct arg *args, struct sample *smp, const char *kw, 
> void *private)
> +{
> +if (!args[0].data.sint || args[0].data.sint == 4) {
> +uint32_t rnd[4] = { 0, 0, 0, 0 };
> +uint64_t last = 0;
> +int byte = 0;
> +uint8_t bits = 0;
> +unsigned int rand_max_bits = my_flsl(RAND_MAX);
> +
> +while (byte < 4) {
> +while (bits < 32) {
> +last |= (uint64_t)random() << bits;
> +bits += rand_max_bits;
> +}
> +rnd[byte++] = last;
> +last >>= 32u;
> +bits  -= 32;
> +}
> +
> +chunk_printf(, "%8.8x-%4.4x-4%3.3x-%4.4x-%12.12llx",
 ^
Instead of hardcoding the 4 here (it's easy to miss within all the
digits for the format specifiers) I suggest to instead "force" the bits
to match the version within the random values using bit operations.

> + rnd[0], // first 32 bits
> + rnd[1] % 65536, // getting fist 16 bits
 ^ 
Hexadecimal notation and a bit mask would capture the intent better:
rand[1] & 0x. The comment is redundant then.

Typo in "first".

> + (rnd[1] >> 16u) % 4096,  // getting bits 17 - 28 (12 
> bits), leaving out bits 29 - 32

With the suggestion regarding the version number this turns into:

((rand[1] >> 16u) & 0xFFF) | 0x4000 // the highest 4 bits indicate the
UUID version

> + (rnd[2] % 16384) + 32768,  // getting first 14 bits, 
> setting the first two bits to 1 and 0...

I'd do (rnd[2] & 0x3FFF) | 0x8000. // the highest 2 bits indicate the
UUID variant (10)

> + (*(((uint64_t*) rnd) + 1) >> 14u) % 281474976710656 // 
> getting 48 bits (last 18 of rnd[2] + first 30 of rnd[3], 281474976710656 = 
> 2^48)

Instead of casting this to a 64 bit value I guess it's easier to
understand if you add the remaining 48 bits in two steps, the remaining
18 of the 3rd byte and the 30 bits of the 4th byte.

> +);
> +
> +smp->data.type = SMP_T_STR;
> +smp->flags = SMP_F_VOL_TEST | SMP_F_MAY_CHANGE;;
 ^^
Duplicate semicolon.

> +smp->data.u.str = trash;
> +
> +return 1;
> +}
> +
> +// more implementations of other uuid formats possible here
> +return 0;
> +
> +}
> +
>  /* Note: must not be declared  as its list will be overwritten.
>   * Note: fetches that may return multiple types must be declared as the 
> lowest
>   * common denominator, the type that can be casted into all other ones. For
> @@ -3214,6 +3255,7 @@ static struct sample_fetch_kw_list smp_kws = {ILH, {
>   { "rand", smp_fetch_rand,  ARG1(0,SINT), NULL, SMP_T_SINT, 
> SMP_USE_INTRN },
>   { "stopping", smp_fetch_stopping, 0, NULL, SMP_T_BOOL, 
> SMP_USE_INTRN },
>   { "stopping", smp_fetch_stopping, 0, NULL, SMP_T_BOOL, 
> SMP_USE_INTRN },
> + { "uuid", smp_fetch_uuid,  ARG1(0, SINT),  NULL, SMP_T_STR, 
> SMP_USE_INTRN },
>  
>   { "cpu_calls",smp_fetch_cpu_calls,  0,   NULL, SMP_T_SINT, 
> SMP_USE_INTRN },
>   { "cpu_ns_avg",   smp_fetch_cpu_ns_avg, 0,   NULL, SMP_T_SINT, 
> SMP_USE_INTRN },
> -- 
> 2.23.0




Re: RFC uuid for log-format

2019-09-10 Thread Schimweg, Luca
Hey,

I just integrated your change requests into my fork and created a PR: 
https://github.com/haproxy/haproxy/pull/271, would be happy if anyone could 
give me some feedback!

Best regards,
Luca

On 06.09.19, 11:41, "Willy Tarreau"  wrote:

On Fri, Sep 06, 2019 at 11:33:32AM +0200, Geoff Simmons wrote:
> On 9/6/19 11:22, Willy Tarreau wrote:
> > 
> > We can then have 4 being the only one implemented for now, we can then
> > set it to the default one if unspecified, but I really want to support
> > such a parameter so that it's easy to extend the form once a new one
> > arrives.
> 
> Still picking nits (although I think this is a good plan) -- if uuid()
> takes a parameter, and version 4 is the only one that's implemented,
> then uuid(1) or uuid(3) etc throws an error?

Yes, that's just the argument parser which will return this.

Willy





Re: RFC uuid for log-format

2019-09-06 Thread Willy Tarreau
On Fri, Sep 06, 2019 at 11:33:32AM +0200, Geoff Simmons wrote:
> On 9/6/19 11:22, Willy Tarreau wrote:
> > 
> > We can then have 4 being the only one implemented for now, we can then
> > set it to the default one if unspecified, but I really want to support
> > such a parameter so that it's easy to extend the form once a new one
> > arrives.
> 
> Still picking nits (although I think this is a good plan) -- if uuid()
> takes a parameter, and version 4 is the only one that's implemented,
> then uuid(1) or uuid(3) etc throws an error?

Yes, that's just the argument parser which will return this.

Willy



Re: RFC uuid for log-format

2019-09-06 Thread Willy Tarreau
On Fri, Sep 06, 2019 at 11:04:44AM +0200, Tim Düsterhus wrote:
> Willy,
> Luca,
> 
> Am 06.09.19 um 05:17 schrieb Willy Tarreau:
> >> smp->data.type = SMP_T_STR;
> >> smp->flags = SMP_F_MAY_CHANGE;
> 
> Comparing this with the time fetches I believe there is 'SMP_F_VOL_TEST'
> missing, no?

Yep good point.

Also RAND_MAX is not the same on all platforms. We're used to having 31
bits on Linux, it's 15 bits only on others like Windows. I think we'd
need to preload 128 bits of random into local variables based on RAND_MAX
then use these variables to produce the output. It could be something more
or less like this :

   uint32_t rnd[4] = { 0, 0, 0, 0 };
   uint64_t last = 0;
   int byte = 0;
   int bits = 0;

   while (byte < 4) {
   while (bits < 32) {
  last |= (uint64_t)random() << bits;
  bits += my_flsl(RAND_MAX);
   }
   rnd[byte++] = last;
   last >>= 32;
   bits  -= 32;
   }

With something like this (modulo the bugs I wrote) you should end up
with 128 bits available in rnd[] regardless of RAND_MAX. 

Willy



Re: RFC uuid for log-format

2019-09-06 Thread Geoff Simmons
On 9/6/19 11:22, Willy Tarreau wrote:
> 
> We can then have 4 being the only one implemented for now, we can then
> set it to the default one if unspecified, but I really want to support
> such a parameter so that it's easy to extend the form once a new one
> arrives.

Still picking nits (although I think this is a good plan) -- if uuid()
takes a parameter, and version 4 is the only one that's implemented,
then uuid(1) or uuid(3) etc throws an error?


Best,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de



signature.asc
Description: OpenPGP digital signature


Re: RFC uuid for log-format

2019-09-06 Thread Tim Düsterhus
Willy,

Am 06.09.19 um 11:22 schrieb Willy Tarreau:
> That's what I really meant in order to support different formats that
> one would like to implement, without having to be exclusively tied to
> those described in RFC4122. But after thinking a bit about it, I guess
> that if most users expect the number here to match the version this
> would create more confusion to have a different one, and the
> unique-id-format still allows to build a complete one from scratch for
> those needing anything different.

Yes I believe that most users think of RFC 4122 when seeing UUID and
thus would expect `uuid(1)` to generate a v1 UUID as specified in RFC 4122.

Java: https://docs.oracle.com/javase/8/docs/api/java/util/UUID.html
JavaScript (npm): https://www.npmjs.com/package/uuid
A PHP lib: https://github.com/ramsey/uuid
Rust: https://docs.rs/uuid/0.6.5/uuid/
Python: https://docs.python.org/2/library/uuid.html

... they all mention RFC 4122. And that's just the first page when
searching Google for "UUID".

Best regards
Tim Düsterhus



Re: RFC uuid for log-format

2019-09-06 Thread Willy Tarreau
On Fri, Sep 06, 2019 at 10:54:41AM +0200, Tim Düsterhus wrote:
> Willy,
> 
> Am 06.09.19 um 10:48 schrieb Willy Tarreau:
> > Let's simply declare that uuid(1) follows the v4 format of RFC4122 with
> > all fields random. Then we may add new types later on when needed.
> 
> I'm not sure whether that's a typo, but I'd suggest `uuid(4)` for a
> version 4 UUID.

That's what I really meant in order to support different formats that
one would like to implement, without having to be exclusively tied to
those described in RFC4122. But after thinking a bit about it, I guess
that if most users expect the number here to match the version this
would create more confusion to have a different one, and the
unique-id-format still allows to build a complete one from scratch for
those needing anything different.

Thus I guess I'm finally fine with leaving only the official version
number there and considering that anything different will continue to
be handled by manual configuration as has been done over the years.

We can then have 4 being the only one implemented for now, we can then
set it to the default one if unspecified, but I really want to support
such a parameter so that it's easy to extend the form once a new one
arrives.

Regards,
Willy



Re: RFC uuid for log-format

2019-09-06 Thread Tim Düsterhus
Willy,
Luca,

Am 06.09.19 um 05:17 schrieb Willy Tarreau:
>> smp->data.type = SMP_T_STR;
>> smp->flags = SMP_F_MAY_CHANGE;

Comparing this with the time fetches I believe there is 'SMP_F_VOL_TEST'
missing, no?

>> smp->data.type = SMP_T_STR;
>> smp->data.u.str.area = smp_fetch_uuid_res;
>> smp->data.u.str.data = strlen(smp_fetch_uuid_res);
> 

Best regards
Tim Düsterhus



Re: RFC uuid for log-format

2019-09-06 Thread Tim Düsterhus
Willy,

Am 06.09.19 um 10:48 schrieb Willy Tarreau:
> Let's simply declare that uuid(1) follows the v4 format of RFC4122 with
> all fields random. Then we may add new types later on when needed.

I'm not sure whether that's a typo, but I'd suggest `uuid(4)` for a
version 4 UUID.

Best regards
Tim Düsterhus



Re: RFC uuid for log-format

2019-09-06 Thread Willy Tarreau
On Fri, Sep 06, 2019 at 10:35:08AM +0200, Geoff Simmons wrote:
> On 9/6/19 09:57, Schimweg, Luca wrote:
> > 
> > UUIDs generally have different versions. I implemented version 4
> > right now, but if you want to we can implement different UUID types.
> > I personally think one is enough, because most people won't care if
> > they get an UUID v1, v2, ...,
> I agree with this. Most people want UUIDs just for the uniqueness, and
> don't care if there's something like a MAC address or timestamp in there
> somewhere.
> 
> I would suggest just v4 first, and the rest only if anyone ever asks for
> them.

I agree with this as well.

> I doubt that anyone ever will.

I'm certain some will want to. In distributed architectures it's often a
requirement to be able to partially isolate your uuids, even just for
basic reasons like distributing the logs to multiple servers without
losing ordering for example.

Let's simply declare that uuid(1) follows the v4 format of RFC4122 with
all fields random. Then we may add new types later on when needed.
 
Cheers,
Willy



Re: RFC uuid for log-format

2019-09-06 Thread Geoff Simmons
On 9/6/19 09:57, Schimweg, Luca wrote:
> 
> UUIDs generally have different versions. I implemented version 4
> right now, but if you want to we can implement different UUID types.
> I personally think one is enough, because most people won't care if
> they get an UUID v1, v2, ...,
I agree with this. Most people want UUIDs just for the uniqueness, and
don't care if there's something like a MAC address or timestamp in there
somewhere.

I would suggest just v4 first, and the rest only if anyone ever asks for
them. I doubt that anyone ever will.


Best,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de



signature.asc
Description: OpenPGP digital signature


Re: RFC uuid for log-format

2019-09-06 Thread Schimweg, Luca
Hey Willy,

thanks for the suggestions, I'll integrate them later or on monday.

UUIDs generally have different versions. I implemented version 4 right now, but 
if you want to we can implement different UUID types. I personally think one is 
enough, because most people won't care if they get an UUID v1, v2, ..., but we 
could definitely do that, for example as uuid(1), uuid(2),  
In my opinion, implementing version 4 is the simplest one, which fulfills all 
requirements.

I think providing UUIDs as simple fetch is a good idea, because they do are a 
standardized format that is used quite much. PostgreSQL even has a Datatype 
dedicated to uuids.

Would you generally be interesting in adding such a feature into HAProxy? Then 
I would continue working on that topic.

Best regards,
Luca


On 06.09.19, 05:18, "Willy Tarreau"  wrote:

On Thu, Sep 05, 2019 at 05:13:11PM +, Schimweg, Luca wrote:
> Hey,
> 
> just for you to get an impression, a quick draft of what a change could 
look like:
> 
> 
https://github.com/lucaschimweg/haproxy/commit/5306395c062ac20b6d030d48c48290d5f08db5cc

( pasted here for discussion)

> char smp_fetch_uuid_res[37];

Note, this is not thread-safe, but you can use the trash from within the
function instead, then you can simply use chunk_printf(, ...) to
replace the sprintf().

> static int
> smp_fetch_uuid(const struct arg *args, struct sample *smp, const char 
*kw, void *private)
> {
> sprintf(smp_fetch_uuid_res, "%8.8x-%4.4x-4%3.3x-%4.4x-%7.7x%5.5x",
> (unsigned int) ((random() + 2*random()) % 4294967296), // 
random() only gives 31 bits of randomness, using it twice to get 32 bits

Just being nit-picking, but here you're creating a bias by adding two 
randoms,
as 25% of the values will be represented twice as frequently as the other 
ones
(those both even and in the first half). As a general rule one must never 
add
randoms with overlapping bits.

> (unsigned int) (random() % 65536), // getting 16 bits
> (unsigned int) (random() % 4096),  // getting 12 bits
> (unsigned int) (random() % 16384) + 32768,  // getting 14 
bits, setting the first two bits to 1 and 0...
> (unsigned int) (random() % 268435456), // getting 28 bits
> (unsigned int) (random() % 1048576) // getting 20 bits
> );

> 
> smp->data.type = SMP_T_STR;
> smp->flags = SMP_F_MAY_CHANGE;
> smp->data.type = SMP_T_STR;
> smp->data.u.str.area = smp_fetch_uuid_res;
> smp->data.u.str.data = strlen(smp_fetch_uuid_res);

Here since you know the length in advance, better not measure it again
(it's returned by sprintf(), or will be automatically calculated by
chunk_printf).

> I didn't check with the contributing guidelines, etc... yet, but this 
commit
> would make the feature work. 
> In the end, Willy has to decide whether we want a feature like this.

The only problem I'm having with this is that we're using a very generic
name "uuid" for a single very specific implementation that will definitely
not suit everyone. Even RFC4122 suggests parts involving timestamps and
host-based identification. What could *possibly* constitute a good first
step would be to mandate an argument which indicates the type of UUID we
want to produce. This function would for now support only one type, this
one, but be easily extended to produce others, which would just require
one extra line in the doc indicating type and description. Just my two
cents.

Thanks,
Willy




Re: RFC uuid for log-format

2019-09-05 Thread Willy Tarreau
On Thu, Sep 05, 2019 at 05:13:11PM +, Schimweg, Luca wrote:
> Hey,
> 
> just for you to get an impression, a quick draft of what a change could look 
> like:
> 
> https://github.com/lucaschimweg/haproxy/commit/5306395c062ac20b6d030d48c48290d5f08db5cc

( pasted here for discussion)

> char smp_fetch_uuid_res[37];

Note, this is not thread-safe, but you can use the trash from within the
function instead, then you can simply use chunk_printf(, ...) to
replace the sprintf().

> static int
> smp_fetch_uuid(const struct arg *args, struct sample *smp, const char *kw, 
> void *private)
> {
> sprintf(smp_fetch_uuid_res, "%8.8x-%4.4x-4%3.3x-%4.4x-%7.7x%5.5x",
> (unsigned int) ((random() + 2*random()) % 4294967296), // 
> random() only gives 31 bits of randomness, using it twice to get 32 bits

Just being nit-picking, but here you're creating a bias by adding two randoms,
as 25% of the values will be represented twice as frequently as the other ones
(those both even and in the first half). As a general rule one must never add
randoms with overlapping bits.

> (unsigned int) (random() % 65536), // getting 16 bits
> (unsigned int) (random() % 4096),  // getting 12 bits
> (unsigned int) (random() % 16384) + 32768,  // getting 14 bits, 
> setting the first two bits to 1 and 0...
> (unsigned int) (random() % 268435456), // getting 28 bits
> (unsigned int) (random() % 1048576) // getting 20 bits
> );

> 
> smp->data.type = SMP_T_STR;
> smp->flags = SMP_F_MAY_CHANGE;
> smp->data.type = SMP_T_STR;
> smp->data.u.str.area = smp_fetch_uuid_res;
> smp->data.u.str.data = strlen(smp_fetch_uuid_res);

Here since you know the length in advance, better not measure it again
(it's returned by sprintf(), or will be automatically calculated by
chunk_printf).

> I didn't check with the contributing guidelines, etc... yet, but this commit
> would make the feature work. 
> In the end, Willy has to decide whether we want a feature like this.

The only problem I'm having with this is that we're using a very generic
name "uuid" for a single very specific implementation that will definitely
not suit everyone. Even RFC4122 suggests parts involving timestamps and
host-based identification. What could *possibly* constitute a good first
step would be to mandate an argument which indicates the type of UUID we
want to produce. This function would for now support only one type, this
one, but be easily extended to produce others, which would just require
one extra line in the doc indicating type and description. Just my two
cents.

Thanks,
Willy



Re: RFC uuid for log-format

2019-09-05 Thread Willy Tarreau
Hi,

On Thu, Sep 05, 2019 at 05:31:36PM +0200, Lukas Tribus wrote:
> > unique-id-format 
> > %[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]
> >
> > To be honest I think for a simple UUID, we should not be required to but 
> > such a complicated and long line into the config. Also, I think that the 
> > performance impact of such a line is much bigger than the generation of a 
> > UUID in plain C code. We could just create a ACL generator called uuid or 
> > something similar which would do the job. The config would get much clearer.
> >
> > Any opinions to this? I would be willing to implement such a feature if the 
> > maintainers would be ready to merge it in general.
> 
> Are you proposing to implement a RFC4122 compliant UUID?
> Which UUID version?
> Which dependencies would it require?
> 
> I guess something RFC4122 compliant based on openssl for example would
> work. Adding chunks of fragile code nobody is gonna maintain or adding
> additional dependencies like libuuid - probably not.
> 
> I suggest we wait for Willy's feedback, but I want to stress that this
> may not be as simple as 5 lines of standalone C code ... we certainly
> don't want to act like we are generating standard compliant RFC4122
> UUIDs when that is not really the case.

I've not looked at what RFC4122 suggests, but what I can tell is that
we'll get as many UUID formats as there are users anyway. If *some*
formats are standard and cover most people's needs, they are more
likely to be well adopted and we could imagine having a sample fetch
function to produce them more easily. But only if we really think it's
likely that they'll be used. In my experience, people dealing with
logs do *want* to integrate some elements helping with troubleshooting,
like a node name/number, a location in the chain, and sometimes some
timestamps because that helps keeping events ordered when processing
them.

Willy



Re: RFC uuid for log-format

2019-09-05 Thread Schimweg, Luca
Hey,

just for you to get an impression, a quick draft of what a change could look 
like:

https://github.com/lucaschimweg/haproxy/commit/5306395c062ac20b6d030d48c48290d5f08db5cc

I didn't check with the contributing guidelines, etc... yet, but this commit 
would make the feature work. 
In the end, Willy has to decide whether we want a feature like this.

Best regards,
Luca

On 05.09.19, 19:00, "li...@ltri.eu on behalf of Lukas Tribus"  
wrote:

Hello Luca,

On Thu, Sep 5, 2019 at 5:38 PM Schimweg, Luca  wrote:
>
> Hey,
>
> i would suggest adding support for the widely used RFC4122 Version 4 
UUID, which is generated completely random. In lua a script for generating 
these is about 5 lines long, but because of several reasons I don't want to use 
lua in this use-case.
>
> I would definitely not add add additional dependencies. The only thing 
required to generate such a UUID is a random number generator (which is already 
available) and some string-concatenation. Example code comes out with a quick 
GitHub search: https://gist.github.com/kvelakur/9069c9896577c3040030 .
>
> I think adding this feature would not require too much effort.
>
> @Willy, whats your opinion on this?

Sounds good to me, let's confirm with Willy.


Lukas




Re: RFC uuid for log-format

2019-09-05 Thread Lukas Tribus
Hello Luca,

On Thu, Sep 5, 2019 at 5:38 PM Schimweg, Luca  wrote:
>
> Hey,
>
> i would suggest adding support for the widely used RFC4122 Version 4 UUID, 
> which is generated completely random. In lua a script for generating these is 
> about 5 lines long, but because of several reasons I don't want to use lua in 
> this use-case.
>
> I would definitely not add add additional dependencies. The only thing 
> required to generate such a UUID is a random number generator (which is 
> already available) and some string-concatenation. Example code comes out with 
> a quick GitHub search: https://gist.github.com/kvelakur/9069c9896577c3040030 .
>
> I think adding this feature would not require too much effort.
>
> @Willy, whats your opinion on this?

Sounds good to me, let's confirm with Willy.


Lukas



Re: RFC uuid for log-format

2019-09-05 Thread Geoff Simmons
On 9/5/19 17:43, Schimweg, Luca wrote:
> 
> actually, I am doing that, I generate a random number from 0-16383
> and after that I add 32768 onto it. 32768 is 0x8000,
> 32768+16383=49151 is 0xBFFF.
Ah, I missed it, sorry again.


Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de



signature.asc
Description: OpenPGP digital signature


Re: RFC uuid for log-format

2019-09-05 Thread Schimweg, Luca
Hey,

actually, I am doing that, I generate a random number from 0-16383 and after 
that I add 32768 onto it. 32768 is 0x8000, 32768+16383=49151 is 0xBFFF. I 
cannot really come up with a reason why this is too, but the specification is 
fulfilled.

And yeah, we could definitely generate any other format to, but UUIDs are a 
standardized format, so I cannot imagine why it is that hard using it, because 
there are probably many people that want to use it.

Best regards
Luca

On 05.09.19, 17:38, "Geoff Simmons"  wrote:

On 9/5/19 16:58, Schimweg, Luca wrote:
> 
> 
> unique-id-format 
%[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]

This is going to be very nit-picky (I apologize): that comes close but
isn't actually guaranteed to produce an RFC 4122 version 4 UUID (the
"random" UUID).

You're right about the 4 at the beginning of the third group of hex
digits. But the first digit of the fourth group is required to be one of
8, 9, a or b. I've never understood why, but there it is.

Whether or not that matters, it seems to me, depends on requirements. If
you just need 16 bytes of randomness for a request ID, why not just
generate that? base64 encoding would be shorter, by the way -- 25 chars
for base64 with padding, 36 for the UUID format.

But if someone has an interoperability requirement dictating a UUID that
has to meet the standard format, then I agree that a fetch that gets it
just right would be warranted.


Best,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de





Re: RFC uuid for log-format

2019-09-05 Thread Schimweg, Luca
Hey,

i would suggest adding support for the widely used RFC4122 Version 4 UUID, 
which is generated completely random. In lua a script for generating these is 
about 5 lines long, but because of several reasons I don't want to use lua in 
this use-case.

I would definitely not add add additional dependencies. The only thing required 
to generate such a UUID is a random number generator (which is already 
available) and some string-concatenation. Example code comes out with a quick 
GitHub search: https://gist.github.com/kvelakur/9069c9896577c3040030 . 

I think adding this feature would not require too much effort. 

@Willy, whats your opinion on this?


Best regards
Luca

On 05.09.19, 17:32, "li...@ltri.eu on behalf of Lukas Tribus"  
wrote:

Hello,

On Thu, Sep 5, 2019 at 4:58 PM Schimweg, Luca  wrote:
>
> Hey again,
>
> I tried to use rand, but when using it, my generation code for a UUID 
looks like this:
>
> unique-id-format 
%[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]
>
> To be honest I think for a simple UUID, we should not be required to but 
such a complicated and long line into the config. Also, I think that the 
performance impact of such a line is much bigger than the generation of a UUID 
in plain C code. We could just create a ACL generator called uuid or something 
similar which would do the job. The config would get much clearer.
>
> Any opinions to this? I would be willing to implement such a feature if 
the maintainers would be ready to merge it in general.

Are you proposing to implement a RFC4122 compliant UUID?
Which UUID version?
Which dependencies would it require?

I guess something RFC4122 compliant based on openssl for example would
work. Adding chunks of fragile code nobody is gonna maintain or adding
additional dependencies like libuuid - probably not.

I suggest we wait for Willy's feedback, but I want to stress that this
may not be as simple as 5 lines of standalone C code ... we certainly
don't want to act like we are generating standard compliant RFC4122
UUIDs when that is not really the case.


Lukas




Re: RFC uuid for log-format

2019-09-05 Thread Geoff Simmons
On 9/5/19 16:58, Schimweg, Luca wrote:
> 
> 
> unique-id-format 
> %[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]

This is going to be very nit-picky (I apologize): that comes close but
isn't actually guaranteed to produce an RFC 4122 version 4 UUID (the
"random" UUID).

You're right about the 4 at the beginning of the third group of hex
digits. But the first digit of the fourth group is required to be one of
8, 9, a or b. I've never understood why, but there it is.

Whether or not that matters, it seems to me, depends on requirements. If
you just need 16 bytes of randomness for a request ID, why not just
generate that? base64 encoding would be shorter, by the way -- 25 chars
for base64 with padding, 36 for the UUID format.

But if someone has an interoperability requirement dictating a UUID that
has to meet the standard format, then I agree that a fetch that gets it
just right would be warranted.


Best,
Geoff
-- 
** * * UPLEX - Nils Goroll Systemoptimierung

Scheffelstraße 32
22301 Hamburg

Tel +49 40 2880 5731
Mob +49 176 636 90917
Fax +49 40 42949753

http://uplex.de



signature.asc
Description: OpenPGP digital signature


Re: RFC uuid for log-format

2019-09-05 Thread Lukas Tribus
Hello,

On Thu, Sep 5, 2019 at 4:58 PM Schimweg, Luca  wrote:
>
> Hey again,
>
> I tried to use rand, but when using it, my generation code for a UUID looks 
> like this:
>
> unique-id-format 
> %[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]
>
> To be honest I think for a simple UUID, we should not be required to but such 
> a complicated and long line into the config. Also, I think that the 
> performance impact of such a line is much bigger than the generation of a 
> UUID in plain C code. We could just create a ACL generator called uuid or 
> something similar which would do the job. The config would get much clearer.
>
> Any opinions to this? I would be willing to implement such a feature if the 
> maintainers would be ready to merge it in general.

Are you proposing to implement a RFC4122 compliant UUID?
Which UUID version?
Which dependencies would it require?

I guess something RFC4122 compliant based on openssl for example would
work. Adding chunks of fragile code nobody is gonna maintain or adding
additional dependencies like libuuid - probably not.

I suggest we wait for Willy's feedback, but I want to stress that this
may not be as simple as 5 lines of standalone C code ... we certainly
don't want to act like we are generating standard compliant RFC4122
UUIDs when that is not really the case.


Lukas



Re: RFC uuid for log-format

2019-09-05 Thread Schimweg, Luca
Hey again,

I tried to use rand, but when using it, my generation code for a UUID looks 
like this: 

unique-id-format 
%[rand,hex,bytes(8,8),lower]-%[rand(65536),hex,bytes(12,4),lower]-4%[rand(4096),hex,bytes(13,3),lower]-%[rand(16384),add(32768),hex,bytes(12,4),lower]-%[rand(65536),hex,bytes(12,4),lower]%[rand,hex,bytes(8,8),lower]

To be honest I think for a simple UUID, we should not be required to but such a 
complicated and long line into the config. Also, I think that the performance 
impact of such a line is much bigger than the generation of a UUID in plain C 
code. We could just create a ACL generator called uuid or something similar 
which would do the job. The config would get much clearer.

Any opinions to this? I would be willing to implement such a feature if the 
maintainers would be ready to merge it in general.

Best regards
Luca

On 03.09.19, 10:34, "li...@ltri.eu on behalf of Lukas Tribus"  
wrote:

Hello Luca,

On Tue, 3 Sep 2019 at 09:18, Schimweg, Luca  wrote:
>
> Hey,
>
>
>
> for one use case I have, I would need a variable like %uuid in 
log-formats,
> which just generates a random UUID. The use-case would be, to be able
> to set the unique-id-format to this uuid, so that we can have a random 
uuid
> set as request-id for any incoming request. Right now, it’s quite 
difficult to
> get a random and unique request-id, the random uuid approach would
> definitely help with that.

I assume the rand [1] does not suffice. In this case, I'd suggest to
use LUA for this, maybe by using some library like lua-resty-jit-uuid
[2].


Lukas

[1] https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-rand
[2] https://github.com/thibaultcha/lua-resty-jit-uuid




Re: RFC uuid for log-format

2019-09-03 Thread Schimweg, Luca
Hey,

we are using a similar procedure right now, but we got the requirement to use 
the format of, or similar to, a uuid.

Best regards,
Luca

From: Conrad Hoffmann 
Sent: Tuesday, September 3, 2019 4:45:57 PM
To: Schimweg, Luca ; Lukas Tribus 
Cc: haproxy@formilux.org 
Subject: Re: RFC uuid for log-format

Hi,

On Tue, 2019-09-03 at 14:07 +, Schimweg, Luca wrote:
> thanks for mentioning rand, I didn't know about it... With some
> rand()s I was able to generate a UUID, I'll have to do some
> performance checks, but I think it will be fair enough. Then we don't
> need a %uuid or similar from my point of view.

FWIW, we use (among others) %Ts (timestamp), %pid (PID b/c multi-
process setup), %rc (retry count), and %rt (session request counter),
which produces some reasonably unique id, though our constraint is only
for it to be (mostly) unique across one server. You can also throw in
the server IP or name of course.

Cheers,
Conrad


>
> Thanks,
> Luca
>
> On 03.09.19, 10:34, "li...@ltri.eu on behalf of Lukas Tribus" <
> li...@ltri.eu> wrote:
>
> Hello Luca,
>
> On Tue, 3 Sep 2019 at 09:18, Schimweg, Luca <
> luca.schim...@sap.com> wrote:
> >
> > Hey,
> >
> >
> >
> > for one use case I have, I would need a variable like %uuid in
> log-formats,
> > which just generates a random UUID. The use-case would be, to
> be able
> > to set the unique-id-format to this uuid, so that we can have a
> random uuid
> > set as request-id for any incoming request. Right now, it’s
> quite difficult to
> > get a random and unique request-id, the random uuid approach
> would
> > definitely help with that.
>
> I assume the rand [1] does not suffice. In this case, I'd suggest
> to
> use LUA for this, maybe by using some library like lua-resty-jit-
> uuid
> [2].
>
>
> Lukas
>
> [1]
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-rand
> [2] https://github.com/thibaultcha/lua-resty-jit-uuid
>
>
--
Conrad Hoffmann
Systems and Traffic Engineering

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

Managing Director: Artem Fishman | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG
Charlottenburg  | HRB 110657B



Re: RFC uuid for log-format

2019-09-03 Thread Conrad Hoffmann
Hi,

On Tue, 2019-09-03 at 14:07 +, Schimweg, Luca wrote:
> thanks for mentioning rand, I didn't know about it... With some
> rand()s I was able to generate a UUID, I'll have to do some
> performance checks, but I think it will be fair enough. Then we don't
> need a %uuid or similar from my point of view.

FWIW, we use (among others) %Ts (timestamp), %pid (PID b/c multi-
process setup), %rc (retry count), and %rt (session request counter),
which produces some reasonably unique id, though our constraint is only
for it to be (mostly) unique across one server. You can also throw in
the server IP or name of course.

Cheers,
Conrad


> 
> Thanks,
> Luca
> 
> On 03.09.19, 10:34, "li...@ltri.eu on behalf of Lukas Tribus" <
> li...@ltri.eu> wrote:
> 
> Hello Luca,
> 
> On Tue, 3 Sep 2019 at 09:18, Schimweg, Luca <
> luca.schim...@sap.com> wrote:
> >
> > Hey,
> >
> >
> >
> > for one use case I have, I would need a variable like %uuid in
> log-formats,
> > which just generates a random UUID. The use-case would be, to
> be able
> > to set the unique-id-format to this uuid, so that we can have a
> random uuid
> > set as request-id for any incoming request. Right now, it’s
> quite difficult to
> > get a random and unique request-id, the random uuid approach
> would
> > definitely help with that.
> 
> I assume the rand [1] does not suffice. In this case, I'd suggest
> to
> use LUA for this, maybe by using some library like lua-resty-jit-
> uuid
> [2].
> 
> 
> Lukas
> 
> [1] 
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-rand
> [2] https://github.com/thibaultcha/lua-resty-jit-uuid
> 
> 
-- 
Conrad Hoffmann
Systems and Traffic Engineering

SoundCloud Ltd. | Rheinsberger Str. 76/77, 10115 Berlin, Germany

Managing Director: Artem Fishman | Incorporated in England & Wales
with Company No. 6343600 | Local Branch Office | AG
Charlottenburg  | HRB 110657B  




Re: RFC uuid for log-format

2019-09-03 Thread Schimweg, Luca
Hey,

thanks for mentioning rand, I didn't know about it... With some rand()s I was 
able to generate a UUID, I'll have to do some performance checks, but I think 
it will be fair enough. Then we don't need a %uuid or similar from my point of 
view.

Thanks,
Luca

On 03.09.19, 10:34, "li...@ltri.eu on behalf of Lukas Tribus"  
wrote:

Hello Luca,

On Tue, 3 Sep 2019 at 09:18, Schimweg, Luca  wrote:
>
> Hey,
>
>
>
> for one use case I have, I would need a variable like %uuid in 
log-formats,
> which just generates a random UUID. The use-case would be, to be able
> to set the unique-id-format to this uuid, so that we can have a random 
uuid
> set as request-id for any incoming request. Right now, it’s quite 
difficult to
> get a random and unique request-id, the random uuid approach would
> definitely help with that.

I assume the rand [1] does not suffice. In this case, I'd suggest to
use LUA for this, maybe by using some library like lua-resty-jit-uuid
[2].


Lukas

[1] https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-rand
[2] https://github.com/thibaultcha/lua-resty-jit-uuid




Re: RFC uuid for log-format

2019-09-03 Thread Lukas Tribus
Hello Luca,

On Tue, 3 Sep 2019 at 09:18, Schimweg, Luca  wrote:
>
> Hey,
>
>
>
> for one use case I have, I would need a variable like %uuid in log-formats,
> which just generates a random UUID. The use-case would be, to be able
> to set the unique-id-format to this uuid, so that we can have a random uuid
> set as request-id for any incoming request. Right now, it’s quite difficult to
> get a random and unique request-id, the random uuid approach would
> definitely help with that.

I assume the rand [1] does not suffice. In this case, I'd suggest to
use LUA for this, maybe by using some library like lua-resty-jit-uuid
[2].


Lukas

[1] https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#7.3.2-rand
[2] https://github.com/thibaultcha/lua-resty-jit-uuid



Re: RFC uuid for log-format

2019-09-03 Thread Schimweg, Luca
I would be open to implement this feature, as it is currently not possible to 
implement this behavior (at least afaik). Would there be interest in such a 
feature from the community?

From: "Schimweg, Luca" 
Date: Tuesday, 3. September 2019 at 09:19
To: "haproxy@formilux.org" 
Subject: [CAUTION] RFC uuid for log-format

Hey,

for one use case I have, I would need a variable like %uuid in log-formats, 
which just generates a random UUID. The use-case would be, to be able to set 
the unique-id-format to this uuid, so that we can have a random uuid set as 
request-id for any incoming request. Right now, it’s quite difficult to get a 
random and unique request-id, the random uuid approach would definitely help 
with that.

Best regards
Luca