Re: [OE-core] [PATCH v2] curl: Update from 8.2.1 to 8.3.0

2023-09-27 Thread Robert Joslyn via lists.openembedded.org 

On 9/26/23 8:51 PM, Khem Raj wrote:

On Tue, Sep 26, 2023 at 7:55 PM Robert Joslyn
 wrote:

On 9/26/23 7:55 AM, Khem Raj wrote:

I am seeing a ptest failure on qemx86-64/glibc

Failed ptests:
{'curl': ['test_1474', 'curl']}

In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause
# the test to fail), this test is marked flaky to avoid it being run in
the CI
# builds which are often run on overloaded servers.

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that
could explain why I haven't seen it fail normally. I was able to induce
a failure by running a very heavy load while running that test on my
desktop (which is older and slower than my build server). If you'd like,
I can send a follow up patch to skip tests marked as flaky.


another run resulted in yet another failure

   test 0587...
FAIL: 587: protoc!
There was no content at all in the file log/2/server.input.
Server glitch? Total curl failure? Returned: 42

prior to this upgrade my test setup never saw curl failures and it has
same amount of load etc.


Test 587 is also marked flaky by upstream. There are 12 tests marked as 
flaky total, and since you've just hit two of them, I'm much more 
inclined to just skip them. Right before this version bump there was 
also an update to run the curl ptests in parallel, which could be 
contributing to the change in test behavior.


Thanks,

Robert


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188321): 
https://lists.openembedded.org/g/openembedded-core/message/188321
Mute This Topic: https://lists.openembedded.org/mt/101543494/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] curl: Update from 8.2.1 to 8.3.0

2023-09-26 Thread Khem Raj 
On Tue, Sep 26, 2023 at 7:55 PM Robert Joslyn
 wrote:
>
> On 9/26/23 7:55 AM, Khem Raj wrote:
> > I am seeing a ptest failure on qemx86-64/glibc
> >
> >Failed ptests:
> >{'curl': ['test_1474', 'curl']}
>
> In looking at the test, it is marked as flaky with the comment:
>
> # Because of the timing sensitivity (scheduling delays of 500 msec can cause
> # the test to fail), this test is marked flaky to avoid it being run in
> the CI
> # builds which are often run on overloaded servers.
>
> https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21
>
> My server is pretty lightly loaded while running the tests, so that
> could explain why I haven't seen it fail normally. I was able to induce
> a failure by running a very heavy load while running that test on my
> desktop (which is older and slower than my build server). If you'd like,
> I can send a follow up patch to skip tests marked as flaky.
>

another run resulted in yet another failure

  test 0587...
   FAIL: 587: protoc!
   There was no content at all in the file log/2/server.input.
   Server glitch? Total curl failure? Returned: 42

prior to this upgrade my test setup never saw curl failures and it has
same amount of load etc.


> Thanks,
>
> Robert
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188290): 
https://lists.openembedded.org/g/openembedded-core/message/188290
Mute This Topic: https://lists.openembedded.org/mt/101543494/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] curl: Update from 8.2.1 to 8.3.0

2023-09-26 Thread Robert Joslyn via lists.openembedded.org 

On 9/26/23 7:55 AM, Khem Raj wrote:

I am seeing a ptest failure on qemx86-64/glibc

   Failed ptests:
   {'curl': ['test_1474', 'curl']}


In looking at the test, it is marked as flaky with the comment:

# Because of the timing sensitivity (scheduling delays of 500 msec can cause
# the test to fail), this test is marked flaky to avoid it being run in 
the CI

# builds which are often run on overloaded servers.

https://github.com/curl/curl/blob/curl-8_3_0/tests/data/test1474#L21

My server is pretty lightly loaded while running the tests, so that 
could explain why I haven't seen it fail normally. I was able to induce 
a failure by running a very heavy load while running that test on my 
desktop (which is older and slower than my build server). If you'd like, 
I can send a follow up patch to skip tests marked as flaky.


Thanks,

Robert


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#188287): 
https://lists.openembedded.org/g/openembedded-core/message/188287
Mute This Topic: https://lists.openembedded.org/mt/101543494/21656
Group Owner: openembedded-core+ow...@lists.openembedded.org
Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [OE-core] [PATCH v2] curl: Update from 8.2.1 to 8.3.0

2023-09-26 Thread Khem Raj 
I am seeing a ptest failure on qemx86-64/glibc

  Failed ptests:
  {'curl': ['test_1474', 'curl']}

On Sat, Sep 23, 2023 at 10:25 AM Robert Joslyn via
lists.openembedded.org
 wrote:
>
> From: Robert Joslyn 
>
> NSS support was removed, so adjust PACKAGECONFIG options.
>
> The --enable-crypto-auth option was removed and split into separate
> options for basic-auth, bearer-auth, digest-auth, kerberos-auth,
> negotiate-auth, and aws. Enable these new options since upstream enables
> them by default.
>
> Disable test 1279 since this requires libcurl and hangs the tests.
>
> Signed-off-by: Robert Joslyn 
> ---
> Changes from v1:
> Enable aws authentication by default. Upstream enables it by default and
> test 1964 fails with it disabled.
>
>  meta/recipes-support/curl/curl/disable-tests   |  1 +
>  .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}  | 14 +-
>  2 files changed, 10 insertions(+), 5 deletions(-)
>  rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)
>
> diff --git a/meta/recipes-support/curl/curl/disable-tests 
> b/meta/recipes-support/curl/curl/disable-tests
> index b687b2bb76..fdac795662 100644
> --- a/meta/recipes-support/curl/curl/disable-tests
> +++ b/meta/recipes-support/curl/curl/disable-tests
> @@ -21,6 +21,7 @@
>  # This test is scanning the source tree
>  1222
>  # These CRL tests need --libcurl option to be enabled
> +1279
>  1400
>  1401
>  1402
> diff --git a/meta/recipes-support/curl/curl_8.2.1.bb 
> b/meta/recipes-support/curl/curl_8.3.0.bb
> similarity index 88%
> rename from meta/recipes-support/curl/curl_8.2.1.bb
> rename to meta/recipes-support/curl/curl_8.3.0.bb
> index b86c6f3bba..646ac01914 100644
> --- a/meta/recipes-support/curl/curl_8.2.1.bb
> +++ b/meta/recipes-support/curl/curl_8.3.0.bb
> @@ -14,7 +14,7 @@ SRC_URI = " \
>  file://run-ptest \
>  file://disable-tests \
>  "
> -SRC_URI[sha256sum] = 
> "dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
> +SRC_URI[sha256sum] = 
> "376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
>
>  # Curl has used many names over the years...
>  CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl 
> daniel_stenberg:curl"
> @@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header 
> ptest
>  # Entropy source for random PACKAGECONFIG option
>  RANDOM ?= "/dev/urandom"
>
> -PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn 
> openssl proxy random threaded-resolver verbose zlib"
> +PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} aws 
> basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random 
> threaded-resolver verbose zlib"
>  PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver 
> verbose zlib"
>  PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver 
> verbose zlib"
>
>  # 'ares' and 'threaded-resolver' are mutually exclusive
>  PACKAGECONFIG[ares] = 
> "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
> +PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
> +PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
> +PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
>  PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
>  PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
>  # Don't use this in production
>  PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
>  PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
> +PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
>  PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
>  PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
>  PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
>  PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
> +PACKAGECONFIG[kerberos-auth] = 
> "--enable-kerberos-auth,--disable-kerberos-auth"
>  PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
>  PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
>  PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
> @@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = 
> "--with-libidn2,--without-libidn2,libidn2"
>  PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
>  PACKAGECONFIG[mbedtls] = 
> "--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
>  PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
> +PACKAGECONFIG[negotiate-auth] = 
> "--enable-negotiate-auth,--disable-negotiate-auth"
>  PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
>  PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
>  PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
> @@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = 
> "--with-librtmp,--without-librtmp,rtmpdump"
>  PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
>  PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
>  PACKAGECONFIG[smtp] =

[OE-core] [PATCH v2] curl: Update from 8.2.1 to 8.3.0

2023-09-23 Thread Robert Joslyn via lists.openembedded.org 
From: Robert Joslyn 

NSS support was removed, so adjust PACKAGECONFIG options.

The --enable-crypto-auth option was removed and split into separate
options for basic-auth, bearer-auth, digest-auth, kerberos-auth,
negotiate-auth, and aws. Enable these new options since upstream enables
them by default.

Disable test 1279 since this requires libcurl and hangs the tests.

Signed-off-by: Robert Joslyn 
---
Changes from v1:
Enable aws authentication by default. Upstream enables it by default and
test 1964 fails with it disabled.

 meta/recipes-support/curl/curl/disable-tests   |  1 +
 .../curl/{curl_8.2.1.bb => curl_8.3.0.bb}  | 14 +-
 2 files changed, 10 insertions(+), 5 deletions(-)
 rename meta/recipes-support/curl/{curl_8.2.1.bb => curl_8.3.0.bb} (88%)

diff --git a/meta/recipes-support/curl/curl/disable-tests 
b/meta/recipes-support/curl/curl/disable-tests
index b687b2bb76..fdac795662 100644
--- a/meta/recipes-support/curl/curl/disable-tests
+++ b/meta/recipes-support/curl/curl/disable-tests
@@ -21,6 +21,7 @@
 # This test is scanning the source tree
 1222
 # These CRL tests need --libcurl option to be enabled
+1279
 1400
 1401
 1402
diff --git a/meta/recipes-support/curl/curl_8.2.1.bb 
b/meta/recipes-support/curl/curl_8.3.0.bb
similarity index 88%
rename from meta/recipes-support/curl/curl_8.2.1.bb
rename to meta/recipes-support/curl/curl_8.3.0.bb
index b86c6f3bba..646ac01914 100644
--- a/meta/recipes-support/curl/curl_8.2.1.bb
+++ b/meta/recipes-support/curl/curl_8.3.0.bb
@@ -14,7 +14,7 @@ SRC_URI = " \
 file://run-ptest \
 file://disable-tests \
 "
-SRC_URI[sha256sum] = 
"dd322f6bd0a20e6cebdfd388f69e98c3d183bed792cf4713c8a7ef498cba4894"
+SRC_URI[sha256sum] = 
"376d627767d6c4f05105ab6d497b0d9aba7111770dd9d995225478209c37ea63"
 
 # Curl has used many names over the years...
 CVE_PRODUCT = "haxx:curl haxx:libcurl curl:curl curl:libcurl libcurl:libcurl 
daniel_stenberg:curl"
@@ -24,21 +24,26 @@ inherit autotools pkgconfig binconfig multilib_header ptest
 # Entropy source for random PACKAGECONFIG option
 RANDOM ?= "/dev/urandom"
 
-PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} libidn 
openssl proxy random threaded-resolver verbose zlib"
+PACKAGECONFIG ??= "${@bb.utils.filter('DISTRO_FEATURES', 'ipv6', d)} aws 
basic-auth bearer-auth digest-auth negotiate-auth libidn openssl proxy random 
threaded-resolver verbose zlib"
 PACKAGECONFIG:class-native = "ipv6 openssl proxy random threaded-resolver 
verbose zlib"
 PACKAGECONFIG:class-nativesdk = "ipv6 openssl proxy random threaded-resolver 
verbose zlib"
 
 # 'ares' and 'threaded-resolver' are mutually exclusive
 PACKAGECONFIG[ares] = "--enable-ares,--disable-ares,c-ares,,,threaded-resolver"
+PACKAGECONFIG[aws] = "--enable-aws,--disable-aws"
+PACKAGECONFIG[basic-auth] = "--enable-basic-auth,--disable-basic-auth"
+PACKAGECONFIG[bearer-auth] = "--enable-bearer-auth,--disable-bearer-auth"
 PACKAGECONFIG[brotli] = "--with-brotli,--without-brotli,brotli"
 PACKAGECONFIG[builtinmanual] = "--enable-manual,--disable-manual"
 # Don't use this in production
 PACKAGECONFIG[debug] = "--enable-debug,--disable-debug"
 PACKAGECONFIG[dict] = "--enable-dict,--disable-dict,"
+PACKAGECONFIG[digest-auth] = "--enable-digest-auth,--disable-digest-auth"
 PACKAGECONFIG[gnutls] = "--with-gnutls,--without-gnutls,gnutls"
 PACKAGECONFIG[gopher] = "--enable-gopher,--disable-gopher,"
 PACKAGECONFIG[imap] = "--enable-imap,--disable-imap,"
 PACKAGECONFIG[ipv6] = "--enable-ipv6,--disable-ipv6,"
+PACKAGECONFIG[kerberos-auth] = "--enable-kerberos-auth,--disable-kerberos-auth"
 PACKAGECONFIG[krb5] = "--with-gssapi,--without-gssapi,krb5"
 PACKAGECONFIG[ldap] = "--enable-ldap,--disable-ldap,openldap"
 PACKAGECONFIG[ldaps] = "--enable-ldaps,--disable-ldaps,openldap"
@@ -47,6 +52,7 @@ PACKAGECONFIG[libidn] = 
"--with-libidn2,--without-libidn2,libidn2"
 PACKAGECONFIG[libssh2] = "--with-libssh2,--without-libssh2,libssh2"
 PACKAGECONFIG[mbedtls] = 
"--with-mbedtls=${STAGING_DIR_TARGET},--without-mbedtls,mbedtls"
 PACKAGECONFIG[mqtt] = "--enable-mqtt,--disable-mqtt,"
+PACKAGECONFIG[negotiate-auth] = 
"--enable-negotiate-auth,--disable-negotiate-auth"
 PACKAGECONFIG[nghttp2] = "--with-nghttp2,--without-nghttp2,nghttp2"
 PACKAGECONFIG[openssl] = "--with-openssl,--without-openssl,openssl"
 PACKAGECONFIG[pop3] = "--enable-pop3,--disable-pop3,"
@@ -56,7 +62,6 @@ PACKAGECONFIG[rtmpdump] = 
"--with-librtmp,--without-librtmp,rtmpdump"
 PACKAGECONFIG[rtsp] = "--enable-rtsp,--disable-rtsp,"
 PACKAGECONFIG[smb] = "--enable-smb,--disable-smb,"
 PACKAGECONFIG[smtp] = "--enable-smtp,--disable-smtp,"
-PACKAGECONFIG[nss] = "--with-nss,--without-nss,nss"
 PACKAGECONFIG[telnet] = "--enable-telnet,--disable-telnet,"
 PACKAGECONFIG[tftp] = "--enable-tftp,--disable-tftp,"
 PACKAGECONFIG[threaded-resolver] = 
"--enable-threaded-resolver,--disable-threaded-resolverares"
@@ -67,11 +72,10 @@ PACKAGECONFIG[zstd] = "--with-zstd,--without-zstd,zstd"
 EXTRA_OECONF = " \