Re: libcurl hangs in curl_multi_cleanup
On Wed, 23 Jun 2021, firefo...@interia.pl wrote: So, since 215db086e0 test660 ensures wrong behavior of libcurl. Since we try to have tests for *everything* curl can do, so the fact that we verify bugs is what is *supposed* to happen. Until we realize it is a bug, it isn't a bug and then it works as it should... -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2021-06-10, o godz. 17:48:26 Daniel Stenberg napisał(a): > On Thu, 10 Jun 2021, Firefox OS wrote: > > > The story is the same as 2 years ago > > You perceive it as the same, I do not. The previous issue was > reproduced with and then verified with test 660 and that test case > still runs fine today. > And now I see why it still runs fine today: in commit 215db086e0 you added "A002 LOGOUT" to the expected output :) While the commit message of 4225e5db1b which introduced test660 was: "... just makes sure LOGOUT is *not* issued on disconnect" So, since 215db086e0 test660 ensures wrong behavior of libcurl. -- --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Tue, 22 Jun 2021, FirefoxOS via curl-library wrote: Thanks for creating a test case. I've now made a PR that I believe fixes this issue: https://github.com/curl/curl/pull/7288 ... and the test 677 is included there and reproduces the bug before this fix. -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2021-06-18, o godz. 23:53:57
Daniel Stenberg napisał(a):
> On Fri, 18 Jun 2021, Firefox OS via curl-library wrote:
>
> > Do you mean imap://...? Yes, it does. And the callstack looks the
> > same.
>
> I cannot reproduce. I converted "curl_imap_teardown_issue.c" into
> test case 677 in this commit:
> https://github.com/curl/curl/commit/4401aede3133d1db9e3cc907b3c6640eda06c57a
>
> You can check out this branch:
>https://github.com/curl/curl/tree/bagder/multi_cleanup-imap-test
>
> And if you build the code and the tests run 677 like:
>
> cd tests && make -sj7 && ./runtests.pl 677
>
> Can you figure out what we need to change to make lib677.c hang?
>
Thanks for creating a test case.
The problem cannot be directly reproduced using curl test suite because
ftpserver.pl doesn't support IMAP IDLE mode (RFC2177). Please have a
look at repro1.patch which adds very limited support of idle mode to
ftpserver.pl (so that idle mode can be entered but not exited). The
result is now the same as I observe using a real IMAP server:
---cut here---
test 0677...[IMAP with CONNECT_ONLY, custom command then exit]
--pe-v- OK (1 out of 1 , remaining: 00:00, took 126.621s, duration:
02:06)
TESTDONE: 1 tests were considered during 127 seconds.
TESTDONE: 1 tests out of 1 reported OK: 100%
---cut here---
(Please also note that I replaced A1 IDLE\n with A1 IDLE\r\n because
ftpserver.pl doesn't seem to accept commands terminated with just \n,
contrary to the server I used before.)
What I wasn't aware of before I saw your test677 file is that
after libcurl "finishes" CONNECT_ONLY connection and passes
ACTIVE_SOCKET to the application, it still wants to proceed with A002
LOGOUT command (waiting for a "correct" answer) despite it has no idea
what the state of the connection can be and what the server will do
after receiving such string. Since the responsibility for the
conversation has been already transferred to the application, I expect
libcurl to avoid sending anything. I expressed this in repro2.patch
(forget about repro1.patch) and now, without touching ftpserver.pl, the
result is as follows:
---cut here---
test 0677...[IMAP with CONNECT_ONLY, custom command then exit]
677: protocol FAILED:
--- log/check-expected 2021-06-22 17:30:42.767593317 +0200
+++ log/check-generated 2021-06-22 17:30:42.767593317 +0200
@@ -1,2 +1,3 @@
A001 CAPABILITY[CR][LF]
A1 IDLE[CR][LF]
+A002 LOGOUT[CR][LF]
- abort tests
TESTDONE: 1 tests were considered during 7 seconds.
TESTDONE: 0 tests out of 1 reported OK: 0%
TESTFAIL: These test cases failed: 677
---cut here---
diff --git a/tests/data/test677 b/tests/data/test677
index 15526a67e..3f84dce81 100644
--- a/tests/data/test677
+++ b/tests/data/test677
@@ -37,7 +37,7 @@ imap://%HOSTIP:%IMAPPORT/677
A001 CAPABILITY
-A1 IDLE
+A1 IDLE
A002 LOGOUT
diff --git a/tests/ftpserver.pl b/tests/ftpserver.pl
index 9040478ec..6db38e1e8 100755
--- a/tests/ftpserver.pl
+++ b/tests/ftpserver.pl
@@ -643,6 +643,7 @@ sub protocolsetup {
'STATUS' => \_imap,
'STORE' => \_imap,
'UID'=> \_imap,
+'IDLE' => \_imap,
);
%displaytext = (
'welcome' => join("",
@@ -1587,6 +1588,16 @@ sub COPY_imap {
return 0;
}
+sub IDLE_imap {
+logmsg "IDLE_imap\n";
+
+sendcontrol "+ entering idle mode\r\n";
+%commandfunc = ();
+$proto = "imap-idle";
+
+return 0;
+}
+
sub UID_imap {
my ($args) = @_;
my ($command) = split(/ /, $args, 1);
@@ -3217,7 +3228,9 @@ while(1) {
}
}
else {
-sendcontrol "500 Unrecognized command\r\n";
+if ($proto ne "imap-idle") {
+sendcontrol "500 Unrecognized command\r\n";
+}
last;
}
diff --git a/tests/libtest/lib677.c b/tests/libtest/lib677.c
index 74675c882..316ddc988 100644
--- a/tests/libtest/lib677.c
+++ b/tests/libtest/lib677.c
@@ -25,7 +25,7 @@
#include "warnless.h"
#include "memdebug.h"
-static const char cmd[] = "A1 IDLE\n";
+static const char cmd[] = "A1 IDLE\r\n";
static char buf[1024];
int test(char *URL)
diff --git a/tests/data/test677 b/tests/data/test677
index 15526a67e..e3d9041fd 100644
--- a/tests/data/test677
+++ b/tests/data/test677
@@ -37,8 +37,7 @@ imap://%HOSTIP:%IMAPPORT/677
A001 CAPABILITY
-A1 IDLE
-A002 LOGOUT
+A1 IDLE
diff --git a/tests/libtest/lib677.c b/tests/libtest/lib677.c
index 74675c882..316ddc988 100644
--- a/tests/libtest/lib677.c
+++ b/tests/libtest/lib677.c
@@ -25,7 +25,7 @@
#include "warnless.h"
#include "memdebug.h"
-static const char cmd[] = "A1 IDLE\n";
+static const char cmd[] = "A1 IDLE\r\n";
static char buf[1024];
int test(char *URL)
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Fri, 18 Jun 2021, Firefox OS via curl-library wrote: Do you mean imap://...? Yes, it does. And the callstack looks the same. I cannot reproduce. I converted "curl_imap_teardown_issue.c" into test case 677 in this commit: https://github.com/curl/curl/commit/4401aede3133d1db9e3cc907b3c6640eda06c57a You can check out this branch: https://github.com/curl/curl/tree/bagder/multi_cleanup-imap-test And if you build the code and the tests run 677 like: cd tests && make -sj7 && ./runtests.pl 677 Can you figure out what we need to change to make lib677.c hang? -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2021-06-18, o godz. 09:31:42 Daniel Stenberg napisał(a): > On Thu, 17 Jun 2021, FirefoxOS via curl-library wrote: > > > Can you help me understand what does test660 actually check? > > > The issue is easily reproducible using curl_imap_teardown_issue.c > > Then clearly test660 doesn't run the same sequence. > > Does the curl_imap_teardown_issue.c code also cause a problem if run against > an IMAP server without using TLS? > Do you mean imap://...? Yes, it does. And the callstack looks the same. -- --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Thu, 17 Jun 2021, FirefoxOS via curl-library wrote: Can you help me understand what does test660 actually check? The issue is easily reproducible using curl_imap_teardown_issue.c Then clearly test660 doesn't run the same sequence. Does the curl_imap_teardown_issue.c code also cause a problem if run against an IMAP server without using TLS? -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2021-06-10, o godz. 17:48:26 Daniel Stenberg napisał(a): > On Thu, 10 Jun 2021, Firefox OS wrote: > > > The story is the same as 2 years ago > > You perceive it as the same, I do not. The previous issue was > reproduced with and then verified with test 660 and that test case > still runs fine today. > Hello, Can you help me understand what does test660 actually check? Do I understand correctly that it executes libtest/lib597.c? Does it check if the cleanup sequence documented at https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html hangs inside curl_multi_cleanup hangs? Or does it rather check only if the "connect only" phase occurs quickly, and then cleans up everything using "undocumented cleanup sequence"? The issue is easily reproducible using curl_imap_teardown_issue.c from https://curl.se/mail/lib-2019-04/0052.html with libcurl-7.76.1-4.fc34.x86_64, and also with current git version (80e1054fe5179c55104446a979369cdb3aceafc6) configured like this: ./configure --enable-debug --enable-imap --with-openssl I'm using URL imaps://poczta.interia.pl and the same account I'm mailing this from (the connection succeeds and idle mode is entered properly). Under gdb I see that curl_multi_cleanup() tries to disconnect some strange connection from connection cache having connect_only=0, so dead_connection=false and the fix mentioned earlier (Curl_disconnect: treat all CONNECT_ONLY connections as "dead") doesn't take effect. Thanks for help. Best regards -- --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Thu, 10 Jun 2021, Firefox OS wrote: The story is the same as 2 years ago You perceive it as the same, I do not. The previous issue was reproduced with and then verified with test 660 and that test case still runs fine today. -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2021-06-10, o godz. 08:14:05 Daniel Stenberg napisał(a): > On Thu, 10 Jun 2021, Firefox OS via curl-library wrote: > > > Unfortunately the bug is there again. > > At least in 7.76.1 and 7.70.0. > > Particularly, curl_multi_cleanup() hangs for exactly 2 minutes. > > Please tell us the whole story. > > What do you do and when does it hang? We want to be able to reproduce. > > What curl version with what components? > Hi, The story is the same as 2 years ago, that's why I only replied to that thread. Please have a look at https://curl.se/mail/lib-2019-04/0052.html. The program curl_imap_teardown_issue.c attached there is now able to reproduce the original problem again (hung curl_multi_cleanup) but with curl library v7.76.1 shipped with Fedora 34. I also see hung curl_multi_cleanup() with cross-compiled curl library in latest version (7.77.0, not 7.70.0 - sorry), and I guess the issue is the same as in 7.76.1, although I'm not 100% sure. --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Thu, 10 Jun 2021, Firefox OS via curl-library wrote: Unfortunately the bug is there again. At least in 7.76.1 and 7.70.0. Particularly, curl_multi_cleanup() hangs for exactly 2 minutes. Please tell us the whole story. What do you do and when does it hang? We want to be able to reproduce. What curl version with what components? -- / daniel.haxx.se | Commercial curl support up to 24x7 is available! | Private help, bug fixes, support, ports, new features | https://www.wolfssl.com/contact/ --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2019-05-06, o godz. 23:59:25 Firefox OS via curl-library napisał(a): > Dnia 2019-04-21, o godz. 23:21:07 > Daniel Stenberg via curl-library napisał(a): > > > On Fri, 19 Apr 2019, Daniel Stenberg via curl-library wrote: > > > > > My proposed fix for this issue is now here: > > > > > > https://github.com/curl/curl/pull/3795 > > > > That one landed in git too and will ship in the next release. > > > > Hi, > > I've rebuilt libcurl-7.64.1 patched with just > https://github.com/curl/curl/commit/e06b8bdae6a6e552d7430c00853d18b9378eac29 > and the issue seems to be gone. > > Thanks Hi, Unfortunately the bug is there again. At least in 7.76.1 and 7.70.0. Particularly, curl_multi_cleanup() hangs for exactly 2 minutes. #0 0x77e355bf in poll () from /lib64/libc.so.6 #1 0x77f5c61e in Curl_poll () from /lib64/libcurl.so.4 #2 0x77f5c756 in Curl_socket_check () from /lib64/libcurl.so.4 #3 0x77f550de in Curl_pp_statemach () from /lib64/libcurl.so.4 #4 0x77f4bb7d in imap_disconnect.lto_priv () from /lib64/libcurl.so.4 #5 0x77f6b53f in Curl_disconnect () from /lib64/libcurl.so.4 #6 0x77f24917 in Curl_conncache_close_all_connections () from /lib64/libcurl.so.4 #7 0x77f5308b in curl_multi_cleanup () from /lib64/libcurl.so.4 #8 0x0040139a in main (argc=, argv=) at curl_imap_teardown_issue.c:89 -- --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
Dnia 2019-04-21, o godz. 23:21:07 Daniel Stenberg via curl-library napisał(a): > On Fri, 19 Apr 2019, Daniel Stenberg via curl-library wrote: > > > My proposed fix for this issue is now here: > > > > https://github.com/curl/curl/pull/3795 > > That one landed in git too and will ship in the next release. > Hi, I've rebuilt libcurl-7.64.1 patched with just https://github.com/curl/curl/commit/e06b8bdae6a6e552d7430c00853d18b9378eac29 and the issue seems to be gone. Thanks --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Fri, 19 Apr 2019, Daniel Stenberg via curl-library wrote: My proposed fix for this issue is now here: https://github.com/curl/curl/pull/3795 That one landed in git too and will ship in the next release. -- / daniel.haxx.se --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
Re: libcurl hangs in curl_multi_cleanup
On Wed, 17 Apr 2019, Firefox OS via curl-library wrote: The program logs in to an imaps:// server (using CURLOPT_CONNECT_ONLY), then enters idle mode using curl_easy_send and receives answer using curl_easy_recv. Then sleeps for a few seconds and tries to clean up everything and quit. I followed the order given at https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html but still curl_multi_cleanup() doesn't exit for a long time. The issue occurs on libcurl-7.61.1 as well as on latest release curl-7.64.1. Am I doing something wrong, or is it a bug? It looks like a bug. I would guess because of the CONNECT_ONLY + IMAP combination. After a CONNECT_ONLY connection, the shut down if that connection can't be done by the IMAP rules anymore since libcurl has lost the IMAP state and it can only really diconnect the connection. I'll look into it. -- / daniel.haxx.se --- Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library Etiquette: https://curl.haxx.se/mail/etiquette.html
libcurl hangs in curl_multi_cleanup
Hi all
I wrote a simplified program (attached) to demonstrate a problem I'm dealing
with.
The problem is that libcurl doesn't allow the program to quit because it hangs
inside curl_multi_cleanup:
(gdb) bt
#0 0x77e463f1 in poll () from /lib64/libc.so.6
#1 0x77f58a46 in poll (__timeout=1000, __nfds=,
__fds=0x7fffdc00) at /usr/include/bits/poll2.h:41
#2 Curl_socket_check (readfd0=readfd0@entry=3, readfd1=readfd1@entry=-1,
writefd=-1, timeout_ms=, timeout_ms@entry=1000)
at ../../lib/select.c:217
#3 0x77f66347 in Curl_pp_statemach (pp=pp@entry=0x422370,
block=block@entry=true) at ../../lib/pingpong.c:112
#4 0x77f61121 in imap_block_statemach (conn=)
at ../../lib/imap.c:1365
#5 imap_disconnect (conn=0x421cc0, dead_connection=)
at ../../lib/imap.c:1624
#6 0x77f3e15d in Curl_disconnect (data=0x4175c0,
conn=conn@entry=0x421cc0, dead_connection=dead_connection@entry=false)
at ../../lib/url.c:781
#7 0x77f6e8f3 in Curl_conncache_close_all_connections (
connc=connc@entry=0x409d70) at ../../lib/conncache.c:573
#8 0x77f535d1 in curl_multi_cleanup (multi=multi@entry=0x409c60)
at ../../lib/multi.c:2244
#9 0x0040139a in main (argc=, argv=)
at curl_imap_teardown_issue.c:89
The program logs in to an imaps:// server (using CURLOPT_CONNECT_ONLY), then
enters idle mode using curl_easy_send and receives answer using curl_easy_recv.
Then sleeps for a few seconds and tries to clean up everything and quit. I
followed the order given at
https://curl.haxx.se/libcurl/c/curl_multi_cleanup.html but still
curl_multi_cleanup() doesn't exit for a long time.
The issue occurs on libcurl-7.61.1 as well as on latest release curl-7.64.1.
Am I doing something wrong, or is it a bug?
--
#include
#include
#include
#include
static const char cmd[] = "A1 IDLE\n";
static char buf[1024];
int main(int argc, char **argv)
{
CURLM *mcurl;
CURL *curl;
int mrun, sock = CURL_SOCKET_BAD;
time_t start = time(NULL);
int state = 0;
ssize_t pos = 0;
if (argc < 2) {
fprintf(stderr, "Usage: %s imaps://mailserver.com u...@mailserver.com password\n", argv[0]);
return 1;
}
assert((mcurl = curl_multi_init()));
assert((curl = curl_easy_init()));
assert(curl_easy_setopt(curl, CURLOPT_VERBOSE, 1L) == CURLE_OK);
assert(curl_easy_setopt(curl, CURLOPT_URL, argv[1]) == CURLE_OK);
if (argc > 2)
assert(curl_easy_setopt(curl, CURLOPT_USERNAME, argv[2]) == CURLE_OK);
if (argc > 3)
assert(curl_easy_setopt(curl, CURLOPT_PASSWORD, argv[3]) == CURLE_OK);
assert(curl_easy_setopt(curl, CURLOPT_CONNECT_ONLY, 1L) == CURLE_OK);
//assert(curl_easy_setopt(curl, CURLOPT_MAXCONNECTS, 1L) == CURLE_OK);
assert(curl_multi_add_handle(mcurl, curl) == CURLM_OK);
while (time(NULL) - start < 5) {
struct curl_waitfd waitfd;
assert(curl_multi_perform(mcurl, ) == CURLM_OK);
for (;;) {
int i;
struct CURLMsg *m = curl_multi_info_read(mcurl, );
if (!m)
break;
if (m->msg == CURLMSG_DONE && m->easy_handle == curl) {
assert(curl_easy_getinfo(curl, CURLINFO_ACTIVESOCKET, ) == CURLE_OK);
assert(sock != CURL_SOCKET_BAD);
}
}
if (sock >= 0) {
waitfd.events = state ? CURL_WAIT_POLLIN : CURL_WAIT_POLLOUT;
waitfd.revents = 0;
assert(curl_easy_getinfo(curl, CURLINFO_ACTIVESOCKET, ) == CURLE_OK);
waitfd.fd = sock;
}
assert(curl_multi_wait(mcurl, , sock < 0 ? 0 : 1, 5000, ) == CURLM_OK);
if (sock >= 0 && (waitfd.revents & waitfd.events)) {
size_t len = 0;
if (!state) {
assert(curl_easy_send(curl, cmd + pos, sizeof(cmd) - 1 - pos, ) == CURLE_OK);
if (len > 0)
pos += len;
else
pos = 0;
if (pos == sizeof(cmd) - 1) {
state++;
pos = 0;
}
} else if (pos < sizeof(buf)) {
assert(curl_easy_recv(curl, buf + pos, sizeof(buf) - pos, ) == CURLE_OK);
if (len > 0)
pos += len;
}
if (len <= 0)
sock = -1;
}
}
if (state) {
fwrite(buf, pos, 1, stdout);
putchar('\n');
}
assert(curl_multi_remove_handle(mcurl, curl) == CURLM_OK);
curl_easy_cleanup(curl);
curl_multi_cleanup(mcurl);
return 0;
}
---
Unsubscribe: https://cool.haxx.se/list/listinfo/curl-library
Etiquette: https://curl.haxx.se/mail/etiquette.html
