Re: libksane seems to break QProcess::start calls
On Saturday, 5 March 2022 00:31:43 PST Tobias Leupold wrote: > f it's only the SANE backend that breaks this, you're of course right. Unfortunately, we can't be sure of that. It's hard to get Unix signals right (we're discussing another instance of it in Qt right now, on [1]) > Within what I could grasp, I filed an issue: > https://gitlab.com/sane-project/backends/-/issues/582 > > I fear fixing this is far beyond my programming skills. I hardly understand > what's going on here at all ... I'll add some comments. [1] https://codereview.qt-project.org/c/qt/qtbase/+/380826 -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel DPG Cloud Engineering
Re: libksane seems to break QProcess::start calls
> It shouldn't have any side-effects. In fact, the code exists to *avoid* > side-effects in the first place. > > Of course, it assumes that there isn't buggy code elsewhere. If there is, > then all bets are lost. I'm not going to document how to bugfix other code, > especially complex code like this. And we've already established that the > SANE backend is buggy. Sorry, I got that wrong. I thought that always if one subclasses a QProcess, one would have to care about implementing some virtual functions to get it working correctly. If it's only the SANE backend that breaks this, you're of course right. > > If you're really bored some time, maybe you want to file an issue for the > > SANE Pixma backend and tell the devs what exactly is wrong there and/or > > how to fix this? > > I'm not. I'll explain if you file or if you want to actually fix, but I > don't have that much free time available. Within what I could grasp, I filed an issue: https://gitlab.com/sane-project/backends/-/issues/582 I fear fixing this is far beyond my programming skills. I hardly understand what's going on here at all ... > Well, if you use QProcess to call the full SANE backend, that will also > avoid the problem. Yeah, maybe if I would acquire the images not by using libksane, but by calling scanimage via QProcess, I would not have any problem. But apart from losing the functionality I wanted to implement (setting the scanner options in a nice GUI way) that would only work around the real problem, which is a buggy implementation of the plustek backend, wouldn't it? > "If you write buggy code, the application may misbehave" is not > documentation. Let's hope the SANE devs fix it ;-)
Re: libksane seems to break QProcess::start calls
On Friday, 4 March 2022 13:47:07 PST Tobias Leupold wrote: > I don't know if this would be still possible for the Qt 5 docs, but a small > hint like "If you subclass this, be aware that ..." woudn't hurt ... As > said, I would never have thought that this would have such side-effects! It shouldn't have any side-effects. In fact, the code exists to *avoid* side- effects in the first place. Of course, it assumes that there isn't buggy code elsewhere. If there is, then all bets are lost. I'm not going to document how to bugfix other code, especially complex code like this. And we've already established that the SANE backend is buggy. > > Anyway, please note that simply your problem still exists even without > > subclassing on kernels older than 5.4. For those, we have to use the > > SIGCHLD handler anyway. This must be fixed in the SANE backend. > > If you're really bored some time, maybe you want to file an issue for the > SANE Pixma backend and tell the devs what exactly is wrong there and/or how > to fix this? I'm not. I'll explain if you file or if you want to actually fix, but I don't have that much free time available. > > Or you can work around it by doing what it is doing: fork(). > > Well, I think, if I'm safe with Qt >=5.15 and Kernel >=5.4, for now, I'd > stick with the simple implementation that works now. Forking some part of a > Qt program is something I even know less about than what can happen if I > call external programs via QProcess ... I'm quite happy to have it in a > working state now! Well, if you use QProcess to call the full SANE backend, that will also avoid the problem. > Surely, the fact that some SANE backends may cause issues with process calls > with older kernels should be mentioned in the still-to-write docs. But at > the moment, I think I should first see if anybody besides me ever considers > the project to be meaningful and if it would be worth it to make it safe > for older environments. Although it may be worth it to mess with this stuff > to learn more about process handling, threads and forks and so on. "If you write buggy code, the application may misbehave" is not documentation. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel DPG Cloud Engineering
Re: libksane seems to break QProcess::start calls
> It's a side-effect. The problem is the QProcess::setupChildProcess virtual > in Qt 5, that had been there since QProcess was introduced the the > Paleolithic Era. When we use CLONE_PIDFD, we have to call clone() directly, > which means the hooks installed by pthread_atfork() do not get run. In > particular, this may mean the mutexes locked by other threads are all still > locked, including one inside malloc(). That would mean the user's code in > setupChildProcess() could deadlock depending on the kernel version. Thanks again for clarifying this! > ... So it was simplified to "if you derive, assume setupChildProcess > was overridden." I don't know if this would be still possible for the Qt 5 docs, but a small hint like "If you subclass this, be aware that ..." woudn't hurt ... As said, I would never have thought that this would have such side-effects! Sure, Qt 6 has been out for some months now, but I think Qt 5 may be still used for quite some time. Here at Gentoo, we e.g. don't even have Qt 6 in the main package repository yet, and normally, Gentoo is not much hesitant to add new versions ... but that's another issue. > Anyway, please note that simply your problem still exists even without > subclassing on kernels older than 5.4. For those, we have to use the SIGCHLD > handler anyway. This must be fixed in the SANE backend. If you're really bored some time, maybe you want to file an issue for the SANE Pixma backend and tell the devs what exactly is wrong there and/or how to fix this? > Or you can work around it by doing what it is doing: fork(). Well, I think, if I'm safe with Qt >=5.15 and Kernel >=5.4, for now, I'd stick with the simple implementation that works now. Forking some part of a Qt program is something I even know less about than what can happen if I call external programs via QProcess ... I'm quite happy to have it in a working state now! Surely, the fact that some SANE backends may cause issues with process calls with older kernels should be mentioned in the still-to-write docs. But at the moment, I think I should first see if anybody besides me ever considers the project to be meaningful and if it would be worth it to make it safe for older environments. Although it may be worth it to mess with this stuff to learn more about process handling, threads and forks and so on. I think, first, I'll get the whole thing in shape with a program icon, installation target, message extraction script and so on. So that it _could_ be used. And then, I'll see if anybody actually wants to use it ;-)
Re: libksane seems to break QProcess::start calls
On Friday, 4 March 2022 10:52:00 PST Tobias Leupold wrote: > I simply made this a QObject subclass and rather then using start() etc. > directly, I created a QProcess object and did the same stuff with this one. > > No freezes anymore. No matter what scanner I use. > > I never thought subclassing the QProcess would change the behavior in such a > radical way :-O Is there some passage in the docs that says "Never ever > subclass a QProcess unless you exactly know what you are doing, otherwise, > you will experience the weirdest problems"? :-D It's a side-effect. The problem is the QProcess::setupChildProcess virtual in Qt 5, that had been there since QProcess was introduced the the Paleolithic Era. When we use CLONE_PIDFD, we have to call clone() directly, which means the hooks installed by pthread_atfork() do not get run. In particular, this may mean the mutexes locked by other threads are all still locked, including one inside malloc(). That would mean the user's code in setupChildProcess() could deadlock depending on the kernel version. To avoid this possibility, we needed to know if you'd overridden setupChildProcess(). When I introduced the CLONE_PIDFD content, we originally added a very hacky way of detecting that, but that was way too fragile. So it was simplified to "if you derive, assume setupChildProcess was overridden." Qt 6 removed this virtual and replaced it with a callback using std::function (QProcess::setChildProcessModifier). So we can be much more sure that you have code that you want to run on the child side before execve(). Anyway, please note that simply your problem still exists even without subclassing on kernels older than 5.4. For those, we have to use the SIGCHLD handler anyway. This must be fixed in the SANE backend. Or you can work around it by doing what it is doing: fork(). Run the SANE backend entirely in a child process, so it can't affect the main application's SIGCHLD handler. If you need to share memory, you can do that by memory- mapping a file before fork(), as that gets shared between parent and child. A memfd would be ideal, but if that fails, you can use a regular QTemporaryFile. On Linux, QSharedMemory is just a memory-mapped file on a tmpfs (/dev/shm) after all, only it's hiding behind two or three layers of abstraction. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel DPG Cloud Engineering
Re: libksane seems to break QProcess::start calls
Hey Thiago, Am Freitag, 4. März 2022, 18:35:23 CET schrieb Thiago Macieira: > And this is the reason why you got the symptom of QProcess breaking. We'd > have expected that the eight mention be the same as the fifth, to keep the > pattern going, but it isn't. It's the same as the second and sixth > mentions. The SIGCHLD handler that ran is that stupidly short one, which we > now know to be definitely buggy. Thanks a lot for the comprehensive analysis! I don't grasp this compleley -- or, to be honest, not at all ... most likely, one has to be a real programmer to do so ;-) But it's definitely quite impressive how you can read this from the trace! But this: Am Freitag, 4. März 2022, 18:02:37 CET schrieb Thiago Macieira: > So the question is why this brscan didn't even attempt to use pidfd. There's > some code in Qt5's QProcess to attempt to detect whether you've subclassed > QProcess and skip using the pidfd feature: > > if (typeid(*q) != typeid(QProcess)) > ffdflags |= FFD_USE_FORK; > > https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qprocess_unix.cpp? > h=5.15#n462 made me try one thing: I used a small helper class subclassing QProcess to execute the external commands. It did not much, just took a command line like you would type it in a shell, along with the in and out file, extracted the program parameter from it and assembled the arguments QStringList. And then ran QProcess::start and checked if the start succeeded, and catched STDERR if the program would not exit with 0. I simply made this a QObject subclass and rather then using start() etc. directly, I created a QProcess object and did the same stuff with this one. No freezes anymore. No matter what scanner I use. I never thought subclassing the QProcess would change the behavior in such a radical way :-O Is there some passage in the docs that says "Never ever subclass a QProcess unless you exactly know what you are doing, otherwise, you will experience the weirdest problems"? :-D I would never have figured this out. Thank you very very much for having a look at this and pointing me in the right direction. Cheers, Tobias
Re: libksane seems to break QProcess::start calls
On Friday, 4 March 2022 09:02:37 PST Thiago Macieira wrote:
> > I uploaded the traces here:
> > https://l3u.de/tmp/strace_brscan.txt.xz
> > https://l3u.de/tmp/strace_plustek.txt.xz
> >
> > Thanks again for all help!
>
> I'll take a look. Let's see... the brscan trace doesn't have any use of
> PIDFD.
Let's see the plustek scan. It also has zero uses of PIDFD (probably libksane
is using KProcess, which triggers the QProcess pidfd protection in Qt 5). So
PIDFD is not the issue.
Let's see what it's doing. The first mention of SIGCHLD is here:
[pid 20601] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|
CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fb873fff910) = 20602
[pid 20601] rt_sigaction(SIGCHLD, {sa_handler=0x7fb894726350, sa_mask=[CHLD],
sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fb8cd33a5a0},
{sa_handler=SIG_DFL, sa_mask=[], sa_flags=0}, 8) = 0
That's NOT QProcess. I can tell because the sequence is wrong, possibly buggy:
it's a bad idea to install the SIGCHLD handler *after* you've fork()ed your
child, because the child might die before you get there (unless you construct
the child code so that it can't). QProcess also doesn't set the sa_mask, which
this code did.
And there's no execve() in PID 20602, so it definitely isn't QProcess. This is
just something using fork() to run some code that may or may not crash, may or
may not hang so it may need to be kill()ed.
The second mention of SIGCHLD is this child dying:
[pid 20601] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=20602,
si_uid=1000, si_status=0, si_utime=3, si_stime=31} ---
[pid 20601] rt_sigreturn({mask=[ALRM]}) = 0
I don't know what the signal handler did, because it made no system calls
before returning. That indicates a poorly written signal handler (it probably
wrote to a volatile variable). That's also probably buggy.
The third mention of SIGCHLD is QProcess/forkfd:
[pid 20579] rt_sigaction(SIGCHLD, {sa_handler=0x7fb8cda513e0, sa_mask=[],
sa_flags=SA_RESTORER|SA_SIGINFO|SA_NOCLDSTOP, sa_restorer=0x7fb8cd33a5a0},
{sa_handler=0x7fb894726350, sa_mask=[CHLD], sa_flags=SA_RESTORER|SA_RESTART,
sa_restorer=0x7fb8cd33a5a0}, 8) = 0
[pid 20579] rt_sigaction(SIGPIPE, {sa_handler=SIG_IGN, sa_mask=[],
sa_flags=SA_RESTORER, sa_restorer=0x7fb8cd33a5a0}, NULL, 8) = 0
[pid 20579] futex(0x7fb8cdd7f3e8, FUTEX_WAKE_PRIVATE, 2147483647) = 0
[pid 20579] pipe2([24, 25], O_CLOEXEC) = 0
[pid 20579] eventfd2(0, EFD_CLOEXEC)= 26
[pid 20579] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|
CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fb8ca53ba90) = 20605
[pid 20579] write(26, "*\0\0\0\0\0\0\0", 8) = 8
Note the same sequence as the previous commit, with a pipe and an eventfd,
which gets value 42. Note also how the handler for SIGCHLD is installed before
we fork(), with no mask.
This child's death is properly reported in the fourth mention of SIGCHLD:
[pid 20579] --- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=20605,
si_uid=1000, si_status=0, si_utime=25, si_stime=0} ---
[pid 20579] waitid(P_ALL, 0, {si_signo=SIGCHLD, si_code=CLD_EXITED,
si_pid=20605, si_uid=1000, si_status=0, si_utime=0, si_stime=0}, WNOHANG|
WEXITED|WNOWAIT, NULL) = 0
[pid 20579] wait4(20605, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], WNOHANG,
{ru_utime={tv_sec=0, tv_usec=248891}, ru_stime={tv_sec=0, tv_usec=3966}, ...})
= 20605
[pid 20579] write(25, "\1\0\0\0\0\0\0\0\0\0\0\0\0\0\0\0;
\314\3\0\0\0\0\0\0\0\0\0\0\0\0\0"..., 152) = 152
[pid 20579] close(25) = 0
[pid 20579] waitid(P_ALL, 0, 0x7ffc6caac790, WNOHANG|WEXITED|WNOWAIT, NULL) =
-1 ECHILD (Keine Kind-Prozesse)
[pid 20579] rt_sigreturn({mask=[]}) = 2
And here you see a proper SIGCHLD handler, that actually does some work before
returning.
The fifth mention of SIGCHLD is the same as the first:
[pid 20636] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|
CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fb8720ab910) = 20637
[pid 20636] rt_sigaction(SIGCHLD, {sa_handler=0x7fb894726350, sa_mask=[CHLD],
sa_flags=SA_RESTORER|SA_RESTART, sa_restorer=0x7fb8cd33a5a0},
{sa_handler=0x7fb8cda513e0, sa_mask=[], sa_flags=SA_RESTORER|SA_SIGINFO|
SA_NOCLDSTOP, sa_restorer=0x7fb8cd33a5a0}, 8) = 0
This installs *again* the same SIGCHLD handler at pointer address
0x7fb894726350 after fork(). This is the first certain bug in this code, third
possible bug overall. Since it never uninstalled its handler, it shouldn't
attempt to install it again. There's NO condition under doing that is a
correct thing to do in a multithreaded application. Zero.
The sixth mention is the same as the second.
The seventh mention is forkfd again, but like a proper SIGCHLD citizen, it's
different from the fourth mention by not installing the handler again. It just
starts the child via fork():
[pid 20579] pipe2([24, 25], O_CLOEXEC) = 0
[pid 20579] eventfd2(0, EFD_CLOEXEC)= 26
[pid 20579] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|
CLONE_CHILD_SETTID|SIGCHLD,
Re: libksane seems to break QProcess::start calls
On Thursday, 3 March 2022 23:32:11 PST Tobias Leupold wrote: > I did it like this. I created one strace for the scanner not causing > problems and scanned two pages. This resulted in 27,096 lines of output. > > Then I created one for the scanner causing the issue, also scanning two > pages. This yielded a solid 270,807 lines :-O > > To be honest, I don't have a clue for what I search in those files ... > would you be so kind to have a look? Or can I lessen the output, so this > becomes more readable? > > I uploaded the traces here: > https://l3u.de/tmp/strace_brscan.txt.xz > https://l3u.de/tmp/strace_plustek.txt.xz > > Thanks again for all help! I'll take a look. Let's see... the brscan trace doesn't have any use of PIDFD. Having *zero* uses means usually means it isn't QProcess because QProcess (forkfd) always tries it at least once, to determine if the support is there. But it certainly looks like QProcess: [pid 20386] pipe2([12, 17], O_CLOEXEC) = 0 [pid 20386] write(5, "\1\0\0\0\0\0\0\0", 8) = 8 [pid 20386] write(5, "\1\0\0\0\0\0\0\0", 8) = 8 [pid 20386] pipe2([18, 19], O_CLOEXEC) = 0 [pid 20386] write(5, "\1\0\0\0\0\0\0\0", 8) = 8 [pid 20386] pipe2([20, 21], O_CLOEXEC) = 0 [pid 20386] write(5, "\1\0\0\0\0\0\0\0", 8) = 8 [pid 20386] pipe2([22, 23], O_CLOEXEC) = 0 [pid 20386] write(5, "\1\0\0\0\0\0\0\0", 8) = 8 That's four pipes with a write of 64-bit value 1 interspersed. That's really QProcess and the writes are the QSocketNotifier creations (the write() are telling the event loop that something has changed, so it can't go to sleep). And here's forkfd: [pid 20386] pipe2([24, 25], O_CLOEXEC) = 0 [pid 20386] eventfd2(0, EFD_CLOEXEC)= 26 [pid 20386] clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID| CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7f5cce3eea90) = 20425 [pid 20386] write(26, "*\0\0\0\0\0\0\0", 8 This is very distinctive: one more pipe, one eventfd and writing of value 42 into it (the answer to life, the universe and everything). Making the code sequence unique pays off (I can fingerprint it in straces). So the question is why this brscan didn't even attempt to use pidfd. There's some code in Qt5's QProcess to attempt to detect whether you've subclassed QProcess and skip using the pidfd feature: if (typeid(*q) != typeid(QProcess)) ffdflags |= FFD_USE_FORK; https://code.qt.io/cgit/qt/qtbase.git/tree/src/corelib/io/qprocess_unix.cpp? h=5.15#n462 -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel DPG Cloud Engineering
Re: libksane seems to break QProcess::start calls
Hi Thiago and all, thanks a lot for the input so far! I'm really a bit lost here ... Am 04.03.22 um 01:06 schrieb Thiago Macieira: > Fix the problem where the problem is. I would really love to, esp. if this could be some upstream issue. And if not, I'd also love to fix it ;-) You can strace the process. Add the options -f -bexecve to see the current process and its forks & threads, but stop when it executes a child process. You should see SIGCHLD line, like this: I did it like this. I created one strace for the scanner not causing problems and scanned two pages. This resulted in 27,096 lines of output. Then I created one for the scanner causing the issue, also scanning two pages. This yielded a solid 270,807 lines :-O To be honest, I don't have a clue for what I search in those files ... would you be so kind to have a look? Or can I lessen the output, so this becomes more readable? I uploaded the traces here: https://l3u.de/tmp/strace_brscan.txt.xz https://l3u.de/tmp/strace_plustek.txt.xz Thanks again for all help! Cheers, Tobias
Re: libksane seems to break QProcess::start calls
On Thursday, 3 March 2022 11:00:07 PST Tobias Leupold wrote:
> Am Donnerstag, 3. März 2022, 19:06:21 CET schrieb Stefan Brüns:
> > Are you using either Qt5 < 5.15 or a kernel version which does not support
> > CLONE_FD? - then you are relying on SIGCHLD for process exit notification.
> >
> > CLONE_FD: https://lwn.net/Articles/636646/
> > Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/
> >
> > sane-backends/backend/plustek-usbhw.c messes with the signal handlers and
> > fails to restore it: `sigaction(..., ..., NULL)`
If CLONE_PIDFD is active, we don't use the SIGCHLD handler to be notified of
process exit, but SIGCHLD must not be ignored with SIG_IGN. That causes the
kernel not to notify anything.
https://gitlab.com/sane-project/backends/-/blob/master/backend/plustek-usbhw.c
I see sigaction, but it is only messing with SIGALRM. Yes, it should restore
previous the handler (probably SIG_DFL) after the alarm is done, but this
isn't the issue. It also messes with the signal block mask, but I don't see a
problem there either: it blocks and unblocks SIGALRM.
> I use Qt 5.15.2 and Gentoo's kernel 5.15.16. Is there something I can check
> to figure out if this could cause the problem? Apparently, there's no
> kernel option I can set?
You can strace the process. Add the options -f -bexecve to see the current
process and its forks & threads, but stop when it executes a child process.
You should see SIGCHLD line, like this:
kill(669903, SIGTERM) = 0
poll([{fd=3, events=0}, {fd=5, events=0}, {fd=6, events=POLLIN}, {fd=8,
events=POLLIN}, {fd=12, events=POLLIN}], 5, 5000) = 4 ([{fd=5,
revents=POLLERR}, {fd=6, revents=POLLHUP}, {fd=8, revents=POLLHUP}, {fd=12,
revents=POLLIN}])
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=669903, si_uid=1000,
si_status=SIGTERM, si_utime=0, si_stime=0} ---
fcntl(12, F_GETFL) = 0x802 (flags O_RDWR|O_NONBLOCK)
waitid(P_PIDFD, 12, {si_signo=SIGCHLD, si_code=CLD_KILLED, si_pid=669903,
si_uid=1000, si_status=SIGTERM, si_utime=0, si_stime=0}, WNOHANG|WEXITED,
NULL) = 0
fd 12 is the pidfd in this case.
Anyway, search for SIGCHLD in the trace and see if anything is setting a
handler for it. On kernels >= 5.4, Qt doesn't do it, so if you see any mention
of it aside from deliveries like above, it came from elsewhere.
> Apart from that, the scanner in question actually does use the plustek
> backend! I fear there's no way to fix this in my code?!
Fix the problem where the problem is.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel DPG Cloud Engineering
Re: libksane seems to break QProcess::start calls
Am Donnerstag, 3. März 2022, 19:06:21 CET schrieb Stefan Brüns: > Are you using either Qt5 < 5.15 or a kernel version which does not support > CLONE_FD? - then you are relying on SIGCHLD for process exit notification. > > CLONE_FD: https://lwn.net/Articles/636646/ > Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/ > > sane-backends/backend/plustek-usbhw.c messes with the signal handlers and > fails to restore it: `sigaction(..., ..., NULL)` I use Qt 5.15.2 and Gentoo's kernel 5.15.16. Is there something I can check to figure out if this could cause the problem? Apparently, there's no kernel option I can set? Apart from that, the scanner in question actually does use the plustek backend! I fear there's no way to fix this in my code?!
Re: libksane seems to break QProcess::start calls
On Donnerstag, 3. März 2022 13:54:32 CET Tobias Leupold wrote: > Hi all :-) > > I have a very odd problem, and I have no idea what could cause this or even > how to debug this. maybe, someone of you can give me a hint. > > I revently wrote a small helper program for one special purpose: Scanning > documents at a defined size, post processing them a bit and saving the > processed, compressed images as a PDF file to e.g. send them via mail. The > sources can be found at https://invent.kde.org/tleupold/scandoc/ . Are you using either Qt5 < 5.15 or a kernel version which does not support CLONE_FD? - then you are relying on SIGCHLD for process exit notification. CLONE_FD: https://lwn.net/Articles/636646/ Qt5: https://codereview.qt-project.org/c/qt/qtbase/+/108456/ sane-backends/backend/plustek-usbhw.c messes with the signal handlers and fails to restore it: `sigaction(..., ..., NULL)` Regards, Stefan -- Stefan Brüns / Bergstraße 21 / 52062 Aachen home: +49 241 53809034 mobile: +49 151 50412019 signature.asc Description: This is a digitally signed message part.
Re: libksane seems to break QProcess::start calls
Hi Klaas! > Interesting, I wrote a very similar little utility called PDFQuirk: > https://dragotin.github.io/quirksite/ Well, apparently, this is a quite common task to do ;-) > PDFQuirk has a very similar idea, yet it does not use libksane, but the > command line tool scanimage directly, started via QProcess. The idea > behind that is that the users should not need to tweak around with > scanner settings after the 'admin' set them, as in the average office, > the scanner does not change so often... I also thought about using scanimage (actually, scandoc is the GUI version of a shell script I used until now). But being able to set the scanner settings via the GUI (esp. the page size) seemed nicer to me. Once set, the settings are saved and restored per scanner, so that one only has to set them once, too. > As this only happens with one scanner and works fine with the other > (right?), Correct > I would point to the scanner driver that is loaded by sane. > Have you tried the scanimage command line utility from SANE if it works > with the scanner in question? Using scanimage on the console works without a problem. The thing I really don't get is how scanning two images can ever affect a QProcess started somewhere else ... I mean the convert process does work, but QProcess thinks that it doesn't finish ... but only after two scans ...
Re: libksane seems to break QProcess::start calls
Am 03.03.22 um 13:54 schrieb Tobias Leupold:
Hi Tobias,
I have a very odd problem, and I have no idea what could cause this or even
how to debug this. maybe, someone of you can give me a hint.
I revently wrote a small helper program for one special purpose: Scanning
documents at a defined size, post processing them a bit and saving the
processed, compressed images as a PDF file to e.g. send them via mail. The
sources can be found at https://invent.kde.org/tleupold/scandoc/ .
Interesting, I wrote a very similar little utility called PDFQuirk:
https://dragotin.github.io/quirksite/
It uses libksane, ImageMagick's convert and pdfjam as helper programs. This
may be too special or too hacky to become e.g. an official KDE extragear
program, but that's another thing.
PDFQuirk has a very similar idea, yet it does not use libksane, but the
command line tool scanimage directly, started via QProcess. The idea
behind that is that the users should not need to tweak around with
scanner settings after the 'admin' set them, as in the average office,
the scanner does not change so often...
However the problem is:
As said, the program uses convert to post process the scanned images. I use
QProcess to run the respective command, in a procedural, synchronous way, as
the command is typically finished within fractions of a second. The call
strips down to:
QProcess process;
process.start(command, arguments);
waitForFinished();
Using one scanner I have (it's a Brother MFC device), this works without a
problem. Everything is fine. But using another one, a CanoScan LiDE 25, a
really strange problem happens:
As this only happens with one scanner and works fine with the other
(right?), I would point to the scanner driver that is loaded by sane.
Have you tried the scanimage command line utility from SANE if it works
with the scanner in question?
regards,
Klaas
After having scanned the first page, everything works as expected. But after
having scanned the second one, the QProcess call doesn't exit anymore. It runs
normally, and the output file is created. But it doesn't return, until it's
killed by the waitForFinished() call. ps lists the process as "defunct".
As expected, the GUI freezes for 30 seconds (the default timeout). But after
the process is killed, the GUI is still frozen for another 30 seconds (why?!),
then it becomes responsive again and the post processed image is added like if
the call would have exited normally.
It's also not about the "convert" call. Each and every QProcess I start after
the seconds scan does not exit anymore. Even something like "dmesg" or such.
After the first scan, everything is fine, after the second scan,
QProcess::start does not exit anymore. As long as I don't do a second scan, I
can start as many QProcess processes as I want, and all exit normally. But not
anymore after the second page.
I also tried to create the QProcess on the heap, and to implement the command
run asynchronously. The result is the same: After the first scan, the process
returns normally, after the second scan, it doesn't exit anymore ("can't
start, already running").
To make it even more peculiar: At first, I implemented the convert process to
read from stdin and write to stdout, piped the image data to it, and read the
output to get the processed image directly. This caused no problem, no matter
how much scans I did. But later on, I needed to call programs not reading from
stdin.
So ... how can that even happen? Where do the 30 seconds unresponsiveness come
from, after the QProcess has already been killed?! Is this something that
libksane causes? How can it influence a completely unrealted QProcess call? Or
did I simply write crappy code?!
If anybody has any idea about this, I would really appreciate some
enlightenment ;-)
Thanks for all help in advance!
Cheers, Tobias
libksane seems to break QProcess::start calls
Hi all :-)
I have a very odd problem, and I have no idea what could cause this or even
how to debug this. maybe, someone of you can give me a hint.
I revently wrote a small helper program for one special purpose: Scanning
documents at a defined size, post processing them a bit and saving the
processed, compressed images as a PDF file to e.g. send them via mail. The
sources can be found at https://invent.kde.org/tleupold/scandoc/ .
It uses libksane, ImageMagick's convert and pdfjam as helper programs. This
may be too special or too hacky to become e.g. an official KDE extragear
program, but that's another thing.
However the problem is:
As said, the program uses convert to post process the scanned images. I use
QProcess to run the respective command, in a procedural, synchronous way, as
the command is typically finished within fractions of a second. The call
strips down to:
QProcess process;
process.start(command, arguments);
waitForFinished();
Using one scanner I have (it's a Brother MFC device), this works without a
problem. Everything is fine. But using another one, a CanoScan LiDE 25, a
really strange problem happens:
After having scanned the first page, everything works as expected. But after
having scanned the second one, the QProcess call doesn't exit anymore. It runs
normally, and the output file is created. But it doesn't return, until it's
killed by the waitForFinished() call. ps lists the process as "defunct".
As expected, the GUI freezes for 30 seconds (the default timeout). But after
the process is killed, the GUI is still frozen for another 30 seconds (why?!),
then it becomes responsive again and the post processed image is added like if
the call would have exited normally.
It's also not about the "convert" call. Each and every QProcess I start after
the seconds scan does not exit anymore. Even something like "dmesg" or such.
After the first scan, everything is fine, after the second scan,
QProcess::start does not exit anymore. As long as I don't do a second scan, I
can start as many QProcess processes as I want, and all exit normally. But not
anymore after the second page.
I also tried to create the QProcess on the heap, and to implement the command
run asynchronously. The result is the same: After the first scan, the process
returns normally, after the second scan, it doesn't exit anymore ("can't
start, already running").
To make it even more peculiar: At first, I implemented the convert process to
read from stdin and write to stdout, piped the image data to it, and read the
output to get the processed image directly. This caused no problem, no matter
how much scans I did. But later on, I needed to call programs not reading from
stdin.
So ... how can that even happen? Where do the 30 seconds unresponsiveness come
from, after the QProcess has already been killed?! Is this something that
libksane causes? How can it influence a completely unrealted QProcess call? Or
did I simply write crappy code?!
If anybody has any idea about this, I would really appreciate some
enlightenment ;-)
Thanks for all help in advance!
Cheers, Tobias
