Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-08-05 Thread Thiago Macieira
On Thursday 10 July 2014 10:22:14 Thiago Macieira wrote:
 On Thursday 10 July 2014 07:56:28 Koehne Kai wrote:
  Provide a define QT_LOG_TO_CONSOLE  that let QCoreApplication.h record
  whether it should log to console (QT_LOG_TO_CONSOLE=1), to the system
  (QT_LOG_TO_CONSOLE=0), or both (QT_LOG_TO_CONSOLE=2).
  
  Qmake can wrap this into convenience CONFIG arguments:
  
  CONFIG+=log_to_console log_to_system
 
 It's easier to set an AA flag.
 
 http://qt-project.org/doc/qt-5/qcoreapplication.html#setAttribute
 http://qt-project.org/doc/qt-5/qt.html#ApplicationAttribute-enum
 
 PS: doc team, the second link above is unreadable.

There was no other support for this suggestion, so I'm not taking it forward.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-08-05 Thread Thiago Macieira
On Friday 11 July 2014 14:47:04 Joerg Bornemann wrote:
 On 11-Jul-14 14:14, Joerg Bornemann wrote:
  Here's the code. Maybe I missed something.
 
 Ossi showed me what I missed.
 We cannot just change stdout unconditionally.
 So no reason to not do this, as long as the application call is prefixed
 with start /wait to run it in the foreground.

Please comment on https://codereview.qt-project.org/89357 with the exact code 
that needs to be written for Windows.

I won't have access to a Windows machine until after the feature freeze.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Koehne Kai


 -Original Message-
 From: development-bounces+kai.koehne=digia@qt-project.org
 [...]
 Some background: on Sailfish, we launch all major system services (and
 UI) through systemd services ( user session for the UI parts). Those
 processes may then choose to execute processes of their own (the most
 prominent example being the UI environment which will launch applications
 in response to user input). With a default setup, systemd will log everything
 sent to stderr of systems-launched processes, and child-launched processes
 will have their stderr output attributed to the systemd service which
 launched them.

Out of curiosity: Would an opt-in solution (the application explicitly has to 
tell Qt to log to journal, e.g. through an API, maybe even through 
qtlogging.ini file) work for your use case?

I'm wondering whether such a setup wouldn't be a good one for desktop 
distributions, too. I can see that e.g. warning and errors from kwin are best 
kept in journald. Not sure whether that means though that you necessarily want 
all spurious warnings and messages from every single KDE app in there.

Regards

Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Hunger Tobias


On Thu, Jul 10, 2014 at 7:14 PM, Thiago Macieira 
thiago.macie...@intel.com wrote:

  I'm not asking that distributions turn journald on if they don't want to.
  Journald is optional.

Good to have that fact stated clearly:-)

snip section on remote debugging and having a way to force stderr

  It's there in two ways:
 
  1) Option -t to ssh, which forces the allocation of a TTY and, thus, 
logging
  to stderr
  2) The environment variable
 
  Note that it's also possible to query the system log remotely too. I 
believe
  that is done on Android, isn't it? From all I can find on the 
Internet, in
  order to enable capture of stderr, you need to have a rooted device.

AFAIK android is not using systemd/journald, so what we do there does 
not directly apply to the situation here.

  I meant that each plugin for remote systems needs to decide how it's 
going to
  approach this problem. If it can access the remote log, all the 
better. If it
  can't, the best way is to use ssh -t -- with the added benefit that 
it causes
  stdio to become line-buffered. If ssh is not getting used, make sure the
  environment variable is set.

Sure, we can come up with case-by-case ways to get to data.

But I really want to keep the simple remote debugging use case working. 
That currently requires sshd and gdbserver to be installed on the device 
and I would personally prefer not to have additional dependencies on top 
of that.

I found some documentation on how to configure journald to forward its 
logs to a pre-configured remote machine, which does not fit our use 
case. We need to access the logs from the machine of the developer and 
that will rarely be the central log server (I hope;-). At least 
journalctl can produce machine readable output, so we might get away by 
just retrieving that by running journalctl -o json. That is way more 
data than what we currently get via stderr, so this will need more 
bandwidth than the current approach. That might already be prohibitive 
on some hardware. But without code that is just speculation.

  And there's another advantage: if Creator is able to query the system 
log, it
  is also able to print the output of an already-running application 
that you
  attached the debugger to.

I see why you would want journal data in your output, but I do see a 
problems making that work for remote debugging/running. Even sailfish is 
apparently forcing output to stderr when starting applications on-device 
via Qt Creator.

Does somebody have experience with reading data from journald remotely?

Best Regards,
Tobias
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Robin Burchell
On Fri, Jul 11, 2014 at 11:13 AM, Hunger Tobias tobias.hun...@digia.com wrote:
 I see why you would want journal data in your output, but I do see a
 problems making that work for remote debugging/running. Even sailfish is
 apparently forcing output to stderr when starting applications on-device
 via Qt Creator.

Don't let our existing solution limit you. There may be better
solutions, we were time-constrained, and now resource-constrained
(i.e. we have to pick and choose what we put effort into..) so we
haven't looked into changing this any more, as far as I know.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Corentin Jabot
Looks like there are many different uses cases and point of views, we may
not reach an agreement.

I agree that the default settings should be as unified as possible, but, on
the other and, each platform
having different capabilities, it make sense to set the default behavior on
a platform-per-platform basis;
there are only 2 or 3 possible choices after-all.
II the documentation states explicitly what the behavior is on each platform
and that the behavior is to be expected on that platform, I don't think
people will get confused.

I really think we need a way to override the log output in the application
with an api such as Denis proposed,
in addition to the existing environment variable.

Having a global or per-application file may be to much, things should be
kept simple.

I still don't think qt creator should read the output from the journal when
it can avoid it.
Reading the system log + stdout, merging both in a sane way ( which means
qt creator will have to time stamp the output from stdout)
looks overly complicated to me.
The extra information the journal provide will be lost anyway since Qt
Creator's output is very basic (but, sufficient most of the time).
And if the order in which each line is displayed is inaccurate, it could
make the debugging much harder.
The extra work of reading the journal may also create a delay in the
output, which is annoying when debugging.

The debugging with an other IDE should also work out-of-the-box. It's not
that debugging should be handled specifically,
more like debugging should not be handled differently, compared to any
another application, and the burden should not be put
on the IDE.
That being said, if it comes to this, I have no problem with Qt creator
setting QT_LOGGING_TO_CONSOLE by default.

As for attaching windows applications to their parent's console
automatically, that would be really neat.




2014-07-11 11:25 GMT+02:00 Robin Burchell robin...@viroteck.net:

 On Fri, Jul 11, 2014 at 11:13 AM, Hunger Tobias tobias.hun...@digia.com
 wrote:
  I see why you would want journal data in your output, but I do see a
  problems making that work for remote debugging/running. Even sailfish is
  apparently forcing output to stderr when starting applications on-device
  via Qt Creator.

 Don't let our existing solution limit you. There may be better
 solutions, we were time-constrained, and now resource-constrained
 (i.e. we have to pick and choose what we put effort into..) so we
 haven't looked into changing this any more, as far as I know.
 ___
 Development mailing list
 Development@qt-project.org
 http://lists.qt-project.org/mailman/listinfo/development

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Joerg Bornemann
On 10-Jul-14 20:03, Thiago Macieira wrote:

 That means we can replace the GetConsoleWindow() call with
 AttachConsole(ATTACH_PARENT_PROCESS). If it succeeds or if it returns
 ERROR_ACCESS_DENIED, we have a console.

 For the Windows folks around: should we do this?

I've just played around with this and unfortunately channel redirection 
does not work with this approach.

 myapp  foo.txt
starts myapp in the background like myapp  on *nix would do.
The log is written to the console window and doesn't appear in the file.
 cmd /c myapp  foo.txt
Blocks the shell as expected but redirection still doesn't work.

Here's the code. Maybe I missed something.

---snip---
#include QApplication
#include qt_windows.h
#include cstdio
#include fcntl.h
#include io.h

int main(int argc, char *argv[])
{
 QApplication a(argc, argv);
 if (!AttachConsole(ATTACH_PARENT_PROCESS))
 return 1;
 HANDLE h = GetStdHandle(STD_OUTPUT_HANDLE);
 int fd = _open_osfhandle((intptr_t)h, _O_TEXT);
 if (fd = 0)
 return 2;
 *stdout = *_fdopen(fd, w);
 setvbuf(stdout, NULL, _IONBF, 0 );
 printf(Hello World from GUI app!\n);
 QMetaObject::invokeMethod(a, quit, Qt::QueuedConnection);
 return a.exec();
}
---snap---


Joerg
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Joerg Bornemann
On 11-Jul-14 01:18, Thiago Macieira wrote:

 Logging to both is a problem if you have something that is able to read from
 both. On Windows, all debuggers and IDEs read from both. Therefore, the option
 is out.

That's not true. Visual Studio displays output from OutputDebugString in 
its debug output pane. The stderr/stdout output is printed to the 
console window that gets opened when you run your console application.
The output is separated and that makes perfect sense. Qt Creator could 
have a separate output pane for OutputDebugString and we're good.

BTW QTestLib did log to both until 5.2.1.
See 7fab8eb56b9309734d1b34d3d935d6a4cf14ce6e


BR,

Joerg
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Joerg Bornemann
On 11-Jul-14 14:14, Joerg Bornemann wrote:

 Here's the code. Maybe I missed something.

Ossi showed me what I missed.
We cannot just change stdout unconditionally.
So no reason to not do this, as long as the application call is prefixed 
with start /wait to run it in the foreground.

---snip---
#include QApplication
#include qt_windows.h
#include cstdio
#include fcntl.h
#include io.h

int main(int argc, char *argv[])
{
 QApplication a(argc, argv);
 HANDLE h = GetStdHandle(STD_OUTPUT_HANDLE);
 if (h == INVALID_HANDLE_VALUE || h == NULL) {
 if (!AttachConsole(ATTACH_PARENT_PROCESS))
 return 1;
 h = GetStdHandle(STD_OUTPUT_HANDLE);
 int fd = _open_osfhandle((intptr_t)h, _O_TEXT);
 if (fd = 0)
 return 2;
 *stdout = *_fdopen(fd, w);
 setvbuf(stdout, NULL, _IONBF, 0 );
 }
 printf(Hello World from GUI app!\n);
 QMetaObject::invokeMethod(a, quit, Qt::QueuedConnection);
 return a.exec();
}
---snap---


Joerg
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Thiago Macieira
On Friday 11 July 2014 09:13:30 Hunger Tobias wrote:
   Note that it's also possible to query the system log remotely too. I
believe
   that is done on Android, isn't it? From all I can find on the
   Internet, in
   order to enable capture of stderr, you need to have a rooted device.
 
 AFAIK android is not using systemd/journald, so what we do there does
 not directly apply to the situation here.

It does because we use a logging system very similar to journald and the 
criteria to choose when to use that or when to use stderr is the same.

Instead of sd_journald_send, we use __android_log_print. That's the only 
difference.

   I meant that each plugin for remote systems needs to decide how it's
   going to
   approach this problem. If it can access the remote log, all the
   better. If it
   can't, the best way is to use ssh -t -- with the added benefit that
   it causes
   stdio to become line-buffered. If ssh is not getting used, make sure the
   environment variable is set.
 
 Sure, we can come up with case-by-case ways to get to data.
 
 But I really want to keep the simple remote debugging use case working.
 That currently requires sshd and gdbserver to be installed on the device
 and I would personally prefer not to have additional dependencies on top
 of that.

That's there. I've given you two possible solutions to get the log sent to 
stderr in those cases.

 I found some documentation on how to configure journald to forward its
 logs to a pre-configured remote machine, which does not fit our use
 case. We need to access the logs from the machine of the developer and
 that will rarely be the central log server (I hope;-). At least
 journalctl can produce machine readable output, so we might get away by
 just retrieving that by running journalctl -o json. That is way more
 data than what we currently get via stderr, so this will need more
 bandwidth than the current approach. That might already be prohibitive
 on some hardware. But without code that is just speculation.

Right. That would be a third option, but it would also require figuring out 
what the target system is doing. Maybe for remote debugging we don't want to 
mess with journald or something else. Suppose you were ssh'ing into an OS X 
machine. Then we're not talking about journald, but about asl.

   And there's another advantage: if Creator is able to query the system
   log, it
   is also able to print the output of an already-running application
   that you
   attached the debugger to.
 
 I see why you would want journal data in your output, but I do see a
 problems making that work for remote debugging/running. Even sailfish is
 apparently forcing output to stderr when starting applications on-device
 via Qt Creator.

When I wrote the paragraph above, I was thinking of local debugging and 
attaching to already-running processes, processes that Creator did not launch. 
For local debugging, it's more reasonable.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Thiago Macieira
On Friday 11 July 2014 08:59:35 Alejandro Exojo wrote:
  I agree. Which is why my proposal email does not talk about debug mode.
 
 Oh, I'm confused then. It does in the Patch Set 6, no?

The commit does not match the proposal.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-11 Thread Thiago Macieira
On Friday 11 July 2014 14:32:51 Joerg Bornemann wrote:
 BTW QTestLib did log to both until 5.2.1.
 See 7fab8eb56b9309734d1b34d3d935d6a4cf14ce6e

To be clear: since 5.2.0 until 5.2.1.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Koehne Kai


 -Original Message-
 From: development-bounces+kai.koehne=digia@qt-project.org
 [...]
 On the contrary, the end-user on Windows expects the application logs to be
 in a file somewhere near the executable if it's not a system service. So in my
 opinion the logs shouldn't write into system unless explicitly asked to.

I think logging to a file is a useful option. But I disagree that it should be 
the default, at least for Windows.
 
First, you've the problem to find a file 'somewhere near the executable' that 
you can write to. For end-users, you don't necessarily have write permissions 
to Program Files, and even if you have it, you don't want data to be put there, 
either. The only place that should always work would probably be %localappdata%.

But even if you manage to do that: You need to take care of concurrent accesses 
(multiple instances of your application might run). Then someone comes along 
and complains about his user directory hitting the quota, so you've to 
implement some cleanup/rotation ... It gets quite a big task.

Then there's the IDE side: You can adapt Qt Creator, but Visual Studio etc 
won't know a thing about your log file.

Despite all its limits (one global buffer, only plain string logging, only one 
debugger/viewer can attach ...), the Windows system log is the recommended way 
to log debugging information for GUI applications. I'm not arguing against 
adding such a file based backend, but it should be the default. 

 Moreover afaik not every enterprise user has access to System Logs on
 Windows.

First time I hear this  (and I don't think we have had a lot complains about it 
so far?). Anyway, this is one of the things where I'd rather tell people to fix 
their policies ( I know, I know, it's a hard fight ...). Also, you can just run 
your application in the debugger (as long as you've the rights to do this ;), 
or implement a simple log-to-file in 10 lines of code yourself.

Regards

Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Koehne Kai


 -Original Message-
 From: development-bounces+kai.koehne=digia@qt-project.org
 [mailto:development-bounces+kai.koehne=digia@qt-project.org] On
 Behalf Of Thiago Macieira
 Sent: Wednesday, July 09, 2014 11:44 PM
 To: development@qt-project.org
 Subject: [Development] Cake and eating it too for qDebug printing to system
 log - https://codereview.qt-project.org/89357
 
 == Problematic ==
 Qt 5.3 and dev are currently able to send the output of qDebug, qWarning
 and the rest of the logging framework to somewhere other than stderr.
 That's done on Windows, BlackBerry, Android, and on Linux systems with
 journald.
 Additionally, there's a pending patch for OS X to use the Apple System Log [5]
 and I had a patch for MeeGo that made it go to the standard Unix syslog.
 
 We have a problem of wanting a cake and eating it too:
 
 1) we want applications to send their logging to the system log where
 possible, which usually logs more information like the log level, time,
 application name and PID. Such a system is searchable and may allow us to
 add additional information, like the category (for category logging), file 
 name,
 line number, function context, etc.
 
 2) we also want to see the output when we run applications from the
 terminal or capture the output with QProcess.
 
 The absence of output causes bug reports like [2] and [4].
 
 == Solutions ==
 === Log to both ===
 Aside from the extra overhead, this causes systems that capture both stderr
 and the system log to record and display the same message twice. That's the
 source of task [3].
 
 This is not an option.
 
 === Log to stderr only ===
 That results in loss of information, since the system logs are richer and more
 searchable.
 
 What's more, on Windows that would result in no output at all for GUI
 applications. I can't find a lot of information on whether GUI applications
 launched from an open command prompt would show anything. IIRC, they
 don't.
 
 This is not an option.
 
 === Log to system logs only ===
 This causes reports like [4]. But note that the bug is actually in Creator, 
 for
 failing to read the log store and display the data, like it does on Windows 
 with
 the debug output.
 
 This is an option.
 

We might consider also deferring the decision to the application:

=== Defer decision to application ===

Provide a define QT_LOG_TO_CONSOLE  that let QCoreApplication.h record whether 
it should log to console (QT_LOG_TO_CONSOLE=1), to the system 
(QT_LOG_TO_CONSOLE=0), or both (QT_LOG_TO_CONSOLE=2). 

Qmake can wrap this into convenience CONFIG arguments:

CONFIG+=log_to_console log_to_system

And individual (system) mkspecs might set a default, so e.g. Windows might get 
QT_LOG_TO_CONSOLE=2 for gui apps.

If nothing is set, let the heuristic kick in ...

Regards

Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Dmitriy Purgin
Sorry for my vague statements :)

Actually I wasn't talking about logging to file as a default for
Windows. On the contrary I think this has to be done by each
application individually -- someone needs as you rightly say 10 lines
of code just to make sure everything went okay, others need a large
logging facility with rotation, archiving, severity of messages and
maybe even mirroring some messages to a database. There's no universal
solution here, the former would complain about dependencies, the
latter would extend the existing logger or invent their own.

What I'm trying to say is that logging to file seems to be more common
approach on Windows than logging to Event Log accessed from the
Control Panel - Administrative unless it is a service or application
produced by Microsoft :) Access to Event Log by end-user can really be
restricted by Domain Policies which affects mostly enterprise users,
though I totally agree that those who are affected should do something
about it.

Cheers

2014-07-10 12:46 GMT+06:00 Koehne Kai kai.koe...@digia.com:


 -Original Message-
 From: development-bounces+kai.koehne=digia@qt-project.org
 [...]
 On the contrary, the end-user on Windows expects the application logs to be
 in a file somewhere near the executable if it's not a system service. So in 
 my
 opinion the logs shouldn't write into system unless explicitly asked to.

 I think logging to a file is a useful option. But I disagree that it should 
 be the default, at least for Windows.

 First, you've the problem to find a file 'somewhere near the executable' that 
 you can write to. For end-users, you don't necessarily have write permissions 
 to Program Files, and even if you have it, you don't want data to be put 
 there, either. The only place that should always work would probably be 
 %localappdata%.

 But even if you manage to do that: You need to take care of concurrent 
 accesses (multiple instances of your application might run). Then someone 
 comes along and complains about his user directory hitting the quota, so 
 you've to implement some cleanup/rotation ... It gets quite a big task.

 Then there's the IDE side: You can adapt Qt Creator, but Visual Studio etc 
 won't know a thing about your log file.

 Despite all its limits (one global buffer, only plain string logging, only 
 one debugger/viewer can attach ...), the Windows system log is the 
 recommended way to log debugging information for GUI applications. I'm not 
 arguing against adding such a file based backend, but it should be the 
 default.

 Moreover afaik not every enterprise user has access to System Logs on
 Windows.

 First time I hear this  (and I don't think we have had a lot complains about 
 it so far?). Anyway, this is one of the things where I'd rather tell people 
 to fix their policies ( I know, I know, it's a hard fight ...). Also, you can 
 just run your application in the debugger (as long as you've the rights to do 
 this ;), or implement a simple log-to-file in 10 lines of code yourself.

 Regards

 Kai
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Koehne Kai


 -Original Message-
 From: development-bounces+kai.koehne=digia@qt-project.org
 [...]
 Here's what I propose:
 
 Heuristically determine at runtime. On all platforms (including Windows), use
 only the fact that a terminal is present. If a terminal is present, write to 
 it
 using stderr. Otherwise, write to the system log.
 
 * Implications for Qt Creator:
 1) it must be able to read from the system log for desktop debugging
 (especially when running applications without Run in terminal). That's
 already done on Windows. It will be necessary to add support for journald
 (Linux) and asl (OS X).

Well, actually your latest patch that checks for stderr!=/dev/null will just 
work fine with current Qt Creator. That's for journal/Linux at least, haven't 
had a look yet how asl works.

Qt Creator needs to continue listening to stderr in any case: You still want to 
see stdout/stderr, after all. It's true that Qt Creator can in theory profit 
from additional information piped through systemd, but integrating both streams 
also has it's issues: E.g. the order of qDebug/stderr messages is no longer 
consistent, and interleaving lines might need special attention (though I guess 
we handle that already for stdout/stderr).

 2) remote debugging: per platform, it needs to be decided whether we can
 access the log on the remote device. If so, that's the best situation. If it 
 isn't,
 then we need to force Qt to log to stderr.
 
  2a) set QT_LOGGING_TO_CONSOLE=1 in the environment
  2b) force the allocation of a terminal (if using ssh, pass the -t option)

See above. Unless stderr somehow ends up in the system log too we need to 
anyway have the terminal.
 
 I recommend advising Linux distributions to disable journald logging at least
 until Creator is updated. And the support for asl on OSX should be held until
 the same time.

I agree (though I'm not sure whether Qt Creator is the blocker, or if it's 
enough to handle the issue in Qt. See above).

 * Implications for unit tests:
 Except for the unit tests testing logging to the system log in the first 
 place,
 QT_LOGGING_TO_CONSOLE=1 should be set. I think QtTest itself should do
 that.
 
 * Support for journald in the pre-built Creator binary:
 1) new plugin, link to libsystemd-journal.so.0. If the library isn't present 
 in the
 target system, the plugin won't load.
 
 2) dlopen libsystemd-journal.so.0. If it isn't present, then Qt can't very 
 well
 be logging to it.
 
 3) use QProcess to run journalctl -f
 
 * Support for journald in the pre-built Qt binaries:
 Disabled.
 --
 Thiago Macieira - thiago.macieira (AT) intel.com
   Software Architect - Intel Open Source Technology Center
 
 ___
 Development mailing list
 Development@qt-project.org
 http://lists.qt-project.org/mailman/listinfo/development
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Denis Shienkov
Perhaps, it makes sense to select an output message sink in runtime?
E.g. before QCoreApplication, by means of specify some QLogging flags, to
override the installMessageHandler and so on?

Maybe make sense to introduce a some pre-defined qInstallMessageHandler's()
or something others functions, like:

qInstallMessageDestination(enum MessageDestinationFlag)

, where

enum MessageDestinationFlag
{
MessageToConsole,
MessageToDebugger,
MessageToSystemLog,

   and etc.
}

For example, then user can select manually what output need:

int main()
{
qInstallMessageDestination(MessageToConsole | MessageToSystemLog);

QCoreApplication app;
}

Besides, in Windows there are four various directions of a  messages output:

1) To the Console

2) To the Debug output, using the OutputDebugString function:
http://msdn.microsoft.com/ru-ru/library/windows/desktop/aa363362%28v=vs.85%29.aspx

3) To the SystemLog (Journal), using the
RegisterEventSource:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363678%28v=vs.85%29.aspx

ReportEvent:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa363679%28v=vs.85%29.aspx

functions (as already was made in  QtService from qt-solutions).

4) Using WPP Software Tracing feature:
http://msdn.microsoft.com/en-us/library/windows/hardware/ff556204%28v=vs.85%29.aspx


Thus, I prefer that the user himself chose where he wants to print out the
log messages.


Best regards,
Denis







2014-07-10 12:03 GMT+04:00 Dmitriy Purgin dpur...@gmail.com:

 Sorry for my vague statements :)

 Actually I wasn't talking about logging to file as a default for
 Windows. On the contrary I think this has to be done by each
 application individually -- someone needs as you rightly say 10 lines
 of code just to make sure everything went okay, others need a large
 logging facility with rotation, archiving, severity of messages and
 maybe even mirroring some messages to a database. There's no universal
 solution here, the former would complain about dependencies, the
 latter would extend the existing logger or invent their own.

 What I'm trying to say is that logging to file seems to be more common
 approach on Windows than logging to Event Log accessed from the
 Control Panel - Administrative unless it is a service or application
 produced by Microsoft :) Access to Event Log by end-user can really be
 restricted by Domain Policies which affects mostly enterprise users,
 though I totally agree that those who are affected should do something
 about it.

 Cheers

 2014-07-10 12:46 GMT+06:00 Koehne Kai kai.koe...@digia.com:
 
 
  -Original Message-
  From: development-bounces+kai.koehne=digia@qt-project.org
  [...]
  On the contrary, the end-user on Windows expects the application logs
 to be
  in a file somewhere near the executable if it's not a system service.
 So in my
  opinion the logs shouldn't write into system unless explicitly asked to.
 
  I think logging to a file is a useful option. But I disagree that it
 should be the default, at least for Windows.
 
  First, you've the problem to find a file 'somewhere near the executable'
 that you can write to. For end-users, you don't necessarily have write
 permissions to Program Files, and even if you have it, you don't want data
 to be put there, either. The only place that should always work would
 probably be %localappdata%.
 
  But even if you manage to do that: You need to take care of concurrent
 accesses (multiple instances of your application might run). Then someone
 comes along and complains about his user directory hitting the quota, so
 you've to implement some cleanup/rotation ... It gets quite a big task.
 
  Then there's the IDE side: You can adapt Qt Creator, but Visual Studio
 etc won't know a thing about your log file.
 
  Despite all its limits (one global buffer, only plain string logging,
 only one debugger/viewer can attach ...), the Windows system log is the
 recommended way to log debugging information for GUI applications. I'm not
 arguing against adding such a file based backend, but it should be the
 default.
 
  Moreover afaik not every enterprise user has access to System Logs on
  Windows.
 
  First time I hear this  (and I don't think we have had a lot complains
 about it so far?). Anyway, this is one of the things where I'd rather tell
 people to fix their policies ( I know, I know, it's a hard fight ...).
 Also, you can just run your application in the debugger (as long as you've
 the rights to do this ;), or implement a simple log-to-file in 10 lines of
 code yourself.
 
  Regards
 
  Kai
 ___
 Development mailing list
 Development@qt-project.org
 http://lists.qt-project.org/mailman/listinfo/development

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Tobias Hunger
Hi Thiago,

Basically I agree with your statements, but I do not think we can rely
on journald
at this time.

The first problem is of course systemd itself: Ubuntu is one of the
biggest distros out
there and we can not reasonably assume that to be running systemd
before 14.04 is
at the end of its life-cycle. Considering that this is a long term
support release that
will be a while. But even when systemd is in use journald might be disabled or
misconfigured -- as you already said yourself. I hope gnome and KDE
starting to depend
on systemd user sessions will get that sorted out soon, but at this
time those changes are
not yet deployed in distributions.

So I think being able to rely on logging to journald is still a long
time off. So what is
the fallback? Stderr again?

Adding Journald into the picture will also make retrieving warnings
from remote machines
a lot more difficult. Currently we rely on SSH and/or gdbserver to
retrieve all application
output on Unix systems. I am not sure at this time how that would work
if the application
starts sending output to the journal. So having a way to force stderr
in favor of system logs
would be greatly appreciated.

Best Regards,
Tobias

PS: Creator not supporting getting logs from system logs on Unix systems
is not a bug, it is a feature nobody requested so far:-)
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Shawn Rutledge
What if we use qtlogging.ini (or equivalent on other systems) to
control whether the logs go to the system log or not?  I agree that
every ordinary KDE user doesn't need to have unread logs taking up
disk space, but maybe developers would like to turn on output to the
journal permanently for after-the-fact diagnosis when something
unexpectedly crashes or misbehaves.

There is already a discoverability problem about the available
categories, so I was thinking we need a nice tool to manage
qtlogging.ini (a widget-based app or a menu-driven console app).  This
tool could turn on and off the system logging too.  Maybe it could be
a feature of qtdiag instead of a separate tool (since logging is a
kind of diagnostic device).

On Windows I never liked CONFIG += console, because it's never in the
.pro by default (usually not in manual tests or examples, and not in
Creator-generated pro files either) and yet I would like to always
have qDebug output in the console when the app is running in a
console.  So detecting whether the app is running in a console and
automatically sending debug output there would mean that we no longer
need that flag in the .pro, right?  If it's technically possible, we
should have done that years ago.

I never want a separate pop-up console otherwise because it's an
annoyance and has too few features.  A log viewer which allows
searching and filtering would be far more useful.  So another idea is
what about including with the Qt SDK a widget-based log viewer, with
UI for turning the categories on and off, and also some searching and
filtering features?  If you are using it, the logging output would be
redirected there instead of stderr or the console... but the question
is how do you use it: by having it start the application?  or have UI
to select and attach to already-running applications?

I also don't like Creator's default run in terminal checkbox because
again I don't get to choose which terminal, and the one it chooses is
always 80x25, white background, has no search feature, etc.  So I have
to remember to turn off the checkbox because the default is not what I
want.  IMO the Application Output pane should have enough features
that it can be treated as the console in which the app is running.
What is missing, that we don't already do this?

Or if we write a log viewer tool, Creator could use that as the
Application Output pane instead of needing to write a separate one.
Or maybe it's just a matter of packaging Creator's output pane into a
standalone log viewer tool, and add more filtering features to both.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 08:13:09 Koehne Kai wrote:
  -Original Message-
  From: development-bounces+kai.koehne=digia@qt-project.org
  [...]
  Here's what I propose:
  
  Heuristically determine at runtime. On all platforms (including Windows),
  use only the fact that a terminal is present. If a terminal is present,
  write to it using stderr. Otherwise, write to the system log.
  
  * Implications for Qt Creator:
  1) it must be able to read from the system log for desktop debugging
  (especially when running applications without Run in terminal). That's
  already done on Windows. It will be necessary to add support for journald
  (Linux) and asl (OS X).
 
 Well, actually your latest patch that checks for stderr!=/dev/null will
 just work fine with current Qt Creator. That's for journal/Linux at least,
 haven't had a look yet how asl works.

The latest patch does not match the proposal.

I want to remove the stderr check for Unix and for Windows, which means 
Creator should check the system logs.

 Qt Creator needs to continue listening to stderr in any case: You still want
 to see stdout/stderr, after all. It's true that Qt Creator can in theory
 profit from additional information piped through systemd, but integrating
 both streams also has it's issues: E.g. the order of qDebug/stderr messages
 is no longer consistent, and interleaving lines might need special
 attention (though I guess we handle that already for stdout/stderr).

Agreed it must continue to read.

As for the synchronisation, journald gives you the message's absolute 
timestamp, so you'll just need to do the same for each line read via QProcess.

  2) remote debugging: per platform, it needs to be decided whether we can
  access the log on the remote device. If so, that's the best situation. If
  it isn't, then we need to force Qt to log to stderr.
  
   2a) set QT_LOGGING_TO_CONSOLE=1 in the environment
   2b) force the allocation of a terminal (if using ssh, pass the -t option)
 
 See above. Unless stderr somehow ends up in the system log too we need to
 anyway have the terminal.

Addressed above.

  I recommend advising Linux distributions to disable journald logging at
  least until Creator is updated. And the support for asl on OSX should be
  held until the same time.
 
 I agree (though I'm not sure whether Qt Creator is the blocker, or if it's
 enough to handle the issue in Qt. See above).

Creator must be able to handle reading from the system log before logging to 
the system is enabled for that system. On Linux, the code is already there, so 
we have to tell distributions *not* to enable journald yet. For OS X, it's 
putting a hold on https://codereview.qt-project.org/82922.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 11:05:03 Tobias Hunger wrote:
 Hi Thiago,
 
 Basically I agree with your statements, but I do not think we can rely
 on journald at this time.

Agreed.

 The first problem is of course systemd itself: Ubuntu is one of the
 biggest distros out
 there and we can not reasonably assume that to be running systemd
 before 14.04 is
 at the end of its life-cycle. Considering that this is a long term
 support release that
 will be a while. But even when systemd is in use journald might be disabled
 or misconfigured -- as you already said yourself. I hope gnome and KDE
 starting to depend
 on systemd user sessions will get that sorted out soon, but at this
 time those changes are
 not yet deployed in distributions.

I'm not asking that distributions turn journald on if they don't want to. 
Journald is optional.

However, they *can't* turn it on until we figure out how to read back what gets 
logged there. At this point, the showstopper is in journald itself.

 So I think being able to rely on logging to journald is still a long
 time off. So what is the fallback? Stderr again?

Yes. If you don't turn journald on (it's optional), then we always fall back 
to stderr.

 Adding Journald into the picture will also make retrieving warnings
 from remote machines
 a lot more difficult. Currently we rely on SSH and/or gdbserver to
 retrieve all application
 output on Unix systems. I am not sure at this time how that would work
 if the application
 starts sending output to the journal. So having a way to force stderr
 in favor of system logs
 would be greatly appreciated.

It's there in two ways:

1) Option -t to ssh, which forces the allocation of a TTY and, thus, logging 
to stderr
2) The environment variable

Note that it's also possible to query the system log remotely too. I believe 
that is done on Android, isn't it? From all I can find on the Internet, in 
order to enable capture of stderr, you need to have a rooted device.

I meant that each plugin for remote systems needs to decide how it's going to 
approach this problem. If it can access the remote log, all the better. If it 
can't, the best way is to use ssh -t -- with the added benefit that it causes 
stdio to become line-buffered. If ssh is not getting used, make sure the 
environment variable is set.

And there's another advantage: if Creator is able to query the system log, it 
is also able to print the output of an already-running application that you 
attached the debugger to.

 PS: Creator not supporting getting logs from system logs on Unix systems
 is not a bug, it is a feature nobody requested so far:-)

Yup.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 16:50:22 Shawn Rutledge wrote:
 What if we use qtlogging.ini (or equivalent on other systems) to
 control whether the logs go to the system log or not?  I agree that
 every ordinary KDE user doesn't need to have unread logs taking up
 disk space, but maybe developers would like to turn on output to the
 journal permanently for after-the-fact diagnosis when something
 unexpectedly crashes or misbehaves.

That doesn't solve the problem of the default. What do we do when 
qtlogging.ini isn't present?

And how do you debug an application that got installed on a system that 
already has qtlogging.ini?

 On Windows I never liked CONFIG += console, because it's never in the
 .pro by default (usually not in manual tests or examples, and not in
 Creator-generated pro files either) and yet I would like to always
 have qDebug output in the console when the app is running in a
 console.  So detecting whether the app is running in a console and
 automatically sending debug output there would mean that we no longer
 need that flag in the .pro, right?  If it's technically possible, we
 should have done that years ago.

I don't think it's possible at all. If the application was compiled to the GUI 
subsystem, it *has* no stdout and stderr. We can open a new console window, 
but I don't think you can write to an existing one.

 I also don't like Creator's default run in terminal checkbox because
 again I don't get to choose which terminal, and the one it chooses is
 always 80x25, white background, has no search feature, etc.  So I have
 to remember to turn off the checkbox because the default is not what I
 want.  IMO the Application Output pane should have enough features
 that it can be treated as the console in which the app is running.
 What is missing, that we don't already do this?

Colour support, TTY support, job control, etc. TTYs on Unix are much more 
complex than pipes.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 07:56:28 Koehne Kai wrote:
 Provide a define QT_LOG_TO_CONSOLE  that let QCoreApplication.h record
 whether it should log to console (QT_LOG_TO_CONSOLE=1), to the system
 (QT_LOG_TO_CONSOLE=0), or both (QT_LOG_TO_CONSOLE=2). 
 
 Qmake can wrap this into convenience CONFIG arguments:
 
 CONFIG+=log_to_console log_to_system

It's easier to set an AA flag.

http://qt-project.org/doc/qt-5/qcoreapplication.html#setAttribute
http://qt-project.org/doc/qt-5/qt.html#ApplicationAttribute-enum

PS: doc team, the second link above is unreadable.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Oswald Buddenhagen
On Thu, Jul 10, 2014 at 10:17:52AM -0700, Thiago Macieira wrote:
 On Thursday 10 July 2014 16:50:22 Shawn Rutledge wrote:
  On Windows I never liked CONFIG += console, because it's never in the
  .pro by default (usually not in manual tests or examples, and not in
  Creator-generated pro files either) and yet I would like to always
  have qDebug output in the console when the app is running in a
  console.  So detecting whether the app is running in a console and
  automatically sending debug output there would mean that we no longer
  need that flag in the .pro, right?  If it's technically possible, we
  should have done that years ago.
 
 I don't think it's possible at all. If the application was compiled to the 
 GUI 
 subsystem, it *has* no stdout and stderr. We can open a new console window, 
 but I don't think you can write to an existing one.
 
windows is stupid, but it can't be that stupid, can it? = google =

AttachConsole(ATTACH_PARENT_PROCESS)
GetStdHandle(STD_ERROR_HANDLE)
(and _open_osfhandle(h) if one cares for stdio)
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 19:34:59 Oswald Buddenhagen wrote:
 On Thu, Jul 10, 2014 at 10:17:52AM -0700, Thiago Macieira wrote:
  On Thursday 10 July 2014 16:50:22 Shawn Rutledge wrote:
   On Windows I never liked CONFIG += console, because it's never in the
   .pro by default (usually not in manual tests or examples, and not in
   Creator-generated pro files either) and yet I would like to always
   have qDebug output in the console when the app is running in a
   console.  So detecting whether the app is running in a console and
   automatically sending debug output there would mean that we no longer
   need that flag in the .pro, right?  If it's technically possible, we
   should have done that years ago.
  
  I don't think it's possible at all. If the application was compiled to the
  GUI subsystem, it *has* no stdout and stderr. We can open a new console
  window, but I don't think you can write to an existing one.
 
 windows is stupid, but it can't be that stupid, can it? = google =
 
 AttachConsole(ATTACH_PARENT_PROCESS)
 GetStdHandle(STD_ERROR_HANDLE)
 (and _open_osfhandle(h) if one cares for stdio)

That means we can replace the GetConsoleWindow() call with 
AttachConsole(ATTACH_PARENT_PROCESS). If it succeeds or if it returns 
ERROR_ACCESS_DENIED, we have a console.

For the Windows folks around: should we do this?
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Shawn Rutledge
On 10 July 2014 19:17, Thiago Macieira thiago.macie...@intel.com wrote:
 On Thursday 10 July 2014 16:50:22 Shawn Rutledge wrote:
 What if we use qtlogging.ini (or equivalent on other systems) to
 control whether the logs go to the system log or not?  I agree that
 every ordinary KDE user doesn't need to have unread logs taking up
 disk space, but maybe developers would like to turn on output to the
 journal permanently for after-the-fact diagnosis when something
 unexpectedly crashes or misbehaves.

 That doesn't solve the problem of the default. What do we do when
 qtlogging.ini isn't present?

I don't think it would need to go to the journal by default.  If users
don't know how to turn it on, they probably don't need the feature
either.  We just need to make it easy to turn on.

 And how do you debug an application that got installed on a system that
 already has qtlogging.ini?

Edit it and run the application again and try to reproduce the bug
again?  Being able to debug after something has already happened is a
rare privilege anyway.  If you expect problems you can turn on the
logging in advance though.

Or maybe it could be configured on a per-application basis in addition
to the defaults.  Maybe we invent a standardized easy way that an app
can allow its own QSettings to control the logging, like have a
standard [qtlogging] section.  We could make it opt-in by requiring
that you call a method on your QSettings object to enable the standard
logging section, and after that, users will be able to edit the
settings file and turn on logging categories just as they can in
qtlogging.ini.  Or it could be always-available if that doesn't bother
anybody...

 On Windows I never liked CONFIG += console, because it's never in the
 .pro by default (usually not in manual tests or examples, and not in
 Creator-generated pro files either) and yet I would like to always
 have qDebug output in the console when the app is running in a
 console.  So detecting whether the app is running in a console and
 automatically sending debug output there would mean that we no longer
 need that flag in the .pro, right?  If it's technically possible, we
 should have done that years ago.

 I don't think it's possible at all. If the application was compiled to the GUI
 subsystem, it *has* no stdout and stderr. We can open a new console window,
 but I don't think you can write to an existing one.

I thought so, but I'm glad Ossi found a possible strategy. ;-)

 I also don't like Creator's default run in terminal checkbox because
 again I don't get to choose which terminal, and the one it chooses is
 always 80x25, white background, has no search feature, etc.  So I have
 to remember to turn off the checkbox because the default is not what I
 want.  IMO the Application Output pane should have enough features
 that it can be treated as the console in which the app is running.
 What is missing, that we don't already do this?

 Colour support, TTY support, job control, etc. TTYs on Unix are much more
 complex than pipes.

What are the use cases?  Somehow I don't usually need more than what
the Creator output pane already provides... except that categorized
logging opens up a possibility to control what you see there.

But even if we go that far, it's not so hard to find some code for a
terminal that we could use (there's even a certain QtQuick one I
happen to know of ;-)  And some people would like having a terminal
window in Creator anyway; I've seen it in other IDEs.  But of course
it would add more bloat.

Anyway just being able to see and manage the logging output in Creator
does not preclude running the application in a terminal if that's what
you want to do, right?  But IMO it should not be the default, and it
wouldn't hurt if it was more configurable.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Alejandro Exojo
El Wednesday 09 July 2014, Thiago Macieira escribió:
 === Log to both ===
 Aside from the extra overhead, this causes systems that capture both stderr
 and the system log to record and display the same message twice. That's the
 source of task [3].
 
 This is not an option.

Why not? Task [3] is an issue in Qt Creator, isn't it? Why it needs to read 
the system log on Windows?

Logging to the console _and_ journald has proven very useful to us. We did it 
specifically for an application using Qt's API, and developers can code, press 
Run (in Creator), and see the output in the same window with all the 
QT_MESSAGE_PATTERN goodness. Afterwards, if it's too noisy, or if you disabled 
some category, you can check the logs on journald (we logged all categories, 
even the disabled ones, to the journal, since we had or own message handler).

It's not hard at all to do it on your application if you want to fine tune it. 
Actually, I never expected to see the journald logging in Qt, but having it 
IMHO it's a boon, since you can use the feature for 3rd party applications.

Only downside is that you might need to collect what your application outputs 
to stderr from a non-Qt library. Then yes, you might get duplicated messages.

 === Heuristically determine at runtime where to send the log ===
 On Unix, I have implemented the same checks.
 
 We can also use the fact of whether the main application is in debug mode
 or not.

I see this as two orthogonal things. Why would one prefer the console on a 
debug build? You might need to put the program in debug mode on the target 
hardware (only device where you have the peripherals), and need the enriched 
system logging and the debug symbols.

This is the most inconvenient change, since you might need to change the 
systemd unit file to set the QT_LOGGING_TO_CONSOLE environment variable to have 
the built in journald logging.

 == Summary of systems ==
 System services on Linux with journald:
  - default stderr: captured into system log
  - system logging: available
  - is stderr useful: yes, for launching from terminal

Note that you can teach systemd to ignore stderr/stdout if wanted. If the app 
is written with Qt, that can be a sane default.
 
 And then some desktop Linux distributions decided to enable journald. The
 effect of that is task [6]: Qt Creator shows no output because it goes to
 journald.

But this comment is/was in the code:

We use isatty to catch the obvious case of someone running something 
interactively. We also support environment variables for Qt Creator use (...)

I thought that it was agreed/known to Qt Creator developers that 
QT_LOGGING_TO_CONSOLE might be gently defaulted to true for applications run 
by Creator.

That environment variable seems like API to me, and a useful one that 
Creator could use. Of course they couldn't predict some distributions would 
pass -journald to configure.


From the other messages:

 * Support for journald in the pre-built Qt binaries:
 Disabled.

Probably will be that way for a long time. If Qt binaries are built with and 
old Ubuntu, they just don't have libsystemd-journal.

 Note: if journald is writing to volatile storage, regular users can't read
 the log. Therefore, system logging is not available.
 
 Linux distributions should not enable journald logging unless regular users
 can read the output.

I think users can read their user journal, independently of the storage. This 
is as user nobody:

$ systemd-journalctl
Showing user generated messages only. Users in the group 'adm' can see all 
messages. Pass -q to turn this message off.

-- 
Alex (a.k.a. suy) | GPG ID 0x0B8B0BC2
http://barnacity.net/ | http://disperso.net
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Alejandro Exojo
El Thursday 10 July 2014, Koehne Kai escribió:
 Provide a define QT_LOG_TO_CONSOLE  that let QCoreApplication.h record
 whether it should log to console (QT_LOG_TO_CONSOLE=1), to the system
 (QT_LOG_TO_CONSOLE=0), or both (QT_LOG_TO_CONSOLE=2). 

Would it be too problematic to make the environment variable 
(QT_LOGGING_TO_CONSOLE, which already is in use) accept also a 2 for logging 
to both sources?

To me, the really interesting feature of providing such code inside of Qt is 
to enable it without having to touch legacy/third party code. Having it for 
your application is also nice, but you'll likely need to override it somehow 
to integrate it with your configuration or UI.

-- 
Alex (a.k.a. suy) | GPG ID 0x0B8B0BC2
http://barnacity.net/ | http://disperso.net
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Friday 11 July 2014 00:27:51 Alejandro Exojo wrote:
 Would it be too problematic to make the environment variable 
 (QT_LOGGING_TO_CONSOLE, which already is in use) accept also a 2 for
 logging  to both sources?

Problematic, no. It can be easily implemented.

But it's an option we discarded. It makes no sense to log to both 
destinations. If you're using a tool that can read the syslog, then it doesn't 
matter which one it was sent to, but it will be a problem if you send to both. 
If you're using a tool that can't, then you want to send to stderr only.

 To me, the really interesting feature of providing such code inside of Qt
 is  to enable it without having to touch legacy/third party code. Having it
 for your application is also nice, but you'll likely need to override it
 somehow to integrate it with your configuration or UI.

If you have legacy, set to 1.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Friday 11 July 2014 00:17:48 Alejandro Exojo wrote:
 El Wednesday 09 July 2014, Thiago Macieira escribió:
  === Log to both ===
  Aside from the extra overhead, this causes systems that capture both
  stderr
  and the system log to record and display the same message twice. That's
  the
  source of task [3].
  
  This is not an option.
 
 Why not? Task [3] is an issue in Qt Creator, isn't it? Why it needs to read
 the system log on Windows?

Because that's the way to do it on Windows. It's been established for 15 or 20 
years that applications log via OutputDebugString. GUI applications can't 
write to the console unless they AllocConsole or attach to an existing one -- 
and attaching is only permitted since Windows XP.

The big problem I found when implementing this is that systemd and launchd 
will capture your stdout and stderr into the system log. So if we log to both, 
then the message appears duplicated in the syslog for all applications 
launched by those tools.

 Logging to the console _and_ journald has proven very useful to us. We did
 it specifically for an application using Qt's API, and developers can code,
 press Run (in Creator), and see the output in the same window with all the
 QT_MESSAGE_PATTERN goodness. Afterwards, if it's too noisy, or if you
 disabled some category, you can check the logs on journald (we logged all
 categories, even the disabled ones, to the journal, since we had or own
 message handler).

You can still install your handler to do that. We're talking about the default 
and I'd like to have a consistent behaviour across different OS. It makes my 
implementation simpler.

Logging to both is a problem if you have something that is able to read from 
both. On Windows, all debuggers and IDEs read from both. Therefore, the option 
is out.

 Only downside is that you might need to collect what your application
 outputs to stderr from a non-Qt library. Then yes, you might get duplicated
 messages.

There you go. That's the default on OS X and will probably become the default 
on Linux if the desktops switch to user-mode systemd.

  === Heuristically determine at runtime where to send the log ===
  On Unix, I have implemented the same checks.
  
  We can also use the fact of whether the main application is in debug mode
  or not.
 
 I see this as two orthogonal things. Why would one prefer the console on a
 debug build? You might need to put the program in debug mode on the target
 hardware (only device where you have the peripherals), and need the enriched
 system logging and the debug symbols.

I agree. Which is why my proposal email does not talk about debug mode.

 This is the most inconvenient change, since you might need to change the
 systemd unit file to set the QT_LOGGING_TO_CONSOLE environment variable to
 have the built in journald logging.

If we did it like that and if you ran debug-mode applications in your system. 
That's not a common situation. And besides, systemd will capture stderr 
anyway, just will less information.

 I thought that it was agreed/known to Qt Creator developers that
 QT_LOGGING_TO_CONSOLE might be gently defaulted to true for applications run
 by Creator.

It isn't. Creator devs are skeptical about modifying the environment by 
default. It's bad enough that it modifies PATH during the build process -- that 
means if you build with a Qt whose tools are in /usr/bin, you lose any 
overrides you placed in PATH (such as icecc).

  Note: if journald is writing to volatile storage, regular users can't read
  the log. Therefore, system logging is not available.
  
  Linux distributions should not enable journald logging unless regular
  users
  can read the output.
 
 I think users can read their user journal, independently of the storage.
 This is as user nobody:
 
 $ systemd-journalctl
 Showing user generated messages only. Users in the group 'adm' can see all
 messages. Pass -q to turn this message off.

$ systemd-journalctl 
No journal files were found.

Like I said, it depends on the journald configuration. Distributions should not 
enable journald unless they also configure journald in a way that users can 
read back what they've logged.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Lisandro Damián Nicanor Pérez Meyer
On Thursday 10 July 2014 10:07:56 Thiago Macieira wrote:
[snip]
 Creator must be able to handle reading from the system log before logging to
 the system is enabled for that system. On Linux, the code is already there,
 so we have to tell distributions *not* to enable journald yet.

AFAIU from this thread, journald support should not be enabled except regular 
users can read the output. Now what I'm missing here is: if Qt is built with 
journald support, can it be still be used if journald is *not* present?

Yes, I happen to be maintaining Qt for a distro in which users can still 
choose the init system.

-- 
UNIX is basically a simple operating system, but you have to be a
genius to understand the simplicity.
  Dennis Ritchie

Lisandro Damián Nicanor Pérez Meyer
http://perezmeyer.com.ar/
http://perezmeyer.blogspot.com/


signature.asc
Description: This is a digitally signed message part.
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Alejandro Exojo
El Friday 11 July 2014, Lisandro Damián Nicanor Pérez Meyer escribió:
 AFAIU from this thread, journald support should not be enabled except
 regular  users can read the output. Now what I'm missing here is: if Qt
 is built with journald support, can it be still be used if journald is not
 present?

No. Quote from: 
http://www.freedesktop.org/software/systemd/man/sd_journal_send.html

If systemd-journald(8) is not running (the socket is not present), those 
functions do nothing, and also return 0.
 
 Yes, I happen to be maintaining Qt for a distro in which users can still 
 choose the init system.

And unfortunately, there is no clean way to test for it. There is the 
sd_booted() function, but:

Internally, this function checks whether the directory /run/systemd/system/ 
exists. A simple check like this can also be implemented trivially in shell or 
any other language.

Which wasn't enough in my tests. Since I used systemd to boot the (developer) 
system, but I ran the app on Creator, it would return true all the time, and I 
got no output on Creator.

It might be even worse if systemd-shim provides that directory. Is it worth to 
try sd_journal_open() and see if it fails? Or check some other way if systemd 
actually launched the application?

Also, the way is done now (more after the patch on review) is to check for the 
console first, and then the decision is done. If the heuristics to find a 
console are good enough for you, having journald support is irrelevant, since 
if you are in a console you'll get the stderr printing.

-- 
Alex (a.k.a. suy) | GPG ID 0x0B8B0BC2
http://barnacity.net/ | http://disperso.net
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Thursday 10 July 2014 23:53:20 Lisandro Damián Nicanor Pérez Meyer wrote:
 On Thursday 10 July 2014 10:07:56 Thiago Macieira wrote:
 [snip]
 
  Creator must be able to handle reading from the system log before logging
  to the system is enabled for that system. On Linux, the code is already
  there, so we have to tell distributions *not* to enable journald yet.
 
 AFAIU from this thread, journald support should not be enabled except
 regular users can read the output. Now what I'm missing here is: if Qt is
 built with journald support, can it be still be used if journald is *not*
 present?
 
 Yes, I happen to be maintaining Qt for a distro in which users can still
 choose the init system.

Don't enable journald if your system doesn't use journald.

As for having journald optionally in the distro, if we can detect at runtime 
and fallback if it's not running, we should implement it.

If we can't detect, then don't enable the feature in your distro unless you're 
sure that journald will be running (excepted misconfiguration, of course).

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-10 Thread Thiago Macieira
On Friday 11 July 2014 07:45:23 Alejandro Exojo wrote:
 It might be even worse if systemd-shim provides that directory. Is it worth
 to  try sd_journal_open() and see if it fails? Or check some other way if
 systemd actually launched the application?

No and no. 

sd_journal_open might fail because the application can't open the log file, but 
logging to the journal is the intention. We're asking that distros not enable 
logging there if the user can't open the journal, but some embedded devices 
may want exactly that. So this option is out.

Checking if the application was launched by systemd is also not an option IMHO 
because it would mean subprocesses wouldn't use journald, but instead go to 
stderr, which in turn makes that output be attributed to the parent process.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-09 Thread Thiago Macieira
On Wednesday 09 July 2014 14:43:36 Thiago Macieira wrote:
 === Log to system logs only ===
 This causes reports like [4]. But note that the bug is actually in Creator, 
 for failing to read the log store and display the data, like it does on
 Windows with the debug output.
 
 This is an option.
 
 === Heuristically determine at runtime where to send the log ===
 Which is what I'm trying to do above.
 
 On Windows, we check whether we have a console and whether stderr is not
 NUL.
 
 On Unix, I have implemented the same checks.
 
 We can also use the fact of whether the main application is in debug mode
 or  not.

Here's what I propose:

Heuristically determine at runtime. On all platforms (including Windows), use 
only the fact that a terminal is present. If a terminal is present, write to 
it using stderr. Otherwise, write to the system log.

* Implications for Qt Creator:
1) it must be able to read from the system log for desktop debugging 
(especially when running applications without Run in terminal). That's 
already done on Windows. It will be necessary to add support for journald 
(Linux) and asl (OS X).

2) remote debugging: per platform, it needs to be decided whether we can 
access the log on the remote device. If so, that's the best situation. If it 
isn't, then we need to force Qt to log to stderr.

 2a) set QT_LOGGING_TO_CONSOLE=1 in the environment
 2b) force the allocation of a terminal (if using ssh, pass the -t option)

I recommend advising Linux distributions to disable journald logging at least 
until Creator is updated. And the support for asl on OSX should be held until 
the same time.

* Implications for unit tests:
Except for the unit tests testing logging to the system log in the first place, 
QT_LOGGING_TO_CONSOLE=1 should be set. I think QtTest itself should do that.

* Support for journald in the pre-built Creator binary:
1) new plugin, link to libsystemd-journal.so.0. If the library isn't present 
in the target system, the plugin won't load.

2) dlopen libsystemd-journal.so.0. If it isn't present, then Qt can't very 
well be logging to it.

3) use QProcess to run journalctl -f

* Support for journald in the pre-built Qt binaries:
Disabled.
-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-09 Thread Thiago Macieira
On Wednesday 09 July 2014 14:43:36 Thiago Macieira wrote:
 Current Linux desktops with journald:
  - default stderr: captured into ~/.xsession-errors
  - system logging available: available
  - is stderr useful: yes, for launching from terminal
 [when Linux desktops start using user-mode systemd, it will be as below]

Note: if journald is writing to volatile storage, regular users can't read the 
log. Therefore, system logging is *not* available.

Linux distributions should not enable journald logging unless regular users 
can read the output.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-09 Thread Corentin Jabot
I feel the urge to give my two cents on that.

At least on unix desktop platforms, I expect to get stderr output on the
console.
So, detecting its presence is certainly a good approach.

When there is no console, I probably don't care about the logs at all.

I *may* care about error/warning messages, granted that the application
manages its
outputs diligently, which is often not the case.

One argument is that the system logging offers better rotating  search
facilities,
which is true, but the logs would still be cluttered by probably useless
messages.
There are tons of signal/slot missing throughout KDE applications, as a end
user
what can I possibly do about it ?


I do not have a huge ~/.xsession-error file, but for those who have,
I think it's a problem for distributions to solve, not Qt.
On the other end, I usually have a large /home partition and relatively
small root partition,
so, of to evil, I would prefer my home to be cluttered. I think that's true
for most
Linux installations;

Depending on its features or importance to the system or to the user an
application
is either important, in which case some log should be conserved or
unimportant logs should
certainly be dropped.

A think an application should be assumed to be unimportant ( not to log to
the system's journal)
by default. The choice should be given both to the developer ( with an api)
and maintainers/end user (environment variable) to modify this behavior.

As for Qt Creator, my opinion is that the logs of an in-development
application become
irrelevant the second you press the rebuild button and so its output should
always
go to stderr only.

For platforms which offer no or poor console support, not-in-development
applications logs
should arguably go to the system journal to be forever forgotten :
I don't expect end-users to know how access those logs, even less to
provide the developers
with them when something goes wrong.


Corentin



2014-07-10 1:20 GMT+02:00 Thiago Macieira thiago.macie...@intel.com:

 On Wednesday 09 July 2014 14:43:36 Thiago Macieira wrote:
  Current Linux desktops with journald:
   - default stderr: captured into ~/.xsession-errors
   - system logging available: available
   - is stderr useful: yes, for launching from terminal
  [when Linux desktops start using user-mode systemd, it will be as below]

 Note: if journald is writing to volatile storage, regular users can't read
 the
 log. Therefore, system logging is *not* available.

 Linux distributions should not enable journald logging unless regular users
 can read the output.

 --
 Thiago Macieira - thiago.macieira (AT) intel.com
   Software Architect - Intel Open Source Technology Center

 ___
 Development mailing list
 Development@qt-project.org
 http://lists.qt-project.org/mailman/listinfo/development

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development


Re: [Development] Cake and eating it too for qDebug printing to system log - https://codereview.qt-project.org/89357

2014-07-09 Thread Thiago Macieira
On Thursday 10 July 2014 02:54:12 Corentin Jabot wrote:
 At least on unix desktop platforms, I expect to get stderr output on the
 console.
 So, detecting its presence is certainly a good approach.
 
 When there is no console, I probably don't care about the logs at all.
 
 I *may* care about error/warning messages, granted that the application
 manages its outputs diligently, which is often not the case.

As we discussed on IRC: *if* you care about the warnings, then you want to find 
them. And reading ~/.xsession-errors is not acceptable. Mine has close to 1.2 
million lines in 7 days of uptime (roughly 7000 lines per hour), in an output 
that doesn't log application name, PID, TID or function name. We can turn 
those on with the logging framework, but they're not easily searchable.

 One argument is that the system logging offers better rotating  search
 facilities, which is true, but the logs would still be cluttered by probably
 useless messages.
 There are tons of signal/slot missing throughout KDE applications, as a end
 user what can I possibly do about it ?

Hence the searchability.

$ journalctl -l /home/thiago/obj/qt/qt5/qtbase/bin/qdbusviewer
-- Logs begin at Wed 2014-07-09 16:14:48 PDT, end at Wed 2014-07-09 19:23:29 
PDT. --
Jul 09 19:23:29 tjmaciei-mobl4 qdbusviewer[36992]: Connecting to deprecated 
signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)

$ journalctl -l QT_CATEGORY=default
-- Logs begin at Wed 2014-07-09 16:14:48 PDT, end at Wed 2014-07-09 19:23:29 
PDT. --
Jul 09 19:23:29 tjmaciei-mobl4 qdbusviewer[36992]: Connecting to deprecated 
signal QDBusConnectionInterface::serviceOwnerChanged(QString,QString,QString)

I can even search for which Qt categories are getting logged:

$ journalctl -F QT_CATEGORY
default

 I do not have a huge ~/.xsession-error file, but for those who have,
 I think it's a problem for distributions to solve, not Qt.

Agreed. But we have to integrate with the mechanisms that the distros provide.

 A think an application should be assumed to be unimportant ( not to log to
 the system's journal) by default. The choice should be given both to the
 developer ( with an api) and maintainers/end user (environment variable) to
 modify this behavior.

I disagree. I think the application should log and then the log system can 
decide whether to keep it or throw it away.

The one thing I'm missing here is a user-configurable log system. If I'm a 
developer of a Qt application and I'm using the system Qt, I want to get my 
logs and not clutter the system. Which is where the recommendation comes from: 
Linux distributions should not enable a logging system that regular users 
can't access.

 As for Qt Creator, my opinion is that the logs of an in-development
 application become irrelevant the second you press the rebuild button and so
 its output should always go to stderr only.

Yes and no. Creator must show it, but I don't think it matters to the user 
where it got the output from.

If Creator can read from the log, it can get more information than just plain 
pipes. It could, for example, allow you to jump to the source line that 
produced the line in question. Granted, that could be done with a proper 
message formatting too, but it would be harder because stdout and stderr are 
just plain buffers (they're not even line buffered), whereas the log would be 
properly packetised.

 For platforms which offer no or poor console support, not-in-development
 applications logs should arguably go to the system journal to be forever
 forgotten :I don't expect end-users to know how access those logs, even less
 to provide the developers with them when something goes wrong.

That's the case on Windows: GUI apps have stderr connected to the blackhole 
and the OutputDebugString buffer is only 32k in size. Use it or lose it.

But again: it can be used. Hence the suggestion to use the system log.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel Open Source Technology Center

___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development