Re: F21 Self Contained Change: Remote Journal Logging

2014-06-10 Thread Adam Williamson
On Fri, 2014-05-02 at 17:51 +0200, Zbigniew Jędrzejewski-Szmek wrote: It's kind of implicit in the Change proposal. When you submit a Change, you are indicating that you want this to be something that Fedora promotes (both from an engineering standpoint and a marketing one). I modifed the

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-07 Thread Marc Deop i Argemí
On Tuesday 22 April 2014 06:34:48 Lennart Poettering wrote: On Wed, 16.04.14 12:46, Bill Nottingham (nott...@splat.cc) wrote: Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com) said:

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-05 Thread Glen Turner
I am pretty sure HTTP(s) is the right choice Hi Lennart, The choice of HTTPS does complicate the network infrastructure moving log records into a network management QoS class (ie, making sure that remote logging works during a DoS attack caused by malware). If you feel that HTTPS is the

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-05 Thread Lennart Poettering
On Mon, 05.05.14 18:54, Glen Turner (g...@gdt.id.au) wrote: I am pretty sure HTTP(s) is the right choice Hi Lennart, The choice of HTTPS does complicate the network infrastructure moving log records into a network management QoS class (ie, making sure that remote logging works

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-05 Thread Florian Weimer
On 05/05/2014 12:36 PM, Lennart Poettering wrote: If you feel that HTTPS is the correct protocol then please consider using another port number than 443. It's port 19531 by default. Have you requested official assignment? It's not yet listed in

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-05 Thread Lennart Poettering
On Mon, 05.05.14 12:49, Florian Weimer (fwei...@redhat.com) wrote: On 05/05/2014 12:36 PM, Lennart Poettering wrote: If you feel that HTTPS is the correct protocol then please consider using another port number than 443. It's port 19531 by default. Have you requested official

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-05 Thread Florian Weimer
On 05/05/2014 12:52 PM, Lennart Poettering wrote: Have you requested official assignment? It's not yet listed in http://www.iana.org/assignments/service-names-port-numbers/service-names-port-numbers.txt. No I have not. What's the benefit of doing so? It avoids collisions, and tools like

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-02 Thread Kevin Fenzi
On Thu, 1 May 2014 16:40:48 +0200 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: ...snip... This was proposed as a Change exactly for the purpose of gathering feedback, even of the fargoing kind like that. There's lot of merit in the proposed protocol modifications, but it's a fairly

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-02 Thread Stephen Gallagher
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/01/2014 10:40 AM, Zbigniew Jędrzejewski-Szmek wrote: On Wed, Apr 30, 2014 at 12:07:25PM -0600, Kevin Fenzi wrote: Hi, sorry for the late reply, I'm away on a workshop... So, this change went to fesco last week, but there were some

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-02 Thread Lennart Poettering
On Fri, 02.05.14 10:53, Stephen Gallagher (sgall...@redhat.com) wrote: He also writes particularly since it appears to have been developed without the input of the journald creators. The code in question has been reviewed on the systemd mailing list, and discussed internally. Also,

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-02 Thread Zbigniew Jędrzejewski-Szmek
On Fri, May 02, 2014 at 10:53:59AM -0400, Stephen Gallagher wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/01/2014 10:40 AM, Zbigniew Jędrzejewski-Szmek wrote: On Wed, Apr 30, 2014 at 12:07:25PM -0600, Kevin Fenzi wrote: Hi, sorry for the late reply, I'm away on a workshop...

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-02 Thread Stephen Gallagher
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/02/2014 11:40 AM, Lennart Poettering wrote: On Fri, 02.05.14 10:53, Stephen Gallagher (sgall...@redhat.com) wrote: He also writes particularly since it appears to have been developed without the input of the journald creators. The code

Re: F21 Self Contained Change: Remote Journal Logging

2014-05-01 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 30, 2014 at 12:07:25PM -0600, Kevin Fenzi wrote: Hi, sorry for the late reply, I'm away on a workshop... So, this change went to fesco last week, but there were some questions/issues around it. Could change owners respond to: 1) sgallagh wasn't sure this was a self contained

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-30 Thread Kevin Fenzi
So, this change went to fesco last week, but there were some questions/issues around it. Could change owners respond to: 1) sgallagh wasn't sure this was a self contained change: see: https://fedorahosted.org/fesco/ticket/1250#comment:19 2) FESCo in general wondered if we advertised this as a

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Florian Weimer
On 04/16/2014 06:46 PM, Bill Nottingham wrote: I understand the pull vs push distinction ... I'm just not clear why pull would ever be a model you'd want to use. (vs something like a local cockpit agent.) Isn't remote Windows event logging pull-only (unless you somehow gate it to syslog)?

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Simo Sorce
On Tue, 2014-04-22 at 06:34 +0200, Lennart Poettering wrote: On Wed, 16.04.14 12:46, Bill Nottingham (nott...@splat.cc) wrote: Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com)

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Matthew Miller
On Tue, Apr 22, 2014 at 06:34:48AM +0200, Lennart Poettering wrote: Pull is the only model that scales, since the centralized log infrastructure can schedule when it pulls from where and thus do this according to available resources. THe push model is prone to logging bursts overwhelming log

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Miloslav Trmač
2014-04-22 15:10 GMT+02:00 Simo Sorce s...@redhat.com: A good protocol would allow to send a first small packet that establish a connection and a reply that can push back on the client w/o requiring huge bandwidth to be spent. Isn't that an inherent capability of TCP? If it is not

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Simo Sorce
On Tue, 2014-04-22 at 19:04 +0200, Miloslav Trmač wrote: 2014-04-22 15:10 GMT+02:00 Simo Sorce s...@redhat.com: A good protocol would allow to send a first small packet that establish a connection and a reply that can push back on the client w/o requiring huge bandwidth to be spent.

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Miloslav Trmač
2014-04-22 20:19 GMT+02:00 Simo Sorce s...@redhat.com: On Tue, 2014-04-22 at 19:04 +0200, Miloslav Trmač wrote: 2014-04-22 15:10 GMT+02:00 Simo Sorce s...@redhat.com: A good protocol would allow to send a first small packet that establish a connection and a reply that can push back on

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Simo Sorce
On Tue, 2014-04-22 at 20:58 +0200, Miloslav Trmač wrote: 2014-04-22 20:19 GMT+02:00 Simo Sorce s...@redhat.com: On Tue, 2014-04-22 at 19:04 +0200, Miloslav Trmač wrote: 2014-04-22 15:10 GMT+02:00 Simo Sorce s...@redhat.com: A good protocol would allow to send a first small

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Apr 22, 2014 at 06:34:48AM +0200, Lennart Poettering wrote: On Wed, 16.04.14 12:46, Bill Nottingham (nott...@splat.cc) wrote: Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Apr 22, 2014 at 03:32:26PM -0400, Simo Sorce wrote: On Tue, 2014-04-22 at 20:58 +0200, Miloslav Trmač wrote: 2014-04-22 20:19 GMT+02:00 Simo Sorce s...@redhat.com: On Tue, 2014-04-22 at 19:04 +0200, Miloslav Trmač wrote: 2014-04-22 15:10 GMT+02:00 Simo Sorce s...@redhat.com:

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Lennart Poettering
On Tue, 22.04.14 09:10, Simo Sorce (s...@redhat.com) wrote: I am pretty sure that a pull model should be the default for everything we do, and push only be done where realtimish behaviour is desired to do live debugging or suchlike. I am pretty sure the push model concept is one of the

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-22 Thread Simo Sorce
On Wed, 2014-04-23 at 05:36 +0200, Lennart Poettering wrote: On Tue, 22.04.14 09:10, Simo Sorce (s...@redhat.com) wrote: I am pretty sure that a pull model should be the default for everything we do, and push only be done where realtimish behaviour is desired to do live debugging or

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-21 Thread Lennart Poettering
On Wed, 16.04.14 12:46, Bill Nottingham (nott...@splat.cc) wrote: Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com) said: = Proposed Self Contained Change: Remote Journal Logging

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Martin Langhoff
On Wed, Apr 16, 2014 at 5:08 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: So I'll ask you about this other aspect -- what about stateless clients with very limited or no local storage? Not supported by this, unfortunately. There needs to be at least temporary storage in tmpfs for

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Zbigniew Jędrzejewski-Szmek
On Thu, Apr 17, 2014 at 07:33:16AM -0400, Martin Langhoff wrote: On Wed, Apr 16, 2014 at 5:08 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: So I'll ask you about this other aspect -- what about stateless clients with very limited or no local storage? Not supported by this,

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Miloslav Trmač
Hello, 2014-04-16 15:04 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: I'll reconsider using SASL instead. I have the HTTPS-transport version almost ready, so for now I'll go with that, to have a working solution. There's still some other questions, mostly related to how the data

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Miloslav Trmač
2014-04-16 19:08 GMT+02:00 Chris Adams li...@cmadams.net: It would be good if systemd could use or extend an existing logging protocol, rather than invent yet another method. Yes. Going by the feature page and from what I can see from journal-remote.c, because Transfer-Encoding: chunked

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Zbigniew Jędrzejewski-Szmek
On Thu, Apr 17, 2014 at 10:17:28PM +0200, Miloslav Trmač wrote: 2014-04-16 19:08 GMT+02:00 Chris Adams li...@cmadams.net: It would be good if systemd could use or extend an existing logging protocol, rather than invent yet another method. Yes. Going by the feature page and from what

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Zbigniew Jędrzejewski-Szmek
On Thu, Apr 17, 2014 at 10:12:24PM +0200, Miloslav Trmač wrote: Hello, 2014-04-16 15:04 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: I'll reconsider using SASL instead. I have the HTTPS-transport version almost ready, so for now I'll go with that, to have a working solution.

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-17 Thread Miloslav Trmač
2014-04-17 23:34 GMT+02:00 Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl: On Thu, Apr 17, 2014 at 10:17:28PM +0200, Miloslav Trmač wrote: 2014-04-16 19:08 GMT+02:00 Chris Adams li...@cmadams.net: It would be good if systemd could use or extend an existing logging protocol, rather than

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Apr 15, 2014 at 03:30:57PM -0400, Simo Sorce wrote: On Tue, 2014-04-15 at 20:28 +0200, Zbigniew Jędrzejewski-Szmek wrote: On Tue, Apr 15, 2014 at 11:00:45AM -0400, Simo Sorce wrote: On Mon, 2014-04-14 at 15:07 +0200, Jaroslav Reznik wrote: = Proposed Self Contained Change: Remote

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Frank Ch. Eigler
Zbigniew =?utf-8?Q?J=C4=99drzejewski-Szmek?= zbys...@in.waw.pl writes: [...] Using HTTP makes it possible to use e.g. use curl to upload some logs from the commandline. It should also be fairly easy for people to write e.g. Python code to upload logs. [...] Are you envisioning these journal

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 16, 2014 at 11:39:07AM -0400, Frank Ch. Eigler wrote: Zbigniew =?utf-8?Q?J=C4=99drzejewski-Szmek?= zbys...@in.waw.pl writes: [...] Using HTTP makes it possible to use e.g. use curl to upload some logs from the commandline. It should also be fairly easy for people to write

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Bill Nottingham
Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com) said: = Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Simo Sorce
On Wed, 2014-04-16 at 15:04 +0200, Zbigniew Jędrzejewski-Szmek wrote: On Tue, Apr 15, 2014 at 03:30:57PM -0400, Simo Sorce wrote: I'd imagine that in a setup with a few servers one would create the certificates on the receiver machine, copypasting some instructions from Fedora docs,

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Martin Langhoff
On Mon, Apr 14, 2014 at 9:07 AM, Jaroslav Reznik jrez...@redhat.com wrote: The communication between the two daemons is done over standard HTTPS, Interesting. One quirk of current syslog-style remote logging over UDP is that it is fairly tolerant to dataloss. With quite a bit of experience in

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Chris Adams
Once upon a time, Martin Langhoff martin.langh...@gmail.com said: On Mon, Apr 14, 2014 at 9:07 AM, Jaroslav Reznik jrez...@redhat.com wrote: The communication between the two daemons is done over standard HTTPS, Interesting. One quirk of current syslog-style remote logging over UDP is that

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 16, 2014 at 12:46:15PM -0400, Bill Nottingham wrote: Zbigniew Jędrzejewski-Szmek (zbys...@in.waw.pl) said: On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com) said: = Proposed Self Contained Change: Remote Journal Logging =

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 16, 2014 at 12:50:53PM -0400, Martin Langhoff wrote: On Mon, Apr 14, 2014 at 9:07 AM, Jaroslav Reznik jrez...@redhat.com wrote: The communication between the two daemons is done over standard HTTPS, I hear you holler OMG you have to build full redundancy in your logging backend;

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Martin Langhoff
On Wed, Apr 16, 2014 at 4:40 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: the upload client is like any other journal client -- it is fully asynchronous wrt. to journald writing log entries. (It's something like 'journalctl -o export|curl -X POST https://some.where/upload'.)

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 16, 2014 at 12:48:21PM -0400, Simo Sorce wrote: On Wed, 2014-04-16 at 15:04 +0200, Zbigniew Jędrzejewski-Szmek wrote: On Tue, Apr 15, 2014 at 03:30:57PM -0400, Simo Sorce wrote: I'd imagine that in a setup with a few servers one would create the certificates on the

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-16 Thread Zbigniew Jędrzejewski-Szmek
On Wed, Apr 16, 2014 at 04:57:25PM -0400, Martin Langhoff wrote: On Wed, Apr 16, 2014 at 4:40 PM, Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl wrote: the upload client is like any other journal client -- it is fully asynchronous wrt. to journald writing log entries. (It's something like

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-15 Thread Simo Sorce
On Mon, 2014-04-14 at 15:07 +0200, Jaroslav Reznik wrote: = Proposed Self Contained Change: Remote Journal Logging = The communication between the two daemons is done over standard HTTPS, following rather simple rules, so it is possible to create alternate implementations without much

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-15 Thread Kevin Fenzi
To be clear here, all this is implemented in the two daemons right? When you say it uses https, thats natively done in the daemons, they don't need apache or some other https implementor in the way? Which ssl stack does this use? nss? openssl? gnutls? something else? kevin signature.asc

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-15 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Apr 15, 2014 at 09:25:39AM -0600, Kevin Fenzi wrote: To be clear here, all this is implemented in the two daemons right? When you say it uses https, thats natively done in the daemons, they don't need apache or some other https implementor in the way? Yes, it's implemented in two

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-15 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Apr 15, 2014 at 11:00:45AM -0400, Simo Sorce wrote: On Mon, 2014-04-14 at 15:07 +0200, Jaroslav Reznik wrote: = Proposed Self Contained Change: Remote Journal Logging = The communication between the two daemons is done over standard HTTPS, following rather simple rules, so it is

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-15 Thread Simo Sorce
On Tue, 2014-04-15 at 20:28 +0200, Zbigniew Jędrzejewski-Szmek wrote: On Tue, Apr 15, 2014 at 11:00:45AM -0400, Simo Sorce wrote: On Mon, 2014-04-14 at 15:07 +0200, Jaroslav Reznik wrote: = Proposed Self Contained Change: Remote Journal Logging = The communication between the two

F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Jaroslav Reznik
= Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging Change owner(s): Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl Systemd journal can be configured to forward events to a remote server. Entries are forwarded including full

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Alexander Bokovoy
On Mon, 14 Apr 2014, Jaroslav Reznik wrote: = Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging Change owner(s): Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl Systemd journal can be configured to forward events to a remote

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 14, 2014 at 05:19:17PM +0300, Alexander Bokovoy wrote: How certificates are managed for sender and receiver parts? By some external means... This could be automated, e.g. using certmaster, but I don't want to tie to a specific certificate distribution implementation. Who generates

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Alexander Bokovoy
On Mon, 14 Apr 2014, Zbigniew Jędrzejewski-Szmek wrote: On Mon, Apr 14, 2014 at 05:19:17PM +0300, Alexander Bokovoy wrote: How certificates are managed for sender and receiver parts? By some external means... This could be automated, e.g. using certmaster, but I don't want to tie to a specific

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Bill Nottingham
Jaroslav Reznik (jrez...@redhat.com) said: = Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging Change owner(s): Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl Systemd journal can be configured to forward events to a

Re: F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Zbigniew Jędrzejewski-Szmek
On Mon, Apr 14, 2014 at 04:20:16PM -0400, Bill Nottingham wrote: Jaroslav Reznik (jrez...@redhat.com) said: = Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging Change owner(s): Zbigniew Jędrzejewski-Szmek

F21 Self Contained Change: Remote Journal Logging

2014-04-14 Thread Jaroslav Reznik
= Proposed Self Contained Change: Remote Journal Logging = https://fedoraproject.org/wiki/Changes/Remote_Journal_Logging Change owner(s): Zbigniew Jędrzejewski-Szmek zbys...@in.waw.pl Systemd journal can be configured to forward events to a remote server. Entries are forwarded including full