Re: [systemd-devel] hostnamectl question

2014-06-17 Thread Mantas Mikulėnas
On Tue, Jun 17, 2014 at 8:46 AM, Vasiliy Tolstov v.tols...@selfip.ru wrote:
 I have a problem with uuid on my servers - they all blade servers and
 uuid in smbios identical.
 But i need random uniq id for each system. I can't relay on mac
 address os something like this.
 AS i see systemd have machine id and boot id. And in diskless systems
 they regenerated every boot.
 What is the difference of boot id and machined id ?

Boot ID is *always* generated by the kernel on every boot and not
stored anywhere, only available from sysctl.

Machine ID is usually static -- generated once after installation, and
stored in /etc/machine-id. It only changes on diskless systems or if
/etc is unwritable for some other reason. (systemd can get the machine
ID from /sys/class/dmi/id/product_uuid, but only does so on qemu-kvm
VMs.)

-- 
Mantas Mikulėnas graw...@gmail.com
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] hostnamectl question

2014-06-17 Thread Vasiliy Tolstov
2014-06-17 10:32 GMT+04:00 Mantas Mikulėnas graw...@gmail.com:
 Boot ID is *always* generated by the kernel on every boot and not
 stored anywhere, only available from sysctl.

 Machine ID is usually static -- generated once after installation, and
 stored in /etc/machine-id. It only changes on diskless systems or if
 /etc is unwritable for some other reason. (systemd can get the machine
 ID from /sys/class/dmi/id/product_uuid, but only does so on qemu-kvm
 VMs.)


Thanks!

-- 
Vasiliy Tolstov,
e-mail: v.tols...@selfip.ru
jabber: v...@selfip.ru
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Duncan
Goffredo Baroncelli posted on Mon, 16 Jun 2014 20:47:49 +0200 as
excerpted:

 The test #6 and #7 suggested that the fsync(2) amd posix_fallocate(3)
 calls aren't the root cause of the problem. Even without these the
 system.journal file still fragments.
 
 [1] http://kreijack.blogspot.it/2014/06/btrfs-and-systemd-journal.html

Thanks for injecting some concrete facts in the form of test results into 
things. =:^)

So I was barking up the wrong tree with fallocate.

But I had the base problem correct -- fragmentation -- and good 
recommendations for working around it -- NOCOW and/or (auto)defrag.
(And for that matter, for those like me where /var/log is a dedicated 
partition, simply putting /var/log on something other than btrfs would 
nicely work around the issue too.

The experts are working on it now, so I'll step out for the most part as 
what else I could add at this point would be simply noise.  But I did 
consider it important to acknowledge that I had been wrong about the 
fallocate.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] PATCH: fix systemd-bootchart svg background

2014-06-17 Thread Frederic Crozat
Le lundi 16 juin 2014 à 10:13 -0700, Kok, Auke-jan H a écrit :
 On Mon, Jun 16, 2014 at 9:52 AM, Frederic Crozat fcro...@suse.com wrote:
  Hi all,
 
  attached is a fix for SVG generated by systemd-bootchart, similar to a
  fix already done in systemd-analyze.
 
 yeah, that's a nice change. Looks good to me.

Great. Could somebody push it on git, I don't have write privilege
there ?

Thanks !

-- 
Frederic Crozat
Project Manager Enterprise Desktop
SUSE

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Duncan
Lennart Poettering posted on Mon, 16 Jun 2014 00:39:39 +0200 as excerpted:

 Well, quite frankly I am not entirely sure why fallocate()

I was barking up the wrong tree with fallocate().  Sorry.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Duncan
Lennart Poettering posted on Mon, 16 Jun 2014 00:43:34 +0200 as excerpted:

  At least here, I interpreted that remark as primarily sarcastic
  commentary on the systemd devs' apparent attitude, which can be
  (controversially) summarized as: Systemd doesn't have problems
  because it's perfect.  Therefore, any problems you have with systemd
  must instead be with other components which systemd depends on.
 
 Come on, sorry, but this is fud. Really... ;-)
 
 Interestingly, I never commented on anything in this area, and neither
 did anybody else from the systemd side afaics. THe entire btrfs defrag
 thing i wasn't aware of before this thread started on the system ML a
 few days ago. I am not sure where you take your ideas about our
 attitude from. God, with behaviour like that you just make us ignore
 you, Duncan.

Sorry.

As you'll note, I said can be controversially...

I never stated that *I* held that position personally, as I was taking 
the third-person observer position.  As such, I've seen that attitude 
expressed by others in multiple threads when systemd comes up (as Josef 
alluded to as well), and that I simply interpreted the remark to which I 
was alluding (And that's now a btrfs problem :/) as a sarcastic 
reference to that attitude... which again I never claimed as my own.

Thus the whoosh, in reference to what I interpreted as sarcasm, which 
certainly doesn't require fully agreement with in ordered to understand.  
(Tho as I mentioned elsewhere, I can certainly see where they're coming 
from given the recent kernel debug kerfuffle and the like... and I'm 
certainly not alone there... but I'm /trying/ to steer a reasonably 
neutral path while appreciating both sides.)

In fact, I specifically stated elsewhere that in fact I recently switched 
to systemd -- by choice as I'm on gentoo which still defaults to openrc 
-- myself.  Certainly I would not have done so if I believed systemd was 
as bad as all that, and the fact that I HAVE done so definitely implies a 
rather large amount of both trust and respect in the systemd devs, or I'd 
not be willing to run their code.

But never-the-less I can see the viewpoint from both sides now, and do 
try to maintain a reasonable neutrality.  I guess I should have made that 
more explicit in the original post, but as they say, hindsight is 20/20. 
=:^\

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd not initializing

2014-06-17 Thread Jay D Bhatt
Hi,

From the log I attached previously, I think is log statement saying as below:

 Starting Login Service...

But then , it gets stuck at 

[  OK  ] Reached target Login Prompts.

So, I think there would be problem with systemd-login.service file for so? Can 
anybody point, why there is no move ahead after this statement?

Thanks,
Jay

~~Disclaimer~~~
Information contained and transmitted by this e-mail is confidential and 
proprietary to IGATE and its affiliates and is intended for use only by the 
recipient. If you are not the intended recipient, you are hereby notified that 
any dissemination, distribution, copying or use of this e-mail is strictly 
prohibited and you are requested to delete this e-mail immediately and notify 
the originator or mailad...@igate.com mailto:mailad...@igate.com. IGATE does 
not enter into any agreement with any party by e-mail. Any views expressed by 
an individual do not necessarily reflect the view of IGATE. IGATE is not 
responsible for the consequences of any actions taken on the basis of 
information provided, through this email. The contents of an attachment to this 
e-mail may contain software viruses, which could damage your own computer 
system. While IGATE has taken every reasonable precaution to minimise this 
risk, we cannot accept liability for any damage which you sustain as a result 
of softwar
 e viruses. You should carry out your own virus checks before opening an 
attachment. To know more about IGATE please visit www.igate.com 
http://www.igate.com.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] man: systemd.net devwork

2014-06-17 Thread poma


up-to-date systemd.net workdev.


poma

diff --git a/man/systemd.netdev.xml b/man/systemd.netdev.xml
index c17ae9e..3bb72a8 100644
--- a/man/systemd.netdev.xml
+++ b/man/systemd.netdev.xml
@@ -142,11 +142,36 @@
 termvarnameKind=/varname/term
 listitem
 paraThe netdev kind. Currently, literalbridge/literal,
-literalbond/literal, literalvlan/literal and
-literalmacvlan/literal are supported. This option
+literalbond/literal, literalvlan/literal,
+literalmacvlan/literal, literalvxlan/literal,
+literalipip/literal, literalsit/literal,
+literalgre/literal, literalvti/literal and
+literalveth/literal are supported. This option
 is compulsory./para
 /listitem
 /varlistentry
+varlistentry
+termvarnameMTUBytes=/varname/term
+listitem
+paraThe maximum transmission unit in bytes to
+set for the device. The usual suffixes K, M, G,
+are supported and are understood to the base of
+1024./para
+/listitem
+/varlistentry
+varlistentry
+termvarnameMACAddress=/varname/term
+listitem
+paraThe MAC address to use. Currently,
+literalbridge/literal, literalvlan/literal,
+literalmacvlan/literal, literalvxlan/literal,
+literalipip/literal, literalsit/literal,
+literalgre/literal, literalvti/literal and
+literalveth/literal are supported.
+It is currently tested only with the literalbridge/literal.
+/para
+/listitem
+/varlistentry
 /variablelist
 
 paraThe literal[VLAN]/literal section only applies for netdevs of kind literalvlan/literal,
@@ -177,6 +202,101 @@
 /varlistentry
 /variablelist
 
+paraThe literal[VXLAN]/literal section only applies for netdevs of kind
+literalvxlan/literal, and accepts the following key:/para
+
+variablelist class='network-directives'
+varlistentry
+termvarnameId=/varname/term
+listitem
+paraThe VXLAN ID to use./para
+/listitem
+/varlistentry
+varlistentry
+termvarnameGroup=/varname/term
+listitem
+paraAn assigned multicast group IP address./para
+/listitem
+/varlistentry
+varlistentry
+termvarnameTOS=/varname/term
+listitem
+paraThe Type Of Service byte value for a vxlan interface./para
+/listitem
+/varlistentry
+varlistentry
+termvarnameTTL=/varname/term
+listitem
+paraA fixed Time To Live N on Virtual eXtensible Local Area Network packets.
+N is a number in the range 1-255. 0 is a special value meaning that packets
+inherit the TTL value./para
+/listitem

Re: [systemd-devel] PATCH: fix systemd-bootchart svg background

2014-06-17 Thread Tom Gundersen
On Tue, Jun 17, 2014 at 9:58 AM, Frederic Crozat fcro...@suse.com wrote:
 Le lundi 16 juin 2014 à 10:13 -0700, Kok, Auke-jan H a écrit :
 On Mon, Jun 16, 2014 at 9:52 AM, Frederic Crozat fcro...@suse.com wrote:
  Hi all,
 
  attached is a fix for SVG generated by systemd-bootchart, similar to a
  fix already done in systemd-analyze.

 yeah, that's a nice change. Looks good to me.

 Great. Could somebody push it on git, I don't have write privilege
 there ?

Done.

-t
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] Systemd-networkd: Prepend static nameserver to DHCP-discovered nameservers

2014-06-17 Thread Runiq

Hello all,

I'm using systemd 213 on Arch Linux, and systemd-networkd/resolved with 
DHCP to connect to the internet. I'm also running a caching DNS server 
on 127.0.0.1.


I'd like to make this caching server the first DNS server in the list, 
but I'd also like to use the nameservers discovered by systemd-resolved. 
Using a static resolv.conf isn't really possible, because I connect to 
networks with different DNS settings.


I know I can set *fallback* DNS servers in /etc/systemd/resolved.conf, 
but is there a way with systemd-networkd to prepend my local DNS server 
to those discovered by DHCP?


I've asked this question on superuser.com [1] but didn't get a 
satisfactory reply.


Thanks,
Patrice

[1] http://superuser.com/q/767827/260249


pgppYs2St3j_F.pgp
Description: PGP signature
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Systemd-networkd: Prepend static nameserver to DHCP-discovered nameservers

2014-06-17 Thread Mantas Mikulėnas
I am not sure about the part where you talk about different networks' DNS
settings.

I mean, as long as the first-listed server responds – and localhost always
responds – then the fallback servers won't be used at all.

So what's the point of having both 127.0.0.1 and external servers listed
(whether static or not)? Having them static won't make any difference.

-- 
Mantas Mikulėnas graw...@gmail.com
// sent from phone
On Jun 17, 2014 2:14 PM, Runiq ru...@archlinux.us wrote:

 Hello all,

 I'm using systemd 213 on Arch Linux, and systemd-networkd/resolved with
 DHCP to connect to the internet. I'm also running a caching DNS server on
 127.0.0.1.

 I'd like to make this caching server the first DNS server in the list, but
 I'd also like to use the nameservers discovered by systemd-resolved. Using
 a static resolv.conf isn't really possible, because I connect to networks
 with different DNS settings.

 I know I can set *fallback* DNS servers in /etc/systemd/resolved.conf, but
 is there a way with systemd-networkd to prepend my local DNS server to
 those discovered by DHCP?

 I've asked this question on superuser.com [1] but didn't get a
 satisfactory reply.

 Thanks,
 Patrice

 [1] http://superuser.com/q/767827/260249

 ___
 systemd-devel mailing list
 systemd-devel@lists.freedesktop.org
 http://lists.freedesktop.org/mailman/listinfo/systemd-devel


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Systemd-networkd: Prepend static nameserver to DHCP-discovered nameservers

2014-06-17 Thread Mantas Mikulėnas
Actually, one thing I just remembered.

resolved never actually writes to /etc/resolv.conf, if I remember
correctly. It only writes to a .conf in /run, and /etc/resolv.conf is just
a symlink to the latter.

So you could just have a static /etc/resolv.conf with 127.0.0.1 in it, and
tell your cache to look in /run/systemd/resolv.conf for the upstream
servers' addresses. I'm sure at least dnsmasq has that ability.

-- 
Mantas Mikulėnas graw...@gmail.com
// sent from phone
On Jun 17, 2014 2:14 PM, Runiq ru...@archlinux.us wrote:

 Hello all,

 I'm using systemd 213 on Arch Linux, and systemd-networkd/resolved with
 DHCP to connect to the internet. I'm also running a caching DNS server on
 127.0.0.1.

 I'd like to make this caching server the first DNS server in the list, but
 I'd also like to use the nameservers discovered by systemd-resolved. Using
 a static resolv.conf isn't really possible, because I connect to networks
 with different DNS settings.

 I know I can set *fallback* DNS servers in /etc/systemd/resolved.conf, but
 is there a way with systemd-networkd to prepend my local DNS server to
 those discovered by DHCP?

 I've asked this question on superuser.com [1] but didn't get a
 satisfactory reply.

 Thanks,
 Patrice

 [1] http://superuser.com/q/767827/260249

 ___
 systemd-devel mailing list
 systemd-devel@lists.freedesktop.org
 http://lists.freedesktop.org/mailman/listinfo/systemd-devel


___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Martin
On 17/06/14 09:17, Duncan wrote:
[...]
 
 But never-the-less I can see the viewpoint from both sides now, and do 
 try to maintain a reasonable neutrality.  I guess I should have made that 
 more explicit in the original post, but as they say, hindsight is 20/20. 
 =:^\

Hey! All good for a giggle and excellent for friendly attention to get
things properly fixed :-)

And this thread may even hit the news for Phoronix and LWN and others,
all in true FLOSS open development and thorough debug :-P

And... And... We get a few bits of systemd, btrfs, and filesystem
interactions fixed for how things /should/ work best.


Thanks (and for others also) for stirring a rapid chase! ;-)

Regards,
Martin

(A fellow Gentoo-er ;-) )

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] systemd not initializing

2014-06-17 Thread Kirill Elagin
Please, try to ssh into the system when it is stuck and show us the log
from `journalctl -b`.

Or, alternatively, do as earlier, reboot the stuck system, boot with
init=/bin/sh and get the log with `journalctl -b -1`.
But ssh is preferred as it will let you troubleshoot the issue further
interactively.
On Jun 17, 2014 1:23 PM, Jay D Bhatt jay.bh...@igate.com wrote:

 Hi,

 From the log I attached previously, I think is log statement saying as
 below:

  Starting Login Service...

 But then , it gets stuck at

 [  OK  ] Reached target Login Prompts.

 So, I think there would be problem with systemd-login.service file for so?
 Can anybody point, why there is no move ahead after this statement?

 Thanks,
 Jay


 ~~Disclaimer~~~
 Information contained and transmitted by this e-mail is confidential and
 proprietary to IGATE and its affiliates and is intended for use only by the
 recipient. If you are not the intended recipient, you are hereby notified
 that any dissemination, distribution, copying or use of this e-mail is
 strictly prohibited and you are requested to delete this e-mail immediately
 and notify the originator or mailad...@igate.com mailto:
 mailad...@igate.com. IGATE does not enter into any agreement with any
 party by e-mail. Any views expressed by an individual do not necessarily
 reflect the view of IGATE. IGATE is not responsible for the consequences of
 any actions taken on the basis of information provided, through this email.
 The contents of an attachment to this e-mail may contain software viruses,
 which could damage your own computer system. While IGATE has taken every
 reasonable precaution to minimise this risk, we cannot accept liability for
 any damage which you sustain as a result of softwar
  e viruses. You should carry out your own virus checks before opening an
 attachment. To know more about IGATE please visit www.igate.com 
 http://www.igate.com.

 
 ___
 systemd-devel mailing list
 systemd-devel@lists.freedesktop.org
 http://lists.freedesktop.org/mailman/listinfo/systemd-devel

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Systemd-networkd: Prepend static nameserver to DHCP-discovered nameservers

2014-06-17 Thread David Timothy Strauss
On Tue, Jun 17, 2014 at 4:33 AM, Mantas Mikulėnas graw...@gmail.com wrote:
 I mean, as long as the first-listed server responds – and localhost always
 responds – then the fallback servers won't be used at all.

Localhost can be subject to two types of failure:
 * The local daemon being down.
 * The resources the local daemon uses being inaccessible. It is
possible for root servers, for example, to be inaccessible while the
DHCP-issued nameservers can still be. Not all local daemons will
prefer to use the DHCP-issued nameservers as their upstream,
particularly if DNSSec is preferred/required.

Therefore, it can be useful to have a local daemon followed by
fallback to using the DHCP-issued nameservers.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] util.h: KDBUS_ITEM_VALID fixed

2014-06-17 Thread Kay Sievers
On Tue, Jun 17, 2014 at 12:38 PM, Jacek Janczyk j.janc...@samsung.com wrote:
 Missing character leading to wrong variable name restored.

 Signed-off-by: Jacek Janczyk j.janc...@samsung.com
 ---
  util.h |2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

Applied.

Thanks,
Kay
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Goffredo Baroncelli
Hi Kai,


I investigate a bit why readahead doesn't defrag the journal.
I put in CC also the mailing list for the record.

On 06/17/2014 03:33 AM, Kai Krakow wrote:
[...]
 Instead, for me, the readahead collector catches access to my system
 journal and thus defragments it. That's not the case for your system
 which explains why it won't be defragmented for you when enabling
 readahead.

The readahead program skips file greater than 10M. If you look at its man page, 
it documents the switch --file-size-max. The default value is 
READAHEAD_FILE_SIZE_MAX which is equal to 10M.
So only when the system.journal is smaller than this value, the readahead 
defrags it.
But this happens only few times. 

This behavior seems to me reasonable to avoid blocking a system when a big file 
is touched during the boot.

BR
G.Baroncelli

-- 
gpg @keyserver.linux.it: Goffredo Baroncelli (kreijackATinwind.it
Key fingerprint BBF5 1610 0B64 DAC6 5F7D  17B2 0EDA 9B37 8B82 E0B5
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] [PATCH] build-sys: add missing backslash

2014-06-17 Thread Ronny Chevalier
---
 Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/Makefile.am b/Makefile.am
index 0213c38..e428141 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -3537,7 +3537,7 @@ libsystemd_journal_internal_la_SOURCES = \
src/journal/mmap-cache.h
 
 # using _CFLAGS = in the conditional below would suppress AM_CFLAGS
-libsystemd_journal_internal_la_CFLAGS =
+libsystemd_journal_internal_la_CFLAGS = \
$(AM_CFLAGS)
 
 libsystemd_journal_internal_la_LIBADD =
-- 
2.0.0

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Chris Murphy

On Jun 16, 2014, at 7:13 PM, cwillu cwi...@cwillu.com wrote:

 It's not a mmap problem, it's a small writes with an msync or fsync
 after each one problem.
 
 For the case of sequential writes (via write or mmap), padding writes
 to page boundaries would help, if the wasted space isn't an issue.
 Another approach, again assuming all other writes are appends, would
 be to periodically (but frequently enough that the pages are still in
 cache) read a chunk of the file and write it back in-place, with or
 without an fsync. On the other hand, if you can afford to lose some
 logs on a crash, not fsyncing/msyncing after each write will also
 eliminate the fragmentation.

Normally I'd be willing to give up ~30 seconds of journal to not have 
fragmented journals. But then if I use systemd.log_level=debug I'd like that to 
trigger more frequent flushing to make sure as little of the journaling is lost 
as possible. Does that make sense?


Chris Murphy
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Kai Krakow
Goffredo Baroncelli kreij...@libero.it schrieb:

 I investigate a bit why readahead doesn't defrag the journal.
 I put in CC also the mailing list for the record.
 
 On 06/17/2014 03:33 AM, Kai Krakow wrote:
 [...]
 Instead, for me, the readahead collector catches access to my system
 journal and thus defragments it. That's not the case for your system
 which explains why it won't be defragmented for you when enabling
 readahead.
 
 The readahead program skips file greater than 10M. If you look at its man
 page, it documents the switch --file-size-max. The default value is
 READAHEAD_FILE_SIZE_MAX which is equal to 10M. So only when the
 system.journal is smaller than this value, the readahead defrags it. But
 this happens only few times.
 
 This behavior seems to me reasonable to avoid blocking a system when a big
 file is touched during the boot.

Ah okay, good catch... Well, systemd does not generate massive amounts of 
logs for me. Maybe you want to play around with SystemMaxUse in 
journald.conf to let the file size stay below 10M?

Actually, for servers using syslog, I set the log file rotation to kick in 
after 10M worth of logging (or after 7 days, whatever comes first) so our 
rsync-diffed backups stay small for log files as otherwise it would start 
creating huge diffs just for backups when I do the rsync --backup way of 
doing backups.

So that may actually be a not too bad option. I'm not using it with my home 
system because I do snapshot-based backups to btrfs and diffs are just much 
smaller (using rsync --no-whole-file --inplace).

We recently switched to zfs based backup storage for our servers, and in 
that process switched to using rsync --no-whole-file --inplace, too. So we 
won't need the small log rotation scheme any longer but I feel comfortable 
with it so I stick to this.

BTW: I checked our systemd-enabled servers and fragmentation of journal 
files is very very high there, too. Way above 5000 extents. The journals are 
on xfs so it suffers that problem, too (which already has been pointed out 
somewhere in this thread multiple times). But to my surprise, this does not 
affect the performance of journalctl there. The bad performance is only for 
fragmentation on btrfs, not xfs. IMHO, this pushes the pointer a little bit 
more towards systemd, tho I couldn't suggest how to fix it. Time for the 
filesystem hackers to kick in.

BTW: The servers are virtualized so by default, systemd-readahead does 
nothing there because of ConditionVirtual in the service files. OTOH, it 
won't defragment there anyways because xfs is not supported for this in 
systemd, altough it would probably be possible because there is xfs_fsr. 
But my feeling is that xfs_fsr is more or less just locking the file, 
copying it, then replaces the original. It cannot work on files in use. This 
is different for btrfs which only chokes on defragmenting files currently 
mmap'ed (e.g., binaries being executed, read: busy text files). Following 
that thought, I wonder why it seems to defragment system.journal for me 
because after all I learned here, systemd writes to the files mmap'ed.

-- 
Replies to list only preferred.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Kai Krakow
Chris Murphy li...@colorremedies.com schrieb:

 It's not a mmap problem, it's a small writes with an msync or fsync
 after each one problem.
 
 For the case of sequential writes (via write or mmap), padding writes
 to page boundaries would help, if the wasted space isn't an issue.
 Another approach, again assuming all other writes are appends, would
 be to periodically (but frequently enough that the pages are still in
 cache) read a chunk of the file and write it back in-place, with or
 without an fsync. On the other hand, if you can afford to lose some
 logs on a crash, not fsyncing/msyncing after each write will also
 eliminate the fragmentation.
 
 Normally I'd be willing to give up ~30 seconds of journal to not have
 fragmented journals. But then if I use systemd.log_level=debug I'd like
 that to trigger more frequent flushing to make sure as little of the
 journaling is lost as possible. Does that make sense?

AFAIR systemd-journald already does explicit flushing when more important 
log entries hit the log. Well, of course debug is one of the least 
important log levels in that chain. But using the same code paths, it should 
be trivial to implement a configurable trigger which also forces explicit 
flushing...

-- 
Replies to list only preferred.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Dave Reisner
This lets KERNELDIR apply to the install target as well so that you can
do something such as the following will Just Work™:

  make KERNELDIR=/lib/modules/3.15.0-foo install
---
 Makefile | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/Makefile b/Makefile
index c593b51..fe4dd58 100644
--- a/Makefile
+++ b/Makefile
@@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
 
 obj-m += kdbus$(EXT).o
 
-KERNELDIR  ?= /lib/modules/$(shell uname -r)/build
+KERNELDIR  ?= /lib/modules/$(shell uname -r)
 PWD:= $(shell pwd)
 
 all: module test
@@ -26,7 +26,7 @@ test::
$(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
 
 module:
-   $(MAKE) -C $(KERNELDIR) M=$(PWD)
+   $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
 
 clean:
rm -f *.o *~ core .depend .*.cmd *.ko *.mod.c
@@ -38,15 +38,15 @@ check:
test/test-kdbus
 
 install: module
-   mkdir -p /lib/modules/$(shell uname -r)/kernel/drivers/kdbus$(EXT)/
-   cp -f kdbus$(EXT).ko /lib/modules/$(shell uname 
-r)/kernel/drivers/kdbus$(EXT)/
-   depmod $(shell uname -r)
+   mkdir -p $(KERNELDIR)/kernel/drivers/kdbus$(EXT)/
+   cp -f kdbus$(EXT).ko $(KERNELDIR)/kernel/drivers/kdbus$(EXT)/
+   depmod $(notdir $(patsubst %/, %, $(KERNELDIR)))
 
 uninstall:
-   rm -f /lib/modules/$(shell uname -r)/kernel/drivers/kdbus/kdbus$(EXT).ko
+   rm -f $(KERNELDIR)/kernel/drivers/kdbus/kdbus$(EXT).ko
 
 coccicheck:
-   $(MAKE) -C $(KERNELDIR) M=$(PWD) coccicheck
+   $(MAKE) -C $(KERNELDIR)/build M=$(PWD) coccicheck
 
 tt: all
sudo sh -c 'dmesg -c  /dev/null'
-- 
2.0.0

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Greg KH
On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
 This lets KERNELDIR apply to the install target as well so that you can
 do something such as the following will Just Work™:
 
   make KERNELDIR=/lib/modules/3.15.0-foo install
 ---
  Makefile | 14 +++---
  1 file changed, 7 insertions(+), 7 deletions(-)
 
 diff --git a/Makefile b/Makefile
 index c593b51..fe4dd58 100644
 --- a/Makefile
 +++ b/Makefile
 @@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
  
  obj-m += kdbus$(EXT).o
  
 -KERNELDIR?= /lib/modules/$(shell uname -r)/build
 +KERNELDIR?= /lib/modules/$(shell uname -r)
  PWD  := $(shell pwd)
  
  all: module test
 @@ -26,7 +26,7 @@ test::
   $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
  
  module:
 - $(MAKE) -C $(KERNELDIR) M=$(PWD)
 + $(MAKE) -C $(KERNELDIR)/build M=$(PWD)

Nope, you just broke the build on my machine when I wanted to build
against the kernel source tree in /home/gregkh/linux/ that does not have
a build/ subdirectory in it.

What is wrong with putting the build/ trailing subdir in your build
command line?  What is currently broken today with the build system?

And long-term, this will not be an issue at all as the code will be
merged into the kernel tree.

thanks,

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Dave Reisner
On Tue, Jun 17, 2014 at 12:28:27PM -0700, Greg KH wrote:
 On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
  This lets KERNELDIR apply to the install target as well so that you can
  do something such as the following will Just Work™:
  
make KERNELDIR=/lib/modules/3.15.0-foo install
  ---
   Makefile | 14 +++---
   1 file changed, 7 insertions(+), 7 deletions(-)
  
  diff --git a/Makefile b/Makefile
  index c593b51..fe4dd58 100644
  --- a/Makefile
  +++ b/Makefile
  @@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
   
   obj-m += kdbus$(EXT).o
   
  -KERNELDIR  ?= /lib/modules/$(shell uname -r)/build
  +KERNELDIR  ?= /lib/modules/$(shell uname -r)
   PWD:= $(shell pwd)
   
   all: module test
  @@ -26,7 +26,7 @@ test::
  $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
   
   module:
  -   $(MAKE) -C $(KERNELDIR) M=$(PWD)
  +   $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
 
 Nope, you just broke the build on my machine when I wanted to build
 against the kernel source tree in /home/gregkh/linux/ that does not have
 a build/ subdirectory in it.

Fair enough. I figured this would be the response I get [0].

 What is wrong with putting the build/ trailing subdir in your build
 command line?  What is currently broken today with the build system?

As the commit message implies, make install doesn't work for non-running
kernels -- it will always install to the current kernel. So, after a
kernel upgrade, I have to make the subdirectories myself, copy over the
module, and run depmod before I can build an initramfs for the new
kernel (and subsequently reboot).

 And long-term, this will not be an issue at all as the code will be
 merged into the kernel tree.

Sure, but unless I'm missing something, the status quo makes things a
bit tedious for rebuilds. Most other out of tree modules I've come in
contact with make this a trivial operation.

Cheers,
Dave

[0] http://xkcd.com/1172/
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Greg KH
On Tue, Jun 17, 2014 at 03:53:15PM -0400, Dave Reisner wrote:
 On Tue, Jun 17, 2014 at 12:28:27PM -0700, Greg KH wrote:
  On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
   This lets KERNELDIR apply to the install target as well so that you can
   do something such as the following will Just Work™:
   
 make KERNELDIR=/lib/modules/3.15.0-foo install
   ---
Makefile | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
   
   diff --git a/Makefile b/Makefile
   index c593b51..fe4dd58 100644
   --- a/Makefile
   +++ b/Makefile
   @@ -17,7 +17,7 @@ kdbus$(EXT)-y := \

obj-m += kdbus$(EXT).o

   -KERNELDIR?= /lib/modules/$(shell uname -r)/build
   +KERNELDIR?= /lib/modules/$(shell uname -r)
PWD  := $(shell pwd)

all: module test
   @@ -26,7 +26,7 @@ test::
 $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)

module:
   - $(MAKE) -C $(KERNELDIR) M=$(PWD)
   + $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
  
  Nope, you just broke the build on my machine when I wanted to build
  against the kernel source tree in /home/gregkh/linux/ that does not have
  a build/ subdirectory in it.
 
 Fair enough. I figured this would be the response I get [0].

Um, that should be the workflow of _any_ kernel developer, we all build
against local kernel directories, not one that is installed in /lib/.
Some of us don't ever boot kernels installed in /lib :)

  What is wrong with putting the build/ trailing subdir in your build
  command line?  What is currently broken today with the build system?
 
 As the commit message implies, make install doesn't work for non-running
 kernels -- it will always install to the current kernel. So, after a
 kernel upgrade, I have to make the subdirectories myself, copy over the
 module, and run depmod before I can build an initramfs for the new
 kernel (and subsequently reboot).

Then fix up the install target, don't break the build target :)

  And long-term, this will not be an issue at all as the code will be
  merged into the kernel tree.
 
 Sure, but unless I'm missing something, the status quo makes things a
 bit tedious for rebuilds. Most other out of tree modules I've come in
 contact with make this a trivial operation.

What do their makefiles look like that is different here?  The module:
target should be the same, right?

thanks,

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Simon Peeters
2014-06-17 22:16 GMT+02:00 Greg KH g...@kroah.com:
 On Tue, Jun 17, 2014 at 03:53:15PM -0400, Dave Reisner wrote:
 On Tue, Jun 17, 2014 at 12:28:27PM -0700, Greg KH wrote:
  On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
   This lets KERNELDIR apply to the install target as well so that you can
   do something such as the following will Just Work™:
  
 make KERNELDIR=/lib/modules/3.15.0-foo install
   ---
Makefile | 14 +++---
1 file changed, 7 insertions(+), 7 deletions(-)
  
   diff --git a/Makefile b/Makefile
   index c593b51..fe4dd58 100644
   --- a/Makefile
   +++ b/Makefile
   @@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
  
obj-m += kdbus$(EXT).o
  
   -KERNELDIR?= /lib/modules/$(shell uname -r)/build
   +KERNELDIR?= /lib/modules/$(shell uname -r)
PWD  := $(shell pwd)
  
all: module test
   @@ -26,7 +26,7 @@ test::
 $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
  
module:
   - $(MAKE) -C $(KERNELDIR) M=$(PWD)
   + $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
 
  Nope, you just broke the build on my machine when I wanted to build
  against the kernel source tree in /home/gregkh/linux/ that does not have
  a build/ subdirectory in it.

 Fair enough. I figured this would be the response I get [0].

 Um, that should be the workflow of _any_ kernel developer, we all build
 against local kernel directories, not one that is installed in /lib/.
 Some of us don't ever boot kernels installed in /lib :)


Hej all,

I have (locally) a patch that acomplishesh almost the same, but in a
way that does not break gregs setup.

What I did was:
 - Add a KERNELVER variable defaulting to $(shell uname -r)
 - replace all occurences of $(shell uname -r) with $(KERNELVER)

maybe this is a viable solution?

It allows people to use the default paths (/lib/modules/*/build) but
for a different kernel version, while still allowing users to set
KERNELDIR to whatever they want.

I hope this is usefull


Simon Peeters

PS: if you want me to sent this as a decent patch, let me know and I
do it once I am on my dev machine.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Greg KH
On Tue, Jun 17, 2014 at 10:43:39PM +0200, Simon Peeters wrote:
 2014-06-17 22:16 GMT+02:00 Greg KH g...@kroah.com:
  On Tue, Jun 17, 2014 at 03:53:15PM -0400, Dave Reisner wrote:
  On Tue, Jun 17, 2014 at 12:28:27PM -0700, Greg KH wrote:
   On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
This lets KERNELDIR apply to the install target as well so that you can
do something such as the following will Just Work™:
   
  make KERNELDIR=/lib/modules/3.15.0-foo install
---
 Makefile | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)
   
diff --git a/Makefile b/Makefile
index c593b51..fe4dd58 100644
--- a/Makefile
+++ b/Makefile
@@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
   
 obj-m += kdbus$(EXT).o
   
-KERNELDIR?= /lib/modules/$(shell uname -r)/build
+KERNELDIR?= /lib/modules/$(shell uname -r)
 PWD  := $(shell pwd)
   
 all: module test
@@ -26,7 +26,7 @@ test::
  $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
   
 module:
- $(MAKE) -C $(KERNELDIR) M=$(PWD)
+ $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
  
   Nope, you just broke the build on my machine when I wanted to build
   against the kernel source tree in /home/gregkh/linux/ that does not have
   a build/ subdirectory in it.
 
  Fair enough. I figured this would be the response I get [0].
 
  Um, that should be the workflow of _any_ kernel developer, we all build
  against local kernel directories, not one that is installed in /lib/.
  Some of us don't ever boot kernels installed in /lib :)
 
 
 Hej all,
 
 I have (locally) a patch that acomplishesh almost the same, but in a
 way that does not break gregs setup.
 
 What I did was:
  - Add a KERNELVER variable defaulting to $(shell uname -r)
  - replace all occurences of $(shell uname -r) with $(KERNELVER)
 
 maybe this is a viable solution?

Yes, it's the correct solution, please send it.

thanks,

greg k-h
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Dave Reisner
On Tue, Jun 17, 2014 at 10:43:39PM +0200, Simon Peeters wrote:
 2014-06-17 22:16 GMT+02:00 Greg KH g...@kroah.com:
  On Tue, Jun 17, 2014 at 03:53:15PM -0400, Dave Reisner wrote:
  On Tue, Jun 17, 2014 at 12:28:27PM -0700, Greg KH wrote:
   On Tue, Jun 17, 2014 at 02:52:43PM -0400, Dave Reisner wrote:
This lets KERNELDIR apply to the install target as well so that you can
do something such as the following will Just Work™:
   
  make KERNELDIR=/lib/modules/3.15.0-foo install
---
 Makefile | 14 +++---
 1 file changed, 7 insertions(+), 7 deletions(-)
   
diff --git a/Makefile b/Makefile
index c593b51..fe4dd58 100644
--- a/Makefile
+++ b/Makefile
@@ -17,7 +17,7 @@ kdbus$(EXT)-y := \
   
 obj-m += kdbus$(EXT).o
   
-KERNELDIR?= /lib/modules/$(shell uname -r)/build
+KERNELDIR?= /lib/modules/$(shell uname -r)
 PWD  := $(shell pwd)
   
 all: module test
@@ -26,7 +26,7 @@ test::
  $(MAKE) -C test KBUILD_MODNAME=kdbus$(EXT)
   
 module:
- $(MAKE) -C $(KERNELDIR) M=$(PWD)
+ $(MAKE) -C $(KERNELDIR)/build M=$(PWD)
  
   Nope, you just broke the build on my machine when I wanted to build
   against the kernel source tree in /home/gregkh/linux/ that does not have
   a build/ subdirectory in it.
 
  Fair enough. I figured this would be the response I get [0].
 
  Um, that should be the workflow of _any_ kernel developer, we all build
  against local kernel directories, not one that is installed in /lib/.
  Some of us don't ever boot kernels installed in /lib :)
 
 
 Hej all,
 
 I have (locally) a patch that acomplishesh almost the same, but in a
 way that does not break gregs setup.
 
 What I did was:
  - Add a KERNELVER variable defaulting to $(shell uname -r)
  - replace all occurences of $(shell uname -r) with $(KERNELVER)
 
 maybe this is a viable solution?

I just wrote the same patch, except I used KVER instead of KERNELVER to
make it more visually distinct from KERNELDIR.

 It allows people to use the default paths (/lib/modules/*/build) but
 for a different kernel version, while still allowing users to set
 KERNELDIR to whatever they want.

There's an opportunity to do weird things if you were to set KERNELDIR
instead of KVER for the install target. I don't think this is anything
to worry about, though.

 I hope this is usefull

Works for me...

 
 
 Simon Peeters
 
 PS: if you want me to sent this as a decent patch, let me know and I
 do it once I am on my dev machine.
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Kai Krakow
Filipe Brandenburger filbran...@google.com schrieb:

 On Mon, Jun 16, 2014 at 6:13 PM, cwillu cwi...@cwillu.com wrote:
 For the case of sequential writes (via write or mmap), padding writes
 to page boundaries would help, if the wasted space isn't an issue.
 Another approach, again assuming all other writes are appends, would
 be to periodically (but frequently enough that the pages are still in
 cache) read a chunk of the file and write it back in-place, with or
 without an fsync. On the other hand, if you can afford to lose some
 logs on a crash, not fsyncing/msyncing after each write will also
 eliminate the fragmentation.
 
 I was wondering if something could be done in btrfs to improve
 performance under this workload... Something like a defrag on demand
 for a case where mostly appends are happening.
 
 When there are small appends with fsync/msync, they become new
 fragments (as expected), but once the writes go past a block boundary,
 btrfs could defragment the previous block in background, since it's
 not really expected to change again.
 
 That could potentially achieve performance close to chattr +C without
 the drawbacks of disabling copy-on-write.

I thought about something like that, too. I'm pretty sure it really doesn't 
matter if your 500G image file is split across 1 extents - as long as at 
least chunks of extents are kept together and rebuilt as one extent. That 
means, instead of letting autodefrag work on the whole file just let it 
operate on a chunk of it within some sane boundaries - maybe 8MB chunks, - 
of course without splitting existing extents if those already cross a chunk 
boundary. That way, it would still reduce head movements a lot while 
maintaining good performance during defragmentation. Your idea would be the 
missing companion to that (it is some sort of slow-growing-file-detection).

If I remember correctly, MacOSX implements a similar adaptic defragmentation 
strategy for its HFS+ filesystem, tho the underlying semantics are probably 
quite different. And it acts upon opening the file instead upon writing to 
the file, so it is probably limited to smallish files only (which I don't 
think makes so much sense on its own, for small files locality to 
semantically similar files is much more important, i.e. files needed during 
boot, files needed for starting a specific application).

If, after those strategies, it is still important to get your file's chunks 
cleanly aligned one after each other, one could still run a manual defrag 
which does the complete story.

BTW: Is it possible to physically relocate files in btrfs? I think that is 
more important than defragmentation. Is such a thing possible with the 
defrag IOCTL? My current understanding is that defragmentating a file just 
rewrites it somewhere random as a contiguous block - which is not always the 
best option because it can hurt boot performance a lot and thus reverses the 
effect of what is being tried to achieve. It feels a bit like playing 
lottery when I defragment my boot files only to learn that the boot process 
now is slower instead of faster. :-\

-- 
Replies to list only preferred.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Lennart Poettering
On Mon, 16.06.14 09:05, Josef Bacik (jba...@fb.com) wrote:

 So you are doing all the right things from what I can tell, I'm just
 a little confused about when you guys run fsync.  From what I can
 tell it's only when you open the journal file and when you switch it
 to offline.  I didn't look too much past this point so I don't
 know how often these things happen.  Are you taking an individual
 message, writing it, updating the head of the file and then
 fsync'ing?  Or are you getting a good bit of dirty log data and
 fsyncing occasionally?

The latter. Basically when opening a file for writing we mark it in the
header as online, then fsync() it. When we close a file we fsync() it,
then change the header to offline, ans sync() again. Also, 5min after
each write we will also put things to offline, until the next write,
when we will put things to online again. Finally, if something is logged
at priorities EMERG, ALERT or CRIT we will sync immediately (which
actually should never happen in real-life, unless something is really
broken -- a simple way to check if anything like this got written is
journalctl -p crit).

Also, we rotate and start a new file every now and then, when we hit a
size limit, but that is usually very seldom.

Putting this together: we should normally fsync() only very
infrequently. 

 What would cause btrfs problems is if you fallocate(), write a small
 chunk, fsync, write a small chunk again, fsync again etc.  Fallocate
 saves you the first write around, but if the next write is within
 the same block as the previous write we'll end up triggering cow and
 enter fragmented territory.  If this is what is what journald is
 doing then that would be good to know, if not I'd like to know what
 is happening since we shouldn't be fragmenting this badly.

Hmm, the only way I see that that would happen is if a lot of stuff is
logged at these super-high log levels mentioned above. But then again,
that never really should happen in real-life.

Could anyone who's expereiencing the slowdowns have a look on the
journalctl output menionted above? Do you have more than a few lines
printed like that?

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


[systemd-devel] [PATCH] ease installation on non-running kernels

2014-06-17 Thread Simon Peeters
---
 Makefile | 11 ++-
 1 file changed, 6 insertions(+), 5 deletions(-)

diff --git a/Makefile b/Makefile
index c593b51..178257b 100644
--- a/Makefile
+++ b/Makefile
@@ -17,7 +17,8 @@ kdbus$(EXT)-y := \
 
 obj-m += kdbus$(EXT).o
 
-KERNELDIR  ?= /lib/modules/$(shell uname -r)/build
+KERNELVER  ?= $(shell uname -r)
+KERNELDIR  ?= /lib/modules/$(KERNELVER)/build
 PWD:= $(shell pwd)
 
 all: module test
@@ -38,12 +39,12 @@ check:
test/test-kdbus
 
 install: module
-   mkdir -p /lib/modules/$(shell uname -r)/kernel/drivers/kdbus$(EXT)/
-   cp -f kdbus$(EXT).ko /lib/modules/$(shell uname 
-r)/kernel/drivers/kdbus$(EXT)/
-   depmod $(shell uname -r)
+   mkdir -p /lib/modules/$(KERNELVER)/kernel/drivers/kdbus$(EXT)/
+   cp -f kdbus$(EXT).ko 
/lib/modules/$(KERNELVER)/kernel/drivers/kdbus$(EXT)/
+   depmod $(KERNELVER)
 
 uninstall:
-   rm -f /lib/modules/$(shell uname -r)/kernel/drivers/kdbus/kdbus$(EXT).ko
+   rm -f /lib/modules/$(KERNELVER)/kernel/drivers/kdbus/kdbus$(EXT).ko
 
 coccicheck:
$(MAKE) -C $(KERNELDIR) M=$(PWD) coccicheck
-- 
2.0.0

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] [PATCH] build-sys: add missing backslash

2014-06-17 Thread Zbigniew Jędrzejewski-Szmek
On Tue, Jun 17, 2014 at 07:26:14PM +0200, Ronny Chevalier wrote:
 ---
  Makefile.am | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
 
 diff --git a/Makefile.am b/Makefile.am
 index 0213c38..e428141 100644
 --- a/Makefile.am
 +++ b/Makefile.am
 @@ -3537,7 +3537,7 @@ libsystemd_journal_internal_la_SOURCES = \
   src/journal/mmap-cache.h
  
  # using _CFLAGS = in the conditional below would suppress AM_CFLAGS
 -libsystemd_journal_internal_la_CFLAGS =
 +libsystemd_journal_internal_la_CFLAGS = \
   $(AM_CFLAGS)
Applied.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Kai Krakow
Lennart Poettering lenn...@poettering.net schrieb:

 [...] Finally, if something is logged
 at priorities EMERG, ALERT or CRIT we will sync immediately (which
 actually should never happen in real-life, unless something is really
 broken -- a simple way to check if anything like this got written is
 journalctl -p crit).
[...]
 Hmm, the only way I see that that would happen is if a lot of stuff is
 logged at these super-high log levels mentioned above. But then again,
 that never really should happen in real-life.

Unless you use those HP utilities:

# journalctl -p crit
-- Logs begin at Mo 2014-05-05 02:26:26 CEST, end at Di 2014-06-17 23:33:48 
CEST. --
Mai 20 10:17:21 jupiter hp-systray[1366]: hp-systray[1366]: error: option -s 
not recognized
-- Reboot --
Mai 22 22:37:15 jupiter hp-systray[1413]: hp-systray[1413]: error: option -s 
not recognized
-- Reboot --
Jun 03 08:49:17 jupiter hp-systray[1501]: hp-systray[1501]: error: option -s 
not recognized
-- Reboot --
Jun 09 13:45:31 jupiter hp-systray[1397]: hp-systray[1397]: error: option -s 
not recognized
-- Reboot --
Jun 11 00:21:16 jupiter hp-systray[1405]: hp-systray[1405]: error: option -s 
not recognized
-- Reboot --
Jun 14 00:30:49 jupiter hp-systray[1434]: hp-systray[1434]: error: option -s 
not recognized
-- Reboot --
Jun 14 22:54:43 jupiter hp-systray[1416]: hp-systray[1416]: error: option -s 
not recognized
-- Reboot --
Jun 14 23:11:11 jupiter hp-systray[1437]: hp-systray[1437]: error: option -s 
not recognized
-- Reboot --
Jun 15 14:53:44 jupiter hp-systray[1447]: hp-systray[1447]: error: option -s 
not recognized
-- Reboot --
Jun 15 16:53:39 jupiter hp-systray[1404]: hp-systray[1404]: error: option -s 
not recognized
-- Reboot --
Jun 15 17:38:40 jupiter hp-systray[1397]: hp-systray[1397]: error: option -s 
not recognized
-- Reboot --
Jun 15 18:34:37 jupiter hp-systray[1375]: hp-systray[1375]: error: option -s 
not recognized
-- Reboot --
Jun 17 20:38:32 jupiter hp-systray[1406]: hp-systray[1406]: error: option -s 
not recognized

 Could anyone who's expereiencing the slowdowns have a look on the
 journalctl output menionted above? Do you have more than a few lines
 printed like that?

Nasty message but no slowdowns. But it's printed just during desktop session 
start so it's no problem. Still, I wonder what this utility tries to do 
(seems to put itself into autostart with an -s switch but without 
supporting it, probably a packaging and/or KDE issue). At least HP thinks: 
Oh boy - that's really critical! Whoohoo. Why not let the system explode? 
:~)

-- 
Replies to list only preferred.

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Duncan
Kai Krakow posted on Tue, 17 Jun 2014 23:02:14 +0200 as excerpted:

  I'm pretty sure it really
 doesn't matter if your 500G image file is split across 1 extents -
 as long as at least chunks of extents are kept together and rebuilt as
 one extent.

It's worth noting that in btrfs terms, chunk has a specific meaning.  
Btrfs allocates chunks from free space, 1 GiB at a time for data 
chunks, 256 MiB at a time for metadata.[1]  And that btrfs-specific 
meaning does tend to distort your example case, somewhat, tho I expect 
most folks understand what you mean.  The problem is that we're running 
out of generic terms that mean chunk-like and are thus experiencing 
namespace collision!

Anyway, btrfs chunks, data chunks being of interest here, get allocated 
on demand.  The practical effect is thus that the maximum btrfs extent 
size is 1 GiB.  As such, the least-extents-possible ideal for that 500 
GiB image file would be 500 extents.

 That means, instead of letting autodefrag work on the whole
 file just let it operate on a chunk of it within some sane boundaries -
 maybe 8MB chunks, - of course without splitting existing extents if
 those already cross a chunk boundary.

As stated, chunk has a special meaning in btrfs.  Data chunks are 1 GiB 
in size and no extent can cross a btrfs chunk boundary.

 BTW: Is it possible to physically relocate files in btrfs?

Currently, ENOTIMPLEMENTED.  AFAIK it's possible and is on the list, but 
so are a number of other nice-to-haves, so it might be awhile.  
Actually just checked the wiki.  The closest specific feature point 
listed says hot-data-tracking and moving to faster devices, noting that 
there's actually some current work on the generic (not-btrfs-specific) 
feature, to be made available via VFS.  Your specific use-case will 
probably be part of that general implementation.

---
[1] Btrfs chunk allocation: 1 GiB data chunks size, 256 MiB metadata 
chunk size by default, but there are various exceptions.  The first of 
course is when space gets tight.  The last allocation will be whatever is 
left.  Second, there's mixed-data/metadata mode, the default when the 
filesystem is = 1 GiB.  Mixed chunks are (like metadata) 256 MiB but 
contain data and metadata mixed, sacrificing speed efficiency for more 
efficient space management.  Third, it's worth noting that the single-
device-btrfs default is dup-mode-metadata (with the same default for 
mixed-mode), so while they're 256 MiB each, two get allocated at once, 
thus allocating metadata half a GiB at a time.  Multi-device-btrfs 
metadata defaults to raid1-mode, still allocated in chunk pairs but one 
on each of two separate devices.  Data mode always defaults to single, 
tho on multi-device-btrfs both data and metadata can be (individually) 
set to various raid modes, each with its own specific allocation pattern.

-- 
Duncan - List replies preferred.   No HTML msgs.
Every nonfree program has a lord, a master --
and if you use the program, he is your master.  Richard Stallman

___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel


Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-17 Thread Tollef Fog Heen
]] Lennart Poettering 

(trimming Ccs)

 I am invoking fallocate() in advance, because we write those files with
 mmap() and that of course would normally triggered SIGBUS already on the
 most boring of reasons, such as disk full/quota full or so. Hence,
 before we do anything like that, we invoke fallocate() to ensure that
 the space is actually available... As far as I can see, that pretty much
 in line with what fallocate() is supposed to be useful for, the man page
 says this explicitly:

OOI, why not use write + read-only mmap for reading?  In testing
performance for another project, that's significantly faster than rw
mmap.

-- 
Tollef Fog Heen
UNIX is user friendly, it's just picky about who its friends are
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
http://lists.freedesktop.org/mailman/listinfo/systemd-devel