Re: [systemd-devel] Hashmap iteration is too costly

[vcaputo]

On Mon, Jul 17, 2017 at 10:12:12AM +0200, Lennart Poettering wrote:
> On Fri, 14.07.17 03:18, vcap...@pengaru.com (vcap...@pengaru.com) wrote:
> 
> > The current hashmap iteration incurs at least one function call per
> > iteration, and in my observations using gcc 6.3 w/-g -O2, it's two:
> > 
> >  internal_hashmap_iterate()
> >  hashmap_iterate_entry()
> > 
> > for every element in the hashmap.
> > 
> > In profiles of `journalctl -b --no-pager` having multiple (50) log files,
> > the hashmap iteration shows up as ~10% of the CPU in callgrind
> > output.
> 
> I wonder if it wouldn't be easiest to simply bypass hashmap native
> iteration for this case. i.e. add a linked list for the open journal
> files which we can easily traverse in O(1) time?
> 

What part of hashmap_iterate_in_insertion_order() grows in complexity with
increasing N?

> The hashmap implementation has been optimized to require very little
> memory, because we have so many hashmap/sets around in PID 1. It's not
> optimized to make iteration fast though.
> 

I don't believe these are mutually exclusive.

Regards,
Vito Caputo
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hashmap iteration is too costly

[Zbigniew Jędrzejewski-Szmek]

On Mon, Jul 17, 2017 at 10:12:12AM +0200, Lennart Poettering wrote:
> On Fri, 14.07.17 03:18, vcap...@pengaru.com (vcap...@pengaru.com) wrote:
> 
> > The current hashmap iteration incurs at least one function call per
> > iteration, and in my observations using gcc 6.3 w/-g -O2, it's two:
> > 
> >  internal_hashmap_iterate()
> >  hashmap_iterate_entry()
> > 
> > for every element in the hashmap.
> > 
> > In profiles of `journalctl -b --no-pager` having multiple (50) log files,
> > the hashmap iteration shows up as ~10% of the CPU in callgrind
> > output.
> 
> I wonder if it wouldn't be easiest to simply bypass hashmap native
> iteration for this case. i.e. add a linked list for the open journal
> files which we can easily traverse in O(1) time?
> 
> The hashmap implementation has been optimized to require very little
> memory, because we have so many hashmap/sets around in PID 1. It's not
> optimized to make iteration fast though.

See also https://github.com/systemd/systemd/pull/6365:
test-hashmap took more than two minutes on rpi3.

Zbyszek
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

[systemd-devel] systemd-nspawn network-interface doesn't return interface on container reboot

[Dmitry Kulida]

Hi Everyone.

I have below trouble.

I start my container with --network-interface option as below:
ExecStart=/usr/bin/systemd-nspawn -M %i.%H --quiet --keep-unit --boot
--link-journal=auto --network-veth *--network-interface=dummy6*
--capability=CAP_NET_RAW --directory=/var/lib/container/%i

Everything works fine but when I try to reboot my container the dummy6 is
not returned back to host machine's namespace and my container fails to
start next time. Only host machine reboot helps.

Do we have some way to return the interface back to host's machine
namespace even maybe manually.

Thanks for help in advance!

--
Dmitry
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Hashmap iteration is too costly

[Lennart Poettering]

On Fri, 14.07.17 03:18, vcap...@pengaru.com (vcap...@pengaru.com) wrote:

> The current hashmap iteration incurs at least one function call per
> iteration, and in my observations using gcc 6.3 w/-g -O2, it's two:
> 
>  internal_hashmap_iterate()
>  hashmap_iterate_entry()
> 
> for every element in the hashmap.
> 
> In profiles of `journalctl -b --no-pager` having multiple (50) log files,
> the hashmap iteration shows up as ~10% of the CPU in callgrind
> output.

I wonder if it wouldn't be easiest to simply bypass hashmap native
iteration for this case. i.e. add a linked list for the open journal
files which we can easily traverse in O(1) time?

The hashmap implementation has been optimized to require very little
memory, because we have so many hashmap/sets around in PID 1. It's not
optimized to make iteration fast though.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel

Re: [systemd-devel] Realtime scheduling with CONFIG_RT_GROUP_SCHED=y

[Lennart Poettering]

On Wed, 12.07.17 11:41, Lars Kellogg-Stedman (l...@redhat.com) wrote:

> On Thu, Jul 6, 2017 at 10:56 AM, Lars Kellogg-Stedman 
> wrote:
> 
> > In older versions of systemd one could handle this using the directives
> > described in https://www.freedesktop.org/wiki/Software/systemd/
> > MyServiceCantGetRealtime/, but unfortunately that document, despite being
> > the number 1 search result for pretty much anything involving "systemd" and
> > "realtime", is obsolete and those directives no longer exist.
> >
> > Is there a way to make this work correctly with modern versions of
> > systemd?  I've hacked around it for now by creating
> > /etc/systemd/system/myservice.service.d/realtime.conf that moves the
> > service back to the root cgroup and then uses chrt to set the scheduling
> > policy:
> >
> 
> It looks like systemd sets up cgroups before calling ExecStartPre, which
> means I can emulate the behavior of those obsolete directives by running:
> 
> ExecStartPre=/bin/sh -c 'echo 55 >
> /sys/fs/cgroup/cpu,cpuacct/cpu.rt_runtime_us'
> ExecStartPre=/bin/sh -c 'echo 20 >
> /sys/fs/cgroup/cpu,cpuacct/system.slice/cpu.rt_runtime_us'
> ExecStartPre=/bin/sh -c 'echo 20 >
> /sys/fs/cgroup/cpu,cpuacct/system.slice/myservice.service/cpu.rt_runtime_us'
> 
> In an environment where CONFIG_RT_GROUP_SCHED is set, is this the best way
> of addressing the problem?

Yeah, this would probably work, but you should really set
CPUAccounting=1 too, as an indirect way to ensure your unit appears in
the "cpu"/"cpuacct" cgroup hierarchy in the first place.

And I'd probably turn this into a proper shell script, that
dynamically reads the path from /proc/self/cgroup and then propagates
things up properly.

Lennart

-- 
Lennart Poettering, Red Hat
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel