https://bugs.kde.org/show_bug.cgi?id=524947
Bug ID: 524947
Summary: Akonadi's agent registration path makes blocking D-Bus
calls on login
Classification: Frameworks and Libraries
Product: Akonadi
Version First unspecified
Reported In:
Platform: Arch Linux
OS: Linux
Status: REPORTED
Severity: major
Priority: NOR
Component: server
Assignee: [email protected]
Reporter: [email protected]
CC: [email protected]
Target Milestone: ---
Created attachment 195609
--> https://bugs.kde.org/attachment.cgi?id=195609&action=edit
Extra details (text files)
DESCRIPTION
Akonadi's agent registration path makes blocking D-Bus calls in both directions
at the same time, on both processes' main threads. akonadi_control blocks
waiting on an agent, while every `Akonadi::AgentManager` client (including the
agents themselves, and plasmashell via the `pimevents` calendar plugin) blocks
waiting on akonadi_control. Neither side can service the other's call because
neither is running its event loop.
Nothing breaks the deadlock except the 25 second default D-Bus reply timeout.
Startup then proceeds by one agent, and deadlocks again. With five Google
resources configured, a cold Akonadi start takes just over two minutes, and
plasmashell is completely frozen for that entire time on every login.
Every measured delay is an exact multiple of 25000 ms. No actual work is
happening during any of it.
STEPS TO REPRODUCE
1. Configure five `akonadi_google_resource` instances (five Google accounts),
plus a few local resources. Nine agent instances total in this case.
2. Enable the `pimevents` plugin in the Plasma digital clock applet, which is
what makes plasmashell an `Akonadi::AgentManager` client.
3. Log in to a Plasma Wayland session, or equivalently run `akonadictl restart`
in a running session.
OBSERVED RESULT
The desktop appears and then freezes solidly. Panels do not respond, the task
manager is stuck on the first frame of its layout animation, no hover events
land. kwin and krunner are separate processes and stay usable, so the session
is not fully dead, but plasmashell is unusable for 124 to 150 seconds depending
on agent count.
akonadi_control logs a run of these, spaced 25 or 50 seconds apart:
akonadi_control[1813498]: "D-Bus communication error
'org.freedesktop.DBus.Error.NoReply':
'Did not receive a reply. Possible causes include: the remote application did
not send a
reply, the message bus security policy blocked the reply, the reply timeout
expired, or
the network connection was broken.'"
Login boot timeline, thirteen agent instances:
21:42:09.688 Akonadi server is now operational
21:42:09.765 3 google resources begin startup
21:42:59.859 +50s 2x NoReply -> 4th google resource begins
21:43:49.857 +50s 1x NoReply -> 5th google resource begins
21:44:39.858 +50s 1x NoReply -> settled
150 seconds total, four timeout rounds. Trimming to nine agent instances gave
124 seconds and still four rounds, so the count of configured agents is not the
main driver.
EXPECTED RESULT
Agent registration completes without blocking either process' main thread.
plasmashell, and any other `Akonadi::AgentManager` client, stays responsive
while Akonadi starts.
SOFTWARE/OS VERSIONS
Operating System: Arch Linux
KDE Plasma Version: 6.7.4
KDE Frameworks Version: 6.29.0
Qt Version: 6.11.2
Kernel Version: 7.1.9-zen1-2-zen (64-bit)
Graphics Platform: Wayland
akonadi: 26.08.0
kdepim-runtime: 26.08.0
libkgapi: 26.08.0
ADDITIONAL INFORMATION
Both halves of the deadlock
Captured with `eu-stack` against a controlled `akonadictl restart`, with
plasmashell stopped beforehand so it was not a participant. debuginfod resolves
the symbols fine.
akonadi_control, main thread, identical at t+12s and t+40s:
#3 QWaitCondition::wait(QMutex*, QDeadlineTimer)
#4 QDBusPendingCallPrivate::waitForFinished()
#5 AgentInstance::obtainAgentInterface()
#6 AgentManager::serviceOwnerChanged(QString const&, QString const&, QString
const&)
#7 QObject::event(QEvent*)
#16 QCoreApplication::exec()
#17 main
akonadi_google_resource, main thread, at the same instant. This stack is
identical in every agent, and identical in plasmashell:
#3 QWaitCondition::wait(QMutex*, QDeadlineTimer)
#4 QDBusPendingCallPrivate::waitForFinished()
#5 QDBusPendingReplyBase::argumentAt(int) const
#6 QDBusPendingReply<QString>::argumentAt<0>() const
#7 Akonadi::AgentManagerPrivate::fillAgentInstance(QString const&) const
#8 Akonadi::AgentManagerPrivate::agentInstanceAdded(QString const&)
#9 doActivate<false>(QObject*, int, void**)
#10 OrgFreedesktopAkonadiAgentManagerInterface::qt_static_metacall(...)
#12 QDBusConnectionPrivate::deliverCall(QObject*, QDBusMessage const&, ...)
#23 QCoreApplication::exec()
#24 int Akonadi::ResourceWidgetBase::init<GoogleResource>(int, char**)
So `AgentManager::serviceOwnerChanged` calls into the agent and blocks, while
the agent is inside its own `agentInstanceAdded` handler calling back into
akonadi_control and blocking. Both are on the main thread, inside event
delivery, so neither can answer.
Which calls, and how long they took
>From `dbus-monitor --session --profile` over the same restart, reply latency
matched by serial:
74.5s -> org.freedesktop.Akonadi.Control AgentManager.agentInstanceType
50.0s -> org.freedesktop.Akonadi.Control AgentManager.agentInstanceOnline
50.0s -> org.freedesktop.Akonadi.Control
AgentManager.agentInstanceAccountId
50.0s -> akonadi_google_resource_2 Agent.Account.accountId
50.0s -> akonadi_google_resource_2 Resource.name
25.0s -> akonadi_google_resource_0 Resource.name
Client to control: `agentInstanceType`, `agentInstanceOnline`,
`agentInstanceAccountId`, all from `fillAgentInstance`. Control to agent:
`Resource.name` and `Agent.Account.accountId`, both from
`obtainAgentInterface`.
plasmashell is a bystander
plasmashell has no Akonadi-specific code in its stack beyond the client
library. It blocks purely because `pimevents` makes it an
`Akonadi::AgentManager` client, and every client blocks in `agentInstanceAdded`
for as long as akonadi_control is stuck. Any process using AgentManager will do
the same. kalendarac does too.
An Akonadi that is already warm never triggers this: restarting plasmashell
against a settled Akonadi produces no block at all, because no
`agentInstanceAdded` signals fire. Likewise, restarting a single agent on its
own is instant and clean. It takes a mass registration to form the cycle.
Side effect: the initial Google sync fails every time
Because the resources' event loops are frozen through the 25 second windows,
the HTTP/2 connections to Google time out and are closed server side. Every
cold start logs:
akonadi_google_resource[1813852]: stream 1 error: "Connection closed"
akonadi_google_resource[1813852]: stream 1 finished with error: "HTTP/2
protocol error"
akonadi_google_resource[1813852]: Got error: KGAPI2::CalendarFetchJob(0x...)
"Invalid response content type"
akonadi_google_resource[1813852]: Got error: KGAPI2::TaskListFetchJob(0x...)
"Invalid response content type"
These land exactly on the timeout boundaries. Calendars only populate on a
later retry. Fixing the deadlock should fix this too.
Ruled out
- Not KWallet. All five OAuth tokens are present in the `Akonadi Google` folder
and read back in 3 ms.
- Not the network. No TCP connections are open at all while the agents are
stalled.
- Not the database. MariaDB check and schema init complete in 0.4 s.
- Not agent count. Thirteen agents took 150 s, nine took 124 s, both four
rounds.
Suggested direction
`AgentManagerPrivate::fillAgentInstance` fetches type, online state and account
id with three separate blocking replies, from inside a signal handler. Making
those async, or having control push the instance data along with the
`agentInstanceAdded` signal instead of making clients ask for it, would remove
the client half of the cycle. The control half,
`AgentInstance::obtainAgentInterface`, would want the same treatment.
--
You are receiving this mail because:
You are watching all bug changes.
