Re: [systemd-devel] libsystemd/sd_bus: trouble understanding how to parse complex responses
Hi Lennart, sorry for the late reaction, thanks to your help (especially the busctl monitor hint) I was able to figure out what was going on. On Wed, Sep 09, 2020 at 06:31:26PM +0200, Lennart Poettering wrote: [...] > > At this point I am not sure if sd_bus actually behaves correctly and > > GetManagedObjects() returns only one path or if I messed up the parsing of > > the reply somehow? And I am not sure how to check that... > > > > I would like to avoid parsing XML if I can, but of course that option is > > still there; nevertheless, I would first like to understand what is going on > > with the GetManagedObjects() call and if the problem is really there or > > somewhere else. > > Consider using "busctl monitor" to see what the message your client > receives actually contains. The issue was that "busctl tree" returned a list of paths which I assumed would be also present in the response of GetManagedObjects(), however - this was not the case. Instead of being an object path on the same level as shown in busctl tree, the SIM path which I saw in the GetManagedObjects() reply was returned in a property. I simply missed that when looking at the busctl response in the console. This explains why I was not getting the second path in the array when parsing with sd_bus - it was really not there, I just thought it would be, because of what I have seen in the busctl tree output. So once again, thank you for the hints, it's solved now. Kind regars, Jin ___ systemd-devel mailing list systemd-devel@lists.freedesktop.org https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libsystemd/sd_bus: trouble understanding how to parse complex responses
On Di, 08.09.20 20:51, Sergey 'Jin' Bostandzhyan (j...@mediatomb.cc) wrote:
> >
> > https://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format
>
> Oh, so you mean that when I use "busctl call" it would detect and fake a
> GetManagedObjects() method invocation and internally use XML
> introspection?
No. I meant that "busctl introspect" and "busctl tree" use the XML
data, and not GetManagedObjects(). If you invoke "busctl call" then
you get what you ask for, we do no magic redirect.
> Is it somehow possible to figure this out via command line tools if the
> problem is in the missing GetManagedObjects() implementation or not?
Well, the introspection data should advertise GetManagedObjects()
method if it's supported.
>
> > > while (sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY,
> > > "{oa{sa{sv}}}") > 0)
> > > {
> >
> > Hmm, this is what you'd write when you have an array of arrays...
>
> Uhm, how do I correctly "enter" a container like this one:
> "a{oa{sa{sv}}}" ?
Using sd_bus_message_enter_container() as you do, but you#d not call
it in a loop like you do, as that only makes sense if the actual type
string is "aa{oa{sa{sv}}}", i.e. starts with an array of arrays.
> I guess this is correct, what GetManagedObjects() returns is an array of
> dictionaries "a{oa{sa{sv}}}" or am I misinterpreting it?
Well, the way dbus encodes dictionaries is as "array of dictionary
entries. That's what the "a{…}" means.
> sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY, "oa{sa{sv}}")
> returns 1 on the first call, thats where I get the first path that
> equals to "/org/freedesktop/ModemManager1/Modem/0" and it returns 0 on the
> next invocation.
It returns 0 if you reached the end of the surrounding container, i.e. the
end of the structure or array.
> > Normally, you#d enter the array once and then iterate through the
> > dict entries contained therein. i.e. unless there's an "aa" object
> > somewhere (i.e. "array of array") you'd do the while loop around the
> > dict entries, not the array object.
>
> Could you please clarify how iteration works in sd_bus sense? Is my
> understanding correct that I need to use sd_bus_message_skip() before
> exiting the container in order to iterate to the next item, if I am not
> interested in parsing all of the nested items? Or can I just "exit" at any
> time and expect the next "enter" invocation to fast forward to the
> next entry?
You are supposed to be at the end of the structure/array when calling
sd_bus_message_exit_container(). Otherwise you'll get EBUSY. We should
probably document that in the man page.
> Based on what you are saying the inner while loop that I had right after
> entering the array should have returned all items. So I start by entering
> the array:
>
> // starting with a{oa{sa{sv}}}
> sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY, "{oa{sa{sv}}}")
>
> And loop over dictionary key-value pairs:
>
> // brings me down into oa{sa{sv}}}, I am only interested in the object paths:
> while (sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY,
> "oa{sa{sv}}") > 0) {
> sd_bus_message_read_basic(m, SD_BUS_TYPE_OBJECT_PATH, );
> sd_bus_message_skip(m, "a{sa{sv}}");
> sd_bus_message_exit_container(m);
> }
>
> Does the above look correct? As mentioned, I am not getting errors, return
> codes are fine, so it all looks normal, but I am only reading out
> the first path, on the second iteration
> sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY,
> "oa{sa{sv}}") returns 0 which aborts the loop.
This does look correct to me.
> At this point I am not sure if sd_bus actually behaves correctly and
> GetManagedObjects() returns only one path or if I messed up the parsing of
> the reply somehow? And I am not sure how to check that...
>
> I would like to avoid parsing XML if I can, but of course that option is
> still there; nevertheless, I would first like to understand what is going on
> with the GetManagedObjects() call and if the problem is really there or
> somewhere else.
Consider using "busctl monitor" to see what the message your client
receives actually contains.
Lennart
--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libsystemd/sd_bus: trouble understanding how to parse complex responses
Hi Lennart,
On Tue, Sep 08, 2020 at 03:58:34PM +0200, Lennart Poettering wrote:
> > Service /org/freedesktop/ModemManager1:
> > Failed to introspect object / of service /org/freedesktop/ModemManager1:
> > Invali argument
> > No objects discovered.
> >
> > The tree looks fine, the error message is a bit weird?
>
> Hmm, consider running this with the env var SYSTEMD_LOG_LEVEL=debug
> set, maybe that explains where the error is generated.
interestingly enough the same error does not happen today, it just returns
the tree, thanks for the log level hint though - that is indeed
very helpful!
> > The only things I am interested in are the paths:
> > /org/freedesktop/ModemManager1/Modem/0
> > /org/freedesktop/ModemManager1/SIM/0
> >
> > The above paths may change when the modem is replugged, so I'd like to query
> > them dynamically.
> >
> > I learned that the call to list what I need is "GetManagedObjects":
> > busctl call org.freedesktop.ModemManager1
> > /org/freedesktop/ModemManager1 org.freedesktop.DBus.ObjectManager
> > GetManagedObjects
>
> GetManagedObjects() is only available in some services, it's an
> optional interface. "busctl" uses the XML introspection logic to
> enumerate objects, i.e. this:
>
> https://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format
Oh, so you mean that when I use "busctl call" it would detect and fake a
GetManagedObjects() method invocation and internally use XML introspection?
That's good to know since I was using busctl to debug/test and I was assuming
that it does the calls as instructed.
> Unlike GetManagedObjects() the XML introspection stuff is implemented
> by most services.
Understood.
Is it somehow possible to figure this out via command line tools if the
problem is in the missing GetManagedObjects() implementation or not?
> > while (sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY, "{oa{sa{sv}}}")
> > > 0)
> > {
>
> Hmm, this is what you'd write when you have an array of arrays...
Uhm, how do I correctly "enter" a container like this one: "a{oa{sa{sv}}}" ?
I guess this is correct, what GetManagedObjects() returns is an array of
dictionaries "a{oa{sa{sv}}}" or am I misinterpreting it?
> sd_bus_message_enter_container() returns < 0 on error, == 0 if we
> reached the end of the surrounding container, and > 0 if all was
> good. I'd recommend checking for error cases which you ignore
> here. The errors tell you where things go wrong.
I already had error checks everywhere except loops, so I added that now.
sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY, "oa{sa{sv}}")
returns 1 on the first call, thats where I get the first path that
equals to "/org/freedesktop/ModemManager1/Modem/0" and it returns 0 on the
next invocation.
This leads me to the next question below...
> Normally, you#d enter the array once and then iterate through the
> dict entries contained therein. i.e. unless there's an "aa" object
> somewhere (i.e. "array of array") you'd do the while loop around the
> dict entries, not the array object.
Could you please clarify how iteration works in sd_bus sense? Is my
understanding correct that I need to use sd_bus_message_skip() before
exiting the container in order to iterate to the next item, if I am not
interested in parsing all of the nested items? Or can I just "exit" at any
time and expect the next "enter" invocation to fast forward to the
next entry?
Based on what you are saying the inner while loop that I had right after
entering the array should have returned all items. So I start by entering
the array:
// starting with a{oa{sa{sv}}}
sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY, "{oa{sa{sv}}}")
And loop over dictionary key-value pairs:
// brings me down into oa{sa{sv}}}, I am only interested in the object paths:
while (sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY, "oa{sa{sv}}")
> 0) {
sd_bus_message_read_basic(m, SD_BUS_TYPE_OBJECT_PATH, );
sd_bus_message_skip(m, "a{sa{sv}}");
sd_bus_message_exit_container(m);
}
Does the above look correct? As mentioned, I am not getting errors, return
codes are fine, so it all looks normal, but I am only reading out
the first path, on the second iteration
sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY, "oa{sa{sv}}") returns
0 which aborts the loop.
At this point I am not sure if sd_bus actually behaves correctly and
GetManagedObjects() returns only one path or if I messed up the parsing of
the reply somehow? And I am not sure how to check that...
I would like to avoid parsing XML if I can, but of course that option is
still there; nevertheless, I would first like to understand what is going on
with the GetManagedObjects() call and if the problem is really there or
somewhere else.
Kind regars,
Jin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
Re: [systemd-devel] libsystemd/sd_bus: trouble understanding how to parse complex responses
On Mo, 07.09.20 19:40, Sergey 'Jin' Bostandzhyan (j...@mediatomb.cc) wrote:
> Hi,
>
> I have spent quite some time on this, but I don't seem to be getting
> anywhere, so I hope someone could point me in the right direction.
>
> I am learning sd_bus, my goal is to get a list of available modem services
> from ModemManager. To make sure that what I am looking for is actually there,
> I first checked with busctl:
>
> Service org.freedesktop.ModemManager1:
> └─/org
> └─/org/freedesktop
> └─/org/freedesktop/ModemManager1
> ├─/org/freedesktop/ModemManager1/Modem
> │ └─/org/freedesktop/ModemManager1/Modem/0
> └─/org/freedesktop/ModemManager1/SIM
> └─/org/freedesktop/ModemManager1/SIM/0
>
> Service /org/freedesktop/ModemManager1:
> Failed to introspect object / of service /org/freedesktop/ModemManager1:
> Invali argument
> No objects discovered.
>
> The tree looks fine, the error message is a bit weird?
Hmm, consider running this with the env var SYSTEMD_LOG_LEVEL=debug
set, maybe that explains where the error is generated.
> The only things I am interested in are the paths:
> /org/freedesktop/ModemManager1/Modem/0
> /org/freedesktop/ModemManager1/SIM/0
>
> The above paths may change when the modem is replugged, so I'd like to query
> them dynamically.
>
> I learned that the call to list what I need is "GetManagedObjects":
> busctl call org.freedesktop.ModemManager1
> /org/freedesktop/ModemManager1 org.freedesktop.DBus.ObjectManager
> GetManagedObjects
GetManagedObjects() is only available in some services, it's an
optional interface. "busctl" uses the XML introspection logic to
enumerate objects, i.e. this:
https://dbus.freedesktop.org/doc/dbus-specification.html#introspection-format
Unlike GetManagedObjects() the XML introspection stuff is implemented
by most services.
> while (sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY, "{oa{sa{sv}}}") >
> 0)
> {
Hmm, this is what you'd write when you have an array of arrays...
sd_bus_message_enter_container() returns < 0 on error, == 0 if we
reached the end of the surrounding container, and > 0 if all was
good. I'd recommend checking for error cases which you ignore
here. The errors tell you where things go wrong.
Normally, you#d enter the array once and then iterate through the
dict entries contained therein. i.e. unless there's an "aa" object
somewhere (i.e. "array of array") you'd do the while loop around the
dict entries, not the array object.
Lennart
--
Lennart Poettering, Berlin
___
systemd-devel mailing list
systemd-devel@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/systemd-devel
[systemd-devel] libsystemd/sd_bus: trouble understanding how to parse complex responses
Hi,
I have spent quite some time on this, but I don't seem to be getting
anywhere, so I hope someone could point me in the right direction.
I am learning sd_bus, my goal is to get a list of available modem services
from ModemManager. To make sure that what I am looking for is actually there,
I first checked with busctl:
Service org.freedesktop.ModemManager1:
└─/org
└─/org/freedesktop
└─/org/freedesktop/ModemManager1
├─/org/freedesktop/ModemManager1/Modem
│ └─/org/freedesktop/ModemManager1/Modem/0
└─/org/freedesktop/ModemManager1/SIM
└─/org/freedesktop/ModemManager1/SIM/0
Service /org/freedesktop/ModemManager1:
Failed to introspect object / of service /org/freedesktop/ModemManager1: Invali
argument
No objects discovered.
The tree looks fine, the error message is a bit weird?
The only things I am interested in are the paths:
/org/freedesktop/ModemManager1/Modem/0
/org/freedesktop/ModemManager1/SIM/0
The above paths may change when the modem is replugged, so I'd like to query
them dynamically.
I learned that the call to list what I need is "GetManagedObjects":
busctl call org.freedesktop.ModemManager1 /org/freedesktop/ModemManager1
org.freedesktop.DBus.ObjectManager GetManagedObjects
It returns a data structure of the type a{oa{sa{sv}}} and the
object paths "/org/freedesktop/ModemManager1/Modem/0" and
"/org/freedesktop/ModemManager1/SIM/0" are indeed present in its
output.
However, when trying to parse the response in the code I only get as far as
reading the first /org/freedesktop/ModemManager1/Modem/0 but I never get
to the second /org/freedesktop/ModemManager1/SIM/0 entry.
What I am doing is (full and compilable source is attached, here I am leaving
out the error checks etc):
while (sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY, "{oa{sa{sv}}}") > 0)
{
while (sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY,
"oa{sa{sv}}") > 0)
{
const char *path;
sd_bus_message_read_basic(m, SD_BUS_TYPE_OBJECT_PATH, );
fprintf(stderr, "Object path: %s\n", path);
sd_bus_message_skip(m, "a{sa{sv}}");
/* exit dictionary entry */
sd_bus_message_exit_container(m);
} // while SD_BUS_TYPE_DICT_ENTRY
/* exit array */
sd_bus_message_exit_container(m);
} // while SD_BUS_TYPE_ARRAY
I do not get any errors in the "real" version (attached), but it never
gets further to the SIM entry. What am I missing?
Kind regards,
Jin
/* compile: gcc `pkg-config --cflags --libs libsystemd` listpaths.c -o listpaths
Looking for:
/org/freedesktop/ModemManager1/Modem/0
/org/freedesktop/ModemManager1/SIM/0
$ busctl tree org.freedesktop.ModemManager1 /org/freedesktop/ModemManager1
Service org.freedesktop.ModemManager1:
└─/org
└─/org/freedesktop
└─/org/freedesktop/ModemManager1
├─/org/freedesktop/ModemManager1/Modem
│ └─/org/freedesktop/ModemManager1/Modem/0
└─/org/freedesktop/ModemManager1/SIM
└─/org/freedesktop/ModemManager1/SIM/0
*/
#include
#include
#include
#include
#include
int main()
{
sd_bus *bus = NULL;
if (sd_bus_open_system() < 0)
{
return 1;
}
while (sd_bus_is_ready(bus) <= 0)
{
sd_bus_wait(bus, 1000);
sd_bus_process(bus, NULL);
}
printf("Bus ready.\n");
sd_bus_message *m = NULL;
sd_bus_error error = SD_BUS_ERROR_NULL;
int ret = sd_bus_call_method(bus, "org.freedesktop.ModemManager1",
"/org/freedesktop/ModemManager1",
"org.freedesktop.DBus.ObjectManager",
"GetManagedObjects", , , NULL);
if (ret < 0)
{
printf("Could not get object list\n");
return ret;
}
while (sd_bus_message_enter_container(m, SD_BUS_TYPE_ARRAY,
"{oa{sa{sv}}}") > 0)
{
while (sd_bus_message_enter_container(m, SD_BUS_TYPE_DICT_ENTRY,
"oa{sa{sv}}") > 0)
{
const char *path;
if (sd_bus_message_read_basic(m, SD_BUS_TYPE_OBJECT_PATH, ) <
0)
{
printf("Could not read object path\n");
goto finish;
}
fprintf(stderr, "Object path: %s\n", path);
if (sd_bus_message_skip(m, "a{sa{sv}}") < 0)
{
printf("Could not skip message\n");
goto finish;
}
/* exit dictionary entry */
if (sd_bus_message_exit_container(m) < 0)
{
printf("Could not exit container (inner loop)\n");
goto finish;
}
} // while SD_BUS_TYPE_DICT_ENTRY
/* exit array */
if (sd_bus_message_exit_container(m) < 0)
{
printf("Could not exit container (outer loop)\n");
goto finish;
}
