Re: [MeeGo-dev] Trying to sync Buteo with Google
Hi, -Original Message- From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On Behalf Of ext Patrick Ohly Sent: Wednesday, October 13, 2010 2:59 PM To: Zhu, Yongsheng Cc: Kavuri Sateesh (Nokia-MS/Bangalore); meego-dev@meego.com Subject: Re: [MeeGo-dev] Trying to sync Buteo with Google On Wed, 2010-10-13 at 09:42 +0100, Zhu, Yongsheng wrote: Hi, Sateesh Patrick I'm working on trying to sync Buteo with Google service via SyncML protocol. Currently it's blocked because Google service sends 'unauthorized' error. See the attached syncML messages buteo-msgs.zip. Since SyncEvolution can sync with Google correctly, as a reference, I also dump its SyncML messages. I'm so curious about this since two messages from Buteo and SyncEvolution are almost same. Indeed, no apparent relevant difference stands out. Perhaps there is some difference outside of the message content itself? Do both programs use the same message encoding (should be WBXML for SyncEvolution)? Is the HTTP POST from Buteo going to the redirected URL? The HTTP POST doesn't go the redirected URL AFAIK. It goes to the URL that's mentioned in the profile. If that's all the same, then I only see one possible approach: minimize the differences until it succeeds. Sateesh, you said that the Google config hasn't been included yet because not enough testing was done. Has any testing been done? Did you get past the authentication problem? -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Trying to sync Buteo with Google
Hi Patrick and Yongsheng, I found the root cause here. So the post was going to the redirected URI, that wasn't the issue. The respURI sent to us by Google was percent encoded and there was a bug in the stack in creation of QUrls from percent encoded URLs. I fixed that and it works fine. Will be pushing the changes soon to buteo-syncml. Regards, Deepak -Original Message- From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On Behalf Of ext Patrick Ohly Sent: Thursday, October 14, 2010 2:15 PM To: Zhu, Yongsheng Cc: Kavuri Sateesh (Nokia-MS/Bangalore); meego-dev@meego.com Subject: Re: [MeeGo-dev] Trying to sync Buteo with Google On Thu, 2010-10-14 at 09:28 +0100, Zhu, Yongsheng wrote: Deepak said that Buteo does not follow the redirect that it gets in the first response from the SyncML server. This redirect is part of the SyncML protocol; if Deepak is right, then this is the root cause of the problem and needs to be fixed. You mean respURI in the 'response' syncml message? I found it has been used to send next message. Yes. So it already behaves as it should. Good. Sending credentials each time is merely a workaround and not one that we can rely on (Google might change their behavior for handling ensuing messages at the starting URL). Yes, it should be a workaround to help us overcome this one. Fair enough. Note that there have been some authentication related changes recently in the upstream buteo-syncml repo. The commit messages don't say which server provided the motivation for these changes; perhaps they also help with Google. In the meantime, let's use your patch. -- Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: ext Patrick Ohly [mailto:patrick.o...@intel.com] Sent: Friday, January 14, 2011 9:22 PM To: Kavuri Sateesh (Nokia-MS/Bangalore); amitc...@gmail.com; Kodihalli Deepak (Nokia-MS/Bangalore); Puranik Santosh (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; tuomas.kat...@ixonos.com; mhuna...@gmail.com; juha.paalija...@ixonos.com; Duggirala Karthik.2 (Nokia-MS/Helsinki) Cc: MeeGo-dev; Drews, Paul Subject: Buteo MTP + gadgetfs Permissions === Currently the code runs as part of mtp_test when started as root. That's necessary because /dev/gadget must be read/writable. But eventually the code needs to run as plugin as normal user inside msycnd. Initially the process accesses /dev/gadget/driver to configure USB, then some new entries (like /dev/gadget/ep-a) which are created by gadgetfs based on that configuration. We could do a chmod as root to grant the msyncd process read/write permission for /dev/gadget/driver, but that doesn't help with the additional files, which are exclusively for root. If anyone knows how to solve this, ideally without patching the kernel, please speak up... This can be done by writing a udev script where you can set the required permissions, add the files to a group, etc. Starting the MTP server plugin == mtp.xml specifies: profile name=mtp type=server key name=usb_transport value=true/ /profile Under which circumstances will that server be activated right now in MeeGo's msyncd? If I read the buteo-syncfw code correctly, it would be activated if USB gets connected, which in turn is detected if usbmoded support is enabled - which as far as I know, is not the case in MeeGo because usbmoded is not in MeeGo. Yes, usb_moded is not in meego, but I hope that doesn't mean that USB connectivity can't be determined. Would it be acceptable to create a pseudo Sync::CONNECTIVITY_ALWAYS transport? A server plugin specifying that in its XML config then would be activated as soon as msyncd starts and remain active as long as msyncd runs, and mtp.xml could be changed to use that. That's already there, but it would be pointless to load the MTP plug-in always. Paths, configuration Why is /usr/share/mtp/deviceinfo.xml copied to /home/user/.mtpdeviceinfo.xml? The code says that it must be writable, but it did not become clear to me why the file must be writable: That's because device properties can be get/set by the initiator. We need to store them persistently if there's a change. How can the /home/user/MyDocs path be made configurable? Would deviceinfo.xml be a good place? /home/user/MyDocs represents the root of a storage, as defined in the MTP1 spec. We can have multiple storages, and each would have their own root. The root can even be an abstraction, for eg if we have a contacts storage. However, deviceinfo.xml would not be the right place for this. I agree that a storage plug-in should declare it's root, so we need a confguration scheme for storage plug-in's now. As of now, as you know, we have only one storage plug-in. media and Phone Memory are also hard-coded as description of that directory. Are these presented to the MTP host? Yes, they form part of the storage info as specified in MTP1. What about providing access to more than one directory? Sorry, I didn't understand. Do you mean more than one root, if so that's possible of course via multiple MTP storages, or do you mean multiple directories under the root? The latter works of course, because everything under the root is enumeratable. Are symbolic links followed? In other words, if I set up ~/MyDocs/ Videos - ~/Videos Pictures - ~/Pictures Music - ~/Music ... would that provide access also to the content of ~/Videos and ~/Pictures? Yes, it could be prevented by the storage though. The initiator wouldn't know, as the purpose of the MTP storage is to abstract the internals. The code currently tries to load the libfsstorage.so module from /usr/share/mtp, but it is installed in /usr/lib/mtp, which is indeed where it should be. Okay to change the code? Yes. MTP transport API = This is from an email that Paul sent privately earlier, still needs an answer as far as I know. - snip -- I [Paul] ran into a design issue around Zero-Length-Packets while doing this. The USB spec (paraphrased) says that transfers exceeding 1 packet size get transmitted and ended as follows: () All packets up to the last shall be the maximum packet size. If your higher-level protocol lets you know the size of the transfer ahead of time (e.g., fixed-size, info from a prior transaction, etc.) then you are done when all the data is transferred. No additional logic or packets necessary. () However, if you don't know the length ahead of time (e.g., the length is variable, or the sender decides to truncate early) then
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: ext Patrick Ohly [mailto:patrick.o...@intel.com] Sent: Monday, January 17, 2011 1:14 PM To: Kodihalli Deepak (Nokia-MS/Bangalore) Cc: Kavuri Sateesh (Nokia-MS/Bangalore); amitc...@gmail.com; Puranik Santosh (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; tuomas.kat...@ixonos.com; mhuna...@gmail.com; juha.paalija...@ixonos.com; Duggirala Karthik.2 (Nokia-MS/Helsinki); meego-dev@meego.com; Drews, Paul Subject: RE: Buteo MTP + gadgetfs On Mo, 2011-01-17 at 04:39 +, deepak.kodiha...@nokia.com wrote: Would it be acceptable to create a pseudo Sync::CONNECTIVITY_ALWAYS transport? A server plugin specifying that in its XML config then would be activated as soon as msyncd starts and remain active as long as msyncd runs, and mtp.xml could be changed to use that. That's already there, but it would be pointless to load the MTP plug-in always. I agree that a true usb_moded replacement would be nice, but in the meantime I don't see an alternative to loading the plugin all the time. How is this already there? I only see CONNECTIVITY_USB, CONNECTIVITY_INTERNET, CONNECTIVITY_BT. There'a a load_without_transport profile key, it could be used for this purpose. /home/user/MyDocs represents the root of a storage, as defined in the MTP1 spec. Yes, but the spec doesn't mandate what the local path has to be, does it? I agree with Alexander, it would be nice if all the standard freedesktop dirs could be exposed via MTP instead of forcing devices to use a single MyDocs directory. Like I said, we should leave this decision to the storage plug-in. We can use XDG user-dirs for file-system storages, but how about a contacts storage? We can have multiple storages, and each would have their own root. I think it would be more natural to have one root which contains multiple, independent directories. Well that's as per the MTP spec, each storage has it's own root. I can tell the host that my contacts storage's root it /contacts, and the host will show that as a directory with contacts objects contained within, but this doesn't mean there's a /contacts directory on the device. The root is an abstraction provided by the MTP storage(s). How about this idea: the root of the MTP plugin filestorage is /etc/mtp/filestorage/root. It can be a symlink to /home/user/MyDocs (same result as right now), or a directory with symlinks to multiple other directories. That might work only for file-system storages. Best Regards Patrick Ohly Senior Software Engineer Intel GmbH Open Source Technology Center Pützstr. 5 Phone: +49-228-2493652 53129 Bonn Germany Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: ext Patrick Ohly [mailto:patrick.o...@intel.com] Sent: Monday, January 17, 2011 5:06 PM To: Kodihalli Deepak (Nokia-MS/Bangalore) Cc: Kavuri Sateesh (Nokia-MS/Bangalore); amitc...@gmail.com; Puranik Santosh (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; tuomas.kat...@ixonos.com; mhuna...@gmail.com; juha.paalija...@ixonos.com; Duggirala Karthik.2 (Nokia-MS/Helsinki); meego-dev@meego.com; Drews, Paul Subject: RE: Buteo MTP + gadgetfs Speaking about plain files, can you comment on this special Playlist sub-directory which is currently hard-coded in the source? Can this be removed together with making the file storage plugin work without prescribing a specific file layout? windows expects a playlists directory to be present in the root, containing abstract playlist files (if there are playlists on the device), in order to be able to sync playlists. Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: ext Drews, Paul [mailto:paul.dr...@intel.com] Sent: Tuesday, January 18, 2011 1:17 AM To: Kodihalli Deepak (Nokia-MS/Bangalore); Ohly, Patrick; Kavuri Sateesh (Nokia-MS/Bangalore); amitc...@gmail.com; Puranik Santosh (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; tuomas.kat...@ixonos.com; mhuna...@gmail.com; juha.paalija...@ixonos.com; Duggirala Karthik.2 (Nokia-MS/Helsinki) Cc: meego-dev@meego.com Subject: RE: Buteo MTP + gadgetfs Here's what I'm worried about: Case A: MTP responder sends data over bulk pipe where the length is known ahead of time. Data takes multiple packets and is an exact multiple of packet length. MTP responder asks for fsync on the last sendData() call. Since the length is known ahead of time, a ZLP should _not_ be sent according to the USB spec. Even though the initiator knows the length in advance, it expects a ZLP if the packet size is a multiple of the USB max packet length. We have seen this with both windows and linux (libmtp) initiators. Case B: MTP responder sends data over bulk pipe but the length is _not_ known ahead of time. Data takes multiple packets and is an exact multiple of packet length. MTP responder asks for fsync on the last sendData() call. Since the length is _not_ known ahead of time, a ZLP _should_ be sent according to the USB spec. AFAIK, this is not possible with MTP1 at least, i.e the length _is_ always known. Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: ext Patrick Ohly [mailto:patrick.o...@intel.com] Sent: Monday, January 17, 2011 5:27 PM To: Kodihalli Deepak (Nokia-MS/Bangalore) Cc: Kavuri Sateesh (Nokia-MS/Bangalore); amitc...@gmail.com; Puranik Santosh (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; tuomas.kat...@ixonos.com; mhuna...@gmail.com; juha.paalija...@ixonos.com; Duggirala Karthik.2 (Nokia-MS/Helsinki); meego-dev@meego.com; Drews, Paul Subject: RE: Buteo MTP + gadgetfs On Mo, 2011-01-17 at 11:45 +, deepak.kodiha...@nokia.com wrote: Speaking about plain files, can you comment on this special Playlist sub-directory which is currently hard-coded in the source? Can this be removed together with making the file storage plugin work without prescribing a specific file layout? windows expects a playlists directory to be present in the root, containing abstract playlist files (if there are playlists on the device), in order to be able to sync playlists. What happens if there is no such directory? Unability to sync playlists or something worse? That would mean no playlist sync with windows (media player), nothing more. Is there some code in MeeGo which maps these playlists back to something used in MeeGo (whatever that is - not sure at the moment)? How would/does that fit into the file storage plugin? Yes, that's tracker. You can check the code in our fsstorageplugin which deals with reading and writing playlists from tracker. -- Best Regards Patrick Ohly Senior Software Engineer Intel GmbH Open Source Technology Center Pützstr. 5 Phone: +49-228-2493652 53129 Bonn Germany Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Buteo MTP + gadgetfs
Hi, -Original Message- From: meego-dev-boun...@meego.com [mailto:meego-dev-boun...@meego.com] On Behalf Of ext Patrick Ohly Sent: Monday, January 31, 2011 6:01 PM To: Tony Cc: Kavuri Sateesh (Nokia-MS/Bangalore); Duggirala Karthik.2 (Nokia-MS/Helsinki); meego-dev@meego.com; tuomas.kat...@ixonos.com; Aggarwal Amit.5 (Nokia-MS/Bangalore); kalle.aalto...@sasken.com; mhuna...@gmail.com Subject: Re: [MeeGo-dev] Buteo MTP + gadgetfs The trouble is that the kernel code that supports reads and writes for gadgetfs endpoints doesn't support non-blocking I/O properly. The select() notification and associated QSocketNotifier fires immediately and continuously even without data ready to read, and a read() on the file-desciptor blocks waiting for data even with O_NONBLOCK set. This is strange. If the fd is not set to O_NONBLOCK, then of course IO on it would block, but does that mean that if one poll()'s or does a select() on that fd, they would get notified even if there's no data to be read? This must be happening due to a bug in functionfs. Best Regards, Patrick Ohly The content of this message is my personal opinion only and although I am an employee of Intel, the statements I make here in no way represent Intel's position on the issue, nor am I authorized to speak on behalf of Intel on this matter. Regards, Deepak ___ MeeGo-dev mailing list MeeGo-dev@meego.com http://lists.meego.com/listinfo/meego-dev
Re: [MeeGo-dev] Help: msyncd periodical polling every 10 seconds
Hi,
-Original Message-
From: meego-dev-boun...@meego.com
[mailto:meego-dev-boun...@meego.com] On Behalf Of ext Yung, Winson W
Sent: Monday, March 07, 2011 8:28 AM
To: Sateesh Babu
Cc: meego-dev@meego.com
Subject: Re: [MeeGo-dev] Help: msyncd periodical polling every
10 seconds
Thanks for the hint, however your steps didn't yield any new
findings, here is what all I see after msyncd was launched.
I am using the msyncd that came with
buteo-syncfw-0.5.4-3.15.i586.rpm package, and I was running it
on Intel Medfield MeeGo platform. Once the program entering
the main loop, there is nothing coming out of console even
though set_sync_log_level was set to 4.
Maybe lsof might give you more information : lsof -p `pidof msyncd`.
You could then check on which fd the poll happens and see what that fd
corresponds to.
Regards,
Deepak
[meego@localhost ~]$ cd /usr/bin
[meego@localhost bin]$ export DISPLAY=:0.0 [meego@localhost
bin]$ cd /usr/bin [meego@localhost bin]$ ./msyncd [1] 1193
[meego@localhost bin]$ Warning: QDBusObjectPath: invalid path
Warning: QDBusObjectPath: invalid path
Warning: QDBusObjectPath: invalid path
Warning: Object::connect: No such signal
Buteo::NetworkManager::sessionConnected() in TransportTracker.cpp:78
Warning: Object::connect: No such signal
Buteo::NetworkManager::sessionError() in TransportTracker.cpp:80
Warning: ProfileManager.cpp 1054 : Profile file not found:
/home/meego/.sync/profiles/storage/hsms.x
Warning: ProfileManager.cpp 151 : Failed to load profile: hsms
[meego@localhost bin]$ tail -f ~/.sync/synchronizer.log
Debug: SyncDBusConnection.cpp 75 : DBus Connection Name:
sync-session-bus-0
Debug: SyncDBusConnection.cpp 76 : DBus is Connected: true
Debug: SyncDBusConnection.cpp 77 : DBus Connection ID: :1.1
Debug: static QDBusConnection
Buteo::SyncDBusConnection::sessionBus() :Exit, execution time: 2 ms
Debug: SyncBackup.cpp 54 : Registered sync backup to D-Bus
Debug: Buteo::SyncBackup::SyncBackup() :Exit, execution time: 7 ms
Debug: bool Buteo::Synchronizer::initialize() :Exit,
execution time: 500 ms
Debug: SyncSigHandler::SyncSigHandler(QObject*, const char*) :Entry
Debug: SyncSigHandler::SyncSigHandler(QObject*, const char*)
:Exit, execution time: 1 ms
Debug: main.cpp 94 : Entering event loop
/Winson
-Original Message-
From: Sateesh Babu [mailto:sateesh.kav...@gmail.com]
Sent: Friday, March 04, 2011 9:29 PM
To: Yung, Winson W
Cc: meego-dev@meego.com
Subject: Re: [MeeGo-dev] Help: msyncd periodical polling every 10
seconds
Hi,
Which hardware are you running this test (N900/Netbook/Aava/...) and
what is the software version you are using?
In order check whats going on, do the following:
# echo 4 /etc/sync/set_sync_log_level # pkill -9 msyncd # restart
msyncd # tail -f /home/user/.sync/synchronizer.log
This should give more information of what is going on in the
background
--
Sateesh
On Sat, Mar 5, 2011 at 2:34 AM, Yung, Winson W
winson.w.y...@intel.com
wrote:
Hi all, I have been noticing that msyncd (MeeGo generic sync service
middleware) has exhibited a pattern of polling every 10
seconds while
system
is idle (at MeeGo home page). I am trying to identify what causes
this periodical polling but without much progress. This is
part of my
effort to
keep the system awake from unnecessary SW wake up in order to
maintain
its
low power state. Can someone provide suggestions here on what is the
likely
cause of the polling, and/or how to track down this? This has been
challenging because it uses glib and Qt infrastructure, and
I have no
visibility (i.e. debug logging) of what's going on inside these
libraries.
Here I include a portion of the strace output of the msyncd, you can
see the
lines (high lighted in red) repeats roughly every 10 seconds because
the
poll(timeout=10sec).
clock_gettime(CLOCK_MONOTONIC, {836, 938422967}) = 0
clock_gettime(CLOCK_MONOTONIC, {836, 947477034}) = 0
socket(PF_INET, SOCK_STREAM|SOCK_CLOEXEC, IPPROTO_IP) = 18
socket(PF_NETLINK, SOCK_RAW, 0) = 19
bind(19, {sa_family=AF_NETLINK, pid=0, groups=}, 12) = 0
getsockname(19, {sa_family=AF_NETLINK, pid=802, groups=},
[12]) = 0
time(NULL) = 5741
sendto(19, \24\0\0\0\22\0\1\3m\26\0\0\0\0\0\0\0\0\0\0, 20, 0,
{sa_family=AF_NETLINK, pid=0, groups=}, 12) = 20
recvmsg(19, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
groups=},
msg_iov(1)=[{\250\1\0\0\20\0\2\0m\26\0\0\\3\0\0\0\0\4\3\1\0\
0\0I\0\1\
0
\0\0\0\0...,
4096}], msg_controllen=0, msg_flags=0}, 0) 4
recvmsg(19, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
groups=},
msg_iov(1)=[{\24\0\0\0\3\0\2\0m\26\0\0\\3\0\0\0\0\0\0\1\0\0\
0I\0\1\0\
0
\0\0\0...,
4096}], msg_controllen=0, msg_flags=0}, 0) = 0
sendto(19, \24\0\0\0\26\0\1\3n\26\0\0\0\0\0\0\0\0\0\0, 20, 0,
{sa_family=AF_NETLINK, pid=0, groups=}, 12) = 20
recvmsg(19, {msg_name(12)={sa_family=AF_NETLINK, pid=0,
groups=},
