Re: [Tracker] No contents indexing taking place
On Thu, 2012-03-15 at 15:18 +, Martyn Russell wrote: On 07/03/12 19:24, Karl Relton wrote: The only miner that uses tracker-miner-web.c in the current source tree (as far as I can see) is the Flickr miner. So I compiled that and started poking it. It does indeed show the problematic behaviour we feared: i.e. initially the interfaces/methods advertised in tracker-miner-object.c are OK, but those advertised in tracker-miner-web.c do NOT appear immediately. Walking up to an already running deamon is ok because by then everything is up running. So if a user tried to connect to the Miner.Flickr service (e.g. using the GetAssociationData method) they will be disappointed, getting the unknown interface error. As you say, the fix is not so easy, so I guess you may want to release 0.14 anyway and then think how best to guaruntee the interfaces come up. Karl, do you mind creating a bug report and I will comment there about a potential solution - just so this doesn't get lost in the abyss please? :) Thanks, You are welcome: https://bugzilla.gnome.org/show_bug.cgi?id=672259 ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On 07/03/12 19:24, Karl Relton wrote: The only miner that uses tracker-miner-web.c in the current source tree (as far as I can see) is the Flickr miner. So I compiled that and started poking it. It does indeed show the problematic behaviour we feared: i.e. initially the interfaces/methods advertised in tracker-miner-object.c are OK, but those advertised in tracker-miner-web.c do NOT appear immediately. Walking up to an already running deamon is ok because by then everything is up running. So if a user tried to connect to the Miner.Flickr service (e.g. using the GetAssociationData method) they will be disappointed, getting the unknown interface error. As you say, the fix is not so easy, so I guess you may want to release 0.14 anyway and then think how best to guaruntee the interfaces come up. Karl, do you mind creating a bug report and I will comment there about a potential solution - just so this doesn't get lost in the abyss please? :) Thanks, -- Regards, Martyn Founder and CEO of Lanedo GmbH. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Mon, 2012-03-05 at 12:23 +, Martyn Russell wrote: I believe I have diagnosed and proposed a fix for the problem. Please see https://bugzilla.gnome.org/show_bug.cgi?id=671314 for a full write up and proposed patch that fixes it. Thank you for investigating this fully, great patch. I've added to it and applied it to master now. IT should be in the 0.14.0 release i've been trying to get out for a week or so now. You mentioned in the bug comments that there could be other places with the same issue. I have tried to do a code trawl, and believe that where the code is pure .c all is okay except perhaps libtracker-miner/tracker-miner-web.c. For the .vala code, I can't really decipher it though. For tracker-miner-web.c I am concerned by the comment at line 193: /* No need to RequestName again as already done by the parent TrackerMiner object */ As far as I can see the parent object in tracker-miner-object.c creates the connection, registers its own interface, and then sets the name on the bus ... and this happens before (AFAICT) the child object registers its interface object. So I would imagine that the web miner might not get all its interfaces registered correctly. Karl ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On 07/03/12 09:58, Karl Relton wrote: On Mon, 2012-03-05 at 12:23 +, Martyn Russell wrote: I believe I have diagnosed and proposed a fix for the problem. Please see https://bugzilla.gnome.org/show_bug.cgi?id=671314 for a full write up and proposed patch that fixes it. Thank you for investigating this fully, great patch. I've added to it and applied it to master now. IT should be in the 0.14.0 release i've been trying to get out for a week or so now. You mentioned in the bug comments that there could be other places with the same issue. Indeed. I have tried to do a code trawl, and believe that where the code is pure .c all is okay except perhaps libtracker-miner/tracker-miner-web.c. For the .vala code, I can't really decipher it though. I checked already, the writeback code was the only place and I changed that already IIRC. For tracker-miner-web.c I am concerned by the comment at line 193: /* No need to RequestName again as already done by the parent TrackerMiner object */ Yes it is. The libtracker-miner library requests the name for you so assuming the library is done correctly, ALL miners should work properly. As far as I can see the parent object in tracker-miner-object.c creates the connection, registers its own interface, and then sets the name on the bus ... and this happens before (AFAICT) the child object registers its interface object. So I would imagine that the web miner might not get all its interfaces registered correctly. Yea, I think you could be right here. Are you seeing problems with the web miners then? One approach here might be to do the name request in _init() not _initable_init(), but that's quite a risky change to make at this late stage before we release. -- Regards, Martyn Founder and CEO of Lanedo GmbH. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Wed, 2012-03-07 at 10:45 +, Martyn Russell wrote: On 07/03/12 09:58, Karl Relton wrote: On Mon, 2012-03-05 at 12:23 +, Martyn Russell wrote: I believe I have diagnosed and proposed a fix for the problem. Please see https://bugzilla.gnome.org/show_bug.cgi?id=671314 for a full write up and proposed patch that fixes it. Thank you for investigating this fully, great patch. I've added to it and applied it to master now. IT should be in the 0.14.0 release i've been trying to get out for a week or so now. You mentioned in the bug comments that there could be other places with the same issue. Indeed. I have tried to do a code trawl, and believe that where the code is pure .c all is okay except perhaps libtracker-miner/tracker-miner-web.c. For the .vala code, I can't really decipher it though. I checked already, the writeback code was the only place and I changed that already IIRC. For tracker-miner-web.c I am concerned by the comment at line 193: /* No need to RequestName again as already done by the parent TrackerMiner object */ Yes it is. The libtracker-miner library requests the name for you so assuming the library is done correctly, ALL miners should work properly. As far as I can see the parent object in tracker-miner-object.c creates the connection, registers its own interface, and then sets the name on the bus ... and this happens before (AFAICT) the child object registers its interface object. So I would imagine that the web miner might not get all its interfaces registered correctly. Yea, I think you could be right here. Are you seeing problems with the web miners then? One approach here might be to do the name request in _init() not _initable_init(), but that's quite a risky change to make at this late stage before we release. I haven't got web miners installed at the moment. I'll see if I can install it and prod it with dbus-send. That might take me a wee while though. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Wed, 2012-03-07 at 10:50 +, Karl Relton wrote: On Wed, 2012-03-07 at 10:45 +, Martyn Russell wrote: For tracker-miner-web.c I am concerned by the comment at line 193: /* No need to RequestName again as already done by the parent TrackerMiner object */ Yes it is. The libtracker-miner library requests the name for you so assuming the library is done correctly, ALL miners should work properly. As far as I can see the parent object in tracker-miner-object.c creates the connection, registers its own interface, and then sets the name on the bus ... and this happens before (AFAICT) the child object registers its interface object. So I would imagine that the web miner might not get all its interfaces registered correctly. Yea, I think you could be right here. Are you seeing problems with the web miners then? One approach here might be to do the name request in _init() not _initable_init(), but that's quite a risky change to make at this late stage before we release. I haven't got web miners installed at the moment. I'll see if I can install it and prod it with dbus-send. That might take me a wee while though. The only miner that uses tracker-miner-web.c in the current source tree (as far as I can see) is the Flickr miner. So I compiled that and started poking it. It does indeed show the problematic behaviour we feared: i.e. initially the interfaces/methods advertised in tracker-miner-object.c are OK, but those advertised in tracker-miner-web.c do NOT appear immediately. Walking up to an already running deamon is ok because by then everything is up running. So if a user tried to connect to the Miner.Flickr service (e.g. using the GetAssociationData method) they will be disappointed, getting the unknown interface error. As you say, the fix is not so easy, so I guess you may want to release 0.14 anyway and then think how best to guaruntee the interfaces come up. Regards Karl ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On 04/03/12 18:11, Karl Relton wrote: On Thu, 2012-03-01 at 23:19 +, Karl Relton wrote: I've found out some more - the tracker-extract process is in fact being started (by dbus), but for some reason the initial call to it is getting an unknown interface error. Subsequent calls to it (while it is still running), will work however. By watching what processes are running, I can see the tracker-extract process start, and then gracefully exit 30 seconds later. The first file change that resulted in its invocation doesn't get any indexing done (because it gets the unknown interface error), but if I change the file again (or another file) before the 30 second inactivity timeout then indexing will occur with respect to that second change. So the problem seems to be that first call. Not sure if it is an error (or timeout) in dbus, or an error in the way tracker-miner-fs/tracker-extract are trying to interact with each other. Could it be that tracker-extract is taking too long to register the interface (make it available) in dbus? I believe I have diagnosed and proposed a fix for the problem. Please see https://bugzilla.gnome.org/show_bug.cgi?id=671314 for a full write up and proposed patch that fixes it. Thank you for investigating this fully, great patch. I've added to it and applied it to master now. IT should be in the 0.14.0 release i've been trying to get out for a week or so now. -- Regards, Martyn Founder and CEO of Lanedo GmbH. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Thu, 2012-03-01 at 23:19 +, Karl Relton wrote: I've found out some more - the tracker-extract process is in fact being started (by dbus), but for some reason the initial call to it is getting an unknown interface error. Subsequent calls to it (while it is still running), will work however. By watching what processes are running, I can see the tracker-extract process start, and then gracefully exit 30 seconds later. The first file change that resulted in its invocation doesn't get any indexing done (because it gets the unknown interface error), but if I change the file again (or another file) before the 30 second inactivity timeout then indexing will occur with respect to that second change. So the problem seems to be that first call. Not sure if it is an error (or timeout) in dbus, or an error in the way tracker-miner-fs/tracker-extract are trying to interact with each other. Could it be that tracker-extract is taking too long to register the interface (make it available) in dbus? I believe I have diagnosed and proposed a fix for the problem. Please see https://bugzilla.gnome.org/show_bug.cgi?id=671314 for a full write up and proposed patch that fixes it. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On 01/03/12 08:39, Karl Relton wrote:
Tracker 0.12 in Ubuntu Precise Pangolin doesn't seem to be indexing any
file contents at all these days - not even words in plain text files.
But I see in tracker-miner-fs.log :
29 Feb 2012, 21:18:57: Tracker: tracker-backend.vala:84:
Tracker.Sparql.Backend.query_async(): 'SELECT ?s WHERE { ?s nie:url
file:///home/parents/Documents/how_about3.txt}'
29 Feb 2012, 21:18:58: Tracker: Processing file
'file:///home/parents/Documents/how_about3.txt'...
29 Feb 2012, 21:18:58: Tracker: Processed 0/0, estimated unknown time
left, less than one second elapsed
29 Feb 2012, 21:18:58: Tracker: Loading extractor rules...
(/usr/share/tracker/extract-rules)
29 Feb 2012, 21:18:58: Tracker: Loaded rule '10-abw.rule'
... snip ...
29 Feb 2012, 21:18:58: Tracker: Loaded rule '90-text-generic.rule'
29 Feb 2012, 21:18:58: Tracker: Extractor rules loaded
29 Feb 2012, 21:19:00: Tracker: Could not process
'file:///home/parents/Documents/how_about3.txt':
GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface
`org.freedesktop.Tracker1.Extract' on object at
path /org/freedesktop/Tracker1/Extract
That certainly looks like a broken install.
Out of interest, can you start the tracker-extract process manually and
does it fix the issue?
/usr/libexec/tracker-extract -v 3
Also, you don't have older Tracker versions installed on your machine do
you (that could conflict API wise)?
I guess that UknownMethod message is bad news, stopping the extractor
from actually being called.
Indeed.
What would cause this - is there some dbus configuration file missing? I
see the interface is defined in /usr/share/tracker/tracker-extract.xml
(the same as in oneiric, as far as I can tell).
Presumably this file is in place too and points to the right binary:
/usr/share/dbus-1/services/org.freedesktop.Tracker1.Extract.service
--
Regards,
Martyn
Founder and CEO of Lanedo GmbH.
___
tracker-list mailing list
tracker-list@gnome.org
http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Thu, 2012-03-01 at 09:44 +, Martyn Russell wrote:
On 01/03/12 08:39, Karl Relton wrote:
Tracker 0.12 in Ubuntu Precise Pangolin doesn't seem to be indexing any
file contents at all these days - not even words in plain text files.
But I see in tracker-miner-fs.log :
29 Feb 2012, 21:18:57: Tracker: tracker-backend.vala:84:
Tracker.Sparql.Backend.query_async(): 'SELECT ?s WHERE { ?s nie:url
file:///home/parents/Documents/how_about3.txt}'
29 Feb 2012, 21:18:58: Tracker: Processing file
'file:///home/parents/Documents/how_about3.txt'...
29 Feb 2012, 21:18:58: Tracker: Processed 0/0, estimated unknown time
left, less than one second elapsed
29 Feb 2012, 21:18:58: Tracker: Loading extractor rules...
(/usr/share/tracker/extract-rules)
29 Feb 2012, 21:18:58: Tracker: Loaded rule '10-abw.rule'
... snip ...
29 Feb 2012, 21:18:58: Tracker: Loaded rule '90-text-generic.rule'
29 Feb 2012, 21:18:58: Tracker: Extractor rules loaded
29 Feb 2012, 21:19:00: Tracker: Could not process
'file:///home/parents/Documents/how_about3.txt':
GDBus.Error:org.freedesktop.DBus.Error.UnknownMethod: No such interface
`org.freedesktop.Tracker1.Extract' on object at
path /org/freedesktop/Tracker1/Extract
That certainly looks like a broken install.
Out of interest, can you start the tracker-extract process manually and
does it fix the issue?
/usr/libexec/tracker-extract -v 3
Yes (for me it is in /usr/lib/tracker/tracker-extract). Starting this
manually then the file does get indexed, and found in subsequent
searches.
Also, you don't have older Tracker versions installed on your machine do
you (that could conflict API wise)?
Not as far as I can tell.
I guess that UknownMethod message is bad news, stopping the extractor
from actually being called.
Indeed.
What would cause this - is there some dbus configuration file missing? I
see the interface is defined in /usr/share/tracker/tracker-extract.xml
(the same as in oneiric, as far as I can tell).
Presumably this file is in place too and points to the right binary:
/usr/share/dbus-1/services/org.freedesktop.Tracker1.Extract.service
Yes - this file is in place and points correctly
to /usr/lib/tracker/tracker-extract
As another test I killed my tracker-store and tracker-miner-fs process,
and then just restarted tracker-miner-fs. This then seemed to invoke
tracker-store automatically (I assume through a dbus service request) -
so dbus and the other deamons seem to be playing ball, just not
tracker-extract.
Karl
___
tracker-list mailing list
tracker-list@gnome.org
http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On 01/03/12 10:47, Karl Relton wrote: That certainly looks like a broken install. Out of interest, can you start the tracker-extract process manually and does it fix the issue? /usr/libexec/tracker-extract -v 3 Yes (for me it is in /usr/lib/tracker/tracker-extract). Starting this manually then the file does get indexed, and found in subsequent searches. snip Presumably this file is in place too and points to the right binary: /usr/share/dbus-1/services/org.freedesktop.Tracker1.Extract.service Yes - this file is in place and points correctly to /usr/lib/tracker/tracker-extract NOTE: The above two locations for tracker-extract are *NOT* the same. This looks quite suspicious to me. I would wager that running these two commands outputs different results: /usr/libexec/tracker-extract -V /usr/lib/tracker/tracker-extract -V In any case, I wouldn't expect both locations to have that binary. So it certainly looks like you need to clean your $prefix and re-make install. -- Regards, Martyn Founder and CEO of Lanedo GmbH. ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
Re: [Tracker] No contents indexing taking place
On Thu, 2012-03-01 at 15:31 +, Martyn Russell wrote: On 01/03/12 10:47, Karl Relton wrote: That certainly looks like a broken install. Out of interest, can you start the tracker-extract process manually and does it fix the issue? /usr/libexec/tracker-extract -v 3 Yes (for me it is in /usr/lib/tracker/tracker-extract). Starting this manually then the file does get indexed, and found in subsequent searches. snip Presumably this file is in place too and points to the right binary: /usr/share/dbus-1/services/org.freedesktop.Tracker1.Extract.service Yes - this file is in place and points correctly to /usr/lib/tracker/tracker-extract NOTE: The above two locations for tracker-extract are *NOT* the same. This looks quite suspicious to me. I would wager that running these two commands outputs different results: /usr/libexec/tracker-extract -V /usr/lib/tracker/tracker-extract -V In any case, I wouldn't expect both locations to have that binary. So it certainly looks like you need to clean your $prefix and re-make install. laptop1:~ /usr/libexec/tracker/tracker-extract -V /usr/libexec/tracker/tracker-extract: Command not found. laptop1:~ /usr/lib/tracker/tracker-extract -V Tracker 0.12.10 This program is free software and comes without any warranty. It is licensed under version 2 or later of the General Public License which can be viewed at: http://www.gnu.org/licenses/gpl.txt Looks like Ubuntu compiles their packages with --libexecdir=/usr/lib, though at the moment I cannot see where they set this. Note the location in my .service file does match where the executable has been installed to. Remember also that tracker-miner-fs seems to be able to start tracker-store okay (presumably via dbus /usr/share/dbus-1/services/org.freedesktop.Tracker1.service, which points to /usr/lib/tracker/tracker-store), so I think the packages are built consistently. Karl ___ tracker-list mailing list tracker-list@gnome.org http://mail.gnome.org/mailman/listinfo/tracker-list
