thanks for looking Gert. The problem seems to be, when indexing from scratch,
after creating an empty index and restarting solr, the resource is indexed but
only using its DC. So no searches on its dcterms work. It only indexes on
DCTERMS when the resource is indexed via pid.
Alistair
--
mov eax,1
mov ebx,0
int 80h
From: Gert Schmeltz Pedersen <gerts...@gmail.com<mailto:gerts...@gmail.com>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Monday, 19 August 2013 12:35
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
Thank you, Alistair. Sorry that I replied to your previous email, the later one
was placed in my spam folder by my email client, and I only noticed it this
morning. I include that email in the history below for completeness.
These two log extracts
the initial ingest from all foxml files for one specific resource that shows
the behaviour - initial–ingest-fgs.log
the log from indexing that resource from pid - index–from-pid.log
both show the generated index document, and they are identical. As I said in
the previous reply, I thought that the problem was that one of them would lack
the dcterms fields. Since they are identical, which is as it should be, I think
I need your explanation again about what the problem is.
Gert
On 19/08/2013, at 10.26, Alistair Young wrote:
the later email should contain that info Gert. Just wondering if the initial
indexing gets hold of a cached version of the resource (before the DCTERMS
namespace was renamed to be the same as the XSLT).
thanks,
Alistair
--
mov eax,1
mov ebx,0
int 80h
From: Gert Schmeltz Pedersen <gerts...@gmail.com<mailto:gerts...@gmail.com>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Friday, 16 August 2013 09:25
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
Thank you, Alistair. Your log lines show me part of one index document
generation, which contains the dct (dcterms) fields. However, if I understand
you right, the problem is that you get an index document without the dct
fields, when you call the updateIndex in one of the other ways (fromPid,
fromFoxmlFiles, triggered by ingest/update). So, in order to compare the
successful case and the problematic case, I need to see also the log lines from
the problematic index document generation, and I need to see all the log lines
of both operations, that is, from the INFO log line starting the operation to
the INFO log line at the end of the operation.
Gert
On 15/08/2013, at 10.58, Alistair Young wrote:
I've dug deeper into the logs Gert and attached are:
the initial ingest from all foxml files for one specific resource that shows
the behaviour - initial–ingest-fgs.log
the log from indexing that resource from pid - index–from-pid.log
and the corresponding solr log extracts for both indexing job - solr–extract.log
There appear to be two ingests initially for that resource into solr?
DEBUG - 2013-08-15 09:31:47.790
DEBUG - 2013-08-15 09:34:29.375
they're from the same indexing job, not run separately
thanks,
Alistair
--
On 14/08/2013, at 17.52, Alistair Young wrote:
Hope this is of some use…
Alistair
-----------------
mov eax,1
mov ebx,0
int 80
From: Gert Schmeltz Pedersen <gerts...@gmail.com<mailto:gerts...@gmail.com>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Wednesday, 14 August 2013 16:15
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
Let me see the log lines.
Gert
On 14/08/2013, at 17.04, Alistair Young wrote:
Having said that, I have to force index via pid to get it to index the updated
resources.
Renamed all dcterms datastreams namespaces to be same as xslt in the
objectStore. Rebuilt the resourceIndex and database. Blew away the solr index.
Index from scratch and it won't index dcterms on the resources that had their
dcterms renamed. Have to force update on them by pid and then it indexes their
dcterms.
Any idea why an initial index would fail while an individual index via pid
would work?
Alistair
-----------------
mov eax,1
mov ebx,0
int 80
From: Alistair Young <alistair.yo...@uhi.ac.uk<mailto:alistair.yo...@uhi.ac.uk>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Wednesday, 14 August 2013 09:53
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
Thanks for that Gert. Finally tracked it down to different representations of
the same namespace in the repo. Some objects have their dcterms namespace
declared as per the xslt, others don't. I feel some metadata munging coming on.
Thanks for confirming the single xslt though. At the end of this hoping to
produce a tutorial on fedora/gsearch/solr. This seems to be the last thorn in
the bush.
Cheers,
Alistair
From: Gert Schmeltz Pedersen <gerts...@gmail.com<mailto:gerts...@gmail.com>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Tuesday, 13 August 2013 20:51
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
No separate xslt, gsearch uses the same indexing xslt, whether triggered by an
ingest/update of the Fedora object, or by an updateIndex fromPid operation, or
by an updateIndex fromFoxmlFiles operation. If you get different indexing
documents in these three cases, then the way to find out why, is to compare the
log lines in debug mode. I may help, if you send me those log lines.
Gert
On 13/08/2013, at 18.47, Alistair Young wrote:
Nup , not that. Even uploading the DCTERMS first it doesn't index them. Is
there a separate xslt it uses for updating a resource? The symptoms are
identical to when the main xslt was lacking the dcterms namespace.
Alistair
-------------------
Alistair Young
Àrd Innleadair air Bathair-bog
UHI@Sabhal Mòr Ostaig
From: Alistair Young <alistair.yo...@uhi.ac.uk<mailto:alistair.yo...@uhi.ac.uk>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Tuesday, 13 August 2013 16:44
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: Re: [fcrepo-user] gsearch not indexing dcterms on upload
I think the problem is down to the multi part commit technique being used when
uploading to Fedora:
1 – new object pid is generated by a POST
2 – the DC datastream is then PUT
3 - the DCTERMS datastream is then PUT
4 - the content datastream is then POST
so I suspect 3 (or 4) is not getting to solr. gsearch indexes on 2 but not on 3
I'm assuming. All the datastreams are present in the foxml files when doing a
new index from scratch so they work.
Is it possible to turn off auto indexing on upload and instead force it via a
REST call to gsearch once the complete resource is uploaded?
Alistair
--
mov eax,1
mov ebx,0
int 80h
From: Alistair Young <alistair.yo...@uhi.ac.uk<mailto:alistair.yo...@uhi.ac.uk>>
Reply-To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Date: Tuesday, 13 August 2013 16:10
To: "Support and info exchange list for Fedora users."
<fedora-commons-users@lists.sourceforge.net<mailto:fedora-commons-users@lists.sourceforge.net>>
Subject: [fcrepo-user] gsearch not indexing dcterms on upload
I think I must be missing a file mod or something. Indexing from scratch, using
from foxml files works fine and dcterms in all resources are indexed as the
namespaces are in the xslt. When uploading a new resource to fedora it gets
indexed but only on dc, not dcterms. Does gsearch use a different xslt when
indexing from an upload rather than a clean start?
Alistair
--
mov eax,1
mov ebx,0
int 80h
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net<mailto:Fedora-commons-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net<mailto:Fedora-commons-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
<fglogs.tar.gz>------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net<mailto:Fedora-commons-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net<mailto:Fedora-commons-users@lists.sourceforge.net>
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite!
It's a free troubleshooting tool designed for production.
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://pubads.g.doubleclick.net/gampad/clk?id=48897031&iu=/4140/ostg.clktrk
_______________________________________________
Fedora-commons-users mailing list
Fedora-commons-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/fedora-commons-users
