Noble,
Thanks for continuing to assist me on trying to come up a config that works.
A couple of questions/clarifications:
1) I had to introduce the artificial comboID and the transformer because
of a conflict with a parallel entity on the id (vets and owners).
2) I don't think there is a conflict
Hi Julio,
The following are my assumptions after studying your given data-config
examples
1. The column id is present in all three tables -- vets, owners and pets.
2. Vets and owners are independent of each other, there is no join required
between them
3. There is a parent-child relationship
Thanks Shalin,
I'll try this asap. Yes, you did understand the sample schema I've been
playing with.
Just a couple of questions to clarify for my own understanding your
proposal.
1) the column comboId doesn't exist on the dB (yet it is specified as a
separate column for both owners and vets in
1. Yes you can add virtual columns. Any column/name which does not exist in
schema.xml are used for joins but not added to the document. The column
attribute is the key using which data is read from the entity's Map. The
name attribute is the solr field to which data is written. Also note that
Another thing to note, the parentDeltaQuery is of no use if deltaQuery is
not specified. In my experience, DataImportHandler is pretty fast and delta
queries may not be needed at all if your dataset is small. We use it without
delta queries even with millions of solr documents and it completes
Hi Noble,
I had forgotten to also list comboId as a uniqueKey in the schema.xml file.
But that didn't make a difference.
It still complained about the Document [null] missing required field: id
for each row it ran into of the outer entity.
If you look at the debug output of the entity:pets (see
hi julio,
You must create an extra field for 'comboid' because you really need
the 'id' for your sub-entities. Your data-config must look as follows.
The pet also has a field called 'id' . It is not a good idea. call it
'petid' or something (both in dataconfig and schema.xml). Please make
sure
The id in pet should be aliased to 'petid' , because id is coming
from both entities there is a conflict
entity name=owners pk=id
query=select id,first_name,last_name FROM owners
transformer=TemplateTransformer
field column=comboId template=owners-${owners.id}/
Shalin,
I experimented with it, and the null pointer exception has been taken care
of. Thank you.
I have a different problem now. I believe it is a syntax/specification
problem.
When importing data, I got the following exceptions:
SEVERE: Exception while adding:
hi Julio,
delete my previous response. In your schema , 'id' is the uniqueKey.
make 'comboid' the unique key. Because that is the target field name
coming out of the entity 'owners'
--Noble
On Tue, Jun 3, 2008 at 9:46 AM, Noble Paul നോബിള് नोब्ळ्
[EMAIL PROTECTED] wrote:
The field 'id' is
The field 'id' is repeated for pet also rename it to something else
say
entity name=pets pk=id
query=SELECT id,name,birth_date,type_id FROM pets WHERE
owner_id='${owners.id}'
parentDeltaQuery=SELECT id FROM owners WHERE
id=${pets.owner_id}
field column=id
julio,
Looks like it is a bug.
We can give u a new TemplateTransformer.java which we will incorporate
in the next patch
--Noble
On Sat, May 31, 2008 at 12:24 AM, Julio Castillo
[EMAIL PROTECTED] wrote:
I'm sorry Shalin, but I still get the same Null Pointer exception. This is
my complete
Hi Julio,
I've fixed the bug, can you please replace the exiting
TemplateTransformer.java in the SOLR-469.patch and use the attached
TemplateTransformer.java file. We'll add the changes to our next
patch. Sorry for all the trouble.
On Sat, May 31, 2008 at 10:31 PM, Noble Paul നോബിള് नोब्ळ्
Not a problem Shalin,
On the contrary, thanks for all your hard work.
I will try it as soon as possible.
** julio
-Original Message-
From: Shalin Shekhar Mangar [mailto:[EMAIL PROTECTED]
Sent: Saturday, May 31, 2008 10:26 AM
To: solr-user@lucene.apache.org
Subject: Re: How to describe
Noble,
I tried the template setting for the id field, but I didn't notice any
different behavior. I also didn't see where this would be reflected.
I looked at the fields and the debug output for the dataImporter and
couldn't see any reference to a modified id name (per the template
instructions).
You need to enable TemplateTransformer for your entity. For example:
entity name=owners pk=id query= transformer=TemplateTransformer
On Fri, May 30, 2008 at 11:31 AM, Julio Castillo
[EMAIL PROTECTED] wrote:
Noble,
I tried the template setting for the id field, but I didn't notice any
Sorry I forgot to mention that.
http://wiki.apache.org/solr/DataImportHandler#head-a6916b30b5d7605a990fb03c4ff461b3736496a9
--Noble
On Fri, May 30, 2008 at 11:37 AM, Shalin Shekhar Mangar
[EMAIL PROTECTED] wrote:
You need to enable TemplateTransformer for your entity. For example:
entity
You might try creating your whole transform as an SQL database view rather
than with the Solr transformer toolkit.
This would also make it easier to directly examine the data to be indexed.
Lance
-Original Message-
From: Julio Castillo [mailto:[EMAIL PROTECTED]
Sent: Thursday, May 29,
The surname is used just as an example of a field.
The NullPointerException is because the same field id tries to use
it's own value in a template. The template cannot contain the same
field on which it is being applied. I'd suggest that you get the id
aliased to another name, for example using a
I'm sorry Shalin, but I still get the same Null Pointer exception. This is
my complete dataconfig.xml (I remove the parallel entity to narrow down the
scope of the problem).
dataConfig
document name=doc-1
entity name=vets pk=id
query=select id as idAlias,first_name,last_name FROM
Hi Julio,
The first data-config is correct.
You're running DataImportHandler in debug mode which creates only the
first 10 documents by default. You can also add count=N to index only
the first N documents. But this is intended only for debugging
purposes. If you want to do a full-import just
Thanks Shalin,
I tried putting everything under the same document (two different unrelated
entities), and got a bit further.
My problem now appears to be both of them stepping on each other due to id
conflicts. Currently my id is defined in my schema as
field name=id type=long indexed=true
That will happen only if id is the uniqueKey in Solr and the id coming
from both your tables have same values. In that case, they will
overwrite each other. You will need a separate uniqueKey (on other
than id field).
On Fri, May 30, 2008 at 6:34 AM, Julio Castillo [EMAIL PROTECTED] wrote:
Consider constructing the id concatenating an extra string for each
document . You can construct that field using the TeplateTransformer.
in the entity owners keep the id as
field column=id name=id template=owners-${owners.id}/
and in vets
field column=id name=id template=vets-${vets.id}/
or
24 matches
Mail list logo
