Solr For
Hi, Looking for solr config for Job Site. In a job site there are 2 main searches. 1) Employee can search for job ( based on skill set, job location, title, salary ) 2) Employer can search for employees ( based on skill set, exp, location, ) Should i have a separate config xml for both searches? Thanks, Baskar
Equivalent Java client code
Hi, I use the below query to check if the both of the word are there in the document. http://localhost:18080/solr/select?defType=dismax&pf=tagged_skills ^100&q=java+ejb+Spring&qf=tagged_skills&q.op=AND I am not sure how to execute the same thing using Solr Java client. SolrQuery query = new SolrQuery(); query.setQuery( "tagged_skills:\"java+and+ejb\"" ); QueryResponse rsp = server.query( query ); SolrDocumentList docs = rsp.getResults(); Also not sure how to check if the given search is there in title and tageed_skills field. Could you please help me on this? Thanks, Baskar.S
Re: Solr Java Client
Hi Gora, Thanks a lot for your reply. "As people have already advised you, the best way to decide how to organise your data in the Solr index depends on the searches that you want to make. This is not entirely clear from your description above. The flattening sample that you show above would be suitable if the user is to search by 'child' attributes, but can be simplified otherwise." *Yes, the search is based on the child attributes.* "How would you know that the user name has been changed? Is there a modification date for that table. If so, it would make sense to check that against the last time indexing to Solr was done. A DIH delta-import makes this straightforward." *As of now, there is no special column to know if the username has been changed. * *But, whenever the user update his name, i can track that in my java code and send the update to Solr. * *Here, I am planning to use Solr java client. * *But, all the above things are possible with Java client and also with delta-import. * *I am looking for changing the solr data whenever there is a change in the database.* *Even there is a small delay i am fine with that. * *Which one you will suggest? * *Solr Java client or DIH delta-import * *My application running on server A, database on server B and solr will be on server C. * *If i am supposed to use, Solr Java client, i may need to hit the database sometimes to get some parent data and then need to send the same to Solr. * *Guess, its a unnecessary trip. * *So confused here, if i need to go with Java client or DIH delta import. * Thanks, Baskar.S On Mon, Sep 16, 2013 at 9:23 AM, Gora Mohanty wrote: > On 16 September 2013 02:47, Baskar Sikkayan wrote: > [...] > > Have a question now. > > > > I know in solr its flat file system and the data will be in denormalized > > form. > > > > My question : > > > > Have 3 tables, > > > > 1) user (userid, firstname, lastname, ...) > > 2) master (masterid, skills, ...) > > 3) child (childid, masterid, userid, ...) > > > > In solr, i have added all these field for each document. > > > > Example, > > > > childid,masterid,userid,skills,firstname,lastname > > > > Real Data Example, > > > > 1(childid),1(masterid),1(userid),"java,jsp","baskar","sks" > > 2(childid),1(masterid),1(userid),"java,jsp","baskar","sks" > > 3(childid),1(masterid),1(userid),"java,jsp","baskar","sks" > > As people have already advised you, the best way to decide > how to organise your data in the Solr index depends on the > searches that you want to make. This is not entirely clear > from your description above. The flattening sample that you > show above would be suitable if the user is to search by > 'child' attributes, but can be simplified otherwise. > > > The above data sample is from solr document. > > In my search result, i will have to show all these fields. > > > > User may change the name at any time.The same has to be updated in solr. > > > > In this case, i need to find all the child id that belongs to the user > and > > update the username with those child ids. > > > > Please tell me if there is any other better approach than this. > > How would you know that the user name has been changed? > Is there a modification date for that table. If so, it would make > sense to check that against the last time indexing to Solr was > done. A DIH delta-import makes this straightforward. > > Updates as you suggest above would be the normal way to handle > things. You should batch your updates, say by running an update > script at periodic intervals. > > Regards, > Gora >
Re: Solr Java Client
Hi,
I read few chapters of solr cookbook and hope, I have good basic
understanding of Apache solr and Datahandler.
I was able to modify the field and all. I am able to understand now, what i
was advised to do.
Have a question now.
I know in solr its flat file system and the data will be in denormalized
form.
My question :
Have 3 tables,
1) user (userid, firstname, lastname, ...)
2) master (masterid, skills, ...)
3) child (childid, masterid, userid, ...)
In solr, i have added all these field for each document.
Example,
childid,masterid,userid,skills,firstname,lastname
Real Data Example,
1(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
2(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
3(childid),1(masterid),1(userid),"java,jsp","baskar","sks"
The above data sample is from solr document.
In my search result, i will have to show all these fields.
User may change the name at any time.The same has to be updated in solr.
In this case, i need to find all the child id that belongs to the user and
update the username with those child ids.
Please tell me if there is any other better approach than this.
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:42 PM, Gora Mohanty wrote:
> On 15 September 2013 21:36, Baskar Sikkayan wrote:
> >
> > Hi,
> > Thanks for the reply. I got it now. Is there XSD for schema.xml and
> > solrconfig.xml?
> > Also, if i have two different fields with different filed type, how can
> > differentiate it in SolrJava client.
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> >
> > Is there any option here to put the field type?
> [...]
>
> Field types are defined in schema.xml.
>
> I would again suggest approaching things systematically,
> and understanding how the various pieces work.
>
> Regards,
> Gora
>
Re: Solr Java Client
Hi,
Thanks for the reply. I got it now. Is there XSD for schema.xml and
solrconfig.xml?
Also, if i have two different fields with different filed type, how can
differentiate it in SolrJava client.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
Is there any option here to put the field type?
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:20 PM, Erick Erickson wrote:
> Put the data in two different fields. You can also include
> a type field and use the fq=type:type1 or fq=type:type2
> if you need to distinguish them separately.
>
> Best,
> Erick
>
>
> On Sun, Sep 15, 2013 at 11:00 AM, Baskar Sikkayan >wrote:
>
> > Hi Erick,
> > Thanks a lot for your reply. I am still not clear.
> > I will have 2 different searches.
> > So, there will be 2 different kind of documents with different fields.
> >
> > But the example below, gives me a impression that
> >
> > SolrInputDocument doc = new SolrInputDocument();even there are 2 diff
> > searches, the data is getting stored in the same
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> >
> > If i use the same doc, how to diff data for 2 different searches.
> > Could you explain me clear on this??//
> >
> >
> > Thanks,
> > Baskar.S
> >
> >
> >
> > On Sun, Sep 15, 2013 at 7:57 PM, Erick Erickson > >wrote:
> >
> > > Listen to Wunder. You really have to switch
> > > gears and think in terms of searching rather
> > > than database tables.
> > >
> > > First, there's no requirement that all documents
> > > have the same fields. There's very little penalty
> > > for this.
> > >
> > > Second, you have to get over normalized data.
> > > It's unclear how many employees/records you have,
> > > but unless it's a really big set of records (i.e multiple
> > > tens of millions) you don't have to worry much
> > > about saving space.
> > >
> > > BTW, your java example works, but you do NOT
> > > want to commit after every document. Configure
> > > your auto soft and hard commits to do this on a
> > > time-based basis.
> > >
> > > So I'd recommend you approach it differently.
> > > Don't get stuck on your current data model. Instead
> > > define the searches you want to make and _then_
> > > decide the appropriate fields in the docs.
> > >
> > > But to your question, you can certainly create
> > > two cores. That's a good strategy if there is
> > > little data shared between the two types of users.
> > > It's probably conceptually cleaner as well.
> > >
> > > Best,
> > > Erick
> > >
> > >
> > > On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan > > >wrote:
> > >
> > > > Hi,
> > > > Thank you very much for your reply.
> > > > Let me clearly explain my requirement.
> > > >
> > > > Its a kind of job site.
> > > >
> > > > I have 2 tables that need to be added in solr.
> > > >
> > > > 1) employee ( id, name, skills, location, .)
> > > > 2) job_postings ( id, name, job title, description, salary,
> > date_posted,
> > > > )
> > > >
> > > >
> > > > Here, there are 2 different kind of searches in my application.
> > > >
> > > > 1) employee search by employer
> > > > 2) job postings search by employee
> > > >
> > > > New row can be added at any time. Also the existing row can be
> altered
> > at
> > > > any time. For example, employee mobile number can be changed at any
> > time
> > > > and the solr data has to be updated.
> > > >
> > > > Here there are 2 different searches. So i need to maintain two
> > different
> > > > kind of indexes in solr.
> > > >
> > > > The java example says, to add a document, below is the sample code.
> My
> > > > doubt is, how can i add two different kind o
Re: Solr Java Client
Hi Erick,
Thanks a lot for your reply. I am still not clear.
I will have 2 different searches.
So, there will be 2 different kind of documents with different fields.
But the example below, gives me a impression that
SolrInputDocument doc = new SolrInputDocument();even there are 2 diff
searches, the data is getting stored in the same
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
If i use the same doc, how to diff data for 2 different searches.
Could you explain me clear on this??//
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 7:57 PM, Erick Erickson wrote:
> Listen to Wunder. You really have to switch
> gears and think in terms of searching rather
> than database tables.
>
> First, there's no requirement that all documents
> have the same fields. There's very little penalty
> for this.
>
> Second, you have to get over normalized data.
> It's unclear how many employees/records you have,
> but unless it's a really big set of records (i.e multiple
> tens of millions) you don't have to worry much
> about saving space.
>
> BTW, your java example works, but you do NOT
> want to commit after every document. Configure
> your auto soft and hard commits to do this on a
> time-based basis.
>
> So I'd recommend you approach it differently.
> Don't get stuck on your current data model. Instead
> define the searches you want to make and _then_
> decide the appropriate fields in the docs.
>
> But to your question, you can certainly create
> two cores. That's a good strategy if there is
> little data shared between the two types of users.
> It's probably conceptually cleaner as well.
>
> Best,
> Erick
>
>
> On Sun, Sep 15, 2013 at 8:18 AM, Baskar Sikkayan >wrote:
>
> > Hi,
> > Thank you very much for your reply.
> > Let me clearly explain my requirement.
> >
> > Its a kind of job site.
> >
> > I have 2 tables that need to be added in solr.
> >
> > 1) employee ( id, name, skills, location, .)
> > 2) job_postings ( id, name, job title, description, salary, date_posted,
> > )
> >
> >
> > Here, there are 2 different kind of searches in my application.
> >
> > 1) employee search by employer
> > 2) job postings search by employee
> >
> > New row can be added at any time. Also the existing row can be altered at
> > any time. For example, employee mobile number can be changed at any time
> > and the solr data has to be updated.
> >
> > Here there are 2 different searches. So i need to maintain two different
> > kind of indexes in solr.
> >
> > The java example says, to add a document, below is the sample code. My
> > doubt is, how can i add two different kind of search data here.
> > Should i create a 2 different core for this 2 different searches?
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> > server.commit();
> >
> > Thanks,
> > Baskar.S
> >
> >
> > On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood > >wrote:
> >
> > > Solr does not have tables, and you can't add an index.
> > >
> > > Solr's data model is flat, like a single table with lots of columns.
> > Think
> > > about creating a view where each row has all the information for one
> > search
> > > result. It includes everything that is searched (indexed=true in the
> > > schema) and returned (stored=true in the schema).
> > >
> > > wunder
> > >
> > > On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
> > >
> > > > Question is not clear to me. Please be more elaborative in your
> query.
> > > Why do u want to store index to DB tables?
> > > >
> > > > Rgds
> > > > AJ
> > > >
> > > > On 15-Sep-2013, at 7:20, Baskar Sikkayan
> wrote:
> > > >
> > > >> How to add index to 3 diff tables from java ...
> > > >>
> > > >>
> > > >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha
> > wrote:
> > > >>
> > > >>> Add a field called "so
Re: Solr Java Client
Hi,
Thank you very much for your reply.
Let me clearly explain my requirement.
Its a kind of job site.
I have 2 tables that need to be added in solr.
1) employee ( id, name, skills, location, .)
2) job_postings ( id, name, job title, description, salary, date_posted,
)
Here, there are 2 different kind of searches in my application.
1) employee search by employer
2) job postings search by employee
New row can be added at any time. Also the existing row can be altered at
any time. For example, employee mobile number can be changed at any time
and the solr data has to be updated.
Here there are 2 different searches. So i need to maintain two different
kind of indexes in solr.
The java example says, to add a document, below is the sample code. My
doubt is, how can i add two different kind of search data here.
Should i create a 2 different core for this 2 different searches?
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit();
Thanks,
Baskar.S
On Sun, Sep 15, 2013 at 9:14 AM, Walter Underwood wrote:
> Solr does not have tables, and you can't add an index.
>
> Solr's data model is flat, like a single table with lots of columns. Think
> about creating a view where each row has all the information for one search
> result. It includes everything that is searched (indexed=true in the
> schema) and returned (stored=true in the schema).
>
> wunder
>
> On Sep 14, 2013, at 7:32 PM, Amit Jha wrote:
>
> > Question is not clear to me. Please be more elaborative in your query.
> Why do u want to store index to DB tables?
> >
> > Rgds
> > AJ
> >
> > On 15-Sep-2013, at 7:20, Baskar Sikkayan wrote:
> >
> >> How to add index to 3 diff tables from java ...
> >>
> >>
> >> On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha wrote:
> >>
> >>> Add a field called "source" in schema.xml and value would be your table
> >>> names.
> >>>
> >>>
> >>>
> >>> Rgds
> >>> AJ
> >>>
> >>> On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
> >>>
> >>>> Hi,
> >>>> I am new to Solr and trying to use Solr java client instead of using
> the
> >>>> Data handler.
> >>>> Is there any configuration i need to do for this?
> >>>>
> >>>> I got the following sample code.
> >>>>
> >>>> SolrInputDocument doc = new SolrInputDocument();
> >>>>
> >>>>doc.addField("cat", "book");
> >>>>doc.addField("id", "book-" + i);
> >>>>doc.addField("name", "The Legend of the Hobbit part " + i);
> >>>>server.add(doc);
> >>>>server.commit(); // periodically flush
> >>>>
> >>>> I am confused here. I am going to index 3 different tables for 3
> >>> different
> >>>> kind of searches. Here i dont have any option to differentiate 3 kind
> of
> >>>> indexes.
> >>>> Am i missing anything here. Could anyone please shed some light here?
> >>>>
> >>>> Thanks,
> >>>> Baskar.S
> >>>
>
>
>
>
>
>
Re: Solr Java Client
How to add index to 3 diff tables from java ...
On Sun, Sep 15, 2013 at 6:49 AM, Amit Jha wrote:
> Add a field called "source" in schema.xml and value would be your table
> names.
>
>
>
> Rgds
> AJ
>
> On 15-Sep-2013, at 5:38, Baskar Sikkayan wrote:
>
> > Hi,
> > I am new to Solr and trying to use Solr java client instead of using the
> > Data handler.
> > Is there any configuration i need to do for this?
> >
> > I got the following sample code.
> >
> > SolrInputDocument doc = new SolrInputDocument();
> >
> > doc.addField("cat", "book");
> > doc.addField("id", "book-" + i);
> > doc.addField("name", "The Legend of the Hobbit part " + i);
> > server.add(doc);
> > server.commit(); // periodically flush
> >
> > I am confused here. I am going to index 3 different tables for 3
> different
> > kind of searches. Here i dont have any option to differentiate 3 kind of
> > indexes.
> > Am i missing anything here. Could anyone please shed some light here?
> >
> > Thanks,
> > Baskar.S
>
Re: MySQL Data import handler
Hi, If i am supposed to go with Java client, should i still do any configurations in solrconfig.xml or schema.xml. Thanks, Baskar.S On Sat, Sep 14, 2013 at 8:46 PM, Gora Mohanty wrote: > On 14 September 2013 20:07, Baskar Sikkayan wrote: > > Hi Gora, > > Thanks a lot for your reply. > > My requirement is to combine 3 tables in mysql for search operation and > > planning to sync these 3 tables( not all the columns ) in Apache Solr. > > Whenever there is any change( adding a new row, deleting a row, modifying > > the column data( any column in the 3 tables ) ), the same has to updated > in > > solr. Guess, for this requirement, instead of going with delta-import, > > Apachae Solar java client will be of useful. > [...] > > Yes, if you are comfortable with programming in Java, > the Solr client would be a good alternative, though the > DataImportHandler can also do what you want. > > Regards, > Gora >
Solr Java Client
Hi,
I am new to Solr and trying to use Solr java client instead of using the
Data handler.
Is there any configuration i need to do for this?
I got the following sample code.
SolrInputDocument doc = new SolrInputDocument();
doc.addField("cat", "book");
doc.addField("id", "book-" + i);
doc.addField("name", "The Legend of the Hobbit part " + i);
server.add(doc);
server.commit(); // periodically flush
I am confused here. I am going to index 3 different tables for 3 different
kind of searches. Here i dont have any option to differentiate 3 kind of
indexes.
Am i missing anything here. Could anyone please shed some light here?
Thanks,
Baskar.S
Re: MySQL Data import handler
Hi Gora, Thanks a lot for your reply. My requirement is to combine 3 tables in mysql for search operation and planning to sync these 3 tables( not all the columns ) in Apache Solr. Whenever there is any change( adding a new row, deleting a row, modifying the column data( any column in the 3 tables ) ), the same has to updated in solr. Guess, for this requirement, instead of going with delta-import, Apachae Solar java client will be of useful. Could you please share your view? Thanks, Baskar.S On Sat, Sep 14, 2013 at 7:25 PM, Gora Mohanty wrote: > On 14 September 2013 18:46, Baskar Sikkayan wrote: > > Hi, > > I am new to solar and trying for MySQL data import handler. > > > > I have 3 tables in mysql. > [...] > > Your question is unclear, and you would probably benefit > by doing some basic homework on Solr and importing > data into it. Please start from the Solr Wiki: > http://wiki.apache.org/solr/ > > For indexing from a database, you can use the DataImportHandler: > http://wiki.apache.org/solr/DataImportHandler > http://wiki.apache.org/solr/DIHQuickStart > What you want should be pretty straightforward. After > setting up basic data import, please refer to delta-import > for adding new data to Solr: > http://wiki.apache.org/solr/DataImportHandler#Using_delta-import_command > > Regards, > Gora >
MySQL Data import handler
Hi, I am new to solar and trying for MySQL data import handler. I have 3 tables in mysql. 1) user 2) mast_data 3) child_data The child_data table has foreign key ref of master_data and user. In child_data table, i have the following columns. 1) date 2) fee 3) location 4) type 5) text 6) user_id 7) master_data_id In the user table, display name field might get changed at any time. In the master table certain fields must be indexed in solar. So combining these 3 tables and trying to index . this is for search page ... the data will get added to these tables at any time and also the fields might get changed at any time I am not sure, how to keep the solr updated whenever there is any change in the table ... Also, looking for deleting the index in solr based on the date field Please help me on this Thanks, Baskar.S
