Re: lucene query (sql kind)
Ross - I'm really perplexed by your message. You create HTML from a database so that you can index it with Lucene, yet wish you could simply index the data in your database tied to a primary key directly, right? Well, you're in luck - you already can do this! What are you using for indexing? It sounds like you borrowed the Lucene demo and have just run with that directly. Erik On Jan 28, 2005, at 11:02 AM, Ross Rankin wrote: I agree. My site is all dynamic pages created from the database. Right now, I have to have a process create dummy pages, index them with Lucene, then translate the Lucene results into meaningful links. It actually works better than it sounds, however it could be easier. If I could just give Lucene a query result (i.e. a list of rows) and then have Lucene send me back say the primary key of the rows that match and the other Lucene goodness: ranking, number of hits, etc. Could be pretty powerful and simplify the deployment for database driven applications. [Note: this opinion and $3.00 will get you a coffee at Starbucks] Ross -Original Message- From: PA [mailto:[EMAIL PROTECTED] Sent: Friday, January 28, 2005 6:44 AM To: Lucene Users List Subject: Re: lucene query (sql kind) On Jan 28, 2005, at 12:40, sunil goyal wrote: I want to run dynamic queries against the lucene index. Is there any native syntax available for Lucene so that I can query, by first generating the query in say an XML or SQL like format (cache this query) and then use this query over lucene index. Talking of which, did anyone contemplated the possibility of a JDBC adaptor of sort for Lucene? Cheers -- PA, Onnay Equitursay http://alt.textdrive.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
RE: lucene query (sql kind)
I agree. My site is all dynamic pages created from the database. Right now, I have to have a process create dummy pages, index them with Lucene, then translate the Lucene results into meaningful links. It actually works better than it sounds, however it could be easier. If I could just give Lucene a query result (i.e. a list of rows) and then have Lucene send me back say the primary key of the rows that match and the other Lucene goodness: ranking, number of hits, etc. Could be pretty powerful and simplify the deployment for database driven applications. [Note: this opinion and $3.00 will get you a coffee at Starbucks] Ross -Original Message- From: PA [mailto:[EMAIL PROTECTED] Sent: Friday, January 28, 2005 6:44 AM To: Lucene Users List Subject: Re: lucene query (sql kind) On Jan 28, 2005, at 12:40, sunil goyal wrote: > I want to run dynamic queries against the lucene index. Is there any > native syntax available for Lucene so that I can query, by first > generating the query in say an XML or SQL like format (cache this > query) and then use this query over lucene index. Talking of which, did anyone contemplated the possibility of a JDBC adaptor of sort for Lucene? Cheers -- PA, Onnay Equitursay http://alt.textdrive.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
I like your idea and think you are quite right. I see quite some
people are using lucene to the extreme such that relational database
functionalities are replaced by lucene.
However, storing everything in lucene and use it as a relational type
of database will be kind of re-inventing the wheel. For example,
sorting on the date field, and any other range query.
I think the better way is to look at ways to integrate lucene tightly
into a java relational database, such as HSQL, McKoi or Derby.
In particular, that integration would make it possible for queries
like "contains(...)", which is included in MySQL full text search
syntax and other major relational db vendors.
I would like to contribute any possible help I could for that to happen.
Thanks,
Jian
On Fri, 28 Jan 2005 13:01:40 + (GMT), mark harwood
<[EMAIL PROTECTED]> wrote:
> I've added some user-defined lucene functions to
> HSQLDB and I've been able to run queries like the
> following one:
>
> select top 10 lucene_highlight(adText) from ads where
> pricePounds <200 and lucene_query('bass guitar
> drums',id)>0 order by lucene_score(id) DESC
>
> I've had similar success with Derby (Cloudscape).
> This approach has some appeal and I've been able to
> use the same class as a UDF in both databases but it
> does have issues: it looks like this UDF based
> integration won't scale. The above query took 80
> milliseconds using 10,000 records. Another
> index/database with 50,000 records was taking a matter
> of seconds. I think a scalable integration is likely
> to require modification of the core RDBMS code.
>
> I think it is worth considering developing such a
> tight RDBMS integration if you consider the issues
> commonly associated with using Lucene:
> 1) Sorting on float/date fields and associated memory
> consumption
> 2) Representing numbers/dates in Lucene (eg having to
> pad with sufficent leading zeros and add to index's
> list of terms)
> 3) Retrieving only certain stored fields from a
> document (all storage can be done in db)
> 4) Issues to do with updating volatile data eg price
> data used in sorts
> 5) Manually coding joins with RDBMS content as custom
> filters
> 6) Too-many terms exceptions produced by range queries
> 7) Grouping results eg by website
> 8) Boosting docs based on stored content eg date
>
> I'm not saying there aren't answers to the above using
> Lucene. However,I do wonder if these can be addressed
> more effectively in a project which seeks tighter
> integration with an RDBMS and leveraging its
> capabilities.
>
> Any one else been down this route?
>
>
> ___
> ALL-NEW Yahoo! Messenger - all new features - even more fun!
> http://uk.messenger.yahoo.com
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
I've added some user-defined lucene functions to
HSQLDB and I've been able to run queries like the
following one:
select top 10 lucene_highlight(adText) from ads where
pricePounds <200 and lucene_query('bass guitar
drums',id)>0 order by lucene_score(id) DESC
I've had similar success with Derby (Cloudscape).
This approach has some appeal and I've been able to
use the same class as a UDF in both databases but it
does have issues: it looks like this UDF based
integration won't scale. The above query took 80
milliseconds using 10,000 records. Another
index/database with 50,000 records was taking a matter
of seconds. I think a scalable integration is likely
to require modification of the core RDBMS code.
I think it is worth considering developing such a
tight RDBMS integration if you consider the issues
commonly associated with using Lucene:
1) Sorting on float/date fields and associated memory
consumption
2) Representing numbers/dates in Lucene (eg having to
pad with sufficent leading zeros and add to index's
list of terms)
3) Retrieving only certain stored fields from a
document (all storage can be done in db)
4) Issues to do with updating volatile data eg price
data used in sorts
5) Manually coding joins with RDBMS content as custom
filters
6) Too-many terms exceptions produced by range queries
7) Grouping results eg by website
8) Boosting docs based on stored content eg date
I'm not saying there aren't answers to the above using
Lucene. However,I do wonder if these can be addressed
more effectively in a project which seeks tighter
integration with an RDBMS and leveraging its
capabilities.
Any one else been down this route?
___
ALL-NEW Yahoo! Messenger - all new features - even more fun!
http://uk.messenger.yahoo.com
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
Hello,
Thanks, It works fine.
> The field parameter simply defines the default field for all queries
> without an explicit field specification (:).
> Using 'field AND field' as default field does not make sense but does
> not hurt as long as the default field is not used.
> I'm not sure why you choose that.
I just thought that Query Parser needs to be specifies what it should
expect before hand. So did "field AND field". But I was wrong.
> Further name:\"john\" and name:john should be the same.
Just in case it's not "john" but "hello john" or some phrase.
Regards
Sunil
On Fri, 28 Jan 2005 13:26:26 +0100, Morus Walter <[EMAIL PROTECTED]> wrote:
> sunil goyal writes:
> >
> > I was just trying that...
> >
> > QueryParser qp = new QueryParser("field AND field", new StandardAnalyzer());
> > Query query = qp.parse("name:\"john\" AND age:[10 TO 16]");
> >
> > It works fine with this. Do I need to specify that QueryParser should
> > expect things in order
> > "field AND field". Or can I do without it?
> >
> The field parameter simply defines the default field for all queries
> without an explicit field specification (:).
> Using 'field AND field' as default field does not make sense but does
> not hurt as long as the default field is not used.
> I'm not sure why you choose that.
>
> Further name:\"john\" and name:john should be the same.
>
> HTH
> Morus
>
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
sunil goyal writes:
>
> I was just trying that...
>
> QueryParser qp = new QueryParser("field AND field", new StandardAnalyzer());
> Query query = qp.parse("name:\"john\" AND age:[10 TO 16]");
>
> It works fine with this. Do I need to specify that QueryParser should
> expect things in order
> "field AND field". Or can I do without it?
>
The field parameter simply defines the default field for all queries
without an explicit field specification (:).
Using 'field AND field' as default field does not make sense but does
not hurt as long as the default field is not used.
I'm not sure why you choose that.
Further name:\"john\" and name:john should be the same.
HTH
Morus
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
I've merged some different fields in one query, with the name of one of
these fields as the second parameter in the
static method, and it worked fine.
Also, you can do a little query parser, and build the queries with
BooleanQuery.
David
sunil goyal wrote:
Hello,
I was just trying that...
QueryParser qp = new QueryParser("field AND field", new StandardAnalyzer());
Query query = qp.parse("name:\"john\" AND age:[10 TO 16]");
It works fine with this. Do I need to specify that QueryParser should
expect things in order
"field AND field". Or can I do without it?
The static method of QueryParser.parse(String , String, Analyzer) -
expects the first string to be the query and second to be the field.
Thanks
Regards
Sunil
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
Hello,
I was just trying that...
QueryParser qp = new QueryParser("field AND field", new StandardAnalyzer());
Query query = qp.parse("name:\"john\" AND age:[10 TO 16]");
It works fine with this. Do I need to specify that QueryParser should
expect things in order
"field AND field". Or can I do without it?
The static method of QueryParser.parse(String , String, Analyzer) -
expects the first string to be the query and second to be the field.
Thanks
Regards
Sunil
On Fri, 28 Jan 2005 12:54:27 +0100, David Escuer
<[EMAIL PROTECTED]> wrote:
>
> Hello,
>To build queries, you can generate a query like "(text:house OR
> text:car) AND (keywords:building)", and then
>parse it with the QueryParser.parse method to get the Lucene query.
> Is not 100% sql-like syntax, but it's more clear
>than the lucene syntax.
>
> Hope it helps
>
> David
>
> sunil goyal wrote:
>
> >Hello all,
> >
> >I want to run dynamic queries against the lucene index. Is there any
> >native syntax available for Lucene so that I can query, by first
> >generating the query in say an XML or SQL like format (cache this
> >query) and then use this query over lucene index.
> >
> >
> >e.g. So a lucene query syntax in which I can define a query
> >(name="john" AND age <10) and then I can just use this query to
> >execute over Lucene index.
> >
> >
> >Regards
> >Sunil
> >
> >-
> >To unsubscribe, e-mail: [EMAIL PROTECTED]
> >For additional commands, e-mail: [EMAIL PROTECTED]
> >
> >
>
> -
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
Hello, To build queries, you can generate a query like "(text:house OR text:car) AND (keywords:building)", and then parse it with the QueryParser.parse method to get the Lucene query. Is not 100% sql-like syntax, but it's more clear than the lucene syntax. Hope it helps David sunil goyal wrote: Hello all, I want to run dynamic queries against the lucene index. Is there any native syntax available for Lucene so that I can query, by first generating the query in say an XML or SQL like format (cache this query) and then use this query over lucene index. e.g. So a lucene query syntax in which I can define a query (name="john" AND age <10) and then I can just use this query to execute over Lucene index. Regards Sunil - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
Re: lucene query (sql kind)
On Jan 28, 2005, at 12:40, sunil goyal wrote: I want to run dynamic queries against the lucene index. Is there any native syntax available for Lucene so that I can query, by first generating the query in say an XML or SQL like format (cache this query) and then use this query over lucene index. Talking of which, did anyone contemplated the possibility of a JDBC adaptor of sort for Lucene? Cheers -- PA, Onnay Equitursay http://alt.textdrive.com/ - To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
