Hi team,
I am trying to query lucene documents and then query by
1) Station name
2) Stations with Longitude and Latitude radius of 10 miles
I created StationLuceneLongLatSerializer as
public class StationLuceneLongLatSerializer implements
LuceneSerializer<Station> {
@Override
public Collection<Document> toDocuments(LuceneIndex luceneIndex, Station
station) {
Document doc = new Document();
// Index title of Station
doc.add(new TextField("title", station.getTitle(), Field.Store.YES));
Field[] fields =
SpatialHelper.getIndexableFields(Station.getLongitude(),
Station.getLatitude());
for (Field field : fields) {
doc.add(field);
}
return Collections.singleton(doc);
}
}
and created lucene index as
create lucene index --name=stationLuceneIndex --region=station
--field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer
--serializer=StationLuceneLongLatSerializer
While I retrieve documents using below query :
String[] queries = new String[] {"mong*"};
String[] fields = new String[] {"title"};
Query query1 = QueryParserUtil.parse(queries, fields, new
EnglishAnalyzer());
Query queryFinal =
new BooleanQuery.Builder()
.add(query1, BooleanClause.Occur.MUST)
.build();
String queryString = queryFinal.toString("title");
System.out.println("query from util => " + queryString);
LuceneQuery<Long, EbayBike> luceneQuery =
luceneService
.createLuceneQueryFactory()
.setLimit(100)
.create("ebayBikeLuceneIndex", "ukBikes", queryString, "title");
Collection<EbayBike> ebayBikes = luceneQuery.findValues();
I get ZERO rows, but when I create lucene index like below(serializer is
removed...)
create lucene index --name=stationLuceneIndex --region=station
--field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer
--serializer=StationLuceneLongLatSerializer
I get 3 rows.
Where have I gone wrong?
Regards,
Aj
On Thu, Jan 30, 2020 at 7:42 PM vas aj <vasudevan.a...@gmail.com> wrote:
> Hi Dan,
>
> Thanks for the input. Serialization errors are now gone.
>
> However, I get this exception now :
>
> org.apache.geode.cache.client.ServerOperationException: remote server on
> UNKNOWN(6217:loner):55000:e60cf1f7: While performing a remote
> executeRegionFunction
> at
> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp$ExecuteRegionFunctionOpImpl.processResponse(ExecuteRegionFunctionOp.java:442)
> at
> org.apache.geode.cache.client.internal.AbstractOp.processResponse(AbstractOp.java:222)
> at
> org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:195)
> at
> org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:382)
> at
> org.apache.geode.cache.client.internal.AbstractOpWithTimeout.attempt(AbstractOpWithTimeout.java:45)
> at
> org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:263)
> at
> org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:353)
> at
> org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:750)
> at
> org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:138)
> at
> org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:785)
> at
> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp.execute(ExecuteRegionFunctionOp.java:92)
> at
> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:687)
> at
> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200)
> at
> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154)
> at
> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379)
> at
> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121)
> at
> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99)
> at
> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85)
> at
> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78)
> at com.lbc.station.service.controller.*StationController*
> .testStationsOfInterest(StationController.java:65)
> at
> com.lbc.station.service.controller.$StationControllerDefinition$$exec1.invokeInternal(Unknown
> Source)
> . . .
> . . .
> . . .
> *Caused by: java.lang.ClassNotFoundException:
> com.lbc.station.service.controller.StationController*
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> at java.lang.Class.forName0(Native Method)
> at java.lang.Class.forName(Class.java:348)
> at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686)
> at
> org.apache.geode.internal.InternalDataSerializer$DSObjectInputStream.resolveClass(InternalDataSerializer.java:3668)
> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
> at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1716)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1556)
> at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
> at
> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
> at
> org.apache.geode.internal.InternalDataSerializer.readSerializable(InternalDataSerializer.java:2846)
> at
> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2790)
> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968)
> at
> org.apache.geode.cache.lucene.internal.distributed.LuceneFunctionContext.fromData(LuceneFunctionContext.java:106)
> at
> org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2372)
> at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:997)
> at
> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2668)
> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968)
> at
> org.apache.geode.internal.util.BlobHelper.deserializeBlob(BlobHelper.java:99)
> at
> org.apache.geode.internal.cache.tier.sockets.CacheServerHelper.deserialize(CacheServerHelper.java:73)
> at
> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:351)
> at
> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:359)
> at
> org.apache.geode.internal.cache.tier.sockets.command.ExecuteRegionFunction66.cmdExecute(ExecuteRegionFunction66.java:101)
> at
> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:183)
> at
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMessage(ServerConnection.java:848)
> at
> org.apache.geode.internal.cache.tier.sockets.OriginalServerConnection.doOneMessage(OriginalServerConnection.java:72)
> at
> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1212)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at
> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.lambda$initializeServerConnectionThreadPool$3(AcceptorImpl.java:666)
> at
> org.apache.geode.internal.logging.LoggingThreadFactory.lambda$newThread$0(LoggingThreadFactory.java:121)
> ... 1 common frames omitted
>
>
>
> Regards,
> Ajay Vasudevan
>
> On Thu, Jan 30, 2020 at 1:03 AM Dan Smith <dsm...@pivotal.io> wrote:
>
>> Hi vas,
>>
>> The problem is that your entire query needs to be inside the lamba, like
>> below:
>>
>> index -> {
>> return new BooleanQuery.Builder()
>> //Query object must be constructed here, inside
>> the lambda
>> Query queryToExecute = new
>> StandardQueryParser().parse(queryString, "stationNameIndex");
>> .add(queryToExecute, BooleanClause.Occur.MUST)
>> .build();
>> });
>>
>> The reason you have to do this is that this entire object is serialized
>> and evaluated *on the server*. Unfortunately, lucene queries are not
>> serializable. So in the code you send, the query is constructed on the
>> client and then we try to serialize it and send it to the server, which
>> doesn't work.
>>
>> //Query is created on the client
>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>> "stationNameIndex");
>>
>> LuceneQuery<Long, Station> luceneQuery =
>> luceneService
>> .createLuceneQueryFactory()
>> .setLimit(100)
>> .create(
>> "stationNameIndex",
>> "stations-region",
>> index -> {
>> return new BooleanQuery.Builder()
>> //Using a Query object here that was constructed
>> outside this block means that it must be serialized to the server. This
>> won't work.
>> .add(queryToExecute, BooleanClause.Occur.MUST)
>> .build();
>> });
>>
>> On Wed, Jan 29, 2020 at 3:23 PM vas aj <vasudevan.a...@gmail.com> wrote:
>>
>>> Changed Query queryToExecute = new StandardQueryParser().parse(queryString,
>>> "stationNameIndex");
>>> to Query queryToExecute = new StandardQueryParser().parse(queryString,
>>> "stationName");
>>>
>>> But still, get the error
>>>
>>> Caused by: java.io.NotSerializableException:
>>> org.apache.lucene.search.PrefixQuery
>>> . . .
>>>
>>> Caused by: org.apache.geode.SerializationException: failed serializing
>>> object
>>> . . .
>>>
>>> Caused by: java.io.NotSerializableException:
>>> org.apache.lucene.search.PrefixQuery
>>>
>>> Help is needed, please
>>>
>>> On Wed, Jan 29, 2020 at 11:13 PM vas aj <vasudevan.a...@gmail.com>
>>> wrote:
>>>
>>>> Rechecked my lucene index on Geode and they are as follows:
>>>>
>>>> create lucene index --name=stationNameIndex --region=stations-region
>>>> --field=stationName,longitude,latitude
>>>> --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer,DEFAULT,DEFAULT
>>>>
>>>> and in java code
>>>>
>>>> queryString = "stationName: piccadilly*";
>>>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>> "stationNameIndex");
>>>>
>>>> LuceneQuery<Long, Station> luceneQuery =
>>>> luceneService
>>>> .createLuceneQueryFactory()
>>>> .setLimit(100)
>>>> .create(
>>>> "stationNameIndex",
>>>> "stations-region",
>>>> index -> {
>>>> return new BooleanQuery.Builder()
>>>> .add(queryToExecute, BooleanClause.Occur.MUST)
>>>> .build();
>>>> });
>>>>
>>>> luceneQuery.findValues() throws Serialization exception but when I use
>>>> the
>>>>
>>>> LuceneQuery<Long, Station> luceneQuery =
>>>> luceneService
>>>> .createLuceneQueryFactory()
>>>> .setLimit(100)
>>>> .create(
>>>> "stationNameIndex",
>>>> "stations-region",
>>>> queryString,
>>>> "stationName"
>>>> });
>>>>
>>>> It works.
>>>>
>>>> What has gone wrong?
>>>>
>>>> On Wed, Jan 29, 2020 at 9:19 PM Xiaojian Zhou <gz...@pivotal.io> wrote:
>>>>
>>>>> You should use the stationName not station if your object is indexed
>>>>> on the field, i.e. stationName.
>>>>>
>>>>> Even class Station implements DataSerializable, lucene index are still
>>>>> only index on the object's fields, such as stationName.
>>>>>
>>>>> On Wed, Jan 29, 2020 at 1:04 PM vas aj <vasudevan.a...@gmail.com>
>>>>> wrote:
>>>>>
>>>>>> Sorry
>>>>>> queryString = "station: piccadilly*" to get the mentioned error.
>>>>>>
>>>>>> On Wed, Jan 29, 2020 at 9:01 PM vas aj <vasudevan.a...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Hi team,
>>>>>>>
>>>>>>> The below code :
>>>>>>>
>>>>>>> queryString = "piccadilly*";
>>>>>>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>>>>> "stationNameIndex");
>>>>>>>
>>>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>>>> luceneService
>>>>>>> .createLuceneQueryFactory()
>>>>>>> .setLimit(100)
>>>>>>> .create(
>>>>>>> "stationNameIndex",
>>>>>>> "stations-region",
>>>>>>> index -> {
>>>>>>> return new BooleanQuery.Builder()
>>>>>>> .add(queryToExecute,
>>>>>>> BooleanClause.Occur.MUST)
>>>>>>> .build();
>>>>>>> });
>>>>>>> Collection<Station> stations = luceneQuery.findValues();
>>>>>>>
>>>>>>> *throws the below error : *
>>>>>>>
>>>>>>> Error while querying => {}
>>>>>>> org.apache.geode.SerializationException: failed serializing object
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.serializeAndAddPart(Message.java:408)
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:350)
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:329)
>>>>>>> at
>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp$ExecuteRegionFunctionSingleHopOpImpl.<init>(ExecuteRegionFunctionSingleHopOp.java:180)
>>>>>>> at
>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.lambda$executeFunction$1(ServerRegionProxy.java:699)
>>>>>>> at
>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.constructAndGetExecuteFunctionTasks(ExecuteRegionFunctionSingleHopOp.java:131)
>>>>>>> at
>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.execute(ExecuteRegionFunctionSingleHopOp.java:84)
>>>>>>> at
>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:701)
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200)
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154)
>>>>>>> at
>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379)
>>>>>>> at
>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121)
>>>>>>> at
>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99)
>>>>>>> at
>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85)
>>>>>>> at
>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78)
>>>>>>>
>>>>>>> whereas
>>>>>>>
>>>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>>>> luceneService
>>>>>>> .createLuceneQueryFactory()
>>>>>>> .setLimit(100)
>>>>>>> .create(
>>>>>>> "stationNameIndex",
>>>>>>> "stations-region",
>>>>>>> queryString,
>>>>>>> "stationName"
>>>>>>> });
>>>>>>>
>>>>>>> works perfectly & Station
>>>>>>> implements org.apache.geode.DataSerializable.
>>>>>>>
>>>>>>> What am I missing?.
>>>>>>> Please help.
>>>>>>>
>>>>>>> Regards,
>>>>>>> Aj
>>>>>>>
>>>>>>> On Tue, Jan 28, 2020 at 11:31 PM Xiaojian Zhou <gz...@pivotal.io>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Maybe you should change one or both of the "MUST" to "SHOULD". And
>>>>>>>> it's better to switch the order of the 2 subqueries.
>>>>>>>>
>>>>>>>>
>>>>>>>> On Tue, Jan 28, 2020 at 2:23 PM vas aj <vasudevan.a...@gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Thanks Dan for the quick help.
>>>>>>>>> However the query failed.
>>>>>>>>>
>>>>>>>>> Working on the same.
>>>>>>>>>
>>>>>>>>> On Tue, 28 Jan 2020, 19:55 Dan Smith, <dsm...@pivotal.io> wrote:
>>>>>>>>>
>>>>>>>>>> I think you could probably use a BooleanQuery to do this.
>>>>>>>>>> Something like this, although I haven't tested it. Your LuceneQuery1
>>>>>>>>>> just
>>>>>>>>>> results in a call to StandardQueryParser.parse, so you can combine
>>>>>>>>>> that
>>>>>>>>>> with your spatial query:
>>>>>>>>>>
>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 =
>>>>>>>>>> luceneService
>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>> .create("stationNameIndex", "stations-region", index -> {
>>>>>>>>>> BooleanQuery query = new BooleanQuery();
>>>>>>>>>> query.add(new
>>>>>>>>>> StandardQueryParser().parse(queryString, name),
>>>>>>>>>> BooleanQuery.Occur.MUST);
>>>>>>>>>> query.add(SpatialHelper.findWithin(-122.8515139,
>>>>>>>>>> 45.5099231, 0.5), , BooleanQuery.Occur.MUST);
>>>>>>>>>> return query;
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> On Tue, Jan 28, 2020 at 11:36 AM vas aj <vasudevan.a...@gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi team,
>>>>>>>>>>>
>>>>>>>>>>> I want to execute a *LuceneQuery* that searches both a
>>>>>>>>>>> queryString(E.g. "station:Piccadilly*") as well as nearest station
>>>>>>>>>>> within
>>>>>>>>>>> .5 miles.(E.g. SpatialHelper.findWithin(lat, long, 0.5))
>>>>>>>>>>>
>>>>>>>>>>> In other words, is there any means of combining *luceneQuery1*
>>>>>>>>>>> & *luceneQuery2* and executing them as
>>>>>>>>>>> Collection<Station> nearestStations = *luceneQuery*
>>>>>>>>>>> .findValues();
>>>>>>>>>>>
>>>>>>>>>>> where luceneQuery = luceneQuery1 + luceneQuery2
>>>>>>>>>>>
>>>>>>>>>>> *LuceneQuery1*
>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery1 =
>>>>>>>>>>> luceneService
>>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>>> .setLimit(100)
>>>>>>>>>>> .create("stationNameIndex", "stations-region",
>>>>>>>>>>> queryString, "name");
>>>>>>>>>>>
>>>>>>>>>>> *LuceneQuery2*
>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 =
>>>>>>>>>>> luceneService
>>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>>> .create("stationNameIndex", "stations-region", index ->
>>>>>>>>>>> SpatialHelper.findWithin(-122.8515139, 45.5099231, 0.5);
>>>>>>>>>>>
>>>>>>>>>>> Regards
>>>>>>>>>>> Aj
>>>>>>>>>>>
>>>>>>>>>>
