Hi team,
RESOLVED the error by adding dependency "spatial4j-0.7.jar" in docker
container,
Log file @ <<my-server-name>>/my-server-name.log gave me the clue of
Spatial4J class was not found !
Regards,
Aj
On Fri, Jan 31, 2020 at 10:04 PM vas aj <vasudevan.a...@gmail.com> wrote:
> Hi team,
>
> I am trying to query lucene documents and then query by
> 1) Station name
> 2) Stations with Longitude and Latitude radius of 10 miles
>
> I created StationLuceneLongLatSerializer as
> public class StationLuceneLongLatSerializer implements
> LuceneSerializer<Station> {
> @Override
> public Collection<Document> toDocuments(LuceneIndex luceneIndex, Station
> station) {
> Document doc = new Document();
> // Index title of Station
> doc.add(new TextField("title", station.getTitle(), Field.Store.YES));
>
> Field[] fields =
> SpatialHelper.getIndexableFields(Station.getLongitude(),
> Station.getLatitude());
>
> for (Field field : fields) {
> doc.add(field);
> }
>
> return Collections.singleton(doc);
> }
> }
>
> and created lucene index as
> create lucene index --name=stationLuceneIndex --region=station
> --field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer
> --serializer=StationLuceneLongLatSerializer
>
> While I retrieve documents using below query :
> String[] queries = new String[] {"mong*"};
> String[] fields = new String[] {"title"};
>
> Query query1 = QueryParserUtil.parse(queries, fields, new
> EnglishAnalyzer());
> Query queryFinal =
> new BooleanQuery.Builder()
> .add(query1, BooleanClause.Occur.MUST)
> .build();
>
> String queryString = queryFinal.toString("title");
> System.out.println("query from util => " + queryString);
>
> LuceneQuery<Long, EbayBike> luceneQuery =
> luceneService
> .createLuceneQueryFactory()
> .setLimit(100)
> .create("ebayBikeLuceneIndex", "ukBikes", queryString, "title");
>
> Collection<EbayBike> ebayBikes = luceneQuery.findValues();
>
> I get ZERO rows, but when I create lucene index like below(serializer is
> removed...)
>
> create lucene index --name=stationLuceneIndex --region=station
> --field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer
> --serializer=StationLuceneLongLatSerializer
>
> I get 3 rows.
>
> Where have I gone wrong?
>
> Regards,
> Aj
>
> On Thu, Jan 30, 2020 at 7:42 PM vas aj <vasudevan.a...@gmail.com> wrote:
>
>> Hi Dan,
>>
>> Thanks for the input. Serialization errors are now gone.
>>
>> However, I get this exception now :
>>
>> org.apache.geode.cache.client.ServerOperationException: remote server on
>> UNKNOWN(6217:loner):55000:e60cf1f7: While performing a remote
>> executeRegionFunction
>> at
>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp$ExecuteRegionFunctionOpImpl.processResponse(ExecuteRegionFunctionOp.java:442)
>> at
>> org.apache.geode.cache.client.internal.AbstractOp.processResponse(AbstractOp.java:222)
>> at
>> org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:195)
>> at
>> org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:382)
>> at
>> org.apache.geode.cache.client.internal.AbstractOpWithTimeout.attempt(AbstractOpWithTimeout.java:45)
>> at
>> org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:263)
>> at
>> org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:353)
>> at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:750)
>> at
>> org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:138)
>> at
>> org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:785)
>> at
>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp.execute(ExecuteRegionFunctionOp.java:92)
>> at
>> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:687)
>> at
>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200)
>> at
>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154)
>> at
>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379)
>> at
>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121)
>> at
>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99)
>> at
>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85)
>> at
>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78)
>> at com.lbc.station.service.controller.*StationController*
>> .testStationsOfInterest(StationController.java:65)
>> at
>> com.lbc.station.service.controller.$StationControllerDefinition$$exec1.invokeInternal(Unknown
>> Source)
>> . . .
>> . . .
>> . . .
>> *Caused by: java.lang.ClassNotFoundException:
>> com.lbc.station.service.controller.StationController*
>> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
>> at java.lang.Class.forName0(Native Method)
>> at java.lang.Class.forName(Class.java:348)
>> at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686)
>> at
>> org.apache.geode.internal.InternalDataSerializer$DSObjectInputStream.resolveClass(InternalDataSerializer.java:3668)
>> at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868)
>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751)
>> at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1716)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1556)
>> at
>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287)
>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211)
>> at
>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069)
>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573)
>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431)
>> at
>> org.apache.geode.internal.InternalDataSerializer.readSerializable(InternalDataSerializer.java:2846)
>> at
>> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2790)
>> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968)
>> at
>> org.apache.geode.cache.lucene.internal.distributed.LuceneFunctionContext.fromData(LuceneFunctionContext.java:106)
>> at
>> org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2372)
>> at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:997)
>> at
>> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2668)
>> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968)
>> at
>> org.apache.geode.internal.util.BlobHelper.deserializeBlob(BlobHelper.java:99)
>> at
>> org.apache.geode.internal.cache.tier.sockets.CacheServerHelper.deserialize(CacheServerHelper.java:73)
>> at
>> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:351)
>> at
>> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:359)
>> at
>> org.apache.geode.internal.cache.tier.sockets.command.ExecuteRegionFunction66.cmdExecute(ExecuteRegionFunction66.java:101)
>> at
>> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:183)
>> at
>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMessage(ServerConnection.java:848)
>> at
>> org.apache.geode.internal.cache.tier.sockets.OriginalServerConnection.doOneMessage(OriginalServerConnection.java:72)
>> at
>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1212)
>> at
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>> at
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>> at
>> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.lambda$initializeServerConnectionThreadPool$3(AcceptorImpl.java:666)
>> at
>> org.apache.geode.internal.logging.LoggingThreadFactory.lambda$newThread$0(LoggingThreadFactory.java:121)
>> ... 1 common frames omitted
>>
>>
>>
>> Regards,
>> Ajay Vasudevan
>>
>> On Thu, Jan 30, 2020 at 1:03 AM Dan Smith <dsm...@pivotal.io> wrote:
>>
>>> Hi vas,
>>>
>>> The problem is that your entire query needs to be inside the lamba, like
>>> below:
>>>
>>> index -> {
>>> return new BooleanQuery.Builder()
>>> //Query object must be constructed here, inside
>>> the lambda
>>> Query queryToExecute = new
>>> StandardQueryParser().parse(queryString, "stationNameIndex");
>>> .add(queryToExecute, BooleanClause.Occur.MUST)
>>> .build();
>>> });
>>>
>>> The reason you have to do this is that this entire object is serialized
>>> and evaluated *on the server*. Unfortunately, lucene queries are not
>>> serializable. So in the code you send, the query is constructed on the
>>> client and then we try to serialize it and send it to the server, which
>>> doesn't work.
>>>
>>> //Query is created on the client
>>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>>> "stationNameIndex");
>>>
>>> LuceneQuery<Long, Station> luceneQuery =
>>> luceneService
>>> .createLuceneQueryFactory()
>>> .setLimit(100)
>>> .create(
>>> "stationNameIndex",
>>> "stations-region",
>>> index -> {
>>> return new BooleanQuery.Builder()
>>> //Using a Query object here that was
>>> constructed outside this block means that it must be serialized to the
>>> server. This won't work.
>>> .add(queryToExecute, BooleanClause.Occur.MUST)
>>> .build();
>>> });
>>>
>>> On Wed, Jan 29, 2020 at 3:23 PM vas aj <vasudevan.a...@gmail.com> wrote:
>>>
>>>> Changed Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>> "stationNameIndex");
>>>> to Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>> "stationName");
>>>>
>>>> But still, get the error
>>>>
>>>> Caused by: java.io.NotSerializableException:
>>>> org.apache.lucene.search.PrefixQuery
>>>> . . .
>>>>
>>>> Caused by: org.apache.geode.SerializationException: failed serializing
>>>> object
>>>> . . .
>>>>
>>>> Caused by: java.io.NotSerializableException:
>>>> org.apache.lucene.search.PrefixQuery
>>>>
>>>> Help is needed, please
>>>>
>>>> On Wed, Jan 29, 2020 at 11:13 PM vas aj <vasudevan.a...@gmail.com>
>>>> wrote:
>>>>
>>>>> Rechecked my lucene index on Geode and they are as follows:
>>>>>
>>>>> create lucene index --name=stationNameIndex --region=stations-region
>>>>> --field=stationName,longitude,latitude
>>>>> --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer,DEFAULT,DEFAULT
>>>>>
>>>>> and in java code
>>>>>
>>>>> queryString = "stationName: piccadilly*";
>>>>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>>> "stationNameIndex");
>>>>>
>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>> luceneService
>>>>> .createLuceneQueryFactory()
>>>>> .setLimit(100)
>>>>> .create(
>>>>> "stationNameIndex",
>>>>> "stations-region",
>>>>> index -> {
>>>>> return new BooleanQuery.Builder()
>>>>> .add(queryToExecute, BooleanClause.Occur.MUST)
>>>>> .build();
>>>>> });
>>>>>
>>>>> luceneQuery.findValues() throws Serialization exception but when I use
>>>>> the
>>>>>
>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>> luceneService
>>>>> .createLuceneQueryFactory()
>>>>> .setLimit(100)
>>>>> .create(
>>>>> "stationNameIndex",
>>>>> "stations-region",
>>>>> queryString,
>>>>> "stationName"
>>>>> });
>>>>>
>>>>> It works.
>>>>>
>>>>> What has gone wrong?
>>>>>
>>>>> On Wed, Jan 29, 2020 at 9:19 PM Xiaojian Zhou <gz...@pivotal.io>
>>>>> wrote:
>>>>>
>>>>>> You should use the stationName not station if your object is indexed
>>>>>> on the field, i.e. stationName.
>>>>>>
>>>>>> Even class Station implements DataSerializable, lucene index are
>>>>>> still only index on the object's fields, such as stationName.
>>>>>>
>>>>>> On Wed, Jan 29, 2020 at 1:04 PM vas aj <vasudevan.a...@gmail.com>
>>>>>> wrote:
>>>>>>
>>>>>>> Sorry
>>>>>>> queryString = "station: piccadilly*" to get the mentioned error.
>>>>>>>
>>>>>>> On Wed, Jan 29, 2020 at 9:01 PM vas aj <vasudevan.a...@gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Hi team,
>>>>>>>>
>>>>>>>> The below code :
>>>>>>>>
>>>>>>>> queryString = "piccadilly*";
>>>>>>>> Query queryToExecute = new StandardQueryParser().parse(queryString,
>>>>>>>> "stationNameIndex");
>>>>>>>>
>>>>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>>>>> luceneService
>>>>>>>> .createLuceneQueryFactory()
>>>>>>>> .setLimit(100)
>>>>>>>> .create(
>>>>>>>> "stationNameIndex",
>>>>>>>> "stations-region",
>>>>>>>> index -> {
>>>>>>>> return new BooleanQuery.Builder()
>>>>>>>> .add(queryToExecute,
>>>>>>>> BooleanClause.Occur.MUST)
>>>>>>>> .build();
>>>>>>>> });
>>>>>>>> Collection<Station> stations = luceneQuery.findValues();
>>>>>>>>
>>>>>>>> *throws the below error : *
>>>>>>>>
>>>>>>>> Error while querying => {}
>>>>>>>> org.apache.geode.SerializationException: failed serializing object
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.serializeAndAddPart(Message.java:408)
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:350)
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:329)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp$ExecuteRegionFunctionSingleHopOpImpl.<init>(ExecuteRegionFunctionSingleHopOp.java:180)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.lambda$executeFunction$1(ServerRegionProxy.java:699)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.constructAndGetExecuteFunctionTasks(ExecuteRegionFunctionSingleHopOp.java:131)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.execute(ExecuteRegionFunctionSingleHopOp.java:84)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:701)
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200)
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154)
>>>>>>>> at
>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85)
>>>>>>>> at
>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78)
>>>>>>>>
>>>>>>>> whereas
>>>>>>>>
>>>>>>>> LuceneQuery<Long, Station> luceneQuery =
>>>>>>>> luceneService
>>>>>>>> .createLuceneQueryFactory()
>>>>>>>> .setLimit(100)
>>>>>>>> .create(
>>>>>>>> "stationNameIndex",
>>>>>>>> "stations-region",
>>>>>>>> queryString,
>>>>>>>> "stationName"
>>>>>>>> });
>>>>>>>>
>>>>>>>> works perfectly & Station
>>>>>>>> implements org.apache.geode.DataSerializable.
>>>>>>>>
>>>>>>>> What am I missing?.
>>>>>>>> Please help.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Aj
>>>>>>>>
>>>>>>>> On Tue, Jan 28, 2020 at 11:31 PM Xiaojian Zhou <gz...@pivotal.io>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> Maybe you should change one or both of the "MUST" to "SHOULD". And
>>>>>>>>> it's better to switch the order of the 2 subqueries.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Tue, Jan 28, 2020 at 2:23 PM vas aj <vasudevan.a...@gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> Thanks Dan for the quick help.
>>>>>>>>>> However the query failed.
>>>>>>>>>>
>>>>>>>>>> Working on the same.
>>>>>>>>>>
>>>>>>>>>> On Tue, 28 Jan 2020, 19:55 Dan Smith, <dsm...@pivotal.io> wrote:
>>>>>>>>>>
>>>>>>>>>>> I think you could probably use a BooleanQuery to do this.
>>>>>>>>>>> Something like this, although I haven't tested it. Your
>>>>>>>>>>> LuceneQuery1 just
>>>>>>>>>>> results in a call to StandardQueryParser.parse, so you can combine
>>>>>>>>>>> that
>>>>>>>>>>> with your spatial query:
>>>>>>>>>>>
>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 =
>>>>>>>>>>> luceneService
>>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>>> .create("stationNameIndex", "stations-region", index ->
>>>>>>>>>>> {
>>>>>>>>>>> BooleanQuery query = new BooleanQuery();
>>>>>>>>>>> query.add(new
>>>>>>>>>>> StandardQueryParser().parse(queryString, name),
>>>>>>>>>>> BooleanQuery.Occur.MUST);
>>>>>>>>>>> query.add(SpatialHelper.findWithin(-122.8515139,
>>>>>>>>>>> 45.5099231, 0.5), , BooleanQuery.Occur.MUST);
>>>>>>>>>>> return query;
>>>>>>>>>>> }
>>>>>>>>>>>
>>>>>>>>>>> On Tue, Jan 28, 2020 at 11:36 AM vas aj <
>>>>>>>>>>> vasudevan.a...@gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi team,
>>>>>>>>>>>>
>>>>>>>>>>>> I want to execute a *LuceneQuery* that searches both a
>>>>>>>>>>>> queryString(E.g. "station:Piccadilly*") as well as nearest station
>>>>>>>>>>>> within
>>>>>>>>>>>> .5 miles.(E.g. SpatialHelper.findWithin(lat, long, 0.5))
>>>>>>>>>>>>
>>>>>>>>>>>> In other words, is there any means of combining *luceneQuery1*
>>>>>>>>>>>> & *luceneQuery2* and executing them as
>>>>>>>>>>>> Collection<Station> nearestStations = *luceneQuery*
>>>>>>>>>>>> .findValues();
>>>>>>>>>>>>
>>>>>>>>>>>> where luceneQuery = luceneQuery1 + luceneQuery2
>>>>>>>>>>>>
>>>>>>>>>>>> *LuceneQuery1*
>>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery1 =
>>>>>>>>>>>> luceneService
>>>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>>>> .setLimit(100)
>>>>>>>>>>>> .create("stationNameIndex", "stations-region",
>>>>>>>>>>>> queryString, "name");
>>>>>>>>>>>>
>>>>>>>>>>>> *LuceneQuery2*
>>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 =
>>>>>>>>>>>> luceneService
>>>>>>>>>>>> .createLuceneQueryFactory()
>>>>>>>>>>>> .create("stationNameIndex", "stations-region", index
>>>>>>>>>>>> -> SpatialHelper.findWithin(-122.8515139, 45.5099231, 0.5);
>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Aj
>>>>>>>>>>>>
>>>>>>>>>>>
