Glad to hear that! -Dan
On Sun, Feb 2, 2020, 11:52 AM vas aj <vasudevan.a...@gmail.com> wrote: > Hi team, > > RESOLVED the error by adding dependency "spatial4j-0.7.jar" in docker > container, > > Log file @ <<my-server-name>>/my-server-name.log gave me the clue of > Spatial4J class was not found ! > > Regards, > Aj > > On Fri, Jan 31, 2020 at 10:04 PM vas aj <vasudevan.a...@gmail.com> wrote: > >> Hi team, >> >> I am trying to query lucene documents and then query by >> 1) Station name >> 2) Stations with Longitude and Latitude radius of 10 miles >> >> I created StationLuceneLongLatSerializer as >> public class StationLuceneLongLatSerializer implements >> LuceneSerializer<Station> { >> @Override >> public Collection<Document> toDocuments(LuceneIndex luceneIndex, >> Station station) { >> Document doc = new Document(); >> // Index title of Station >> doc.add(new TextField("title", station.getTitle(), Field.Store.YES)); >> >> Field[] fields = >> SpatialHelper.getIndexableFields(Station.getLongitude(), >> Station.getLatitude()); >> >> for (Field field : fields) { >> doc.add(field); >> } >> >> return Collections.singleton(doc); >> } >> } >> >> and created lucene index as >> create lucene index --name=stationLuceneIndex --region=station >> --field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer >> --serializer=StationLuceneLongLatSerializer >> >> While I retrieve documents using below query : >> String[] queries = new String[] {"mong*"}; >> String[] fields = new String[] {"title"}; >> >> Query query1 = QueryParserUtil.parse(queries, fields, new >> EnglishAnalyzer()); >> Query queryFinal = >> new BooleanQuery.Builder() >> .add(query1, BooleanClause.Occur.MUST) >> .build(); >> >> String queryString = queryFinal.toString("title"); >> System.out.println("query from util => " + queryString); >> >> LuceneQuery<Long, EbayBike> luceneQuery = >> luceneService >> .createLuceneQueryFactory() >> .setLimit(100) >> .create("ebayBikeLuceneIndex", "ukBikes", queryString, "title"); >> >> Collection<EbayBike> ebayBikes = luceneQuery.findValues(); >> >> I get ZERO rows, but when I create lucene index like below(serializer is >> removed...) >> >> create lucene index --name=stationLuceneIndex --region=station >> --field=title --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer >> --serializer=StationLuceneLongLatSerializer >> >> I get 3 rows. >> >> Where have I gone wrong? >> >> Regards, >> Aj >> >> On Thu, Jan 30, 2020 at 7:42 PM vas aj <vasudevan.a...@gmail.com> wrote: >> >>> Hi Dan, >>> >>> Thanks for the input. Serialization errors are now gone. >>> >>> However, I get this exception now : >>> >>> org.apache.geode.cache.client.ServerOperationException: remote server on >>> UNKNOWN(6217:loner):55000:e60cf1f7: While performing a remote >>> executeRegionFunction >>> at >>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp$ExecuteRegionFunctionOpImpl.processResponse(ExecuteRegionFunctionOp.java:442) >>> at >>> org.apache.geode.cache.client.internal.AbstractOp.processResponse(AbstractOp.java:222) >>> at >>> org.apache.geode.cache.client.internal.AbstractOp.attemptReadResponse(AbstractOp.java:195) >>> at >>> org.apache.geode.cache.client.internal.AbstractOp.attempt(AbstractOp.java:382) >>> at >>> org.apache.geode.cache.client.internal.AbstractOpWithTimeout.attempt(AbstractOpWithTimeout.java:45) >>> at >>> org.apache.geode.cache.client.internal.ConnectionImpl.execute(ConnectionImpl.java:263) >>> at >>> org.apache.geode.cache.client.internal.pooling.PooledConnection.execute(PooledConnection.java:353) >>> at >>> org.apache.geode.cache.client.internal.OpExecutorImpl.executeWithPossibleReAuthentication(OpExecutorImpl.java:750) >>> at >>> org.apache.geode.cache.client.internal.OpExecutorImpl.execute(OpExecutorImpl.java:138) >>> at >>> org.apache.geode.cache.client.internal.PoolImpl.execute(PoolImpl.java:785) >>> at >>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionOp.execute(ExecuteRegionFunctionOp.java:92) >>> at >>> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:687) >>> at >>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200) >>> at >>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154) >>> at >>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379) >>> at >>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121) >>> at >>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99) >>> at >>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85) >>> at >>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78) >>> at com.lbc.station.service.controller.*StationController* >>> .testStationsOfInterest(StationController.java:65) >>> at >>> com.lbc.station.service.controller.$StationControllerDefinition$$exec1.invokeInternal(Unknown >>> Source) >>> . . . >>> . . . >>> . . . >>> *Caused by: java.lang.ClassNotFoundException: >>> com.lbc.station.service.controller.StationController* >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:382) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:348) >>> at java.io.ObjectInputStream.resolveClass(ObjectInputStream.java:686) >>> at >>> org.apache.geode.internal.InternalDataSerializer$DSObjectInputStream.resolveClass(InternalDataSerializer.java:3668) >>> at >>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1868) >>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1751) >>> at java.io.ObjectInputStream.readClass(ObjectInputStream.java:1716) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1556) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2287) >>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2211) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2069) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1573) >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) >>> at >>> org.apache.geode.internal.InternalDataSerializer.readSerializable(InternalDataSerializer.java:2846) >>> at >>> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2790) >>> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968) >>> at >>> org.apache.geode.cache.lucene.internal.distributed.LuceneFunctionContext.fromData(LuceneFunctionContext.java:106) >>> at >>> org.apache.geode.internal.InternalDataSerializer.invokeFromData(InternalDataSerializer.java:2372) >>> at org.apache.geode.internal.DSFIDFactory.create(DSFIDFactory.java:997) >>> at >>> org.apache.geode.internal.InternalDataSerializer.basicReadObject(InternalDataSerializer.java:2668) >>> at org.apache.geode.DataSerializer.readObject(DataSerializer.java:2968) >>> at >>> org.apache.geode.internal.util.BlobHelper.deserializeBlob(BlobHelper.java:99) >>> at >>> org.apache.geode.internal.cache.tier.sockets.CacheServerHelper.deserialize(CacheServerHelper.java:73) >>> at >>> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:351) >>> at >>> org.apache.geode.internal.cache.tier.sockets.Part.getObject(Part.java:359) >>> at >>> org.apache.geode.internal.cache.tier.sockets.command.ExecuteRegionFunction66.cmdExecute(ExecuteRegionFunction66.java:101) >>> at >>> org.apache.geode.internal.cache.tier.sockets.BaseCommand.execute(BaseCommand.java:183) >>> at >>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.doNormalMessage(ServerConnection.java:848) >>> at >>> org.apache.geode.internal.cache.tier.sockets.OriginalServerConnection.doOneMessage(OriginalServerConnection.java:72) >>> at >>> org.apache.geode.internal.cache.tier.sockets.ServerConnection.run(ServerConnection.java:1212) >>> at >>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) >>> at >>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) >>> at >>> org.apache.geode.internal.cache.tier.sockets.AcceptorImpl.lambda$initializeServerConnectionThreadPool$3(AcceptorImpl.java:666) >>> at >>> org.apache.geode.internal.logging.LoggingThreadFactory.lambda$newThread$0(LoggingThreadFactory.java:121) >>> ... 1 common frames omitted >>> >>> >>> >>> Regards, >>> Ajay Vasudevan >>> >>> On Thu, Jan 30, 2020 at 1:03 AM Dan Smith <dsm...@pivotal.io> wrote: >>> >>>> Hi vas, >>>> >>>> The problem is that your entire query needs to be inside the lamba, >>>> like below: >>>> >>>> index -> { >>>> return new BooleanQuery.Builder() >>>> //Query object must be constructed here, >>>> inside the lambda >>>> Query queryToExecute = new >>>> StandardQueryParser().parse(queryString, "stationNameIndex"); >>>> .add(queryToExecute, BooleanClause.Occur.MUST) >>>> .build(); >>>> }); >>>> >>>> The reason you have to do this is that this entire object is serialized >>>> and evaluated *on the server*. Unfortunately, lucene queries are not >>>> serializable. So in the code you send, the query is constructed on the >>>> client and then we try to serialize it and send it to the server, which >>>> doesn't work. >>>> >>>> //Query is created on the client >>>> Query queryToExecute = new StandardQueryParser().parse(queryString, >>>> "stationNameIndex"); >>>> >>>> LuceneQuery<Long, Station> luceneQuery = >>>> luceneService >>>> .createLuceneQueryFactory() >>>> .setLimit(100) >>>> .create( >>>> "stationNameIndex", >>>> "stations-region", >>>> index -> { >>>> return new BooleanQuery.Builder() >>>> //Using a Query object here that was >>>> constructed outside this block means that it must be serialized to the >>>> server. This won't work. >>>> .add(queryToExecute, BooleanClause.Occur.MUST) >>>> .build(); >>>> }); >>>> >>>> On Wed, Jan 29, 2020 at 3:23 PM vas aj <vasudevan.a...@gmail.com> >>>> wrote: >>>> >>>>> Changed Query queryToExecute = new >>>>> StandardQueryParser().parse(queryString, >>>>> "stationNameIndex"); >>>>> to Query queryToExecute = new >>>>> StandardQueryParser().parse(queryString, "stationName"); >>>>> >>>>> But still, get the error >>>>> >>>>> Caused by: java.io.NotSerializableException: >>>>> org.apache.lucene.search.PrefixQuery >>>>> . . . >>>>> >>>>> Caused by: org.apache.geode.SerializationException: failed serializing >>>>> object >>>>> . . . >>>>> >>>>> Caused by: java.io.NotSerializableException: >>>>> org.apache.lucene.search.PrefixQuery >>>>> >>>>> Help is needed, please >>>>> >>>>> On Wed, Jan 29, 2020 at 11:13 PM vas aj <vasudevan.a...@gmail.com> >>>>> wrote: >>>>> >>>>>> Rechecked my lucene index on Geode and they are as follows: >>>>>> >>>>>> create lucene index --name=stationNameIndex --region=stations-region >>>>>> --field=stationName,longitude,latitude >>>>>> --analyzer=org.apache.lucene.analysis.en.EnglishAnalyzer,DEFAULT,DEFAULT >>>>>> >>>>>> and in java code >>>>>> >>>>>> queryString = "stationName: piccadilly*"; >>>>>> Query queryToExecute = new StandardQueryParser().parse(queryString, >>>>>> "stationNameIndex"); >>>>>> >>>>>> LuceneQuery<Long, Station> luceneQuery = >>>>>> luceneService >>>>>> .createLuceneQueryFactory() >>>>>> .setLimit(100) >>>>>> .create( >>>>>> "stationNameIndex", >>>>>> "stations-region", >>>>>> index -> { >>>>>> return new BooleanQuery.Builder() >>>>>> .add(queryToExecute, BooleanClause.Occur.MUST) >>>>>> .build(); >>>>>> }); >>>>>> >>>>>> luceneQuery.findValues() throws Serialization exception but when I >>>>>> use the >>>>>> >>>>>> LuceneQuery<Long, Station> luceneQuery = >>>>>> luceneService >>>>>> .createLuceneQueryFactory() >>>>>> .setLimit(100) >>>>>> .create( >>>>>> "stationNameIndex", >>>>>> "stations-region", >>>>>> queryString, >>>>>> "stationName" >>>>>> }); >>>>>> >>>>>> It works. >>>>>> >>>>>> What has gone wrong? >>>>>> >>>>>> On Wed, Jan 29, 2020 at 9:19 PM Xiaojian Zhou <gz...@pivotal.io> >>>>>> wrote: >>>>>> >>>>>>> You should use the stationName not station if your object is >>>>>>> indexed on the field, i.e. stationName. >>>>>>> >>>>>>> Even class Station implements DataSerializable, lucene index are >>>>>>> still only index on the object's fields, such as stationName. >>>>>>> >>>>>>> On Wed, Jan 29, 2020 at 1:04 PM vas aj <vasudevan.a...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Sorry >>>>>>>> queryString = "station: piccadilly*" to get the mentioned error. >>>>>>>> >>>>>>>> On Wed, Jan 29, 2020 at 9:01 PM vas aj <vasudevan.a...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi team, >>>>>>>>> >>>>>>>>> The below code : >>>>>>>>> >>>>>>>>> queryString = "piccadilly*"; >>>>>>>>> Query queryToExecute = new >>>>>>>>> StandardQueryParser().parse(queryString, "stationNameIndex"); >>>>>>>>> >>>>>>>>> LuceneQuery<Long, Station> luceneQuery = >>>>>>>>> luceneService >>>>>>>>> .createLuceneQueryFactory() >>>>>>>>> .setLimit(100) >>>>>>>>> .create( >>>>>>>>> "stationNameIndex", >>>>>>>>> "stations-region", >>>>>>>>> index -> { >>>>>>>>> return new BooleanQuery.Builder() >>>>>>>>> .add(queryToExecute, >>>>>>>>> BooleanClause.Occur.MUST) >>>>>>>>> .build(); >>>>>>>>> }); >>>>>>>>> Collection<Station> stations = luceneQuery.findValues(); >>>>>>>>> >>>>>>>>> *throws the below error : * >>>>>>>>> >>>>>>>>> Error while querying => {} >>>>>>>>> org.apache.geode.SerializationException: failed serializing object >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.serializeAndAddPart(Message.java:408) >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:350) >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.tier.sockets.Message.addObjPart(Message.java:329) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp$ExecuteRegionFunctionSingleHopOpImpl.<init>(ExecuteRegionFunctionSingleHopOp.java:180) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.lambda$executeFunction$1(ServerRegionProxy.java:699) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.constructAndGetExecuteFunctionTasks(ExecuteRegionFunctionSingleHopOp.java:131) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.client.internal.ExecuteRegionFunctionSingleHopOp.execute(ExecuteRegionFunctionSingleHopOp.java:84) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.client.internal.ServerRegionProxy.executeFunction(ServerRegionProxy.java:701) >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeOnServer(ServerRegionFunctionExecutor.java:200) >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.executeFunction(ServerRegionFunctionExecutor.java:154) >>>>>>>>> at >>>>>>>>> org.apache.geode.internal.cache.execute.ServerRegionFunctionExecutor.execute(ServerRegionFunctionExecutor.java:379) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findTopEntries(LuceneQueryImpl.java:121) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findPages(LuceneQueryImpl.java:99) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findResults(LuceneQueryImpl.java:85) >>>>>>>>> at >>>>>>>>> org.apache.geode.cache.lucene.internal.LuceneQueryImpl.findValues(LuceneQueryImpl.java:78) >>>>>>>>> >>>>>>>>> whereas >>>>>>>>> >>>>>>>>> LuceneQuery<Long, Station> luceneQuery = >>>>>>>>> luceneService >>>>>>>>> .createLuceneQueryFactory() >>>>>>>>> .setLimit(100) >>>>>>>>> .create( >>>>>>>>> "stationNameIndex", >>>>>>>>> "stations-region", >>>>>>>>> queryString, >>>>>>>>> "stationName" >>>>>>>>> }); >>>>>>>>> >>>>>>>>> works perfectly & Station >>>>>>>>> implements org.apache.geode.DataSerializable. >>>>>>>>> >>>>>>>>> What am I missing?. >>>>>>>>> Please help. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> Aj >>>>>>>>> >>>>>>>>> On Tue, Jan 28, 2020 at 11:31 PM Xiaojian Zhou <gz...@pivotal.io> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Maybe you should change one or both of the "MUST" to "SHOULD". >>>>>>>>>> And it's better to switch the order of the 2 subqueries. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Tue, Jan 28, 2020 at 2:23 PM vas aj <vasudevan.a...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Thanks Dan for the quick help. >>>>>>>>>>> However the query failed. >>>>>>>>>>> >>>>>>>>>>> Working on the same. >>>>>>>>>>> >>>>>>>>>>> On Tue, 28 Jan 2020, 19:55 Dan Smith, <dsm...@pivotal.io> wrote: >>>>>>>>>>> >>>>>>>>>>>> I think you could probably use a BooleanQuery to do this. >>>>>>>>>>>> Something like this, although I haven't tested it. Your >>>>>>>>>>>> LuceneQuery1 just >>>>>>>>>>>> results in a call to StandardQueryParser.parse, so you can combine >>>>>>>>>>>> that >>>>>>>>>>>> with your spatial query: >>>>>>>>>>>> >>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 = >>>>>>>>>>>> luceneService >>>>>>>>>>>> .createLuceneQueryFactory() >>>>>>>>>>>> .create("stationNameIndex", "stations-region", index >>>>>>>>>>>> -> { >>>>>>>>>>>> BooleanQuery query = new BooleanQuery(); >>>>>>>>>>>> query.add(new >>>>>>>>>>>> StandardQueryParser().parse(queryString, name), >>>>>>>>>>>> BooleanQuery.Occur.MUST); >>>>>>>>>>>> query.add(SpatialHelper.findWithin(-122.8515139, >>>>>>>>>>>> 45.5099231, 0.5), , BooleanQuery.Occur.MUST); >>>>>>>>>>>> return query; >>>>>>>>>>>> } >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Jan 28, 2020 at 11:36 AM vas aj < >>>>>>>>>>>> vasudevan.a...@gmail.com> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Hi team, >>>>>>>>>>>>> >>>>>>>>>>>>> I want to execute a *LuceneQuery* that searches both a >>>>>>>>>>>>> queryString(E.g. "station:Piccadilly*") as well as nearest >>>>>>>>>>>>> station within >>>>>>>>>>>>> .5 miles.(E.g. SpatialHelper.findWithin(lat, long, 0.5)) >>>>>>>>>>>>> >>>>>>>>>>>>> In other words, is there any means of combining *luceneQuery1* >>>>>>>>>>>>> & *luceneQuery2* and executing them as >>>>>>>>>>>>> Collection<Station> nearestStations = *luceneQuery* >>>>>>>>>>>>> .findValues(); >>>>>>>>>>>>> >>>>>>>>>>>>> where luceneQuery = luceneQuery1 + luceneQuery2 >>>>>>>>>>>>> >>>>>>>>>>>>> *LuceneQuery1* >>>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery1 = >>>>>>>>>>>>> luceneService >>>>>>>>>>>>> .createLuceneQueryFactory() >>>>>>>>>>>>> .setLimit(100) >>>>>>>>>>>>> .create("stationNameIndex", "stations-region", >>>>>>>>>>>>> queryString, "name"); >>>>>>>>>>>>> >>>>>>>>>>>>> *LuceneQuery2* >>>>>>>>>>>>> LuceneQuery<Long, Station> luceneQuery2 = >>>>>>>>>>>>> luceneService >>>>>>>>>>>>> .createLuceneQueryFactory() >>>>>>>>>>>>> .create("stationNameIndex", "stations-region", index >>>>>>>>>>>>> -> SpatialHelper.findWithin(-122.8515139, 45.5099231, 0.5); >>>>>>>>>>>>> >>>>>>>>>>>>> Regards >>>>>>>>>>>>> Aj >>>>>>>>>>>>> >>>>>>>>>>>>