Re: trouble running indexer with Solr spatial
Po On Jan 28, 2015 1:54 AM, vit bulgako...@yahoo.com wrote: I am using Solr 4.2 I added fieldType name=location_rpt class=solr.SpatialRecursivePrefixTreeFieldType according to a href=http://; https://wiki.apache.org/solr/SolrAdaptersForLuceneSpatial4 and have spatial4j-0.3.jar in my project. When running the indexer I started getting this error: java.lang.NoClassDefFoundError: com/google/common/cache/CacheBuilder at org.apache.solr.schema.AbstractSpatialFieldType.init(AbstractSpatialFieldType.java:82) at org.apache.solr.schema.AbstractSpatialPrefixTreeFieldType.init(AbstractSpatialPrefixTreeFieldType.java:32) at org.apache.solr.schema.SpatialRecursivePrefixTreeFieldType.init(SpatialRecursivePrefixTreeFieldType.java:28) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:357) at java.lang.Class.newInstance(Class.java:310) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:470) at org.apache.solr.core.SolrResourceLoader.newInstance(SolrResourceLoader.java:458) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:81) at org.apache.solr.schema.FieldTypePluginLoader.create(FieldTypePluginLoader.java:43) at org.apache.solr.util.plugin.AbstractPluginLoader.load(AbstractPluginLoader.java:151) at org.apache.solr.schema.IndexSchema.readSchema(IndexSchema.java:374) at org.apache.solr.schema.IndexSchema.init(IndexSchema.java:122) at org.apache.solr.core.SolrCore.init(SolrCore.java:684) at org.apache.solr.core.SolrCore.init(SolrCore.java:618) -- View this message in context: http://lucene.472066.n3.nabble.com/trouble-running-indexer-with-Solr-spatial-tp4182359.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: indexing unique keys
There would be records in millions and field would only be md5 hash. Size
of field would be 32 chars
Can you please point out what would be the best approach ?
Thanks
On Sep 5, 2014 12:47 PM, Mikhail Khludnev mkhlud...@griddynamics.com
wrote:
Hello,
You are asking without giving a context. What's the size of sets, desired
TPS, key length, and even values?
It's hard to answer definitely. It's not primary usage for Lucene, it adds
some unnecessary overhead. However, community collected a few workaround
for such kind of problem. From the other side, as far as I know executing
queries like WHERE x IN (1,,2324) is not a piece of cake for SQL
servers, also.
you can follow link at
https://plus.google.com/u/0/+MichaelMcCandless/posts/8VNydNi3wvK to find a
relevant benchmark. it might help you to get least estimates for the Lucene
solution.
On Thu, Sep 4, 2014 at 5:53 PM, Mark , N nipen.m...@gmail.com wrote:
I have a use-case where we want to store unique keys ( Hashes) which
would
be
used to compare against another set of keys ( Hashes)
For example
Index set= { h1, h2 , h3 , h4 }
comparision set = { h1 , h2 }
result set = h1,h2
Would it be an advantage to store index set in Solr instead of storing
in traditional databases?
Thanks in advance
*Nipen Mark *
--
Sincerely yours
Mikhail Khludnev
Principal Engineer,
Grid Dynamics
http://www.griddynamics.com
mkhlud...@griddynamics.com
Indexing scanned PDFs
we are using SOLr to index pdf documents but there are cases where PDFs are usually a scanned document with no text to extract and index . Is there a plugin or module in SOLR that we can integrate so that it would actually extract a text / OCR and then index? Thanks in advance Chandan Tamrakar
Re: searching for an id
which analyzer are you using to index that field , you can verify that from schema file . thanks On Thu, Jan 31, 2013 at 2:35 PM, b.riez...@pixel-ink.de b.riez...@pixel-ink.de wrote: Hi I have an id wich is a string like this. tx-20130130-4599 i'm using a field without processing, wich i got confirmed via the analyser tool But when i search for that it got split up, so instead of finding that specific entry with that unique id, it finds all entries with tx in it. Any idea how to get rid of that behavior? Best Ben -- Chandan Tamrakar * *
Re: search with spaces
Which Analyzer is being used in the field that was indexed ? May be you can use solradmin to analyze and see how is your index thanks On Thu, Dec 27, 2012 at 2:30 PM, Sangeetha sangeetha...@gmail.com wrote: Hi, I have a text field with value O O Jaane Jaane. When i search with *q=Jaane Jaane* it is giving the results. But if i give *q=O O Jaane Jaane* it is not working? What could be the reason? Thanks, Sangeetha -- View this message in context: http://lucene.472066.n3.nabble.com/search-with-spaces-tp4029265.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: Even after indexing a mysql table,in solr am not able to retreive data after querying
status shows that all your 4 records were not indexed. str name=Total Documents Failed4/str On Fri, Oct 19, 2012 at 12:22 PM, Romita Saha romita.s...@sg.panasonic.comwrote: Hi, Even after indexing a mysql table,in solr am not able to retrieve data after querying. Here is the status after i run http://localhost:8983/solr/db/dataimport str name=Indexing completed. Added/Updated: 0 documents. Deleted 0 documents./str str name=Committed2012-10-19 14:31:28/str str name=Total Documents Processed0/str str name=Total Documents Failed4/str str name=Time taken0:0:0.524/str/lst str name=WARNINGThis response format is experimental. It is likely to change in the future ./str /response My data-config file is : entity name=camera query=SELECT id FROM camera field column=id name=id/ field column=data name=data/ /entity The related schema.xml file is : field name=id type=integer indexed=true stored=true required=true/ field name=data type=string indexed=true stored=true required=true/ In my database, id is of Type int (11) and data is of Type varchar(100) I am new to solr. Could any one please help. Thanks and regards, Romita Saha -- Chandan Tamrakar * *
Re: Solr Clustering
yes there is a solr component if you want to cluster solr documents , check the following link http://wiki.apache.org/solr/ClusteringComponent Carrot2 might be good if you want to cluster few thousands of documents , for example when user search solr , just cluster the search results Mahout is much more scalable and probably you need Hadoop for that thanks chandan On Tue, Sep 4, 2012 at 2:10 PM, Denis Kuzmenok forward...@ukr.net wrote: Original Message Subject: Solr Clustering From: Denis Kuzmenok forward...@ukr.net To: solr-user@lucene.apache.org CC: Hi, all. I know there is carrot2 and mahout for clustering. I want to implement such thing: I fetch documents and want to group them into clusters when they are added to index (i want to filter similar documents for example for 1 week). i need these documents quickly, so i cant rely on some postponed calculations. Each document should have assigned cluster id (like group similar documents into clusters and assign each document its cluster id. It's something similar to news aggregators like google news. I dont need to search for clusters with documents older than 1 week (for example). Each document will have its unique id and saved into DB. But solr will have cluster id field also. Is it possible to implement this with solr/carrot/mahout? -- Chandan Tamrakar * *
Carrot2 using rawtext of field for clustering
Is there any workaround in Solr/Carrot2 So that we could pass tokens that'd been filtered with customer tokenizer/filters instead of rawtext that it currently uses for clustering ? I read an issue in following link too . https://issues.apache.org/jira/browse/SOLR-2917 Is writing our own parsers to filter text documents before indexing to SOLR could be only the right approach currently ? please let me know if anyone have come across this issue and have other better suggestions? -- Chandan Tamrakar * *
Re: Can I update a specific field in solr?
I think You can do that but you need to re-index a whole document again. note that there is nothing like update , its usually delete and then add. thanks On Wed, Jun 8, 2011 at 4:00 PM, ZiLi dangld...@163.com wrote: Hi, I try to update a specific field in solr , but I didn't find anyway to implement this . Anyone who knows how to ? Any suggestions will be appriciate : ) 2011-06-08 ZiLi -- Chandan Tamrakar * *
Re: different indexes for multitenant approach
may be you need multi core feature of solr , you can have a single Solr instance with separate configurations and indexes http://wiki.apache.org/solr/CoreAdmin On Fri, Jun 3, 2011 at 12:04 PM, Naveen Gupta nkgiit...@gmail.com wrote: Hi I want to implement different index strategy where we want to keep indexes with respect to each tennant and we want to maintain indexes separately ... first level of category -- company name second level of category - company name + fields to be indexed then further categories - group of different company name based on some heuristic (hashing) (if it grows furhter) i want to do in the same solr instance. can it be possible ? Thanks Naveen -- Chandan Tamrakar * *
Re: DIH: Exception with Too many connections
looks like you are not being able to connect to database , pls see if you get similar exception when you try to connect from other clients On Tue, May 31, 2011 at 3:01 PM, tiffany tiffany.c...@future.co.jp wrote: Hi all, I'm using DIH and getting the following error. My Solr version is Solr3.1. = ... Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up. at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at com.mysql.jdbc.Util.getInstance(Util.java:381) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2364) at com.mysql.jdbc.ConnectionImpl.init(ConnectionImpl.java:781) at com.mysql.jdbc.JDBC4Connection.init(JDBC4Connection.java:46) at sun.reflect.GeneratedConstructorAccessor94.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:352) at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:284) at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:161) at org.apache.solr.handler.dataimport.JdbcDataSource$1.call(JdbcDataSource.java:128) at org.apache.solr.handler.dataimport.JdbcDataSource.getConnection(JdbcDataSource.java:363) at org.apache.solr.handler.dataimport.JdbcDataSource.access$200(JdbcDataSource.java:39) at org.apache.solr.handler.dataimport.JdbcDataSource$ResultSetIterator.init(JdbcDataSource.java:240) ... 11 more Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Data source rejected establishment of connection, message from server: Too many connections at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown Source) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at com.mysql.jdbc.Util.handleNewInstance(Util.java:406) at com.mysql.jdbc.Util.getInstance(Util.java:381) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:985) at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:956) at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1104) at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2292) ... 24 more = My dataSource setting is something like this: dataSource type=JdbcDataSource driver=com.mysql.jdbc.Driver url=jdbc:mysql://database01/test?autoReconnect=true user=xxx password=xxx batchSize=-1 / Any idea to solve this problem? Thank you! -- View this message in context: http://lucene.472066.n3.nabble.com/DIH-Exception-with-Too-many-connections-tp3005213p3005213.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: Facet Query
which analyzer do you use for indexing ? You could exclude those stop words during indexing http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters On Fri, May 27, 2011 at 1:36 PM, Jasneet Sabharwal jasneet.sabhar...@ngicorporation.com wrote: Hi When I do a facet query on my data, it shows me a list of all the words present in my database with their count. Is it possible to not get the results of common words like a, an, the, http and so one but only get the count of stuff we need like microsoft, ipad, solr, etc. -- Thanx Regards Jasneet Sabharwal -- Chandan Tamrakar * *
Re: What is omitNorms
When you say omitnorms=true for any fields it means SOLR will not store norms . AFAIK , if you do not store these norms then your index size would be smaller and will take less memory . You could safely omit these norms for smaller fields . i.e your indexing time is more. So if you do not store norms you save the memory Norms are used to boosts and field length normalization during indexing time so that short document has higher score Turning the norms on/off may depend on your indexing size and implementations I hope this helps .. thanks On Thu, May 26, 2011 at 11:48 AM, Romi romijain3...@gmail.com wrote: thanks for the link, i got lot information from this document. Can u please tell me how can i verify omitNorms effect in my document indexing or searching. - Romi -- View this message in context: http://lucene.472066.n3.nabble.com/What-is-omitNorms-tp2987547p2987649.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: problem in setting field attribute in schema.xml
I guess you are indexing with property index=false , stored = true if it is , that means you are storing the value on index , so whenever you do *:* you can see the stored value for example if you have a field = ID, Customer_Name and you would only like to index customer_name because this is a field which users is going to search .. then you can just store ID in index without indexing . When customer names matches your index you would also like to show ID to users I do not know what is the purpose on your case . Store fields are usually required when you don't want to index but show on the search results. I hope its clear . You can try and experiment changing these values on a unique fields too .. thanks. On Thu, May 26, 2011 at 4:37 PM, Romi romijain3...@gmail.com wrote: i have done it, i deleted old indexes and created new indexes but still able to search it through *:*, and no result when i search it as field:value. really surprising result. :-O - Romi -- View this message in context: http://lucene.472066.n3.nabble.com/problem-in-setting-field-attribute-in-schema-xml-tp2984126p2988256.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: problem in setting field attribute in schema.xml
Yes as i said earlier . If you want to store the value of field as it is in index without Tokenizing . .for example customer_id which is a unique fields and you don't want to tokenize when you index a field you could tokenize the field values to index based on what tokenizer you use so that users can search .. On Thu, May 26, 2011 at 5:55 PM, Romi romijain3...@gmail.com wrote: did u mean when i set indexed=false and store=true, solr does not index the field's value but store its value as it is??? - Romi -- View this message in context: http://lucene.472066.n3.nabble.com/problem-in-setting-field-attribute-in-schema-xml-tp2984126p2988458.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: What is omitNorms
This is an advance option. pls see the details on following link http://www.lucidimagination.com/Community/Hear-from-the-Experts/Articles/Scaling-Lucene-and-Solr#d0e71 On Thu, May 26, 2011 at 11:12 AM, Romi romijain3...@gmail.com wrote: and i also wanted to know what is difference if i set omitNorms in fieldType or if i set it in field. - Romi -- View this message in context: http://lucene.472066.n3.nabble.com/What-is-omitNorms-tp2987547p2987562.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
Re: Indexing Mails
what kind of emails you want to parse ? MS emails ? You could integrate apache tika but it depends on what kind of emails Tika parser would be able to parse You can define the fields that could be parsed and define that in your xml schema thanks On Tue, May 10, 2011 at 2:07 PM, Jörg Agatz joerg.ag...@googlemail.comwrote: will the E-Mail ID, and the recent E-Mail Ids, indext too? and witch fiels i have to create in schema.xml? -- Chandan Tamrakar * *
Re: Need to create dyanamic indexies base on different document workspaces
It depends on your application design how you want your index There is a feature called solr core . http://wiki.apache.org/solr/CoreAdmin You could still have a single index but a field to differentiate the items in index thanks On Thu, Apr 21, 2011 at 10:55 AM, Gaurav Shingala gaurav.shing...@hotmail.com wrote: Hi, Is there a way to create different solr indexes for different categories? We have different document workspaces and ideally want each workspace to have its own solr index. Thanks, Gaurav -- Chandan Tamrakar * *
Re: Need to create dyanamic indexies base on different document workspaces
Actually you need to put a file named *solr.xml* in the solr.home directory to create the solr core . you can do that programatically if you want to make it dynamic based on your logic pls check the solr core admin document. On Thu, Apr 21, 2011 at 2:52 PM, Gaurav Shingala gaurav.shing...@hotmail.com wrote: Is it possible to create solr core dyanamically? In our case we want each workspace to have its own solr index. Thanks From: chandan.tamra...@nepasoft.com Date: Thu, 21 Apr 2011 11:57:53 +0545 Subject: Re: Need to create dyanamic indexies base on different document workspaces To: solr-user@lucene.apache.org It depends on your application design how you want your index There is a feature called solr core . http://wiki.apache.org/solr/CoreAdmin You could still have a single index but a field to differentiate the items in index thanks On Thu, Apr 21, 2011 at 10:55 AM, Gaurav Shingala gaurav.shing...@hotmail.com wrote: Hi, Is there a way to create different solr indexes for different categories? We have different document workspaces and ideally want each workspace to have its own solr index. Thanks, Gaurav -- Chandan Tamrakar * * -- Chandan Tamrakar * *
Re: ComplexPhraseQueryParser and wildcards
did you get any exceptions ? usually wild card term you mentioned would be expanded before being actually searched . thanks. On Mon, Mar 28, 2011 at 1:24 PM, jmr jmpala...@free.fr wrote: Hi, I'm using ComplexPhraseQueryParser and I'm quite happy with it. However, there are some queries using wildcards nor working. Exemple: I want to do a proximity search between the word compiler and the expression 'cross linker' or 'cross linking' or 'cross linked' ... (cross-linker compiler~50 OR cross-linking compiler~50) is working OK but (cross-link* compiler~50) is not working (returns nothing) Is there another syntax allowing to do sucj query ? Thanks JMR -- View this message in context: http://lucene.472066.n3.nabble.com/ComplexPhraseQueryParser-and-wildcards-tp2742244p2742244.html Sent from the Solr - User mailing list archive at Nabble.com. -- Chandan Tamrakar * *
