Re: Mongo DB Users
Remove please 2014-09-16 6:59 GMT+02:00 Patti Kelroe-Cooke patt...@gmail.com: Remove Kind regards Patti On Mon, Sep 15, 2014 at 5:35 PM, Aaron Susan aaronsus...@gmail.com wrote: Hi, I am here to inform you that we are having a contact list of *Mongo DB Users *would you be interested in it? Data Field’s Consist Of: Name, Job Title, Verified Phone Number, Verified Email Address, Company Name Address Employee Size, Revenue size, SIC Code, Industry Type etc., We also provide other technology users as well depends on your requirement. For Example: *Red Hat * *Terra data * *Net-app * *NuoDB* *MongoHQ ** and many more* We also provide IT Decision Makers, Sales and Marketing Decision Makers, C-level Titles and other titles as per your requirement. Please review and let me know your interest if you are looking for above mentioned users list or other contacts list for your campaigns. Waiting for a positive response! Thanks *Aaron Susan* Data Specialist If you are not the right person, feel free to forward this email to the right person in your organization. To opt out response Remove
How to write core's name in log
Hi,
I have multiple core in Solr and I want to write core name in log through to
lo4j.
I've found in SolrException a method called log(Logger log, Throwable e) but
when It try to build a Exception it haven't core's name.
The Exception is built in toStr() method in SolrException class, so I want
to write core's name in the message of Exception.
I'm thinking to add MDC variable, this will be name of core. Finally I'll
use it in log4j configuration like this in ConversionPattern %X{core}
The idea is that when Solr received a request I'll add this new variable
name of core.
But I don't know if it's a good idea or not.
or Do you already exists any solution for add name of core in log?
Thanks
Joan
Date faceting per last hour, three days and last week
Hi, I'm trying date faceting per last 24 hours, three days and last week, but I don't know how to do it. I have a DateField and I want to set different ranges, it is posible? I understand the example from solr wikihttp://wiki.apache.org/solr/SimpleFacetParameters#Date_Faceting:_per_day_for_the_past_5_daysbut I want to do more gaps with the same field_date. How I do this? Thanks, Joan
Solr + Hadoop
Hi,
I'm trying build solr index with MapReduce (Hadoop) and I'm using
https://issues.apache.org/jira/browse/SOLR-1301 but I've a problem with
hadoop version and this patch.
When I compile this patch, I use 0.21.0 hadoop version, I don't have any
problem but when I'm trying to run my job in Hadoop (0.0.21) I get some
error like this:
*Exception in thread main java.lang.IncompatibleClassChangeError: Found
interface org.apache.hadoop.mapreduce.JobContext, but class was expected*
at
org.apache.solr.hadoop.SolrOutputFormat.checkOutputSpecs(SolrOutputFormat.java:147)
at
org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:373)
at
org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:334)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:960)
at org.apache.hadoop.mapreduce.Job.waitForCompletion(Job.java:976)
I try to override the next method:
*public void checkOutputSpecs(JobContext job) throws IOException {
super.checkOutputSpecs(job);
if (job.getConfiguration().get(SETUP_OK) == null) {
throw new IOException(Solr home cache not set up!);
}
}*
by
* public void checkOutputSpecs(Job job) throws IOException {
super.checkOutputSpecs(job);
if (job.getConfiguration().get(SETUP_OK) == null) {
throw new IOException(Solr home cache not set up!);
}
}*
but I continue receive some error:
*
java.lang.RuntimeException: java.lang.ClassNotFoundException:
org.apache.solr.hadoop.SolrOutputFormat
at
org.apache.hadoop.conf.Configuration.getClass(Configuration.java:1128)
at
org.apache.hadoop.mapreduce.task.JobContextImpl.getOutputFormatClass(JobContextImpl.java:203)
at org.apache.hadoop.mapred.Task.initialize(Task.java:487)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:311)
at org.apache.hadoop.mapred.Child$4.run(Child.java:217)
at java.security.AccessController.doPrivileged(Native Method)*
Please, Is someone using this patch with 0.21.0 Version Hadoop?. Can someone
help me?
Thanks,
Joan
Re: Creating Solr index from map/reduce
Thanks Alexander 2011/1/3 Alexander Kanarsky kanarsky2...@gmail.com Joan, current version of the patch assumes the location and names for the schema and solrconfig files ($SOLR_HOME/conf), it is hardcoded (see the SolrRecordWriter's constructor). Multi-core configuration with separate configuration locations via solr.xml is not supported as for now. As a workaround, you could link or copy the schema and solrconfig files to follow the hardcoded assumption. Thanks, -Alexander On Wed, Dec 29, 2010 at 2:50 AM, Joan joan.monp...@gmail.com wrote: If I rename my custom schema file (schema-xx.xml), whitch is located in SOLR_HOME/schema/, and then I copy it to conf folder and finally I try to run CSVIndexer, it shows me an other error: Caused by: java.lang.RuntimeException: Can't find resource 'solrconfig.xml' in classpath or '/tmp/hadoop-root/mapred/local/taskTracker/archive/localhost/tmp/b7611d6d-9cc7-4237-a240-96ecaab9f21a.solr.zip/conf/' I dont't understand because I've a solr configuration file (solr.xml) where I define all core: core name=core_name instanceDir=solr-data/index config=solr/conf/solrconfig_xx.xml schema=solr/schema/schema_xx.xml properties=solr/conf/solrcore.properties/ But I think that when I run CSVIndexer, it doesn't know that solr.xml exist, and it try to looking for schema.xml and solrconfig.xml by default in default folder (conf) 2010/12/29 Joan joan.monp...@gmail.com Hi, I'm trying generate Solr index from hadoop (map/reduce) so I'm using this patch SOLR-301 https://issues.apache.org/jira/browse/SOLR-1301, however I don't get it. When I try to run CSVIndexer with some arguments: directory Solr index -solr Solr home input, in this case CSV I'm runnig CSVIndexer: HADOOP_INSTALL/bin/hadoop jar my.jar CSVIndexer INDEX_FOLDER -solr /SOLR_HOME CSV FILE PATH Before that I run CSVIndexer, I've put csv file into HDFS. My Solr home hasn't default files configurations, but which is divided into multiple folders /conf /schema I have custom solr file configurations so CSVIndexer can't find schema.xml, obviously It won't be able to find it because this file doesn't exist, in my case, this file is named schema-xx.xml and CSVIndexer is looking for it inside conf folder and It don't know that schema folder exist. And I have solr configuration file (solr.xml) where I configure multiple cores. I tried to modify solr's paths but It still not working . I understand that CSVIndexer copy Solr Home specified into HDFS (/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try to find schema.xml it doesn't exit: 10/12/29 10:18:11 INFO mapred.JobClient: Task Id : attempt_201012291016_0002_r_00_1, Status : FAILED java.lang.IllegalStateException: Failed to initialize record writer for my.jar, attempt_201012291016_0002_r_00_1 at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:253) at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.io.FileNotFoundException: Source '/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml' does not exist at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636) at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606) at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:222) ... 4 more
Creating Solr index from map/reduce
Hi, I'm trying generate Solr index from hadoop (map/reduce) so I'm using this patch SOLR-301 https://issues.apache.org/jira/browse/SOLR-1301, however I don't get it. When I try to run CSVIndexer with some arguments: directory Solr index -solr Solr home input, in this case CSV I'm runnig CSVIndexer: HADOOP_INSTALL/bin/hadoop jar my.jar CSVIndexer INDEX_FOLDER -solr /SOLR_HOME CSV FILE PATH Before that I run CSVIndexer, I've put csv file into HDFS. My Solr home hasn't default files configurations, but which is divided into multiple folders /conf /schema I have custom solr file configurations so CSVIndexer can't find schema.xml, obviously It won't be able to find it because this file doesn't exist, in my case, this file is named schema-xx.xml and CSVIndexer is looking for it inside conf folder and It don't know that schema folder exist. And I have solr configuration file (solr.xml) where I configure multiple cores. I tried to modify solr's paths but It still not working . I understand that CSVIndexer copy Solr Home specified into HDFS (/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try to find schema.xml it doesn't exit: 10/12/29 10:18:11 INFO mapred.JobClient: Task Id : attempt_201012291016_0002_r_00_1, Status : FAILED java.lang.IllegalStateException: Failed to initialize record writer for my.jar, attempt_201012291016_0002_r_00_1 at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:253) at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.io.FileNotFoundException: Source '/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml' does not exist at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636) at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606) at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:222) ... 4 more
Re: Creating Solr index from map/reduce
If I rename my custom schema file (schema-xx.xml), whitch is located in SOLR_HOME/schema/, and then I copy it to conf folder and finally I try to run CSVIndexer, it shows me an other error: Caused by: java.lang.RuntimeException: Can't find resource 'solrconfig.xml' in classpath or '/tmp/hadoop-root/mapred/local/taskTracker/archive/localhost/tmp/b7611d6d-9cc7-4237-a240-96ecaab9f21a.solr.zip/conf/' I dont't understand because I've a solr configuration file (solr.xml) where I define all core: core name=core_name instanceDir=solr-data/index config=solr/conf/solrconfig_xx.xml schema=solr/schema/schema_xx.xml properties=solr/conf/solrcore.properties/ But I think that when I run CSVIndexer, it doesn't know that solr.xml exist, and it try to looking for schema.xml and solrconfig.xml by default in default folder (conf) 2010/12/29 Joan joan.monp...@gmail.com Hi, I'm trying generate Solr index from hadoop (map/reduce) so I'm using this patch SOLR-301 https://issues.apache.org/jira/browse/SOLR-1301, however I don't get it. When I try to run CSVIndexer with some arguments: directory Solr index -solr Solr home input, in this case CSV I'm runnig CSVIndexer: HADOOP_INSTALL/bin/hadoop jar my.jar CSVIndexer INDEX_FOLDER -solr /SOLR_HOME CSV FILE PATH Before that I run CSVIndexer, I've put csv file into HDFS. My Solr home hasn't default files configurations, but which is divided into multiple folders /conf /schema I have custom solr file configurations so CSVIndexer can't find schema.xml, obviously It won't be able to find it because this file doesn't exist, in my case, this file is named schema-xx.xml and CSVIndexer is looking for it inside conf folder and It don't know that schema folder exist. And I have solr configuration file (solr.xml) where I configure multiple cores. I tried to modify solr's paths but It still not working . I understand that CSVIndexer copy Solr Home specified into HDFS (/tmp/hadoop-user/mapred/local/taskTracker/archive/...) and when It try to find schema.xml it doesn't exit: 10/12/29 10:18:11 INFO mapred.JobClient: Task Id : attempt_201012291016_0002_r_00_1, Status : FAILED java.lang.IllegalStateException: Failed to initialize record writer for my.jar, attempt_201012291016_0002_r_00_1 at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:253) at org.apache.solr.hadoop.SolrOutputFormat.getRecordWriter(SolrOutputFormat.java:152) at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:553) at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:408) at org.apache.hadoop.mapred.Child.main(Child.java:170) Caused by: java.io.FileNotFoundException: Source '/tmp/hadoop-guest/mapred/local/taskTracker/archive/localhost/tmp/e8be5bb1-e910-47a1-b5a7-1352dfec2b1f.solr.zip/conf/schema.xml' does not exist at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:636) at org.apache.commons.io.FileUtils.copyFile(FileUtils.java:606) at org.apache.solr.hadoop.SolrRecordWriter.init(SolrRecordWriter.java:222) ... 4 more
Re: storing results
Thanks for your answers, Yes, its true that with boolen queries things are much easier... +(query1) +(query2) should do an and or (query1) (query2) should do an or and this does not need a special ability to parse the queries I like the dismax approach, I think is interesting but then to merge queries is a bit difficult, and I think for this option is better to try the filter, but then there is no OR option, only , AND. And bq (from what I understand from de docs) seems that also performs and AND It's a pity I have to do it for a demo in a few days (and nights) and I don't think the resulting code will be nice enough to be shown. Joan Chris Hostetter wrote: : To do so I need to store the results as a filter, with a given name, so : the user can use it later on. But I need to store this in disk, as I can : not trust on the cache or the web session. : The user should then indicate that the query that is doing now has a : filter (a previous query) and this filter should be added to the query : (this is allowed in solr, i think) but as filter_ID, to be loaded to : solve the query. if you really want to be ableto refer to these previous searches using some sort of identifier, and have them persist for an indefinite amount of time, it's really out of Solr's hands -- if someone were to try and add a fewture like this to Solr, how would it know which queries to remember and generate names for? how long would it store each name? ... these are the kidns of questsions that your app can understand more easily ... you could concievable use Solr to store the name=querystring mappings in little custom solr docs if you wanted, but you have to decide when to create those mappings and when to expire them. in general though, all you really need to remember is hte query string, remembering all of the results really isn't neccessary. The next time your user wants to "refine" his search -- wether it's 10 seconds latter or 10 days later -- just take the old query string and combine it with the new query string. how you combine it depends on how you want the scoring to work, use an "fq" param if the old query shouldn't affect the score, just define the super set of docs, use a BooleanQuery if you want all the clauses from both searches to impact the score. it's important to understand that trying to keep track of the actual results would be very, very, very bad (especially if you want remember them for a really long time) because when the user comes back, the index may have chagned, docs may have shifted ids, or been edited so they no longer match the criteria, or have been deleted completely. -Hoss -- signUPF Joan Codina Filb Departament de Tecnologia Universitat Pompeu Fabra ___ Abans d'imprimir aquest e-mail, pensa si realment s necessari, i en cas de que ho sigui, pensa que si ho fas a doble cara estalvies un 25% del paper, els arbres t'ho agrairan. ___ La informaci d'aquest missatge electrnic s confidencial, personal i intransferible i noms est dirigida a la/les adrea/ces indicades a dalt. Si vost llegeix aquest missatge per equivocaci, l'informem que queda prohibida la seva divulgaci, s o distribuci, completa o en part, i li preguem esborri el missatge original juntament amb els seus fitxers annexos sense llegir-lo ni gravar-lo. Grcies.
storing results
I'm using solr, to build a search engine, and it works great!! Thanks for the job,guys! but... I need to build a searcher that must allow to perform a "search process" for a collection of documents. And this search process may last several days, and be composed of several queries that I must be able to store and reuse. My doubts are: How can I store the search results? How can I add previously stored search results to a new search? Thanks!! -- signUPF Joan Codina Filb Departament de Tecnologia Universitat Pompeu Fabra
Re: storing results
I would like to store the results of a query someway,. Then after the user analyzes some of the documents (and he/she can take some days to do it), the user can try to make a query refinement over the previous result, getting a subset of it. To do so I need to store the results as a filter, with a given name, so the user can use it later on. But I need to store this in disk, as I can not trust on the cache or the web session. The user should then indicate that the query that is doing now has a filter (a previous query) and this filter should be added to the query (this is allowed in solr, i think) but as filter_ID, to be loaded to solve the query. I hope now its clearer what I'm trying to do Joan Mike Klaas wrote: On 3/27/07, Joan Codina [EMAIL PROTECTED] wrote: I'm using solr, to build a search engine, and it works great!! Thanks for the job,guys! Glad it is working for you. but... I need to build a searcher that must allow to perform a search process for a collection of documents. And this search process may last several days, and be composed of several queries that I must be able to store and reuse. My doubts are: How can I store the search results? How can I add previously stored search results to a new search? I'm not sure what you mean by search process, nor really what you mean by the functionality requirements you list. You can certainly store search results and combine them together however you wish, but it seems that that is more on the post-query data-analysis side of things rather than having anything to do with Solr. Could you clarify what you're looking for Solr to do for you? -Mike
