DIH, multiple sources, cores and search: single core with multiple entities or single core per source with search across multiple cores?

2017-07-24 Thread Giovanni De Stefano
Hello guys, I need to index content coming from different sources (db, filesystems, …). Those sources share most fields, only a few are specific to the source. Content coming from different sources changes at different rates. Some sources will generate hundreds of thousands of documents, some

Re: Antw: Re: How to Debug Solr With Eclipse

2017-07-14 Thread Giovanni De Stefano
t; Bayerische Staatsbibliothek > Verbundzentrale des BVB > Referat Verbundnahe Dienste > 80807 München > Tel.: +49(0)89/28638-4445 > Fax: +49(0)89/28638-2605 > E-Mail: rainer.g...@bsb-muenchen.de > > > > >>>> Giov

Re: How to Debug Solr With Eclipse

2017-07-13 Thread Giovanni De Stefano
Hello Rainer, you have the right link: select the version you want and download the -src version. Once un untar the .tgz you can run `ant eclipse` from the command line and then import the generated project in eclipse. Please note that you will need both and and ivy installed (just start with

Re: How to "chain" import handlers: import from DB and from file system

2017-07-10 Thread Giovanni De Stefano
Thank you guys for your advice! I would rather take advantage as much as possible of the existing handlers/processors. I just realised that nested entities in DIH is extremely slow: I fixed that with a view on the DB (that does a join between 2 tables). The other thing I have to do is chain

How to "chain" import handlers: import from DB and from file system

2017-07-09 Thread Giovanni De Stefano
Hello all, I have to index (and search) data organised as followed: many files on the filesystem and each file has extra metadata stored on a DB (the DB table has a reference to the file path). I think I should have 1 Solr document per file with fields coming from both the DB (through DIH)

Does Solr 1.4 really work nicely on Jboss 4?

2009-06-18 Thread Giovanni De Stefano
Hello all, I have a simple question :-) In my project it is mandatory to use Jboss 4.0.1 SP3 and Java 1.5.0_06/08. The software relies on Solr 1.4. Now, I am aware that some JSP Admin pages will not be displayed due to some Java5/6 dependency but this is not a problem because rewriting some of

Re: Does Solr 1.4 really work nicely on Jboss 4?

2009-06-18 Thread Giovanni De Stefano
instead of Tomcat, however instructions for getting it to work in JBoss with Tomcat can be found here: http://wiki.apache.org/solr/SolrJBoss It should work fine on JBoss 4.0.1. - Daryl. On Thu, Jun 18, 2009 at 8:57 AM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello all

Re: dataimport.properties; configure writable location?

2009-05-20 Thread Giovanni De Stefano
Doh, can you please rephrase? Giovanni On Wed, May 20, 2009 at 3:47 PM, Wesley Small wesley.sm...@mtvstaff.comwrote: In Solr 1.3, is there a setting that allows one to modified the where the dataimport.properties file resides? In a production environment, the solrconfig directory needs to

Re: How to retrieve all available Cores in a static way ?

2009-05-20 Thread Giovanni De Stefano
:17 PM, Ryan McKinley ryan...@gmail.com wrote: I cringe to suggest this but you can use the deprecated call: SolrCore.getSolrCore().getCoreContainer() On May 19, 2009, at 11:21 AM, Giovanni De Stefano wrote: Hello all, I have a quick question but I cannot find a quick answer :-) I

How to retrieve all available Cores in a static way ?

2009-05-19 Thread Giovanni De Stefano
Hello all, I have a quick question but I cannot find a quick answer :-) I have a Java client running on the same JVM where Solr is running. The Solr I have is a multicore. How can I retrieve from the Java client the different cores available? I tried with: ... CoreContainer container = new

Re: Access HTTP headers from custom request handler

2009-04-23 Thread Giovanni De Stefano
Hello Hoss, thank you for your reply. I have no problems subclassing the SolrDispatchFilter...but where shall I configure it? :-) I cannot find any doc/wiki explaining how to configure a custom dispatch filter. I believe it should be in solrconfig.xml requestDispatcher ... ...

Access HTTP headers from custom request handler

2009-04-09 Thread Giovanni De Stefano
Hello all, we are writing a custom request handler and we need to implement some business logic according to some HTTP headers. I see there is no easy way to access HTTP headers from the request handler. Moreover it seems to me that the HTTPServletness is lost way before the custom request

Custom DIH: FileDataSource with additional business logic?

2009-04-09 Thread Giovanni De Stefano
Hello, here I am with another question. I am using DIH to index a DB. Additionally I also have to index some files containing Java serialized objects (and I cannot change this... :-( ). I currently have implemented a standalone Java app with the following features: 1) read all files from a

Re: Indexing fields of xml file using solrj

2009-04-01 Thread Giovanni De Stefano
Hello, I believe what you want is DirectXMLRequest. http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/request/DirectXmlRequest.html Cheers, Giovanni On 4/1/09, Radha C. cra...@ceiindia.com wrote: Hi All, I want to index the document fields in a xml file to index using solrj.

Re: multicore

2009-04-01 Thread Giovanni De Stefano
Hello, the starting point is here: http://wiki.apache.org/solr/CoreAdmin Cheers, Giovanni On 4/1/09, Neha Bhardwaj neha_bhard...@persistent.co.in wrote: Hi, I need to create multiple cores for my project. I need to know: how to have multiple cores ? can we start all cores from

Re: Indexing fields of xml file using solrj

2009-04-01 Thread Giovanni De Stefano
I understand Shalin is a guru and I am nobody but... http://lucene.apache.org/solr/api/org/apache/solr/client/solrj/request/DirectXmlRequest.html Is what you need if you want to use Solrj... :-) On 4/1/09, Radha C. cra...@ceiindia.com wrote: Thanks shalin, I need to index the xml which is

Re: Indexing fields of xml file using solrj

2009-04-01 Thread Giovanni De Stefano
in identifying the xml location, I am having the input xml in $solrhome/inputdata/example.xml, Do you have any idea about how to get solrhome location dynamically by using any solrj API class? _ From: Giovanni De Stefano [mailto:giovanni.destef...@gmail.com] Sent: Wednesday, April 01, 2009 7:30 PM

DIH DateFormatTransformer ClassCastException

2009-03-30 Thread Giovanni De Stefano
Hello all, I have a little problem with DIH and DateFormatTransformer. My column in the DB contains an Oracle TIMESTAMP; the corresponding field is of type date in the Solr schema; I specified a DateFormatTransformer for the entity; the field definition is as follow: field column=WHATEVER

Re: DIH DateFormatTransformer ClassCastException

2009-03-30 Thread Giovanni De Stefano
wrote: On Mon, Mar 30, 2009 at 8:52 PM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello all, I have a little problem with DIH and DateFormatTransformer. My column in the DB contains an Oracle TIMESTAMP; the corresponding field is of type date in the Solr schema; I

Re: DIH DateFormatTransformer ClassCastException

2009-03-30 Thread Giovanni De Stefano
...@gmail.com wrote: On Mon, Mar 30, 2009 at 10:59 PM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Unfortunately I have to perform some kind of transformation otherwise the exception is: org.apache.solr.common.SolrException: Invalid Date String:'2008-07-12T22:00:26+0200

Solrj exception posting XML docs

2009-03-27 Thread Giovanni De Stefano
Hello all, I am currently using Solr 1.3 and its Solrj. I am trying to post XML docs directly through Solrj but I get the following exception: 13:12:09,119 ERROR [STDERR] Mar 27, 2009 1:12:09 PM org.apache.solr.common.SolrException log SEVERE: java.lang.NullPointerException at

Re: Search transparently with Solr with multiple cores, different indexes, common response type

2009-03-27 Thread Giovanni De Stefano
Hello Hoss, Steve, thank you very much for your feedbacks, they have been very helpful making me feel more confident now about this architecture. In fact I decided to go for a single shared schema, but keeping multiple indexes (multicore) because those two indexes are very different: one is huge

Re: Solrj exception posting XML docs

2009-03-27 Thread Giovanni De Stefano
code to understand that the command was missing... Anyway I posted my own resolution for future reference :-) Cheers, Giovanni On 3/27/09, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello all, I am currently using Solr 1.3 and its Solrj. I am trying to post XML docs directly

Solr date parsing issue

2009-03-27 Thread Giovanni De Stefano
Hello, I am having a problem indexing a date field. In my schema the date field is defined the standard way: fieldType name=date class=solr.DateField sortMissingLast=true omitNorms=true/ I know the Solr format is 1995-12-31T23:59:59Z, but the dates coming from my sources are in the format

Re: Solr date parsing issue

2009-03-27 Thread Giovanni De Stefano
at 8:17 PM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello, I am having a problem indexing a date field. In my schema the date field is defined the standard way: fieldType name=date class=solr.DateField sortMissingLast=true omitNorms=true/ I know the Solr format

Re: Solr date parsing issue

2009-03-27 Thread Giovanni De Stefano
Hello, yes, I use both: I have a multicore architecture, multiple indexes but I have been able to manage a common schema. Giovanni On 3/27/09, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sat, Mar 28, 2009 at 12:46 AM, Giovanni De Stefano giovanni.destef...@gmail.com wrote

Re: indexing mutiple table

2009-03-26 Thread Giovanni De Stefano
Hello, I believe you should use 2 different indexes, 2 different cores and write a custom request handler or any other client that forwards the query to the cores and merge the results. Cheers, Giovanni On 3/26/09, Radha C. cra...@ceiindia.com wrote: Hi, I am trying to index different

Re: indexing mutiple table

2009-03-26 Thread Giovanni De Stefano
spread over many tables say more than 50 tables, then I have to setup that many cores ? _ From: Giovanni De Stefano [mailto:giovanni.destef...@gmail.com] Sent: Thursday, March 26, 2009 5:04 PM To: solr-user@lucene.apache.org; cra...@ceiindia.com Subject: Re: indexing mutiple table Hello

Re: Field tokenizer question

2009-03-23 Thread Giovanni De Stefano
Hello Ashish, as far as I know solr.StrField is not analized but it is indexed as is (verbatim). Cheers, Giovanni On 3/23/09, Ashish P ashish.ping...@gmail.com wrote: Hey it works. Can you please tell me the reason?? Thanks, Ashish Koji Sekiguchi-2 wrote: Ashish P wrote: I have

Re: directly communicate with solr with java

2009-03-23 Thread Giovanni De Stefano
Hello Julian, perhaps you are looking for Solrj: http://wiki.apache.org/solr/Solrj http://fisk.stjernesludd.net/archives/2-Using-Solrj-A-short-guide-to-getting-started-with-Solrj.html Cheers, Giovanni On 3/22/09, Julian Davchev j...@drun.net wrote: Hi, I got solr setup working smoothly

Re: Do I need maven build for solrj client?

2009-03-23 Thread Giovanni De Stefano
Hello, I have found this simple tutorial here: http://fisk.stjernesludd.net/archives/2-Using-Solrj-A-short-guide-to-getting-started-with-Solrj.html Maybe it helps... Cheers, Giovanni On 3/22/09, Shalin Shekhar Mangar shalinman...@gmail.com wrote: On Sun, Mar 22, 2009 at 11:54 PM, radha c

Search transparently with Solr with multiple cores, different indexes, common response type

2009-03-20 Thread Giovanni De Stefano
Hello all, here I am with another question... :-) I figured that I have to change approach to implement the requirements I have :-( Here it is what I have to index: 1) data A in an Oracle DB Table A 2) data B in an Oracle DB Table B 3) data C in different files Data A, B, and C are slightly

Re: Solr multiple indexes

2009-03-19 Thread Giovanni De Stefano
not yet via Solr. Otis -- Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch - Original Message From: Giovanni De Stefano giovanni.destef...@gmail.com To: solr-user@lucene.apache.org Sent: Wednesday, March 18, 2009 12:56:36 PM Subject: Solr multiple indexes Hello all

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
Guten Tag Ralf, try to look here: http://wiki.apache.org/solr/CoreAdmin It says: You can also specify properties in solr.xml which can be used in the solrconfig.xml and schema.xml files. I believe this might help. Cheers, Giovanni On 3/19/09, Kraus, Ralf | pixelhouse GmbH r...@pixelhouse.de

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
/should move those 199 common lines in solr.xml. Cheers, Giovanni On 3/19/09, Kraus, Ralf | pixelhouse GmbH r...@pixelhouse.de wrote: Giovanni De Stefano schrieb: Hello Ralf, I believe you want to do something like: solr persistent=true sharedLib=lib cores adminPath=/admin/cores core name

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
, Kraus, Ralf | pixelhouse GmbH r...@pixelhouse.de wrote: Giovanni De Stefano schrieb: Guten Tag Ralf, try to look here: http://wiki.apache.org/solr/CoreAdmin It says: You can also specify properties in solr.xml which can be used in the solrconfig.xml and schema.xml files. I believe this might

Re: Combination of solr.xml and solrconfig.xml

2009-03-19 Thread Giovanni De Stefano
there are pure properties that are tokenized in all solrconfig.xml. I hope it helps. Giovanni On 3/19/09, Kraus, Ralf | pixelhouse GmbH r...@pixelhouse.de wrote: Giovanni De Stefano schrieb: On the other hand, if your files have 199 lines in *common*, then yes, you could/should move those 199

Re: Solr: delta-import, help needed

2009-03-18 Thread Giovanni De Stefano
items using DIH you must maintain a deleted rows ids in the db or just flag them as deleted. --Noble On Wed, Mar 18, 2009 at 2:46 PM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello Paul, thank you for your reply. The UPDATE in fact works fine: I only had to update

Solr multiple indexes

2009-03-18 Thread Giovanni De Stefano
Hello all, here I am with another question :-) I have to index the content of two different tables on an Oracle DB. When it comes to only one table, everything is fine: one datasource, one document, one entity in data-config, one uniqueKey in schema.xml etc. It works great. But now I have on

Solr: delta-import, help needed

2009-03-17 Thread Giovanni De Stefano
Hello all, I have a table TEST in an Oracle DB with the following columns: URI (varchar), CONTENT (varchar), CREATION_TIME (date). The primary key both in the DB and Solr is URI. Here is my data-config.xml: dataConfig dataSource driver=oracle.jdbc.driver.OracleDriver

Solr: ERRORs at Startup

2009-03-13 Thread Giovanni De Stefano
Hello everybody, I am currently using: - Solr v1.3.0 - Jboss jboss-5.0.1.GA http://jboss-5.0.1.ga/ - Java jdk 1.5_06 When I start Solr within Jboss I see a lot of errors in the log but Solr seems working (meaning I can see the admin interface but I cannot index my DB...but that is

Solr: is there a default ClobTransformer?

2009-03-13 Thread Giovanni De Stefano
Hello all, I am trying to index an Oracle DB with some Clob columns. Following the doc I see that I need to transform my entity with a ClobTransformer. Now, my log says the following: 12:05:52,901 ERROR [STDERR] Mar 13, 2009 12:05:52 PM org.apache.solr.handler.dataimport.EntityProcessorBase

Re: Solr: ERRORs at Startup

2009-03-13 Thread Giovanni De Stefano
wouldn't worry too much, the content of the messages doesn't imply anything bad going on. Toby. On 13 Mar 2009, at 09:57, Giovanni De Stefano wrote: Hello everybody, I am currently using: Solr v1.3.0 Jboss jboss-5.0.1.GA Java jdk 1.5_06 When I start Solr within Jboss I see a lot

Re: Solr: is there a default ClobTransformer?

2009-03-13 Thread Giovanni De Stefano
at 12:29 PM, Noble Paul നോബിള്‍ नोब्ळ् noble.p...@gmail.com wrote: ClobTranformer is a Solr1.4 feature. which one are you using? On Fri, Mar 13, 2009 at 4:39 PM, Giovanni De Stefano giovanni.destef...@gmail.com wrote: Hello all, I am trying to index an Oracle DB with some Clob columns