Re: Questions regarding indexing JSON data
On Mon, Sep 21, 2015, at 02:53 AM, Kevin Vasko wrote:
> I am new to Apache Solr and have been struggling with indexing some JSON
> files.
>
> I have several TB of twitter data in JSON format that I am having trouble
> posting/indexing. I am trying to use a schemaless schema so I don't have
> to add 200+ records fields manually.
>
> 1.
>
> The first issue is none of the records have '[' or ']' wrapped around the
> records. So it looks like this:
>
> { "created_at": "Sun Apr 19 23:45:45 + 2015","id":
> 5.899379634353e+17, "id_str": "589937963435302912", mailing list>}
>
>
> Just to validate the schemaless portion was working I used a single
> "tweet" and trimmed it down to bare minimum. The brackets not being in
> the origian appears to be a problem as when I tried to process just a
> small portion of one record it requires me to wrap the row in a [ ] (I
> assume to make it an array) to index correctly. Like the following:
>
> [{ "created_at": "Sun Apr 19 23:45:45 + 2015","id":
> 5.899379634353e+17, "id_str": "589937963435302912", list>}]
>
> Is there a way around this? I didn't want to preprocess the TB's of JSON
> data that is in this format to add '[', ',' and '[' around all of the
> data.
>
> 2.
>
> The second issue is some of the fields have null values.
> e.g. "in_reply_to_status_id": null,
>
> I think I figured a way to resolve this by manually adding the field as a
> "strings" type but if I miss one it will kick the file out. Just wanted
> to see if there was something I could add to the schemaless configuration
> to have it pick up null fields as replace them as strings automatically?
> Or is there a better way to handle this?
>
>
> 3.
> The last issue I think my most difficult issue. Which is dealing with
> "nested" or "children" fields in my JSON data.
>
> The data looks like this. https://gist.github.com/gnip/764239. Is there
> anyways to index this information preferably automatically (schemaless
> method) without having to flatten all of my data?
1. Solr is designed to handle large amounts of content. You don't want
to be pushing documents one at a time, as you will be wasting huge
amounts of effort needlessly. Therefore, Solr assumes that when it
receives JSON, it will be in an array of documents. IIRC, when you post
an object {}, it will be considered a partial update instruction.
2. Don't rely upon the schemaless setup. Define your schema - you can't
actually live without one. Relying upon the data to work it out for you
is fraught with risk. Whether you define it via HTTP calls, or via
editing an XML file, is up to you. Just don't rely upon it correctly
guessing.
Also, when you have a 'null', the equivalent in Solr is to omit the
field. There is typically no concept in Solr for storing a null value.
3. Look at block joins, they may well help. But remember a Lucene index
is currently largely flat - you won't get anything like the versatility
out of it that you would from a relational database (in relation to
nested structures) as that isn't what it was designed for. Really,
you're gonna want to identify what you want OUT of your data, and then
identify a data structure that will allow you to achieve it. You cannot
assume that there is a standard way of doing it that will support every
use-case.
Upayavira
Questions regarding indexing JSON data
I am new to Apache Solr and have been struggling with indexing some JSON files.
I have several TB of twitter data in JSON format that I am having trouble
posting/indexing. I am trying to use a schemaless schema so I don't have to add
200+ records fields manually.
1.
The first issue is none of the records have '[' or ']' wrapped around the
records. So it looks like this:
{ "created_at": "Sun Apr 19 23:45:45 + 2015","id": 5.899379634353e+17,
"id_str": "589937963435302912",}
Just to validate the schemaless portion was working I used a single "tweet" and
trimmed it down to bare minimum. The brackets not being in the origian appears
to be a problem as when I tried to process just a small portion of one record
it requires me to wrap the row in a [ ] (I assume to make it an array) to index
correctly. Like the following:
[{ "created_at": "Sun Apr 19 23:45:45 + 2015","id": 5.899379634353e+17,
"id_str": "589937963435302912",}]
Is there a way around this? I didn't want to preprocess the TB's of JSON data
that is in this format to add '[', ',' and '[' around all of the data.
2.
The second issue is some of the fields have null values.
e.g. "in_reply_to_status_id": null,
I think I figured a way to resolve this by manually adding the field as a
"strings" type but if I miss one it will kick the file out. Just wanted to see
if there was something I could add to the schemaless configuration to have it
pick up null fields as replace them as strings automatically? Or is there a
better way to handle this?
3.
The last issue I think my most difficult issue. Which is dealing with "nested"
or "children" fields in my JSON data.
The data looks like this. https://gist.github.com/gnip/764239. Is there anyways
to index this information preferably automatically (schemaless method) without
having to flatten all of my data?
Thanks.
Re: Issue regarding Indexing PDFs into Solr.
It seems that your solrconfig.xml can not find libraries. Here is an example path from solrconfig.xml: 2013/4/29 Krishna Venkateswaran > Hi > > I have installed Solr over Apache Tomcat. > I have used Apache Tomcat v6.x for Solr to work. > > When trying to upload a file using SolrJ to index it into Solr, I am > getting an exception as follows: > > Server at http://localhost:8080/solr-example returned non ok status:500, > message:Internal Server Error > > When I looked up at the internet, I saw that the jars location were issue > and hence I changed them too. > But even then I am still getting this exception. > > Can you help me in this regard? > > I am also adding the logs from Catalina.out below: > > > > Apr 28, 2013 4:22:05 PM org.apache.solr.common.SolrException log > SEVERE: null:org.apache.solr.common.SolrException: lazy loading error > at > > org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:258) > at > > org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240) > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817) > at > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) > at > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) > at > > org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) > at > > org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) > at > > org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) > at > > org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) > at > > org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) > at > > org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) > at > > org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) > at > org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) > at > org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) > at > > org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) > at > org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) > at java.lang.Thread.run(Thread.java:679) > Caused by: org.apache.solr.common.SolrException: Error loading class > 'solr.extraction.ExtractingRequestHandler' > at > > org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:440) > at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:518) > at > org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:592) > at > > org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:249) > ... 17 more > Caused by: java.lang.ClassNotFoundException: > solr.extraction.ExtractingRequestHandler > at java.net.URLClassLoader$1.run(URLClassLoader.java:217) > at java.security.AccessController.doPrivileged(Native Method) > at java.net.URLClassLoader.findClass(URLClassLoader.java:205) > at java.lang.ClassLoader.loadClass(ClassLoader.java:321) > at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:615) > at java.lang.ClassLoader.loadClass(ClassLoader.java:266) > at java.lang.Class.forName0(Native Method) > at java.lang.Class.forName(Class.java:266) > at > > org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:424) > ... 20 more > > Thanks and Regards > Krishna >
Issue regarding Indexing PDFs into Solr.
Hi I have installed Solr over Apache Tomcat. I have used Apache Tomcat v6.x for Solr to work. When trying to upload a file using SolrJ to index it into Solr, I am getting an exception as follows: Server at http://localhost:8080/solr-example returned non ok status:500, message:Internal Server Error When I looked up at the internet, I saw that the jars location were issue and hence I changed them too. But even then I am still getting this exception. Can you help me in this regard? I am also adding the logs from Catalina.out below: Apr 28, 2013 4:22:05 PM org.apache.solr.common.SolrException log SEVERE: null:org.apache.solr.common.SolrException: lazy loading error at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:258) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handleRequest(RequestHandlers.java:240) at org.apache.solr.core.SolrCore.execute(SolrCore.java:1817) at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:639) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:345) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:141) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293) at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:861) at org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:606) at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489) at java.lang.Thread.run(Thread.java:679) Caused by: org.apache.solr.common.SolrException: Error loading class 'solr.extraction.ExtractingRequestHandler' at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:440) at org.apache.solr.core.SolrCore.createInstance(SolrCore.java:518) at org.apache.solr.core.SolrCore.createRequestHandler(SolrCore.java:592) at org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.getWrappedHandler(RequestHandlers.java:249) ... 17 more Caused by: java.lang.ClassNotFoundException: solr.extraction.ExtractingRequestHandler at java.net.URLClassLoader$1.run(URLClassLoader.java:217) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:205) at java.lang.ClassLoader.loadClass(ClassLoader.java:321) at java.net.FactoryURLClassLoader.loadClass(URLClassLoader.java:615) at java.lang.ClassLoader.loadClass(ClassLoader.java:266) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:266) at org.apache.solr.core.SolrResourceLoader.findClass(SolrResourceLoader.java:424) ... 20 more Thanks and Regards Krishna
RE: Regarding Indexing Multiple Columns Best Practise
Thanks Erick!! -Original Message- From: Erick Erickson [mailto:erickerick...@gmail.com] Sent: Friday, March 16, 2012 6:58 PM To: solr-user@lucene.apache.org Subject: Re: Regarding Indexing Multiple Columns Best Practise I would *guess* you won't notice much/any difference. Note that, if you use a fieldType with the increment gap > 1 (the default is often set to 100), phrase queries (slop) will perform differently depending upon which option you choose. Best Erick On Thu, Mar 15, 2012 at 10:49 AM, Husain, Yavar wrote: > Say I have around 30-40 fields (SQL Table Columns) indexed using Solr from > the database. I concatenate those fields into one field by using Solr > copyfield directive and than make it default search field which I search. > > If at the database level itself I perform concatenation of all those fields > into one field and then index that field directly (it will avoid copy > operation of Solr of each field to that concatenated field) will it be a > indexing performance improvement? I am sure it will be but will it make a > big/huge change in indexing running time? > > Thanks > ** > This message may contain confidential or > proprietary information intended only for the use of the > addressee(s) named above or may contain information that is legally > privileged. If you are not the intended addressee, or the person > responsible for delivering it to the intended addressee, you are > hereby notified that reading, disseminating, distributing or copying > this message is strictly prohibited. If you have received this message by > mistake, please immediately notify us by replying to the message and delete > the original message and any copies immediately thereafter. > > Thank you.- > ** > > FAFLD >
Re: Regarding Indexing Multiple Columns Best Practise
I would *guess* you won't notice much/any difference. Note that, if you use a fieldType with the increment gap > 1 (the default is often set to 100), phrase queries (slop) will perform differently depending upon which option you choose. Best Erick On Thu, Mar 15, 2012 at 10:49 AM, Husain, Yavar wrote: > Say I have around 30-40 fields (SQL Table Columns) indexed using Solr from > the database. I concatenate those fields into one field by using Solr > copyfield directive and than make it default search field which I search. > > If at the database level itself I perform concatenation of all those fields > into one field and then index that field directly (it will avoid copy > operation of Solr of each field to that concatenated field) will it be a > indexing performance improvement? I am sure it will be but will it make a > big/huge change in indexing running time? > > Thanks > ** > This message may contain confidential or proprietary information intended > only for the use of the > addressee(s) named above or may contain information that is legally > privileged. If you are > not the intended addressee, or the person responsible for delivering it to > the intended addressee, > you are hereby notified that reading, disseminating, distributing or copying > this message is strictly > prohibited. If you have received this message by mistake, please immediately > notify us by > replying to the message and delete the original message and any copies > immediately thereafter. > > Thank you.- > ** > FAFLD >
Regarding Indexing Multiple Columns Best Practise
Say I have around 30-40 fields (SQL Table Columns) indexed using Solr from the database. I concatenate those fields into one field by using Solr copyfield directive and than make it default search field which I search. If at the database level itself I perform concatenation of all those fields into one field and then index that field directly (it will avoid copy operation of Solr of each field to that concatenated field) will it be a indexing performance improvement? I am sure it will be but will it make a big/huge change in indexing running time? Thanks ** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you.- ** FAFLD
Re: Question regarding indexing multiple languages, stopwords, etc.
Greg, You need to get stopword lists for your 6 languages. Then you need to create new field types just like that 'text' type, one for each language. Point them to the appropriate stopwords files and instead of "English" specify each one of your languages. You can either index each language in its own index or put them all in the same index, in which case you'll want fields like title_en, title_fr, etc. Check http://search-lucene.com/ - this multilingual stuff is a common topic. Otis Sematext :: http://sematext.com/ :: Solr - Lucene - Nutch Lucene ecosystem search :: http://search-lucene.com/ - Original Message > From: Greg Georges > To: "solr-user@lucene.apache.org" > Sent: Mon, February 21, 2011 4:27:46 PM > Subject: Question regarding indexing multiple languages, stopwords, etc. > > Hello all, > > I have gotten my DataImporthandler to index my data from my MySQL database. > I >was looking at the schema tool and noticing that stopwords in different >languages are being indexed as terms. The 6 languages we have are English, >French, Spanish, Chinese, German and Italian. > > Right now I am using the basic schema configuration for English. How do I >define them for others languages? I have looked at the wiki page >(http://wiki.apache.org/solr/LanguageAnalysis) but I would like to have an >example configuration for all the languages I need. Also I need a list of >stopwords for these languages. So far I have this > > > > > > > ignoreCase="true" > words="stopwords.txt" > enablePositionIncrements="true" > /> > generateWordParts="1" >generateNumberParts="1" catenateWords="1" catenateNumbers="1" catenateAll=" >splitOnCaseChange="1"/> > > protected="protwords.txt"/> > > > Thanks in advance > > Greg >
Question regarding indexing multiple languages, stopwords, etc.
Hello all, I have gotten my DataImporthandler to index my data from my MySQL database. I was looking at the schema tool and noticing that stopwords in different languages are being indexed as terms. The 6 languages we have are English, French, Spanish, Chinese, German and Italian. Right now I am using the basic schema configuration for English. How do I define them for others languages? I have looked at the wiki page (http://wiki.apache.org/solr/LanguageAnalysis) but I would like to have an example configuration for all the languages I need. Also I need a list of stopwords for these languages. So far I have this Thanks in advance Greg
Re: query regarding Indexing xml files -db-data-config.xml
Hi Noble,
Thanks for the reply,
As advised I have changed the db-data-config.xml as below. But still the
Indexing completed. Added/Updated: 0 documents. Deleted 0
documents.
Got error as below when baseDir is removed
INFO: last commit = 1242683454570
May 18, 2009 2:55:15 PM org.apache.solr.handler.dataimport.DataImporter
doFullImport
SEVERE: Full Import failed
org.apache.solr.handler.dataimport.DataImportHandlerException: 'baseDir' is
a required attribute Pro
cessing Document # 1
at
org.apache.solr.handler.dataimport.FileListEntityProcessor.init(FileListEntityProcessor.j
ava:76)
at
org.apache.solr.handler.dataimport.DocBuilder.buildDocument(DocBuilder.java:299)
at
org.apache.solr.handler.dataimport.DocBuilder.doFullDump(DocBuilder.java:225)
at
org.apache.solr.handler.dataimport.DocBuilder.execute(DocBuilder.java:167)
at
org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:324)
at
org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:382)
at
org.apache.solr.handler.dataimport.DataImporter$1.run(DataImporter.java:363)
May 18, 2009 2:55:15 PM org.apache.solr.update.DirectUpdateHandler2 rollback
INFO: start rollback
Please advise.
Thanks and regards,
Jay
2009/5/17 Noble Paul നോബിള് नोब्ळ्
> hi ,
> u may not need that enclosing entity , if you only wish to index one file.
>
> baseDir is not required if you give absolute path in the fileName.
>
> no need to mention forEach or fields if you set useSolrAddSchema="true"
>
> On Sat, May 16, 2009 at 1:23 AM, jayakeerthi s
> wrote:
> > Hi All,
> >
> > I am trying to index the fileds from the xml files, here is the
> > configuration that I am using.
> >
> >
> > db-data-config.xml
> >
> >
> >
> >
> > > fileName="c:\test\ipod_other.xml" recursive="true" rootEntity="false"
> > dataSource="null" baseDir="${dataimporter.request.xmlDataDir}">
> >
> > > name="manu"/>
> >
> >
> >
> >
> >
> >
> > Schema.xml has the field "manu"
> >
> > The input xml file used to import the field is
> >
> >
> > F8V7067-APL-KIT
> > Belkin Mobile Power Cord for iPod w/ Dock
> > Belkin
> > electronics
> > connector
> > car power adapter, white
> > 4
> > 19.95
> > 1
> > false
> >
> >
> >
> > doing the full-import this is the response I am getting
> >
> > -
> > 0
> > 0
> > 0
> > 2009-05-15 11:58:00
> > Indexing completed. Added/Updated: 0 documents. Deleted 0
> > documents.
> > 2009-05-15 11:58:00
> > 2009-05-15 11:58:00
> > 0:0:0.172
> >
> > This response format is experimental. It is likely
> to
> > change in the future.
> >
> >
> >
> > Do I missing anything here or is there any format on the input xml,??
> please
> > help resolving this.
> >
> > Thanks and regards,
> > Jay
> >
>
>
>
> --
> -
> Noble Paul | Principal Engineer| AOL | http://aol.com
>
Re: query regarding Indexing xml files -db-data-config.xml
hi ,
u may not need that enclosing entity , if you only wish to index one file.
baseDir is not required if you give absolute path in the fileName.
no need to mention forEach or fields if you set useSolrAddSchema="true"
On Sat, May 16, 2009 at 1:23 AM, jayakeerthi s wrote:
> Hi All,
>
> I am trying to index the fileds from the xml files, here is the
> configuration that I am using.
>
>
> db-data-config.xml
>
>
>
>
> fileName="c:\test\ipod_other.xml" recursive="true" rootEntity="false"
> dataSource="null" baseDir="${dataimporter.request.xmlDataDir}">
>
> name="manu"/>
>
>
>
>
>
>
> Schema.xml has the field "manu"
>
> The input xml file used to import the field is
>
>
> F8V7067-APL-KIT
> Belkin Mobile Power Cord for iPod w/ Dock
> Belkin
> electronics
> connector
> car power adapter, white
> 4
> 19.95
> 1
> false
>
>
>
> doing the full-import this is the response I am getting
>
> -
> 0
> 0
> 0
> 2009-05-15 11:58:00
> Indexing completed. Added/Updated: 0 documents. Deleted 0
> documents.
> 2009-05-15 11:58:00
> 2009-05-15 11:58:00
> 0:0:0.172
>
> This response format is experimental. It is likely to
> change in the future.
>
>
>
> Do I missing anything here or is there any format on the input xml,?? please
> help resolving this.
>
> Thanks and regards,
> Jay
>
--
-
Noble Paul | Principal Engineer| AOL | http://aol.com
Re: query regarding Indexing xml files -db-data-config.xml
Hmmm,
I thought that if you were using the XPathEntityProcessor that
you have to specify an xpath for each of the fields you want
to populate. Unless you are using XPathEntityProcessor's use
useSolrAddSchema mode?
Fergus.
>If that is your complete input file then it looks like you are missing the
>wrapping element:
>
>
>
> F8V7067-APL-KIT>
>> field>
>> Belkin Mobile Power Cord for iPod w/ Dock
>> Belkin
>> electronics
>> connector
>> car power adapter, white
>> 4
>> 19.95
>> 1
>> false
>>
>
>
>
>Is it possible you just forgot to include the ?
>
>-Jay
>
>
>On Fri, May 15, 2009 at 12:53 PM, jayakeerthi s wrote:
>
>> Hi All,
>>
>> I am trying to index the fileds from the xml files, here is the
>> configuration that I am using.
>>
>>
>> db-data-config.xml
>>
>>
>>
>>
>> > fileName="c:\test\ipod_other.xml" recursive="true" rootEntity="false"
>> dataSource="null" baseDir="${dataimporter.request.xmlDataDir}">
>>
>>> name="manu"/>
>>
>>
>>
>>
>>
>>
>> Schema.xml has the field "manu"
>>
>> The input xml file used to import the field is
>>
>>
>> F8V7067-APL-KIT
>> Belkin Mobile Power Cord for iPod w/ Dock
>> Belkin
>> electronics
>> connector
>> car power adapter, white
>> 4
>> 19.95
>> 1
>> false
>>
>>
>>
>> doing the full-import this is the response I am getting
>>
>> -
>> 0
>> 0
>> 0
>> 2009-05-15 11:58:00
>> Indexing completed. Added/Updated: 0 documents. Deleted 0
>> documents.
>> 2009-05-15 11:58:00
>> 2009-05-15 11:58:00
>> 0:0:0.172
>>
>> This response format is experimental. It is likely to
>> change in the future.
>>
>>
>>
>> Do I missing anything here or is there any format on the input xml,??
>> please
>> help resolving this.
>>
>> Thanks and regards,
>> Jay
>>
--
===
Fergus McMenemie Email:fer...@twig.me.uk
Techmore Ltd Phone:(UK) 07721 376021
Unix/Mac/Intranets Analyst Programmer
===
Re: query regarding Indexing xml files -db-data-config.xml
Many thanks for the reply
The complete input xml file is below I missed to include this earlier.
F8V7067-APL-KIT
Belkin Mobile Power Cord for iPod w/ Dock
Belkin
electronics
connector
car power adapter, white
4
19.95
1
false
IW-02
iPod & iPod Mini USB 2.0 Cable
Belkin
electronics
connector
car power adapter for iPod, white
2
11.50
1
false
regards,
Jay
On Fri, May 15, 2009 at 1:14 PM, Jay Hill wrote:
> If that is your complete input file then it looks like you are missing the
> wrapping element:
>
>
>
> F8V7067-APL-KIT >
> > field>
> > Belkin Mobile Power Cord for iPod w/ Dock
> > Belkin
> > electronics
> > connector
> > car power adapter, white
> > 4
> > 19.95
> > 1
> > false
> >
>
>
>
> Is it possible you just forgot to include the ?
>
> -Jay
>
>
> On Fri, May 15, 2009 at 12:53 PM, jayakeerthi s >wrote:
>
> > Hi All,
> >
> > I am trying to index the fileds from the xml files, here is the
> > configuration that I am using.
> >
> >
> > db-data-config.xml
> >
> >
> >
> >
> > > fileName="c:\test\ipod_other.xml" recursive="true" rootEntity="false"
> > dataSource="null" baseDir="${dataimporter.request.xmlDataDir}">
> >
> > > name="manu"/>
> >
> >
> >
> >
> >
> >
> > Schema.xml has the field "manu"
> >
> > The input xml file used to import the field is
> >
> >
> > F8V7067-APL-KIT
> > Belkin Mobile Power Cord for iPod w/ Dock
> > Belkin
> > electronics
> > connector
> > car power adapter, white
> > 4
> > 19.95
> > 1
> > false
> >
> >
> >
> > doing the full-import this is the response I am getting
> >
> > -
> > 0
> > 0
> > 0
> > 2009-05-15 11:58:00
> > Indexing completed. Added/Updated: 0 documents. Deleted 0
> > documents.
> > 2009-05-15 11:58:00
> > 2009-05-15 11:58:00
> > 0:0:0.172
> >
> > This response format is experimental. It is likely
> to
> > change in the future.
> >
> >
> >
> > Do I missing anything here or is there any format on the input xml,??
> > please
> > help resolving this.
> >
> > Thanks and regards,
> > Jay
> >
>
Re: query regarding Indexing xml files -db-data-config.xml
If that is your complete input file then it looks like you are missing the
wrapping element:
F8V7067-APL-KIT
> field>
> Belkin Mobile Power Cord for iPod w/ Dock
> Belkin
> electronics
> connector
> car power adapter, white
> 4
> 19.95
> 1
> false
>
Is it possible you just forgot to include the ?
-Jay
On Fri, May 15, 2009 at 12:53 PM, jayakeerthi s wrote:
> Hi All,
>
> I am trying to index the fileds from the xml files, here is the
> configuration that I am using.
>
>
> db-data-config.xml
>
>
>
>
> fileName="c:\test\ipod_other.xml" recursive="true" rootEntity="false"
> dataSource="null" baseDir="${dataimporter.request.xmlDataDir}">
>
> name="manu"/>
>
>
>
>
>
>
> Schema.xml has the field "manu"
>
> The input xml file used to import the field is
>
>
> F8V7067-APL-KIT
> Belkin Mobile Power Cord for iPod w/ Dock
> Belkin
> electronics
> connector
> car power adapter, white
> 4
> 19.95
> 1
> false
>
>
>
> doing the full-import this is the response I am getting
>
> -
> 0
> 0
> 0
> 2009-05-15 11:58:00
> Indexing completed. Added/Updated: 0 documents. Deleted 0
> documents.
> 2009-05-15 11:58:00
> 2009-05-15 11:58:00
> 0:0:0.172
>
> This response format is experimental. It is likely to
> change in the future.
>
>
>
> Do I missing anything here or is there any format on the input xml,??
> please
> help resolving this.
>
> Thanks and regards,
> Jay
>
query regarding Indexing xml files -db-data-config.xml
Hi All, I am trying to index the fileds from the xml files, here is the configuration that I am using. db-data-config.xml Schema.xml has the field "manu" The input xml file used to import the field is F8V7067-APL-KIT Belkin Mobile Power Cord for iPod w/ Dock Belkin electronics connector car power adapter, white 4 19.95 1 false doing the full-import this is the response I am getting - 0 0 0 2009-05-15 11:58:00 Indexing completed. Added/Updated: 0 documents. Deleted 0 documents. 2009-05-15 11:58:00 2009-05-15 11:58:00 0:0:0.172 This response format is experimental. It is likely to change in the future. Do I missing anything here or is there any format on the input xml,?? please help resolving this. Thanks and regards, Jay
Re: Regarding Indexing
Hi
You can read here and decide which strategy to adopt for maintaining
multiple indexes -
http://wiki.apache.org/solr/MultipleIndexes
I used the 1st option of 'Multiple Solr webapps' as I used single Solr
instance for indexing my 2 different modules.
- Neeti
On Fri, Aug 29, 2008 at 3:07 PM, sanraj25 <[EMAIL PROTECTED]> wrote:
>
>
> I want to store two independent datas in solr index. so I decided to create
> two index.But that's not possible.so i go for multicore concept in solr
> .can u give me step by step procedure to create multicore in solr
>
> regards,
> Santhanaraj R
>
>
>
> Norberto Meijome-2 wrote:
> >
> > On Fri, 29 Aug 2008 00:31:13 -0700 (PDT)
> > sanraj25 <[EMAIL PROTECTED]> wrote:
> >
> >> But still i cant maintain two index.
> >> please help me how to create two cores in solr
> >
> > What specific problem do you have ?
> > B
> >
> > _
> > {Beto|Norberto|Numard} Meijome
> >
> > "Always listen to experts. They'll tell you what can't be done, and why.
> > Then do it."
> > Robert A. Heinlein
> >
> > I speak for myself, not my employer. Contents may be hot. Slippery when
> > wet.
> > Reading disclaimers makes you go blind. Writing them is worse. You have
> > been
> > Warned.
> >
> >
>
> --
> View this message in context:
> http://www.nabble.com/Regarding-Indexing-tp19215093p19216746.html
> Sent from the Solr - User mailing list archive at Nabble.com.
>
>
Re: Regarding Indexing
On Fri, 29 Aug 2008 02:37:10 -0700 (PDT)
sanraj25 <[EMAIL PROTECTED]> wrote:
> I want to store two independent datas in solr index. so I decided to create
> two index.But that's not possible.so i go for multicore concept in solr
> .can u give me step by step procedure to create multicore in solr
Hi,
without specific questions, i doubt myself or others can give you any other
information than the documentation, which can be found at :
http://wiki.apache.org/solr/CoreAdmin
Please make sure you are using (a recent version of ) 1.3.
B
_
{Beto|Norberto|Numard} Meijome
Your reasoning is excellent -- it's only your basic assumptions that are wrong.
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
Re: Regarding Indexing
I want to store two independent datas in solr index. so I decided to create
two index.But that's not possible.so i go for multicore concept in solr
.can u give me step by step procedure to create multicore in solr
regards,
Santhanaraj R
Norberto Meijome-2 wrote:
>
> On Fri, 29 Aug 2008 00:31:13 -0700 (PDT)
> sanraj25 <[EMAIL PROTECTED]> wrote:
>
>> But still i cant maintain two index.
>> please help me how to create two cores in solr
>
> What specific problem do you have ?
> B
>
> _
> {Beto|Norberto|Numard} Meijome
>
> "Always listen to experts. They'll tell you what can't be done, and why.
> Then do it."
> Robert A. Heinlein
>
> I speak for myself, not my employer. Contents may be hot. Slippery when
> wet.
> Reading disclaimers makes you go blind. Writing them is worse. You have
> been
> Warned.
>
>
--
View this message in context:
http://www.nabble.com/Regarding-Indexing-tp19215093p19216746.html
Sent from the Solr - User mailing list archive at Nabble.com.
Re: Regarding Indexing
On Fri, 29 Aug 2008 00:31:13 -0700 (PDT)
sanraj25 <[EMAIL PROTECTED]> wrote:
> But still i cant maintain two index.
> please help me how to create two cores in solr
What specific problem do you have ?
B
_
{Beto|Norberto|Numard} Meijome
"Always listen to experts. They'll tell you what can't be done, and why.
Then do it."
Robert A. Heinlein
I speak for myself, not my employer. Contents may be hot. Slippery when wet.
Reading disclaimers makes you go blind. Writing them is worse. You have been
Warned.
Regarding Indexing
Hi To maintain two separate datas in solr ,I tried to create two index with the help of http://wiki.apache.org/solr/CoreAdmin http://wiki.apache.org/solr/CoreAdmin instruction.From the document I put solr.xml on the solr home directory. But still i cant maintain two index. please help me how to create two cores in solr regards, Santhanaraj R -- View this message in context: http://www.nabble.com/Regarding-Indexing-tp19215093p19215093.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Regarding Indexing data
the data is stored in Lucene format. Lucene is the place to look if you want to know the exact format. Lucene stores only "stored" fields . If you need to just index the data , the actual amount of data may be less than that of the input. --Noble On Mon, Jul 21, 2008 at 6:00 PM, sanraj25 <[EMAIL PROTECTED]> wrote: > > Hi, > I want some exact information regarding solr data. When we are > indexing the data ,they are stored in same size or compressed .Because i > want to use large size of data . so if the size reduced i can store large > data in my server. > > give some instruction to read the documents through admin of solr.Because i > sometimes want to read that documents > > regards, > Santhanaraj R > -- > View this message in context: > http://www.nabble.com/Regarding-Indexing-data-tp18567107p18567107.html > Sent from the Solr - User mailing list archive at Nabble.com. > > -- --Noble Paul
Regarding Indexing data
Hi, I want some exact information regarding solr data. When we are indexing the data ,they are stored in same size or compressed .Because i want to use large size of data . so if the size reduced i can store large data in my server. give some instruction to read the documents through admin of solr.Because i sometimes want to read that documents regards, Santhanaraj R -- View this message in context: http://www.nabble.com/Regarding-Indexing-data-tp18567107p18567107.html Sent from the Solr - User mailing list archive at Nabble.com.
