Re: Getting started with writing parser
On Tue, Jan 25, 2011 at 3:46 PM, Dinesh wrote: > > no i actually changed the directory to mine where i stored the log files.. it > is /home/exam/apa..solr/example/exampledocs > > i specified it in a solr schema.. i created an DataImportHandler for that in > try.xml.. then in that i changed that file name to sample.txt > > that new try.xml is > http://pastebin.com/pfVVA7Hs [...] Let us take this one part at a time. In your inner nested entity,
Re: Getting started with writing parser
no i actually changed the directory to mine where i stored the log files.. it is /home/exam/apa..solr/example/exampledocs i specified it in a solr schema.. i created an DataImportHandler for that in try.xml.. then in that i changed that file name to sample.txt that new try.xml is http://pastebin.com/pfVVA7Hs i changed the log into one word per line thinking there might be error in my regex expression.. now i'm completely stuck.. - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2327920.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Tue, Jan 25, 2011 at 11:44 AM, Dinesh wrote: > > i don't even know whether the regex expression that i'm using for my log is > correct or no.. If it is the same try.xml that you posted earlier, it is very likely not going to work. You seem to have just cut and pasted entries from the Hathi Trust blog, without understanding how they work. Could you take a fresh look at http://wiki.apache.org/solr/DataImportHandler and explain in words the following: * What is your directory structure for storing the log files? * What parts of the log file do you want to keep (you have already explained this in another message)? * How would the above translate into: - A Solr schema - Setting up (a) a data source, (b) processor(s), and (c) transformers. >i very much worried i couldn't proceed in my > project already > 1/3 rd of the timing is over.. please help.. this is just the first stage.. > after this i have ti setup up all the log to be redirected to SYSLOG and > from there i'll send it to SOLR server.. then i have to analyse all the > data's that i obtained from DNS, DHCP, WIFI, SWITCES.. and i have to prepare > a user based report on his actions.. please help me cause the day's i have > keeps reducing.. my project leader is questioning me a lot.. pls.. [...] Well, I am sorry, but at least I strongly feel that we should not be doing your work for you, and especially not if it is a student project, as seems to be the case. If you can address the above points one by one (stay on this thread, please), people should be able to help you. However, it is up to you to get to understand Solr well enough. Regards, Gora
Re: Getting started with writing parser
i want to take the month, time, DHCPMESSAGE, from_mac, gateway_ip, net_ADDR - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2327738.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Tue, Jan 25, 2011 at 10:05 AM, Dinesh wrote: > > http://pastebin.com/CkxrEh6h > > this is my sample log [...] And, which portions of the log text do you want to preserve? Does it go into Solr as a single error message, or do you want to separate out parts of it. Regards, Gora
Re: Getting started with writing parser
i don't even know whether the regex expression that i'm using for my log is correct or no.. i very much worried i couldn't proceed in my project already 1/3 rd of the timing is over.. please help.. this is just the first stage.. after this i have ti setup up all the log to be redirected to SYSLOG and from there i'll send it to SOLR server.. then i have to analyse all the data's that i obtained from DNS, DHCP, WIFI, SWITCES.. and i have to prepare a user based report on his actions.. please help me cause the day's i have keeps reducing.. my project leader is questioning me a lot.. pls.. - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2326917.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
http://pastebin.com/CkxrEh6h this is my sample log - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2326646.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Mon, Jan 24, 2011 at 2:28 PM, Dinesh wrote: > > my solrconfig.xml > > http://pastebin.com/XDg0L4di > > my schema.xml > > http://pastebin.com/3Vqvr3C0 > > my try.xml > > http://pastebin.com/YWsB37ZW [...] OK, thanks for the above. You also need to: * Give us a sample of your log files (for crying out loud, this has got to be the fifth time that I have asked you for this). * Tell us what happens when you run with the above configuration. From a cursory look at try.xml, you have not really understood how it works, or how to configure it for your needs. Regards, Gora
Re: Getting started with writing parser
my solrconfig.xml http://pastebin.com/XDg0L4di my schema.xml http://pastebin.com/3Vqvr3C0 my try.xml http://pastebin.com/YWsB37ZW - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2318218.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Mon, Jan 24, 2011 at 11:54 AM, Dinesh wrote: > > i did all the configurations correctly.. previously i missed a configuration > file Sorry, what are you trying to configure now? The built-in Solr example, or the setup for your log files? Did you get the built-in Solr example to work? How were things working earlier that you were getting Solr running, but facing an error on POST. Please proceed systematically, and do not jump back and forth between steps. >after adding it i'm getting a new > error called > > Unknown FieldType: 'string' used in > QueryElevationComponent > > i found it was defined in solrconfig.xml [...] Please make your schema.xml, and solrconfig.xml available on the web somewhere, say on http://pastebin.com/ . Regards, Gora P.S. I will not be in network connectivity from now till late tonight, but others might be able to help in the meantime.
Re: Getting started with writing parser
i did all the configurations correctly.. previously i missed a configuration file after adding it i'm getting a new error called Unknown FieldType: 'string' used in QueryElevationComponent i found it was defined in solrconfig.xml i didn't change any of the line in that but i don't know why am i getting error - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2317618.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Mon, Jan 24, 2011 at 11:18 AM, Dinesh wrote: > > i tried those examples.. is it compuslory that i should make it into XML, how > does it index CSV.. You will have to convert either into XML, or CSV, but neither of those should be too difficult. > should i post my entire schema that i made it > myself and > the text file that i tried to index.. Post the schema. How big is the text file? If it is more than, say 50 lines, put it up on the web somewhere, and post a link to it. If you are going to do that for the text file, also do it for the schema, and post links to both. Regards, Gora
Re: Getting started with writing parser
i tried those examples.. is it compuslory that i should make it into XML, how does it index CSV.. should i post my entire schema that i made it myself and the text file that i tried to index.. - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2317521.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Mon, Jan 24, 2011 at 10:47 AM, Dinesh wrote: > > i tried editing the schema file and indexing my own log.. the error that i > got is > > root@karunya-desktop:/home/karunya/apache-solr-1.4.1/example/exampledocs# > java -jar post.jar sample.txt > SimplePostTool: version 1.2 > SimplePostTool: WARNING: Make sure your XML documents are encoded in UTF-8, > other encodings are not currently supported > SimplePostTool: POSTing files to http://localhost:8983/solr/update.. > SimplePostTool: POSTing file sample.txt > SimplePostTool: FATAL: Solr returned an error: [...] Most likely, you are trying to send a plain text file to Solr, instead of the XML that it is expecting. Please see http://lucene.apache.org/solr/tutorial.html#Indexing+Data for an example of how to index XML files to Solr via a POST. That references files in example/exampledocs/ in your Solr source code directory that can serve as examples. E.g., see example/exampledocs/solr.xml. You can try first to see that you can get the built-in Solr examples running, by following the instructions from the beginning of http://lucene.apache.org/solr/tutorial.html. Once, that is done, and if you describe the format of your log files, and what data you want to retain from them, people can help you further. Regards, Gora
Re: Getting started with writing parser
i tried editing the schema file and indexing my own log.. the error that i got is root@karunya-desktop:/home/karunya/apache-solr-1.4.1/example/exampledocs# java -jar post.jar sample.txt SimplePostTool: version 1.2 SimplePostTool: WARNING: Make sure your XML documents are encoded in UTF-8, other encodings are not currently supported SimplePostTool: POSTing files to http://localhost:8983/solr/update.. SimplePostTool: POSTing file sample.txt SimplePostTool: FATAL: Solr returned an error: Severe_errors_in_solr_configuration__Check_your_log_files_for_more_detailed_information_on_what_may_be_wrong__If_you_want_solr_to_continue_after_configuration_errors_changeabortOnConfigurationErrorfalseabortOnConfigurationError__in_null___orgapachesolrcommonSolrException_Unknown_fieldtype_text_specified_on_field_month__at_orgapachesolrschemaIndexSchemareadSchemaIndexSchemajava477__at_orgapachesolrschemaIndexSchemainitIndexSchemajava95__at_orgapachesolrcoreSolrCoreinitSolrCorejava520__at_orgapachesolrcoreCoreContainer$InitializerinitializeCoreContainerjava137__at_orgapachesolrservletSolrDispatchFilterinitSolrDispatchFilterjava83__at_orgmortbayjettyservletFilterHolderdoStartFilterHolderjava99__at_orgmortbaycomponentAbstractLifeCyclestartAbstractLifeCyclejava40__at_orgmortbayjettyservletServletHandlerinitializeServletHandlerjava594__at_orgmortbayjettyservletContextstartContextContextjava139__at_orgmortbayjettywebappWebAppContextstartContextWebAppContextjava1218__at_orgmortbayjettyhandlerContextHandlerdoStartContextHandlerjava500__at_orgmortbayjettywebappWebAppContextdoStartWebAppContextjava448__at_orgmortbaycomponentAbstractLifeCyclestartAbstractLifeCyclejava40__at_orgmortbayjettyhandlerHandlerCollectiondoStartHandlerCollectionjava147__at_orgmortbayjettyhandlerContextHandlerCollectiondoStartContextHandlerCollectionjava161__at_orgmortbaycomponentAbstractLifeCyclestartAbstractLifeCyclejava40__at_orgmortbayjettyhandlerHandlerCollectiondoStartHandlerCollectionjava147__at_orgmortbaycomponentAbstractLifeCyclestartAbstractLifeCyclejava40__at_orgmortbayjettyhandlerHandlerWrapperdoStartHandlerWrapperjava117__at_orgmortbayjettyServerdoStartServerjava210__at_orgmortbaycomponentAbstractLifeCyclestartAbstractLifeCyclejava40__at_orgmortbayxmlXmlConfigurationmain please help me solve this - DINESHKUMAR . M I am neither especially clever nor especially gifted. I am only very, very curious. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2317421.html Sent from the Solr - User mailing list archive at Nabble.com.
Re: Getting started with writing parser
On Tue, Jan 18, 2011 at 11:59 AM, Dinesh wrote: > > how to write a parser program that will convert log files into XML.. [...] There is no point to starting multiple threads on this issue, hoping that someone will somehow solve your problem. You have been given the following: * Links that should help you get started, including an example of someone indexing Solr's own logs. * Some ideas on how to proceed. * Requests to try the above suggestions out, and ask specific questions when you run into issues. * A suggestion to contact a local expert in Solr. * Multiple requests for a sample of your log files. Please show some signs that you have tried the above suggestions. Otherwise, I am afraid that it will be difficult, if not impossible. for people on this list to help you out. Regards, Gora
Getting started with writing parser
how to write a parser program that will convert log files into XML.. -- View this message in context: http://lucene.472066.n3.nabble.com/Getting-started-with-writing-parser-tp2278092p2278092.html Sent from the Solr - User mailing list archive at Nabble.com.
