[Casting] values on update/csv

[Adam Estrada]

All,

I have a csv file and I want to store one of the fields as a tdouble type.
It does not like that at all...Is there a way to cast the string value to a
tdouble?

Thanks,
Adam

Re: [Casting] values on update/csv

[Markus Jelsma]

Should be no problem but please paste the log output etc.

 All,
 
 I have a csv file and I want to store one of the fields as a tdouble type.
 It does not like that at all...Is there a way to cast the string value to a
 tdouble?
 
 Thanks,
 Adam

Re: [Casting] values on update/csv

[Adam Estrada]

Hi,

I am using curl to run the following and as soon as I convert the field type
from string to tdouble, I get the errors you see below.

0:0:0:0:0:0:0:1 -  -  [08/12/2010:23:28:27 +] GET
/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,name,asciiname,lat,lng,countrycode,population,elevation,gtopo30,timezone,modificationdate,catstream.file=C:\tmp\allCountries\xaa.csvoverwrite=truestream.contentType=text/plain;charset=utf-8
HTTP/1.1 500 4023

I am trying to index coordinates in decimal degrees so many of them have
negative values. Could this be the problem?


Dec 8, 2010 6:28:27 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NumberFormatException: For input string: lat
at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
at java.lang.Double.parseDouble(Unknown Source)
at org.apache.solr.schema.TrieField.createField(TrieField.java:431)
at
org.apache.solr.schema.SchemaField.createField(SchemaField.java:94)
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.jav
a:246)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpd
ateProcessorFactory.java:60)
at
org.apache.solr.handler.CSVLoader.doAdd(CSVRequestHandler.java:386)
at
org.apache.solr.handler.SingleThreadedCSVLoader.addDoc(CSVRequestHand
ler.java:400)
at
org.apache.solr.handler.CSVLoader.load(CSVRequestHandler.java:363)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
ntentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:131)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle
Request(RequestHandlers.java:233)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
at
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter
.java:338)
at
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilte
r.java:241)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(Servlet
Handler.java:1089)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:3
65)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.jav
a:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:1
81)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:7
12)
at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405)

at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHand
lerCollection.java:211)
at
org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.
java:114)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:1
39)
at org.mortbay.jetty.Server.handle(Server.java:285)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:50
2)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpCo
nnection.java:821)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:513)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at
org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.
java:226)
at
org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool
.java:442)

Dec 8, 2010 6:28:27 PM org.apache.solr.core.SolrCore execute
INFO: [] webapp=/solr path=/update/csv
params={fieldnames=id,name,asciiname,lat,
lng,countrycode,population,elevation,gtopo30,timezone,modificationdate,catcommi
t=trueoverwrite=truestream.contentType=text/plain;charset%3Dutf-8separator=,
stream.file=C:\tmp\allCountries\xaa.csv} status=500 QTime=52
Dec 8, 2010 6:28:27 PM org.apache.solr.common.SolrException log
SEVERE: java.lang.NumberFormatException: For input string: lat
at sun.misc.FloatingDecimal.readJavaFormatString(Unknown Source)
at java.lang.Double.parseDouble(Unknown Source)
at org.apache.solr.schema.TrieField.createField(TrieField.java:431)
at
org.apache.solr.schema.SchemaField.createField(SchemaField.java:94)
at
org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.jav
a:246)
at
org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpd
ateProcessorFactory.java:60)
at
org.apache.solr.handler.CSVLoader.doAdd(CSVRequestHandler.java:386)
at
org.apache.solr.handler.SingleThreadedCSVLoader.addDoc(CSVRequestHand
ler.java:400)
at
org.apache.solr.handler.CSVLoader.load(CSVRequestHandler.java:363)
at
org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(Co
ntentStreamHandlerBase.java:54)
at
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandl
erBase.java:131)
at
org.apache.solr.core.RequestHandlers$LazyRequestHandlerWrapper.handle

Re: [Casting] values on update/csv

[Chris Hostetter]

: I am using curl to run the following and as soon as I convert the field type
: from string to tdouble, I get the errors you see below.
: 
: 0:0:0:0:0:0:0:1 -  -  [08/12/2010:23:28:27 +] GET
: 
/solr/update/csv?commit=trueseparator=%2Cfieldnames=id,name,asciiname,lat,lng,countrycode,population,elevation,gtopo30,timezone,modificationdate,catstream.file=C:\tmp\allCountries\xaa.csvoverwrite=truestream.contentType=text/plain;charset=utf-8
: HTTP/1.1 500 4023
: 
: I am trying to index coordinates in decimal degrees so many of them have
: negative values. Could this be the problem?
...
: Dec 8, 2010 6:28:27 PM org.apache.solr.common.SolrException log
: SEVERE: java.lang.NumberFormatException: For input string: lat

the problem is you are trying to index the literal string lat as a 
numberic value.

Check your CSV file and look for that string.

(the CSV Handler, by default, assumes the first line of hte file contains 
column names, but it looks like in your case perhaps there is another line 
in the file that also contains the column names? ... just a hunch)




-Hoss