for reference implementation to Sanitize Unknown SolrFields, you can see below link
https://github.com/cloudera/cdk/blob/master/cdk-morphlines/cdk-morphlines-solr-core/src/main/java/com/cloudera/cdk/morphline/solr/SanitizeUnknownSolrFieldsBuilder.java On Sat, Aug 2, 2014 at 8:24 PM, Umesh Prasad <umesh.i...@gmail.com> wrote: > Solr schema over REST https://wiki.apache.org/solr/SchemaRESTAPI > > https://cwiki.apache.org/confluence/display/solr/Schema+API > > You can use that for getting required fields and validate at client side .. > > > > > > On 31 July 2014 14:32, Liram Vardi <lir...@checkpoint.com> wrote: > > > Hi Jack, > > Thank you for your reply. > > This is the Solr stack trace. As you can see, the missing field is > > "hourOfDay". > > > > Thanks, > > Liram > > > > 2014-07-30 14:27:54,934 ERROR [qtp-608368492-19] (SolrException.java:108) > > - org.apache.solr.common.SolrException: > > [doc=53b16126-0000-0002-2b03-17ac4d4a07b6] missing required field: > hourOfDay > > at > > > org.apache.solr.update.DocumentBuilder.toDocument(DocumentBuilder.java:189) > > at > > > org.apache.solr.update.AddUpdateCommand.getLuceneDocument(AddUpdateCommand.java:73) > > at > > > org.apache.solr.update.DirectUpdateHandler2.addDoc(DirectUpdateHandler2.java:210) > > at > > > org.apache.solr.update.processor.RunUpdateProcessor.processAdd(RunUpdateProcessorFactory.java:69) > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor.doLocalAdd(DistributedUpdateProcessor.java:556) > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor.versionAdd(DistributedUpdateProcessor.java:692) > > at > > > org.apache.solr.update.processor.DistributedUpdateProcessor.processAdd(DistributedUpdateProcessor.java:435) > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) > > at > > > org.apache.solr.update.processor.AbstractDefaultValueUpdateProcessorFactory$DefaultValueUpdateProcessor.processAdd(AbstractDefaultValueUpdateProcessorFactory.java:94) > > at > > > org.apache.solr.update.processor.UpdateRequestProcessor.processAdd(UpdateRequestProcessor.java:51) > > at > > > com.checkpoint.solr_plugins.MulticoreUpdateRequestProcessor.processAdd(MulticoreUpdateRequestProcessorFactory.java:152) > > at > > > org.apache.solr.handler.loader.XMLLoader.processUpdate(XMLLoader.java:246) > > at > > org.apache.solr.handler.loader.XMLLoader.load(XMLLoader.java:173) > > at > > > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904) > > at > > > com.checkpoint.solr_plugins.MulticoreUpdateRequestProcessor.processAdd(MulticoreUpdateRequestProcessorFactory.java:248) > > at > > > org.apache.solr.handler.loader.JavabinLoader$1.update(JavabinLoader.java:86) > > at > > > org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readOuterMostDocIterator(JavaBinUpdateRequestCodec.java:143) > > at > > > org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readIterator(JavaBinUpdateRequestCodec.java:123) > > at > > org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:220) > > at > > > org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec$1.readNamedList(JavaBinUpdateRequestCodec.java:108) > > at > > org.apache.solr.common.util.JavaBinCodec.readVal(JavaBinCodec.java:185) > > at > > org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:111) > > at > > > org.apache.solr.client.solrj.request.JavaBinUpdateRequestCodec.unmarshal(JavaBinUpdateRequestCodec.java:150) > > at > > > org.apache.solr.handler.loader.JavabinLoader.parseAndLoadDocs(JavabinLoader.java:96) > > at > > org.apache.solr.handler.loader.JavabinLoader.load(JavabinLoader.java:55) > > at > > > org.apache.solr.handler.UpdateRequestHandler$1.load(UpdateRequestHandler.java:92) > > at > > > org.apache.solr.handler.ContentStreamHandlerBase.handleRequestBody(ContentStreamHandlerBase.java:74) > > at > > > org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135) > > at org.apache.solr.core.SolrCore.execute(SolrCore.java:1904) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:659) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:362) > > at > > > org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:158) > > at > > > org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1474) > > at > > > org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:499) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:137) > > at > > > org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:557) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:231) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1086) > > at > > org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:428) > > at > > > org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:193) > > at > > > org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1020) > > at > > > org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:135) > > at > > > org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:255) > > at > > > org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:154) > > at > > > org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:116) > > at org.eclipse.jetty.server.Server.handle(Server.java:370) > > at > > > org.eclipse.jetty.server.AbstractHttpConnection.handleRequest(AbstractHttpConnection.java:489) > > at > > > org.eclipse.jetty.server.BlockingHttpConnection.handleRequest(BlockingHttpConnection.java:53) > > at > > > org.eclipse.jetty.server.AbstractHttpConnection.content(AbstractHttpConnection.java:960) > > at > > > org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler.content(AbstractHttpConnection.java:1021) > > at > org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:957) > > > > -----Original Message----- > > From: Jack Krupansky [mailto:j...@basetechnology.com] > > Sent: Wednesday, July 30, 2014 5:53 PM > > To: solr-user@lucene.apache.org > > Subject: Re: Identify specific document insert error inside a solrj batch > > request > > > > Agreed that this is a problem with Solr. If it was merely "bad input", > > Solr should be returning a 4xx error. > > > > I don't know if we already have a Jira for this. If not, one should be > > filed. > > > > There are two issues: > > > > 1. The status code should be 4xx with an appropriate message about "bad > > input". > > > > 2. The offset of the offending document should be reported so that the > app > > can locate the problem to resolve it. > > > > Give us the actual server stack trace so we can verify whether this was > > simply "user error" or some defect in Solr itself. > > > > -- Jack Krupansky > > > > -----Original Message----- > > From: Liram Vardi > > Sent: Wednesday, July 30, 2014 9:25 AM > > To: solr-user@lucene.apache.org > > Subject: Identify specific document insert error inside a solrj batch > > request > > > > Hi All, > > > > I have a question regarding the use of HttpSolrServer (SolrJ). > > I have a collection of SolrInputDocuments I want to send to Solr as a > > batch. > > Now, let's assume that one of the docs inside this collection is > corrupted > > (missing some "required" field). > > When I send the batch of docs to solr using > HttpSolrServer.add(Collection< > > SolrInputDocument> docs) I am getting the following general exception: > > > > "org.apache.solr.client.solrj.impl.HttpSolrServer$RemoteSolrException: > > Server at http://172.23.3.91:8210/solr/template returned non ok > > status:500, > > message:Server Error" > > > > When I check Solr log, I can identify exactly which is the corrupted > > document. > > > > My question: > > Is it possible to identify the problematic document at the client side? > > (for > > recovery purposes) > > > > Thanks, > > Liram > > > > > > Email secured by Check Point > > > > > > -- > --- > Thanks & Regards > Umesh Prasad >
