Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[gpssolr2020]

Thanks. We will try with more heap.

And we noticed that zookeeper(open jdk) and Solr(sun jdk) is using different
jvm. Will this really cause this OOM issue ?.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-exception-while-deleting-30k-s-tp4109259p4110538.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[Yago Riveiro]

You have your GC tuned? 

In the pass I had a lot of problems with zookeeper as a result of GC pauses 
because my heap was too big.

Increase your heap to 20G or more, and use some of the configurations exposed 
in this thread http://wiki.apache.org/solr/ShawnHeisey

The first works fine for me with a heap with 24G. 

-- 
Yago Riveiro
Sent with Sparrow (http://www.sparrowmailapp.com/?sig)


On Tuesday, January 7, 2014 at 3:54 PM, gpssolr2020 wrote:

> Thanks all for the reply.
> 
> we are working on that to reduce the delete query size.
> 
> But after that we faced one more issue.The issue is our batch process is
> able to delete 16k records but we got an OOM exception in one server.(out of
> 4 server in solrcloud).We are using solr 4.2 and zookeeper 3.4.5. and we are
> using sunjdk. we increased the heap size from 12 to 16 GB but no luck.we
> have 250GB ram dedicated only for solr and we have 100 million records.
> 
> we have We are getting the following error during the deletion process.
> 
> 
> org.apache.solr.common.SolrException; Error opening realtime searcher for
> deleteByQuery:java.lang.OutOfMemoryError: Java heap space
> 
> 
> org.apache.solr.common.SolrException; Error while trying to recover.
> core=pe_shard1_replica1:java.lang.IllegalStateException: this writ
> er hit an OutOfMemoryError; cannot commit
> 
> org.apache.solr.common.SolrException; auto commit
> error...:java.lang.IllegalStateException: this writer hit an
> OutOfMemoryError; cannot
> commit
> 
> 
> ERROR - org.apache.solr.common.SolrException;
> org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are
> disabled.
> 
> 
> Note:Initially we got the same error when we index more than one million
> record at a time.So to fix that we divided data into two chunks and it is
> working fine now.
> 
> Please advise ..
> 
> 
> Thanks...
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-exception-while-deleting-30k-s-tp4109259p4109993.html
> Sent from the Solr - User mailing list archive at Nabble.com 
> (http://Nabble.com).
> 
> 

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[gpssolr2020]

Thanks all for the reply.

we are working on that to reduce the delete query size.

But after that we faced one more issue.The issue is our batch process is
able to delete 16k records but we got an OOM exception in one server.(out of
4 server in solrcloud).We are using solr 4.2 and zookeeper 3.4.5. and we are
using sunjdk. we increased the heap size from 12 to 16 GB but no luck.we
have 250GB ram dedicated only for solr and we have 100 million records.

we have We are getting the following error during the deletion process.


org.apache.solr.common.SolrException; Error opening realtime searcher for
deleteByQuery:java.lang.OutOfMemoryError: Java heap space


 org.apache.solr.common.SolrException; Error while trying to recover.
core=pe_shard1_replica1:java.lang.IllegalStateException: this writ
er hit an OutOfMemoryError; cannot commit

 org.apache.solr.common.SolrException; auto commit
error...:java.lang.IllegalStateException: this writer hit an
OutOfMemoryError; cannot
 commit


ERROR -  org.apache.solr.common.SolrException;
org.apache.solr.common.SolrException: Cannot talk to ZooKeeper - Updates are
disabled.


Note:Initially we got the same error when we index more than one million
record at a time.So to fix that we divided data into two chunks and it is
working fine now.

Please advise ..


Thanks...



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-exception-while-deleting-30k-s-tp4109259p4109993.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[Mark Miller]

Just an FYI, newer version of Solr will deploy the proper error message rather 
than that cryptic one.

- Mark

On Jan 3, 2014, at 12:54 AM, Shawn Heisey  wrote:

> On 1/2/2014 10:22 PM, gpssolr2020 wrote:
>> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60)
>> or the data in not in 'javabin' format
> 
> 
> 
>> (Account:123+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
>> (Account:345+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
>> (Account:569+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
>> (Account:789+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+..+OR+30k
>> th record)
>> 
>> But we are able to delete small number of records without any issues.
>> 
>> Can anyone please help us on this.
> 
> Walter is right about the error message.  This indicates that Solr is
> returning an error response that's in HTML or XML format rather than the
> expected javabin.  The server log should actually contain more
> information about what actually went wrong.
> 
> This is probably happening because the request is too big.  In my
> indexing code, I send deleteByQuery requests in batches of 1000 for this
> very reason.
> 
> If that date range is the same for all of the deletes in a batch, you
> could make the request considerably smaller by using the following format:
> 
> DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"] AND Account:(123
> OR 345 OR 569 OR 789 OR  3th-value)
> 
> The other option is to increase the max form size of your servlet
> container.  In 4.1 or later, this is actually controlled by Solr
> directly, not the container config, with the formdataUploadLimitInKB
> attribute on the requestParsers tag in solrconfig.xml.  The default is
> 2048, or 2MB.  With 3 clauses like you have described, the request
> will be a little bit more than 2MB.
> 
> There is a bug in the early Jetty 8 versions (that included with Solr
> 4.0) that prevented the form size from being set in the jetty config:
> 
> https://bugs.eclipse.org/bugs/show_bug.cgi?id=397130
> 
> The Jetty 6 that is included with Solr 1.x and 3.x can be properly
> configured.
> 
> Thanks,
> Shawn
> 

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[Shawn Heisey]

On 1/2/2014 10:22 PM, gpssolr2020 wrote:
> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60)
> or the data in not in 'javabin' format



> (Account:123+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:345+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:569+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:789+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+..+OR+30k
> th record)
> 
> But we are able to delete small number of records without any issues.
> 
> Can anyone please help us on this.

Walter is right about the error message.  This indicates that Solr is
returning an error response that's in HTML or XML format rather than the
expected javabin.  The server log should actually contain more
information about what actually went wrong.

This is probably happening because the request is too big.  In my
indexing code, I send deleteByQuery requests in batches of 1000 for this
very reason.

If that date range is the same for all of the deletes in a batch, you
could make the request considerably smaller by using the following format:

DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"] AND Account:(123
OR 345 OR 569 OR 789 OR  3th-value)

The other option is to increase the max form size of your servlet
container.  In 4.1 or later, this is actually controlled by Solr
directly, not the container config, with the formdataUploadLimitInKB
attribute on the requestParsers tag in solrconfig.xml.  The default is
2048, or 2MB.  With 3 clauses like you have described, the request
will be a little bit more than 2MB.

There is a bug in the early Jetty 8 versions (that included with Solr
4.0) that prevented the form size from being set in the jetty config:

https://bugs.eclipse.org/bugs/show_bug.cgi?id=397130

The Jetty 6 that is included with Solr 1.x and 3.x can be properly
configured.

Thanks,
Shawn

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format exception while deleting 30k records

[Walter Underwood]

60 in ASCII is '<'. Is it returning XML? Or maybe an error message?

wunder

On Jan 2, 2014, at 9:22 PM, gpssolr2020  wrote:

> Hi,
> 
> We are getting the  below error message while trying to delete 30k records
> from solr.
> 
> Error occured while invoking endpoint on Solr: 
> org.apache.solr.client.solrj.SolrServerException: Error executing query
>   at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:98)
>   at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:36)
>   at
> com.ntrs.wpf.search.batch.BatchProcessorImpl.run(BatchProcessorImpl.java:57)
>   at
> com.ntrs.wpf.search.batch.SolrIndexingController.purgePositions(SolrIndexingController.java:235)
>   at
> com.ntrs.wpf.search.batch.SolrIndexingController.execute(SolrIndexingController.java:106)
>   at
> com.ntrs.wpf.search.batch.BatchFrameworkManager.main(BatchFrameworkManager.java:63)
>   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>   at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
>   at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:616)
>   at
> org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoader.java:56)
> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60)
> or the data in not in 'javabin' format
>   at
> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:109)
>   at
> org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
>   at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:387)
>   at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:181)
>   at
> org.apache.solr.client.solrj.request.QueryRequest.process(QueryRequest.java:90)
>   ... 10 more
> 
> 
> we are forming the delete query as below.
> 
> (Account:123+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:345+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:569+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+
> (Account:789+AND+DATE:["2013-11-29T00:00:00Z"+TO+"2013-11-29T23:59:59Z"])+OR+..+OR+30k
> th record)
> 
> But we are able to delete small number of records without any issues.
> 
> Can anyone please help us on this.
> 
> 
> 
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-exception-while-deleting-30k-s-tp4109259.html
> Sent from the Solr - User mailing list archive at Nabble.com.

--
Walter Underwood
wun...@wunderwood.org

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Erick Erickson]

The process looks like this:

each shard returns the top 100K documents
(actually the doc ID and whatever your
sort criteria is, often just the score).
_from every shard_ and the node that
distributes that request then takes those
900K items and merges the list to get the
100K that satisfy the request. Then the
node sends out a second request to each
shard to get the actual document, perhaps
asking for all 100K from a single node.

As Walter and Shalin say, don't do this. Solr
isn't built to return massive result sets.
Performance will suffer and you'll run into
"interesting" limits like this.

Why do you want to do this in the first place?

Best
Erick


On Fri, Jul 5, 2013 at 10:26 AM, eakarsu  wrote:

> I am using Solr 4.3.1 on solrcloud with 10 nodes.
>
> I added 3 million documents from a csv file with this command
>
> curl
> '
> http://localhost:8080/solr/trcollection2/update/csv?stream.file=/home/hduser/csvFile.csv&skipLines=1&fieldnames=,cache,segment,digest,tstamp,lang,url,,content,id,title,boost&stream.contentType=text/p
> lain;charset=utf-8'
>
> Then I query the data, fetching first 100K documents with this. But I am
> getting "Invalid version (expected 2, but 60) or the data in not in
> 'javabin' format" error. I have appended what I got in output file
> "alltrcollection_309mil.csv"
>
> I appreciate if you can help me on this
>
> hduser@host1:~$ curl -o alltrcollection_309mil.csv
> '
> http://localhost:8080/solr/trcollection2/select?q=*%3A*&rows=10&wt=xml&indent=true
> '
>   % Total% Received % Xferd  Average Speed   TimeTime Time
> Current
>  Dload  Upload   Total   SpentLeft
> Speed
> 100  3439 0 3439 0 0103  0 --:--:--  0:00:33 --:--:--
> 1049
> hduser@host1:~$ more alltrcollection_309mil.csv
> 
> 
>
> 
>   500
>   33307
>   
> true
> *:*
> xml
> 10
>   
> 
> 
>   java.lang.RuntimeException: Invalid version (expected 2,
> but 60) or the data in not in 'javabin' format
>   org.apache.solr.common.SolrException:
> java.lang.RuntimeException: Invalid version (expected 2, but 60) or the
> data
> in not in 'javabin' format
> at
>
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:302)
> at
>
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1820)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
> at
>
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
>
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
>
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
>
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
>
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
>
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
> at
>
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at
>
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
> at
>
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
>
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at
>
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
>
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60)
> or the data in not in 'javabin' format
> at
> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:109)
> at
>
> org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
> at
>
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at
>
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
> at
>
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
>  

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Walter Underwood]

Since it works to fetch 10K rows and doesn't work to fetch 100K rows in a 
single request, I very strongly suggest that you use the request that work. 
Make ten requests of 10K rows each. Or even better, 100 requests of 1K rows 
each.
Large requests make large memory demands. 

wunder

On Jul 5, 2013, at 7:58 AM, Shalin Shekhar Mangar wrote:

> Oops I actually meant to say that search engines *are not* optimized
> for large pages. See https://issues.apache.org/jira/browse/SOLR-1726
> 
> Well one of the shards involved in the request is throwing an error.
> Check the logs of your shards. You can also add a shards.info=true
> param to your search which should return the responses of each shard.
> 
> On Fri, Jul 5, 2013 at 8:18 PM, eakarsu  wrote:
>> Thanks for your answer,
>> 
>> I can fetch 10K documents without any issue. I don't think we are having out
>> of memory exception because each tomcat server in cluster has 8GB memory
>> allocated.
>> 

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Shalin Shekhar Mangar]

Oops I actually meant to say that search engines *are not* optimized
for large pages. See https://issues.apache.org/jira/browse/SOLR-1726

Well one of the shards involved in the request is throwing an error.
Check the logs of your shards. You can also add a shards.info=true
param to your search which should return the responses of each shard.

On Fri, Jul 5, 2013 at 8:18 PM, eakarsu  wrote:
> Thanks for your answer,
>
> I can fetch 10K documents without any issue. I don't think we are having out
> of memory exception because each tomcat server in cluster has 8GB memory
> allocated.
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-tp4075739p4075749.html
> Sent from the Solr - User mailing list archive at Nabble.com.



-- 
Regards,
Shalin Shekhar Mangar.

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[eakarsu]

Thanks for your answer,

I can fetch 10K documents without any issue. I don't think we are having out
of memory exception because each tomcat server in cluster has 8GB memory
allocated.



--
View this message in context: 
http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-tp4075739p4075749.html
Sent from the Solr - User mailing list archive at Nabble.com.

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Shalin Shekhar Mangar]

Can you try to fetch a smaller number of documents? Search engines are
optimized for returning large pages. My guess is that one of the
shards is returning an error (maybe an OutOfMemoryError) for this
query.

On Fri, Jul 5, 2013 at 7:56 PM, eakarsu  wrote:
> I am using Solr 4.3.1 on solrcloud with 10 nodes.
>
> I added 3 million documents from a csv file with this command
>
> curl
> 'http://localhost:8080/solr/trcollection2/update/csv?stream.file=/home/hduser/csvFile.csv&skipLines=1&fieldnames=,cache,segment,digest,tstamp,lang,url,,content,id,title,boost&stream.contentType=text/p
> lain;charset=utf-8'
>
> Then I query the data, fetching first 100K documents with this. But I am
> getting "Invalid version (expected 2, but 60) or the data in not in
> 'javabin' format" error. I have appended what I got in output file
> "alltrcollection_309mil.csv"
>
> I appreciate if you can help me on this
>
> hduser@host1:~$ curl -o alltrcollection_309mil.csv
> 'http://localhost:8080/solr/trcollection2/select?q=*%3A*&rows=10&wt=xml&indent=true'
>   % Total% Received % Xferd  Average Speed   TimeTime Time
> Current
>  Dload  Upload   Total   SpentLeft
> Speed
> 100  3439 0 3439 0 0103  0 --:--:--  0:00:33 --:--:--
> 1049
> hduser@host1:~$ more alltrcollection_309mil.csv
> 
> 
>
> 
>   500
>   33307
>   
> true
> *:*
> xml
> 10
>   
> 
> 
>   java.lang.RuntimeException: Invalid version (expected 2,
> but 60) or the data in not in 'javabin' format
>   org.apache.solr.common.SolrException:
> java.lang.RuntimeException: Invalid version (expected 2, but 60) or the data
> in not in 'javabin' format
> at
> org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:302)
> at
> org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:135)
> at org.apache.solr.core.SolrCore.execute(SolrCore.java:1820)
> at
> org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:656)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:359)
> at
> org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:155)
> at
> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
> at
> org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
> at
> org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
> at
> org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
> at
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
> at
> org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
> at
> org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:931)
> at
> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
> at
> org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:407)
> at
> org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1004)
> at
> org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
> at
> org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> at java.lang.Thread.run(Thread.java:722)
> Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60)
> or the data in not in 'javabin' format
> at
> org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:109)
> at
> org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
> at
> org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:180)
> at
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:156)
> at
> org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:119)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at
> java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
> at java.util.concurrent.FutureTask.run(FutureTask.java:166)
> ... 3 more
> 
>   500
> 
> 
>
>
>
> --
> View this message in context: 
> http://lucene.472066.n3.nabble.com/Invalid-version-expected-2-but-60-or-the-data-in-not-in-javabin-format-tp4075739.html
> S

RE: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Sergiu Bivol]

Adding the original message.

Thank you
Sergiu

-Original Message-
From: Sergiu Bivol [mailto:sbi...@blackberry.com]
Sent: Thursday, May 09, 2013 2:50 PM
To: solr-user@lucene.apache.org
Subject: RE: Invalid version (expected 2, but 60) or the data in not in 
'javabin' format

I have a similar problem. With 5 shards, querying 500K rows fails, but 400K is 
fine.
Querying individual shards for 1.5 million rows works.
All solr instances are v4.2.1 and running on separate Ubuntu VMs.
It is not random, can be always reproduced by adding &rows=50 to a query 
where numFound is > 500K

Is this a configuration issue, where some setting can be increased?

-
From: Ahmet Arslan 
Subject: Invalid version (expected 2, but 60) or the data in not in 'javabin' 
format
Date: Mon, 21 Jan 2013 22:35:10 GMT

Hi,

I am was hitting the following exception when doing distributed search.
I am faceting on an int field named contentID. For some queries it was giving 
this error.
For some queries it just works fine.

localhost:8080/solr/kanu/select/?shards=localhost:8080/solr/rega,localhost:8080/solr/kanu&indent=true&q=karar&start=0&rows=15&hl=false&wt=xml&facet=true&facet.limit=-1&facet.sort=false&json.nl=arrarr&fq=isXml:false&mm=100%&facet.field=contentID&f.contentID.facet.mincount=2

Same search URL works fine for cores (kanu and rega) individually.

Plus if I use rega core as base search URL it works too. e.g.
localhost:8080/solr/rega/select/?shards=localhost:8080...

I see that rega core has lots of unique values for contentID field.
So my conclusion is, if a shard response is "too big" this happens.

This is a bad usage of faceting and I will remove faceting on that field since 
it was added
accidentally.

I still want to share stack traces since error message is somehow misleading.

Jan 21, 2013 10:36:53 PM org.apache.solr.common.SolrException log
SEVERE: null:org.apache.solr.common.SolrException: java.lang.RuntimeException: 
Invalid version
(expected 2, but 60) or the data in not in 'javabin' format
at 
org.apache.solr.handler.component.SearchHandler.handleRequestBody(SearchHandler.java:300)
at 
org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:129)
at org.apache.solr.core.SolrCore.execute(SolrCore.java:1701)
at 
org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:455)
at 
org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:276)
at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
at 
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:293)
at 
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:859)
at 
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:602)
at 
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:489)
at java.lang.Thread.run(Thread.java:662)
Caused by: java.lang.RuntimeException: Invalid version (expected 2, but 60) or 
the data in
not in 'javabin' format
at 
org.apache.solr.common.util.JavaBinCodec.unmarshal(JavaBinCodec.java:109)
at 
org.apache.solr.client.solrj.impl.BinaryResponseParser.processResponse(BinaryResponseParser.java:41)
at 
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:385)
at 
org.apache.solr.client.solrj.impl.HttpSolrServer.request(HttpSolrServer.java:182)
at 
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:166)
at 
org.apache.solr.handler.component.HttpShardHandler$1.call(HttpShardHandler.java:133)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPo

RE: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Sergiu Bivol]

I have a similar problem. With 5 shards, querying 500K rows fails, but 400K is 
fine.
Querying individual shards for 1.5 million rows works.
All solr instances are v4.2.1 and running on separate Ubuntu VMs.
It is not random, can be always reproduced by adding &rows=50 to a query 
where numFound is > 500K

Is this a configuration issue, where some setting can be increased?

-
This transmission (including any attachments) may contain confidential 
information, privileged material (including material protected by the 
solicitor-client or other applicable privileges), or constitute non-public 
information. Any use of this information by anyone other than the intended 
recipient is prohibited. If you have received this transmission in error, 
please immediately reply to the sender and delete this information from your 
system. Use, dissemination, distribution, or reproduction of this transmission 
by unintended recipients is not authorized and may be unlawful.

RE: Invalid version (expected 2, but 60) or the data in not in 'javabin'

[Shahar Davidson]

Thanks for the prompt reply Mark.

Just to give you some background, I'm simulating a multi-shard environment by 
running more than 200 Solr Cores on a single machine (machine does not seem to 
be stressed) and I'm running a distributed facet.
The Solr server is running trunk 1404975 with SOLR-2894 patch applied over it 
(the one from Nov. 12th).
While I'm running the distributed request, other clients are sending various 
search requests to the Solr server.
This issue is randomly happening and does not reproduce constantly.
As I wrote earlier, I applied the Debugging.patch from SOLR-3258 to see the 
actual response and noticed that an actual XML reply was received and the XML 
itself was corrupt (as if a chunk of text was taken out right from the middle 
of it).

Since this reproduces randomly, the only thing that comes to mind is some kind 
of concurrency related problem.

Any help would be greatly appreciated,

Shahar.

-Original Message-
From: Mark Miller [mailto:markrmil...@gmail.com] 
Sent: Wednesday, December 26, 2012 4:21 AM
To: solr-user@lucene.apache.org
Subject: Re: Invalid version (expected 2, but 60) or the data in not in 
'javabin'

The problem is not necessary xml - it seems to be anything that is not valid 
javabin - I've just most often seen it with 404s that return an html error.

I'm not sure if there is a jira issue or not, but this type of thing should be 
failing in a more user friendly way.

As to why your response is corrupt, I have no guesses.

This is easily repeatable? It's happening every time, or randomly?

- Mark

On Dec 25, 2012, at 4:23 AM, Shahar Davidson  wrote:

> Thanks Otis.
> 
> I went through every piece of info that I could lay may hands on.
> Most of them are about incompatible SolrJ versions (that's not my case) and 
> there was one message from Mark Miller that Solr may respond with an XML  
> instead of javabin in case there was some kind of http error being returned 
> (that's not my case either).
> 
> I'm using distributed search.
> I added some debug output to print out the response once the "Invalid 
> version" exception is caught (in JavaBinCode.unmarshal() ).
> What I saw is that the response actually contains the facet response in XML 
> format, yet I also noticed that the response is corrupt (i.e. as if a chunk 
> of text has been taken out of the middle of the reply - some kind of overrun 
> perhaps?).
> 
> Any help would be appreciated.
> 
> Thanks,
> 
> Shahar.
> 
> 
> -Original Message-
> From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com]
> Sent: Friday, December 21, 2012 6:23 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Invalid version (expected 2, but 60) or the data in not in 
> 'javabin'
> 
> Hi,
> 
> Have a look at http://search-lucene.com/?q=invalid+version+javabin
> 
> Otis
> --
> Solr Monitoring - http://sematext.com/spm/index.html
> Search Analytics - http://sematext.com/search-analytics/index.html
> 
> 
> 
> 
> On Wed, Dec 19, 2012 at 11:23 AM, Shahar Davidson 
> wrote:
> 
>> Hi,
>> 
>> I'm encountering this error randomly when running a distributed facet.
>> (i.e. I'm sending the exact same request, yet this does not reproduce
>> consistently)
>> I have about  180 shards that are being queried.
>> It seems that when Solr distributes the request to the shards one , 
>> or perhaps more, shards return an  XML reply instead of  Javabin.
>> 
>> I added some debug output to JavaBinCode.unmarshal  (as done in the 
>> debugging.patch of SOLR-3258) to check whether the XML reply holds an 
>> error or not, and I noticed that the XML actually holds the response 
>> from one of the shards.
>> 
>> I'm using the patch provided in SOLR-2894 on top of trunk 1404975.
>> 
>> Has anyone encountered such an issue? Any ideas?
>> 
>> Thanks,
>> 
>> Shahar.
>> 
> 
> 
> Email secured by Check Point


Email secured by Check Point

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin'

[Mark Miller]

The problem is not necessary xml - it seems to be anything that is not valid 
javabin - I've just most often seen it with 404s that return an html error.

I'm not sure if there is a jira issue or not, but this type of thing should be 
failing in a more user friendly way.

As to why your response is corrupt, I have no guesses.

This is easily repeatable? It's happening every time, or randomly?

- Mark

On Dec 25, 2012, at 4:23 AM, Shahar Davidson  wrote:

> Thanks Otis.
> 
> I went through every piece of info that I could lay may hands on.
> Most of them are about incompatible SolrJ versions (that's not my case) and 
> there was one message from Mark Miller that Solr may respond with an XML  
> instead of javabin in case there was some kind of http error being returned 
> (that's not my case either).
> 
> I'm using distributed search.
> I added some debug output to print out the response once the "Invalid 
> version" exception is caught (in JavaBinCode.unmarshal() ).
> What I saw is that the response actually contains the facet response in XML 
> format, yet I also noticed that the response is corrupt (i.e. as if a chunk 
> of text has been taken out of the middle of the reply - some kind of overrun 
> perhaps?).
> 
> Any help would be appreciated.
> 
> Thanks,
> 
> Shahar.
> 
> 
> -Original Message-
> From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] 
> Sent: Friday, December 21, 2012 6:23 AM
> To: solr-user@lucene.apache.org
> Subject: Re: Invalid version (expected 2, but 60) or the data in not in 
> 'javabin'
> 
> Hi,
> 
> Have a look at http://search-lucene.com/?q=invalid+version+javabin
> 
> Otis
> --
> Solr Monitoring - http://sematext.com/spm/index.html
> Search Analytics - http://sematext.com/search-analytics/index.html
> 
> 
> 
> 
> On Wed, Dec 19, 2012 at 11:23 AM, Shahar Davidson 
> wrote:
> 
>> Hi,
>> 
>> I'm encountering this error randomly when running a distributed facet.
>> (i.e. I'm sending the exact same request, yet this does not reproduce
>> consistently)
>> I have about  180 shards that are being queried.
>> It seems that when Solr distributes the request to the shards one , or 
>> perhaps more, shards return an  XML reply instead of  Javabin.
>> 
>> I added some debug output to JavaBinCode.unmarshal  (as done in the 
>> debugging.patch of SOLR-3258) to check whether the XML reply holds an 
>> error or not, and I noticed that the XML actually holds the response 
>> from one of the shards.
>> 
>> I'm using the patch provided in SOLR-2894 on top of trunk 1404975.
>> 
>> Has anyone encountered such an issue? Any ideas?
>> 
>> Thanks,
>> 
>> Shahar.
>> 
> 
> 
> Email secured by Check Point

RE: Invalid version (expected 2, but 60) or the data in not in 'javabin'

[Shahar Davidson]

Thanks Otis.

I went through every piece of info that I could lay may hands on.
Most of them are about incompatible SolrJ versions (that's not my case) and 
there was one message from Mark Miller that Solr may respond with an XML  
instead of javabin in case there was some kind of http error being returned 
(that's not my case either).

I'm using distributed search.
I added some debug output to print out the response once the "Invalid version" 
exception is caught (in JavaBinCode.unmarshal() ).
What I saw is that the response actually contains the facet response in XML 
format, yet I also noticed that the response is corrupt (i.e. as if a chunk of 
text has been taken out of the middle of the reply - some kind of overrun 
perhaps?).

Any help would be appreciated.

Thanks,

Shahar.


-Original Message-
From: Otis Gospodnetic [mailto:otis.gospodne...@gmail.com] 
Sent: Friday, December 21, 2012 6:23 AM
To: solr-user@lucene.apache.org
Subject: Re: Invalid version (expected 2, but 60) or the data in not in 
'javabin'

Hi,

Have a look at http://search-lucene.com/?q=invalid+version+javabin

Otis
--
Solr Monitoring - http://sematext.com/spm/index.html
Search Analytics - http://sematext.com/search-analytics/index.html




On Wed, Dec 19, 2012 at 11:23 AM, Shahar Davidson wrote:

> Hi,
>
> I'm encountering this error randomly when running a distributed facet.
>  (i.e. I'm sending the exact same request, yet this does not reproduce
> consistently)
> I have about  180 shards that are being queried.
> It seems that when Solr distributes the request to the shards one , or 
> perhaps more, shards return an  XML reply instead of  Javabin.
>
> I added some debug output to JavaBinCode.unmarshal  (as done in the 
> debugging.patch of SOLR-3258) to check whether the XML reply holds an 
> error or not, and I noticed that the XML actually holds the response 
> from one of the shards.
>
> I'm using the patch provided in SOLR-2894 on top of trunk 1404975.
>
> Has anyone encountered such an issue? Any ideas?
>
> Thanks,
>
> Shahar.
>


Email secured by Check Point

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin'

[Otis Gospodnetic]

Hi,

Have a look at http://search-lucene.com/?q=invalid+version+javabin

Otis
--
Solr Monitoring - http://sematext.com/spm/index.html
Search Analytics - http://sematext.com/search-analytics/index.html




On Wed, Dec 19, 2012 at 11:23 AM, Shahar Davidson wrote:

> Hi,
>
> I'm encountering this error randomly when running a distributed facet.
>  (i.e. I'm sending the exact same request, yet this does not reproduce
> consistently)
> I have about  180 shards that are being queried.
> It seems that when Solr distributes the request to the shards one , or
> perhaps more, shards return an  XML reply instead of  Javabin.
>
> I added some debug output to JavaBinCode.unmarshal  (as done in the
> debugging.patch of SOLR-3258) to check whether the XML reply holds an error
> or not, and I noticed that the XML actually holds the response from one of
> the shards.
>
> I'm using the patch provided in SOLR-2894 on top of trunk 1404975.
>
> Has anyone encountered such an issue? Any ideas?
>
> Thanks,
>
> Shahar.
>

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Tomás Fernández Löbbe]

This looks like you are using a SolrJ version different than the Solr
server version you are using. Make sure that server and client are using
the same Solr version.

On Mon, Mar 19, 2012 at 8:02 AM, Markus Jelsma
wrote:

> You probably have a non-char codepoint hanging around somewhere. You can
> strip them away:
> http://unicode.org/cldr/**utility/list-unicodeset.jsp?a=**
> [:Noncharacter_Code_Point=True
> **:]
>
>
>
> On Mon, 19 Mar 2012 10:33:35 +0800, "怪侠" <87863...@qq.com> wrote:
>
>> Hi, all.
>>  I want to update the file's index. The folowing is my code:
>>  ContentStreamUpdateRequest up = new ContentStreamUpdateRequest(
>>  "/update/extract");
>>up.addFile(file);
>>up.setParam("uprefix", "attr_");
>>up.setParam("fmap.content", "attr_content");
>>up.setParam("literal.id", file.getPath());
>>up.setAction(**AbstractUpdateRequest.ACTION.**COMMIT, false, false);
>>solr.request(up);
>>
>>  and I always get the error:
>>  java.lang.RuntimeException: Invalid version (expected 2, but 60) or
>> the data in not in 'javabin' format
>>
>>  and the error in solr server is:
>>  Error processing "legacy" update
>> command:com.ctc.wstx.exc.**WstxIOException: Invalid UTF-8 middle byte
>> 0xe3 (at char #10, byte #-1).
>>
>>  Could anyone tell me how to solve it?
>>
>>  Thanks very much.
>>
>
> --
> Markus Jelsma - CTO - Openindex
> http://www.linkedin.com/in/**markus17
> 050-8536600 / 06-50258350
>

Re: Invalid version (expected 2, but 60) or the data in not in 'javabin' format

[Markus Jelsma]

You probably have a non-char codepoint hanging around somewhere. You 
can strip them away:

http://unicode.org/cldr/utility/list-unicodeset.jsp?a=[:Noncharacter_Code_Point=True:]


On Mon, 19 Mar 2012 10:33:35 +0800, "怪侠" <87863...@qq.com> wrote:

Hi, all.
 I want to update the file's index. The folowing is my code:
  ContentStreamUpdateRequest up = new ContentStreamUpdateRequest(
  "/update/extract");
up.addFile(file);
up.setParam("uprefix", "attr_");
up.setParam("fmap.content", "attr_content");
up.setParam("literal.id", file.getPath());
up.setAction(AbstractUpdateRequest.ACTION.COMMIT, false, false);
solr.request(up);

 and I always get the error:
 java.lang.RuntimeException: Invalid version (expected 2, but 60) or
the data in not in 'javabin' format

 and the error in solr server is:
 Error processing "legacy" update
command:com.ctc.wstx.exc.WstxIOException: Invalid UTF-8 middle byte
0xe3 (at char #10, byte #-1).

 Could anyone tell me how to solve it?

 Thanks very much.


--
Markus Jelsma - CTO - Openindex
http://www.linkedin.com/in/markus17
050-8536600 / 06-50258350