[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16393741#comment-16393741
]
Shawn Heisey commented on SOLR-11049:
-
Just became aware of this issue due to the mailing list.
It's awesome that there's a workaround, and it does look like the reference
guide was updated.
But anytime somebody performs an action and it doesn't work, Solr should not
return a success status (0), and there should be at least one log entry
explaining what went wrong.
Separately: Do we need to be worried about the fact that the failed upload took
24 seconds? I'm guessing that there was at least one timeout involved with
this. I would have expected ZK to reject the upload quite quickly, and to do
it in a way that Solr *can* detect as an error. It would be good to figure out
whether it was Solr or ZK that misbehaved here.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
>Priority: Major
> Fix For: 7.2, master (8.0)
>
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16131308#comment-16131308
]
Christine Poerschke commented on SOLR-11049:
bq. Zookeeper isn't suited to storing large data like this. I think you should
explore using the blob store for this purpose.
Thanks [~shalinmangar] for the blob store pointer!
Just to cross-reference tickets here, [~yuyano]'s SOLR-11250 takes a wrapper
approach i.e. a wrapper model is still stored in ZooKeeper but the wrapped
model's definition is stored externally.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110708#comment-16110708
]
Stefan Langenmaier commented on SOLR-11049:
---
[~shalinmangar] - thanks for the clarification :)
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110675#comment-16110675
]
Shalin Shekhar Mangar commented on SOLR-11049:
--
[~stefan.langenmaier] -- my comment that you should explore the blob store
(instead of increasing jute.maxBuffer setting) was meant for Christine and not
you. The capability to load from the blob store is not yet implemented.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16110518#comment-16110518
]
Stefan Langenmaier commented on SOLR-11049:
---
Thanks for the pointers, but I seem to be a little bit lost. According to the
documentation "[the blob store] can be used to upload a jar file which contains
standard solr components such as RequestHandlers, SearchComponents, or other
custom code you have written for Solr."
While the LTR model is used in request hanlders, as far as I understand it, it
is uploaded and created through another mechanism and then Solr is repsonsible
for storing it.
I dont see yet how this is supposed to work.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107364#comment-16107364
]
Matthias Krueger commented on SOLR-11049:
-
See https://lucene.apache.org/solr/guide/6_6/blob-store-api.html or
https://cwiki.apache.org/confluence/display/solr/Blob+Store+API.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16107150#comment-16107150
]
Stefan Langenmaier commented on SOLR-11049:
---
Where can I found out more about this blob store?
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
[jira] [Commented] (SOLR-11049) Solr in cloud mode silently fails uploading a big LTR model
[
https://issues.apache.org/jira/browse/SOLR-11049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16105969#comment-16105969
]
Shalin Shekhar Mangar commented on SOLR-11049:
--
Zookeeper isn't suited to storing large data like this. I think you should
explore using the blob store for this purpose.
> Solr in cloud mode silently fails uploading a big LTR model
> ---
>
> Key: SOLR-11049
> URL: https://issues.apache.org/jira/browse/SOLR-11049
> Project: Solr
> Issue Type: Bug
> Security Level: Public(Default Security Level. Issues are Public)
> Components: contrib - LTR
> Environment: tested with Solr 6.6 an integrated zookeeper
>Reporter: Stefan Langenmaier
> Attachments: SOLR-11049.patch
>
>
> Hi,
> I'm using Solr in cloud mode, I have a MultipleAdditiveTreesModel with about
> 3MB in size. When I upload the model with
> {noformat}
> curl -v -XPUT 'http://localhost:8983/solr/tmdb/schema/model-store'
> --data-binary @/big-tree.model -H 'Content-type:application/json'
> {noformat}
> I get the following response
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":24318}
> }
> {code}
> This looks kind of slow but without an error. When I check the config the
> model is not visible and when I try to run a query that uses the model I get
> the following error
> {code:html}
> "error":{
> "metadata":[
> "error-class","org.apache.solr.common.SolrException",
> "root-error-class","org.apache.solr.common.SolrException"],
> "msg":"cannot find model bigTreeModel",
> "code":400}
> {code}
> When I upload the model to solr where I increased the zookeeper znode size
> limit with
> {noformat}
> -Djute.maxbuffer=0x1ff
> {noformat}
> the same model upload succeeds much faster
> {code:html}
> {
> "responseHeader":{
> "status":0,
> "QTime":689}
> }
> {code}
> The model is visible in the configuration and queries that use it run without
> error.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
-
To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org
For additional commands, e-mail: dev-h...@lucene.apache.org
