Re: Fetch a binary field

[Barbet Alain]

Hi !

Because this field containt a zipped xml that is bigger than all
others fields & I don't need it for searching, just for display. Yes
it would be better if this field is outside the Lucene base, but as I
have hundred of bases like that, with millions of documents for each,
no I can't change this & reindex the stuff ...

Any other idea ?

2017-08-11 23:55 GMT+02:00 Dave :
> Why didn't you set it to be indexed? Sure it would be a small dent in an index
>
>> On Aug 11, 2017, at 5:20 PM, Barbet Alain  wrote:
>>
>> Re,
>> I take a look on the source code where this msg happen
>> https://github.com/apache/lucene-solr/blob/master/solr/core/src/java/org/apache/solr/schema/SchemaField.java#L186
>>
>> I use version 6.5 who differ from master.
>> In 6.5:
>> if (! (indexed() || hasDocValues()) ) {
>> As my field is not indexed (and hasDocValues() is false as binary),
>> this test fail.
>> Ok but so, what is the way to get this field as I can see it in Luke
>> (in hex format) ?
>>
>>
>> 2017-08-11 15:41 GMT+02:00 Barbet Alain :
>>> Hi !
>>>
>>>
>>> I've a Lucene base coming from a C++ program linked with Lucene++, a
>>> port of Lucene 3.5.9. When I open this base with Luke, it show Lucene
>>> 2.9. Can see a binary field I have in Luke, with data encoded in
>>> base64.
>>>
>>>
>>> I have upgrade this base from 2.9 => 4.0 => 5.0 =>6.0 so I can use it
>>> with Solr 6.5.1. I rebuild a schema for this base, who have one field
>>> in binary:
>>>
>>>
>>> 
>>>
>>>
>>> When I try to retrieve this field with a query (with SOLR admin
>>> interface) it's fail with "can not use FieldCache on a field which is
>>> neither indexed nor has doc values: document"
>>>
>>>
>>> I can retrieve others fields, but can't find a way for this one. Does
>>> someone have an idea ? (At the end I want do this with php, but as it
>>> fail with Solr interface too ).
>>>
>>> Thank you for any help !

Re: missing documents after restart

[John Blythe]

haha, can't have that now!

-- 
*John Blythe*
Product Manager & Lead Developer

251.605.3071 | j...@curvolabs.com
www.curvolabs.com

58 Adams Ave
Evansville, IN 47713

On Fri, Aug 11, 2017 at 2:44 PM, Erick Erickson 
wrote:

> Thanks for closing this out, I was breaking out in hives ;)
>
> Erick
>
> On Fri, Aug 11, 2017 at 11:31 AM, John Blythe  wrote:
> > Looks like part of our nightly processing was restarting the solr server
> > before all indexing was done bc of using a blunt object approach of doing
> > so at designated times, doh!
> >
> > On Tue, Aug 8, 2017 at 9:35 PM John Blythe  wrote:
> >
> >> Thanks Erick. I don't think all of those ifs are in place. Must be
> >> something in our nightly process that is conflicting. Will dive in
> tomorrow
> >> to figure out and report back.
> >>
> >> On Tue, Aug 8, 2017 at 1:27 PM Erick Erickson 
> >> wrote:
> >>
> >>> First, are you absolutely sure you're committing before shutting down?
> >>> Hard commit in this case, openSearcher shouldn't matter.
> >>>
> >>> SolrCloud? And if not SolrCloud, how are you shutting Solr down? "Kill
> >>> -9" is evil.
> >>>
> >>> If you have transaction logs enabled then you shouldn't be losing
> >>> docs, any uncommitted docs should be replayed from the transaction
> >>> log.
> >>>
> >>> This should absolutely _not_ be happening assuming hard commits
> >>> happen. One possible explanation is that you have hard commits turned
> >>> off and are using soft commits _and_ the transaction log isn't
> >>> enabled. And you kill Solr with evil intent. But that's a long chain
> >>> of "ifs"
> >>>
> >>> Best,
> >>> Erick
> >>>
> >>> On Tue, Aug 8, 2017 at 6:02 AM, John Blythe 
> wrote:
> >>> > hi all.
> >>> >
> >>> > i have a core that contains about 22 million documents. when the solr
> >>> > server is restarted it drops to 200-400k. the dashbaord says that
> it's
> >>> both
> >>> > optimized and current.
> >>> >
> >>> > is there config issues i need to address in solr or the server? not
> >>> really
> >>> > sure where to begin in hunting this down.
> >>> >
> >>> > thanks-
> >>>
> >> --
> >> --
> >> *John Blythe*
> >> Product Manager & Lead Developer
> >>
> >> 251.605.3071 | j...@curvolabs.com
> >> www.curvolabs.com
> >>
> >> 58 Adams Ave
> >> Evansville, IN 47713
> >>
> > --
> > --
> > *John Blythe*
> > Product Manager & Lead Developer
> >
> > 251.605.3071 | j...@curvolabs.com
> > www.curvolabs.com
> >
> > 58 Adams Ave
> > Evansville, IN 47713
>

Re: MongoDb vs Solr

[Dave]

Personally I say use a rdbms for data storage, it's what it's for. Solr is for 
search and retrieve and the expense of possible loss of all data, in which case 
you rebuild it. 

> On Aug 12, 2017, at 11:26 AM, Muwonge Ronald  wrote:
> 
> Hi Solr can use mongodb for storage and you can play with the data as it
> grows depending on your data goals.Ease of learning doesn't mean
> happiness.I recommend you use both for serious projects that won't collapse
> soon.
> Ronny
>> On 5 Aug 2017 02:16, "Francesco Viscomi"  wrote:
>> 
>> Hi all,
>> why i have to choose solr if mongoDb is easier to learn and to use?
>> Both are NoSql database, is there a good reason to chose solr and not
>> mongoDb?
>> 
>> thanks really much
>> 
>> --
>> Ing. Viscomi Francesco
>> 

Re: MongoDb vs Solr

[Jcd]

Hello,
I have implemented a NoSQL key/value DB where I index the value field needed 
for retrieval plus the key value to get back the DB record when searching and 
it works pretty well.
I use Lucene but you can most probably do the same with SOLR.
Best wishes 
JCD

Envoyé de mon iPhone

> Le 12 août 2017 à 21:06, Leonardo Perez Pulido  a 
> écrit :
> 
> The main reason to chose Solr is the capacity of it has of return results
> sorted by relevance. Solr returns documents in ranked order based on how
> relevant each document is to the user's query.
> Solr isn't a database, although it can 'store' data.
> Regards.
> 
>> On Sat, Aug 12, 2017 at 2:27 PM, Dave  wrote:
>> 
>> Personally I say use a rdbms for data storage, it's what it's for. Solr is
>> for search and retrieve and the expense of possible loss of all data, in
>> which case you rebuild it.
>> 
>>> On Aug 12, 2017, at 11:26 AM, Muwonge Ronald  wrote:
>>> 
>>> Hi Solr can use mongodb for storage and you can play with the data as it
>>> grows depending on your data goals.Ease of learning doesn't mean
>>> happiness.I recommend you use both for serious projects that won't
>> collapse
>>> soon.
>>> Ronny
 On 5 Aug 2017 02:16, "Francesco Viscomi"  wrote:
 
 Hi all,
 why i have to choose solr if mongoDb is easier to learn and to use?
 Both are NoSql database, is there a good reason to chose solr and not
 mongoDb?
 
 thanks really much
 
 --
 Ing. Viscomi Francesco
 
>> 

Re: MongoDb vs Solr

[Leonardo Perez Pulido]

The main reason to chose Solr is the capacity of it has of return results
sorted by relevance. Solr returns documents in ranked order based on how
relevant each document is to the user's query.
Solr isn't a database, although it can 'store' data.
Regards.

On Sat, Aug 12, 2017 at 2:27 PM, Dave  wrote:

> Personally I say use a rdbms for data storage, it's what it's for. Solr is
> for search and retrieve and the expense of possible loss of all data, in
> which case you rebuild it.
>
> > On Aug 12, 2017, at 11:26 AM, Muwonge Ronald  wrote:
> >
> > Hi Solr can use mongodb for storage and you can play with the data as it
> > grows depending on your data goals.Ease of learning doesn't mean
> > happiness.I recommend you use both for serious projects that won't
> collapse
> > soon.
> > Ronny
> >> On 5 Aug 2017 02:16, "Francesco Viscomi"  wrote:
> >>
> >> Hi all,
> >> why i have to choose solr if mongoDb is easier to learn and to use?
> >> Both are NoSql database, is there a good reason to chose solr and not
> >> mongoDb?
> >>
> >> thanks really much
> >>
> >> --
> >> Ing. Viscomi Francesco
> >>
>

Re: MongoDb vs Solr

[Muwonge Ronald]

Hi Solr can use mongodb for storage and you can play with the data as it
grows depending on your data goals.Ease of learning doesn't mean
happiness.I recommend you use both for serious projects that won't collapse
soon.
Ronny
On 5 Aug 2017 02:16, "Francesco Viscomi"  wrote:

> Hi all,
> why i have to choose solr if mongoDb is easier to learn and to use?
> Both are NoSql database, is there a good reason to chose solr and not
> mongoDb?
>
> thanks really much
>
> --
> Ing. Viscomi Francesco
>

Re: MongoDb vs Solr

[Lars Karlsson]

Agree, but saying

"but Solr is generally not considered to be durable across crashes or kill
-9."

Such statements are misleading, even Cassandra can loose data in such case.

and talking about RDBMS and ACID = non scalable persistence and dealing
with replication hell.

On Sat, 12 Aug 2017 at 17:26, Muwonge Ronald  wrote:

> Hi Solr can use mongodb for storage and you can play with the data as it
> grows depending on your data goals.Ease of learning doesn't mean
> happiness.I recommend you use both for serious projects that won't collapse
> soon.
> Ronny
> On 5 Aug 2017 02:16, "Francesco Viscomi"  wrote:
>
> > Hi all,
> > why i have to choose solr if mongoDb is easier to learn and to use?
> > Both are NoSql database, is there a good reason to chose solr and not
> > mongoDb?
> >
> > thanks really much
> >
> > --
> > Ing. Viscomi Francesco
> >
>

Unable to write response, client closed connection or we are shutting down

[Nawab Zada Asad Iqbal]

Hi,

I am executing a query performance test against my solr 6.6 setup and I
noticed following exception every now and then. What do I need to do?

Aug 11, 2017 08:40:07 AM INFO  (qtp761960786-250) [   x:filesearch]
o.a.s.s.HttpSolrCall Unable to write response, client closed connection or
we are shutting down
org.eclipse.jetty.io.EofException
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:199)
at org.eclipse.jetty.io.WriteFlusher.flush(WriteFlusher.java:420)
at org.eclipse.jetty.io.WriteFlusher.completeWrite(
WriteFlusher.java:375)
at org.eclipse.jetty.io.SelectChannelEndPoint$3.run(
SelectChannelEndPoint.java:107)
at org.eclipse.jetty.io.SelectChannelEndPoint.onSelected(
SelectChannelEndPoint.java:193)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.
processSelected(ManagedSelector.java:283)
at org.eclipse.jetty.io.ManagedSelector$SelectorProducer.produce(
ManagedSelector.java:181)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
executeProduceConsume(ExecuteProduceConsume.java:249)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.
produceConsume(ExecuteProduceConsume.java:148)
at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(
ExecuteProduceConsume.java:136)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(
QueuedThreadPool.java:671)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(
QueuedThreadPool.java:589)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.io.IOException: Broken pipe
at sun.nio.ch.FileDispatcherImpl.write0(Native Method)
at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:47)
at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:93)
at sun.nio.ch.IOUtil.write(IOUtil.java:51)
at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:471)
at org.eclipse.jetty.io.ChannelEndPoint.flush(ChannelEndPoint.java:177)



Apart from that, I also noticed that the query response time is longer than
I expected, while the memory utilization stays <= 35%. I thought that
somewhere I have set maxThreads (Jetty) to a very low number, however I am
falling back on default which is 1 (so that shouldn't be a problem).


Thanks
Nawab