Re: Question about solr commits

2020-10-08 Thread Erick Erickson
This is a bit confused. There will be only one timer that starts at time T when
the first doc comes in. At T+ 15 seconds, all docs that have been received since
time T will be committed. The first doc to hit Solr _after_ T+15 seconds starts
a single new timer and the process repeats.

Best,
rick

> On Oct 8, 2020, at 2:26 PM, Rahul Goswami  wrote:
> 
> Shawn,
> So if the autoCommit interval is 15 seconds, and one update request arrives
> at t=0 and another at t=10 seconds, then will there be two timers one
> expiring at t=15 and another at t=25 seconds, but this would amount to ONLY
> ONE commit at t=15 since that one would include changes from both updates.
> Is this understanding correct ?
> 
> Thanks,
> Rahul
> 
> On Wed, Oct 7, 2020 at 11:39 PM yaswanth kumar 
> wrote:
> 
>> Thank you very much both Eric and Shawn
>> 
>> Sent from my iPhone
>> 
>>> On Oct 7, 2020, at 10:41 PM, Shawn Heisey  wrote:
>>> 
>>> On 10/7/2020 4:40 PM, yaswanth kumar wrote:
 I have the below in my solrconfig.xml
 

  ${solr.Data.dir:}


  ${solr.autoCommit.maxTime:6}
  false


  ${solr.autoSoftCommit.maxTime:5000}

  
 Does this mean even though we are always sending data with commit=false
>> on
 update solr api, the above should do the commit every minute (6 ms)
 right?
>>> 
>>> Assuming that you have not defined the "solr.autoCommit.maxTime" and/or
>> "solr.autoSoftCommit.maxTime" properties, this config has autoCommit set to
>> 60 seconds without opening a searcher, and autoSoftCommit set to 5 seconds.
>>> 
>>> So five seconds after any indexing begins, Solr will do a soft commit.
>> When that commit finishes, changes to the index will be visible to
>> queries.  One minute after any indexing begins, Solr will do a hard commit,
>> which guarantees that data is written to disk, but it will NOT open a new
>> searcher, which means that when the hard commit happens, any pending
>> changes to the index will not be visible.
>>> 
>>> It's not "every five seconds" or "every 60 seconds" ... When any changes
>> are made, Solr starts a timer.  When the timer expires, the commit is
>> fired.  If no changes are made, no commits happen, because the timer isn't
>> started.
>>> 
>>> Thanks,
>>> Shawn
>> 



Re: Question about solr commits

2020-10-08 Thread Rahul Goswami
Shawn,
So if the autoCommit interval is 15 seconds, and one update request arrives
at t=0 and another at t=10 seconds, then will there be two timers one
expiring at t=15 and another at t=25 seconds, but this would amount to ONLY
ONE commit at t=15 since that one would include changes from both updates.
Is this understanding correct ?

Thanks,
Rahul

On Wed, Oct 7, 2020 at 11:39 PM yaswanth kumar 
wrote:

> Thank you very much both Eric and Shawn
>
> Sent from my iPhone
>
> > On Oct 7, 2020, at 10:41 PM, Shawn Heisey  wrote:
> >
> > On 10/7/2020 4:40 PM, yaswanth kumar wrote:
> >> I have the below in my solrconfig.xml
> >> 
> >> 
> >>   ${solr.Data.dir:}
> >> 
> >> 
> >>   ${solr.autoCommit.maxTime:6}
> >>   false
> >> 
> >> 
> >>   ${solr.autoSoftCommit.maxTime:5000}
> >> 
> >>   
> >> Does this mean even though we are always sending data with commit=false
> on
> >> update solr api, the above should do the commit every minute (6 ms)
> >> right?
> >
> > Assuming that you have not defined the "solr.autoCommit.maxTime" and/or
> "solr.autoSoftCommit.maxTime" properties, this config has autoCommit set to
> 60 seconds without opening a searcher, and autoSoftCommit set to 5 seconds.
> >
> > So five seconds after any indexing begins, Solr will do a soft commit.
> When that commit finishes, changes to the index will be visible to
> queries.  One minute after any indexing begins, Solr will do a hard commit,
> which guarantees that data is written to disk, but it will NOT open a new
> searcher, which means that when the hard commit happens, any pending
> changes to the index will not be visible.
> >
> > It's not "every five seconds" or "every 60 seconds" ... When any changes
> are made, Solr starts a timer.  When the timer expires, the commit is
> fired.  If no changes are made, no commits happen, because the timer isn't
> started.
> >
> > Thanks,
> > Shawn
>


Re: Question about solr commits

2020-10-07 Thread yaswanth kumar
Thank you very much both Eric and Shawn

Sent from my iPhone

> On Oct 7, 2020, at 10:41 PM, Shawn Heisey  wrote:
> 
> On 10/7/2020 4:40 PM, yaswanth kumar wrote:
>> I have the below in my solrconfig.xml
>> 
>> 
>>   ${solr.Data.dir:}
>> 
>> 
>>   ${solr.autoCommit.maxTime:6}
>>   false
>> 
>> 
>>   ${solr.autoSoftCommit.maxTime:5000}
>> 
>>   
>> Does this mean even though we are always sending data with commit=false on
>> update solr api, the above should do the commit every minute (6 ms)
>> right?
> 
> Assuming that you have not defined the "solr.autoCommit.maxTime" and/or 
> "solr.autoSoftCommit.maxTime" properties, this config has autoCommit set to 
> 60 seconds without opening a searcher, and autoSoftCommit set to 5 seconds.
> 
> So five seconds after any indexing begins, Solr will do a soft commit. When 
> that commit finishes, changes to the index will be visible to queries.  One 
> minute after any indexing begins, Solr will do a hard commit, which 
> guarantees that data is written to disk, but it will NOT open a new searcher, 
> which means that when the hard commit happens, any pending changes to the 
> index will not be visible.
> 
> It's not "every five seconds" or "every 60 seconds" ... When any changes are 
> made, Solr starts a timer.  When the timer expires, the commit is fired.  If 
> no changes are made, no commits happen, because the timer isn't started.
> 
> Thanks,
> Shawn


Re: Question about solr commits

2020-10-07 Thread Shawn Heisey

On 10/7/2020 4:40 PM, yaswanth kumar wrote:

I have the below in my solrconfig.xml


 
   ${solr.Data.dir:}
 
 
   ${solr.autoCommit.maxTime:6}
   false
 
 
   ${solr.autoSoftCommit.maxTime:5000}
 
   

Does this mean even though we are always sending data with commit=false on
update solr api, the above should do the commit every minute (6 ms)
right?


Assuming that you have not defined the "solr.autoCommit.maxTime" and/or 
"solr.autoSoftCommit.maxTime" properties, this config has autoCommit set 
to 60 seconds without opening a searcher, and autoSoftCommit set to 5 
seconds.


So five seconds after any indexing begins, Solr will do a soft commit. 
When that commit finishes, changes to the index will be visible to 
queries.  One minute after any indexing begins, Solr will do a hard 
commit, which guarantees that data is written to disk, but it will NOT 
open a new searcher, which means that when the hard commit happens, any 
pending changes to the index will not be visible.


It's not "every five seconds" or "every 60 seconds" ... When any changes 
are made, Solr starts a timer.  When the timer expires, the commit is 
fired.  If no changes are made, no commits happen, because the timer 
isn't started.


Thanks,
Shawn


Re: Question about solr commits

2020-10-07 Thread Erick Erickson
Yes.

> On Oct 7, 2020, at 6:40 PM, yaswanth kumar  wrote:
> 
> I have the below in my solrconfig.xml
> 
> 
>
>  ${solr.Data.dir:}
>
>
>  ${solr.autoCommit.maxTime:6}
>  false
>
>
>  ${solr.autoSoftCommit.maxTime:5000}
>
>  
> 
> Does this mean even though we are always sending data with commit=false on
> update solr api, the above should do the commit every minute (6 ms)
> right?
> 
> -- 
> Thanks & Regards,
> Yaswanth Kumar Konathala.
> yaswanth...@gmail.com



Question about solr commits

2020-10-07 Thread yaswanth kumar
I have the below in my solrconfig.xml



  ${solr.Data.dir:}


  ${solr.autoCommit.maxTime:6}
  false


  ${solr.autoSoftCommit.maxTime:5000}

  

Does this mean even though we are always sending data with commit=false on
update solr api, the above should do the commit every minute (6 ms)
right?

-- 
Thanks & Regards,
Yaswanth Kumar Konathala.
yaswanth...@gmail.com