Re: [Koha] Help testing marc staging performance issues

2018-10-05 Thread Barton Chittenden 
So, looking around, I see some bugs that seem to cover the same ground:

Bug 17240 - "Allow processes that rely on background jobs run in Plack mode"

Bug 15032 - [Plack] Scripts that fork (like stage-marc-import.pl) don't
work as expected

Bug 20342 - Fix table locking issue, progress bar not updating, and allow
use of plack for Stage MARC Import


On Thu, Sep 20, 2018 at 5:02 PM Barton Chittenden <
bar...@bywatersolutions.com> wrote:

> Tomas,
>
> Do you know if there's a bug open for this? If not, would you mind filing
> one?
>
> Thanks,
>
> --Barton
>
> On Wed, Sep 19, 2018 at 7:57 PM, Tomas Cohen Arazi 
> wrote:
>
>> We should really do a two-pass processing, so the first thing is
>> calculating the batch size. That way Ajax will not timeout.
>>
>> El mié., 19 de sep. de 2018 20:04, Barton Chittenden <
>> bar...@bywatersolutions.com> escribió:
>>
>>> On Wed, Sep 19, 2018 at 6:29 PM, Jonathan Druart <
>>> jonathan.dru...@bugs.koha-community.org> wrote:
>>>
>>> > Hi Barton,
>>> >
>>> > > I believe this typically happens under plack.
>>> >
>>> > These scripts should not run under plack, check your apache config file
>>> > (search for "ProxyPass" in apache-shared-intranet-plack.conf)
>>> >
>>>
>>> You're exactly right, those are excluded:
>>>
>>> ProxyPass "/cgi-bin/koha/tools/stage-marc-import.pl" "!"
>>> ProxyPass "/cgi-bin/koha/tools/upload-cover-image.pl" "!"
>>>
>>> All, please disregard what I said about plack, apparently my brain was
>>> playing tricks with me when I wrote that. The rest of the message stands:
>>> If you've seen errors importing large batches of Marc records and gotten
>>> the message "there was an error.", I'd like more information so that I
>>> can
>>> file a decent bug report.
>>>
>>> Thanks,
>>>
>>> --Barton
>>> ___
>>> Koha mailing list  http://koha-community.org
>>> Koha@lists.katipo.co.nz
>>> https://lists.katipo.co.nz/mailman/listinfo/koha
>>>
>> --
>> Tomás Cohen Arazi
>> Theke Solutions (https://theke.io )
>> ✆ +54 9351 3513384
>> GPG: B2F3C15F
>>
>
>
___
Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Help testing marc staging performance issues

2018-09-20 Thread Barton Chittenden 
Tomas,

Do you know if there's a bug open for this? If not, would you mind filing
one?

Thanks,

--Barton

On Wed, Sep 19, 2018 at 7:57 PM, Tomas Cohen Arazi 
wrote:

> We should really do a two-pass processing, so the first thing is
> calculating the batch size. That way Ajax will not timeout.
>
> El mié., 19 de sep. de 2018 20:04, Barton Chittenden <
> bar...@bywatersolutions.com> escribió:
>
>> On Wed, Sep 19, 2018 at 6:29 PM, Jonathan Druart <
>> jonathan.dru...@bugs.koha-community.org> wrote:
>>
>> > Hi Barton,
>> >
>> > > I believe this typically happens under plack.
>> >
>> > These scripts should not run under plack, check your apache config file
>> > (search for "ProxyPass" in apache-shared-intranet-plack.conf)
>> >
>>
>> You're exactly right, those are excluded:
>>
>> ProxyPass "/cgi-bin/koha/tools/stage-marc-import.pl" "!"
>> ProxyPass "/cgi-bin/koha/tools/upload-cover-image.pl" "!"
>>
>> All, please disregard what I said about plack, apparently my brain was
>> playing tricks with me when I wrote that. The rest of the message stands:
>> If you've seen errors importing large batches of Marc records and gotten
>> the message "there was an error.", I'd like more information so that I can
>> file a decent bug report.
>>
>> Thanks,
>>
>> --Barton
>> ___
>> Koha mailing list  http://koha-community.org
>> Koha@lists.katipo.co.nz
>> https://lists.katipo.co.nz/mailman/listinfo/koha
>>
> --
> Tomás Cohen Arazi
> Theke Solutions (https://theke.io )
> ✆ +54 9351 3513384
> GPG: B2F3C15F
>
___
Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Help testing marc staging performance issues

2018-09-19 Thread Tomas Cohen Arazi 
We should really do a two-pass processing, so the first thing is
calculating the batch size. That way Ajax will not timeout.

El mié., 19 de sep. de 2018 20:04, Barton Chittenden <
bar...@bywatersolutions.com> escribió:

> On Wed, Sep 19, 2018 at 6:29 PM, Jonathan Druart <
> jonathan.dru...@bugs.koha-community.org> wrote:
>
> > Hi Barton,
> >
> > > I believe this typically happens under plack.
> >
> > These scripts should not run under plack, check your apache config file
> > (search for "ProxyPass" in apache-shared-intranet-plack.conf)
> >
>
> You're exactly right, those are excluded:
>
> ProxyPass "/cgi-bin/koha/tools/stage-marc-import.pl" "!"
> ProxyPass "/cgi-bin/koha/tools/upload-cover-image.pl" "!"
>
> All, please disregard what I said about plack, apparently my brain was
> playing tricks with me when I wrote that. The rest of the message stands:
> If you've seen errors importing large batches of Marc records and gotten
> the message "there was an error.", I'd like more information so that I can
> file a decent bug report.
>
> Thanks,
>
> --Barton
> ___
> Koha mailing list  http://koha-community.org
> Koha@lists.katipo.co.nz
> https://lists.katipo.co.nz/mailman/listinfo/koha
>
-- 
Tomás Cohen Arazi
Theke Solutions (https://theke.io )
✆ +54 9351 3513384
GPG: B2F3C15F
___
Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Help testing marc staging performance issues

2018-09-19 Thread Barton Chittenden 
On Wed, Sep 19, 2018 at 6:29 PM, Jonathan Druart <
jonathan.dru...@bugs.koha-community.org> wrote:

> Hi Barton,
>
> > I believe this typically happens under plack.
>
> These scripts should not run under plack, check your apache config file
> (search for "ProxyPass" in apache-shared-intranet-plack.conf)
>

You're exactly right, those are excluded:

ProxyPass "/cgi-bin/koha/tools/stage-marc-import.pl" "!"
ProxyPass "/cgi-bin/koha/tools/upload-cover-image.pl" "!"

All, please disregard what I said about plack, apparently my brain was
playing tricks with me when I wrote that. The rest of the message stands:
If you've seen errors importing large batches of Marc records and gotten
the message "there was an error.", I'd like more information so that I can
file a decent bug report.

Thanks,

--Barton
___
Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Help testing marc staging performance issues

2018-09-19 Thread Jonathan Druart 
Hi Barton,

> I believe this typically happens under plack.

These scripts should not run under plack, check your apache config file
(search for "ProxyPass" in apache-shared-intranet-plack.conf)

Cheers,
Jonathan

On Wed, 19 Sep 2018 at 19:17 Barton Chittenden 
wrote:

> Hi, All!
>
> At ByWater, we've had a few reports of
>
> "Home › Tools › Stage MARC records for import" timing out on large import
> batches, with the message "there was an error."
>
> When this happens, there are typically two batches under
>
> Home › Tools › Manage staged MARC records
>
> The first batch will show a number of records, and the second batch will
> show 0 records.
>
> I believe this typically happens under plack. I'm having a *very* difficult
> time re-creating the error, so I don't have any plack error logs, I don't
> have any good numbers on how many records cause the problem.
>
> One of our libraries told me that the problem seems to be worse when using
> matching rules.
>
> If you've seen issues like this, I would love to know
>
> 1) What version of Koha you're using
> 2) How many records you were importing
> 3) If you were doing any sort of record matching
> 4) Any other relevant details
> 5) If you have access to the server error logs, I'd like to know what
> messages you see there.
>
> I'd like to file a bug, but without being able to replicate the problem, I
> can't get any traction there.
>
> Thanks,
>
> --Barton
> ___
> Koha mailing list  http://koha-community.org
> Koha@lists.katipo.co.nz
> https://lists.katipo.co.nz/mailman/listinfo/koha
>
___
Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
https://lists.katipo.co.nz/mailman/listinfo/koha