Re: [Koha] Large number of action_logs records

[Fridolin SOMERS]

Hi,

You can use daily cron : cleanup_database.pl --logs DAYS

https://git.koha-community.org/Koha-community/Koha/src/branch/master/misc/cronjobs/cleanup_database.pl#L88

Le 31/08/2023 à 06:17, Coehoorn, Joel a écrit :
I found the --incremental was missing. This is an installation that has 
evolved since before 2016, so I suspect it predates that addition. I've 
added it and am monitoring to see if growth is more reasonable.


What I may submit as a bug report or feature request is an option to 
rotate these logs. So we'd add a background task to delete records older 
than some reasonable (possibly configurable) number. This would keep the 
table from growing indefinitely.


*Joel Coehoorn*
Director of Information Technology
*York University*
Office: 402-363-5603 | jcoeho...@york.edu  | 
york.edu 




On Thu, Aug 31, 2023 at 3:22 AM Fridolin SOMERS 
mailto:fridolin.som...@biblibre.com>> wrote:


Do you use update_totalissues.pl  with
--incremental ?

Le 30/08/2023 à 21:37, Jonathan Druart a écrit :
 > Hi Joel,
 >
 > We could skip the logging when the biblio is modified by this
cronjob,
 > please report this problem in a new bug report.
 >
 > Regards,
 > Jonathan
 >
 > Le mar. 29 août 2023 à 21:22, Coehoorn, Joel mailto:jcoeho...@york.edu>> a écrit :
 >>
 >> We're seeing a large quantity of records filling the action_logs
table...
 >> nearly 18,000 in the last 24 hours. This in turn uses a LOT of disk
 >> space... it was briefly over 90GB until I intervened, and
because MySQL is
 >> dumb about shrinking tables we were fortunate I caught it in time.
 >>
 >> It looks like the update_totalissues.pl
 script is running every night and
 >> putting the entire biblio record into the info column in the
table for
 >> every record we have.
 >>
 >> What could be causing this, and how can I stop it (without
disabling the
 >> job completely, which I assume is part of Koha and important)?
 >>
 >> *Joel Coehoorn*
 >> Director of Information Technology
 >> *York University*
 >> Office: 402-363-5603 | jcoeho...@york.edu
 | york.edu 
 >>
 >> *Please contact helpd...@york.edu 
mailto:helpd...@york.edu>> for technical
 >> assistance.*
 >>
 >>
 >> The mission of York University is to transform lives through
 >> Christ-centered education and to equip students for lifelong
service to
 >> God, family, and society
 >> ___
 >>
 >> Koha mailing list http://koha-community.org

 >> Koha@lists.katipo.co.nz 
 >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

 > ___
 >
 > Koha mailing list http://koha-community.org

 > Koha@lists.katipo.co.nz 
 > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha


-- 
Fridolin SOMERS 
>
Software and system maintainer 濾
BibLibre, France
___

Koha mailing list http://koha-community.org 
Koha@lists.katipo.co.nz 
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha




--
Fridolin SOMERS 
Software and system maintainer 濾
BibLibre, France
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] [Koha-devel] needed help in koha cataloguing module

[David Nind]

Hi Zahid.

From my limited knowledge, I think using setting up MARC overlay rules is
what you would need to do.

Happy for anyone who has setup these types of rules to comment on whether
this is the best way to do what you want.

The ByWater Solutions Team has a nice clear video about setting up these
rules (recorded for 21.11, so some things may have changed): see
https://bywatersolutions.com/education/monday-minutes-protecting-marc-fields
and https://www.youtube.com/watch?v=w2xDi1K07Qg

There is also a section in the manual about record overlay rules (I've
linked to the latest version, there may be some differences with 22.11):
https://koha-community.org/manual/23.05/en/html/administration.html#record-overlay-rules

Without knowing exactly why you want to stop cataloguers from adding or
modifying the 245 indicators and subfields (I'm assuming MARC21 here), such
as you only want specific cataloguers to edit these subfields, there may be
some other things to consider:
- You can use permissions, so you can remove editing catalog records for
staff users (Patrons > [patron account] > More > Set permissions > Edit
catalog)
- For your frameworks, you could change the visibility of subfields in the
OPAC, staff interface, and editor (Administration > Catalog > MARC
bibliographic) (or remove them from the framework, but removing 245$a
wouldn't make any sense)
- There is probably a report you could create that would identify changes
made to these subfields, and then take corrective action (education about
your cataloguing standards, etc) (I'm not An SQL/Reports person)
- If not enabled, you could enable the CataloguingLog system preference to
record who has made changes, and monitor this


David Nind
New Zealand

PS I've only sent my reply to the General Koha Mailing List.


On Wed, 30 Aug 2023 at 21:44, Zahid Iqbal  wrote:

> Dear all,
> i am using koha 22.11 version
> i want to disable the indicator box and box for (a) (b) (c) etc so user
> can not type and change the [a] [b] [c] to [e] [f] [3] etc. On
> adding/modifying books koha page koha/cataloguing/addbiblio.pl when i
> click on 245 (*a*) it allow me to change (*a*) i want to disable/readonly
> [a] and two boxes for indicators so it cannot be changed.is there any
> system preference value to restrict the user? or any other suggestion?
> should i look into MARCOverlayRules ?
> needed help in this problem.
> Best regards,
> --
> *Zahid*
> ___
> Koha-devel mailing list
> koha-de...@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Performance improvement suggestions

[Furst, Kevin]

After running the tuner and making some changes to my config, I believe the
improvements are satisfactory for now.

*Kevin Furst*
Technology Director
Pine River-Backus Schools
o | 218.587.8030
@ | kfu...@prbschools.org

Technology Office/Help Desk:  218-587-8510

*Confidentiality Notice: This E-mail message, including any attachments, is
for the sole use of the intended recipient(s) and may contain confidential
and privileged information. Any unauthorized review, use, disclosure or
distribution is prohibited. If you are not the intended recipient, please
contact the sender by reply E-mail and destroy all copies of the original
message.*


On Thu, Aug 31, 2023 at 1:11 PM Fridolin SOMERS <
fridolin.som...@biblibre.com> wrote:

> Please answer to the mailling list instead of me directly.
>
> Le 31/08/2023 à 03:11, Furst, Kevin a écrit :
> > 1. Plack is enabled and running
> > 2. Haven't run db tuner you suggested yet. Will look at it today.
> > 3. Hardware is Intel Xeon E5520 8GB RAM  and (8) 2TB SSDs in a RAID.
> >
> >
> > *Kevin Furst*
> > Technology Director
> > Pine River-Backus Schools
> > o | 218.587.8030
> > @ | kfu...@prbschools.org 
> >
> > Technology Office/Help Desk:  218-587-8510
> >
> > /Confidentiality Notice: This E-mail message, including any attachments,
> > is for the sole use of the intended recipient(s) and may contain
> > confidential and privileged information. Any unauthorized review, use,
> > disclosure or distribution is prohibited. If you are not the intended
> > recipient, please contact the sender by reply E-mail and destroy all
> > copies of the original message./
> >
> >
> > On Wed, Aug 30, 2023 at 8:19 PM Fridolin SOMERS
> > mailto:fridolin.som...@biblibre.com>>
> wrote:
> >
> > Hi,
> >
> > Did you tune DB server ? Like with mysqltuner.pl <
> http://mysqltuner.pl>
> >
> > Did you enable Plack mode ?
> > https://wiki.koha-community.org/wiki/Koha_on_Debian#Setup_plack
> > 
> >
> > Please give more informations on hardware.
> >
> > Best regards,
> >
> > Le 30/08/2023 à 09:54, Furst, Kevin a écrit :
> >  > Hi,
> >  >
> >  > I'm running v.23.05.x on Ubuntu 20.04. I've currently got zebra
> > running for
> >  > the search engine. It works correctly but is really slow. We're
> > talking ~10
> >  > seconds for a return on catalog search query and maybe 5-6
> > seconds on a
> >  > patron search in the staff side. Our library only has ~1000
> > patrons and 20k
> >  > titles.
> >  >
> >  > I'm wondering if anyone has any performance enhancement
> > recommendations on
> >  > the server-side that they've had success with that would help
> > speed up
> >  > those times.
> >  >
> >  > *Kevin Furst*
> >  > Technology Director
> >  > Pine River-Backus Schools
> >  > o | 218.587.8030
> >  > @ | kfu...@prbschools.org 
> >  >
> >  > Technology Office/Help Desk:  218-587-8510
> >  >
> >  > *Confidentiality Notice: This E-mail message, including any
> > attachments, is
> >  > for the sole use of the intended recipient(s) and may contain
> > confidential
> >  > and privileged information. Any unauthorized review, use,
> > disclosure or
> >  > distribution is prohibited. If you are not the intended
> > recipient, please
> >  > contact the sender by reply E-mail and destroy all copies of the
> > original
> >  > message.*
> >  > ___
> >  >
> >  > Koha mailing list http://koha-community.org
> > 
> >  > Koha@lists.katipo.co.nz 
> >  > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > 
> >
> > --
> > Fridolin SOMERS  > >
> > Software and system maintainer 濾
> > BibLibre, France
> > ___
> >
> > Koha mailing list http://koha-community.org <
> http://koha-community.org>
> > Koha@lists.katipo.co.nz 
> > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > 
> >
>
> --
> Fridolin SOMERS 
> Software and system maintainer 濾
> BibLibre, France
> ___
>
> Koha mailing list  http://koha-community.org
> Koha@lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Performance improvement suggestions

[Fridolin SOMERS]

Please answer to the mailling list instead of me directly.

Le 31/08/2023 à 03:11, Furst, Kevin a écrit :

1. Plack is enabled and running
2. Haven't run db tuner you suggested yet. Will look at it today.
3. Hardware is Intel Xeon E5520 8GB RAM  and (8) 2TB SSDs in a RAID.


*Kevin Furst*
Technology Director
Pine River-Backus Schools
o | 218.587.8030
@ | kfu...@prbschools.org 

Technology Office/Help Desk:  218-587-8510

/Confidentiality Notice: This E-mail message, including any attachments, 
is for the sole use of the intended recipient(s) and may contain 
confidential and privileged information. Any unauthorized review, use, 
disclosure or distribution is prohibited. If you are not the intended 
recipient, please contact the sender by reply E-mail and destroy all 
copies of the original message./



On Wed, Aug 30, 2023 at 8:19 PM Fridolin SOMERS 
mailto:fridolin.som...@biblibre.com>> wrote:


Hi,

Did you tune DB server ? Like with mysqltuner.pl 

Did you enable Plack mode ?
https://wiki.koha-community.org/wiki/Koha_on_Debian#Setup_plack


Please give more informations on hardware.

Best regards,

Le 30/08/2023 à 09:54, Furst, Kevin a écrit :
 > Hi,
 >
 > I'm running v.23.05.x on Ubuntu 20.04. I've currently got zebra
running for
 > the search engine. It works correctly but is really slow. We're
talking ~10
 > seconds for a return on catalog search query and maybe 5-6
seconds on a
 > patron search in the staff side. Our library only has ~1000
patrons and 20k
 > titles.
 >
 > I'm wondering if anyone has any performance enhancement
recommendations on
 > the server-side that they've had success with that would help
speed up
 > those times.
 >
 > *Kevin Furst*
 > Technology Director
 > Pine River-Backus Schools
 > o | 218.587.8030
 > @ | kfu...@prbschools.org 
 >
 > Technology Office/Help Desk:  218-587-8510
 >
 > *Confidentiality Notice: This E-mail message, including any
attachments, is
 > for the sole use of the intended recipient(s) and may contain
confidential
 > and privileged information. Any unauthorized review, use,
disclosure or
 > distribution is prohibited. If you are not the intended
recipient, please
 > contact the sender by reply E-mail and destroy all copies of the
original
 > message.*
 > ___
 >
 > Koha mailing list http://koha-community.org

 > Koha@lists.katipo.co.nz 
 > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha


-- 
Fridolin SOMERS 
>
Software and system maintainer 濾
BibLibre, France
___

Koha mailing list http://koha-community.org 
Koha@lists.katipo.co.nz 
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha




--
Fridolin SOMERS 
Software and system maintainer 濾
BibLibre, France
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Large number of action_logs records

[Coehoorn, Joel]

I found the --incremental was missing. This is an installation that has
evolved since before 2016, so I suspect it predates that addition. I've
added it and am monitoring to see if growth is more reasonable.

What I may submit as a bug report or feature request is an option to rotate
these logs. So we'd add a background task to delete records older than some
reasonable (possibly configurable) number. This would keep the table from
growing indefinitely.

*Joel Coehoorn*
Director of Information Technology
*York University*
Office: 402-363-5603 | jcoeho...@york.edu | york.edu



On Thu, Aug 31, 2023 at 3:22 AM Fridolin SOMERS <
fridolin.som...@biblibre.com> wrote:

> Do you use update_totalissues.pl with --incremental ?
>
> Le 30/08/2023 à 21:37, Jonathan Druart a écrit :
> > Hi Joel,
> >
> > We could skip the logging when the biblio is modified by this cronjob,
> > please report this problem in a new bug report.
> >
> > Regards,
> > Jonathan
> >
> > Le mar. 29 août 2023 à 21:22, Coehoorn, Joel  a
> écrit :
> >>
> >> We're seeing a large quantity of records filling the action_logs
> table...
> >> nearly 18,000 in the last 24 hours. This in turn uses a LOT of disk
> >> space... it was briefly over 90GB until I intervened, and because MySQL
> is
> >> dumb about shrinking tables we were fortunate I caught it in time.
> >>
> >> It looks like the update_totalissues.pl script is running every night
> and
> >> putting the entire biblio record into the info column in the table for
> >> every record we have.
> >>
> >> What could be causing this, and how can I stop it (without disabling the
> >> job completely, which I assume is part of Koha and important)?
> >>
> >> *Joel Coehoorn*
> >> Director of Information Technology
> >> *York University*
> >> Office: 402-363-5603 | jcoeho...@york.edu | york.edu
> >>
> >> *Please contact helpd...@york.edu  for technical
> >> assistance.*
> >>
> >>
> >> The mission of York University is to transform lives through
> >> Christ-centered education and to equip students for lifelong service to
> >> God, family, and society
> >> ___
> >>
> >> Koha mailing list  http://koha-community.org
> >> Koha@lists.katipo.co.nz
> >> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
> > ___
> >
> > Koha mailing list  http://koha-community.org
> > Koha@lists.katipo.co.nz
> > Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>
> --
> Fridolin SOMERS 
> Software and system maintainer 濾
> BibLibre, France
> ___
>
> Koha mailing list  http://koha-community.org
> Koha@lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
>
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Slow Item Edit Routines

[Rudy Hinojosa]

I only have 9 item types. I figured out what happened. My client is 
using a consortium, and discovered that 3 libraries were sharing an item 
type collectively, which is fine. However, there is a table called 
itemtype_branches that had 30k duplicates of this collection, and this 
was causing the edit screen to choke. The edit screen was attempting to 
load these duplicate records in the item type dropdown. After further 
review of this table, there is no index on it that prevents duplicates. 
So there is a C4 somewhere that is performing a union against the items 
and itemtype_branches table. This could be a bug. I manually deleted the 
duplicates, and left the 3 matching collections to each appropriate 
branch. Now the edit screen works fine. I don't know how those 
duplicates got in there.


On 8/30/23 02:00, Jonathan Druart wrote:

Hi Rudy,

Do you have a lot of item types?

Do you have a lot of items for this bibliographic record?

Regards,
Jonathan

Le mar. 22 août 2023 à 15:39, Rudy Hinojosa
  a écrit :

Using v23.05.02 release has been a great experience thus far, but I am
noticing that the system is doing something CPU or process intrinsic
when it comes to editing an item. I'm unable to edit any single field or
trigger a drop down until at least 15 seconds have passed. I'm leaning
that there is some process probably pre-loading  and populating the KOHA
ITEM TYPE drop downs. The system becomes hog tied until this process is
completed. After that, it's smooth sailing and fast saving as well.
I've disabled catalog logging changes for now to see if that helps, but
outside of that, does anyone have any suggestions. The server is beefy.
The user loads are light. All systems are nominal including memcached,
and no warning in the about us screens.

Rudy Hinojosa




___

Koha mailing listhttp://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe:https://lists.katipo.co.nz/mailman/listinfo/koha

--

photo   
Rudy E. Hinojosa
CEO/President,Lightwave Library

+1 737-372-7779   | lightwavelibrary.com 



rudy.hinoj...@lightwavelibrary.com 



25101 Ronald W. Reagan Blvd #49, Georgetown TX 78633 



___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

[Koha] Koha Community Newsletter: August 2023

[Koha Newsletter]

The Koha Community Newsletter for August 2023 is here:

https://koha-community.org/koha-community-newsletter-august-2023/

Many thanks to everyone who submitted articles and news to this newsletter!

Please feel free to email me with any corrections or suggestions.
--
Michael Kuhn
Editor, Koha Community Newsletter
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Large number of action_logs records

[Fridolin SOMERS]

Do you use update_totalissues.pl with --incremental ?

Le 30/08/2023 à 21:37, Jonathan Druart a écrit :

Hi Joel,

We could skip the logging when the biblio is modified by this cronjob,
please report this problem in a new bug report.

Regards,
Jonathan

Le mar. 29 août 2023 à 21:22, Coehoorn, Joel  a écrit :


We're seeing a large quantity of records filling the action_logs table...
nearly 18,000 in the last 24 hours. This in turn uses a LOT of disk
space... it was briefly over 90GB until I intervened, and because MySQL is
dumb about shrinking tables we were fortunate I caught it in time.

It looks like the update_totalissues.pl script is running every night and
putting the entire biblio record into the info column in the table for
every record we have.

What could be causing this, and how can I stop it (without disabling the
job completely, which I assume is part of Koha and important)?

*Joel Coehoorn*
Director of Information Technology
*York University*
Office: 402-363-5603 | jcoeho...@york.edu | york.edu

*Please contact helpd...@york.edu  for technical
assistance.*


The mission of York University is to transform lives through
Christ-centered education and to equip students for lifelong service to
God, family, and society
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha


--
Fridolin SOMERS 
Software and system maintainer 濾
BibLibre, France
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha

Re: [Koha] Large number of action_logs records

[Jonathan Druart]

Hi Joel,

We could skip the logging when the biblio is modified by this cronjob,
please report this problem in a new bug report.

Regards,
Jonathan

Le mar. 29 août 2023 à 21:22, Coehoorn, Joel  a écrit :
>
> We're seeing a large quantity of records filling the action_logs table...
> nearly 18,000 in the last 24 hours. This in turn uses a LOT of disk
> space... it was briefly over 90GB until I intervened, and because MySQL is
> dumb about shrinking tables we were fortunate I caught it in time.
>
> It looks like the update_totalissues.pl script is running every night and
> putting the entire biblio record into the info column in the table for
> every record we have.
>
> What could be causing this, and how can I stop it (without disabling the
> job completely, which I assume is part of Koha and important)?
>
> *Joel Coehoorn*
> Director of Information Technology
> *York University*
> Office: 402-363-5603 | jcoeho...@york.edu | york.edu
>
> *Please contact helpd...@york.edu  for technical
> assistance.*
>
>
> The mission of York University is to transform lives through
> Christ-centered education and to equip students for lifelong service to
> God, family, and society
> ___
>
> Koha mailing list  http://koha-community.org
> Koha@lists.katipo.co.nz
> Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha
___

Koha mailing list  http://koha-community.org
Koha@lists.katipo.co.nz
Unsubscribe: https://lists.katipo.co.nz/mailman/listinfo/koha