Re: [Koha-devel] Koha CSRF protection

[Jonathan Druart via Koha-devel]

We want to know which requests to protect (ie. Requiring a csrf token):
those having a op starting with cud-
Otherwise you could GET something that should be POSTed.
I've tried to describe this change as best as I could on the wiki, please
adjust if it's not clear enough.
https://wiki.koha-community.org/wiki/Coding_Guidelines#CSRF_protection

On Fri, 12 Apr 2024, 15:00 Julian Maurice via Koha-devel, <
koha-devel@lists.koha-community.org> wrote:

> Hi,
>
> I'm a bit late on the topic but I had a look at the different bugs and
> patches during hackfest (mainly because it didn't work for me, I will
> open a new bug report for that).
>
> There is something in it that seems to cause bugs and I don't see a
> reason for it: it's the "cud-" thing.
>
> As I understand it, now every request that create/update/delete
> something should be POST (or PUT/DELETE/PATCH) requests and have an 'op'
> parameter whose value start with 'cud-' and all other requests should be
> GET (or OPTIONS/TRACE/HEAD) requests and if they have an 'op' parameter
> it should not start with "cud-".
> Why do we need the "cud-" prefix if we can use the HTTP method for
> detecting which requests need to be protected ?
>
> What seems strange is that the current implementation will allow a POST
> request without an 'op' parameter, but will block a POST request with an
> 'op' parameter that does not start with 'cud-'.
> It looks like we could get rid of this prefix check without losing
> anything. What did I miss ?
>
> Le 04/03/2024 à 08:37, Marcel de Rooy via Koha-devel a écrit :
> > Great work!
> >
> > *From:*Koha-devel  *On
> > Behalf Of *Nick Clemens via Koha-devel
> > *Sent:* Friday, March 1, 2024 2:26 PM
> > *To:* Koha Devel ; Koha
> > 
> > *Subject:* [Koha-devel] Koha CSRF protection
> >
> > Hello all!
> >
> > We have pushed the CSRF work from 34478 and related bugs today. We know
> > there are more follow-ups needed, and have filed a series of bugs under
> > an omnibus:
> >
> > https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=36192
> > 
> >
> > We have a framapad where issues can be reported/found:
> >
> > https://annuel.framapad.org/p/koha_34478_remaining
> > 
> >
> > And we have bugs for each of the sections of the document. We need all
> > developers to submit patches when they encounter issues, and for other
> > users testing master to report found issues on the pad. Testers can
> > report issues on the pad as well.
> >
> > There is a new coding guideline - all POSTs to forms in Koha will need
> > to include a csrf token:
> >
> > https://wiki.koha-community.org/wiki/Coding_Guidelines#Security
> > 
> >
> > This has been a big work, many thanks to all involved, and there is
> > still work to be done, but this is an important fix that we must do.
> >
> > You can reach out to me on IRC (kidclamp) or via email and I will do my
> > best to help anyone contribute.
> >
> > Thanks,
> >
> > Nick
> >
> >
> > --
> >
> > Nick Clemens
> >
> > ByWater Solutions
> >
> > bywatersolutions.com 
> >
> > Phone: (888) 900-8944
> >
> > Pronouns: (he/him/his)
> > Timezone: Eastern
> >
> > Follow us:
> >
> > 
> > 
> > 
> > 
> >
> >
> > ___
> > Koha-devel mailing list
> > Koha-devel@lists.koha-community.org
> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> > website : https://www.koha-community.org/
> > git : https://git.koha-community.org/
> > bugs : https://bugs.koha-community.org/
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] calendar change request on koha-community.org

[Jonathan Druart via Koha-devel]

Hi Paul,

I've adjusted the description of the event.

Cheers,
Jonathan

Le lun. 4 mars 2024 à 15:26, Paul Poulain via Koha-devel
 a écrit :
>
> Hello all,
>
> I don't know who to ask, so asking widely, sorry.
> The hackfest announcement on https://koha-community.org/calendar/ is correct. 
> However, it points to an old email, could it point to: 
> https://lists.koha-community.org/pipermail/koha-devel/2024-February/048495.html
>
> Thanks
>
> --
> Paul Poulain, Associé-gérant / co-owner
> BibLibre, Services en logiciels libres pour les bibliothèques
> BibLibre, Open Source software and services for libraries
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Session corruption in koha-testing-docker master

[Jonathan Druart via Koha-devel]

Seen it for the first time right now.
It is inside "cas_ticket" and it contains the columns for Koha::Patrons
Seems related to bug 34893

Le mar. 13 févr. 2024 à 01:36, David Cook via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :

> Hi all,
>
>
>
> The last two days I’ve noticed that my session is getting corrupted while
> I’m working on master in koha-testing-docker. I’m not touching the session
> myself, but seemingly randomly the session gets all kinds of DIBC data
> stuffed into it, and Koha seems to be parse it and Koha throws a fatal
> error, which can only be resolved by deleting the problematic session from
> the table.
>
>
>
> Just curious if anyone else is seeing this. I really don’t think it’s
> related to anything I’m working on, but I also can’t reliably reproduce it.
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Biblios not always removed from Elasticsearch index after delete

[Jonathan Druart via Koha-devel]

Hi Emmi,

Search for "No job found" in the worker logs. Do you have jobs (in DB
table background_jobs) stuck in "new" (with
type="update_elastic_index")?
If so, search in the data column for your biblionumbers.
See also bug 35819.

Regards,
Jonathan

Le lun. 29 janv. 2024 à 13:32, Emmi Takkinen via Koha-devel
 a écrit :
>
> Hello all!
>
> We've had some reports that biblios which have been deleted via "Merge 
> records" aren't always deleted from index and are shown in the search result 
> list. When one opens such a result, Koha of course says there is no such a 
> record in database. Some of these biblios have been deleted from database 
> weeks ago and I'm able to remove them manually from index. So it's not a case 
> where Elasticsearch is just slow deleting them or that there is something 
> wrong with the Elastic object. I've also searched our logs for errors, but 
> haven't found anything that could explain this.
>
> Has anyone faced a same sort of problem? We are running Koha version 
> 22.11.00.001 
> 
>  with Elasticsearch 6.8.9.
>
> Regards,
> Emmi
> ---
> Emmi Takkinen
> Järjestelmäkehittäjä
> Koha-Suomi Oy
> emmi.takki...@koha-suomi.fi
> +358 40 021 3514
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] 22.11.12 having package dependency problems?

[Jonathan Druart via Koha-devel]

The problem should be fixed for live packages.

Le mer. 24 janv. 2024 à 06:59, David Cook via Koha-devel
 a écrit :
>
> I help out a few libraries running 22.11 straight from the community, and two 
> libraries lately have had Koha go down during automatic upgrades to 
> 22.11.12-1.
>
>
>
> koha-common mostly installs but fails to complete due to unmet dependencies:
>
>
>
> apt install koha-common -s
>
> Reading package lists... Done
>
> Building dependency tree... Done
>
> Reading state information... Done
>
> koha-common is already the newest version (22.11.12-1).
>
> You might want to run 'apt --fix-broken install' to correct these.
>
> The following packages have unmet dependencies:
>
> koha-common : Depends: koha-l10n but it is not going to be installed
>
> E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or 
> specify a solution).
>
>
>
> You can remedy this by running apt-get install koha-l10n, but I’m curious why 
> this is happening. I imagine it must be happening to more libraries around 
> the world as well…
>
>
>
> Maybe when the koha-common upgrade ran the koha-l10n package wasn’t available?
>
>
>
> Has anyone else bumped into this?
>
>
>
> I didn’t realize that we were backporting koha-l10n to other releases than 
> 23.11, so it hasn’t been on my radar until now.
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] 22.11.12 having package dependency problems?

[Jonathan Druart via Koha-devel]

Please report it.

There is another one, related:
https://gitlab.com/koha-community/koha-l10n/-/issues/1

Yours should be reported on bugzilla I'd say.

Le mer. 24 janv. 2024 à 06:59, David Cook via Koha-devel
 a écrit :
>
> I help out a few libraries running 22.11 straight from the community, and two 
> libraries lately have had Koha go down during automatic upgrades to 
> 22.11.12-1.
>
>
>
> koha-common mostly installs but fails to complete due to unmet dependencies:
>
>
>
> apt install koha-common -s
>
> Reading package lists... Done
>
> Building dependency tree... Done
>
> Reading state information... Done
>
> koha-common is already the newest version (22.11.12-1).
>
> You might want to run 'apt --fix-broken install' to correct these.
>
> The following packages have unmet dependencies:
>
> koha-common : Depends: koha-l10n but it is not going to be installed
>
> E: Unmet dependencies. Try 'apt --fix-broken install' with no packages (or 
> specify a solution).
>
>
>
> You can remedy this by running apt-get install koha-l10n, but I’m curious why 
> this is happening. I imagine it must be happening to more libraries around 
> the world as well…
>
>
>
> Maybe when the koha-common upgrade ran the koha-l10n package wasn’t available?
>
>
>
> Has anyone else bumped into this?
>
>
>
> I didn’t realize that we were backporting koha-l10n to other releases than 
> 23.11, so it hasn’t been on my radar until now.
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Translation server migrated

[Jonathan Druart via Koha-devel]

Hello,

The translation server has been migrated from Pootle to Weblate.
It's a whole new server with a complete new workflow.
The goal was to have it ready to translate 23.11, but everything is not set
up yet (downtimes may appear).

You can login with your Pootle credentials, and you should have the same
privileges (part of a language team/administrator of the team).
Note that no SMTP server (email) is configured yet.
There are some heavy tasks that will be run from 1:00 to 2:00 UTC and from
3:00 to 5:00 (or 6:00? I don't know yet!), expect a slow response time
during these slots (can be adjusted later if needed).

More information (about what has been done and what still need to be done)
can be found on the wiki page
https://wiki.koha-community.org/wiki/Translation_server_migration_weblate

Please report any problems you encounter!

Happy translating!

Cheers,
Jonathan

PS: email to the general mailing list about this migration will be sent
next week.
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Anybody still using tarballs?

[Jonathan Druart via Koha-devel]

But it won't embed po files.
Please see my reply to Victor.

Le vendredi 10 novembre 2023, Tomas Cohen Arazi  a
écrit :
> Gitlab allows you to download a tar.gz, and commits are GPG signed
already.
> Maybe we can change the workflow. We should take some time to think about
it next cycle.
> El mar, 7 nov 2023 a las 17:22, Jonathan Druart via Koha-devel (<
koha-devel@lists.koha-community.org>) escribió:
>>
>> Hi,
>> I am suggesting removing the tarball from our release process. I don't
think anybody is using it anyway.
>> It's extra work for RMaints, and it does not seem necessary.
>> I am going to suggest removing the po files from the Koha src code
starting from November releases, and that will require yet an extra
(unnecessary?) step to embed the po files to generate the tarball.
>> What do you think?
>> Cheers,
>> Jonathan
>> ___
>> Koha-devel mailing list
>> Koha-devel@lists.koha-community.org
>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : https://www.koha-community.org/
>> git : https://git.koha-community.org/
>> bugs : https://bugs.koha-community.org/
>
>
> --
> Tomás Cohen Arazi
> Theke Solutions (https://theke.io)
> ✆ +54 9351 3513384
> GPG: B2F3C15F
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Anybody still using tarballs?

[Jonathan Druart via Koha-devel]

You missed the original request, Victor. The .po files will be missing from
the src and so the git extract won't be enough.

I think we better need to have a job triggered on tags pushed that will
build and push the .tar.gz files.

Le ven. 10 nov. 2023 à 04:52, Victor Grousset/tuxayo via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :

> Hi :)
>
> To still provide tarballs without release process steps to maintain and
> perform every time there is still this:
> https://git.koha-community.org/Koha-community/Koha/tags
> No files for signature though. Though, where can people find the rmaint
> public keys anyway? So I'm not sure anyone uses this to get Koha.
>
> Hum, after trying, the forge is a bit weird about the tarball feature,
> it seems the link doesn't work. But after like 30 sec of processing it
> works. (might need to click again) And subsequent clicks work directly.
>
> Direct link still work though:
> https://git.koha-community.org/Koha-community/Koha/archive/v23.05.05.tar.gz
>
> Oh no when using the direct link the 1st time, it ends up in a 500 error
> and next try it works. (timeout?)
>
> So having the release process get the tarball URL to trigger the
> generation would work. Still a bit wack but could be an alternative to
> keep tarballs without much work. (same behavior on Forgejo, tested on
> codeberg.org but it's likely with a fast enough server the request
> doesn't timeout or something)
>
> Cheers,
>
> --
> Victor Grousset/tuxayo
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Anybody still using tarballs?

[Jonathan Druart via Koha-devel]

Hi,

I am suggesting removing the tarball from our release process. I don't
think anybody is using it anyway.
It's extra work for RMaints, and it does not seem necessary.
I am going to suggest removing the po files from the Koha src code starting
from November releases, and that will require yet an extra (unnecessary?)
step to embed the po files to generate the tarball.

What do you think?

Cheers,
Jonathan
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Using Elasticsearch in koha-testing-docker

[Jonathan Druart via Koha-devel]

About ktd you need to include one of the ES/OS compose files (
docker-compose.es* docker-compose.os*), there are bin/ktd switches.
You also need KOHA_ELASTICSEARCH set to "yes" in the .env file.

Le mar. 7 nov. 2023 à 03:55, David Cook via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :

> Hi all,
>
>
>
> I’m trying to use Elasticsearch in koha-testing-docker, and I would
> appreciate some assistance.
>
>
>
> Here’s the steps I’m taking:
>
>1. Launch koha-testing-docker
>2. Update Koha’s sysprefs
>   1. Change SearchEngine from “Zebra” to “Elasticsearch” at
>   
> http://localhost:8081/cgi-bin/koha/admin/preferences.pl?op=search=SearchEngine
>3. (Re)build Elasticsearch indexes:
>   1. koha-elasticsearch --rebuild kohadev
>4. Start Elasticsearch indexer:
>   1. koha-es-indexer --start kohadev
>
>
>
> Is there anything else to do? In a prod install, do I just need to update
> the “elasticsearch” config in koha-conf.xml?
>
>
>
> Do I leave all the Zebra things running or can I turn those off? I recall
> reading somewhere that zebra_queue might fill up if I turn it off?
>
>
>
> Thanks all!
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Last month before the release

[Jonathan Druart via Koha-devel]

selenium/patrons_search.t has been failing randomly for a while.
Failures should be reported on bug 31199.

Le lun. 6 nov. 2023 à 07:03, David Cook  a écrit :

> Also, regarding
> https://jenkins.koha-community.org/job/Koha_Master/lastCompletedBuild/testReport/
> :
>
>
>
> I’ve got a fix for “t_db_dependent_Koha_Patrons_Import_t.No warning raised
> by import_patrons” at
> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=35264
>
>
>
> It looks like Katrin fixed
> “xt_author_valid_templates_t./kohadevbox/koha/koha-tmpl/intranet-tmpl/prog/en/modules/circ/
> circulation_batch_checkouts.tt”
>
>
>
> And I can’t reproduce the problem on master for
> “t_db_dependent_selenium_patrons_search_t.Search patrons”, so I’m guessing
> someone has already fixed that one?
>
>
>
> Hopefully the next run will be fail free :D
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> *From:* David Cook 
> *Sent:* Monday, 6 November 2023 10:32 AM
> *To:* 'Jonathan Druart' 
> *Cc:* 'Tomas Cohen Arazi' ; 'koha-devel' <
> koha-devel@lists.koha-community.org>
> *Subject:* RE: [Koha-devel] Last month before the release
>
>
>
> Thanks, Jonathan. Yeah, I think having a live chat sounds like a plan.
> Communication is good.
>
>
>
> Timezones, especially mine, make it tough, but being able to point at
> documentation for policies and procedures is helpful.
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> *From:* Jonathan Druart 
> *Sent:* Friday, 3 November 2023 7:10 PM
> *To:* David Cook 
> *Cc:* Tomas Cohen Arazi ; koha-devel <
> koha-devel@lists.koha-community.org>
> *Subject:* Re: [Koha-devel] Last month before the release
>
>
>
> David, if you don't know Jenkins and you are part of the QA team, I guess
> you are not alone to be lost with our CI structure and it means we have a
> big problem. Either motivation, or tools or communication.
> I am not answering you now but maybe we need to document a bit better how
> it works for the next release, responsibilize people (asking them to fix
> what they broke instead of fixing for them) and finally maybe have a live
> chat beginning of the cycle with RMaints and QA team to (re) explain how it
> works.
>
> Le vendredi 3 novembre 2023, David Cook via Koha-devel <
> koha-devel@lists.koha-community.org> a écrit :
> > It’s not clear to me though how you look at Jenkins runs. Is it going to
> https://dashboard.koha-community.org/ and then clicking on one of the
> badges to go to Jenkins like
> https://jenkins.koha-community.org/job/Koha_Master_U22/ and then clicking
> on “Latest Test Result”?
> >
> >
> >
> > David Cook
> >
> > Senior Software Engineer
> >
> > Prosentient Systems
> >
> > Suite 7.03
> >
> > 6a Glen St
> >
> > Milsons Point NSW 2061
> >
> > Australia
> >
> >
> >
> > Office: 02 9212 0899
> >
> > Online: 02 8005 0595
> >
> >
> >
> > From: Tomas Cohen Arazi 
> > Sent: Friday, 3 November 2023 11:25 AM
> > To: David Cook 
> > Cc: koha-devel 
> > Subject: Re: [Koha-devel] Last month before the release
> >
> >
> >
> > The suggested sequence is:
> >
> >
> >
> > 1. You notice your stuff for pushed, yay!
> >
> > 2. You look at Jenkins runs for the relevant branches (e.g.
> Koha_Master_* tasks)
> >
> > 3. If some test gets broken that day, you try too see if it was you.
> Beware sometimes it is not obvious.
> >
> >
> >
> > Today there was a failure in the holds API tests, because of a bug that
> didn't touch the API, but made the code stricter (?) so the loose tests we
> had written had to be tweaked so they didn't trigger a silly error.
> >
> >
> >
> > So the devs did right, didn't introduce a bug, but made the tests fail.
> But it is all of us, specially those who got patches pushed, who should be
> looking after those failures. Team work
> >
> >
> >
> > Thanks!
> >
> >
> >
> > El jue, 2 nov 2023 21:15, David Cook 
> escribió:
> >
> > What’s the best way to do this?
> >
> >
> >
> > I think Jenkins might send emails, but I think my mail server blocked it
> ages ago as spam.
> >
> >
> >
> > David Cook
> >
> > Senior Software Engineer
> >
> > Prosentient Systems
> >
> > Suite 7.03
> >
> > 6a Glen St
> >
> > Milsons Point NSW 2061
> >
> > Australia
> >
> >
> >
> > Office: 02 9212 0899
> >
> > Online: 02 8005 0595
> >
> >
> >
> > From: Koha-devel  On
> Behalf Of Tomas Cohen Arazi via Koha-devel
> > Sent: Friday, 3 November 2023 12:31 AM
> > To: koha-devel 
> > Subject: [Koha-devel] Last month before the release
> >
> >
> >
> > Hi all.
> >
> >
> >
> > We are almost done with the release, a couple medium/big things to push
> today/tomorrow and then bug fixing.
> >
> >
> >
> > I wanted to ask y'all to please view the Jenkins tasks status anytime
> after your stuff has been pushed. I know I 

Re: [Koha-devel] Last month before the release

[Jonathan Druart via Koha-devel]

Because of 26170, I left a comment there.

Le ven. 3 nov. 2023 à 13:49, Tomas Cohen Arazi via Koha-devel
 a écrit :
>
> Hi all, this is master's latest failures:
>
> https://jenkins.koha-community.org/job/Koha_Master/lastCompletedBuild/testReport/
>
> The selenium failure has been there for a while, and the other ones are 
> recent.
>
> El jue, 2 nov 2023 a las 10:30, Tomas Cohen Arazi () 
> escribió:
>>
>> Hi all.
>>
>> We are almost done with the release, a couple medium/big things to push 
>> today/tomorrow and then bug fixing.
>>
>> I wanted to ask y'all to please view the Jenkins tasks status anytime after 
>> your stuff has been pushed. I know I said this many times, but some devs 
>> clearly break some tests that are trivially fixable and the team needs you 
>> to take care of your breakages.
>>
>> I'm doing my best to track things and fix, but you should all be worried 
>> about that.
>>
>> --
>> Tomás Cohen Arazi
>> Theke Solutions (https://theke.io)
>> ✆ +54 9351 3513384
>> GPG: B2F3C15F
>
>
>
> --
> Tomás Cohen Arazi
> Theke Solutions (https://theke.io)
> ✆ +54 9351 3513384
> GPG: B2F3C15F
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Last month before the release

[Jonathan Druart via Koha-devel]

David, if you don't know Jenkins and you are part of the QA team, I guess
you are not alone to be lost with our CI structure and it means we have a
big problem. Either motivation, or tools or communication.
I am not answering you now but maybe we need to document a bit better how
it works for the next release, responsibilize people (asking them to fix
what they broke instead of fixing for them) and finally maybe have a live
chat beginning of the cycle with RMaints and QA team to (re) explain how it
works.

Le vendredi 3 novembre 2023, David Cook via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :
> It’s not clear to me though how you look at Jenkins runs. Is it going to
https://dashboard.koha-community.org/ and then clicking on one of the
badges to go to Jenkins like
https://jenkins.koha-community.org/job/Koha_Master_U22/ and then clicking
on “Latest Test Result”?
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> From: Tomas Cohen Arazi 
> Sent: Friday, 3 November 2023 11:25 AM
> To: David Cook 
> Cc: koha-devel 
> Subject: Re: [Koha-devel] Last month before the release
>
>
>
> The suggested sequence is:
>
>
>
> 1. You notice your stuff for pushed, yay!
>
> 2. You look at Jenkins runs for the relevant branches (e.g. Koha_Master_*
tasks)
>
> 3. If some test gets broken that day, you try too see if it was you.
Beware sometimes it is not obvious.
>
>
>
> Today there was a failure in the holds API tests, because of a bug that
didn't touch the API, but made the code stricter (?) so the loose tests we
had written had to be tweaked so they didn't trigger a silly error.
>
>
>
> So the devs did right, didn't introduce a bug, but made the tests fail.
But it is all of us, specially those who got patches pushed, who should be
looking after those failures. Team work
>
>
>
> Thanks!
>
>
>
> El jue, 2 nov 2023 21:15, David Cook  escribió:
>
> What’s the best way to do this?
>
>
>
> I think Jenkins might send emails, but I think my mail server blocked it
ages ago as spam.
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> From: Koha-devel  On Behalf
Of Tomas Cohen Arazi via Koha-devel
> Sent: Friday, 3 November 2023 12:31 AM
> To: koha-devel 
> Subject: [Koha-devel] Last month before the release
>
>
>
> Hi all.
>
>
>
> We are almost done with the release, a couple medium/big things to push
today/tomorrow and then bug fixing.
>
>
>
> I wanted to ask y'all to please view the Jenkins tasks status anytime
after your stuff has been pushed. I know I said this many times, but some
devs clearly break some tests that are trivially fixable and the team needs
you to take care of your breakages.
>
>
>
> I'm doing my best to track things and fix, but you should all be worried
about that.
>
>
>
> --
>
> Tomás Cohen Arazi
>
> Theke Solutions (https://theke.io)
> ✆ +54 9351 3513384
> GPG: B2F3C15F
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Memory footprint of Koha processes

[Jonathan Druart via Koha-devel]

Some ideas:
1. You could hack the worker to process all queues, then you could have
only one worker (?)
2. Stop the worker, modify Koha::BackgroundJob->enqueue to call ->process
(not with $self, you will need to refetch the job)
3. You can remove the "use" statements from plack.psgi to reduce plack's
memory footprint (but will be slower then)
4. Investigate why the worker is now taking 150M while it was 100 after
28413
5. Care about my bugs when I am working on memory footprint, so that I
don't have the feeling to do something nobody cares about... 28410 ;)

Le lun. 23 oct. 2023 à 21:57, Philippe Blouin via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :

> Howdy!
>
> I'm trying to run a server with MANY koha instances (for educational
> purposes), but although the usage is very small, the resources required
> just make everything explode.
>
> background_jobs_worker.pl for instance uses 170M per instance, and plack
> uses 200M without anything being loaded.
>
> Any suggestion how this could be hacked (locally) to allow for 200 koha
> instances on a machine without requiring 100G of memory ?  The object model
> seems at fault here, but I see no way to go around "use
> Koha::BackgroundJobs;" at the top of the script.
>
> Thanks!
> --
> Philippe Blouin,
> Directeur de la technologie
>
> Tél.  : (833) 465-4276, poste 230
> philippe.blo...@inlibro.com
> inLibro | pour esprit libre | www.inLibro.com
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] I'm not getting your emails on bugs...

[Jonathan Druart via Koha-devel]

It's enabled, so the problem is not coming from there.

Le lun. 23 oct. 2023 à 04:59, David Cook via Koha-devel <
koha-devel@lists.koha-community.org> a écrit :

> Hi all,
>
>
>
> It looks like my “bugmail has been disabled” for the last 10 days on
> Bugzilla, so if I haven’t responded to a comment on Bugzilla, it’s not
> because I’m ignoring you. It’s because I’m not getting the emails. (If an
> administrator sees this, please re-enable my bugmail. Thanks!)
>
>
>
> I’ve been trying to weed through the contents from
> koha-b...@lists.koha-community.org which I still receive but there’s so
> many emails I’m likely to miss the vast majority.
>
>
>
> So if you need anything from me urgently, shoot me an email directly?
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Background jobs quirks

[Jonathan Druart via Koha-devel]

Hum weird, isn't DBIC supposed to deal with reconnection?
You could add the find in a try, then find again in the catch... (for debug
purpose!)
Or force a new DBMS connection for each job (C4::Context->dbh({new => 1})),
but that's not a long term solution.

Le jeu. 19 oct. 2023 à 08:04, David Cook  a
écrit :

> Just one last one….
>
>
>
> I put a Carp on to the SIGPIPE and sure enough it is the
> Koha::BackgroundJobs->find() causing the SIGPIPE to be raised.
>
>
>
> Of course, ignoring the SIGPIPE isn’t going to help because the job won’t
> be fetched.
>
>
>
> I’m not sure what the answer here is… and unfortunately I’ve ran out of
> time today. With my work task, I’ll probably abandon using the background
> jobs for the moment as I need to do something more reliable. I’ll think
> about this one later…
>
>
>
> at /kohadevbox/koha/misc/background_jobs_worker.pl line 62.
>
> main::__ANON__("PIPE") called at
> /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 932
>
> eval {...} called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line
> 932
>
>
> DBIx::Class::Storage::DBI::connected(DBIx::Class::Storage::DBI::mysql=HASH(0x55b81d683d78))
> called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm line 850
>
>
> DBIx::Class::Storage::DBI::__ANON__(DBIx::Class::Storage::BlockRunner=HASH(0x55b81f9d8d28))
> called at /usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 190
>
> DBIx::Class::Storage::BlockRunner::__ANON__() called at
> /usr/share/perl5/Context/Preserve.pm line 38
>
> Context::Preserve::preserve_context(CODE(0x55b81fa01620), "replace",
> CODE(0x55b81fa016c8)) called at
> /usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 213
>
>
> DBIx::Class::Storage::BlockRunner::_run(DBIx::Class::Storage::BlockRunner=HASH(0x55b81f9d8d28),
> CODE(0x55b81f9d8ce0)) called at
> /usr/share/perl5/DBIx/Class/Storage/BlockRunner.pm line 105
>
>
> DBIx::Class::Storage::BlockRunner::run(DBIx::Class::Storage::BlockRunner=HASH(0x55b81f9d8d28),
> CODE(0x55b81f9d8ce0)) called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm
> line 856
>
> DBIx::Class::Storage::DBI::dbh_do(undef, undef, "SELECT `me`.`id`,
> `me`.`status`, `me`.`progress`, `me`.`size`"..., ARRAY(0x55b81f453d30),
> ARRAY(0x55b81f81d380)) called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm
> line 1939
>
>
> DBIx::Class::Storage::DBI::_execute(DBIx::Class::Storage::DBI::mysql=HASH(0x55b81d683d78),
> "select", ARRAY(0x55b81f9d3278), ARRAY(0x55b81d7db5a8),
> HASH(0x55b81f16bd08), HASH(0x55b81f6cc0b8)) called at
> /usr/share/perl5/DBIx/Class/Stor
>
> age/DBI.pm line 2584
>
>
> DBIx::Class::Storage::DBI::_select(DBIx::Class::Storage::DBI::mysql=HASH(0x55b81d683d78),
> ARRAY(0x55b81f9d3278), ARRAY(0x55b81d7db5a8), HASH(0x55b81f16bd08),
> HASH(0x55b81f4ce6c0)) called at /usr/share/perl5/DBIx/Class/Storage/DBI.pm
>
> line 2761
>
>
> DBIx::Class::Storage::DBI::select_single(DBIx::Class::Storage::DBI::mysql=HASH(0x55b81d683d78),
> ARRAY(0x55b81f9d3278), ARRAY(0x55b81d7db5a8), HASH(0x55b81f16bd08),
> HASH(0x55b81f4ce6c0)) called at /usr/share/perl5/DBIx/Class/ResultSet
>
> .pm line 1101
>
>
> DBIx::Class::ResultSet::single(DBIx::Class::ResultSet=HASH(0x55b81d89b180))
> called at /usr/share/perl5/DBIx/Class/ResultSet.pm line 910
>
>
> DBIx::Class::ResultSet::find(DBIx::Class::ResultSet=HASH(0x55b81f9d6b68),
> 12) called at /kohadevbox/koha/Koha/Objects.pm line 96
>
> Koha::Objects::find("Koha::BackgroundJobs", 12) called at
> /kohadevbox/koha/misc/background_jobs_worker.pl line 132
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
>
> *From:* David Cook 
> *Sent:* Thursday, October 19, 2023 4:26 PM
> *To:* 'Koha Devel' 
> *Cc:* 'Jonathan Druart' ; 'Tomas
> Cohen Arazi' 
> *Subject:* RE: [Koha-devel] Background jobs quirks
>
>
>
> Success!
>
>
>
> I created the persistent database connection between the background job
> worker and the database, and then I restarted the database.
>
>
>
> When I tried to enqueue a background job, it’s now stuck in a “New” status
> with a Progress of “null/1”, and /var/log/koha/kohadev/worker-error.log
> says the following:
>
> 20231019 05:02:14 kohadev-koha-worker-long_tasks: client (pid 23247)
> killed by signal 13, respawning
>
>
>
> Now Starman seems to have managed to re-establish a connection without any
> errors. It’s not clear why that might be. I suppose Starman/Plack might be
> suppressing SIGPIPE errors. We might have some subtle issues under the hood
> without even realizing it.
>
>
>
> (That suddenly reminds me of that MYSQL_OPT_RECONNECT issue from August.
> MySQL 8.0.34/8.1.0 deprecate automatic reconnection. I would hope that
> DBD::MySQL would take care of that, but as we might recall that module
> isn’t really maintained…)
>
>
>
> I am testing on 22.11 so maybe it’s a bit different in master… but it’s
> interesting.
>
>
>
> David Cook
>
> Senior Software