Re: [Koha-devel] Follow-up patches and why not to use them

[Joonas Kylmälä]

Hi Marcel,

On 18/06/2021 09:45, Marcel de Rooy wrote:
> Agree with Julian here. It depends.
> You could squash follow-up patches yourself too before reverting them?
> In some cases patches tell us a nice story, in a lot of cases it might
> be confusing or messy..

I think you might not have seen yet my follow-up email yet to Julian
sent on the 7th of June. Basically squashing or reverting blindly is not
an option when you want to review that the changes make sense on top of
the new code base. Then the "nice story" or "confusing/messy story"
part: as I said in the follow-up email to Julian my wish was to have as
well commits with one logical change in each so then we should have
*always* a nice story to read.

Would love to hear your opinion now with these points clarified.

Joonas
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Follow-up patches and why not to use them

[Joonas Kylmälä]

Hi,

On 04/06/2021 15:42, Julian Maurice wrote:
> Do you know you can revert multiple commits at once (ie. only one
> "revert commit" that revert a series of commits) ? Would that make it
> easier for cases like that ?
> And when trying to find all commits of a particular bug, git log --grep
> is your friend.

Yup, however you still need to verify all the changes make sense on top
of the new code base so that doesn't help.

> By enforcing a "1 commit" rule we would lose the (often) logical
> separation of patches that makes review easier.

I didn't suggest 1 commit hard rule per bug. I'm after commits
containing one logical change also.

Joonas
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Follow-up patches and why not to use them

[Joonas Kylmälä]

Hi,

I just bumped in another case of follow-up patch style causing us
trouble. In bug
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28490 I had to
spend considerable amount of time just reverting all the problematic
patches and making sure I didn't miss any related patches, instead of
just reverting one patch and knowing I would be good to go with that.
Reverting this many patches also means a lot more review work yet again
because you have no commit messages to reference to easily or if you
want to use those you have to jump from patch to patch to find out what
the change does. And this extra review work needs to be done by two
people when reverting such follow-up patch series!

If we instead asked the original author to fix the patches then this
work would need to be done only ~once. The argument against this I have
heard is that it makes the review work harder because you don't know
what has changed since the previous version. I think however that is not
very useful because the second reviewer doesn't benefit from this at all
and makes their work harder and also the first reviewer even sometimes
might review the revision after many days by which time they have
forgotten already the context so basically they end up in the same
situation as the second reviewer and have to jump between patches.

Just my two cents, I hope we can stop doing follow-ups and instead have
commits which contain one single change to decrease our time spent on
review and make sure we don't miss any problems due to having to jump
between so many contexts.

Joonas
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Replacement of mailman (mailing lists)

[Joonas Kylmälä]

Hi,

On 03/06/2021 16:07, Jonathan Druart wrote:
> Should we keep the historical mailing list or move to something
> modern, like discourse or flarum?
> Is it something:
> - we really want/need
> - we don't want/need
> - neat we could have

I personally don't have a need for forum type of solution, and I
actually prefer mailing lists so I can have all updates from all
different projects at one inbox.

> Currently the lists are not maintained (messages are not reviewed/no
> moderation), the mailman version is old, looks like there are a lot of
> email bounding, etc.

Besides the emails bounding issue, the maintenance burden would be the
same with forum software, right? Also, regarding this specific case of
mailman being outdated, would it be possible to just enable
unattended-upgrades (if debian based server) with automatic reboot to
solve this?

About email moderation, I have not noticed getting any spam through this
mailing list. If the email is not accepted won't it bounce so if someone
wants to send something (like a big file) they can contact via irc or
other channels to say there is a problem.

Joonas
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Deadlock issues when running the tests in parallel, and maybe not only in the tests?

[Joonas Kylmälä]

Hmm, based on the line
https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh#L204
I would think the ENV variable is actually set so maybe it is not this.
Any more info on the tables it locks on?

Regards,
Joonas

On 25/05/2021 18:54, Joonas Kylmälä wrote:
> Hey Victor,
> 
> can you please check whether the $ENV{KOHA_TESTING} variable is set
> correctly in your CI setup, and if it is not, does it fix the issue?
> Please see the code in C4::Circulation::SendCirculationAlert(), it is
> the only place where we do manual db locking as far as I know. It is
> something we should fix by re-architecturing the message_queue database
> table for digest messages. However, I'm interested in knowing if the
> deadlocks now happen because of this and whether it is unit test
> specific issue because at least based on the comments in
> SendCirculationAlert() it appears to be so.
> 
> Regards,
> Joonas
> 
> On 25/05/2021 18:45, Victor Grousset/tuxayo wrote:
>> Hi :)
>>
>> IIUC this is where the CI calls "prove" to run the tests
>> https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh#L213
>>
>>
>> And this is the place where KOHA_PROVE_CPUS is set.
>> https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh#L155
>>
>> It's not set in ".env" file so it should use "nproc" and this calls
>> "prove" with more than one thread.
>>
>> Is that really the case? I tried locally use to "prove -j 4" and I'm
>> getting deadlocks with the database after a few minutes.
>>
>> And do these deadlock in tests mean that Koha can hit them while running
>> in production?
>>
>> I expect the tests to fail due to messing up with each other data but a
>> deadlock is something else.
>>
>>
>> Cheers,
>>
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Deadlock issues when running the tests in parallel, and maybe not only in the tests?

[Joonas Kylmälä]

Hey Victor,

can you please check whether the $ENV{KOHA_TESTING} variable is set
correctly in your CI setup, and if it is not, does it fix the issue?
Please see the code in C4::Circulation::SendCirculationAlert(), it is
the only place where we do manual db locking as far as I know. It is
something we should fix by re-architecturing the message_queue database
table for digest messages. However, I'm interested in knowing if the
deadlocks now happen because of this and whether it is unit test
specific issue because at least based on the comments in
SendCirculationAlert() it appears to be so.

Regards,
Joonas

On 25/05/2021 18:45, Victor Grousset/tuxayo wrote:
> Hi :)
> 
> IIUC this is where the CI calls "prove" to run the tests
> https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh#L213
> 
> 
> And this is the place where KOHA_PROVE_CPUS is set.
> https://gitlab.com/koha-community/koha-testing-docker/-/blob/master/files/run.sh#L155
> 
> It's not set in ".env" file so it should use "nproc" and this calls
> "prove" with more than one thread.
> 
> Is that really the case? I tried locally use to "prove -j 4" and I'm
> getting deadlocks with the database after a few minutes.
> 
> And do these deadlock in tests mean that Koha can hit them while running
> in production?
> 
> I expect the tests to fail due to messing up with each other data but a
> deadlock is something else.
> 
> 
> Cheers,
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] [Koha] Upgrading bugzilla server

[Joonas Kylmälä]

It works now, thanks a lot! :)

On 07/04/2021 11:47, Chris Cormack wrote:
> I think that was it, can you try an attachment now please
> 
> Chris
> 
> On 7/04/21 8:37 pm, Chris Cormack wrote:
>> That might be it, or it might be something up with the mail daemon I
>> think it might be timing out trying to send the mail
>>
>> Chris
>>
>> On 7/04/21 8:33 pm, Joonas Kylmälä wrote:
>>> The 500 error went away from me as well for viewing also now. Could it
>>> be that some nginx default timeout value changed during the upgrade? The
>>> error message is:
>>>
>>> "
>>> 504 Gateway Time-out
>>> 
>>> 504 Gateway Time-out
>>> nginx/1.14.0 (Ubuntu)
>>> 
>>> 
>>> "
>>>
>>> And now I was able to get one patch to
>>> https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28013 but the
>>> other ones didn't go through.
>>>
>>> Joonas
>>>
>>> On 07/04/2021 11:27, Chris Cormack wrote:
>>>> Hmm not getting a 500 on that anymore.
>>>>
>>>> Looks like it attached it ok, not sure why it is timing out
>>>>
>>>> Chris
>>>>
>>>> On 7/04/21 7:48 pm, Joonas Kylmälä wrote:
>>>>> To reproduce the problem you can try open
>>>>> <https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119143>. It
>>>>> gives me right now Internal Server Error 500. The attachment upload via
>>>>> git-bz gave 504.
>>>>>
>>>>> Joonas
>>>>>
>>>>> On 07/04/2021 10:46, Chris Cormack wrote:
>>>>>> Interesting, bugzilla hasn't changed at all. It was an OS upgrade from
>>>>>> xenial to bionic.
>>>>>>
>>>>>> I suspect its bots hammering the site again I'll go block some ips
>>>>>>
>>>>>> Chris 
>>>>>>
>>>>>> On Wed, 7 Apr 2021, 7:40 pm Joonas Kylmälä, >>>>> <mailto:joonas.kylm...@helsinki.fi>> wrote:
>>>>>>
>>>>>> Hi Chris,
>>>>>>
>>>>>> thanks. The attachment viewing and adding new attachments stopped
>>>>>> working. It gives 504 internal server error.
>>>>>>
>>>>>> Joonas
>>>>>>
>>>>>> On 07/04/2021 00:29, Chris Cormack wrote:
>>>>>> > And it's finished.
>>>>>> >
>>>>>> > Please let me know if you spot anything weird
>>>>>> >
>>>>>> > Chris
>>>>>> >
>>>>>> > On Wed, 7 Apr 2021 at 09:10, Chris Cormack
>>>>>> mailto:ch...@bigballofwax.co.nz>> wrote:
>>>>>> >>
>>>>>> >> Hi all
>>>>>> >>
>>>>>> >> I am upgrading the OS on the bugzilla server, so things might go 
>>>>>> up
>>>>>> >> and down a bit
>>>>>> >>
>>>>>> >> I'll update when its all good to go again
>>>>>> >>
>>>>>> >> Chris
>>>>>> > ___
>>>>>> > Koha-devel mailing list
>>>>>> > Koha-devel@lists.koha-community.org
>>>>>> <mailto:Koha-devel@lists.koha-community.org>
>>>>>> > 
>>>>>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>>>>> > website : https://www.koha-community.org/
>>>>>> > git : https://git.koha-community.org/
>>>>>> > bugs : https://bugs.koha-community.org/
>>>>>> >
>>>>>>
>>>>>> -- 
>>>>>> Joonas Kylmälä
>>>>>> Tietojärjestelmäasiantuntija
>>>>>>
>>>>>> Kansalliskirjasto
>>>>>> Kirjastoverkkopalvelut
>>>>>> PL 15 (Unioninkatu 36)
>>>>>> 00014 Helsingin yliopisto
>>>>>>
>>>>>
>>>
>> ___
>> Koha-devel mailing list
>> Koha-devel@lists.koha-community.org
>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : https://www.koha-community.org/
>> git : https://git.koha-community.org/
>> bugs : https://bugs.koha-community.org/
>>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] [Koha] Upgrading bugzilla server

[Joonas Kylmälä]

The 500 error went away from me as well for viewing also now. Could it
be that some nginx default timeout value changed during the upgrade? The
error message is:

"
504 Gateway Time-out

504 Gateway Time-out
nginx/1.14.0 (Ubuntu)


"

And now I was able to get one patch to
https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=28013 but the
other ones didn't go through.

Joonas

On 07/04/2021 11:27, Chris Cormack wrote:
> Hmm not getting a 500 on that anymore.
> 
> Looks like it attached it ok, not sure why it is timing out
> 
> Chris
> 
> On 7/04/21 7:48 pm, Joonas Kylmälä wrote:
>> To reproduce the problem you can try open
>> <https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119143>. It
>> gives me right now Internal Server Error 500. The attachment upload via
>> git-bz gave 504.
>>
>> Joonas
>>
>> On 07/04/2021 10:46, Chris Cormack wrote:
>>> Interesting, bugzilla hasn't changed at all. It was an OS upgrade from
>>> xenial to bionic.
>>>
>>> I suspect its bots hammering the site again I'll go block some ips
>>>
>>> Chris 
>>>
>>> On Wed, 7 Apr 2021, 7:40 pm Joonas Kylmälä, >> <mailto:joonas.kylm...@helsinki.fi>> wrote:
>>>
>>> Hi Chris,
>>>
>>> thanks. The attachment viewing and adding new attachments stopped
>>> working. It gives 504 internal server error.
>>>
>>> Joonas
>>>
>>> On 07/04/2021 00:29, Chris Cormack wrote:
>>> > And it's finished.
>>> >
>>> > Please let me know if you spot anything weird
>>> >
>>> > Chris
>>> >
>>> > On Wed, 7 Apr 2021 at 09:10, Chris Cormack
>>> mailto:ch...@bigballofwax.co.nz>> wrote:
>>> >>
>>> >> Hi all
>>> >>
>>> >> I am upgrading the OS on the bugzilla server, so things might go up
>>> >> and down a bit
>>> >>
>>> >> I'll update when its all good to go again
>>> >>
>>> >> Chris
>>> > ___
>>> > Koha-devel mailing list
>>> > Koha-devel@lists.koha-community.org
>>> <mailto:Koha-devel@lists.koha-community.org>
>>> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>>> > website : https://www.koha-community.org/
>>> > git : https://git.koha-community.org/
>>> > bugs : https://bugs.koha-community.org/
>>> >
>>>
>>> -- 
>>> Joonas Kylmälä
>>> Tietojärjestelmäasiantuntija
>>>
>>> Kansalliskirjasto
>>> Kirjastoverkkopalvelut
>>> PL 15 (Unioninkatu 36)
>>> 00014 Helsingin yliopisto
>>>
>>

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Upgrading bugzilla server

[Joonas Kylmälä]

To reproduce the problem you can try open
<https://bugs.koha-community.org/bugzilla3/attachment.cgi?id=119143>. It
gives me right now Internal Server Error 500. The attachment upload via
git-bz gave 504.

Joonas

On 07/04/2021 10:46, Chris Cormack wrote:
> Interesting, bugzilla hasn't changed at all. It was an OS upgrade from
> xenial to bionic.
> 
> I suspect its bots hammering the site again I'll go block some ips
> 
> Chris 
> 
> On Wed, 7 Apr 2021, 7:40 pm Joonas Kylmälä,  <mailto:joonas.kylm...@helsinki.fi>> wrote:
> 
> Hi Chris,
> 
> thanks. The attachment viewing and adding new attachments stopped
> working. It gives 504 internal server error.
> 
> Joonas
> 
> On 07/04/2021 00:29, Chris Cormack wrote:
> > And it's finished.
> >
> > Please let me know if you spot anything weird
> >
> > Chris
> >
> > On Wed, 7 Apr 2021 at 09:10, Chris Cormack
> mailto:ch...@bigballofwax.co.nz>> wrote:
> >>
> >> Hi all
> >>
> >> I am upgrading the OS on the bugzilla server, so things might go up
> >> and down a bit
> >>
> >> I'll update when its all good to go again
> >>
> >> Chris
> > ___
> > Koha-devel mailing list
> > Koha-devel@lists.koha-community.org
> <mailto:Koha-devel@lists.koha-community.org>
> > https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> > website : https://www.koha-community.org/
> > git : https://git.koha-community.org/
>     > bugs : https://bugs.koha-community.org/
> >
> 
> -- 
> Joonas Kylmälä
> Tietojärjestelmäasiantuntija
> 
> Kansalliskirjasto
> Kirjastoverkkopalvelut
> PL 15 (Unioninkatu 36)
> 00014 Helsingin yliopisto
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Upgrading bugzilla server

[Joonas Kylmälä]

Hi Chris,

thanks. The attachment viewing and adding new attachments stopped
working. It gives 504 internal server error.

Joonas

On 07/04/2021 00:29, Chris Cormack wrote:
> And it's finished.
> 
> Please let me know if you spot anything weird
> 
> Chris
> 
> On Wed, 7 Apr 2021 at 09:10, Chris Cormack  wrote:
>>
>> Hi all
>>
>> I am upgrading the OS on the bugzilla server, so things might go up
>> and down a bit
>>
>> I'll update when its all good to go again
>>
>> Chris
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

On 03/03/2021 12:25, Julian Maurice wrote:
> Le 03/03/2021 à 11:18, Joonas Kylmälä a écrit :
>> Not really sure it is more work than writing the current db upgrade
>> procedures, maybe tiny bit, and a bit more if you want to be able to
>> revert the DB upgrade as well (which is optional and I don't think we
>> should even consider that for MVP implementation of this).
> 
> You at least need to modify the code so that it handles the intermediary
> state, don't you ?
> In the case of a column/table rename, you need to make sure both
> columns/tables stay in sync until the whole upgrade process is finished.

You are right, that's the extra stuff needed.

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Hi,

On 03/03/2021 11:39, Julian Maurice wrote:
> I'm not too much concerned about the "push-test-release" process, but
> more about how we consistently make the code aware of the multiple
> possible database states. Surely we'd have to enforce a new set of rules
> for writing database updates. And we'd have to test these changes very
> carefully, since a tiny mistake could mean a loss of data. Of course you
> have backups, but DB rollback means downtime, right ?

Depends on the rollback I guess, some things you could probably revert
without downtime. The possibility of loss of data due to programming
error happens now as well and is no different.

> So, that sounds like a lot of extra work for avoiding a few seconds of
> downtime (most of the time).

Not really sure it is more work than writing the current db upgrade
procedures, maybe tiny bit, and a bit more if you want to be able to
revert the DB upgrade as well (which is optional and I don't think we
should even consider that for MVP implementation of this).

> Also, what about template translations ? It takes some time to generate
> translated templates, and it can do weird things if the new code is used
> while using the old templates (or the old code with the new templates).
> Wouldn't that prevent zero-downtime upgrade ?

You redirect the traffic during upgrade to other nodes that are in
working state and after the node being upgraded is ready to receive
traffic you then can re-enable the traffic redirection there.

Joonas

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Hi

On 02/03/2021 17:56, Julian Maurice wrote:
> Does it mean that the application (Perl) code should be aware of that
> intermediary state where both columns exist at the same time ? If so,
> for how long should it be aware of that old database state ?

Yup. And answer to the second question: as long as the DB migration
takes, sometimes just a few seconds, sometimes hours (we had e.g. some
character encoding change in biblio metadata table if I'm not mistaken
and it took a loong time).

> Does it mean that, for every database schema update, you have to do
> something like this:
> - git reset --hard commit_for_1st_step
> - updatedatabase (add new column) && reload starman
> - git reset --hard commit_for_2nd_step
> - updatedatabase (remove old column) && reload starman
> ?

If you just do the reload starman before updatedatabase then yeah, I
think this represents the idea :) I found more verbose explanation from
here:
http://alexandre-masselot.blogspot.com/2012/08/continuous-deployment-in-perl-code-folks.html

> 
> Do you know another open source software that allows rolling upgrade ?
> It seems like a great idea, but from what I just read online it looks
> really hard to do it correctly.

I don't know any other software but not sure it is that difficult. What
I gather, we could use e.g. Jenkins to push the code / new debian
package that would be created for every commit. Jenkins would make sure
db schema migration is finished before installing a newer version and
the steps would be repeated until we are at the latest release.

And I want to add to to the other email I sent earlier, to clarify the
non-distributed setup for David: the one node setup would probably be
best to be done in the current fashion, shutdown koha, run all db
migrations, start koha. It seems hacky and useless to try do
downtimeless migration with that given you also don't care the service
being down during HW maintanance, etc.

Joonas

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

On 03/03/2021 09:38, Joonas Kylmälä wrote:
> Downtime, removal of odd maintenance hours and shorter feedback loop are
> the main benefits I see here.

ahah, one more thing I was thinking: I hate to spend time on doing
releases so I wanted to automate so far as just pushing to production
git branch and there it goes on it's way through unit tests and to the
prod :)

Joonas

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Hi,

On 03/03/2021 00:47, dc...@prosentient.com.au wrote:
> It seems to me that we would have to train every developer on how to write 
> their Perl code and database changes to work with this conceptual model.  

Yeah, that's correct if there is DB changes.


> That being said, I am intrigued. I hate downtime as much as the next person.

Downtime, removal of odd maintenance hours and shorter feedback loop are
the main benefits I see here.


> But how do you propose that the Koha community (and not just your 
> organisation) would make this work? At the moment, the Koha community 
> releases new versions once every month. Some libraries pick those upgrades up 
> automatically, some have sysadmins initiate upgrades, etc. I think we have to 
> assume that no human intervention is a requirement for Koha. So I'm not sure 
> how a 2-step upgrade would work for libraries without sysadmins who were 
> specifically knowledgeable about Koha. 

For such installations, non-distributed ones, there would be no change.
You could just upgrade the debian package and it would do all the
upgrade steps. If you have Koha web server on multiple different servers
then you have to do similar coordination as you have had to do already
with Koha but just with different commands.

Joonas

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Correction: "create another
column with the same name" -> create another
column with the *new* name"

On 02/03/2021 17:00, Joonas Kylmälä wrote:
> Hi,
> 
> for example if we rename a DB column in Koha now we need to stop all the
> koha web server instances (or they stop responding because because koha
> checks that db revision is correct), do DB upgrade, start all Koha web
> server instances. With rolling upgrade we would first create another
> column with the same name and write & read data to both columns during
> the 1st step of upgrade, then in 2nd step when all web server nodes are
> using latest version we could upgrade again to newer version and just
> delete the old column and start using new one.
> 
> Ahaha, there is probably better explanation online if you search
> "rolling database schema upgrade" but hopefully the above clarfied my
> intention.
> 
> Joonas
> 
> On 02/03/2021 16:47, Magnus Enger wrote:
>> Den 02.03.2021 15:33, skrev Joonas Kylmälä:
>>> Hi,
>>>
>>> I opened a bug report for introducing rolling DB upgrade support for
>>> Koha. I would like to hear if you have anything to object to that or
>>> should we just go ahead and develop that? :)
>>
>> I must confess I am not 100% certain what you mean by rolling updates
>> here. Could you give some more details?
>>
>> Best regards,
>> Magnus
>> Libriotech
>>
>> ___
>> Koha-devel mailing list
>> Koha-devel@lists.koha-community.org
>> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
>> website : https://www.koha-community.org/
>> git : https://git.koha-community.org/
>> bugs : https://bugs.koha-community.org/
> 

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Hi,

for example if we rename a DB column in Koha now we need to stop all the
koha web server instances (or they stop responding because because koha
checks that db revision is correct), do DB upgrade, start all Koha web
server instances. With rolling upgrade we would first create another
column with the same name and write & read data to both columns during
the 1st step of upgrade, then in 2nd step when all web server nodes are
using latest version we could upgrade again to newer version and just
delete the old column and start using new one.

Ahaha, there is probably better explanation online if you search
"rolling database schema upgrade" but hopefully the above clarfied my
intention.

Joonas

On 02/03/2021 16:47, Magnus Enger wrote:
> Den 02.03.2021 15:33, skrev Joonas Kylmälä:
>> Hi,
>>
>> I opened a bug report for introducing rolling DB upgrade support for
>> Koha. I would like to hear if you have anything to object to that or
>> should we just go ahead and develop that? :)
> 
> I must confess I am not 100% certain what you mean by rolling updates
> here. Could you give some more details?
> 
> Best regards,
> Magnus
> Libriotech
> 
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Rolling DB upgrades

[Joonas Kylmälä]

Hi,

I opened a bug report for introducing rolling DB upgrade support for
Koha. I would like to hear if you have anything to object to that or
should we just go ahead and develop that? :)

https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=27832

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

Re: [Koha-devel] Elasticsearch changing licences

[Joonas Kylmälä]

Ḧi,

yeah, just heard about this. We might want to start looking into some
community developed solution, maybe solr?

Joonas

On 15/01/2021 07:22, dc...@prosentient.com.au wrote:
> Hi all,
> 
>  
> 
> Have folk heard that Elastic is changing the Elasticsearch licence?
> 
>  
> 
> Depending on your sources and your beliefs, this may or may not matter,
> but certainly something to be aware of.
> 
>  
> 
> https://www.elastic.co/pricing/faq/licensing
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Performance issue with Elasticsearch in record merging (Bug 22690)

[Joonas Kylmälä]

Hi,

there has been this really bad performance issue with Elasticsearch in
the record merging feature. The bug is
. The
fix has been around for a while now and was signed-off at a time but had
to go through bunch of changes due to rebase conflicts and is needing
now sign-off again. Would anyone be interested in taking a look into this?

Joonas
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

Re: [Koha-devel] Request for sign-off: component part records feature (Bug 11175)

[Joonas Kylmälä]

Hi,

On 08/09/2020 10:15, dc...@prosentient.com.au wrote:
> I've just taken a look at comment 59 but I'm not sure why you decided to 
> rescue it based on that comment. 

I did it so that the GUI code does not need to be rewritten because it
takes time.


> I'm happy to leave some feedback on the bug report. I'll do that quickly now.

Thanks!

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

Re: [Koha-devel] Request for sign-off: component part records feature (Bug 11175)

[Joonas Kylmälä]

Hi,

thanks for taking a look! There is already some discussion in the bug
report about Elasticsearch and about the backend implementation but that
is from 2018 and earlier so I'm not sure it is up to date and is that's
why you have concerns about the approach? Please let us know in the bug
report comments, and feel free to mark "In discussion" if needed.

Also please see comment 59 for why I decided to rescue this and whether
it is okay in your opinion.

Joonas

On 08/09/2020 04:27, dc...@prosentient.com.au wrote:
> Hi Joonas,
> 
> I've taken another look at the report (my last look was in 2014), but I don't 
> think that this is the right approach. It also doesn't take into account 
> Elasticsearch. 
> 
> I might be wrong, but I don't imagine that this would pass QA or be pushed to 
> master. I'm actually tempted to mark it as "In Discussion" to be honest.
> 
> David Cook
> Software Engineer
> Prosentient Systems
> 72/330 Wattle St
> Ultimo, NSW 2007
> Australia
> 
> Office: 02 9212 0899
> Online: 02 8005 0595


-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

[Koha-devel] Request for sign-off: component part records feature (Bug 11175)

[Joonas Kylmälä]

Hi,

there has been this long-standing feature pending from 2013 for Koha for
listing component part records in the host record biblio details page. I
have rescued the original patches by submitting follow-ups but now I
haven't got any sign-offs for 2 months. I think the issue is that this
needs developers to do the sign-off since the zebra index needs to be
re-indexed to test this. So if you are a developer of a library that
uses component part records I would like to ask you take a look at this
bug 11175 [1] and sign-off if you just can! Thanks already beforehand! :)

[1] https://bugs.koha-community.org/bugzilla3/show_bug.cgi?id=11175

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

Re: [Koha-devel] [External] Re: [External] Re: [External] Re: Error updating database

[Joonas Kylmälä]

On 09/06/2020 10:43, Adrian Hilt wrote:
> After that update now my users can't add items to a MARC record, they get 
> 
> SOFTWARE ERROR
> can't open /var/log/koha/keough/intranet-error.log (permission denied)
> at /usr/share/perl5/Log/Log4perl/Appender/File.pm line 124

chown the file with the koha user you are using and it should fix this.

-- 
Joonas Kylmälä
Tietojärjestelmäasiantuntija

Kansalliskirjasto
Kirjastoverkkopalvelut
PL 15 (Unioninkatu 36)
00014 Helsingin yliopisto
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/

Re: [Koha-devel] Bug 18936 - Circulation Rules Ratification

[Joonas Kylmälä]

Hi,

if you are still interested to know the why part, it is explained in the
wiki:
.
I definitely agree it should have been in the commits too but I got
maybe too lax as a QA person since we had so many regressions fixed
already (took around 2 weeks to do the QA) the earliest patches were
send and revised already 2.5 years ago so this has been a long process
but I guess not a real excuse either.

As for the developer changelog, I'm fine with reading just the git
history but for this to work ideally I still think the follow-ups from
the patch author itself are bad and we need to be strict about enforcing
good commit messages!

On 04/02/2020 18:58, Julian Maurice wrote:
> Thanks for the clarification, but my comment was more about "how people
> could find this information" rather than "give me this information".
> Currently it's not in the bug, not in commit messages, not in any kind
> of documentation.
> 
> During the last developer irc meeting it was decided to reinforce good
> commit message guideline, and I think that could have been useful here.
> Specifically the "why" part is missing and the "what" part could have
> been more verbose.
> 
> Apart from this particular bug, I think everyone would benefit from a
> "developer changelog" where all this kind of changes (database renames,
> subroutines removals, ...) would be written.
> 
> Just my 2 cents...
> 
> PS: Not everyone has time to look at all the bugs discussed on the
> mailing list. Please don't make me feel guilty about it :)
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : http://www.koha-community.org/
git : http://git.koha-community.org/
bugs : http://bugs.koha-community.org/