Re: [Koha-devel] Interesting syscalls when Starman starting up

2023-04-26 Thread David Cook 
I was a late adopter to the REST API . And I’m just talking about 
optimizations. 

 

Yeah, the plugins are the most variable part with the spec, and certainly 
introduce some risk. Originally when I was looking at the schema validation, I 
thought about caching a MD5 checksum of the spec and caching a successful 
validation. So going forward if the checksum stayed the same, then there was no 
point in re-validating. That’s something I might look at bringing into Koha at 
some stage.

 

Still experimenting though. Got a lot on the plate at the moment, so probably 
won’t be looking at this again for a while. Thought I’d share though. 

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

From: Tomas Cohen Arazi  
Sent: Wednesday, 26 April 2023 9:57 PM
To: David Cook 
Cc: Koha Devel 
Subject: Re: [Koha-devel] Interesting syscalls when Starman starting up

 

Koha has been using the Swagger/OpenAPI plugin since... 2015.

 

The validation step is important, especially when there are API plugins. 
Plugins can break the API, and we need to check that. I'd like to hear opinions 
about that,  though. And I am open to changing my mind.

 

Best regards

 

El mié, 26 abr 2023 a las 4:13, David Cook (mailto:dc...@prosentient.com.au> >) escribió:

Hey all,

 

Years ago, I chatted with the author of Mojolicious::Plugin::OpenAPI about 
skipping the schema validation as it was causing extremely high server load.

 

He added a config option for it in a newer version, and that version is 
available for Koha 22.11. I’ve tried turning on that config option but not sure 
if it’s working. I have more exploring to do.

 

When I startup Starman for Koha 22.11, I still see a huge server load, and when 
I use strace, I see a long loop of the following:

 

newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755, st_size=4096, 
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml", 
{st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml", 
{st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0

newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096, ...}, 
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755, st_size=4096, 
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger", {st_mode=S_IFDIR|0755, 
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml", 
{st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml", 
{st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0

 

I’m guessing that might be caused by the following lines:

 

$schema->resolve( $spec_file );

my $spec = $schema->bundle->data;

 

But I’ll have to investigate more… 

 

Koha used to start up quickly and with minimal overhead. Back when I was 
looking at Mojolicious::Plugin::OpenAPI, if I skipped the validation, I could 
start up with minimal overhead as well. Looks like things have changed over the 
past few years to keep the problem happening…

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

___
Koha-devel mailing list
Koha-devel@lists.koha-community.org 
<mailto:Koha-devel@lists.koha-community.org> 
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/




 

-- 

Tomás Cohen Arazi

Theke Solutions (http://theke.io <http://theke.io/> )
✆ +

Re: [Koha-devel] Interesting syscalls when Starman starting up

2023-04-26 Thread Tomas Cohen Arazi 
Koha has been using the Swagger/OpenAPI plugin since... 2015.

The validation step is important, especially when there are API plugins.
Plugins can break the API, and we need to check that. I'd like to hear
opinions about that,  though. And I am open to changing my mind.

Best regards

El mié, 26 abr 2023 a las 4:13, David Cook ()
escribió:

> Hey all,
>
>
>
> Years ago, I chatted with the author of Mojolicious::Plugin::OpenAPI about
> skipping the schema validation as it was causing extremely high server load.
>
>
>
> He added a config option for it in a newer version, and that version is
> available for Koha 22.11. I’ve tried turning on that config option but not
> sure if it’s working. I have more exploring to do.
>
>
>
> When I startup Starman for Koha 22.11, I still see a huge server load, and
> when I use strace, I see a long loop of the following:
>
>
>
> newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...},
> AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096,
> ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger",
> {st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
> {st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
> {st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0
>
> newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...},
> AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096,
> ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755,
> st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger",
> {st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
> {st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0
>
> newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
> {st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0
>
>
>
> I’m guessing that might be caused by the following lines:
>
>
>
> $schema->resolve( $spec_file );
>
> my $spec = $schema->bundle->data;
>
>
>
> But I’ll have to investigate more…
>
>
>
> Koha used to start up quickly and with minimal overhead. Back when I was
> looking at Mojolicious::Plugin::OpenAPI, if I skipped the validation, I
> could start up with minimal overhead as well. Looks like things have
> changed over the past few years to keep the problem happening…
>
>
>
> David Cook
>
> Senior Software Engineer
>
> Prosentient Systems
>
> Suite 7.03
>
> 6a Glen St
>
> Milsons Point NSW 2061
>
> Australia
>
>
>
> Office: 02 9212 0899
>
> Online: 02 8005 0595
>
>
> ___
> Koha-devel mailing list
> Koha-devel@lists.koha-community.org
> https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
> website : https://www.koha-community.org/
> git : https://git.koha-community.org/
> bugs : https://bugs.koha-community.org/
>


-- 
Tomás Cohen Arazi
Theke Solutions (http://theke.io)
✆ +54 9351 3513384
GPG: B2F3C15F
___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/

[Koha-devel] Interesting syscalls when Starman starting up

2023-04-26 Thread David Cook 
Hey all,

 

Years ago, I chatted with the author of Mojolicious::Plugin::OpenAPI about
skipping the schema validation as it was causing extremely high server load.

 

He added a config option for it in a newer version, and that version is
available for Koha 22.11. I've tried turning on that config option but not
sure if it's working. I have more exploring to do.

 

When I startup Starman for Koha 22.11, I still see a huge server load, and
when I use strace, I see a long loop of the following:

 

newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...},
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096,
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755, st_size=4096,
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
{st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
{st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0

newfstatat(AT_FDCWD, "/usr", {st_mode=S_IFDIR|0755, st_size=4096, ...},
AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share", {st_mode=S_IFDIR|0755, st_size=4096,
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha", {st_mode=S_IFDIR|0755, st_size=4096,
...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1", {st_mode=S_IFDIR|0755,
st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger",
{st_mode=S_IFDIR|0755, st_size=4096, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
{st_mode=S_IFREG|0644, st_size=27644, ...}, AT_SYMLINK_NOFOLLOW) = 0

newfstatat(AT_FDCWD, "/usr/share/koha/api/v1/swagger/swagger.yaml",
{st_mode=S_IFREG|0644, st_size=27644, ...}, 0) = 0

 

I'm guessing that might be caused by the following lines:

 

$schema->resolve( $spec_file );

my $spec = $schema->bundle->data;

 

But I'll have to investigate more. 

 

Koha used to start up quickly and with minimal overhead. Back when I was
looking at Mojolicious::Plugin::OpenAPI, if I skipped the validation, I
could start up with minimal overhead as well. Looks like things have changed
over the past few years to keep the problem happening.

 

David Cook

Senior Software Engineer

Prosentient Systems

Suite 7.03

6a Glen St

Milsons Point NSW 2061

Australia

 

Office: 02 9212 0899

Online: 02 8005 0595

 

___
Koha-devel mailing list
Koha-devel@lists.koha-community.org
https://lists.koha-community.org/cgi-bin/mailman/listinfo/koha-devel
website : https://www.koha-community.org/
git : https://git.koha-community.org/
bugs : https://bugs.koha-community.org/