[Pharo-dev] Re: Status update of Libgit2 and CI issues

2022-08-23 Thread Nicolas Anquetil 

yes indeed

thanks a lot for handling this during ESUG !

nicolas

On 23/08/2022 17:07, Guillermo Polito wrote:

Hi all,

* after some hiccups, CI servers are up un running.

See for example:

https://github.com/moosetechnology/Moose/runs/7976358526?check_suite_focus=true

* Another extra fix to an issue affecting only OSX machines is on the 
pipe with backports for Pharo10 and Pharo9.


https://github.com/pharo-project/pharo/issues/11561

Please, tell us if you find some other problem.
G

El 19 ago 2022, a las 21:31, Hernán Morales Durand 
 escribió:



Thank you for the update.
I followed the thread and I imagine it was not easy to work on this 
problem.


Best regards,

Hernán


El vie, 19 ago 2022 a las 16:45, Guillermo Polito 
() escribió:


A couple of weeks ago we got reported a big recurrent failure on
CI jobs while cloning using iceberg/libgit
https://github.com/pharo-project/pharo/issues/11481.

TL;DR; So far (Friday ~17h Paris time) we managed to get
everything up and running. A new VM is being released by our CI,
all pull requests are issued and we should have everything
integrated by tonight.

=-=-=-=-=-=-=-=

More on the situation: Errors happened randomly  with

```
error reading from the zlib stream
```

or
```
bad packet length
```

Several people helped in getting a reliable script to reproduce
the issue, plus some instructions to produce a reproduction
environment using a docker container. With that in our hands, we
managed to corner the issue to a new incompatibility between old
libgit2 versions, ssh2, OpenSSL, and (at least) Github.

After testing some configurations, we decided to upgrade the
libgit and related binaries to newer versions:
 - libgit 1.4.4
 - libssh 1.9.0
 - OpenSSL 1.1.1k

This was of course not as straightforward as we would have liked.
Libgit2 1.4.4 was not compatible with its older versions and
broke our ffi bindings.
We needed to provide new VM binaries for all our major platforms,
make a new VM release, upgrade the libgit2 bindings, and make
releases of the libgit2 bindings, iceberg, and backport to Pharo
10 and Pharo 9.

TL;DR; So far (Friday ~17h Paris time) we managed to get
everything up and running. A new VM is being released by our CI,
all pull requests are issued and we should have everything
integrated by tonight.

Libgit+related compilation instructions are updated in the VM wiki

https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-ARM64-Third-Party-Dependencies

https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-X86_64-Third-Party-Dependencies

https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM64-Third-Party-Dependencies

https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM32-Third-Party-Dependencies

Libgit bindings Upgrade
    changes:

https://github.com/pharo-vcs/libgit2-pharo-bindings/commit/0e0100f7d0f3a11c3170d2f19a984a54c01a294b
    release:
https://github.com/pharo-vcs/libgit2-pharo-bindings/releases/tag/v2.2.6

Iceberg Upgrade
    pr: https://github.com/pharo-vcs/iceberg/pull/1605
    release: https://github.com/pharo-vcs/iceberg/releases/tag/v2.0.7

Upgrading library dependencies in the VM
https://github.com/pharo-project/pharo-vm/pull/447
  Release VM 9.0.16
https://github.com/pharo-project/pharo-vm/releases/tag/v9.0.16

Backports to Pharo
  Pharo10 https://github.com/pharo-project/pharo/pull/11545
  Pharo9 https://github.com/pharo-project/pharo/pull/11546

We have left for later to upgrade library dependencies for the
Windows VM (x86-64, ARM64).
Of course, any help in this direction is welcome.

Cheers,
G in behalf of the team




--
Nicolas Anquetil
RMod team -- Inria Lille

[Pharo-dev] Re: Status update of Libgit2 and CI issues

2022-08-23 Thread Guillermo Polito 
Hi all, 

* after some hiccups, CI servers are up un running.

See for example:

https://github.com/moosetechnology/Moose/runs/7976358526?check_suite_focus=true 


* Another extra fix to an issue affecting only OSX machines is on the pipe with 
backports for Pharo10 and Pharo9.

https://github.com/pharo-project/pharo/issues/11561 


Please, tell us if you find some other problem.
G

> El 19 ago 2022, a las 21:31, Hernán Morales Durand  
> escribió:
> 
> 
> Thank you for the update. 
> I followed the thread and I imagine it was not easy to work on this problem.
> 
> Best regards,
> 
> Hernán
> 
> 
> El vie, 19 ago 2022 a las 16:45, Guillermo Polito ( >) escribió:
> A couple of weeks ago we got reported a big recurrent failure on CI jobs 
> while cloning using iceberg/libgit 
> https://github.com/pharo-project/pharo/issues/11481 
> .
> 
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and 
> running. A new VM is being released by our CI, all pull requests are issued 
> and we should have everything integrated by tonight.
> 
> =-=-=-=-=-=-=-=
> 
> More on the situation: Errors happened randomly  with
> 
> ```
> error reading from the zlib stream
> ```
> 
> or
> ```
> bad packet length
> ```
> 
> Several people helped in getting a reliable script to reproduce the issue, 
> plus some instructions to produce a reproduction environment using a docker 
> container. With that in our hands, we managed to corner the issue to a new 
> incompatibility between old libgit2 versions, ssh2, OpenSSL, and (at least) 
> Github.
> 
> After testing some configurations, we decided to upgrade the libgit and 
> related binaries to newer versions:
>  - libgit 1.4.4
>  - libssh 1.9.0
>  - OpenSSL 1.1.1k
> 
> This was of course not as straightforward as we would have liked.
> Libgit2 1.4.4 was not compatible with its older versions and broke our ffi 
> bindings.
> We needed to provide new VM binaries for all our major platforms, make a new 
> VM release, upgrade the libgit2 bindings, and make releases of the libgit2 
> bindings, iceberg, and backport to Pharo 10 and Pharo 9.
> 
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and 
> running. A new VM is being released by our CI, all pull requests are issued 
> and we should have everything integrated by tonight.
> 
> Libgit+related compilation instructions are updated in the VM wiki
>   
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-ARM64-Third-Party-Dependencies
>  
> 
>   
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-X86_64-Third-Party-Dependencies
>  
> 
>   
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM64-Third-Party-Dependencies
>  
> 
>   
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM32-Third-Party-Dependencies
>  
> 
> 
> Libgit bindings Upgrade
> changes: 
> https://github.com/pharo-vcs/libgit2-pharo-bindings/commit/0e0100f7d0f3a11c3170d2f19a984a54c01a294b
>  
> 
> release: 
> https://github.com/pharo-vcs/libgit2-pharo-bindings/releases/tag/v2.2.6 
> 
> 
> Iceberg Upgrade
> pr: https://github.com/pharo-vcs/iceberg/pull/1605 
> 
> release: https://github.com/pharo-vcs/iceberg/releases/tag/v2.0.7 
> 
> 
> Upgrading library dependencies in the VM
>   https://github.com/pharo-project/pharo-vm/pull/447 
> 
>   Release VM 9.0.16 
> https://github.com/pharo-project/pharo-vm/releases/tag/v9.0.16 
> 
> 
> Backports to Pharo
>   Pharo10 https://github.com/pharo-project/pharo/pull/11545 
> 
>   Pharo9  https://github.com/pharo-project/pharo/pull/11546 
> 
> 
> We have left for later to upgrade library dependencies for the Windows VM 
> (x86-64, ARM64).
> Of course, any help in this direction is welcome.
> 
> Cheers,
> G in behalf of the team

[Pharo-dev] Re: Status update of Libgit2 and CI issues

2022-08-19 Thread Hernán Morales Durand 
Thank you for the update.
I followed the thread and I imagine it was not easy to work on this problem.

Best regards,

Hernán


El vie, 19 ago 2022 a las 16:45, Guillermo Polito (<
guillermopol...@gmail.com>) escribió:

> A couple of weeks ago we got reported a big recurrent failure on CI jobs
> while cloning using iceberg/libgit
> https://github.com/pharo-project/pharo/issues/11481.
>
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and
> running. A new VM is being released by our CI, all pull requests are issued
> and we should have everything integrated by tonight.
>
> =-=-=-=-=-=-=-=
>
> More on the situation: Errors happened randomly  with
>
> ```
> error reading from the zlib stream
> ```
>
> or
> ```
> bad packet length
> ```
>
> Several people helped in getting a reliable script to reproduce the issue,
> plus some instructions to produce a reproduction environment using a docker
> container. With that in our hands, we managed to corner the issue to a new
> incompatibility between old libgit2 versions, ssh2, OpenSSL, and (at least)
> Github.
>
> After testing some configurations, we decided to upgrade the libgit and
> related binaries to newer versions:
>  - libgit 1.4.4
>  - libssh 1.9.0
>  - OpenSSL 1.1.1k
>
> This was of course not as straightforward as we would have liked.
> Libgit2 1.4.4 was not compatible with its older versions and broke our ffi
> bindings.
> We needed to provide new VM binaries for all our major platforms, make a
> new VM release, upgrade the libgit2 bindings, and make releases of the
> libgit2 bindings, iceberg, and backport to Pharo 10 and Pharo 9.
>
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and
> running. A new VM is being released by our CI, all pull requests are issued
> and we should have everything integrated by tonight.
>
> Libgit+related compilation instructions are updated in the VM wiki
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-ARM64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-X86_64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM32-Third-Party-Dependencies
>
> Libgit bindings Upgrade
> changes:
> https://github.com/pharo-vcs/libgit2-pharo-bindings/commit/0e0100f7d0f3a11c3170d2f19a984a54c01a294b
> release:
> https://github.com/pharo-vcs/libgit2-pharo-bindings/releases/tag/v2.2.6
>
> Iceberg Upgrade
> pr: https://github.com/pharo-vcs/iceberg/pull/1605
> release: https://github.com/pharo-vcs/iceberg/releases/tag/v2.0.7
>
> Upgrading library dependencies in the VM
>   https://github.com/pharo-project/pharo-vm/pull/447
>   Release VM 9.0.16
> https://github.com/pharo-project/pharo-vm/releases/tag/v9.0.16
>
> Backports to Pharo
>   Pharo10 https://github.com/pharo-project/pharo/pull/11545
>   Pharo9  https://github.com/pharo-project/pharo/pull/11546
>
> We have left for later to upgrade library dependencies for the Windows VM
> (x86-64, ARM64).
> Of course, any help in this direction is welcome.
>
> Cheers,
> G in behalf of the team

[Pharo-dev] Re: Status update of Libgit2 and CI issues

2022-08-19 Thread Gabriel Cotelli 
Thank you for the update. Once the new VM and image versions are built by
the CI I will update also the ba-st docker Pharo images, so they start
working flawlessly again.

On Fri, Aug 19, 2022 at 11:45 AM Guillermo Polito 
wrote:

> A couple of weeks ago we got reported a big recurrent failure on CI jobs
> while cloning using iceberg/libgit
> https://github.com/pharo-project/pharo/issues/11481.
>
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and
> running. A new VM is being released by our CI, all pull requests are issued
> and we should have everything integrated by tonight.
>
> =-=-=-=-=-=-=-=
>
> More on the situation: Errors happened randomly  with
>
> ```
> error reading from the zlib stream
> ```
>
> or
> ```
> bad packet length
> ```
>
> Several people helped in getting a reliable script to reproduce the issue,
> plus some instructions to produce a reproduction environment using a docker
> container. With that in our hands, we managed to corner the issue to a new
> incompatibility between old libgit2 versions, ssh2, OpenSSL, and (at least)
> Github.
>
> After testing some configurations, we decided to upgrade the libgit and
> related binaries to newer versions:
>  - libgit 1.4.4
>  - libssh 1.9.0
>  - OpenSSL 1.1.1k
>
> This was of course not as straightforward as we would have liked.
> Libgit2 1.4.4 was not compatible with its older versions and broke our ffi
> bindings.
> We needed to provide new VM binaries for all our major platforms, make a
> new VM release, upgrade the libgit2 bindings, and make releases of the
> libgit2 bindings, iceberg, and backport to Pharo 10 and Pharo 9.
>
> TL;DR; So far (Friday ~17h Paris time) we managed to get everything up and
> running. A new VM is being released by our CI, all pull requests are issued
> and we should have everything integrated by tonight.
>
> Libgit+related compilation instructions are updated in the VM wiki
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-ARM64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-OSX-X86_64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM64-Third-Party-Dependencies
>
> https://github.com/pharo-project/pharo-vm/wiki/Building-Linux-ARM32-Third-Party-Dependencies
>
> Libgit bindings Upgrade
> changes:
> https://github.com/pharo-vcs/libgit2-pharo-bindings/commit/0e0100f7d0f3a11c3170d2f19a984a54c01a294b
> release:
> https://github.com/pharo-vcs/libgit2-pharo-bindings/releases/tag/v2.2.6
>
> Iceberg Upgrade
> pr: https://github.com/pharo-vcs/iceberg/pull/1605
> release: https://github.com/pharo-vcs/iceberg/releases/tag/v2.0.7
>
> Upgrading library dependencies in the VM
>   https://github.com/pharo-project/pharo-vm/pull/447
>   Release VM 9.0.16
> https://github.com/pharo-project/pharo-vm/releases/tag/v9.0.16
>
> Backports to Pharo
>   Pharo10 https://github.com/pharo-project/pharo/pull/11545
>   Pharo9  https://github.com/pharo-project/pharo/pull/11546
>
> We have left for later to upgrade library dependencies for the Windows VM
> (x86-64, ARM64).
> Of course, any help in this direction is welcome.
>
> Cheers,
> G in behalf of the team