Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Eneas U de Queiroz
On Fri, Mar 26, 2021 at 6:57 PM Felix Fietkau  wrote:
> I fully agree with Eneas here (though I don't like his patch for this
> issue either).

This is the first time I wrote a patch I do NOT want to be applied.  I
just want to keep the status quo.

> Here's a way to fix this:
>
> include/package-defaults.mk has this:
>
>   define Build/Prepare/Default
> $(PKG_UNPACK)
> [ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
> $(Build/Patch)
>   endef

However, this is run before the patches are even applied when QUILT=1.
$(Build/Patch) just builds the quilt patch tree.  A much simpler
solution, if we are really going to change the patches, is to just
$(CP) the file in Build/Configure.  If we move it--no matter
where--then we can't go back and forth with quilt push & pop, which
would hinder its usefulness.

>
> You can adjust it to define this in the package Makefile:
>
> define Build/Prepare
> $(PKG_UNPACK)
> [ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
> mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> $(Build/Patch)
> endef
>
> - Felix

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Rosen Penev
On Fri, Mar 26, 2021 at 2:57 PM Felix Fietkau  wrote:
>
>
> On 2021-03-26 22:39, Eneas U de Queiroz wrote:
> > On Fri, Mar 26, 2021 at 4:28 PM Rosen Penev  wrote:
> >>
> >> On Fri, Mar 26, 2021 at 5:55 AM Eneas U de Queiroz
> >>  wrote:
> >> >
> >> > On Fri, Mar 26, 2021 at 6:26 AM Rosen Penev  wrote:
> >> > > +ifeq ($(QUILT),)
> >> > > +   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> >> > > $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> >> > > +endif
> >> >
> >> > This will break compilation with QUILT, as the rename will never
> >> > happen then.  You're using this strategy with other packages, so I
> >> > won't mention them individually, but this applies to all.
> >> > I would handle it at the patch level by removing the old file and
> >> > creating the new one.
> >> It doesn't break quilt as the rename happens separately. Note that the
> >> patches were modified to refer to the old name.
> > It breaks 'make package/openssl/compile QUILT=1', because the 'mv'
> > command will never run when QUILT is not empty.  You can't run the
> > 'mv' line with quilt because the patches are not applied in
> > Build/Prepare.  However, you must ensure it is run later, or the
> > package will not compile with QUILT=1.  I can point you to
> > openwrt/packages#14894 to see why you can't just skip running stuff
> > when QUILT is defined.
> I fully agree with Eneas here (though I don't like his patch for this
> issue either). Here's a way to fix this:
>
> include/package-defaults.mk has this:
>
>   define Build/Prepare/Default
> $(PKG_UNPACK)
> [ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
> $(Build/Patch)
>   endef
>
> You can adjust it to define this in the package Makefile:
>
> define Build/Prepare
> $(PKG_UNPACK)
> [ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
> mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> $(Build/Patch)
> endef
https://github.com/openwrt/openwrt/pull/4030 is the full patchset.
Makefile hackery was NAKed so I manually "fixed" the patches.
>
> - Felix

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Felix Fietkau


On 2021-03-26 22:39, Eneas U de Queiroz wrote:
> On Fri, Mar 26, 2021 at 4:28 PM Rosen Penev  wrote:
>>
>> On Fri, Mar 26, 2021 at 5:55 AM Eneas U de Queiroz
>>  wrote:
>> >
>> > On Fri, Mar 26, 2021 at 6:26 AM Rosen Penev  wrote:
>> > > +ifeq ($(QUILT),)
>> > > +   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
>> > > $(PKG_BUILD_DIR)/engines/e_devcrypto.c
>> > > +endif
>> >
>> > This will break compilation with QUILT, as the rename will never
>> > happen then.  You're using this strategy with other packages, so I
>> > won't mention them individually, but this applies to all.
>> > I would handle it at the patch level by removing the old file and
>> > creating the new one.
>> It doesn't break quilt as the rename happens separately. Note that the
>> patches were modified to refer to the old name.
> It breaks 'make package/openssl/compile QUILT=1', because the 'mv'
> command will never run when QUILT is not empty.  You can't run the
> 'mv' line with quilt because the patches are not applied in
> Build/Prepare.  However, you must ensure it is run later, or the
> package will not compile with QUILT=1.  I can point you to
> openwrt/packages#14894 to see why you can't just skip running stuff
> when QUILT is defined.
I fully agree with Eneas here (though I don't like his patch for this
issue either). Here's a way to fix this:

include/package-defaults.mk has this:

  define Build/Prepare/Default
$(PKG_UNPACK)
[ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
$(Build/Patch)
  endef

You can adjust it to define this in the package Makefile:

define Build/Prepare
$(PKG_UNPACK)
[ ! -d ./src/ ] || $(CP) ./src/. $(PKG_BUILD_DIR)
mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
$(PKG_BUILD_DIR)/engines/e_devcrypto.c
$(Build/Patch)
endef

- Felix

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Eneas U de Queiroz
On Fri, Mar 26, 2021 at 4:28 PM Rosen Penev  wrote:
>
> On Fri, Mar 26, 2021 at 5:55 AM Eneas U de Queiroz
>  wrote:
> >
> > On Fri, Mar 26, 2021 at 6:26 AM Rosen Penev  wrote:
> > > +ifeq ($(QUILT),)
> > > +   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> > > $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> > > +endif
> >
> > This will break compilation with QUILT, as the rename will never
> > happen then.  You're using this strategy with other packages, so I
> > won't mention them individually, but this applies to all.
> > I would handle it at the patch level by removing the old file and
> > creating the new one.
> It doesn't break quilt as the rename happens separately. Note that the
> patches were modified to refer to the old name.
It breaks 'make package/openssl/compile QUILT=1', because the 'mv'
command will never run when QUILT is not empty.  You can't run the
'mv' line with quilt because the patches are not applied in
Build/Prepare.  However, you must ensure it is run later, or the
package will not compile with QUILT=1.  I can point you to
openwrt/packages#14894 to see why you can't just skip running stuff
when QUILT is defined.
> >
> > Alternatively, you can keep your strategy, adding the rename with
> > QUILT in Build/Configure, with the caveat that it will run every time
> > compile is called, so you'll need to ignore an eventual error, and
> > make sure that the original file was not recreated somewhere in the
> > build process by a previous compile run, which would clobber the
> > patched file.
> I haven't seen any issues. Locally I run make package/x/{clean,refresh}
make package/openssl/compile V=sc  QUILT=1
...
make[4]: *** No rule to make target 'engines/e_devcrypto.c', needed by
'engines/e_devcrypto.o'.  Stop.

BTW, I imagine you don't build with QUILT=1 much, do you?

make -j4 package/openssl/compile
make[2]: Entering directory '/home/equeiroz/src/openwrt/scripts/config'
make[2]: 'conf' is up to date.
make[2]: Leaving directory '/home/equeiroz/src/openwrt/scripts/config'
 make[1] package/openssl/compile
 make[2] -C package/libs/toolchain compile
 make[2] -C package/libs/zlib compile
 make[2] -C package/firmware/prism54-firmware compile
 make[2] -C package/firmware/linux-firmware compile
 make[2] -C package/kernel/linux compile
 make[2] -C package/kernel/cryptodev-linux compile
 make[2] -C package/libs/openssl compile

make -j4 package/openssl/compile QUILT=1
make[2]: Entering directory '/home/equeiroz/src/openwrt/scripts/config'
make[2]: 'conf' is up to date.
make[2]: Leaving directory '/home/equeiroz/src/openwrt/scripts/config'
 make[1] package/openssl/compile
 make[2] -C package/libs/openssl compile

It does not check dependencies every time you call compile.  Try make
package/seafile-server/compile for some fun.

Cheers,

Eneas

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Rosen Penev
On Fri, Mar 26, 2021 at 5:55 AM Eneas U de Queiroz
 wrote:
>
> Hi Rosen
>
> This patch does not apply as is, but don't write a v2 yet.
> I'm testing the bump to 1.1.1k, and I'll handle it from there, by
> using --no-renames with git format-patch.  I'm maintaining the patches
> at https://github.com/cotequeiroz/openssl, and refreshing backports
> with git is much easier than with quilt.  See comments below, as your
> patchset breaks compilation with QUILT.
>
> On Fri, Mar 26, 2021 at 6:26 AM Rosen Penev  wrote:
> >
> > quilt cannot handle file renames and ends up duplicating the file.
> > Instead of doing that, handle the renaming in the Makefile so that
> > the upstream file can change.
> >
> > Signed-off-by: Rosen Penev 
> > ---
> >  package/libs/openssl/Makefile |7 +
> >  ...o-make-the-dev-crypto-engine-dynamic.patch | 2633 +
> >  ...default-to-not-use-digests-in-engine.patch |4 +-
> >  ...to-ignore-error-when-closing-session.patch |4 +-
> >  4 files changed, 151 insertions(+), 2497 deletions(-)
> >
> > diff --git a/package/libs/openssl/Makefile b/package/libs/openssl/Makefile
> > index 436abfd94c..f3113cab6e 100644
> > --- a/package/libs/openssl/Makefile
> > +++ b/package/libs/openssl/Makefile
> > @@ -324,6 +324,13 @@ OPENSSL_TARGET:=linux-$(call 
> > qstrip,$(CONFIG_ARCH))-openwrt
> >
> >  STAMP_CONFIGURED := $(STAMP_CONFIGURED)_$(shell echo $(OPENSSL_OPTIONS) | 
> > mkhash md5)
> >
> > +define Build/Prepare
> > +   $(call Build/Prepare/Default)
>
>
> > +ifeq ($(QUILT),)
> > +   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> > $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> > +endif
>
> This will break compilation with QUILT, as the rename will never
> happen then.  You're using this strategy with other packages, so I
> won't mention them individually, but this applies to all.
> I would handle it at the patch level by removing the old file and
> creating the new one.
It doesn't break quilt as the rename happens separately. Note that the
patches were modified to refer to the old name.
>
> Alternatively, you can keep your strategy, adding the rename with
> QUILT in Build/Configure, with the caveat that it will run every time
> compile is called, so you'll need to ignore an eventual error, and
> make sure that the original file was not recreated somewhere in the
> build process by a previous compile run, which would clobber the
> patched file.
I haven't seen any issues. Locally I run make package/x/{clean,refresh}
>
> Cheers

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


Re: [PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Eneas U de Queiroz
Hi Rosen

This patch does not apply as is, but don't write a v2 yet.
I'm testing the bump to 1.1.1k, and I'll handle it from there, by
using --no-renames with git format-patch.  I'm maintaining the patches
at https://github.com/cotequeiroz/openssl, and refreshing backports
with git is much easier than with quilt.  See comments below, as your
patchset breaks compilation with QUILT.

On Fri, Mar 26, 2021 at 6:26 AM Rosen Penev  wrote:
>
> quilt cannot handle file renames and ends up duplicating the file.
> Instead of doing that, handle the renaming in the Makefile so that
> the upstream file can change.
>
> Signed-off-by: Rosen Penev 
> ---
>  package/libs/openssl/Makefile |7 +
>  ...o-make-the-dev-crypto-engine-dynamic.patch | 2633 +
>  ...default-to-not-use-digests-in-engine.patch |4 +-
>  ...to-ignore-error-when-closing-session.patch |4 +-
>  4 files changed, 151 insertions(+), 2497 deletions(-)
>
> diff --git a/package/libs/openssl/Makefile b/package/libs/openssl/Makefile
> index 436abfd94c..f3113cab6e 100644
> --- a/package/libs/openssl/Makefile
> +++ b/package/libs/openssl/Makefile
> @@ -324,6 +324,13 @@ OPENSSL_TARGET:=linux-$(call 
> qstrip,$(CONFIG_ARCH))-openwrt
>
>  STAMP_CONFIGURED := $(STAMP_CONFIGURED)_$(shell echo $(OPENSSL_OPTIONS) | 
> mkhash md5)
>
> +define Build/Prepare
> +   $(call Build/Prepare/Default)


> +ifeq ($(QUILT),)
> +   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
> $(PKG_BUILD_DIR)/engines/e_devcrypto.c
> +endif

This will break compilation with QUILT, as the rename will never
happen then.  You're using this strategy with other packages, so I
won't mention them individually, but this applies to all.
I would handle it at the patch level by removing the old file and
creating the new one.

Alternatively, you can keep your strategy, adding the rename with
QUILT in Build/Configure, with the caveat that it will run every time
compile is called, so you'll need to ignore an eventual error, and
make sure that the original file was not recreated somewhere in the
build process by a previous compile run, which would clobber the
patched file.

Cheers

___
openwrt-devel mailing list
openwrt-devel@lists.openwrt.org
https://lists.openwrt.org/mailman/listinfo/openwrt-devel


[PATCH] openssl: don't rename a file with quilt

2021-03-26 Thread Rosen Penev
quilt cannot handle file renames and ends up duplicating the file.
Instead of doing that, handle the renaming in the Makefile so that
the upstream file can change.

Signed-off-by: Rosen Penev 
---
 package/libs/openssl/Makefile |7 +
 ...o-make-the-dev-crypto-engine-dynamic.patch | 2633 +
 ...default-to-not-use-digests-in-engine.patch |4 +-
 ...to-ignore-error-when-closing-session.patch |4 +-
 4 files changed, 151 insertions(+), 2497 deletions(-)

diff --git a/package/libs/openssl/Makefile b/package/libs/openssl/Makefile
index 436abfd94c..f3113cab6e 100644
--- a/package/libs/openssl/Makefile
+++ b/package/libs/openssl/Makefile
@@ -324,6 +324,13 @@ OPENSSL_TARGET:=linux-$(call qstrip,$(CONFIG_ARCH))-openwrt
 
 STAMP_CONFIGURED := $(STAMP_CONFIGURED)_$(shell echo $(OPENSSL_OPTIONS) | 
mkhash md5)
 
+define Build/Prepare
+   $(call Build/Prepare/Default)
+ifeq ($(QUILT),)
+   mv $(PKG_BUILD_DIR)/crypto/engine/eng_devcrypto.c 
$(PKG_BUILD_DIR)/engines/e_devcrypto.c
+endif
+endef
+
 define Build/Configure
(cd $(PKG_BUILD_DIR); \
./Configure $(OPENSSL_TARGET) \
diff --git 
a/package/libs/openssl/patches/430-e_devcrypto-make-the-dev-crypto-engine-dynamic.patch
 
b/package/libs/openssl/patches/430-e_devcrypto-make-the-dev-crypto-engine-dynamic.patch
index 4218fbe76c..56d2309b72 100644
--- 
a/package/libs/openssl/patches/430-e_devcrypto-make-the-dev-crypto-engine-dynamic.patch
+++ 
b/package/libs/openssl/patches/430-e_devcrypto-make-the-dev-crypto-engine-dynamic.patch
@@ -107,2461 +107,111 @@ Signed-off-by: Eneas U de Queiroz 

  ENGINES_NO_INST=ossltest dasync
  SOURCE[dasync]=e_dasync.c
 --- a/crypto/engine/eng_devcrypto.c
-+++ /dev/null
-@@ -1,1264 +0,0 @@
--/*
-- * Copyright 2017-2019 The OpenSSL Project Authors. All Rights Reserved.
-- *
-- * Licensed under the OpenSSL license (the "License").  You may not use
-- * this file except in compliance with the License.  You can obtain a copy
-- * in the file LICENSE in the source distribution or at
-- * https://www.openssl.org/source/license.html
-- */
--
--#include "e_os.h"
--#include 
--#include 
--#include 
--#include 
--#include 
--#include 
--#include 
--
--#include 
--#include 
--#include 
--#include 
--#include 
--#include 
--
--#include "crypto/engine.h"
--
--/* #define ENGINE_DEVCRYPTO_DEBUG */
--
--#if CRYPTO_ALGORITHM_MIN < CRYPTO_ALGORITHM_MAX
--# define CHECK_BSD_STYLE_MACROS
--#endif
--
--/*
-- * ONE global file descriptor for all sessions.  This allows operations
-- * such as digest session data copying (see digest_copy()), but is also
-- * saner...  why re-open /dev/crypto for every session?
-- */
--static int cfd;
--#define DEVCRYPTO_REQUIRE_ACCELERATED 0 /* require confirmation of 
acceleration */
--#define DEVCRYPTO_USE_SOFTWARE1 /* allow software drivers */
--#define DEVCRYPTO_REJECT_SOFTWARE 2 /* only disallow confirmed software 
drivers */
--
--#define DEVCRYPTO_DEFAULT_USE_SOFDTRIVERS DEVCRYPTO_REJECT_SOFTWARE
--static int use_softdrivers = DEVCRYPTO_DEFAULT_USE_SOFDTRIVERS;
--
--/*
-- * cipher/digest status & acceleration definitions
-- * Make sure the defaults are set to 0
-- */
--struct driver_info_st {
--enum devcrypto_status_t {
--DEVCRYPTO_STATUS_FAILURE = -3, /* unusable for other reason */
--DEVCRYPTO_STATUS_NO_CIOCCPHASH   = -2, /* hash state copy not 
supported */
--DEVCRYPTO_STATUS_NO_CIOCGSESSION = -1, /* session open failed */
--DEVCRYPTO_STATUS_UNKNOWN =  0, /* not tested yet */
--DEVCRYPTO_STATUS_USABLE  =  1  /* algo can be used */
--} status;
--
--enum devcrypto_accelerated_t {
--DEVCRYPTO_NOT_ACCELERATED= -1, /* software implemented */
--DEVCRYPTO_ACCELERATION_UNKNOWN   =  0, /* acceleration support unkown 
*/
--DEVCRYPTO_ACCELERATED=  1  /* hardware accelerated */
--} accelerated;
--
--char *driver_name;
--};
--
--static int clean_devcrypto_session(struct session_op *sess) {
--if (ioctl(cfd, CIOCFSESSION, >ses) < 0) {
--SYSerr(SYS_F_IOCTL, errno);
--return 0;
--}
--memset(sess, 0, sizeof(struct session_op));
--return 1;
--}
--
--/**
-- *
-- * Ciphers
-- *
-- * Because they all do the same basic operation, we have only one set of
-- * method functions for them all to share, and a mapping table between
-- * NIDs and cryptodev IDs, with all the necessary size data.
-- *
-- */
--
--struct cipher_ctx {
--struct session_op sess;
--int op;  /* COP_ENCRYPT or COP_DECRYPT */
--unsigned long mode;  /* EVP_CIPH_*_MODE */
--
--/* to handle ctr mode being a stream cipher */
--unsigned char partial[EVP_MAX_BLOCK_LENGTH];
--unsigned int blocksize, num;
--};
--
--static const struct cipher_data_st {
--int nid;
--int blocksize;
--int keylen;
--