Re: Problems building 3.0 with dynamic module support
Viktor Dukhovni: > On Wed, Feb 04, 2015 at 03:31:03PM +1300, Peter wrote: > > > Well for now, then I'll just have to remove -pie, but if I can get that > > in as a feature request to make -pie work with shared=yes, then I would > > really appreciate it. Not sure if it should be considered a blocker for > > 3.0.0 or not, though, maybe it could be considered a bugfix to go into > > 3.0.1? > > We've never supported "pie", so if shared libraries don't work with > "pie" that's not a bug. Perhaps "pie" support could be considered > for 3.1. Indeed. PIE support is a new feature. New features are not added during the code freeze. You're welcome to back-port this new feature once we have tested it in Postfix 3.1 with multiple build options (shared/nonshared) and with multiple OS distributions. This does not appear to require tests on multiple hardware architectures. Wietse
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 06:15 PM, Viktor Dukhovni wrote: > And they still work I hope, ... If you can, please also check that > dynamic maps still load. I would hope so but I haven't actually run them yet. I will be pushing them out to my testing repo soon and get some people to test. Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 06:12:07PM +1300, Peter wrote: > On 02/04/2015 05:36 PM, Viktor Dukhovni wrote: > > However, if my quick hack works, let us know, at least we'll know > > what needs to be done to support this at some point later. > > It works, hardening check shows all the executables to be position > independent. And they still work I hope, ... If you can, please also check that dynamic maps still load. -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 05:36 PM, Viktor Dukhovni wrote: > However, if my quick hack works, let us know, at least we'll know > what needs to be done to support this at some point later. It works, hardening check shows all the executables to be position independent. Peter
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 05:36 PM, Viktor Dukhovni wrote: > Yes, but they did not use shared libraries. The compatible thing > to do would be a statically linked build. Once you're changing > the build, you may as well drop PIE support for now. Right, I would not have pursued pie support much further, but if your quick hack works I may as well include the patch until a better solution is offered. It's plenty easy enough to just drop the patch into the src.rpm. > However, if my quick hack works, let us know, at least we'll know > what needs to be done to support this at some point later. Yep, I'll let you know, hopefully soon. Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 05:00:40PM +1300, Peter wrote: > This is more along the lines of, I'm building 3rd-party postfix packages > for CentOS, the current stable postfix packages (sourced from Fedora) > have -pie enabled and so I'd like to keep it enabled if at all possible. Yes, but they did not use shared libraries. The compatible thing to do would be a statically linked build. Once you're changing the build, you may as well drop PIE support for now. However, if my quick hack works, let us know, at least we'll know what needs to be done to support this at some point later. -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 04:07 PM, Viktor Dukhovni wrote: > The low-level details are easy, the hard part is the interface > glue. How should users be able to specify such flags, updating > the INSTALL documentation, ... > > For a preview of a brute-force hack that makes it work, apply > the patch below: > > diff --git a/makedefs b/makedefs > index f7be08c..c8b7d74 100644 > --- a/makedefs > +++ b/makedefs > @@ -1090,7 +1090,7 @@ SYSTYPE = $SYSTYPE > _AR = $_AR > ARFL = $ARFL > _RANLIB = $_RANLIB > -SYSLIBS = $AUXLIBS $SYSLIBS $PLUGIN_AUXLIBS > +SYSLIBS = -pie $AUXLIBS $SYSLIBS $PLUGIN_AUXLIBS > CC = $CC $CCARGS \$(WARN) > OPT = $OPT > DEBUG= $DEBUG > > and configure Postfix with: > > make -f Makefile.init CCARGS="-fPIC ..." AUXLIBS="..." Thanks, I'll stick that patch in the build and see how it works. > This is not a user interface, just a proof of concept. To support > this properly we'd need to automatically enable -fPIC for all > objects when PIE is requested for executables. > > Note, good luck debugging those (even getting a stack trace) if > you ever run into trouble. I've yet to see a gdb that understands > PIE executables, perhaps I have not yet been using a sufficiently > bleeding-edge toolchain. This is more along the lines of, I'm building 3rd-party postfix packages for CentOS, the current stable postfix packages (sourced from Fedora) have -pie enabled and so I'd like to keep it enabled if at all possible. Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 03:40:51PM +1300, Peter wrote: > On 02/04/2015 03:39 PM, Viktor Dukhovni wrote: > > We've never supported "pie", so if shared libraries don't work with > > "pie" that's not a bug. Perhaps "pie" support could be considered > > for 3.1. > > Ok, I'm fine with that. The low-level details are easy, the hard part is the interface glue. How should users be able to specify such flags, updating the INSTALL documentation, ... For a preview of a brute-force hack that makes it work, apply the patch below: diff --git a/makedefs b/makedefs index f7be08c..c8b7d74 100644 --- a/makedefs +++ b/makedefs @@ -1090,7 +1090,7 @@ SYSTYPE = $SYSTYPE _AR= $_AR ARFL = $ARFL _RANLIB= $_RANLIB -SYSLIBS= $AUXLIBS $SYSLIBS $PLUGIN_AUXLIBS +SYSLIBS= -pie $AUXLIBS $SYSLIBS $PLUGIN_AUXLIBS CC = $CC $CCARGS \$(WARN) OPT= $OPT DEBUG = $DEBUG and configure Postfix with: make -f Makefile.init CCARGS="-fPIC ..." AUXLIBS="..." The "SYSLIBS" flags only get used for linked executable programs, not shared libraries, but now every object file must be PIC, hence the extra CCARGS flag. This is not a user interface, just a proof of concept. To support this properly we'd need to automatically enable -fPIC for all objects when PIE is requested for executables. Note, good luck debugging those (even getting a stack trace) if you ever run into trouble. I've yet to see a gdb that understands PIE executables, perhaps I have not yet been using a sufficiently bleeding-edge toolchain. -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 03:39 PM, Viktor Dukhovni wrote: > We've never supported "pie", so if shared libraries don't work with > "pie" that's not a bug. Perhaps "pie" support could be considered > for 3.1. Ok, I'm fine with that. Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 03:31:03PM +1300, Peter wrote: > Well for now, then I'll just have to remove -pie, but if I can get that > in as a feature request to make -pie work with shared=yes, then I would > really appreciate it. Not sure if it should be considered a blocker for > 3.0.0 or not, though, maybe it could be considered a bugfix to go into > 3.0.1? We've never supported "pie", so if shared libraries don't work with "pie" that's not a bug. Perhaps "pie" support could be considered for 3.1. -- Viktor.
Re: Problems building 3.0 with dynamic module support
Am 04.02.2015 um 03:31 schrieb Peter: On 02/04/2015 02:47 PM, Viktor Dukhovni wrote: It may be tricky, Postfix applies "AUXLIBS" when building both the final executables, and the shared libraries, but it seems that "-pie" is not appropriate for shared libraries. Additinal "makedefs" and Makefile.in logic would be required to create linker flags that apply to the executables only. Well for now, then I'll just have to remove -pie, but if I can get that in as a feature request to make -pie work with shared=yes, then I would really appreciate it. Not sure if it should be considered a blocker for 3.0.0 or not, though, maybe it could be considered a bugfix to go into 3.0.1? looks like you don't realize the difference between PIC and PIE PIE = position independent EXECUTABLE PIC = position independent CODE shared libraries (at least on x86_64) are always PIC see the difference between a .so and a executeable below Full RELRO Canary found NX enabledDSO No RPATH No RUNPATH /usr/lib64/mysql/libmysqlclient.so.18.0.0 Full RELRO Canary found NX enabledPIE enabled No RPATH No RUNPATH /usr/libexec/mysqld
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 02:46 PM, li...@rhsoft.net wrote:
> not for dynamic build but that below is from my rpmbuilder and it's a
> hardened build supporting ASLR
> AUXLIBS="-lpcre -L%{_libdir}/mysql -lmysqlclient -lm -L%{_libdir}/sasl2
> -lsasl2 -lssl -lcrypto -pie -Wl,-z,now -Wl,-z,relro,-z,noexecstack"
This is pretty similar to what I had before, it craps out as soon as you
add shared=yes to make makefiles.
Peter
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 02:47 PM, Viktor Dukhovni wrote: > It may be tricky, Postfix applies "AUXLIBS" when building both the > final executables, and the shared libraries, but it seems that > "-pie" is not appropriate for shared libraries. Additinal "makedefs" > and Makefile.in logic would be required to create linker flags that > apply to the executables only. Well for now, then I'll just have to remove -pie, but if I can get that in as a feature request to make -pie work with shared=yes, then I would really appreciate it. Not sure if it should be considered a blocker for 3.0.0 or not, though, maybe it could be considered a bugfix to go into 3.0.1? Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 02:31:37PM +1300, Peter wrote: > Can you suggest the combination with -pie that is supposed to work and > actually *does* work? It may be tricky, Postfix applies "AUXLIBS" when building both the final executables, and the shared libraries, but it seems that "-pie" is not appropriate for shared libraries. Additinal "makedefs" and Makefile.in logic would be required to create linker flags that apply to the executables only. -- Viktor.
Re: Problems building 3.0 with dynamic module support
Am 04.02.2015 um 02:31 schrieb Peter:
On 02/04/2015 01:42 PM, li...@rhsoft.net wrote:
BUT one belongs to CCARGS and the other to AUXLIBS
re-read the previous mails in this thread!
...and from one of *my* previous emails:
make makefiles shared=yes 'CCARGS=-fPIC -fPIE' 'AUXLIBS=-pie'
...also fails
Can you suggest the combination with -pie that is supposed to work and
actually *does* work?
not for dynamic build but that below is from my rpmbuilder and it's a
hardened build supporting ASLR
hardening-check /usr/libexec/postfix/master
/usr/libexec/postfix/master:
Position Independent Executable: yes
Stack protected: yes
Fortify Source functions: yes (some protected functions found)
Read-only relocations: yes
Immediate binding: yes
.rpmrc:
optflags: x86_64 -m64 -O2 -march=corei7 -mtune=corei7 -fopenmp -mmmx
-msse2 -msse3 -msse4.1 -msse4.2 -maes -mfpmath=sse -pipe
-fomit-frame-pointer -finline-functions -finline-limit=60 -fexceptions
-fstack-protector-strong --param=ssp-buffer-size=6 -D_FORTIFY_SOURCE=2
-Wstack-protector -Wformat -Werror=format-security
postfix.spec:
%build
CCARGS="-fPIC -DNO_NIS -DNO_NISPLUS -DNO_EAI -DNO_LMDB -DNO_CDB
-DNO_LDAP -DNO_PGSQL -DNO_SQLITE -DHAS_PCRE -I%{_includedir}/pcre
-DHAS_MYSQL -I%{_includedir}/mysql -DUSE_TLS -DUSE_SASL_AUTH
-DUSE_CYRUS_SASL -I%{_includedir}/sasl
-DDEF_CONFIG_DIR=\\\"%{postfix_config_dir}\\\""
AUXLIBS="-lpcre -L%{_libdir}/mysql -lmysqlclient -lm -L%{_libdir}/sasl2
-lsasl2 -lssl -lcrypto -pie -Wl,-z,now -Wl,-z,relro,-z,noexecstack"
%{__make} %{?_smp_mflags} -f Makefile.init makefiles shared=no
CCARGS="${CCARGS}" AUXLIBS="${AUXLIBS}" DEBUG="" OPT="%{optflags}
-Wno-comment -fno-strict-aliasing"
%{__make} %{?_smp_mflags} CCARGS="${CCARGS}" AUXLIBS="${AUXLIBS}"
DEBUG="" OPT="%{optflags} -Wno-comment -fno-strict-aliasing"
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 01:42 PM, li...@rhsoft.net wrote: > BUT one belongs to CCARGS and the other to AUXLIBS > re-read the previous mails in this thread! ...and from one of *my* previous emails: > make makefiles shared=yes 'CCARGS=-fPIC -fPIE' 'AUXLIBS=-pie' > > ...also fails Can you suggest the combination with -pie that is supposed to work and actually *does* work? Peter
Re: Problems building 3.0 with dynamic module support
Am 03.02.2015 um 23:35 schrieb Peter: On 02/04/2015 11:31 AM, Viktor Dukhovni wrote: make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-fPIE -pie' ...fails Of course it does. You used both "-fPIE" and "-fpie". No, I used both -fPIE and -pie (without the "f") BUT one belongs to CCARGS and the other to AUXLIBS re-read the previous mails in this thread!
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 11:31 AM, Viktor Dukhovni wrote: >> make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-fPIE -pie' >> ...fails > > Of course it does. You used both "-fPIE" and "-fpie". No, I used both -fPIE and -pie (without the "f"). Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 11:11:43AM +1300, Peter wrote: > On 02/04/2015 10:47 AM, Viktor Dukhovni wrote: > > No, not CCARGS, AUXLIBS: > > > > make -f Makefile.in shared=yes "AUXLIBS=-fPIE" makefiles > > make > > > > works with the GCC toolchain on my machine. > > make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-fPIE -pie' > ...fails Of course it does. You used both "-fPIE" and "-fpie". -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 10:47 AM, Viktor Dukhovni wrote: > No, not CCARGS, AUXLIBS: > > make -f Makefile.in shared=yes "AUXLIBS=-fPIE" makefiles > make > > works with the GCC toolchain on my machine. make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-fPIE -pie' ...fails On 02/04/2015 10:49 AM, Wietse Venema wrote: > The -pie is a linker option, so it belongs in AUXLIBS. > > The gcc manpage says that you also need to specify the -fpie or > -fPIE compiler option, so that belongs in CCARGS. make makefiles shared=yes 'CCARGS=-fPIC -fPIE' 'AUXLIBS=-pie' ...also fails I've also tried: make makefiles shared=yes 'CCARGS=-fPIC -fPIE -pie' ...which fails differently (can give details if wanted) make makefiles shared=yes 'CCARGS=-fPIC -pie' ...which works. According to the gcc docs -pie should be passed through to the linker, so in theory it should work this way, but I don't know for sure. Peter
Re: Problems building 3.0 with dynamic module support
Viktor Dukhovni: > On Wed, Feb 04, 2015 at 09:59:28AM +1300, Peter wrote: > > > I simplified it down to this and was still got the error: > > make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' > > If you want PIE support, you'll need to use "-fPIE" (upper-case). > This makes it possible to enable ASLR for the Postfix binaries and > libraries (and any system libraries linked with PIE). The -pie is a linker option, so it belongs in AUXLIBS. The gcc manpage says that you also need to specify the -fpie or -fPIE compiler option, so that belongs in CCARGS. Wietse
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 10:45:23AM +1300, Peter wrote: > On 02/04/2015 10:20 AM, Viktor Dukhovni wrote: > > On Wed, Feb 04, 2015 at 09:59:28AM +1300, Peter wrote: > > > >> I simplified it down to this and was still got the error: > >> make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' > > > > If you want PIE support, you'll need to use "-fPIE" (upper-case). > > This makes it possible to enable ASLR for the Postfix binaries and > > libraries (and any system libraries linked with PIE). > > If I add -fPIE to CCARGS I get: No, not CCARGS, AUXLIBS: make -f Makefile.in shared=yes "AUXLIBS=-fPIE" makefiles make works with the GCC toolchain on my machine. -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 10:20 AM, Viktor Dukhovni wrote: > On Wed, Feb 04, 2015 at 09:59:28AM +1300, Peter wrote: > >> I simplified it down to this and was still got the error: >> make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' > > If you want PIE support, you'll need to use "-fPIE" (upper-case). > This makes it possible to enable ASLR for the Postfix binaries and > libraries (and any system libraries linked with PIE). If I add -fPIE to CCARGS I get: /usr/bin/ld: attr_print0.o: relocation R_X86_64_PC32 against `attr_print0' can not be used when making a shared object; recompile with -fPIC It works with CCARGS="-fPIC -pie" Peter
Re: Problems building 3.0 with dynamic module support
On Wed, Feb 04, 2015 at 09:59:28AM +1300, Peter wrote: > I simplified it down to this and was still got the error: > make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' If you want PIE support, you'll need to use "-fPIE" (upper-case). This makes it possible to enable ASLR for the Postfix binaries and libraries (and any system libraries linked with PIE). -- Viktor.
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 09:59 AM, Peter wrote: > I simplified it down to this and was still got the error: > make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' > > If I remove the -pie from AUXLIBS (either from the simplified version or > the full version) it builds just fine. It also builds just fine if I > remove the shared=yes (and dynamicmaps=yes). So it appears that -pie > doesn't want to work with shared=yes. > > I honestly don't know where or why -pie was added in the first place, so > I'll remove it for now, I don't know if postfix is supposed to be > compatible with that option or not. A bit of googling shows that it's for security hardening of the code. I also found the fix. If I move -pie from AUXLIBS to CCARGS then it appears to build just fine. I think the issue is that with the new AUXLIBS_X attributes those options specified in AUXLIBS are no longer applied to everything so the linker was trying to link some position-independent code against other code that was not compiled with -pie. Moving -pie to CCARGS forces it to be applied to everything and fixes the issue. Peter
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 09:16 AM, Wietse Venema wrote: > OK, show the complete "make makefiles" command that you used without > the insanely complicated Linux build process. I have a few Linux > boxen where I can try that command myself. The full "make makefiles" was: make -f Makefile.init makefiles shared=yes dynamicmaps=yes 'CCARGS=-fPIC -DHAS_LDAP -DLDAP_DEPRECATED=1 -DHAS_PCRE -I/usr/include/pcre -DHAS_MYSQL -I/usr/include/mysql -DHAS_PGSQL -I/usr/include/pgsql -DHAS_SQLITE -DUSE_SASL_AUTH -DUSE_CYRUS_SASL -I/usr/include/sasl -DUSE_TLS -I/usr/kerberos/include -DDEF_CONFIG_DIR=\"/etc/postfix\" ' 'AUXLIBS= -L/usr/lib64/sasl2 -lsasl2 -L/usr/kerberos/lib64 -lssl -lcrypto -ldl -lz -pie -Wl,-z,relro' 'AUXLIBS_LDAP=-lldap -llber' AUXLIBS_PCRE=-lpcre 'AUXLIBS_MYSQL=-L/usr/lib64/mysql -lmysqlclient -lm' AUXLIBS_PGSQL=-lpq 'AUXLIBS_SQLITE=-lsqlite3 -lpthread' DEBUG= 'OPT=-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -Wno-comment' I simplified it down to this and was still got the error: make makefiles shared=yes 'CCARGS=-fPIC' 'AUXLIBS=-pie' If I remove the -pie from AUXLIBS (either from the simplified version or the full version) it builds just fine. It also builds just fine if I remove the shared=yes (and dynamicmaps=yes). So it appears that -pie doesn't want to work with shared=yes. I honestly don't know where or why -pie was added in the first place, so I'll remove it for now, I don't know if postfix is supposed to be compatible with that option or not. Peter
Re: Problems building 3.0 with dynamic module support
Peter: > On 02/04/2015 01:25 AM, Wietse Venema wrote: > > Execute the following commands by themselves, not as part of > > some insnaly complicated Linux build process. > > > > make makefiles > > make > > > > If that works without error, then you made a mistake with the Linux > > build process. > > Still does the same thing. I'm currently in the process of trying to > simplify the options passed to make makefiles that causes it (divide and > conquer style). Also am testing on CentOS 6 as well (this was on CentOS > 5). I'll get back with more details. OK, show the complete "make makefiles" command that you used without the insanely complicated Linux build process. I have a few Linux boxen where I can try that command myself. Wietse
Re: Problems building 3.0 with dynamic module support
On 02/04/2015 01:25 AM, Wietse Venema wrote: > Execute the following commands by themselves, not as part of > some insnaly complicated Linux build process. > > make makefiles > make > > If that works without error, then you made a mistake with the Linux > build process. Still does the same thing. I'm currently in the process of trying to simplify the options passed to make makefiles that causes it (divide and conquer style). Also am testing on CentOS 6 as well (this was on CentOS 5). I'll get back with more details. Peter
Re: Problems building 3.0 with dynamic module support
Peter: > I'm trying to build Postfix 3.0.0 with dynamic loadable module support > (it builds fine without). When I add shared=yes dynamicmaps=yes to make > makefiles I get the following (fpaste of build.log from mock): > http://paste.fedoraproject.org/180820/14229612 (http://ur1.ca/jmm0z) > > Note that the errors in the above paste start after line 906, it looks > like there are missing object files in the gcc command to me. > > Is this a bug in the build process or am I doing something wrong? Execute the following commands by themselves, not as part of some insnaly complicated Linux build process. make makefiles make If that works without error, then you made a mistake with the Linux build process. Wietse
