Re: [rsyslog] Porting rsyslog to an OS that does not support dynamic linking or shared libraries
El mar., 16 oct. 2018 a las 15:48, Satyam Sharma () escribió: > > Hi Rainer, > > Thanks much for responding. > > Your recommendation to use libtool's dlopen()-equivalent API for > static libraries does seem nicely workable for my problem without much > disturbance to existing code. I will definitely try this, and would > be happy to contribute a generalized patch if it would be useful for > rsyslog. > > Just out of curiosity ... the "default built-in" modules (omfile, > omfwd, etc.) that are statically linked with rsyslogd are directly > loaded/initialized at startup in loadBuildInModules(). Could > something like this also be attempted? (Admittedly, this is uglier > solution than above.) Probably yes, but I'd say that's a hack we will not be able to implement into the mainstream. They are just there for compatibility reasons. The libtool way should really do nicely, as of my understanding this is exactly the use case it was developed for (at least they describe so in their doc). I admit I was lazy to implemnt this 100% complete as I never really got into the situation where I could need it nor did any of my customers. Rainer > > Regards, > Satyam > > On Tue, Oct 16, 2018 at 4:35 PM Rainer Gerhards > wrote: > > > > just a quick answer after having just a glimpse over your mail: look > > into libtool, it supports that use case. Fully utilizing this may > > require some changes to rsyslog, e.g. replacing dlopen() by the > > libtool equivalent. IIRC it should be fairly trivial. PRs are happily > > accepted. > > > > HTH > > Rainer > > El mar., 16 oct. 2018 a las 12:56, Satyam Sharma via rsyslog > > () escribió: > > > > > > Hello all, > > > > > > I am porting rsyslog (latest) to an embedded OS that does not support > > > shared libraries or dynamic linking at all. My system does not > > > provide dlopen()/dlsym() and friends, and my toolchain does not create > > > .so's in the first place, so my porting effort is having to update the > > > sources around these limitations. Due to the fundamental design of > > > the core rsyslog runtime, this is turning out to be a challenge, and > > > it would be great if someone can help out with my questions and point > > > me in the right direction. Please bear with the long email. > > > > > > Background: > > > > > > This port is intended to be very small, and only the default bare > > > minimum modules that are not automatically statically built into > > > rsyslogd (i.e., such as zlibw, regexp, net/netstrms/strmsrv/nsd_ptcp, > > > tcpsrv/tcpclt, imtcp, imuxsock, etc.) are required apart from default > > > statically built-in ones (i.e., omfwd, omfile, ompipe, etc.). Nothing > > > else. Naturally, I am aiming to link the aforementioned "not built-in > > > by default" modules statically with my rsyslogd, and hoping to support > > > corresponding functionalities. > > > > > > Build: > > > > > > I have already obtained a build of rsyslog for my intended > > > configuration with "... --enable-static --disable-shared > > > --enable-rsyslogd --disable-rsyslogrt --enable-inet ..." (and ALL > > > extra plugins are --disable'd). I made the following source changes > > > during the build (apart from minor porting edits appropriate for my > > > system): > > > > > > * In runtime/modules.c, I "fake" a situation as if the dlopen() / > > > dlsym() (APIs nonexistent in my OS) have failed and return error/NULL. > > > I would never specify a module that I do not support (i.e., statically > > > build into my rsyslogd) in the rsyslog.conf, so I did not expect this > > > code to be executed in the first place. (But it does; see below.) > > > > > > * In all the "not statically built in by default" modules, I have > > > added the suitable argument to the BEGINmodInit() macro. > > > > > > Error on launch: > > > > > > When I execute this rsyslogd in my OS environment, it fails and exits > > > immediately upon startup (error code 2066). Apparently, 'conf' wants > > > to load and init the 'net' module. I intended for my built liblmnet.a > > > to be statically linked into my rsyslogd, but still it searches for > > > 'lmnet.so' to try and dlopen() it, and as explained above this errors > > > out. > > > > > > Investigation and questions (and some ideas): > > > > > > 1. Running 'nm' on built rsyslogd shows that liblmnet.a / > > > liblmnsd_ptcp.a / libimtcp.a / etc. modules are not getting statically > > > linked into it at all. May be I need to edit tools/Makefile > > > appropriately and make that happen? > > > > > > 2. Is the dlopen() / dlsym() code in runtime/modules.c somehow > > > unavoidable even if those modules are statically built into rsyslogd? > > > Instead of faking an error situation, should I detect the requested > > > aforementioned module and manually call its modInit() function (once > > > they are statically linked in) in there, and setup some data > > > structures by hand to fake as if dlopen() / dlsym() mechanism actually > > > succeeded? > > > > > > Again, it would
Re: [rsyslog] Porting rsyslog to an OS that does not support dynamic linking or shared libraries
Hi Rainer, Thanks much for responding. Your recommendation to use libtool's dlopen()-equivalent API for static libraries does seem nicely workable for my problem without much disturbance to existing code. I will definitely try this, and would be happy to contribute a generalized patch if it would be useful for rsyslog. Just out of curiosity ... the "default built-in" modules (omfile, omfwd, etc.) that are statically linked with rsyslogd are directly loaded/initialized at startup in loadBuildInModules(). Could something like this also be attempted? (Admittedly, this is uglier solution than above.) Regards, Satyam On Tue, Oct 16, 2018 at 4:35 PM Rainer Gerhards wrote: > > just a quick answer after having just a glimpse over your mail: look > into libtool, it supports that use case. Fully utilizing this may > require some changes to rsyslog, e.g. replacing dlopen() by the > libtool equivalent. IIRC it should be fairly trivial. PRs are happily > accepted. > > HTH > Rainer > El mar., 16 oct. 2018 a las 12:56, Satyam Sharma via rsyslog > () escribió: > > > > Hello all, > > > > I am porting rsyslog (latest) to an embedded OS that does not support > > shared libraries or dynamic linking at all. My system does not > > provide dlopen()/dlsym() and friends, and my toolchain does not create > > .so's in the first place, so my porting effort is having to update the > > sources around these limitations. Due to the fundamental design of > > the core rsyslog runtime, this is turning out to be a challenge, and > > it would be great if someone can help out with my questions and point > > me in the right direction. Please bear with the long email. > > > > Background: > > > > This port is intended to be very small, and only the default bare > > minimum modules that are not automatically statically built into > > rsyslogd (i.e., such as zlibw, regexp, net/netstrms/strmsrv/nsd_ptcp, > > tcpsrv/tcpclt, imtcp, imuxsock, etc.) are required apart from default > > statically built-in ones (i.e., omfwd, omfile, ompipe, etc.). Nothing > > else. Naturally, I am aiming to link the aforementioned "not built-in > > by default" modules statically with my rsyslogd, and hoping to support > > corresponding functionalities. > > > > Build: > > > > I have already obtained a build of rsyslog for my intended > > configuration with "... --enable-static --disable-shared > > --enable-rsyslogd --disable-rsyslogrt --enable-inet ..." (and ALL > > extra plugins are --disable'd). I made the following source changes > > during the build (apart from minor porting edits appropriate for my > > system): > > > > * In runtime/modules.c, I "fake" a situation as if the dlopen() / > > dlsym() (APIs nonexistent in my OS) have failed and return error/NULL. > > I would never specify a module that I do not support (i.e., statically > > build into my rsyslogd) in the rsyslog.conf, so I did not expect this > > code to be executed in the first place. (But it does; see below.) > > > > * In all the "not statically built in by default" modules, I have > > added the suitable argument to the BEGINmodInit() macro. > > > > Error on launch: > > > > When I execute this rsyslogd in my OS environment, it fails and exits > > immediately upon startup (error code 2066). Apparently, 'conf' wants > > to load and init the 'net' module. I intended for my built liblmnet.a > > to be statically linked into my rsyslogd, but still it searches for > > 'lmnet.so' to try and dlopen() it, and as explained above this errors > > out. > > > > Investigation and questions (and some ideas): > > > > 1. Running 'nm' on built rsyslogd shows that liblmnet.a / > > liblmnsd_ptcp.a / libimtcp.a / etc. modules are not getting statically > > linked into it at all. May be I need to edit tools/Makefile > > appropriately and make that happen? > > > > 2. Is the dlopen() / dlsym() code in runtime/modules.c somehow > > unavoidable even if those modules are statically built into rsyslogd? > > Instead of faking an error situation, should I detect the requested > > aforementioned module and manually call its modInit() function (once > > they are statically linked in) in there, and setup some data > > structures by hand to fake as if dlopen() / dlsym() mechanism actually > > succeeded? > > > > Again, it would be great if someone with the core design knowledge > > could help me out with this. > > > > Thanks and regards, > > Satyam Sharma > > ___ > > rsyslog mailing list > > http://lists.adiscon.net/mailman/listinfo/rsyslog > > http://www.rsyslog.com/professional-services/ > > What's up with rsyslog? Follow https://twitter.com/rgerhards > > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > > LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog
Re: [rsyslog] Porting rsyslog to an OS that does not support dynamic linking or shared libraries
just a quick answer after having just a glimpse over your mail: look into libtool, it supports that use case. Fully utilizing this may require some changes to rsyslog, e.g. replacing dlopen() by the libtool equivalent. IIRC it should be fairly trivial. PRs are happily accepted. HTH Rainer El mar., 16 oct. 2018 a las 12:56, Satyam Sharma via rsyslog () escribió: > > Hello all, > > I am porting rsyslog (latest) to an embedded OS that does not support > shared libraries or dynamic linking at all. My system does not > provide dlopen()/dlsym() and friends, and my toolchain does not create > .so's in the first place, so my porting effort is having to update the > sources around these limitations. Due to the fundamental design of > the core rsyslog runtime, this is turning out to be a challenge, and > it would be great if someone can help out with my questions and point > me in the right direction. Please bear with the long email. > > Background: > > This port is intended to be very small, and only the default bare > minimum modules that are not automatically statically built into > rsyslogd (i.e., such as zlibw, regexp, net/netstrms/strmsrv/nsd_ptcp, > tcpsrv/tcpclt, imtcp, imuxsock, etc.) are required apart from default > statically built-in ones (i.e., omfwd, omfile, ompipe, etc.). Nothing > else. Naturally, I am aiming to link the aforementioned "not built-in > by default" modules statically with my rsyslogd, and hoping to support > corresponding functionalities. > > Build: > > I have already obtained a build of rsyslog for my intended > configuration with "... --enable-static --disable-shared > --enable-rsyslogd --disable-rsyslogrt --enable-inet ..." (and ALL > extra plugins are --disable'd). I made the following source changes > during the build (apart from minor porting edits appropriate for my > system): > > * In runtime/modules.c, I "fake" a situation as if the dlopen() / > dlsym() (APIs nonexistent in my OS) have failed and return error/NULL. > I would never specify a module that I do not support (i.e., statically > build into my rsyslogd) in the rsyslog.conf, so I did not expect this > code to be executed in the first place. (But it does; see below.) > > * In all the "not statically built in by default" modules, I have > added the suitable argument to the BEGINmodInit() macro. > > Error on launch: > > When I execute this rsyslogd in my OS environment, it fails and exits > immediately upon startup (error code 2066). Apparently, 'conf' wants > to load and init the 'net' module. I intended for my built liblmnet.a > to be statically linked into my rsyslogd, but still it searches for > 'lmnet.so' to try and dlopen() it, and as explained above this errors > out. > > Investigation and questions (and some ideas): > > 1. Running 'nm' on built rsyslogd shows that liblmnet.a / > liblmnsd_ptcp.a / libimtcp.a / etc. modules are not getting statically > linked into it at all. May be I need to edit tools/Makefile > appropriately and make that happen? > > 2. Is the dlopen() / dlsym() code in runtime/modules.c somehow > unavoidable even if those modules are statically built into rsyslogd? > Instead of faking an error situation, should I detect the requested > aforementioned module and manually call its modInit() function (once > they are statically linked in) in there, and setup some data > structures by hand to fake as if dlopen() / dlsym() mechanism actually > succeeded? > > Again, it would be great if someone with the core design knowledge > could help me out with this. > > Thanks and regards, > Satyam Sharma > ___ > rsyslog mailing list > http://lists.adiscon.net/mailman/listinfo/rsyslog > http://www.rsyslog.com/professional-services/ > What's up with rsyslog? Follow https://twitter.com/rgerhards > NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of > sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T > LIKE THAT. ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
[rsyslog] Porting rsyslog to an OS that does not support dynamic linking or shared libraries
Hello all, I am porting rsyslog (latest) to an embedded OS that does not support shared libraries or dynamic linking at all. My system does not provide dlopen()/dlsym() and friends, and my toolchain does not create .so's in the first place, so my porting effort is having to update the sources around these limitations. Due to the fundamental design of the core rsyslog runtime, this is turning out to be a challenge, and it would be great if someone can help out with my questions and point me in the right direction. Please bear with the long email. Background: This port is intended to be very small, and only the default bare minimum modules that are not automatically statically built into rsyslogd (i.e., such as zlibw, regexp, net/netstrms/strmsrv/nsd_ptcp, tcpsrv/tcpclt, imtcp, imuxsock, etc.) are required apart from default statically built-in ones (i.e., omfwd, omfile, ompipe, etc.). Nothing else. Naturally, I am aiming to link the aforementioned "not built-in by default" modules statically with my rsyslogd, and hoping to support corresponding functionalities. Build: I have already obtained a build of rsyslog for my intended configuration with "... --enable-static --disable-shared --enable-rsyslogd --disable-rsyslogrt --enable-inet ..." (and ALL extra plugins are --disable'd). I made the following source changes during the build (apart from minor porting edits appropriate for my system): * In runtime/modules.c, I "fake" a situation as if the dlopen() / dlsym() (APIs nonexistent in my OS) have failed and return error/NULL. I would never specify a module that I do not support (i.e., statically build into my rsyslogd) in the rsyslog.conf, so I did not expect this code to be executed in the first place. (But it does; see below.) * In all the "not statically built in by default" modules, I have added the suitable argument to the BEGINmodInit() macro. Error on launch: When I execute this rsyslogd in my OS environment, it fails and exits immediately upon startup (error code 2066). Apparently, 'conf' wants to load and init the 'net' module. I intended for my built liblmnet.a to be statically linked into my rsyslogd, but still it searches for 'lmnet.so' to try and dlopen() it, and as explained above this errors out. Investigation and questions (and some ideas): 1. Running 'nm' on built rsyslogd shows that liblmnet.a / liblmnsd_ptcp.a / libimtcp.a / etc. modules are not getting statically linked into it at all. May be I need to edit tools/Makefile appropriately and make that happen? 2. Is the dlopen() / dlsym() code in runtime/modules.c somehow unavoidable even if those modules are statically built into rsyslogd? Instead of faking an error situation, should I detect the requested aforementioned module and manually call its modInit() function (once they are statically linked in) in there, and setup some data structures by hand to fake as if dlopen() / dlsym() mechanism actually succeeded? Again, it would be great if someone with the core design knowledge could help me out with this. Thanks and regards, Satyam Sharma ___ rsyslog mailing list http://lists.adiscon.net/mailman/listinfo/rsyslog http://www.rsyslog.com/professional-services/ What's up with rsyslog? Follow https://twitter.com/rgerhards NOTE WELL: This is a PUBLIC mailing list, posts are ARCHIVED by a myriad of sites beyond our control. PLEASE UNSUBSCRIBE and DO NOT POST if you DON'T LIKE THAT.
