Re: TCL and Unicode
On Mon, Mar 22, 2021 at 06:32:29PM +0100, Dimitry Andric wrote: > On 22 Mar 2021, at 14:40, Shawn Webb wrote: > > > > I'm tracking down a regression in ports regarding TCL and unicode. The > > primary victim of the problem is databases/sqlite3. Note that I use > > freebsd-ports on github as my upstream, so I'll be using git commit > > hashes from that repo. > > > > The sqlite3 build failure can be seen at [1]. > > > > If I revert commit 787aad81fc79d441fb0c9a750e6e33b6c0ea7ac6, sqlite3 > > builds fine. I noticed a few key changes from that commit: > > > > The build of sqlite3 depends on TCL: instead of using the distfile > > that has the autoconf artifacts pre-generated, the distfile without > > the autoconf artifacts is used (changing from > > sqlite-autoconf-3340100.tar.gz to sqlite-src-3340100.zip). This > > means that the TCL-based autoconf artifacts must be generated > > locally. At least, partially. It seems that the only part of the > > build that depends on TCL is the sqlite3_analyzer. > > > > Admittedly, this change is somewhat confusing to me. I'm having > > somewhat of a hard time knowing whether it's TCL or sqlite3 itself > > as the main culprit. > > > > Any guidance is appreciated. > > > > [1]: > > http://ci-08.md.hardenedbsd.org/data/hardenedbsd-current_amd64-local/2021-03-21_13h53m43s/logs/errors/sqlite3-3.34.1,1.log > > Hi Shawn, > > It builds fine for me locally on 14.0-CURRENT (as of ~2 days ago), and > indeed the sqlite3.c file is now dynamically generated by tcl. > > In your CI failure case, it looks like something is inserting blobs of > zero bytes into the resulting file, though? So either the file system > is going bad, or tcl is outputting nonsense, for some reason. At least, > I think you'll have to do some investigations in that direction... Looks like compiling lang/tcl86 with the following CFLAGS causes tcl to output invalid unicode for some reason: -ftrivial-auto-var-init=zero -enable-trivial-auto-var-init-zero-knowing-it-will-be-removed-from-clang Thanks for the hint! -- Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc signature.asc Description: PGP signature
Re: TCL and Unicode
On 22 Mar 2021, at 14:40, Shawn Webb wrote: > > I'm tracking down a regression in ports regarding TCL and unicode. The > primary victim of the problem is databases/sqlite3. Note that I use > freebsd-ports on github as my upstream, so I'll be using git commit > hashes from that repo. > > The sqlite3 build failure can be seen at [1]. > > If I revert commit 787aad81fc79d441fb0c9a750e6e33b6c0ea7ac6, sqlite3 > builds fine. I noticed a few key changes from that commit: > > The build of sqlite3 depends on TCL: instead of using the distfile > that has the autoconf artifacts pre-generated, the distfile without > the autoconf artifacts is used (changing from > sqlite-autoconf-3340100.tar.gz to sqlite-src-3340100.zip). This > means that the TCL-based autoconf artifacts must be generated > locally. At least, partially. It seems that the only part of the > build that depends on TCL is the sqlite3_analyzer. > > Admittedly, this change is somewhat confusing to me. I'm having > somewhat of a hard time knowing whether it's TCL or sqlite3 itself > as the main culprit. > > Any guidance is appreciated. > > [1]: > http://ci-08.md.hardenedbsd.org/data/hardenedbsd-current_amd64-local/2021-03-21_13h53m43s/logs/errors/sqlite3-3.34.1,1.log Hi Shawn, It builds fine for me locally on 14.0-CURRENT (as of ~2 days ago), and indeed the sqlite3.c file is now dynamically generated by tcl. In your CI failure case, it looks like something is inserting blobs of zero bytes into the resulting file, though? So either the file system is going bad, or tcl is outputting nonsense, for some reason. At least, I think you'll have to do some investigations in that direction... -Dimitry signature.asc Description: Message signed with OpenPGP
TCL and Unicode
Hey all, I'm tracking down a regression in ports regarding TCL and unicode. The primary victim of the problem is databases/sqlite3. Note that I use freebsd-ports on github as my upstream, so I'll be using git commit hashes from that repo. The sqlite3 build failure can be seen at [1]. If I revert commit 787aad81fc79d441fb0c9a750e6e33b6c0ea7ac6, sqlite3 builds fine. I noticed a few key changes from that commit: The build of sqlite3 depends on TCL: instead of using the distfile that has the autoconf artifacts pre-generated, the distfile without the autoconf artifacts is used (changing from sqlite-autoconf-3340100.tar.gz to sqlite-src-3340100.zip). This means that the TCL-based autoconf artifacts must be generated locally. At least, partially. It seems that the only part of the build that depends on TCL is the sqlite3_analyzer. Admittedly, this change is somewhat confusing to me. I'm having somewhat of a hard time knowing whether it's TCL or sqlite3 itself as the main culprit. Any guidance is appreciated. [1]: http://ci-08.md.hardenedbsd.org/data/hardenedbsd-current_amd64-local/2021-03-21_13h53m43s/logs/errors/sqlite3-3.34.1,1.log Thanks, -- Shawn Webb Cofounder / Security Engineer HardenedBSD https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc signature.asc Description: PGP signature
