Thank you! Mystery solved, I suppose.
On Thu, Nov 14, 2024 at 6:28 AM Martin Simmons <mar...@lispworks.com> wrote:
> The problem is that you have installed a copy of llvm15 that was compiled
> for
> FreeBSD 13.3 ("FreeBSD_version: 1303001" in the pkg info). That depends on
> the new symbol _ZNSt3__122__libcpp_verbose_abortEPKcz from the FreeBSD 13.3
> version of /usr/lib/libc++.so.1, but your jail will have the FreeBSD 13.2
> version of /usr/lib/libc++.so.1 without that symbol.
>
> I suspect that you installed llvm15 using pkg install after FreeBSD 13.2
> was
> EoL. At that point, the 13.x packages on the FreeBSD pkg servers will be
> compiled for FreeBSD 13.3 so you will get the wrong version. The FreeBSD
> pkg
> servers don't support EoL versions of FreeBSD, but I would expect pkg
> install
> to give a warning about this.
>
> The reason why it has only just failed is probably because jit is only
> used if
> PostgreSQL calculates that it is useful:
>
> https://www.postgresql.org/docs/15/jit-decision.html
>
> __Martin
>
>
> >>>>> On Wed, 13 Nov 2024 15:25:27 -0500, B Smith said:
> >
> > Sudo pkg check -s found no issues.
> >
> > Here is the pkg info and ldd outputs:
> > # pkg info llvm15
> > llvm15-15.0.7_10
> > Name : llvm15
> > Version : 15.0.7_10
> > Installed on :
> > Shared Libs required:
> > libzstd.so.1
> > libpython3.11.so.1.0
> > liblua-5.3.so
> > libedit.so.0
> > Shared Libs provided:
> > libomptarget.so.15
> > libomptarget.rtl.amdgpu.so.15
> > libomp.so
> > liblldb.so.15
> > libclang_rt.ubsan_standalone-x86_64.so
> > libclang_rt.ubsan_standalone-i386.so
> > libclang_rt.ubsan_minimal-x86_64.so
> > libclang_rt.ubsan_minimal-i386.so
> > libclang_rt.tsan-x86_64.so
> > libclang_rt.dyndd-x86_64.so
> > libclang_rt.asan-x86_64.so
> > libclang_rt.asan-i386.so
> > libclang.so.15
> > libclang-cpp.so.15
> > libarcher.so
> > libRemarks.so.15
> > libLTO.so.15
> > libLLVM-15.so
> > Annotations :
> > FreeBSD_version: 1303001
> > build_timestamp: 2024-07-09T01:34:12+0000
> > built_by : poudriere-git-3.4.1-30-g79e3edcd
> > cpe : cpe:2.3:a:llvm:llvm:15.0.7:::::freebsd13:x64:10
> > flavor : default
> > port_checkout_unclean: no
> > port_git_hash : be013aced
> > ports_top_checkout_unclean: no
> > ports_top_git_hash: be013aced
> > repo_type : binary
> > repository : FreeBSD
> > Flat size : 966MiB
> > Description :
> > The LLVM Project is a collection of modular and reusable compiler and
> > toolchain technologies.
> >
> > This port includes Clang (a C/C++/Objective-C compiler), LLD (a linker),
> > LLDB (a debugger), an OpenMP runtime library, and the LLVM infrastructure
> > these are built on.
> >
> > # ldd /usr/local/llvm15/lib/libLLVM-15.so
> > /usr/local/llvm15/lib/libLLVM-15.so:
> > librt.so.1 => /usr/lib/librt.so.1 (0x801486000)
> > libexecinfo.so.1 => /usr/lib/libexecinfo.so.1 (0x80148f000)
> > libthr.so.3 => /lib/libthr.so.3 (0x801495000)
> > libm.so.5 => /lib/libm.so.5 (0x8014c2000)
> > libz.so.6 => /lib/libz.so.6 (0x8014fe000)
> > libzstd.so.1 => /usr/local/lib/libzstd.so.1 (0x80151b000)
> > libc++.so.1 => /usr/lib/libc++.so.1 (0x8084a7000)
> > libcxxrt.so.1 => /lib/libcxxrt.so.1 (0x8085ab000)
> > libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x8085cd000)
> > libc.so.7 => /lib/libc.so.7 (0x801083000)
> > libelf.so.2 => /lib/libelf.so.2 (0x801068000)
> >
> > Thanks,
> >
> > B. Smith
> >
> > On Wed, Nov 13, 2024 at 7:40 AM Martin Simmons <mar...@lispworks.com>
> wrote:
> >
> > > What is the output of these commands in the jail?
> > >
> > > pkg info llvm15
> > >
> > > ldd /usr/local/llvm15/lib/libLLVM-15.so
> > >
> > > __Martin
> > >
> > >
> > > >>>>> On Tue, 12 Nov 2024 17:19:34 -0500, B Smith said:
> > > >
> > > > Yes, I should probably upgrade it to 13.4. I'm slow to upgrade my
> jails
> > > > when they are functioning well and they are single-task with no
> > > > outward-facing components (i.e., the Bacula jail is only for running
> > > > Bacula).
> > > >
> > > > I should have mentioned in my original email that the library is
> still
> > > > there. It was installed via pkg.
> > > > Running the same commands you did:
> > > > # locate llvmjit.so
> > > > /usr/local/lib/postgresql/llvmjit.so
> > > > # pkg which /usr/local/lib/postgresql/llvmjit.so
> > > > /usr/local/lib/postgresql/llvmjit.so was installed by package
> > > > postgresql15-server-15.7
> > > >
> > > > Neither llvmjit.so nor libLLVM-15.so show any modification since
> July 9
> > > > (when they were installed).
> > > >
> > > > Thanks.
> > > >
> > > >
> > > > On Tue, Nov 12, 2024 at 5:08 PM Dan Langille <d...@langille.org>
> wrote:
> > > >
> > > > >
> > > > > On Tue, Nov 12, 2024, at 4:13 PM, B. Smith wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I'm running Bacula in a FreeBSD 13.2 jail under TrueNAS Core. This
> > > weekend
> > > > > all my backups proceeded fine, but I noticed that a backup that
> should
> > > have
> > > > > been suspended started up, so I stopped the job. A few other
> > > incremental
> > > > > jobs ran without incident, but then I got an error in a job after
> > > those:
> > > > >
> > > > >
> > > > > side note, time to get that jail update to fix known vulns.
> > > > >
> > > > >
> > > > > Fatal error: backup.c:343 [DE0028] Query failed: DECLARE
> _bac_cursor
> > > > > CURSOR FOR SELECT Path, Filename, FileIndex, JobId, LStat,
> DeltaSeq,
> > > MD5,
> > > > > JobTDate FROM ((SELECT Path.Path, T1.Filename, T1.FileIndex,
> T1.JobId,
> > > > > LStat, DeltaSeq, MD5, JobTDate FROM ( SELECT DISTINCT ON (Filename,
> > > PathId)
> > > > > JobTDate, JobId, FileId, FileIndex, PathId, Filename, LStat, MD5,
> > > DeltaSeq
> > > > > FROM (SELECT FileId, JobId, PathId, Filename, FileIndex, LStat,
> MD5,
> > > > > DeltaSeq FROM File WHERE JobId IN
> > > > >
> > >
> (1561,1633,1642,1750,1759,2001,2010,2019,2028,2037,2046,2055,2064,2073,2082,2091,2100,2109,2118,2127,2136,2145,2182,2191,2200,2209,2218,2227,2236,2245,2254,2263,2272,2281,2340,2471,2517,2755)
> > > > > UNION ALL SELECT File.FileId, File.JobId, PathId, Filename,
> > > File.FileIndex,
> > > > > LStat, MD5, DeltaSeq FROM BaseFiles JOIN File USING (FileId) WHERE
> > > > > BaseFiles.JobId IN
> > > > >
> > >
> (1561,1633,1642,1750,1759,2001,2010,2019,2028,2037,2046,2055,2064,2073,2082,2091,2100,2109,2118,2127,2136,2145,2182,2191,2200,2209,2218,2227,2236,2245,2254,2263,2272,2281,2340,2471,2517,2755)
> > > > > ) AS T JOIN Job USING (JobId) ORDER BY Filename, PathId, JobTDate
> > > DESC )
> > > > > AS T1 JOIN Path ON (Path.PathId = T1.PathId) WHERE FileIndex > 0
> ) )
> > > AS U1
> > > > > ORDER BY JobTDate, FileIndex ASC: ERR=ERROR: could not load
> library
> > > > > "/usr/local/lib/postgresql/llvmjit.so":
> > > > > /usr/local/llvm15/lib/libLLVM-15.so: Undefined symbol
> > > > > "_ZNSt3__122__libcpp_verbose_abortEPKcz"
> > > > >
> > > > > I shutdown Bacula, restarted the jail, and still had the same
> issue. I
> > > did
> > > > > not initiate any updates or changes to the system between the
> working
> > > and
> > > > > non-working jobs, and no real job (other than BackupCatalog) would
> run
> > > > > without the error. Before restoring my database from backup, I used
> > > > > postgres to turn off JIT:
> > > > >
> > > > > alter system set jit=off;
> > > > >
> > > > >
> > > > > JIT is not familiar. Reading
> > > > > https://www.freshports.org/databases/postgresql16-server it is
> > > > >
> > > > > LLVM=on: Build with support for JIT-compiling expressions
> > > > >
> > > > > Looking on my PostgreSQL 16 server, I find:
> > > > >
> > > > > [22:01 pg03 dvl ~] % locate llvmjit.so
> > > > > /usr/local/lib/postgresql/llvmjit.so
> > > > > [22:05 pg03 dvl ~] % pkg which /usr/local/lib/postgresql/llvmjit.so
> > > > > /usr/local/lib/postgresql/llvmjit.so was installed by package
> > > > > postgresql16-server-16.4
> > > > >
> > > > > It sounds like something deleted the file from your system and
> that's
> > > how
> > > > > it became upset.
> > > > >
> > > > > How was PostgreSQL installed there?
> > > > >
> > > > > ALTER SYSTEM
> > > > >
> > > > >
> > > > > Doing this immediately fixed the problem. But I am mystified as to
> how
> > > it
> > > > > suddenly became a problem in the first place. Any suggestions as
> to why
> > > > > this suddenly popped up, and whether JIT is even important for
> Bacula
> > > > > performance?
> > > > >
> > > > >
> > > > > I can't help you there.
> > > > >
> > > > > --
> > > > > Dan Langille
> > > > > d...@langille.org
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
>
_______________________________________________
Bacula-users mailing list
Bacula-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bacula-users
