Re: RFC: Extension Packaging & Lookup

[David E. Wheeler]

On Oct 29, 2024, at 12:23, Paul Ramsey <pram...@cleverelephant.ca> wrote:

> Thanks for this, David,

🤘🏻

> This of course is the area that worries the heck out of me, as someone with 
> extensions that includes not just single system dependencies but long chains 
> of them (depending on GDAL draws in a huge tree).

Yeah. I cited pgsql-http as a simple place to start, on the assumption that 
once we figure out how to properly configure things for one DSO, it the pattern 
should work for any of them in a tree.

> I’m unsure if it will work, but I have wondered if building out the 
> dependencies to install right next to the DSO, and giving the DSO an rpath of 
> “.” would achieve the effect we are looking for.

Given the security issues with library paths, I’d guess that relative paths are 
verboten. But also, Postgres does not `cd` into an extension directory before 
loading it, AFAIK.

> It’s unfortunate (DY)LD_LIBRARY_PATH is dead and dying, but there we are. The 
> trouble I see with somehow coercing the system to load a local copy of system 
> libraries is for (a) common system libs that PostgreSQL itself might be 
> linking (libssl, for example) that then will end up with symbol collisions 
> between the copy loaded by postgres and the copy loaded by the DSO and (b) 
> same thing but for different extensions with the same dependencies. 

Yeah, this is why people tend to depend on system dependencies loaded from 
well-known paths, so libssl will always load the same DSO. I imagine the use of 
LD_LIBRARY_PATH can cause issues today.

> I guess I cannot shake the idea that a lot of interesting extensions are 
> going to have interesting system dependencies, that “exposing an interesting 
> library to postgres” has a high value for an integration system like 
> PostgreSQL. 

Yeah, I think the issue will be to figure out how to manage OS package-provided 
system dependencies in immutable environments like a Docker container. I 
suspect some combination of -rpath compiled into Postgres and mounting 
individual DSO files not included in the base image will be the way to go.

Best,

David