Need help with bundled lib issue for OpenColorIO
I'm working on packaging OpenColorIO for Fedora and ran into an issue with bundled libraries. The project is currently statically compiling in yaml-cpp, tinyxml, and lcms. Upstream doesn't have a problem with unbundling lcms, but is not sure about the other two, here's his explination: hmmm some of the things in ext are not exposed in the public OCIO API; and not being a build expert I'd prefer to not expose additional runtime library dependencies. (tinyxml + yamlcpp, for example). With your implementation, on a fedora build that had one of these libraries installed, would you link to the xml / yaml so(s), or would it use the .a statically at build time? I'd hate to have the dependancies in the core library change depending on build options. What if I pulled in these two libraries into 'core' as source files? (it used to be this way, actually). I'm fine with picking up lcms, etc externally. But i'd like core to be self-contained... --- Are there technical reasons these libraries can not be unbundled? Thanks, Richard -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Need help with bundled lib issue for OpenColorIO
I'm working on packaging OpenColorIO for Fedora and ran into an issue with bundled libraries. The project is currently statically compiling in yaml-cpp, tinyxml, and lcms. Upstream doesn't have a problem with unbundling lcms, but is not sure about the other two, here's his explination: hmmm some of the things in ext are not exposed in the public OCIO API; and not being a build expert I'd prefer to not expose additional runtime library dependencies. (tinyxml + yamlcpp, for example). With your implementation, on a fedora build that had one of these libraries installed, would you link to the xml / yaml so(s), or would it use the .a statically at build time? I'd hate to have the dependancies in the core library change depending on build options. What if I pulled in these two libraries into 'core' as source files? (it used to be this way, actually). I'm fine with picking up lcms, etc externally. But i'd like core to be self-contained... --- Are there technical reasons these libraries can not be unbundled? Only if he's significantly modifying the bundled libs and upstream won't take the changes. If you unbundle and build against the system versions, and it works, that's what you need to do. Always link dynamically if at all possible. -J Thanks, Richard -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel -- in your fear, seek only peace in your fear, seek only love -d. bowie -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Need help with bundled lib issue for OpenColorIO
On Thu, Nov 17, 2011 at 8:55 AM, Jon Ciesla l...@jcomserv.net wrote:
I'm working on packaging OpenColorIO for Fedora and ran into an issue
with bundled libraries.
The project is currently statically compiling in yaml-cpp, tinyxml,
and lcms. Upstream doesn't have a problem with unbundling lcms, but is
not sure about the other two, here's his explination:
hmmm some of the things in ext are not exposed in the public OCIO
API; and not being a build expert I'd prefer to not expose additional
runtime library dependencies. (tinyxml + yamlcpp, for example).
With your implementation, on a fedora build that had one of these
libraries installed, would you link to the xml / yaml so(s), or would
it use the .a statically at build time? I'd hate to have the
dependancies in the core library change depending on build options.
What if I pulled in these two libraries into 'core' as source files?
(it used to be this way, actually). I'm fine with picking up lcms, etc
externally. But i'd like core to be self-contained...
---
Are there technical reasons these libraries can not be unbundled?
Only if he's significantly modifying the bundled libs and upstream won't
take the changes. If you unbundle and build against the system versions,
and it works, that's what you need to do. Always link dynamically if at
all possible.
Ok, so the public/private API's won't be a problem?
He did mention they were patched, some of it for build reasons (no
problem) but some of it was to make it work. He's going to check to
see if those patches have made their way into upstream.
The last problem is a strange one. They have a library, PyOpenColorIO
that provides both C++ symbols, but also python bindings. I assume
they need to go in /usr/lib{,64} but should they also get symlinked to
/usr/lib{,64}/pythonX.X/site-packages?
Thanks,
Richard
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Need help with bundled lib issue for OpenColorIO
On Thu, Nov 17, 2011 at 09:11:06AM -0600, Richard Shaw wrote:
On Thu, Nov 17, 2011 at 8:55 AM, Jon Ciesla l...@jcomserv.net wrote:
I'm working on packaging OpenColorIO for Fedora and ran into an issue
with bundled libraries.
The project is currently statically compiling in yaml-cpp, tinyxml,
and lcms. Upstream doesn't have a problem with unbundling lcms, but is
not sure about the other two, here's his explination:
hmmm some of the things in ext are not exposed in the public OCIO
API; and not being a build expert I'd prefer to not expose additional
runtime library dependencies. (tinyxml + yamlcpp, for example).
With your implementation, on a fedora build that had one of these
libraries installed, would you link to the xml / yaml so(s), or would
it use the .a statically at build time? I'd hate to have the
dependancies in the core library change depending on build options.
What if I pulled in these two libraries into 'core' as source files?
(it used to be this way, actually). I'm fine with picking up lcms, etc
externally. But i'd like core to be self-contained...
---
Are there technical reasons these libraries can not be unbundled?
Only if he's significantly modifying the bundled libs and upstream won't
take the changes. If you unbundle and build against the system versions,
and it works, that's what you need to do. Always link dynamically if at
all possible.
Ok, so the public/private API's won't be a problem?
I'm not quite sure what he means by public/private API. If he's saying that
the end user doesn't see the xml and yaml libraries's APIs that seems like
it's tangential to the issue.
If he's talking about shielding the user from having to download and install
those libraries separately, the strategy we like upstreams to follow is to
keep the bundled code when they ship but at build time choose whether to use
the bundled library or the system library.
Since we have rpm packages that keep dependency information and yum which
does depsolving, the end user would not have to worry about the additional
dependencies on libraries when installing our package -- yum install
OpenColorIO would pull the proper dependencies automatically. When an
OpenColorIO user downloads the package and builds from source, they can use
the bundled versions.
For his question about .so vs .a and pulling the source files in -- the
problems that we're trying to avoid are the bundled libraries having issues
(especially, but not limited to security issues) that need patching. If
there's a single system copy in a dynamic library, there's one package that
we have to update to fix this. If there's static libraries involved, then
we need to rebuild the library package and then find the packages that are
linked statically and rebuild them. If there's libraries that are bundled,
then we have to recompile the system library, we have to find what software
has bundled the libraries (hopefully we've already identified them as in
this case), if there's patches nad changes to the bundled library we have to
merge those with the new version from upstream that fixes the issues, and
then we have to rebuild and reship those libraries.
So for us, the ideal is to link against the .so's and let the package
manager do the work of pulling the right packages for the end user. If
upstream wishes to ship the libraries as a supplement for those who are not
getting them from upstream, that's fine but making it work for both cases
makes for the best experience for the users.
He did mention they were patched, some of it for build reasons (no
problem) but some of it was to make it work. He's going to check to
see if those patches have made their way into upstream.
If upstream hasn't taken them yet, it's possible that our packages could
carry the changes. It depends on the maintainer of the library package,
whether the upstream is going to merge the changes eventually, and whether
they cause incompatibilities. Something to keep in mind.
The last problem is a strange one. They have a library, PyOpenColorIO
that provides both C++ symbols, but also python bindings. I assume
they need to go in /usr/lib{,64} but should they also get symlinked to
/usr/lib{,64}/pythonX.X/site-packages?
Yeah, that is an interesting one. Typically, libraries for C and elf shared
objects that are python extensions are kept in separate files. From your
description, a symlink would be appropriate here.
-Toshio
pgpULoqWBr1Nw.pgp
Description: PGP signature
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Re: Need help with bundled lib issue for OpenColorIO
On Thu, Nov 17, 2011 at 10:31 AM, Toshio Kuratomi a.bad...@gmail.com wrote: [SNIP] Only if he's significantly modifying the bundled libs and upstream won't take the changes. If you unbundle and build against the system versions, and it works, that's what you need to do. Always link dynamically if at all possible. Ok, so the public/private API's won't be a problem? I'm not quite sure what he means by public/private API. If he's saying that the end user doesn't see the xml and yaml libraries's APIs that seems like it's tangential to the issue. I've had some further communication with upstream and they have agreed that the libraries in question need to be unbundled, but may in the meantime seek an exception. The reason for this is that the two libraries are mainly used for serialization and they are concerned that it may not work correctly/dependably across the different versions of the library in different distro's. This is a summary, see this link[1] for his full explanation. I think part of their immediate hesitation is that this is apparently an industrial grade app that has been used by Sony for use in movies (such as Cloudy With A chance of Meatballs) and they want to make sure it works the same on all platforms (the one big positive with bundled libs). He did mention they were patched, some of it for build reasons (no problem) but some of it was to make it work. He's going to check to see if those patches have made their way into upstream. If upstream hasn't taken them yet, it's possible that our packages could carry the changes. It depends on the maintainer of the library package, whether the upstream is going to merge the changes eventually, and whether they cause incompatibilities. Something to keep in mind. They are looking into whether their patches have made it upstream or not, so that's TBD. Thanks, Richard [1] https://github.com/imageworks/OpenColorIO/pull/183#issuecomment-2777004 -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel
