Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2026-01-09 Thread Paavo Helde via Tiff


Admitting I have not read through all the other replies, just wanting to 
make come comments.


In general it would be ideal to have a C++ library with C API interface. 
However, converting the codebase to C++ would probably mean several years 
of instabilities.


Side remark; with implementation in C, I know I would never want to 
contribute to it, with C++ I might.


Having a direct C++ API for the actual implementation is not feasible as 
there is no common C++ ABI. So C API it is. But it could be made better! 
How come I have to know in advance what type of variable certain tag value 
must be read into, if the type of the value is encoded in the TIFF file 
itself in the first place? For starters, no typeless ... functions please!


For C++ one can in principle also provide a header-only C++ API on top of 
the C API.


As for building it, CMake seems to be the best at the moment for 
cross-platform development. One moment, checking... yes, libtiff is built 
with CMake, so everything good!


Just my 2 euro cents
Paavo


On Mon, 29 Dec 2025, Roger Leigh via Tiff wrote:


Date: Mon, 29 Dec 2025 11:08:45 +
From: Roger Leigh via Tiff 
Reply-To: Roger Leigh 
To: Roger Leigh via Tiff 
Subject: [Tiff] Proposal for conversion of the codebase to C++ while retaining
 the existing C API

Hi folks,

I mentioned this as a possibility during discussion in mid December, but I’ve 
been spending some time thinking about this over the last couple of weeks, and 
have taken the time to explore some of the details and write them up.  There’s 
far too much for an email, so I’ve written up a detailed proposal (with some 
help from Claude Opus 4.5 for exploring the codebases in detail) and put it 
here: https://gitlab.com/libtiff/libtiff/-/issues/771

As a very brief bit of context and summary.  I’ve long been unsatisfied with 
the base C API of libtiff.  It’s unsafe, doesn’t support multi-threading well, 
is hard to use correctly, is stateful where it doesn’t need to be, and has 
inconsistent and hard to use error handling.  Some alternatives for comparison:

OME Files TIFF wrapper (by me):
https://gitlab.com/codelibre/ome/ome-files-cpp/-/tree/master/lib/ome/files/tiff?ref_type=heads
A set of C++ wrapper classes over libtiff’s C API.  Adds compile-time type-safe 
field and pixel data access, supports multiple open directories (with 
limitations due to the libtiff API).

Libertiff (by Even)
https://github.com/libertiff/libertiff/tree/master
Header-only, more modern and cleaner than OME Files TIFF wrappers, but exposes 
less functionality overall.  But the interface is very clean, and very well 
done.

Bio-Formats TIFF (by Melissa Linkert)
https://github.com/ome/bioformats/tree/develop/components/formats-bsd/src/loci/formats/tiff
Java implementation with a lot of features and a lot of performance 
optimisations.

The proposal includes elements from all three of these implementations, and is 
really asking the question: what would libtiff look like if we implemented it wholly 
in C++, but also used extern "C” linkage to completely retain the C API and ABI 
as they are today.  I think it could result in a dramatic improvement in the quality 
of implementation and its usability and performance, and open up libtiff to uses it 
currently can’t meet.  But it’s clearly a large change to make.  However, from the 
above examples, it’s something we’ve actually already done several times over 
independently, so it’s not in any way new, it would largely be a repeat of what 
we’ve already done but this time working within libtiff rather than trying to work 
around its limited C API.  The above three implementations are in production use, 
some for well over a decade, so are not untested or unsafe—all of the design 
proposals have been well proven.

Anyway, I’d be grateful for any feedback after you’ve looked over the linked 
proposal.  It’s written as a plan but it’s really just a series of proposed 
ideas.  This is trying to look at the future of libtiff, and how we can take it 
forward while also keeping it fully usable and stable for its existing user 
base.


Kind regards,
Roger
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-31 Thread Roger Leigh via Tiff

> On 30 Dec 2025, at 20:12, Olivier Paquet  wrote:
> Le mar. 30 déc. 2025, à 09 h 50, Roger Leigh via Tiff  > a écrit :
>> On 29 Dec 2025, at 16:43, Olivier Paquet > > wrote:
>  
>> * Type-safe C++ field and pixel data access are header-only inline 
>> templates, which ultimately call into the existing unsafe code.  It’s all 
>> layered on top.
>  
> This is something we should have had ages ago but it is almost entirely 
> unrelated to the implementation language. The only link is that we'll likely 
> need some new safer (non vararg) C APIs and it might be easier to do those 
> with cleaner code inside.
>  
>> On top of this, there are some additional considerations.  I’ve mentioned 
>> them previously on the list I think.  You might have seen the reporting over 
>> the last couple of years about CISA and the phasing-out of unsafe code from 
>> January 2026 onwards:
>> 
>> https://www.cisa.gov/resources-tools/resources/product-security-bad-practices
>> https://www.cisa.gov/sites/default/files/2023-12/The-Case-for-Memory-Safe-Roadmaps-508c.pdf
>> https://www.cisa.gov/news-events/alerts/2025/06/24/new-guidance-released-reducing-memory-related-vulnerabilities
>  
> Not familiar with these but does C++ ever count as safe? Granted, it's 
> somewhat easier to write good code but it still has an awful lot of ways to 
> shoot yourself in the foot.

Absolutely, but it’s all relative, and if used well, we can use C++ to have a 
smaller, simpler, more robust codebase with user-facing interfaces which are 
much harder to use incorrectly.  It’s never going to achieve complete safety, 
but it can get close.

We can also get closer with C.  Not as close as with C++, but we can do a lot 
better than today if we wish to.  The primary concern is API and ABI breakage.  
Avoiding breakage on the C side was intentionally not part of the original 
proposal, because I wanted to minimise friction and inconvenience to existing 
users at all costs. If we have the scope to break things, then there is a lot 
of room for significant improvements on the C side as well.  That would also 
make it much easier to layer a good C++ API on top (or vice versa) as well.  

>> The point here isn’t just about the casts.  It’s that the codebase quality 
>> overall is not at all great, and C the language as well as legacy coding 
>> practices and compatibility concerns are a large part of the problem here.  
>> The vast majority of the problems causing the CVEs would not occur in other 
>> languages, even C++ with its much stricter type conversion rules.  While we 
>> could no doubt do a lot of work to do stricter static analysis etc, there 
>> are diminishing returns and so it’s worth asking the question of is it worth 
>> it, and where the breaking point is where it is not.
> 
> No need to sugar coat it, the code was written before security for image 
> libraries was a thing. It shows. The code of the tools is even worse and 
> seems to be causing many of the bug reports. The thing is, many of its users 
> don't really have security concerns.
>  
>> It’s also that the wider world is starting to make hard demands upon 
>> software developers regarding software development practices.  It’s been 
>> coming in earnest since 2023, and 2026 is when the mandates start to begin 
>> for real.  Whether we like it or not, unsafe code elimination is now being 
>> mandated by national bodies in the USA as well as being copied by other 
>> countries around the world, and this will preclude a lot of legacy C 
>> libraries.  And C++, it might be a lot safer than C but it’s still 
>> imperfect.  If you want to write an application using libtiff, you now need 
>> to justify why you’re using a C library and not a safer alternative.  And 
>> the strictness will increase over time.  At some point you won’t be able to 
>> justify using an unsafe C library at all.  So for me the question is not so 
>> much “do we do this”, as “where do we do this”.  Do we do it as part of 
>> libtiff, and bring the C API along to help service the existing users.  Or 
>> do we create a separate branch or separate repository in the libtiff group 
>> on GitLab and do it there?  Or do we split entirely and do it completely 
>> independently?  Whatever we choose to do, the rest of the world will move on 
>> with or without us, and this is the one of the most important points behind 
>> this work.  Planning to be a part of the future by doing the work to remain 
>> relevant in it.
>  
> All very good questions. Especially as the current C API has some 
> fundamentally unsafe parts which may remain a risk as long as we carry them. 
> A case could be made that it should be deprecated or maintained as a separate 
> "unsafe" project and projects which care about security should only use newer 
> APIs.
> 
>> So the core thing I’d want out of the discussion is an agreed plan.  One 
>> possibility would be as proposed, refactor the 

Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Olivier Paquet via Tiff
Le mar. 30 déc. 2025, à 13 h 28, Even Rouault via Tiff 
a écrit :

> I'm a bit unusure about the thread-safey aspects of your proposal, in
> particular about the need for a mutex per TIFF handle. I would argue
> that's the job of the caller to protect that. One option could be to
> have a opt-in mutex (a bit like SQLite3's  SQLITE_OPEN_NOMUTEX vs
> SQLITE_OPEN_FULLMUTEX : https://sqlite.org/c3ref/open.html).  Having
> some higher level APIs that for example extract from multi-strip/tiles
> at once and may use multithreading internally to decode several
> tiles/strips in parallel would have strong value though (something I do
> on the GDAL side of the TIFF reader typically).
>

Good thread safety is not about adding mutexes, it's about separating
resources so multiple threads can work without the need for locking.

What we need the most is easy access to multithread tile/strip
encoding/decoding. Meaning we need APIs to be able to create multiple codec
instances for a single TIFF file. I've done this manually with
TIFFReadRawTile()/TIFFWriteRawTile() in the past but it's a pain as each
supported codec needs to be reimplemented outside libtiff. It does not
fundamentally require locking inside libtiff however, just some new APIs.

A separate nice thing to have is actual multithread I/O of tiles/strips
(pread()/pwrite() or similar). Reading especially. Writing is less valuable
in my experience, and more complex. I think reading can be done without a
mutex with some care, but one may be desirable to ensure safe use. Writing
certainly requires one.

Layering a simple "read/write this file using multiple threads" API on top
of this is also desirable as it is what many users will want out of libtiff.

Wrapping a TIFF handle around a big mutex to make everything safe seems of
low value to me. That can easily be done by user code if that's how they
want to deal with multiple threads.

Olivier
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Olivier Paquet via Tiff
Le mar. 30 déc. 2025, à 09 h 50, Roger Leigh via Tiff 
a écrit :

> On 29 Dec 2025, at 16:43, Olivier Paquet  wrote:
> > I find the proposal's categorization of "let's build this as C++" as low
> risk bordering on ridiculous. It will break stuff on some platforms, that's
> for certain. The rest of it looks more or less like "let's rewrite
> everything".
>
> It very much was not intended to read this way.  I intend the approach to
> be a measured, incremental series of refactors with no “rewriting the
> world” at any stage.
>

Not at any stage the but the end goal of modern C++ internals means a
mostly complete rewrite. Splitting it out in multiple stages does not
change that. It will be almost entirely new code, likely with entirely new
bugs.

We would need to plan each stage out properly, but the initial stages would
> be something along the lines of:
>
> * Make the existing C codebase compile cleanly with a C++ compiler.  This
> is already done.  For example see
> https://gitlab.com/libtiff/libtiff/-/jobs/12552031204.  Done For GCC and
> Clang; MSVC is already C++.  No problems, all tools and tests compile, all
> tests pass.
> * Rename the C sources to C++.  No code changes.  No ABI changes.
> * Change TIFF structure to class; migrate free functions to class
> methods.  Actual method contents are unchanged.  Retain free functions and
> call class methods for C compatibility.
> * Repeat for other types (Directory, Codec etc.)
>

Those are reasonable and low risk changes but not what I understood from
your issue. There is still the matter of adding C++ as a requirement but
that is not a risk of causing bugs, rather a risk of leaving a lot of old
platforms behind.


> * Type-safe C++ field and pixel data access are header-only inline
> templates, which ultimately call into the existing unsafe code.  It’s all
> layered on top.
>

This is something we should have had ages ago but it is almost entirely
unrelated to the implementation language. The only link is that we'll
likely need some new safer (non vararg) C APIs and it might be easier to do
those with cleaner code inside.

There’s obviously more detail than this in practice, but at no point am I
> proposing a blanket rewrite of *anything*.  It’s a sequence of refactors
> and additions, with the C API and ABI being kept functional and stable at
> every step.
>

If 90% of the code gets refactored into a different language with idioms
from a different century, I call it a rewrite :-) It's just not from
scratch. But there will be a lot more new code than old code from what I
understand.


> On top of this, there are some additional considerations.  I’ve mentioned
> them previously on the list I think.  You might have seen the reporting
> over the last couple of years about CISA and the phasing-out of unsafe code
> from January 2026 onwards:
>
>
> https://www.cisa.gov/resources-tools/resources/product-security-bad-practices
>
> https://www.cisa.gov/sites/default/files/2023-12/The-Case-for-Memory-Safe-Roadmaps-508c.pdf
>
> https://www.cisa.gov/news-events/alerts/2025/06/24/new-guidance-released-reducing-memory-related-vulnerabilities
>

Not familiar with these but does C++ ever count as safe? Granted, it's
somewhat easier to write good code but it still has an awful lot of ways to
shoot yourself in the foot.


> The point here isn’t just about the casts.  It’s that the codebase quality
> overall is not at all great, and C the language as well as legacy coding
> practices and compatibility concerns are a large part of the problem here.
> The vast majority of the problems causing the CVEs would not occur in other
> languages, even C++ with its much stricter type conversion rules.  While we
> could no doubt do a lot of work to do stricter static analysis etc, there
> are diminishing returns and so it’s worth asking the question of is it
> worth it, and where the breaking point is where it is not.
>

No need to sugar coat it, the code was written before security for image
libraries was a thing. It shows. The code of the tools is even worse and
seems to be causing many of the bug reports. The thing is, many of its
users don't really have security concerns.


> It’s also that the wider world is starting to make hard demands upon
> software developers regarding software development practices.  It’s been
> coming in earnest since 2023, and 2026 is when the mandates start to begin
> for real.  Whether we like it or not, unsafe code elimination is now being
> mandated by national bodies in the USA as well as being copied by other
> countries around the world, and this will preclude a lot of legacy C
> libraries.  And C++, it might be a lot safer than C but it’s still
> imperfect.  If you want to write an application using libtiff, you now need
> to justify why you’re using a C library and not a safer alternative.  And
> the strictness will increase over time.  At some point you won’t be able to
> justify using an unsafe C library at all.  So for me the question is not 

Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Greg Troxel via Tiff
Even Rouault via Tiff  writes:

> I'm also on the board of only "supporting" (or aiming at...) "modern"
> environments, that is things  like Linux, Windows, Mac, Android,
> WebAssembly, but maybe not 8/16-bit micro-controllers,
> obsolete/esoteric OS (old UNIXes).

My comments aren't really contradicting you, but I'd add

  *BSD, illumos, and any other maintained system with a reasonably
   modern compiler and that mostly conforms to POSIX

As usual I'd like to see the compiler requirement be very far back, as
with systems that integrate the compiler into base, things get awkward.
Today, supporting gcc 10 + would be ok, and 7 + would be great.  Needing
12 seems unreasonable.
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Even Rouault via Tiff

Roger,

Thanks for all the thoughts you've put into devising this plan. 
Personally, I wouldn't have issues with my usage of libtiff if it used 
C++ for its internals, while retaining existing C API, and I'd be 
curious to hear more details about others for which a dependency to the 
C++ standard library would be a problem what the actual troubles would 
be. There would be a decision to take if we don't want anymore to 
"support"  odd/marginal environments. I'm also on the board of only 
"supporting" (or aiming at...) "modern" environments, that is things  
like Linux, Windows, Mac, Android, WebAssembly, but maybe not 8/16-bit 
micro-controllers, obsolete/esoteric OS (old UNIXes).


That said, I've some concerns about the benefit/(effort x risks) of your 
proposal. Regarding libtiff "security", I see 2 main issues:


- TIFFGetField(). There's no way we can sanitize that one, be libtiff 
being written in C++ or Rust. The interface of the function itself is 
broken by design (va_list of varying number of parameters and types). 
Top priority would be to introduce in the existing code base functions 
like TIFFGetFieldAsUInt32(), TIFFGetFieldAsUInt32Array(), 
TIFFGetFieldAsFloat64(), etc., or something similar that are type safe 
in their API (with the functions doing reasonable type conversions when 
possible). And push hard to deprecate existing usage of TIFFGetField().


- I believe there are a number of issues in the API contract of a number 
of public functions, especially when read/write operations are mixed 
(things like updating existing strips/tiles seem to have been an 
after-though addition, that seem to mostly work now, but not without 
hacks inside). I've fought for years in my use of libtiff in GDAL about 
that. Those issues might be a mix of the core being written in a 
memory-unsafe language, and also logic flaws/unspecified behavior in the 
library regarding which functions can be called in which order.


I'm a bit unusure about the thread-safey aspects of your proposal, in 
particular about the need for a mutex per TIFF handle. I would argue 
that's the job of the caller to protect that. One option could be to 
have a opt-in mutex (a bit like SQLite3's  SQLITE_OPEN_NOMUTEX vs 
SQLITE_OPEN_FULLMUTEX : https://sqlite.org/c3ref/open.html).  Having 
some higher level APIs that for example extract from multi-strip/tiles 
at once and may use multithreading internally to decode several 
tiles/strips in parallel would have strong value though (something I do 
on the GDAL side of the TIFF reader typically).


If we really want to embrace the memory-safe bandwagon, we'd have likely 
no other choice than doing a rewrite in Rust, and providing a C API that 
is *mostly* compatible with the existing one, but possibly with gaps. 
I'm undecided myself if that's something doable given the state of the 
libtiff ecosystem and our own capabilities and availabilities (I'm 
myself a total Rust noob, although I see that would be definitely 
something worth embracing). Regarding Rust, it is not available for all 
architectures, but it is available for all architectures that Debian 
officially support, so I wouldn't be that concerned about the ones missing.


Even

Le 29/12/2025 à 12:08, Roger Leigh via Tiff a écrit :

Hi folks,

I mentioned this as a possibility during discussion in mid December, but I’ve 
been spending some time thinking about this over the last couple of weeks, and 
have taken the time to explore some of the details and write them up.  There’s 
far too much for an email, so I’ve written up a detailed proposal (with some 
help from Claude Opus 4.5 for exploring the codebases in detail) and put it 
here:https://gitlab.com/libtiff/libtiff/-/issues/771

As a very brief bit of context and summary.  I’ve long been unsatisfied with 
the base C API of libtiff.  It’s unsafe, doesn’t support multi-threading well, 
is hard to use correctly, is stateful where it doesn’t need to be, and has 
inconsistent and hard to use error handling.  Some alternatives for comparison:

OME Files TIFF wrapper (by me):
https://gitlab.com/codelibre/ome/ome-files-cpp/-/tree/master/lib/ome/files/tiff?ref_type=heads
A set of C++ wrapper classes over libtiff’s C API.  Adds compile-time type-safe 
field and pixel data access, supports multiple open directories (with 
limitations due to the libtiff API).

Libertiff (by Even)
https://github.com/libertiff/libertiff/tree/master
Header-only, more modern and cleaner than OME Files TIFF wrappers, but exposes 
less functionality overall.  But the interface is very clean, and very well 
done.

Bio-Formats TIFF (by Melissa Linkert)
https://github.com/ome/bioformats/tree/develop/components/formats-bsd/src/loci/formats/tiff
Java implementation with a lot of features and a lot of performance 
optimisations.

The proposal includes elements from all three of these implementations, and is 
really asking the question: what would libtiff look like if we implemented it wholly 
in C++, but also used 

Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Kemp Watson via Tiff
There are many TIFF code implementations. Perhaps a start would be to
define exactly what "libtiff" is.
My view has always been: it is the idiomatic _C_ TIFF library.

Kemp Watson


On Tue, 30 Dec 2025 at 12:25, Bob Friesenhahn via Tiff 
wrote:

> On 12/30/25 08:50, Roger Leigh via Tiff wrote:
>
> > On top of this, there are some additional considerations.  I’ve
> mentioned them previously on the list I think.  You might have seen the
> reporting over the last couple of years about CISA and the phasing-out of
> unsafe code from January 2026 onwards:
> >
> >
> https://www.cisa.gov/resources-tools/resources/product-security-bad-practices
>
> It is worth noting that the CISA documents classify C and C++ the same
> due to being "memory-unsafe" languages.  Adding C++ to the existing C
> will not absolve libtiff from CISA/EU mandates and the "memory-unsafe"
> stigma.  The underlying operating system (e.g. Linux, *BSD, Solaris,
> etc.) is usually written in C.
>
> The languages Ada, C#, Delphi/Object Pascal, Go, Java, JavaScript (when
> used idiomatically), Kotlin, Python, Ruby, Rust, and Swift are on the
> lists of memory-safe languages.
>
> The CVEs for "libtiff" may be found at
> https://www.cvedetails.com/product/3881/Libtiff-Libtiff.html?vendor_id=2224.
>
> Unfortunately, the descriptions in the long list of CVEs are not always
> clear as to if they are in the library, or the utilities, but the vast
> majority appear to be in the utilities.  This is clearly a flaw in the
> CVE system which should have allocated separate CVE notices for libtiff,
> and for the libtiff tools.
>
> Regardless, there is an excessive concern about use of "memory-unsafe"
> languages when it comes to computer security. A great number of computer
> security issues have nothing to do with memory safety.
>
> As regards to libtiff, the major concern is the TIFF format itself which
> offers a wide variety of options (via tags, and the interpretation of
> one or more TAGs at once), and it is not always clear how to do things
> properly.  C++ will not solve this problem unless it is user-facing and
> assures correctness based on rigid design rather than interpretation.
>
> Using Rust as a base would incur much more significant issues than using
> C++ as the base because the Rust run-times and toolchain are cumbersome
> and only available for very limited targets.
>
> Bob
>
>
> ___
> Tiff mailing list
> [email protected]
> https://lists.osgeo.org/mailman/listinfo/tiff
>
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Bob Friesenhahn via Tiff

On 12/30/25 08:50, Roger Leigh via Tiff wrote:


On top of this, there are some additional considerations.  I’ve mentioned them 
previously on the list I think.  You might have seen the reporting over the 
last couple of years about CISA and the phasing-out of unsafe code from January 
2026 onwards:

https://www.cisa.gov/resources-tools/resources/product-security-bad-practices


It is worth noting that the CISA documents classify C and C++ the same 
due to being "memory-unsafe" languages.  Adding C++ to the existing C 
will not absolve libtiff from CISA/EU mandates and the "memory-unsafe" 
stigma.  The underlying operating system (e.g. Linux, *BSD, Solaris, 
etc.) is usually written in C.


The languages Ada, C#, Delphi/Object Pascal, Go, Java, JavaScript (when 
used idiomatically), Kotlin, Python, Ruby, Rust, and Swift are on the 
lists of memory-safe languages.


The CVEs for "libtiff" may be found at 
https://www.cvedetails.com/product/3881/Libtiff-Libtiff.html?vendor_id=2224. 
Unfortunately, the descriptions in the long list of CVEs are not always 
clear as to if they are in the library, or the utilities, but the vast 
majority appear to be in the utilities.  This is clearly a flaw in the 
CVE system which should have allocated separate CVE notices for libtiff, 
and for the libtiff tools.


Regardless, there is an excessive concern about use of "memory-unsafe" 
languages when it comes to computer security. A great number of computer 
security issues have nothing to do with memory safety.


As regards to libtiff, the major concern is the TIFF format itself which 
offers a wide variety of options (via tags, and the interpretation of 
one or more TAGs at once), and it is not always clear how to do things 
properly.  C++ will not solve this problem unless it is user-facing and 
assures correctness based on rigid design rather than interpretation.


Using Rust as a base would incur much more significant issues than using 
C++ as the base because the Rust run-times and toolchain are cumbersome 
and only available for very limited targets.


Bob


___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Lee Howard via Tiff

On 12/30/25 3:26 AM, Roger Leigh via Tiff wrote:


Yet we suffer from a regular stream of CVEs due to their multiple quality 
issues.



How, exactly, is anybody actually suffering from any of the supposed 
CVEs that get pinned on the tools?  The minute somebody points out 
actual suffering is the minute I will make sure to address issues 
promptly.  I didn't write the tools, but the tools clearly weren't 
written with a perspective that the user would reasonably be feeding 
them randomized input both on the command-line as well as with the 
data.  Furthermore, a good portion of the supposed CVEs that have been 
reported against the tools were actually issues in the library and not 
the tools.  The tool was just the vector by which the researcher 
identified the issue in the library.


If by "suffer from" you mean "lament".  Okay, sure.  I get that. We'd 
all like this stuff to have been future-proof from the beginning.  I've 
no argument there.




Do we really need to generate PostScript, over two decades after the whole 
world moved over to PDF?



Yes.



And also generate PDF at the same time?



Yes.



Does that really bring much value?



Yes.



Why are we doing either when there are better applications out there for 
generating PDFs?



I'm not sure what you intend to mean by "better".  There may be other 
applications.  And there may be yet other applications that could be 
developed.  But when a specific tool becomes relied upon then the 
processes and applications that rely upon the tool have a dependency 
upon the tool.  It's not necessarily "better" to just replace the tool 
with something else merely because someone out there identified a 
supposed flaw in the tool if it were used in a way that the application 
never uses it.




But for people like me (and likely yourself) whose primary use of libtiff is to 
link it into our applications, the tools are a complete distraction and an 
unproductive time sink.



It's okay that you don't like the tools.  You don't have to like them.  
Nobody is asking you to like them.  But to repeatedly claim in this 
forum that they're of absolutely no value merely because you don't see 
their value is frustrating when those of us who do see value in them 
have made our case and actually invested time and effort to remedy CVEs, 
etc.


I don't want to wade into the debate about C vs C++ (or Rust or whatever 
our favorite programming language is).  But please leave your opinions 
about the tools out of this debate.


Lee.
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Roger Leigh via Tiff
On 29 Dec 2025, at 16:43, Olivier Paquet  wrote:
> I find the proposal's categorization of "let's build this as C++" as low risk 
> bordering on ridiculous. It will break stuff on some platforms, that's for 
> certain. The rest of it looks more or less like "let's rewrite everything".

It very much was not intended to read this way.  I intend the approach to be a 
measured, incremental series of refactors with no “rewriting the world” at any 
stage.

We would need to plan each stage out properly, but the initial stages would be 
something along the lines of:

* Make the existing C codebase compile cleanly with a C++ compiler.  This is 
already done.  For example see 
https://gitlab.com/libtiff/libtiff/-/jobs/12552031204.  Done For GCC and Clang; 
MSVC is already C++.  No problems, all tools and tests compile, all tests pass.
* Rename the C sources to C++.  No code changes.  No ABI changes.
* Change TIFF structure to class; migrate free functions to class methods.  
Actual method contents are unchanged.  Retain free functions and call class 
methods for C compatibility.
* Repeat for other types (Directory, Codec etc.)
* Type-safe C++ field and pixel data access are header-only inline templates, 
which ultimately call into the existing unsafe code.  It’s all layered on top.

There’s obviously more detail than this in practice, but at no point am I 
proposing a blanket rewrite of *anything*.  It’s a sequence of refactors and 
additions, with the C API and ABI being kept functional and stable at every 
step.

> Now, I imagine that some people will argue that losing support for such 
> "legacy" code is just too bad. So be it, but be aware that this means 
> the old version of the lib will likely live on as a source of security 
> bugs for years to come.
> 
> That's the discussion we need to have before any of the more technical 
> aspects of this. There's no way we can move core libtiff to modern C++ as 
> proposed and retain the wide support libtiff provides. I think even older C++ 
> would be a stretch.
> 
> There also needs to be a discussion about the value of doing this. What does 
> it bring that we can't have without it? It will also bring new bugs. Do the 
> improvements outweigh the problems? Is this true at every point along the way 
> or only 5-10 years from now?

I’ve summarised some of this in my other replies, regarding things it brings 
that we can’t have currently.

On top of this, there are some additional considerations.  I’ve mentioned them 
previously on the list I think.  You might have seen the reporting over the 
last couple of years about CISA and the phasing-out of unsafe code from January 
2026 onwards:

https://www.cisa.gov/resources-tools/resources/product-security-bad-practices
https://www.cisa.gov/sites/default/files/2023-12/The-Case-for-Memory-Safe-Roadmaps-508c.pdf
https://www.cisa.gov/news-events/alerts/2025/06/24/new-guidance-released-reducing-memory-related-vulnerabilities

Nearly everything I’ve done in this project over the last decade has all been 
done with an aim of making libtiff safe, robust and portable.  We’ve introduced 
number of things, from the CMake build system, to GitLab CI, which included 
coverity static analysis and greatly improved our effective portability, 
particularly to Windows which hadn’t previously had regular testing.  It has 
all helped, up to a point.  Libtiff is a higher-quality codebase as a result.  
But it’s not enough.  We have a seemingly endless stream of CVEs and bugs.  And 
it all stems from this being a legacy C codebase which hasn’t had any really 
serious attempt at fixing the fundamentals.  Just look at the changes made here:

https://gitlab.com/libtiff/libtiff/-/compare/warning-additions5..61b5551e0c26685000435885c1ba6cf23b92d416?from_project_id=4721645

And this is merely to add in some stricter compiler warnings!  I haven’t even 
gotten started on actual static analysis and finding more serious problems yet. 
 The C++ conversion would aim to remove every single cast.  It’s detailed in 
the plan.

The point here isn’t just about the casts.  It’s that the codebase quality 
overall is not at all great, and C the language as well as legacy coding 
practices and compatibility concerns are a large part of the problem here.  The 
vast majority of the problems causing the CVEs would not occur in other 
languages, even C++ with its much stricter type conversion rules.  While we 
could no doubt do a lot of work to do stricter static analysis etc, there are 
diminishing returns and so it’s worth asking the question of is it worth it, 
and where the breaking point is where it is not.

It’s also that the wider world is starting to make hard demands upon software 
developers regarding software development practices.  It’s been coming in 
earnest since 2023, and 2026 is when the mandates start to begin for real.  
Whether we like it or not, unsafe code elimination is now being mandated by 
national bodies in the USA as well as being copied by 

Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Roger Leigh via Tiff
On 29 Dec 2025, at 16:05, Robin Watts  wrote:
> 
> On 29/12/2025 11:08, Roger Leigh via Tiff wrote:
>> I mentioned this as a possibility during discussion
> > in mid December,
> 
> Please be aware that moving libtiff to C++ instead of C would cause problems 
> for various pieces of software that depend upon it.
> 
> The most obvious case, from my point of view, is Ghostscript.
> 
> We are careful to avoid any compulsory dependencies that depend upon
> C++, as we have to run on lots of systems where C++ presents a problem.

Please could you provide a list of the platforms and compiler versions which 
would be affected for context.

>> As a very brief bit of context and summary.  I’ve long been
> > unsatisfied with the base C API of libtiff.  It’s unsafe,
> > doesn’t support multi-threading well, is hard to use correctly,
> > is stateful where it doesn’t need to be, and has inconsistent
> > and hard to use error handling.
> 
> All of which case be solved without resorting to C++.

This isn’t the case.  Both libertiff and the ome-files tiff wrappers were 
written to solve problems which could not otherwise be solved in plain C.  They 
both add a lot of additional safety checking at both compile-time and run-time. 
 They weren’t written just for the sake of it, I personally spent several years 
working on this stuff.  This is a summary of the parts in the proposal which 
couldn’t be done in C:

Type Safety

Field Access:
- Type-safe field access at compile-time (get() → uint32_t)
- Type-safe field access at runtime (variant-based FieldValue)
- Compile-time tag-to-type mapping via FieldTraits
- Type-safe enum classes for Compression, Photometric, etc.
- Runtime tag registration with type traits
- User-defined FieldTraits<> specializations

Pixel Data:
- Type-safe pixel spans (std::span, std::span)
- Automatic type-safe unpacking of packed formats (1-bit, 12-bit → byte-aligned)
- PixelTraits for compile-time pixel format validation
- Typed rational pixel types

Error Handling

- Unified std::expected returns (vs 6 different C patterns)
- Typed error codes (enum) instead of string-only errors
- Monadic chaining (.and_then(), .transform())
- All functions can now directly report errors

Resource Management

- RAII file handles (automatic cleanup)
- Move-only semantics preventing accidental copies
- std::unique_ptr-based codec/buffer ownership

Thread Safety

- Per-file std::recursive_mutex for I/O operations
- Thread-safe field caching with std::shared_mutex
- Lock RAII guard for compound operations

Zero-Allocation I/O

- Caller-managed buffers via std::span
- No internal allocations for pixel data

Rational Types

- Multi-precision rationals (Rational16, Rational32, Rational64)
- Exact fractional values without float rounding
- Lossless round-trip for metadata (DPI, GPS, exposure)

Sure, some aspects of these things could be done in C, but mostly that’s not 
the case.  C isn’t capable of the type-safety aspects brought in here.  
Likewise safe resource management.  And portable and safe locking.  This can 
all be done both portably and safely in C++.

>> The proposal includes elements from all three of these
> > implementations, and is really asking the question: what would
> > libtiff look like if we implemented it wholly in C++, but also
> > used extern "C” linkage to completely retain the C API and ABI as they
> > are today.
> 
> Writing it in C++, and offering a C API (unchanged or not) does NOT address 
> the issues I'm concerned about.
> 
> Any use of C++ (in particular the standard library) is a bridge too far for 
> at least some of your target audience.
> 
> Now, I imagine that some people will argue that losing support for such 
> "legacy" code is just too bad. So be it, but be aware that this means the old 
> version of the lib will likely live on as a source of security bugs for years 
> to come.

Making decisions about the scope of what libtiff will and will not support 
would be useful, as well as who are target audience are.  This is the current 
state of things:

If you take a look at the current test matrix: 
(https://gitlab.com/libtiff/libtiff/-/pipelines/2234658359) you’ll see that we 
actively test on a number of platforms.  They are all contemporary—current 
versions of operating systems and compilers, with an “older” set in there as 
well (Ubuntu 22.04LTS, was 20.04 until 24.04 released, Visual Studio 2022; was 
2019 until 2026 was released).  In general, this is the current and previous 
major release of each platform, and current only for rolling releases like 
home-brew.  This is because the primary customers of libtiff are the people who 
distribute it (Linux package managers, BSD ports, Microsoft vcpkg, 
MacPorts/homebrew etc.).  It will of course also be used by application 
developers who are linking it into their applications.  Some of those will also 
be distributed via these package managers, or directly linked in and 
distributed independently as they see fit.

I have always 

Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-30 Thread Roger Leigh via Tiff
On 30 Dec 2025, at 01:49, William Bader  wrote:
> 
> Recently there were issues with not having enough man power to support the 
> tiff utilities. Where will the time come from to do the C++ conversion and 
> maintenance? If the C++ conversion takes a long time to complete to stability 
> and feature parity, is there a chance that the C and C++ versions could fork? 
> What if someone proposes doing a rust conversion of the C++?

Hi William,

The time would come from myself and anyone else interested in this.  I do have 
time for maintenance of the core library, but my interest in the tools is 
limited.

I would say there is a big difference between the core library and the tools in 
terms of maintenance.  We all use the core library, but a lot of the tools have 
a much more limited audience, and some of them are thoroughly obsolete and 
(from my perspective) a fairly pointless maintenance burden to have to deal 
with.  Yet we suffer from a regular stream of CVEs due to their multiple 
quality issues.  Do we really need to generate PostScript, over two decades 
after the whole world moved over to PDF?  And also generate PDF at the same 
time?  Does that really bring much value?  Why are we doing either when there 
are better applications out there for generating PDFs?  You could make the same 
claim for a lot of them.  If they do have any value, it would be to validate 
the behaviour and stability of the C API and the C ABI throughout the 
conversion, so long as they have robust and comprehensive tests (but most 
don’t).  But for people like me (and likely yourself) whose primary use of 
libtiff is to link it into our applications, the tools are a complete 
distraction and an unproductive time sink.

If we implement them in the same repository, I don’t think the C and C++ 
versions would fork, because they would be one and the same thing.  The C API 
would ultimately end up as a thin compatibility wrapper around a C++ core.  
This is exactly what C libraries like Mesa do.  You wouldn’t know they were C++ 
libraries from the outside.

Regarding Rust, there is already a Rust tiff library 
https://github.com/image-rs/image-tiff but it’s quite limited.  If anyone 
wanted to do a Rust conversion, they would currently need to do it through the 
exported C ABI.  It would need to be made a lot safer first.  I don’t see that 
being practical with the current implementation; it might become possible after 
a C++ conversion, plus some improvements to the C API as well.

> Could the proposed C++ API be done as a front-end for C API to give time for 
> testing and experimentation before converting the code base and possibly 
> locking in a design that might have issues?

Initially yes, though I’d want to turn that around at some point (but in a 
completely transparent way).  We would likely not freeze the C++ API during the 
transition, but keep the C API stable and usable at all times.

You can actually see the first step here:
https://gitlab.com/libtiff/libtiff/-/pipelines/2234658359

If you look in the “compat” column, these jobs build libtiff as C++; we already 
have the interfaces marked extern “C” so it’s retaining the C ABI even at this 
stage.  The next step would be to rename the .c files to .cpp and this would 
then be the codebase converted and building as C++ (but with no code changes at 
all).  Everything following that would be small stepwise changes which preserve 
the C API and ABI at every step along the way.

> I have a Linux C application that uses libtiff. If I continue using the C API 
> after the C++ conversion, would I only need to add -lstdc++ to the link after 
> -ltiff?

You shouldn’t need to make any changes at all.  There doesn’t need to be any 
externally-visible dependency upon libstdc++ at all.  Most of the user-facing 
C++ code for users using the new libtiff C++ interface would be header-only 
without any library dependencies.  Code using e.g. standard library types and 
containers which are header-only will not bring in any library dependency.  
Things like std::optional, std::expected, std::array etc.

Kind regards,
Roger

___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-29 Thread William Bader via Tiff
Recently there were issues with not having enough man power to support the tiff 
utilities. Where will the time come from to do the C++ conversion and 
maintenance? If the C++ conversion takes a long time to complete to stability 
and feature parity, is there a chance that the C and C++ versions could fork? 
What if someone proposes doing a rust conversion of the C++?

Could the proposed C++ API be done as a front-end for C API to give time for 
testing and experimentation before converting the code base and possibly 
locking in a design that might have issues?

I have a Linux C application that uses libtiff. If I continue using the C API 
after the C++ conversion, would I only need to add -lstdc++ to the link after 
-ltiff?

William
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-29 Thread Olivier Paquet via Tiff
Le lun. 29 déc. 2025, à 11 h 05, Robin Watts via Tiff 
a écrit :

> On 29/12/2025 11:08, Roger Leigh via Tiff wrote:
> > I mentioned this as a possibility during discussion
>  > in mid December,
>
> Please be aware that moving libtiff to C++ instead of C would cause
> problems for various pieces of software that depend upon it.
>
> The most obvious case, from my point of view, is Ghostscript.
>
> We are careful to avoid any compulsory dependencies that depend upon
> C++, as we have to run on lots of systems where C++ presents a problem.
>

I agree with this point. I find the proposal's categorization of "let's
build this as C++" as low risk bordering on ridiculous. It will break stuff
on some platforms, that's for certain. The rest of it looks more or less
like "let's rewrite everything".


> Now, I imagine that some people will argue that losing support for such
> "legacy" code is just too bad. So be it, but be aware that this means
> the old version of the lib will likely live on as a source of security
> bugs for years to come.


That's the discussion we need to have before any of the more technical
aspects of this. There's no way we can move core libtiff to modern C++ as
proposed and retain the wide support libtiff provides. I think even older
C++ would be a stretch.

There also needs to be a discussion about the value of doing this. What
does it bring that we can't have without it? It will also bring new bugs.
Do the improvements outweigh the problems? Is this true at every point
along the way or only 5-10 years from now?

Olivier
___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff


Re: [Tiff] Proposal for conversion of the codebase to C++ while retaining the existing C API

2025-12-29 Thread Robin Watts via Tiff

On 29/12/2025 11:08, Roger Leigh via Tiff wrote:

I mentioned this as a possibility during discussion

> in mid December,

Please be aware that moving libtiff to C++ instead of C would cause 
problems for various pieces of software that depend upon it.


The most obvious case, from my point of view, is Ghostscript.

We are careful to avoid any compulsory dependencies that depend upon
C++, as we have to run on lots of systems where C++ presents a problem.


As a very brief bit of context and summary.  I’ve long been

> unsatisfied with the base C API of libtiff.  It’s unsafe,
> doesn’t support multi-threading well, is hard to use correctly,
> is stateful where it doesn’t need to be, and has inconsistent
> and hard to use error handling.

All of which case be solved without resorting to C++.


The proposal includes elements from all three of these

> implementations, and is really asking the question: what would
> libtiff look like if we implemented it wholly in C++, but also
> used extern "C” linkage to completely retain the C API and ABI as they
> are today.

Writing it in C++, and offering a C API (unchanged or not) does NOT 
address the issues I'm concerned about.


Any use of C++ (in particular the standard library) is a bridge too far 
for at least some of your target audience.


Now, I imagine that some people will argue that losing support for such 
"legacy" code is just too bad. So be it, but be aware that this means 
the old version of the lib will likely live on as a source of security 
bugs for years to come.


Robin

___
Tiff mailing list
[email protected]
https://lists.osgeo.org/mailman/listinfo/tiff