Re: [Development] qsizetype

2023-03-11 Thread Thiago Macieira
On Saturday, 11 March 2023 00:48:10 PST A. Pönitz wrote:
> Unfortunately for myself (and I am ready to take some blame for this
> ignorance) there was this parallel, and for me on first sight unrelated,
> development of the Q*View zoo, that started to set sizeof(qsizeype) ==
> sizeof(std::size_t) in stone and which currently seems to be the main
> obstacle to have a configurable sizeof(qsizetype). It is surely possible
> (and in my view sensible) to have sizeof(Q*View::size_type) ==
> sizeof(std::size_t) *while keeping* qize_type == int), but I wouldn't bet
> on this kind of outcome.

Not just size_t, and I don't think it's even the most important match. A lot 
of code in Qt 6 has begun assuming that sizeof(qsizetype) == sizeof(void *). 
That is, they are using qsizetype interchangeable with quintptr and qintptr. 
The view classes you're referring to above are not the only example; they may 
be the most visible one and thus the most correctable if we wanted to change, 
but it's very likely that there are many more hidden such uses inside the code 
that would just corrupt pointers left and right.

This is why I replied to Matthias that changing this is going to be more work 
than keep going now.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Cloud Software Architect - Intel DCAI Cloud Engineering


smime.p7s
Description: S/MIME cryptographic signature
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-11 Thread A . Pönitz
On Thu, Mar 09, 2023 at 02:08:48PM +0100, Hasselmann Mathias via Development 
wrote:
> My take on qsizetype: Just revert this failed experiment. 
>
> It's a huge annoyance for little to no benefit.

Correct.

> I'll never understand how this very broken and incomplete experiment could
> make it into Qt's main branch at all.

Strangely enough I don't find any sensibly usable written record of this,
so I can only quote my own memory, which is a system I occacsionally accept
input from but that I don't recommend to trust. But I can at least give my
own reasoning why "this" could happen.

I /believe/ the issue was discussed (at least?) twice on Thursday at the
Qt Contributor summit 2019,

I also /believe/ this was somewhat controversial, and I /believe/ a discussed,
and even /promised/ possible mitigation was to keep qsizetype configurable like
qreal was for a while (for slightly different reasons, and arguably not exactly
helpful either, but...)

That would surely not have been optimal from my personal perspective, but "good
enough" for the time being, as this is structurally one of those "political"
things that is impossible to stop by reason, and there is only a limited number
of problems one can address at a time. My personal hope here was to be able to
chicken out by defining configuring qsizetype to int for my own uses and wait
for history's corrective measures. 

Unfortunately for myself (and I am ready to take some blame for this ignorance) 
there was this parallel, and for me on first sight unrelated, development of
the Q*View zoo, that started to set sizeof(qsizeype) == sizeof(std::size_t)
in stone and which currently seems to be the main obstacle to have a
configurable sizeof(qsizetype). It is surely possible (and in my view sensible)
to have sizeof(Q*View::size_type) == sizeof(std::size_t) *while keeping*
qize_type == int), but I wouldn't bet on this kind of outcome.

Andre'
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-09 Thread Thiago Macieira
On Thursday, 9 March 2023 05:08:48 PST Hasselmann Mathias via Development 
wrote:
> My take on qsizetype: Just revert this failed experiment. It's a huge
> annoyance for little to no benefit. I'll never understand how this very
> broken and incomplete experiment could make it into Qt's main branch at all.

Pretty sure reverting now would be even more disruptive than just fixing the 
remaining issues. It does mean Qt 7 anyway.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Cloud Software Architect - Intel DCAI Cloud Engineering


smime.p7s
Description: S/MIME cryptographic signature
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-09 Thread Hasselmann Mathias via Development
My take on qsizetype: Just revert this failed experiment. It's a huge 
annoyance for little to no benefit. I'll never understand how this very 
broken and incomplete experiment could make it into Qt's main branch at all.


Am 13.09.2022 um 15:12 schrieb Volker Hilsheimer:

On 12 Sep 2022, at 20:04, A. Pönitz  wrote:

On Wed, Sep 07, 2022 at 06:38:30PM +0200, A. Pönitz wrote:

On Mon, Sep 05, 2022 at 05:15:45PM +, Marc Mutz wrote:

[...]
   We have the tools (QT_REMOVED_SINCE + Ivan's work on
   -disable-deprecated-until) to have a user-configurable, rolling BC window
   now We should use these tools to avoid accumulating too much technical
   [...]
   That said, sometimes it's just simpler to do the API change together with
   the rest. I wouldn't worry too much about the effect this has on users of
   Qt APIs: Function argument widening is SC,

I currently fail to understand why all this work needs to have user-visible
consequences *at all* before 7.0 - especially, but not limited, to the now
apparently planned incoming stream of source-incompatible changes including
related deprecations starting to hard-hit users from 6.8 on.

What would have been wrong with starting with

#ifdef I_AM_WORKING_ON_IT
using qsizetyp_ = qsizetype;
#else
using qsizetyp_ = int;
#endif

then have the people working on it (and only those, plus perhaps potential
early adopters) define the macro locally, "port" int to qsizetyp_, and when
everyone is happy with the scope and the implication ofthe change, at 7.0 time,
globally replace qsizetyp_ by qsizetype ?

Why is all this done as operation at an "open heart" instead of having
a "staging" and "production" setup?


Could anyone involved in the decision making that resulted in
the approach taken here please comment?


I can't claim that I was involved in the decision making, but here’s how I see 
it:

We have the tools to change - with some limitations - API signatures without 
breaking either source or binary compatibility. We can deprecate and “weaken” 
old overloads in favour of new overloads; or we can remove the old overload 
completely from the public API and still continue to export the old symbol 
through the module-specific ‘removed_api.cpp’ files.

This is conceptually great news, it gives us a bigger toolbox than what we had 
before. Technically, this is very powerful and useful, allowing us to fix 
mistakes gradually, while giving users control over what kind of deprecation 
warning level they want (from completely silent, up to code no longer 
compiling).

This is IMHO superior to a temporary type alias: A string-based signal/slot 
connection where the signal has been ported ot emit a qsizetyp_ while the slot 
still receives int will fail. So that would break source compatibility. But if 
both slot overloads are still visible for moc when Qt is built, but not to the 
compiler when Qt is used, then those connections will continue to work.

So, I think we have the right tools. The discussion we need to have is when to 
use them. As I have proposed in this thread: this has to be a case by case 
decision.

QTimer should allow timeouts longer than 2^31 msecs, i.e. < 25 days. Great that 
we could fix this before Qt 7.

QDir::count and operator[] now work with qsizetype. I suppose there can be >2^31 files in 
a directory, perhaps more so in 10 years than now. Nevertheless, I do wonder whether this is 
worth the potential source compatibility breakage that is pointed out in the comment 
message. But as long as users need to opt into deprecation warnings explicitly, that is ok 
as well (and would be a “staging" and “production" setup, in practice).


Volker

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development

--
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-09 Thread Ahmad Samir

On 9/3/23 10:14, Marc Mutz via Development wrote:

On 08.03.23 15:30, Ahmad Samir wrote:

So, named casts (static_cast) are better in such cases, as searching for
them in source code is much easier than searching for:
- int(foo)
- (int)foo
- (int)(foo)


In which situation would you want to search for all casts in a piece of source
code? Without the use of a semantic grep tool?



The tool (in this case compiling with -Wshorten-64-to-32, then checking with 
clangd or the build log ...etc) would warn about implicit casts, but not explicit 
ones? now if there is a named cast, then I can search for that much easier than I 
would for the C-style cast (int)foo, or int(foo).


(I haven't checked clang-tidy, but I am guessing it won't be that much different 
than clangd).



I wouldn't like the extra noise of a static_cast when a constructor call
would suffice. Obviously, C-style casts should be avoided (in headers,
IIRC, they're caught by headerscheck).



Good luck with avoiding C-style casts in a huge codebase, e.g. qtbase, there is 
code that's been there since the 90's, and any attempt to "fix" C-style casts in 
the whole codebase would be seen as "churn".



Thanks,
Marc



Thanks,
Ahmad Samir



OpenPGP_signature
Description: OpenPGP digital signature
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-09 Thread Marc Mutz via Development
On 08.03.23 15:30, Ahmad Samir wrote:
> So, named casts (static_cast) are better in such cases, as searching for 
> them in source code is much easier than searching for:
> - int(foo)
> - (int)foo
> - (int)(foo)

In which situation would you want to search for all casts in a piece of source 
code? Without the use of a semantic grep tool?

I wouldn't like the extra noise of a static_cast when a constructor call 
would suffice. Obviously, C-style casts should be avoided (in headers, 
IIRC, they're caught by headerscheck).

Thanks,
Marc

-- 
Marc Mutz 
Principal Software Engineer

The Qt Company
Erich-Thilo-Str. 10 12489
Berlin, Germany
www.qt.io

Geschäftsführer: Mika Pälsi, Juha Varelius, Jouni Lintunen
Sitz der Gesellschaft: Berlin,
Registergericht: Amtsgericht Charlottenburg,
HRB 144331 B

-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2023-03-08 Thread Ahmad Samir

On 1/9/22 12:15, Marc Mutz wrote:

This should be non-issue for function parameters, because widening isn't
an error. The value is less pronounced for return values, because
widening those may cause narrowing in user code, but the hope is that
once we've progressed somewhat, we can enlist compiler support by
globally enabling warnings such as -Wshorten-64-to-32, even though, as
the porting guide included in the
https://bugreports.qt.io/browse/QTBUG-102461  epic describes, this will
not catch manual casts, which is why, sadly, one needs to look at every
int/uint manually.


Sorry for bringing this thread back up from the archives.

So, named casts (static_cast) are better in such cases, as searching for them in 
source code is much easier than searching for:

- int(foo)
- (int)foo
- (int)(foo)

multiplied by the number of other integral types in Qt (qint32, qint64, uint, 
qlonglong, qulonglong... etc).


Thanks,
Ahmad Samir



OpenPGP_signature
Description: OpenPGP digital signature
-- 
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-21 Thread A . Pönitz
On Tue, Sep 13, 2022 at 01:12:57PM +, Volker Hilsheimer wrote:
> > On Wed, Sep 07, 2022 at 06:38:30PM +0200, A. Pönitz wrote:
> >> [...] What would have been wrong with starting with
> >> 
> >> #ifdef I_AM_WORKING_ON_IT using qsizetyp_ = qsizetype; #else using
> >> qsizetyp_ = int; #endif
> >> 
> >> then have the people working on it (and only those, plus perhaps
> >> potential early adopters) define the macro locally, "port" int to
> >> qsizetyp_, and when everyone is happy with the scope and the
> >> implication ofthe change, at 7.0 time, globally replace qsizetyp_
> >> by qsizetype ?
> >> 
> >> Why is all this done as operation at an "open heart" instead of
> >> having a "staging" and "production" setup?
> > 
> > 
> > Could anyone involved in the decision making that resulted in the
> > approach taken here please comment?
> 
> 
> I can't claim that I was involved in the decision making,
> but here’s how I see it:

I appreciate this, albeit I believe it would possibly help to
correct my understanding of the matter if the actual reasoning on
cost and benefits were given by someone actually involved, not just
our combined best guesses.


Some comments (please ignore / handle the ones in [...] in separate
threads to not dilute this here further)

> We have the tools to change - with some limitations - API signatures
> without breaking either source or binary compatibility.

[The introduction of first overloads to functions - and that's one
core 'tool' here - /is/ source-incompatible. That this particular
kind of source-incompatibility was declared "harmless" doesn't
really change the fact.]

> deprecate and “weaken” old overloads in favour of new overloads; or we
> can remove the old overload completely from the public API and still
> continue to export the old symbol through the module-specific
> ‘removed_api.cpp’ files.

> This is conceptually great news, it gives us a bigger toolbox than
> what we had before. Technically, this is very powerful and useful,
> allowing us to fix mistakes gradually, while giving users control over
> what kind of deprecation warning level they want (from completely
> silent, up to code no longer compiling).

[It also means that users are exposed over a lengthy period of time
to these "harmless" source incompatibilities]

> This is IMHO superior to a temporary type alias: A string-based
> signal/slot connection where the signal has been ported ot emit a
> qsizetyp_ while the slot still receives int will fail. So that would
> break source compatibility. But if both slot overloads are still
> visible for moc when Qt is built, but not to the compiler when Qt is
> used, then those connections will continue to work.

[That's a good point, and something that didn't appear to me as a
problem so far, possibly because I have only a few string-based
connects left.  And you are right, this /additional/ incompatibility
hits the users with the current approach only at Qt 7 time, whereas
the temporary alias would hit "as we go". So basically the known
options are not "compile errors as we go plus connect errors at one
time" vs "connect errors as we go plus compile errors at one time".

That's nothing I'd really like to discuss here deeper, because I
actually believe that neither is anywhere close to a good solution,
see below]

> So, I think we have the right tools. The discussion we need to
> have is when to use them. As I have proposed in this thread: this
> has to be a case by case decision.
> 
> QTimer should allow timeouts longer than 2^31 msecs, i.e. < 25
> days.  Great that we could fix this before Qt 7.

[I agree, but this one is completely unrelated to the size of
qsizetype]

> QDir::count and operator[] now work with qsizetype. I suppose there
> can be >2^31 files in a directory, perhaps more so in 10 years than
> now.
> Nevertheless, I do wonder whether this is worth the potential
> source compatibility breakage that is pointed out in the comment
> message. But as long as users need to opt into deprecation warnings
> explicitly, that is ok as well (and would be a “staging" and
> “production" setup, in practice).
 
[Latest at Qt 7 the user has /no choice/...]



I'd like to go back to the beginning of this thread and collect/reword
a few items:


0. To get this out of the way: If all this were about a completely new
codebase, I would see no reason to /not/ use some qsizetype with
sizeof(qsizetype) == sizeof(size_t) and this discussion would not exist.

I also agree that sizeof(qsizetype) == sizeof(size_t) has (limited...)
benefits in interacting with other bits of code, most notably the 
C++ Standard Library.

Consequently, I /do/ consider changing to sizeof(qsizetype) ==
sizeof(size_t) a 'Nice to have' feature. I "just" do not see the effort
necessary to go there anywhere close to even remotely justifiable by
at least two orders of magnitude(!) in any metrics I'd consider appropriate
in the current context.


1. Having a consistent 'main' size type in an application is 

Re: [Development] qsizetype

2022-09-13 Thread Volker Hilsheimer
> On 12 Sep 2022, at 20:04, A. Pönitz  wrote:
> 
> On Wed, Sep 07, 2022 at 06:38:30PM +0200, A. Pönitz wrote:
>> On Mon, Sep 05, 2022 at 05:15:45PM +, Marc Mutz wrote:
>>> [...]
>>>   We have the tools (QT_REMOVED_SINCE + Ivan's work on
>>>   -disable-deprecated-until) to have a user-configurable, rolling BC window
>>>   now We should use these tools to avoid accumulating too much technical
>>>   [...]
>>>   That said, sometimes it's just simpler to do the API change together with
>>>   the rest. I wouldn't worry too much about the effect this has on users of
>>>   Qt APIs: Function argument widening is SC,
>> 
>> I currently fail to understand why all this work needs to have user-visible
>> consequences *at all* before 7.0 - especially, but not limited, to the now
>> apparently planned incoming stream of source-incompatible changes including
>> related deprecations starting to hard-hit users from 6.8 on.
>> 
>> What would have been wrong with starting with
>> 
>> #ifdef I_AM_WORKING_ON_IT
>> using qsizetyp_ = qsizetype; 
>> #else 
>> using qsizetyp_ = int;
>> #endif
>> 
>> then have the people working on it (and only those, plus perhaps potential
>> early adopters) define the macro locally, "port" int to qsizetyp_, and when
>> everyone is happy with the scope and the implication ofthe change, at 7.0 
>> time,
>> globally replace qsizetyp_ by qsizetype ?
>> 
>> Why is all this done as operation at an "open heart" instead of having 
>> a "staging" and "production" setup?
> 
> 
> Could anyone involved in the decision making that resulted in
> the approach taken here please comment?


I can't claim that I was involved in the decision making, but here’s how I see 
it:

We have the tools to change - with some limitations - API signatures without 
breaking either source or binary compatibility. We can deprecate and “weaken” 
old overloads in favour of new overloads; or we can remove the old overload 
completely from the public API and still continue to export the old symbol 
through the module-specific ‘removed_api.cpp’ files.

This is conceptually great news, it gives us a bigger toolbox than what we had 
before. Technically, this is very powerful and useful, allowing us to fix 
mistakes gradually, while giving users control over what kind of deprecation 
warning level they want (from completely silent, up to code no longer 
compiling).

This is IMHO superior to a temporary type alias: A string-based signal/slot 
connection where the signal has been ported ot emit a qsizetyp_ while the slot 
still receives int will fail. So that would break source compatibility. But if 
both slot overloads are still visible for moc when Qt is built, but not to the 
compiler when Qt is used, then those connections will continue to work.

So, I think we have the right tools. The discussion we need to have is when to 
use them. As I have proposed in this thread: this has to be a case by case 
decision.

QTimer should allow timeouts longer than 2^31 msecs, i.e. < 25 days. Great that 
we could fix this before Qt 7.

QDir::count and operator[] now work with qsizetype. I suppose there can be 
>2^31 files in a directory, perhaps more so in 10 years than now. Nevertheless, 
I do wonder whether this is worth the potential source compatibility breakage 
that is pointed out in the comment message. But as long as users need to opt 
into deprecation warnings explicitly, that is ok as well (and would be a 
“staging" and “production" setup, in practice).


Volker

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-12 Thread A . Pönitz
On Wed, Sep 07, 2022 at 06:38:30PM +0200, A. Pönitz wrote:
> On Mon, Sep 05, 2022 at 05:15:45PM +, Marc Mutz wrote:
> >  [...]
> >We have the tools (QT_REMOVED_SINCE + Ivan's work on
> >-disable-deprecated-until) to have a user-configurable, rolling BC window
> >now We should use these tools to avoid accumulating too much technical
> >[...]
> >That said, sometimes it's just simpler to do the API change together with
> >the rest. I wouldn't worry too much about the effect this has on users of
> >Qt APIs: Function argument widening is SC,
> 
> I currently fail to understand why all this work needs to have user-visible
> consequences *at all* before 7.0 - especially, but not limited, to the now
> apparently planned incoming stream of source-incompatible changes including
> related deprecations starting to hard-hit users from 6.8 on.
> 
> What would have been wrong with starting with
> 
> #ifdef I_AM_WORKING_ON_IT
> using qsizetyp_ = qsizetype; 
> #else 
> using qsizetyp_ = int;
> #endif
> 
> then have the people working on it (and only those, plus perhaps potential
> early adopters) define the macro locally, "port" int to qsizetyp_, and when
> everyone is happy with the scope and the implication ofthe change, at 7.0 
> time,
> globally replace qsizetyp_ by qsizetype ?
> 
> Why is all this done as operation at an "open heart" instead of having 
> a "staging" and "production" setup?


Could anyone involved in the decision making that resulted in
the approach taken here please comment?

Andre'
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-08 Thread Thiago Macieira
On Thursday, 8 September 2022 02:23:39 PDT Edward Welbourne wrote:
> The reason we plan for Qt 7 is that it's the next time we can make
> certain kinds of backwards-incompatible changes.  When we decide that
> such a change is needed, we plan for it, because we can't do it yet, or
> indeed any time soon.

Particularly because we realised that we did not do all the "### Qt 6" changes 
for 6.0. This time around, we're actually making the changes right now. Many 
of the changes can be added to Qt 6 itself, if they are simple overloads, and 
those benefit most people. Some others can't and must instead be hidden with 
#ifdef.

The point of this thread is to discuss what work we should do now, if neither 
option above can apply or if it becomes too ugly to maintain.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Cloud Software Architect - Intel DCAI Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-08 Thread Edward Welbourne
Nicolas Fella wrote:
>> The fact that KDE does not use Qt6 yet has rather little to do with
>> the quality of Qt6.

Kevin Kofler (8 September 2022 01:37) replied:
> Where have I claimed that it does? I sense a strawman…
>
> My point is that it takes time for KDE and other downstreams to adopt
> a new major release of Qt, no matter the reason why it does, and that
> moving on to Qt 7 before or shortly after the downstream consumers are
> ported to Qt 6 would be very unhelpful.

Well, since you brought up the strawman, where exaactly did anyone claim
Qt 7 was in any sense so imminent that it'd be "before or shortly after"
KDE has upgraded to Qt 6 ?  Please do not mistake long-term planning for
an intent to release what we're planning any time soon.  I don't know
what the plans for the major version cycle currently are, but the
roadmap for the foreseeable future is all about continuing releases of
Qt 6.

The reason we plan for Qt 7 is that it's the next time we can make
certain kinds of backwards-incompatible changes.  When we decide that
such a change is needed, we plan for it, because we can't do it yet, or
indeed any time soon.

Eddy.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Kevin Kofler via Development
Nicolas Fella wrote:
> The fact that KDE does not use Qt6 yet has rather little to do with the
> quality of Qt6.

Where have I claimed that it does? I sense a strawman…

My point is that it takes time for KDE and other downstreams to adopt a new 
major release of Qt, no matter the reason why it does, and that moving on to 
Qt 7 before or shortly after the downstream consumers are ported to Qt 6 
would be very unhelpful.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Nicolas Fella

Am 07.09.22 um 08:45 schrieb Kevin Kofler via Development:

Alex Blasche wrote:

2.) An alternative might be to make this change in one go for Qt 7. We
would keep Qt 6.x on the status quo but start adding compatible
replacement APi with an absolute change at 7.0 (ifdefs or typedefs come to
mind). Users would only be burdened one time (though it being one BIG time
effort). Such a change would be much simpler in Qt headers.

It scares me that a Qt 7 is already being planned or discussed at all,
considering that your major consumers such as KDE Plasma are not even ported
to Qt 6 yet! (Note that I am talking about *consumers* here, not only about
(paying) customers. The former includes FOSS projects such as KDE.)

Those major/BIC releases need to happen a lot less frequently, or ideally,
stop entirely. At least if you want your consumers to keep up (and you
clearly do or you would not have restricted access to Qt 5.15 LTS).

You should plan for Qt 6.x releases (rather than 7.x) for at least 10 more
years, if not indefinitely.


The fact that KDE does not use Qt6 yet has rather little to do with the
quality of Qt6. The main reason is that we want to polish our own APIs
more before introducing a new major version of them, which will be based
on Qt6. If you are actually interested about what KDE thinks about Qt6
you can find my thoughts at https://www.youtube.com/watch?v=nBzYuZIX1iY.

Please do not use KDE for your arguments and demands without actually
checking with relevant people whether your assumptions are correct.

Nicolas

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Kevin Kofler via Development
Ilya Fedin wrote:
> I believe the fact KDE is not ported to Qt 6 yet is more question of
> bureaucracy coordination of a lot of people in different KDE projects.
> That is, deciding that they want to port to Qt 6 takes time, then every
> project maintainer should do the port and it seems they want to
> announce Qt 6 support only with KF6, it's not like GNOME that changed
> version scheme to not to associate their major version with major
> version of their toolkit.
> 
> Ports themselves seem to be trivial for most of KDE frameworks from
> what I saw: cmake changes, some deprecation of APIs using Qt's
> deprecated APIs, etc.
> 
> I.e. what takes the time seem to be mostly routine, coordination and
> bureaucracy rather than solving big breaking changes while porting.

In practice, porting is always an issue for application developers. See, 
e.g., https://valdyas.org/fading/hacking/happy-porting/ (as one example out 
of many, and no, I am not the author of that blog post).

Application developers and end users both just want their applications to 
keep working. No application developer wants to spend time on porting an 
application to a new incompatible version of a library. Any time wasted on 
that cannot be spent on improving the application.

> A framework without major updates will stagnate. Qt 6 finally added RHI

RHI can be added without breaking source and binary compatibility. In fact, 
that is exactly what was done in 5.14 with the introduction of the QSG_RHI 
environment variable. It is a behind-the-scenes change that does not need to 
affect the public API or ABI.

> and there's still a lot of modern APIs in systems, Qt doesn't provide
> cross-platform abstractions for.

Adding new cross-platform abstractions for additional system APIs (modern or 
not) surely does not require backwards-incompatible changes to what is 
already there. It should just be a matter of adding a new module with a new 
API to Qt, without changing or removing anything existing.

> I don't understand how you can ask a piece of software to not to have
> major updates **indefinitely**...

I am not saying that we should not have Qt releases adding new features. I 
am saying that we should not have (source- and/or binary-) backwards-
incompatible Qt releases. I do not agree with your implied claim that a 
library cannot evolve without breaking existing applications.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Thiago Macieira
On Tuesday, 6 September 2022 23:45:45 PDT Kevin Kofler via Development wrote:
> It scares me that a Qt 7 is already being planned or discussed at all,

We had "### Qt 7" remarks in Qt 6.0.0.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Cloud Software Architect - Intel DCAI Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread A . Pönitz
On Mon, Sep 05, 2022 at 05:15:45PM +, Marc Mutz wrote:
>  [...]
>We have the tools (QT_REMOVED_SINCE + Ivan's work on
>-disable-deprecated-until) to have a user-configurable, rolling BC window
>now We should use these tools to avoid accumulating too much technical
>[...]
>That said, sometimes it's just simpler to do the API change together with
>the rest. I wouldn't worry too much about the effect this has on users of
>Qt APIs: Function argument widening is SC,

I currently fail to understand why all this work needs to have user-visible
consequences *at all* before 7.0 - especially, but not limited, to the now
apparently planned incoming stream of source-incompatible changes including
related deprecations starting to hard-hit users from 6.8 on.

What would have been wrong with starting with

#ifdef I_AM_WORKING_ON_IT
using qsizetyp_ = qsizetype; 
#else 
using qsizetyp_ = int;
#endif

then have the people working on it (and only those, plus perhaps potential
early adopters) define the macro locally, "port" int to qsizetyp_, and when
everyone is happy with the scope and the implication ofthe change, at 7.0 time,
globally replace qsizetyp_ by qsizetype ?

Why is all this done as operation at an "open heart" instead of having 
a "staging" and "production" setup?

Andre'
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Volker Hilsheimer
> On 7 Sep 2022, at 08:36, Alex Blasche  wrote:
> 
> Hi Volker,
> 
>> 
>> From: Development  on behalf of Volker 
>> Hilsheimer 
> 
>> I agree that it would be great if users of Qt could flip on aggressive 
>> compile options to get warnings about narrowing-conversions. 
>> But again, is that worth it? And is that more important to users than having 
>> a few releases of Qt where they don’t have to constantly 
>> chase after new deprecations? I honestly doubt it. I know a few folks in the 
>> Creator and Design Studio teams that would love a Qt 
>> release or two with no new warnings.
> 
>> So, I in principle agree with your plan, but let’s focus on the modules 
>> where there is at least a plausible usecase for >2G data. 
>> For the rest, reach out to the respective maintainers, please.
> 
> I'd like to rephrase the above to ensure I correctly understand Volker here:
> 
> The suggestion is to do the conversion to API's which benefit from >2G data 
> only. In such cases the changes would be done such that we only add the 
> converted API behind guards which come into effect in Qt 7 
> (QT_DEPRECATED_SINCE/QT_REMOVED_SINCE(7,0)). This means we don't need fancy 
> overload solution for getter like 
> 
> int size() vs qsizetype  size()
> 
> as we are talking about a switch at Qt 7 time. Is this your suggestion 
> Volker? 


Short version: In principle, yes. For some definition of “benefit”. I think 
there is a significant amount of code where we can simply live with the status 
quo, even if it’s not ideal. The rest still need to be evaluated case-by-case, 
and waiting for Qt 7 will not always work.

Long version: 

Marc has stated that

>>> 'That said, no actual user-visible size-mismatch bug has yet required the 
>>> change of public interfaces.’


That’s good. As per that statement, this is hopefully not going to be a major 
source of API deprecations or otherwise incompatible changes. But it is a major 
source of work. There are a few no-brainers, like QBuffer [1], or even 
QDataStream [2] - these are bugs, thanks for fixing the one, and please don’t 
wait until Qt 7 with the other.

[1] https://bugreports.qt.io/browse/QTBUG-102171
[2] https://bugreports.qt.io/browse/QTBUG-105034

But then I’m looking at https://bugreports.qt.io/browse/QTBUG-104824 and even 
though QMdiArea doesn’t have any public API operating on int, I don’t expect 
anyone to spend time on changing the implementation to use qsizetype (or auto, 
or ranged-for). And I will understand if maintainers get somewhat upset if they 
are expected to review such patches.


> I would support this solution as I have the same concern about never-ending 
> porting requirements for our users.


Looking at the two cases that are quoted to have required API changes in the 
various JIRA tickets under the https://bugreports.qt.io/browse/QTBUG-102461 
epic:

QByteArrayList::join now supports separators >2G [1]. QVersionNumber, operating 
on a QList as segments, now uses qsizetype in APIs directly related to 
accessing elements or slices of that list [2].

[1] https://codereview.qt-project.org/c/qt/qtbase/+/379810
[2] https://codereview.qt-project.org/c/qt/qtbase/+/389682

If either of those changes would have resulted in deprecation warnings for 
users, should we have made them anyway? I don’t think so.

I’m sure there will be cases where we have to deprecate API in order to fix 
legitimate issues. For changes that result in porting efforts from our users, 
we should have *very* good justification: data loss or corruption, or potential 
security issues, etc. There are hopefully few of those, and we can hopefully 
take each of those case-by-case. The respective maintainer should make that 
call, hence my request to involve them in the discussion, and to not just throw 
patches at them for review.

Speaking as a maintainer: Yes, there are legitimate use cases for e.g. a 
QSqlTableModel to operate on tables with >2G rows. And there are legitimate use 
cases to have a QSlider for a wider range than 4G. We cannot change those APIs 
without breaking compatibility. Indeed, it would be nice to have a consistent 
API, and to support those use cases. But even if we could make those changes 
without breaking ABI compatibility - as long as we are putting a porting burden 
on our users, I would not be on board with making them within Qt 6.


For everything that doesn’t have strong justification, cannot be changed 
without impact on existing client code, or - as far as implementation is 
concerned - has no plausible usecase for dealing with >2G items, we can still 
gradually annotate our code and our API via `#if QT_VERSION` or 
`QT_DEPRECATED_SINCE(7, 0)`. Esp since we hopefully don’t have to think 
systematically about a Qt 7 for several more years, adding those annotations to 
the code seems better than peppering the code with // ‘### Qt 7’ comments that 
someone hopefully greps for, or creating JIRA tickets that we then have a 

Re: [Development] qsizetype

2022-09-07 Thread Ilya Fedin
On Wed, 07 Sep 2022 08:45:45 +0200
Kevin Kofler via Development  wrote:

> It scares me that a Qt 7 is already being planned or discussed at
> all, considering that your major consumers such as KDE Plasma are not
> even ported to Qt 6 yet! (Note that I am talking about *consumers*
> here, not only about (paying) customers. The former includes FOSS
> projects such as KDE.)

I believe the fact KDE is not ported to Qt 6 yet is more question of
bureaucracy coordination of a lot of people in different KDE projects.
That is, deciding that they want to port to Qt 6 takes time, then every
project maintainer should do the port and it seems they want to
announce Qt 6 support only with KF6, it's not like GNOME that changed
version scheme to not to associate their major version with major
version of their toolkit.

Ports themselves seem to be trivial for most of KDE frameworks from
what I saw: cmake changes, some deprecation of APIs using Qt's
deprecated APIs, etc.

I.e. what takes the time seem to be mostly routine, coordination and
bureaucracy rather than solving big breaking changes while porting.

> Those major/BIC releases need to happen a lot less frequently, or
> ideally, stop entirely. At least if you want your consumers to keep
> up (and you clearly do or you would not have restricted access to Qt
> 5.15 LTS).
> 
> You should plan for Qt 6.x releases (rather than 7.x) for at least 10
> more years, if not indefinitely.

A framework without major updates will stagnate. Qt 6 finally added RHI
and there's still a lot of modern APIs in systems, Qt doesn't provide
cross-platform abstractions for.

I don't understand how you can ask a piece of software to not to have
major updates **indefinitely**...
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Volker Hilsheimer
> On 7 Sep 2022, at 09:00, Andreas Aardal Hanssen  wrote:
> 
> On Wed, Sep 7, 2022, at 08:45, Kevin Kofler via Development wrote:
>> Alex Blasche wrote:
>>> 2.) An alternative might be to make this change in one go for Qt 7. We
>>> would keep Qt 6.x on the status quo but start adding compatible
>>> replacement APi with an absolute change at 7.0 (ifdefs or typedefs come to
>>> mind). Users would only be burdened one time (though it being one BIG time
>>> effort). Such a change would be much simpler in Qt headers.
>> It scares me that a Qt 7 is already being planned or discussed at all, 
>> considering that your major consumers such as KDE Plasma are not even ported 
>> to Qt 6 yet! (Note that I am talking about *consumers* here, not only about 
>> (paying) customers. The former includes FOSS projects such as KDE.)
>> Those major/BIC releases need to happen a lot less frequently, or ideally, 
>> stop entirely. At least if you want your consumers to keep up (and you 
>> clearly do or you would not have restricted access to Qt 5.15 LTS).
>> You should plan for Qt 6.x releases (rather than 7.x) for at least 10 more 
>> years, if not indefinitely.
> 
> I second this, considering the breakages in APIs and behavior, and bugs 
> introduced, in Qt 6, I feel discussions about Qt 7 send a very poor signal. 
> Porting to Qt 6 is a huge and costly effort. With so much care being made not 
> to break BC, I would hope more care would be taken to preserve behavioral 
> (QVariant/QMetaType) and API compatibility. I hope KDE not using Qt 6 yet 
> sends a message taken seriously on the Qt side. Stay on Qt 6 for a long time 
> guys.
> 
> Andreas


I understand that seeing Qt 7 in discussions is raising eyebrows, given where 
we are with Qt 6. But we need to be able to discuss what we can do without BiC 
breakage within Qt 6, and what we can only do for Qt 7, and what lessons we 
learned from the Qt 6 work. That doesn’t mean that we don’t plan to continue to 
focus on Qt 6!

Whether, and how often, we will break compatibility (be it binary, or behavior, 
or the result of significant, “world changing” rework) in the future deserves a 
separate discussion thread anyway.


Volker

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Andreas Aardal Hanssen
On Wed, Sep 7, 2022, at 08:45, Kevin Kofler via Development wrote:
> Alex Blasche wrote:
> > 2.) An alternative might be to make this change in one go for Qt 7. We
> > would keep Qt 6.x on the status quo but start adding compatible
> > replacement APi with an absolute change at 7.0 (ifdefs or typedefs come to
> > mind). Users would only be burdened one time (though it being one BIG time
> > effort). Such a change would be much simpler in Qt headers.
> It scares me that a Qt 7 is already being planned or discussed at all, 
> considering that your major consumers such as KDE Plasma are not even ported 
> to Qt 6 yet! (Note that I am talking about *consumers* here, not only about 
> (paying) customers. The former includes FOSS projects such as KDE.)
> Those major/BIC releases need to happen a lot less frequently, or ideally, 
> stop entirely. At least if you want your consumers to keep up (and you 
> clearly do or you would not have restricted access to Qt 5.15 LTS).
> You should plan for Qt 6.x releases (rather than 7.x) for at least 10 more 
> years, if not indefinitely.

I second this, considering the breakages in APIs and behavior, and bugs 
introduced, in Qt 6, I feel discussions about Qt 7 send a very poor signal. 
Porting to Qt 6 is a huge and costly effort. With so much care being made not 
to break BC, I would hope more care would be taken to preserve behavioral 
(QVariant/QMetaType) and API compatibility. I hope KDE not using Qt 6 yet sends 
a message taken seriously on the Qt side. Stay on Qt 6 for a long time guys.

Andreas
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Kevin Kofler via Development
Alex Blasche wrote:
> 2.) An alternative might be to make this change in one go for Qt 7. We
> would keep Qt 6.x on the status quo but start adding compatible
> replacement APi with an absolute change at 7.0 (ifdefs or typedefs come to
> mind). Users would only be burdened one time (though it being one BIG time
> effort). Such a change would be much simpler in Qt headers.

It scares me that a Qt 7 is already being planned or discussed at all, 
considering that your major consumers such as KDE Plasma are not even ported 
to Qt 6 yet! (Note that I am talking about *consumers* here, not only about 
(paying) customers. The former includes FOSS projects such as KDE.)

Those major/BIC releases need to happen a lot less frequently, or ideally, 
stop entirely. At least if you want your consumers to keep up (and you 
clearly do or you would not have restricted access to Qt 5.15 LTS).

You should plan for Qt 6.x releases (rather than 7.x) for at least 10 more 
years, if not indefinitely.

Kevin Kofler

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-07 Thread Alex Blasche
Hi Volker,

>
>From: Development  on behalf of Volker 
>Hilsheimer 

>I agree that it would be great if users of Qt could flip on aggressive compile 
>options to get warnings about narrowing-conversions. 
>But again, is that worth it? And is that more important to users than having a 
>few releases of Qt where they don’t have to constantly 
>chase after new deprecations? I honestly doubt it. I know a few folks in the 
>Creator and Design Studio teams that would love a Qt 
>release or two with no new warnings.

>So, I in principle agree with your plan, but let’s focus on the modules where 
>there is at least a plausible usecase for >2G data. 
>For the rest, reach out to the respective maintainers, please.

I'd like to rephrase the above to ensure I correctly understand Volker here:

The suggestion is to do the conversion to API's which benefit from >2G data 
only. In such cases the changes would be done such that we only add the 
converted API behind guards which come into effect in Qt 7 
(QT_DEPRECATED_SINCE/QT_REMOVED_SINCE(7,0)). This means we don't need fancy 
overload solution for getter like 

int size() vs qsizetype  size()

as we are talking about a switch at Qt 7 time. Is this your suggestion Volker? 

I would support this solution as I have the same concern about never-ending 
porting requirements for our users.

--
Alex
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-06 Thread Volker Hilsheimer
> On 5 Sep 2022, at 19:15, Marc Mutz  wrote:
> 
> Hi,
> 
> Experience shows that we'll have many, many, many things to consider come Qt 
> 7. And as Qt 6 has shown, such trivialities will be left by the wayside. So, 
> playing the devil's advocate here: if this work is too much for Qt 6.x, what 
> makes us think we'll get it done in the frenzy leading up to Qt 7.0?
> 
> We have the tools (QT_REMOVED_SINCE + Ivan's work on 
> -disable-deprecated-until) to have a user-configurable, rolling BC window 
> now. We should use these tools to avoid accumulating too much technical debt 
> until 7.0. From a C++ POV, we now only need major versions to break 
> polymorphic inheritance (incl., but not limited to, virtuals), and signals 
> and slots (the latter because of conflicting SC-solutions for new-style and 
> old-style connects).
> 
> That said, no actual user-visible size-mismatch bug has yet required the 
> change of public interfaces.
> 
> So there's definitely two stages here: 1) fixing the implementation (we must 
> do this) and 2) fixing the API (much less urgent iff our headers are clean 
> w.r.t. aggressive narrowing warnings (-Wnarrowing) so users can enable them 
> for their own projects to catch the bugs at the API boundary).
> 
> So, I think the priority and therefore work order should be:
> 
> 1. Fix all int/uint in the implementation
> 2. Fix our headers (public and private) to be resistant to aggressive 
> narrowing-conversion compiler warnings
> 3. Enable said compiler warnings for (growing subsets of) Qt implementation 
> code
> 4. Fix our APIs
> 
> Seeing as user-side truncation occurs when passing to function arguments, and 
> seeing that function arguments are much less critical when it comes to SC 
> than function return values, we might even split the API update part to fix 
> setters first, and only then getters.
> 
> That said, sometimes it's just simpler to do the API change together with the 
> rest. I wouldn't worry too much about the effect this has on users of Qt 
> APIs: Function argument widening is SC, and function return value widening 
> can be buffered with liberal use of auto on the side of the user (I don't 
> want to count the number of trivial bugs that would have been prevented in 
> the Qt5->6 port if Qt didn't have such a conservative stance on the use of 
> auto).

Hey,

It’s great that we have the tools to make most of the changes gradually, thank 
to you, Ivan and others. And I agree that we should keep going, in those 
modules where the relation between the problem we are solving, and the effort 
it takes, is sensible. Thanks for taking the lead on that.

But for many modules, there is simply no use case for >2G data, and whatever 
theoretical issues people might be able to provoke by passing huge lists or 
indices into APIs are constructed and self-inflicted.

For instance, I don’t see why we need to prepare QGraphicsScene or 
QAbstractAnimation’s implementation to handle >2G items, in any of the lists 
they are operating on. Is every narrowing when converting to fixed points a 
serious problem? I don’t see that. And a lot of people working on keeping Qt’s 
UI and rendering functionality competitive will not want to spend time 
reviewing and arguing about changes that bring 64bit safety to code that has 
absolutely no use for that (e.g. we don’t need >2GB opengl buffers). At least 
you need to make sure that the maintainers of those modules are onboard, are 
willing to join, or at least review your patches.

For those modules, doing things gradually as drive-by’s when we touch the 
respective code anyway, will be fine (at least as far as trivial 
s/int/qsizetype, or refactoring to ranged-for is concerned).


> We should just communicate clearly that for the next few releases, any int in 
> the API is subject to potential porting to qsizetype, and suggest users to 
> prepare their own code for the change, ideally already in the Qt 6.4.0 
> changelog, and then in all following ones.


I agree that it would be great if users of Qt could flip on aggressive compile 
options to get warnings about narrowing-conversions. But again, is that worth 
it? And is that more important to users than having a few releases of Qt where 
they don’t have to constantly chase after new deprecations? I honestly doubt 
it. I know a few folks in the Creator and Design Studio teams that would love a 
Qt release or two with no new warnings.

So, I in principle agree with your plan, but let’s focus on the modules where 
there is at least a plausible usecase for >2G data. For the rest, reach out to 
the respective maintainers, please.


Volker

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-06 Thread Edward Welbourne
Volker Hilsheimer (5 September 2022 17:31) wrote:
>> We have virtual functions that take int and could potentially be fed
>> by the return value of container.size() (or generally need to be able
>> to handle values >2G), so should take a qsizetype (say,
>> QAbstractItemModel::insertRows, overridden by QSqlTableModel). I
>> don’t see how we can change those until Qt 7.

We can, however, get ahead of the game with the Qt 7 version already
sketched out and tested in advance, within #if-ery.  It may bitrot a bit
between now and Qt 7, but getting it substantially ready now should make
the 7.0 preparations less frenetic than what I was doing two years ago,
upon discovering how many ### Qt 6 comments were going unattended.

>> So we will have to live with some inconsistency.

Indeed.

>> For the other modules, esp those related to user interface
>> development, the chance that an int-API is called with a value above
>> 2G seems very low, even if they are operating internally on Qt
>> containers. I might be missing some real world examples though where
>> e.g. QGraphicsScene or a QTextDocument needs to operate on >2G items.

While I mostly agree, I do feel obliged to point out that there are
surprises out there.  It's now about a quarter century since I had to
rework a GIS system's datastore to cope with vector maps with more than
2 Gi of data describing the entities in them, because we had customers
whose datasets were that big.  (As this had to work on systems with
32-bit fpos_t, this involved a virtual file system.  But we had one of
those anyway, so I just had to teach it to be 64-bit safe and split each
logical file up into <= 2 GiB physical files.)  So there have been folk out
there with vast datasets for (at least) a quarter century.  I suspect
most such folk are still in the "specialised" category, so don't really
expect them to be using Qt to manage their vast data-sets, but I do
expect there shall be ever more use-cases where folk gently find the
sizes of their data-sets growing and will be "disappointed" if they hit
a 2 Gi size limit in the software they're using.

In most cases such datasets can and should be handled progressively, but
letting the author of a prototype do it the dumb way - by loading
the whole dataset in one go - will mean they can do the prototype in Qt,
which makes them more likely to still be using Qt when they do the
production version (and optimize away the insanely huge loads).

Marc Mutz (5 September 2022 19:15) wrote:
> Experience shows that we'll have many, many, many things to consider
> come Qt 7. And as Qt 6 has shown, such trivialities will be left by
> the wayside.

See above; indeed, two years ago, I discovered many things, some of them
by no means trivial, that had been punted to the major version and
forgotten about.  That was in addition to all the things I knew about
that needed to happen at the major version change.

This can, however, be at least party addressed by judicious improvements
to process: we could, for example, require every ### Qt 7 comment to
mention a Jira ticket describing the work that needs to be done for Qt
7, of which it's a part, marked suitably in Jira (Fix version 7.0,
perhaps a tag), so that we can search effectively for these tickets when
the time comes.

> So, playing the devil's advocate here: if this work is too much for Qt
> 6.x, what makes us think we'll get it done in the frenzy leading up to
> Qt 7.0?

We can be better prepared than we were for Qt 6.  We can do quite a lot
of preparation now to reduce that frenzy: simple mindless #if-ery on
QT_VERSION can do some of that, the QT[67]_ONLY macros some more.

> We have the tools (QT_REMOVED_SINCE + Ivan's work on
> -disable-deprecated-until) to have a user-configurable, rolling BC
> window now. We should use these tools to avoid accumulating too much
> technical debt until 7.0. From a C++ POV, we now only need major
> versions to break polymorphic inheritance (incl., but not limited to,
> virtuals), and signals and slots (the latter because of conflicting
> SC-solutions for new-style and old-style connects).

All of which also helps us reduce the amount of work we're forced to
leave pending until Qt 7; we can do it now and they'll happen naturally,
without any work during the mad frenzy.  We can then clean up after that
frenzy subsides, rearranging the code to take version >= 7 for granted,
starting after feature-freeze.

> That said, no actual user-visible size-mismatch bug has yet required
> the change of public interfaces.

This is good to hear; but see above.  I do think it's only a matter of
time before some class of client code does run into a need to manage
data-sets we would presently consider insanely huge; and we'll probably
be surprised by which application domain ends up needing it first.

For that matter, the further we progress towards 64-bit usage, the more
"specialized" categories of software there shall be, that presently have
to use custom software to handle their vast 

Re: [Development] qsizetype

2022-09-05 Thread Marc Mutz
Hi,

Experience shows that we'll have many, many, many things to consider come Qt 7. 
And as Qt 6 has shown, such trivialities will be left by the wayside. So, 
playing the devil's advocate here: if this work is too much for Qt 6.x, what 
makes us think we'll get it done in the frenzy leading up to Qt 7.0?

We have the tools (QT_REMOVED_SINCE + Ivan's work on -disable-deprecated-until) 
to have a user-configurable, rolling BC window now. We should use these tools 
to avoid accumulating too much technical debt until 7.0. From a C++ POV, we now 
only need major versions to break polymorphic inheritance (incl., but not 
limited to, virtuals), and signals and slots (the latter because of conflicting 
SC-solutions for new-style and old-style connects).

That said, no actual user-visible size-mismatch bug has yet required the change 
of public interfaces.

So there's definitely two stages here: 1) fixing the implementation (we must do 
this) and 2) fixing the API (much less urgent iff our headers are clean w.r.t. 
aggressive narrowing warnings (-Wnarrowing) so users can enable them for their 
own projects to catch the bugs at the API boundary).

So, I think the priority and therefore work order should be:

1. Fix all int/uint in the implementation
2. Fix our headers (public and private) to be resistant to aggressive 
narrowing-conversion compiler warnings
3. Enable said compiler warnings for (growing subsets of) Qt implementation code
4. Fix our APIs

Seeing as user-side truncation occurs when passing to function arguments, and 
seeing that function arguments are much less critical when it comes to SC than 
function return values, we might even split the API update part to fix setters 
first, and only then getters.

That said, sometimes it's just simpler to do the API change together with the 
rest. I wouldn't worry too much about the effect this has on users of Qt APIs: 
Function argument widening is SC, and function return value widening can be 
buffered with liberal use of auto on the side of the user (I don't want to 
count the number of trivial bugs that would have been prevented in the Qt5->6 
port if Qt didn't have such a conservative stance on the use of auto).

We should just communicate clearly that for the next few releases, any int in 
the API is subject to potential porting to qsizetype, and suggest users to 
prepare their own code for the change, ideally already in the Qt 6.4.0 
changelog, and then in all following ones.

Thanks,
Marc

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-05 Thread Volker Hilsheimer
> On 5 Sep 2022, at 14:46, Alex Blasche  wrote:
>> -Original Message-
>> From: Development  On Behalf Of
>> Marc Mutz

[…]

>> I just want to raise awareness of the issue, which, really, is just an 
>> incomplete
>> port of the Qt containers to qsizetype in 6.0.
>> 
>> Over the last months, however, I've realized that "just the Qt containers"
>> introduces a nasty API inconsistency in Qt: qsizetype in Qt containers and 
>> int
>> elsewhere. This inconsistency is un-Qt-ish and creates problems for our 
>> users.

[…]

> I like the container.size() vs setFoo(int) use case as example to decide how 
> far we want to spin this inconsistency further ahead throughout all of Qt . I 
> totally agree with Marc's argument that it is not Qtish and causes pain to 
> every user.  I guess we have three options:
> 
> 1.) What I am wondering though is if we were to sink all the time into Qt to 
> convert all API's, would Qt be blamed for all the continued porting effect 
> throughout the years on this issue? After all we cannot do this for one 
> single release in one go. It would probably find its way into each release 
> over a couple of years. Does this help Qt in its goal to be easy to use? The 
> compatibility layer does add its own layer of complexity.
> 
> 2.) An alternative might be to make this change in one go for Qt 7. We would 
> keep Qt 6.x on the status quo but start adding compatible replacement APi 
> with an absolute change at 7.0 (ifdefs or typedefs come to mind). Users would 
> only be burdened one time (though it being one BIG time effort). Such a 
> change would be much simpler in Qt headers.
> 
> 3.) On the other hand, we have had a lot of Qt 6 ports already and so far, it 
> did not come up as a major issue in those ports. Could we simply accept this 
> inconsistency? After all, would QSplitter ever need setWidget(qsizetype, 
> QWidget*)? Under this scenario we might agree to convert all internal Qt uses 
> of containers but leave the public API untouched. Of course some dedicated 
> use case, where extra large files/chunks of data should be read, could get 
> the treatment much earlier.
> 
> Personally, I cannot agree whether I like Option 2 or 3 more.


We have virtual functions that take int and could potentially be fed by the 
return value of container.size() (or generally need to be able to handle values 
>2G), so should take a qsizetype (say, QAbstractItemModel::insertRows, 
overridden by QSqlTableModel). I don’t see how we can change those until Qt 7.

So we will have to live with some inconsistency. Given that, and also trying to 
be pragmatic wrt the effort required (potentially also in client code) in 
relation to the likelihood of real issues, I’d say that we should focus the 
proactive effort on QtCore, QtNetwork, QtSQL, QtWebSockets/Channel, and perhaps 
a few more modules that are likely to have to process >2G quantities of data. 
In those modules, perhaps there’s a security component to consider, where wrong 
handling of qsizetype might enable an integer overflow attack.

For the other modules, esp those related to user interface development, the 
chance that an int-API is called with a value above 2G seems very low, even if 
they are operating internally on Qt containers. I might be missing some real 
world examples though where e.g. QGraphicsScene or a QTextDocument needs to 
operate on >2G items.


>> the hope is that  once we've progressed somewhat, we can enlist compiler 
>> support
>> by  globally enabling warnings such as -Wshorten-64-to-32


Would be great if we can do that for the modules above, at least, to avoid 
further deterioration. And once we have established an acceptable baseline, it 
would help a lot with doing the rest (or at least more) of the work for Qt 7.

Volker


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype

2022-09-05 Thread Alex Blasche

> -Original Message-
> From: Development  On Behalf Of
> Marc Mutz

I am taking the hot potato and spin this futher.


> You may have seen the qsizetype patches we've been pushing over the last
> months. This is to give my perspective on the issue and why I think it's
> needed:
> 
> First off, while I made QStringView qsizetype'd, because it was designed to 
> take
> input from STL containers[1], I was not involved in the decision to port even 
> just
> the Qt containers to qsizetype in 6.0.
> 
> [1] (it originally used size_t, which is what the STL containers use as 
> size_type
> but it was requested to use a signed type during review)
> 
> I do observe, though, that, starting with the premiss of a Qt container of 
> size > 2
> Gi, the current code base has a pretty high bug density in this area:

Absolutely this must be fixed. Those are plain bugs.
<...>

> The majority of changes are targeting the second goal. Bugs that fall in the 
> first
> category, like the ones listed above. are user-visible, usually get a commit 
> and
> Jira ticket of their own.
> 
> I don't really want to start a discussion on whether (owning) Qt containers
> _should_  support more than 2 Gi elements in the first place.
> From my pov, that decision has been made in the run-up to 6.0, and everyone
> who knows me knows that I don't much care about the owning Qt containers
> and their shortcomings.

Indeed. It was decided.

 
> I just want to raise awareness of the issue, which, really, is just an 
> incomplete
> port of the Qt containers to qsizetype in 6.0.
> 
> Over the last months, however, I've realized that "just the Qt containers"
> introduces a nasty API inconsistency in Qt: qsizetype in Qt containers and int
> elsewhere. This inconsistency is un-Qt-ish and creates problems for our users.
> 
> First up, it's unclear what a Qt container is: is QRegion a container?
> What does numRects() return when you setRects with a QList with > 2Gi
> elements? Where do we draw the line between Qt container and Qt non-
> container? See, in particular, QDataStream's inability to serialize 
> containers > 2
> Gi (and, in the case of QString, even > 1 Gi).
> 
> Second, the truncation that users face on c.size() (qsizetype) ->
> setFoo() (int) is using modulo arithmetic, not saturation arithmetic, so the 
> result
> is very unpredictable: INT_MAX + 1 elements turn into 0 elements, not INT_MAX,
> etc, creating a lot of opportunities for False Positives or Negatives, even 
> with
> modestly oversized containers (INT_MAX
> + å). The central Qt API guideline is to make APIs easy to use and hard
> to misuse. Both parts are violated here, especially since, while compilers
> regularly warn about signed/unsigned mismatches, they seldomly warn about
> narrowing (proof: Qt compiles). To fix the inconsistency means to port also 
> the
> "non-container" APIs to qsizetype, and handle/detect/defend against overflow
> _centrally_, so we don't inflict the issue on every user of the APIs.

I like the container.size() vs setFoo(int) use case as example to decide how 
far we want to spin this inconsistency further ahead throughout all of Qt . I 
totally agree with Marc's argument that it is not Qtish and causes pain to 
every user.  I guess we have three options:

1.) What I am wondering though is if we were to sink all the time into Qt to 
convert all API's, would Qt be blamed for all the continued porting effect 
throughout the years on this issue? After all we cannot do this for one single 
release in one go. It would probably find its way into each release over a 
couple of years. Does this help Qt in its goal to be easy to use? The 
compatibility layer does add its own layer of complexity.

2.) An alternative might be to make this change in one go for Qt 7. We would 
keep Qt 6.x on the status quo but start adding compatible replacement APi with 
an absolute change at 7.0 (ifdefs or typedefs come to mind). Users would only 
be burdened one time (though it being one BIG time effort). Such a change would 
be much simpler in Qt headers.

3.) On the other hand, we have had a lot of Qt 6 ports already and so far, it 
did not come up as a major issue in those ports. Could we simply accept this 
inconsistency? After all, would QSplitter ever need setWidget(qsizetype, 
QWidget*)? Under this scenario we might agree to convert all internal Qt uses 
of containers but leave the public API untouched. Of course some dedicated use 
case, where extra large files/chunks of data should be read, could get the 
treatment much earlier.

Personally, I cannot agree whether I like Option 2 or 3 more.
--
Alex


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


[Development] qsizetype

2022-09-01 Thread Marc Mutz
Hi,

You may have seen the qsizetype patches we've been pushing over the last 
months. This is to give my perspective on the issue and why I think it's 
needed:

First off, while I made QStringView qsizetype'd, because it was designed 
to take input from STL containers[1], I was not involved in the decision 
to port even just the Qt containers to qsizetype in 6.0.

[1] (it originally used size_t, which is what the STL containers use as 
size_type but it was requested to use a signed type during review)

I do observe, though, that, starting with the premiss of a Qt container 
of size > 2 Gi, the current code base has a pretty high bug density in 
this area:

- https://codereview.qt-project.org/c/qt/qtbase/+/422246
- https://codereview.qt-project.org/c/qt/qtbase/+/422989
- https://codereview.qt-project.org/c/qt/qtbase/+/421912
- https://codereview.qt-project.org/c/qt/qtbase/+/422999
- https://codereview.qt-project.org/c/qt/qtbase/+/403614
- https://codereview.qt-project.org/c/qt/qtbase/+/358114

The goal of the exercise was initially twofold: 1. to fix bugs, assuming 
the Qt containers hold more than 2 Gi of elements and 2. go back to a CI 
build where narrowing conversions cause compile errors. I believe we had 
that in Qt 5, at least on MSVC, and it was apparently disabled for 6.0.

The majority of changes are targeting the second goal. Bugs that fall in 
the first category, like the ones listed above. are user-visible, 
usually get a commit and Jira ticket of their own.

I don't really want to start a discussion on whether (owning) Qt 
containers _should_  support more than 2 Gi elements in the first place. 
From my pov, that decision has been made in the run-up to 6.0, and 
everyone who knows me knows that I don't much care about the owning Qt 
containers and their shortcomings.

I just want to raise awareness of the issue, which, really, is just an 
incomplete port of the Qt containers to qsizetype in 6.0.

Over the last months, however, I've realized that "just the Qt 
containers" introduces a nasty API inconsistency in Qt: qsizetype in Qt 
containers and int elsewhere. This inconsistency is un-Qt-ish and 
creates problems for our users.

First up, it's unclear what a Qt container is: is QRegion a container? 
What does numRects() return when you setRects with a QList with > 2Gi 
elements? Where do we draw the line between Qt container and Qt 
non-container? See, in particular, QDataStream's inability to serialize 
containers > 2 Gi (and, in the case of QString, even > 1 Gi).

Second, the truncation that users face on c.size() (qsizetype) -> 
setFoo() (int) is using modulo arithmetic, not saturation arithmetic, so 
the result is very unpredictable: INT_MAX + 1 elements turn into 0 
elements, not INT_MAX, etc, creating a lot of opportunities for False 
Positives or Negatives, even with modestly oversized containers (INT_MAX 
+ ε). The central Qt API guideline is to make APIs easy to use and hard 
to misuse. Both parts are violated here, especially since, while 
compilers regularly warn about signed/unsigned mismatches, they seldomly 
warn about narrowing (proof: Qt compiles). To fix the inconsistency 
means to port also the "non-container" APIs to qsizetype, and 
handle/detect/defend against overflow _centrally_, so we don't inflict 
the issue on every user of the APIs.

This should be non-issue for function parameters, because widening isn't 
an error. The value is less pronounced for return values, because 
widening those may cause narrowing in user code, but the hope is that 
once we've progressed somewhat, we can enlist compiler support by 
globally enabling warnings such as -Wshorten-64-to-32, even though, as 
the porting guide included in the 
https://bugreports.qt.io/browse/QTBUG-102461 epic describes, this will 
not catch manual casts, which is why, sadly, one needs to look at every 
int/uint manually.

Comments (and contributions) welcome.

Thanks,
Marc

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-09-01 Thread Scott Bloom
> On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development 
>  wrote:
>
> Hi,
>
> Il 27/08/20 02:46, Thiago Macieira ha scritto:
>> A QListView of 2 billion lines with where each line is a QString one
>> to 7 characters in length would be 2G * (24 + 32) = 96 GB of memory use.
>> QListWidget's overhead is much worse.
>
> This isn't accurate; QListView (with the default delegate) doesn't cache 
> data, and only fetches and shows what's visible in its viewport. So the 
> actual consumption is pretty much constant no matter how big is the 
> underlying model.

Correct. But the argument for QTextDocument still holds. If you have such huge 
files, I believe you want to load them using some custom editing component that 
mmaps the file and only loads the relevant parts on the fly. QTextDocument 
allows for rich text editing and requires to hold the content in memory in a 
QString because of that.

QTextDocument should be able to handle very large regular documents this 
without larger issues. Just as a comparison: All Harry Potter books together 
are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 
times larger document) into a QTextDocument should work without issues.

But a 5G set of machine generated data? I would believe you want a special 
class handling that and loading data on the fly, so you have some memory left 
for the rest of your system.

Having said that, I don’t mind changing our API to use qsizetype for 
QTextDocument, I just don’t believe you will find the class to be very snappy 
and usable with such large documents. The piece table itself should be able to 
handle it if you have enough RAM, but the default layout engine probably can’t 
(you could try using your own custom layout though).

Cheers,
Lars
___

To be clear, I am not expecting a QTextDocument as is to perform well under 
these loads (especially the 5gb files) in its normal usage form.

However, i can see someone extending qtextdocument to handle large files using 
one of many strategies.
Thanks
Scott
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-28 Thread Matthew Woehlke

On 25/08/2020 15.15, André Pönitz wrote:

On Mon, Aug 24, 2020 at 09:46:43AM +0200, Mathias Hasselmann wrote:

C++ also has a solution for that problem: [1]

https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/


AAA is a non-solution from the ivory tower.

It's a pain for human reviewers and tools operating on less then a full
translation unit.

(Almost) real-world example:

 for (auto i = foo.guess_stuff(); --i >= 0; )
   do_something(i);

Good? Bad?


Bad, *but*...

  for (auto const i : indexRange(foo.guess_stuff()) | range::reverse)
do_something(i)

...better! ;-) Also answers Scott's question.

(I'll assume `do_something` is actually something complicated enough 
that std::for_each or the like wouldn't be preferred. Also note that 
exact spelling of the above will depend on where you are getting range 
utilities; in my case, I'm lucky if I can use C++14, so I have my own 
implementations.)


BTW, AAA encourages duck typing, which may be a good or bad thing 
depending on your perspective.


--
Matthew
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Giuseppe D'Angelo via Development

Hi,

Il 27/08/20 16:47, Thiago Macieira ha scritto:

So, can someone take a look at what it would take to make the models use 64-
bit and come up with a proper guide for how to maintain code that compiles and
works on both Qt5 and Qt6? The latter is very important: if you can't easily
maintain for both, we just add to users' pain. It might be as simple as a
typedef added to QAbstractItemModel.


Finger in the wind: it's going to be painful. int-based APIs are used

- in virtuals in QAbstractItemModel (rowCount(), but also things like 
moveRows() which take int as parameter, so changing it is a straight API 
break)


- in Q(Persistent)ModelIndex

- in convenience subclasses like QStandardItemModel (and of course in 
the convenience views)


- in selection handling

- in the views' convenience handlers (e.g. QAbstractItemView::rowsInserted)


Long story short, I don't see this happening without also some 
configure-time switch for a "qaimsizetype" or somesuch.


My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Thiago Macieira
On Thursday, 27 August 2020 00:52:10 PDT Lars Knoll wrote:
> > On 27 Aug 2020, at 09:12, Philippe  wrote:

> >>> * QAbstractItemModel
> >>> * QModelIndex

> If I were do design these from scratch, I would certainly use qsizetype
> here, maybe even a qint64 (because one can handle those large data sets on
> 32bit systems as well with on demand loading of the data).
 
Good point. File sizes are qint64, so I agree any quantity that is tied to 
data, not to memory, should either use int or qint64.

So, can someone take a look at what it would take to make the models use 64-
bit and come up with a proper guide for how to maintain code that compiles and 
works on both Qt5 and Qt6? The latter is very important: if you can't easily 
maintain for both, we just add to users' pain. It might be as simple as a 
typedef added to QAbstractItemModel.

Like the q6sizetype typedef that was proposed and I never implemented.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Edward Welbourne
Il 25/08/20 07:49, Thiago Macieira ha scritto:
>> But how about models? This is an honest question. Does it make sense
>> for tables and lists that big? Note that an item*view* has a purpose
>> of being viewed, so how does one display such a huge list, tree or
>> table?

Giuseppe D'Angelo (25 August 2020 11:58) replied:
> Just another thought -- models may not necessarily used directly with
> views but as data sources for other business logic parts of the
> application (including but not necessarily limited to proxy models).
> Given the underlying data sources are 64-bit capable, such models
> should be as well.

and, at a more brutal level: in the late '90s, I ported a Geographic
Information System to cope with database files > 2GB in size (despite
the fact that the underlying OSen didn't; I had to use a virtual file
system), because we had customers whose datasets were big enough to need
it.  I consequently cannot believe, more than two decades later, that
2GB size limits on containers make sense.  Even if we didn't have a
use-case we could think of, some user out there almost certainly has, or
shall within Qt6's life-time have, such use-cases.

Coding to cope with them gracefully may, of course, require care: but we
must support them.

Eddy.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Lars Knoll
On 27 Aug 2020, at 10:45, Lars Knoll 
mailto:lars.kn...@qt.io>> wrote:

On 27 Aug 2020, at 08:32, Lars Knoll 
mailto:lars.kn...@qt.io>> wrote:

On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development 
mailto:development@qt-project.org>> wrote:

Hi,

Il 27/08/20 02:46, Thiago Macieira ha scritto:
A QListView of 2 billion lines with where each line is a QString one to 7
characters in length would be 2G * (24 + 32) = 96 GB of memory use.
QListWidget's overhead is much worse.

This isn't accurate; QListView (with the default delegate) doesn't cache data, 
and only fetches and shows what's visible in its viewport. So the actual 
consumption is pretty much constant no matter how big is the underlying model.

Correct. But the argument for QTextDocument still holds. If you have such huge 
files, I believe you want to load them using some custom editing component that 
mmaps the file and only loads the relevant parts on the fly. QTextDocument 
allows for rich text editing and requires to hold the content in memory in a 
QString because of that.

QTextDocument should be able to handle very large regular documents this 
without larger issues. Just as a comparison: All Harry Potter books together 
are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 
times larger document) into a QTextDocument should work without issues.

But a 5G set of machine generated data? I would believe you want a special 
class handling that and loading data on the fly, so you have some memory left 
for the rest of your system.

Having said that, I don’t mind changing our API to use qsizetype for 
QTextDocument, I just don’t believe you will find the class to be very snappy 
and usable with such large documents. The piece table itself should be able to 
handle it if you have enough RAM, but the default layout engine probably can’t 
(you could try using your own custom layout though).

I’ll give it a try and let’s see. There are very few to no virtual functions 
that would be affected, so changing the classes to use qsizetype might be 
worthwhile.

Did a quick trial, and I don’t think it’s worth it. It’s quite a lot of work 
and will break user code in some places for very little gain. Little gain not 
because there aren’t use cases that need to handle very large documents, but 
because QTextEdit and QTextDocument are probably not the right classes to do so 
given that they load everything into memory at once.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Lars Knoll
On 27 Aug 2020, at 08:32, Lars Knoll 
mailto:lars.kn...@qt.io>> wrote:

On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development 
mailto:development@qt-project.org>> wrote:

Hi,

Il 27/08/20 02:46, Thiago Macieira ha scritto:
A QListView of 2 billion lines with where each line is a QString one to 7
characters in length would be 2G * (24 + 32) = 96 GB of memory use.
QListWidget's overhead is much worse.

This isn't accurate; QListView (with the default delegate) doesn't cache data, 
and only fetches and shows what's visible in its viewport. So the actual 
consumption is pretty much constant no matter how big is the underlying model.

Correct. But the argument for QTextDocument still holds. If you have such huge 
files, I believe you want to load them using some custom editing component that 
mmaps the file and only loads the relevant parts on the fly. QTextDocument 
allows for rich text editing and requires to hold the content in memory in a 
QString because of that.

QTextDocument should be able to handle very large regular documents this 
without larger issues. Just as a comparison: All Harry Potter books together 
are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 
times larger document) into a QTextDocument should work without issues.

But a 5G set of machine generated data? I would believe you want a special 
class handling that and loading data on the fly, so you have some memory left 
for the rest of your system.

Having said that, I don’t mind changing our API to use qsizetype for 
QTextDocument, I just don’t believe you will find the class to be very snappy 
and usable with such large documents. The piece table itself should be able to 
handle it if you have enough RAM, but the default layout engine probably can’t 
(you could try using your own custom layout though).

I’ll give it a try and let’s see. There are very few to no virtual functions 
that would be affected, so changing the classes to use qsizetype might be 
worthwhile.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Lars Knoll

> On 27 Aug 2020, at 09:12, Philippe  wrote:
> 
> On Wed, 26 Aug 2020 06:36:27 +
> Lars Knoll  wrote:
> 
>>> * QAbstractItemModel
>>> * QModelIndex
>> 
>> I don’t think we should port these to use qsizetype.
> 
> If you mean, the effort would be too big, or there is lack of resources
> to do it, or too many API breaks for Qt 5 to 6, then I can agree.
> 
> But if you mean there is no usage need, I disagree.

If I were do design these from scratch, I would certainly use qsizetype here, 
maybe even a qint64 (because one can handle those large data sets on 32bit 
systems as well with on demand loading of the data).

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Philippe
On Wed, 26 Aug 2020 06:36:27 +
Lars Knoll  wrote:

> > * QAbstractItemModel
> > * QModelIndex
> 
> I don’t think we should port these to use qsizetype.

If you mean, the effort would be too big, or there is lack of resources
to do it, or too many API breaks for Qt 5 to 6, then I can agree.

But if you mean there is no usage need, I disagree.

Philippe

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Lars Knoll
> On 27 Aug 2020, at 08:04, Giuseppe D'Angelo via Development 
>  wrote:
> 
> Hi,
> 
> Il 27/08/20 02:46, Thiago Macieira ha scritto:
>> A QListView of 2 billion lines with where each line is a QString one to 7
>> characters in length would be 2G * (24 + 32) = 96 GB of memory use.
>> QListWidget's overhead is much worse.
> 
> This isn't accurate; QListView (with the default delegate) doesn't cache 
> data, and only fetches and shows what's visible in its viewport. So the 
> actual consumption is pretty much constant no matter how big is the 
> underlying model.

Correct. But the argument for QTextDocument still holds. If you have such huge 
files, I believe you want to load them using some custom editing component that 
mmaps the file and only loads the relevant parts on the fly. QTextDocument 
allows for rich text editing and requires to hold the content in memory in a 
QString because of that. 

QTextDocument should be able to handle very large regular documents this 
without larger issues. Just as a comparison: All Harry Potter books together 
are around 1.1M words, ie. around 10M characters. Loading that (or even a 10 
times larger document) into a QTextDocument should work without issues.

But a 5G set of machine generated data? I would believe you want a special 
class handling that and loading data on the fly, so you have some memory left 
for the rest of your system.

Having said that, I don’t mind changing our API to use qsizetype for 
QTextDocument, I just don’t believe you will find the class to be very snappy 
and usable with such large documents. The piece table itself should be able to 
handle it if you have enough RAM, but the default layout engine probably can’t 
(you could try using your own custom layout though).

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-27 Thread Giuseppe D'Angelo via Development

Hi,

Il 27/08/20 02:46, Thiago Macieira ha scritto:

A QListView of 2 billion lines with where each line is a QString one to 7
characters in length would be 2G * (24 + 32) = 96 GB of memory use.
QListWidget's overhead is much worse.


This isn't accurate; QListView (with the default delegate) doesn't cache 
data, and only fetches and shows what's visible in its viewport. So the 
actual consumption is pretty much constant no matter how big is the 
underlying model.


My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Thiago Macieira
On Wednesday, 26 August 2020 17:23:16 PDT Lisandro Damián Nicanor Pérez Meyer 
wrote:
> > Disagree here.  There is good reason many in my industry (Electronic
> > design automation) use the Scintilla editor widget inside Qt apps.
> > Specifically to handle extremely large, in both line count and column
> > count, files.
> > 
> > Many of the text files we work with, are generated library definitions for
> > silicon definitions.  A 5+ gb file is not uncommon.  Or files with a truth
> > table definition with 5000-1000 columns.  I kid you not.
> > 
> > Its one thing to not handle it, and say "too big to open" its another to
> > attempt to and crash because the internal integers cant handle it.
> > 
> > Scintilla handles these massive files relatively easily.
> 
> Yes, this files terms to be huge and are a valid case after all.

Not disagreeing that huge files are valid.

The question is whether the API needs to directly map to such huge files. When 
your data set becomes very big, the layer above those display classes should 
consider tiling or windowing the data.

Let's take a simplest case: a 2 billion character QPlainTextEdit. The text 
alone is 4 GB in memory. QPlainTextEdit (through QTextDocument) will also 
parse all the lines in the text and create blocks for them, whether they're 
visible or not. So not only will this require a lot of memory, it will also be 
very slow.

A QListView of 2 billion lines with where each line is a QString one to 7 
characters in length would be 2G * (24 + 32) = 96 GB of memory use. 
QListWidget's overhead is much worse.

So, I don't dispute the need to *index* very large data sets. I do dispute the 
need for a single Qt graphical class to keep the entire data set parsed, in 
memory.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Lisandro Damián Nicanor Pérez Meyer
Hi!

El mié., 26 ago. 2020 16:59, Scott Bloom  escribió:

> From: Development  On Behalf Of Ville
> Voutilainen
> Sent: Wednesday, August 26, 2020 12:08 AM
> To: Lars Knoll 
> Cc: Qt development mailing list 
> Subject: Re: [Development] qsizetype and classes working with QStrings or
> QList
>
> On Wed, 26 Aug 2020 at 09:39, Lars Knoll  wrote:
> > > QtGui:
> > > * QTextCursor
> > > * QTextDocument (find offset, character{At,Count})
> > > * QTextLayout
> > > * QValidator and subclasses (validate offset)
> >
> > These here are questionable. Editing a text file with more than 2G
> characters? Sounds unlikely.
>
> Disagree here.  There is good reason many in my industry (Electronic
> design automation) use the Scintilla editor widget inside Qt apps.
> Specifically to handle extremely large, in both line count and column
> count, files.
>
> Many of the text files we work with, are generated library definitions for
> silicon definitions.  A 5+ gb file is not uncommon.  Or files with a truth
> table definition with 5000-1000 columns.  I kid you not.
>
> Its one thing to not handle it, and say "too big to open" its another to
> attempt to and crash because the internal integers cant handle it.
>
> Scintilla handles these massive files relatively easily.
>

Yes, this files terms to be huge and are a valid case after all.

>
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Scott Bloom
From: Development  On Behalf Of Ville 
Voutilainen
Sent: Wednesday, August 26, 2020 12:08 AM
To: Lars Knoll 
Cc: Qt development mailing list 
Subject: Re: [Development] qsizetype and classes working with QStrings or QList

On Wed, 26 Aug 2020 at 09:39, Lars Knoll  wrote:
> > QtGui:
> > * QTextCursor
> > * QTextDocument (find offset, character{At,Count})
> > * QTextLayout
> > * QValidator and subclasses (validate offset)
>
> These here are questionable. Editing a text file with more than 2G 
> characters? Sounds unlikely.

Disagree here.  There is good reason many in my industry (Electronic design 
automation) use the Scintilla editor widget inside Qt apps.  Specifically to 
handle extremely large, in both line count and column count, files.

Many of the text files we work with, are generated library definitions for 
silicon definitions.  A 5+ gb file is not uncommon.  Or files with a truth 
table definition with 5000-1000 columns.  I kid you not.

Its one thing to not handle it, and say "too big to open" its another to 
attempt to and crash because the internal integers cant handle it.

Scintilla handles these massive files relatively easily.

Scott
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Giuseppe D'Angelo via Development

Il 25/08/20 21:05, André Pönitz ha scritto:

why I wanted a configure time switch to choose the size of qsizetype).

This doesn't really help if Qt comes with your distribution or even with
the Qt installers.



What I meant is that it would be a porting aid towards Qt 6, rather than 
a switch to support "forever". Qt 7 (or 6.N for a big enough value of N) 
would use a size_t-sized qsizetype exclusively.


My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Marco Bubke
A new Document API where you could memory map the file and save the changes as 
a log on top would be nice. For lines you have to read the file but you don't 
have to hold it completely in memory. An other advantage would be that your 
document would be a binary reflection of the file. But I think this would be 
not a replacement of QDocument.

-Original Message-
From: Development  On Behalf Of Lars Knoll
Sent: Wednesday, August 26, 2020 9:23 AM
To: Ville Voutilainen 
Cc: Qt development mailing list 
Subject: Re: [Development] qsizetype and classes working with QStrings or QList


> On 26 Aug 2020, at 09:07, Ville Voutilainen  
> wrote:
> 
> On Wed, 26 Aug 2020 at 09:39, Lars Knoll  wrote:
>>> QtGui:
>>> * QTextCursor
>>> * QTextDocument (find offset, character{At,Count})
>>> * QTextLayout
>>> * QValidator and subclasses (validate offset)
>> 
>> These here are questionable. Editing a text file with more than 2G 
>> characters? Sounds unlikely.
> 
> Thou Shalt Not Write Debug Log Viewers Or Hex Editors With Qt. :)

Porting that API might not be too much of a problem, I don’t think we have 
virtual methods in there taking offsets.

But QTextDocument is probably not the right data structure for a hex editor or 
a log viewer anyway (why would you store a hex in clear text instead of 
decoding on the fly???).

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Lars Knoll

> On 26 Aug 2020, at 09:07, Ville Voutilainen  
> wrote:
> 
> On Wed, 26 Aug 2020 at 09:39, Lars Knoll  wrote:
>>> QtGui:
>>> * QTextCursor
>>> * QTextDocument (find offset, character{At,Count})
>>> * QTextLayout
>>> * QValidator and subclasses (validate offset)
>> 
>> These here are questionable. Editing a text file with more than 2G 
>> characters? Sounds unlikely.
> 
> Thou Shalt Not Write Debug Log Viewers Or Hex Editors With Qt. :)

Porting that API might not be too much of a problem, I don’t think we have 
virtual methods in there taking offsets.

But QTextDocument is probably not the right data structure for a hex editor or 
a log viewer anyway (why would you store a hex in clear text instead of 
decoding on the fly???).

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Ville Voutilainen
On Wed, 26 Aug 2020 at 09:39, Lars Knoll  wrote:
> > QtGui:
> > * QTextCursor
> > * QTextDocument (find offset, character{At,Count})
> > * QTextLayout
> > * QValidator and subclasses (validate offset)
>
> These here are questionable. Editing a text file with more than 2G 
> characters? Sounds unlikely.

Thou Shalt Not Write Debug Log Viewers Or Hex Editors With Qt. :)
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-26 Thread Lars Knoll
Going back to the original question here:

> On 23 Aug 2020, at 16:06, Marcel Krems  wrote:
> 
> Hi,
> 
> since QString, QList, etc. are using qsizetype for indexing- and 
> size-operations.
> What is the plan with classes working with aforementioned container classes 
> which are still using int in their interfaces?
> If they keep using int there could be a lot of warnings like this one:
> warning: implicit conversion loses integer precision: 'qsizetype' (aka 'long 
> long') to 'int' [-Wshorten-64-to-32]
> Or you have to plaster your code with casts. E.g. every time you pass an 
> index of your container to your model class.
> 
> Some classes which are probably affected:
> QtCore:
> * QAbstractItemModel and subclasses (using QList or std::vector as data 
> storage)
> * QModelIndex

I don’t think we should port these to use qsizetype.

> * QRegularExpression (match offset)
> * QStringMatcher
> * QSyntaxHighlighter
> * QTextBoundaryFinder
> * QXmlString::size

I think we should still fix these, as they are in low level string processing 
classes

> QtGui:
> * QTextCursor
> * QTextDocument (find offset, character{At,Count})
> * QTextLayout
> * QValidator and subclasses (validate offset)

These here are questionable. Editing a text file with more than 2G characters? 
Sounds unlikely.
> 
> QtWidgets:
> * QAbstractItemView and subclasses
> * QLineEdit

Neither should we touch these IMO.

Cheers,
Lars

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Scott Bloom
Or

for( auto ii = 0; ii < std::vector.size(); ++ii )
{
}

since there is no suffix for "size_t" and the size of size_t will depend on 64 
vs 32 bits whats the best way to AAA the index iterator?

Scott


-Original Message-
From: Development [mailto:development-boun...@qt-project.org] On Behalf Of 
André Pönitz
Sent: Tuesday, August 25, 2020 12:15
To: Mathias Hasselmann 
Cc: development@qt-project.org
Subject: Re: [Development] qsizetype and classes working with QStrings or QList

On Mon, Aug 24, 2020 at 09:46:43AM +0200, Mathias Hasselmann wrote:
>C++ also has a solution for that problem: [1]
>
> https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-al
> ways-auto/

AAA is a non-solution from the ivory tower.

It's a pain for human reviewers and tools operating on less then a full 
translation unit.

(Almost) real-world example:

for (auto i = foo.guess_stuff(); --i >= 0; )
  do_something(i);

Good? Bad? 

Andre'

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread André Pönitz
On Mon, Aug 24, 2020 at 09:46:43AM +0200, Mathias Hasselmann wrote:
>C++ also has a solution for that problem: [1]
>
> https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/

AAA is a non-solution from the ivory tower.

It's a pain for human reviewers and tools operating on less then a full
translation unit.

(Almost) real-world example:

for (auto i = foo.guess_stuff(); --i >= 0; )
  do_something(i);

Good? Bad? 

Andre'

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread André Pönitz
On Mon, Aug 24, 2020 at 09:26:54AM +0200, Giuseppe D'Angelo via Development 
wrote:
> On 23/08/2020 16:06, Marcel Krems wrote:
> > If they keep using int there could be a lot of warnings like this one:
> > warning: implicit conversion loses integer precision: 'qsizetype' (aka
> > 'long long') to 'int' [-Wshorten-64-to-32]
> 
> I'm afraid that these warnings will be all over the place anyhow. Just how
> much code has been written against int-based APIs?

Depends on the nature of the application, but in a typical Qt application
where GUI plays a major role, I would expect 'int' all over the place,

As an example, Qt Creator code has 48715 hits for '\bint\b', 1191 '\bsize_t',
most of the latter in 3rd party or test code.

> why I wanted a configure time switch to choose the size of qsizetype).

This doesn't really help if Qt comes with your distribution or even with
the Qt installers.

> But anyhow: yes, as a consistency factor, as many APIs as possible should be
> ported to qsizetype.

For us it was consistent so far. It is getting a pain only now. 

Andre'
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Matthew Woehlke

On 25/08/2020 02.22, Mathias Hasselmann wrote:
I'd really enjoy implementing list models without having 
to litter static casts all over the place. Well, but seems that chance 
got missed once again with qsizetype still being signed.


That's intentional. There are problems with unsigned index types, not 
least of which is they aren't practically useful anyway. Some consider 
it a mistake that size_t is unsigned, and indeed, ssize_t is starting to 
be retrofitted in some places (e.g. `container.ssize()`). If we ever get 
an STL2 it will likely also have signed index types.


Signed index types allow things like indexing-from-end and make tests 
for invalid indices far less painful. Seriously, which of these do you 
prefer?


  if (s.find(ch) < 0)   // option 1
  if (s.find(ch) != std::string::npos)  // option 2

--
Matthew
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Matthew Woehlke

On 25/08/2020 05.58, Giuseppe D'Angelo via Development wrote:

Il 25/08/20 07:49, Thiago Macieira ha scritto:

But how about models? This is an honest question. Does it make sense for
tables and lists that big? Note that an item*view*  has a purpose of 
being viewed, so how does one display such a huge list, tree or

table?


Just another thought -- models may not necessarily used directly with 
views but as data sources for other business logic parts of the 
application (including but not necessarily limited to proxy models). 
Given the underlying data sources are 64-bit capable, such models should 
be as well.


Indeed; this is sort of the approach one of my applications takes. 
Although we do usually end up with a view on top of the model (often 
after a few levels of indirection¹), we also use models (i.e. 
QAbstractItemModel) to drive things like graphical displays. (It would 
take some words to explain our actual use case, but think of using a 
model as the source for a scatter plot and you sort-of get the idea.)


That said, I suspect you would run into fairly severe performance 
problems trying to use a model with more than 2^31 rows...


(¹ We generally have one model that presents the data in an abstract 
fashion, using data roles rather than columns as field identifiers. This 
separates what the data *is* from how it's presented. This may get fed 
into one or more sort/filter proxy models before being fed into a 
"representation" model, which folds, spindles and mutilates the 
fields-as-data-roles model into a fields-as-columns model suitable for 
being fed into e.g. QTreeView. One nice feature is that the 
representation can accept a list of what fields you want to have in the 
view.)


--
Matthew
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Matthew Woehlke

On 25/08/2020 01.24, Philippe wrote:

But then there would be the need to make QAbstractSlider be able to
handle 64 bit quantities too.


Well, since you mentioned it:

https://github.com/Kitware/qtextensions/blob/master/widgets/qtDoubleSlider.h

No, that isn't a two-headed slider, it's a slider that works on 
`double`. Doesn't derive from QAbstractSlider because it can't.


These are hardly uncommon; GIMP has tons of examples, for instance, in 
many, many filter options dialogs (although the widget they use isn't 
entirely like QSlider). Yes, you can sort-of emulate them with integers 
under the hood, but doing so adds an annoying level of dissonance to the 
code.


Q[Abstract]Slider being limited to `int` is indeed an irritation. We 
have QDoubleSpinbox, why not QDoubleSlider?


--
Matthew
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Thiago Macieira
On Monday, 24 August 2020 23:11:17 PDT Philippe wrote:
> > This is an honest question. Does it make sense for
> > tables and lists that big? Note that an item *view* has a purpose of being
> > viewed, so how does one display such a huge list, tree or table?
> 
> I have a concrete case: in the audio domain, it's common to have audio
> files with more than 2 billions audio samples (long recordings).
> I had the wish to display to the user the level of individual audio
> samples, for analysis purposes.

Thank you, that is indeed a valid use-case.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Giuseppe D'Angelo via Development

Il 25/08/20 07:49, Thiago Macieira ha scritto:


But how about models? This is an honest question. Does it make sense for
tables and lists that big? Note that an item*view*  has a purpose of being
viewed, so how does one display such a huge list, tree or table?


Just another thought -- models may not necessarily used directly with 
views but as data sources for other business logic parts of the 
application (including but not necessarily limited to proxy models). 
Given the underlying data sources are 64-bit capable, such models should 
be as well.


My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: Firma crittografica S/MIME
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Иван Комиссаров
Well, yes, when displaying huge databases, for example. This use-case pops up 
quite often from my experience. It is rare to contain more than 2^31 elements, 
yes, but one has to be aware of that case and support it somehow which leads to 
overcompicated code that does «paging», for example. And paging splits the data 
which is poor UX too.
Of course, to make it usable, you have to add filters on top of that, but you’d 
probably need them anyway while paging is just an overcomplicating with no gain.
Note that the model might not contain the data but retrieve it upon request 
from the view, e.g. store some cache.
An example of such a huge model can be the hex model that displays the contents 
of the file. Normally, files are not that big, but they are theoretically 
possible.
Of course, one can say that «nobody needs» it, that this is corner use-case… 
Well, libraries have to support corner use-cases, haven’t they?

Ivan

> 25 авг. 2020 г., в 01:09, Thiago Macieira  
> написал(а):
> 
> On Monday, 24 August 2020 15:10:24 PDT Иван Комиссаров wrote:
>> It would be nice if QAbstractItemModel will support qsizetype instead of
>> int, but I do not see how this is possible considering the fact that
>> rowCount/columnCount return int. I suppose, it is not very hard to patch
>> QModelIndex, but what to do with virtual functions? The user code will
>> break. The only solution I see is to add QAbstractItemModelV2 with the
>> «wide» interface and provide a proxy for the old one.
> 
> Do we need models with more than 2 billion rows or columns?
> 
> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>  Software Architect - Intel DPG Cloud Engineering
> 
> 
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Mathias Hasselmann


Am 25.08.2020 um 01:09 schrieb Thiago Macieira:

On Monday, 24 August 2020 15:10:24 PDT Иван Комиссаров wrote:

It would be nice if QAbstractItemModel will support qsizetype instead of
int, but I do not see how this is possible considering the fact that
rowCount/columnCount return int. I suppose, it is not very hard to patch
QModelIndex, but what to do with virtual functions? The user code will
break. The only solution I see is to add QAbstractItemModelV2 with the
«wide» interface and provide a proxy for the old one.

Do we need models with more than 2 billion rows or columns?
Not really, but I'd really enjoy implementing list models without having 
to litter static casts all over the place. Well, but seems that chance 
got missed once again with qsizetype still being signed. So what.


Ciao
Mathias
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-25 Thread Philippe
> This is an honest question. Does it make sense for 
> tables and lists that big? Note that an item *view* has a purpose of being 
> viewed, so how does one display such a huge list, tree or table?

I have a concrete case: in the audio domain, it's common to have audio
files with more than 2 billions audio samples (long recordings).
I had the wish to display to the user the level of individual audio
samples, for analysis purposes.

Philippe

On Mon, 24 Aug 2020 22:49:42 -0700
Thiago Macieira  wrote:

> On Monday, 24 August 2020 22:24:52 PDT Philippe wrote:
> > > Do we need models with more than 2 billion rows or columns?
> > 
> > More than we need in-memory containers with more than 2 billion entries,
> > no?
> 
> More? We see a lot of data processing bumping up to gigabyte levels. 
> Containers with more than 2 billion items are rare, but I see it happening 
> for 
> QByteArray and QString at least.
> 
> But how about models? This is an honest question. Does it make sense for 
> tables and lists that big? Note that an item *view* has a purpose of being 
> viewed, so how does one display such a huge list, tree or table?
> 
> > For instance, one could wish to display in a list view, the contents of
> > a file with more than 2 billions "entries".
> > 
> > But then there would be the need to make QAbstractSlider be able to
> > handle 64 bit quantities too.
> 
> How does that make a good UX?
> 
> On an 8k resolution (7680 × 4320), each vertical pixel would represent half a 
> million entries.
> 
> I'm not questioning the existence of data sets of more than 2 billion 
> entries. 
> Those exist. I'm asking how one can make a UX that requires more than 2 
> billion rows or columns.
> 
> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel DPG Cloud Engineering
> 
> 
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Thiago Macieira
On Monday, 24 August 2020 22:24:52 PDT Philippe wrote:
> > Do we need models with more than 2 billion rows or columns?
> 
> More than we need in-memory containers with more than 2 billion entries,
> no?

More? We see a lot of data processing bumping up to gigabyte levels. 
Containers with more than 2 billion items are rare, but I see it happening for 
QByteArray and QString at least.

But how about models? This is an honest question. Does it make sense for 
tables and lists that big? Note that an item *view* has a purpose of being 
viewed, so how does one display such a huge list, tree or table?

> For instance, one could wish to display in a list view, the contents of
> a file with more than 2 billions "entries".
> 
> But then there would be the need to make QAbstractSlider be able to
> handle 64 bit quantities too.

How does that make a good UX?

On an 8k resolution (7680 × 4320), each vertical pixel would represent half a 
million entries.

I'm not questioning the existence of data sets of more than 2 billion entries. 
Those exist. I'm asking how one can make a UX that requires more than 2 
billion rows or columns.

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Philippe
> Do we need models with more than 2 billion rows or columns?

More than we need in-memory containers with more than 2 billion entries,
no?

For instance, one could wish to display in a list view, the contents of
a file with more than 2 billions "entries".

But then there would be the need to make QAbstractSlider be able to
handle 64 bit quantities too.

Philippe

On Mon, 24 Aug 2020 16:09:57 -0700
Thiago Macieira  wrote:

> On Monday, 24 August 2020 15:10:24 PDT  ?? wrote:
> > It would be nice if QAbstractItemModel will support qsizetype instead of
> > int, but I do not see how this is possible considering the fact that
> > rowCount/columnCount return int. I suppose, it is not very hard to patch
> > QModelIndex, but what to do with virtual functions? The user code will
> > break. The only solution I see is to add QAbstractItemModelV2 with the
> > «wide» interface and provide a proxy for the old one.
> 
> Do we need models with more than 2 billion rows or columns?
> 
> -- 
> Thiago Macieira - thiago.macieira (AT) intel.com
>   Software Architect - Intel DPG Cloud Engineering
> 
> 
> 
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development


___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Thiago Macieira
On Monday, 24 August 2020 15:10:24 PDT Иван Комиссаров wrote:
> It would be nice if QAbstractItemModel will support qsizetype instead of
> int, but I do not see how this is possible considering the fact that
> rowCount/columnCount return int. I suppose, it is not very hard to patch
> QModelIndex, but what to do with virtual functions? The user code will
> break. The only solution I see is to add QAbstractItemModelV2 with the
> «wide» interface and provide a proxy for the old one.

Do we need models with more than 2 billion rows or columns?

-- 
Thiago Macieira - thiago.macieira (AT) intel.com
  Software Architect - Intel DPG Cloud Engineering



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Иван Комиссаров
It would be nice if QAbstractItemModel will support qsizetype instead of int, 
but I do not see how this is possible considering the fact that 
rowCount/columnCount return int. I suppose, it is not very hard to patch 
QModelIndex, but what to do with virtual functions? The user code will break.
The only solution I see is to add QAbstractItemModelV2 with the «wide» 
interface and provide a proxy for the old one.

Ivan

> 23 авг. 2020 г., в 16:06, Marcel Krems  написал(а):
> 
> Hi,
> 
> since QString, QList, etc. are using qsizetype for indexing- and 
> size-operations.
> What is the plan with classes working with aforementioned container classes 
> which are still using int in their interfaces?
> If they keep using int there could be a lot of warnings like this one:
> warning: implicit conversion loses integer precision: 'qsizetype' (aka 'long 
> long') to 'int' [-Wshorten-64-to-32]
> Or you have to plaster your code with casts. E.g. every time you pass an 
> index of your container to your model class.
> 
> Some classes which are probably affected:
> QtCore:
> * QAbstractItemModel and subclasses (using QList or std::vector as data 
> storage)
> * QModelIndex
> * QRegularExpression (match offset)
> * QStringMatcher
> * QSyntaxHighlighter
> * QTextBoundaryFinder
> * QXmlString::size
> 
> QtGui:
> * QTextCursor
> * QTextDocument (find offset, character{At,Count})
> * QTextLayout
> * QValidator and subclasses (validate offset)
> 
> QtWidgets:
> * QAbstractItemView and subclasses
> * QLineEdit
> 
> 
> Kind regards,
> Marcel
> ___
> Development mailing list
> Development@qt-project.org
> https://lists.qt-project.org/listinfo/development

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Marcel Krems

https://bugreports.qt.io/browse/QTBUG-86224


On 24.08.2020 09:26, Giuseppe D'Angelo via Development wrote:

On 23/08/2020 16:06, Marcel Krems wrote:

If they keep using int there could be a lot of warnings like this one:
warning: implicit conversion loses integer precision: 'qsizetype' (aka
'long long') to 'int' [-Wshorten-64-to-32]


I'm afraid that these warnings will be all over the place anyhow. Just 
how much code has been written against int-based APIs? (For the 
record, that's why I wanted a configure time switch to choose the size 
of qsizetype).


But anyhow: yes, as a consistency factor, as many APIs as possible 
should be ported to qsizetype. Could you please file a bug report?


Thanks,

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development



___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Giuseppe D'Angelo via Development

On 24/08/2020 11:17, Mathias Hasselmann wrote:

Do you have examples showing verifiable evidence, or do you share a feeling?


There has been quite a flurry of patches into Qt fixing the generated 
warnings (shortening 64-to-32, using "%d" in printf, and the like). I 
don't have a way to list them all, I guess this is as good as it gets:



https://codereview.qt-project.org/q/message:qsizetype


My 2 c,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: S/MIME Cryptographic Signature
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Ville Voutilainen
On Mon, 24 Aug 2020 at 15:37, Christian Kandeler
 wrote:
> > I don't have verifiable evidence examples, but the gist of it is this:
> >
> > ConcreteType x = foo(); // this detects API breaks right here, right now
> > ...
> > ...
> > ...
> > some_use_of(x);
> >
> > With AAA, this might become
> >
> > auto x = foo(); // this always compiles
> > ...
> > ...
> > ...
> > some_use_of(x); // you may detect an API break here, or somewhere deep
> > inside some_use_of
> >
> > I wonder where the verifiable evidence is that AAA works at scale.
>
> What about:
>
> some_use_of(foo());
>
> Are you suggesting that this is an anti-pattern?

I fail to see where I might have suggested such a thing. That sort of
uses are unaffected by whether
an API user does or does not buy into AAA. They certainly are another
way to detect (or suffer from)
an API break, and trying to use more auto doesn't solve it.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Christian Kandeler
On Mon, 24 Aug 2020 14:45:19 +0300
Ville Voutilainen  wrote:

> On Mon, 24 Aug 2020 at 12:17, Mathias Hasselmann
>  wrote:
> > >> C++ also has a solution for that problem:  
> > >> https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/
> > > That non-solution is terrible. The very reason for not using deduced
> > > types is to detect API breaks loudly.
> > > The warning does that in dulcet tones, not as loudly as some might
> > > wish because the conversion is implicit.
> > > Buying the AAA snake oil can move the problem elsewhere for a while,
> > > but it's not helpful; it's partially
> > > hiding an API break, and it's unlikely that you want that to continue;
> > > the manifestations of the API break
> > > are going to appear further away from the spots where they could be
> > > first detected.
> >
> > Do you have examples showing verifiable evidence, or do you share a feeling?
> 
> I don't have verifiable evidence examples, but the gist of it is this:
> 
> ConcreteType x = foo(); // this detects API breaks right here, right now
> ...
> ...
> ...
> some_use_of(x);
> 
> With AAA, this might become
> 
> auto x = foo(); // this always compiles
> ...
> ...
> ...
> some_use_of(x); // you may detect an API break here, or somewhere deep
> inside some_use_of
> 
> I wonder where the verifiable evidence is that AAA works at scale.

What about:

some_use_of(foo());

Are you suggesting that this is an anti-pattern?


Christian
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Ville Voutilainen
On Mon, 24 Aug 2020 at 12:17, Mathias Hasselmann
 wrote:
> >> C++ also has a solution for that problem:  
> >> https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/
> > That non-solution is terrible. The very reason for not using deduced
> > types is to detect API breaks loudly.
> > The warning does that in dulcet tones, not as loudly as some might
> > wish because the conversion is implicit.
> > Buying the AAA snake oil can move the problem elsewhere for a while,
> > but it's not helpful; it's partially
> > hiding an API break, and it's unlikely that you want that to continue;
> > the manifestations of the API break
> > are going to appear further away from the spots where they could be
> > first detected.
>
> Do you have examples showing verifiable evidence, or do you share a feeling?

I don't have verifiable evidence examples, but the gist of it is this:

ConcreteType x = foo(); // this detects API breaks right here, right now
...
...
...
some_use_of(x);

With AAA, this might become

auto x = foo(); // this always compiles
...
...
...
some_use_of(x); // you may detect an API break here, or somewhere deep
inside some_use_of

I wonder where the verifiable evidence is that AAA works at scale.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Mathias Hasselmann


Am 24.08.2020 um 11:04 schrieb Ville Voutilainen:

On Mon, 24 Aug 2020 at 10:50, Mathias Hasselmann
 wrote:

Am 24.08.2020 um 09:26 schrieb Giuseppe D'Angelo via Development:

On 23/08/2020 16:06, Marcel Krems wrote:

If they keep using int there could be a lot of warnings like this one:
warning: implicit conversion loses integer precision: 'qsizetype' (aka
'long long') to 'int' [-Wshorten-64-to-32]


I'm afraid that these warnings will be all over the place anyhow. Just how much 
code has been written against int-based APIs? (For the record, that's why I 
wanted a configure time switch to choose the size of qsizetype).

But anyhow: yes, as a consistency factor, as many APIs as possible should be 
ported to qsizetype. Could you please file a bug report?

C++ also has a solution for that problem:  
https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/

That non-solution is terrible. The very reason for not using deduced
types is to detect API breaks loudly.
The warning does that in dulcet tones, not as loudly as some might
wish because the conversion is implicit.
Buying the AAA snake oil can move the problem elsewhere for a while,
but it's not helpful; it's partially
hiding an API break, and it's unlikely that you want that to continue;
the manifestations of the API break
are going to appear further away from the spots where they could be
first detected.


Do you have examples showing verifiable evidence, or do you share a feeling?

Ciao
Mathias

___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Ville Voutilainen
On Mon, 24 Aug 2020 at 10:50, Mathias Hasselmann
 wrote:
>
> Am 24.08.2020 um 09:26 schrieb Giuseppe D'Angelo via Development:
>
> On 23/08/2020 16:06, Marcel Krems wrote:
>
> If they keep using int there could be a lot of warnings like this one:
> warning: implicit conversion loses integer precision: 'qsizetype' (aka
> 'long long') to 'int' [-Wshorten-64-to-32]
>
>
> I'm afraid that these warnings will be all over the place anyhow. Just how 
> much code has been written against int-based APIs? (For the record, that's 
> why I wanted a configure time switch to choose the size of qsizetype).
>
> But anyhow: yes, as a consistency factor, as many APIs as possible should be 
> ported to qsizetype. Could you please file a bug report?
>
> C++ also has a solution for that problem:  
> https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/

That non-solution is terrible. The very reason for not using deduced
types is to detect API breaks loudly.
The warning does that in dulcet tones, not as loudly as some might
wish because the conversion is implicit.
Buying the AAA snake oil can move the problem elsewhere for a while,
but it's not helpful; it's partially
hiding an API break, and it's unlikely that you want that to continue;
the manifestations of the API break
are going to appear further away from the spots where they could be
first detected.
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Mathias Hasselmann

Am 24.08.2020 um 09:26 schrieb Giuseppe D'Angelo via Development:


On 23/08/2020 16:06, Marcel Krems wrote:

If they keep using int there could be a lot of warnings like this one:
warning: implicit conversion loses integer precision: 'qsizetype' (aka
'long long') to 'int' [-Wshorten-64-to-32]


I'm afraid that these warnings will be all over the place anyhow. Just 
how much code has been written against int-based APIs? (For the 
record, that's why I wanted a configure time switch to choose the size 
of qsizetype).


But anyhow: yes, as a consistency factor, as many APIs as possible 
should be ported to qsizetype. Could you please file a bug report?


C++ also has a solution for that problem: 
https://herbsutter.com/2013/08/12/gotw-94-solution-aaa-style-almost-always-auto/


Ciao,
Mathias
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


Re: [Development] qsizetype and classes working with QStrings or QList

2020-08-24 Thread Giuseppe D'Angelo via Development

On 23/08/2020 16:06, Marcel Krems wrote:

If they keep using int there could be a lot of warnings like this one:
warning: implicit conversion loses integer precision: 'qsizetype' (aka
'long long') to 'int' [-Wshorten-64-to-32]


I'm afraid that these warnings will be all over the place anyhow. Just 
how much code has been written against int-based APIs? (For the record, 
that's why I wanted a configure time switch to choose the size of 
qsizetype).


But anyhow: yes, as a consistency factor, as many APIs as possible 
should be ported to qsizetype. Could you please file a bug report?


Thanks,
--
Giuseppe D'Angelo | giuseppe.dang...@kdab.com | Senior Software Engineer
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, http://www.kdab.com
KDAB - The Qt, C++ and OpenGL Experts



smime.p7s
Description: S/MIME Cryptographic Signature
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development


[Development] qsizetype and classes working with QStrings or QList

2020-08-23 Thread Marcel Krems

Hi,

since QString, QList, etc. are using qsizetype for indexing- and 
size-operations.
What is the plan with classes working with aforementioned container 
classes which are still using int in their interfaces?

If they keep using int there could be a lot of warnings like this one:
warning: implicit conversion loses integer precision: 'qsizetype' (aka 
'long long') to 'int' [-Wshorten-64-to-32]
Or you have to plaster your code with casts. E.g. every time you pass an 
index of your container to your model class.


Some classes which are probably affected:
QtCore:
* QAbstractItemModel and subclasses (using QList or std::vector as data 
storage)

* QModelIndex
* QRegularExpression (match offset)
* QStringMatcher
* QSyntaxHighlighter
* QTextBoundaryFinder
* QXmlString::size

QtGui:
* QTextCursor
* QTextDocument (find offset, character{At,Count})
* QTextLayout
* QValidator and subclasses (validate offset)

QtWidgets:
* QAbstractItemView and subclasses
* QLineEdit


Kind regards,
Marcel
___
Development mailing list
Development@qt-project.org
https://lists.qt-project.org/listinfo/development