Re: [Development] A QtCore class for event-driven jobs
On Tuesday 17 September 2013 15:08:15 Buddenhagen Oswald wrote: agreed, but for the same reason composite jobs are a non-starter: they are managers with unlimited potential for scheduling policy api growth. Don't go there. Very different; composite jobs are an optional feature useful as an implementation helper for the job. Optional feature for internal implementation detail means no limitation, you don't have to use it if you don't want to. Anyway, I'm postponing this, possibly for ever. If anyone wants to use exactly what I described in this email thread, you are welcome to use the KCoreAddons framework, a standalone library that sits directly on top of QtCore. -- David Faure | david.fa...@kdab.com | Managing Director KDAB France KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
[sending from crappy mobile outlook client] agreed, but for the same reason composite jobs are a non-starter: they are managers with unlimited potential for scheduling policy api growth. Don't go there. -Original Message- From: David Faure Sent: 07.09.2013, 11:21 To: development@qt-project.org Subject: Re: [Development] A QtCore class for event-driven jobs On Saturday 07 September 2013 00:55:26 Konstantin Ritt wrote: Simply can not ignore this thread :) Like Andre, I have written my own jobs implementation, too. And I completely agree that not having some kind of job manager makes QJob API of quite limited usefulness I disagree. One can choose to associate jobs with some sort of manager, but that manager will always be framework-specific, so this can be done in the frameworks themselves. Examples: KIO associates jobs with a singleton scheduler. Akonadi associates jobs with an imap session by passing a parameter to the constructors. All of this doesn't affect the basic job class. Plz don't forget that some generic API in Qt, i.e. like async jobs, should be flexible enough to be used for quite different tasks by variety of users Exactly. This is why it should be usable with and without a job manager. , not only just to make a standard base in future KIO implementation. This is about much more than KIO, my initial email had a list of existing use cases. job = someoperation(some parameters); Yes, you can of course have factory methods in your framework. Just like QNAM::get/put/post. But surely that doesn't mean QtCore (QJob) itself needs to have the concept of a job manager. We'd have two then... manager-enqueue(job); The job's start implementation can do the enqueuing. This way, job not necessarily should derive from QObject; all required signals and methods could be provided by some kind of job watcher [or job sequence watcher] or even by the manager itself. I don't see what's gained by that. Moreover, in case where one needs i.e. to visualize jobs sequence execution progress with just a single progressbar, having to connect to every job's progress/finished/error/whatever signals and/or suspend/resume/abort slots once the job is started makes the implementation not really convenient to the user. Connecting to the manager just once looks like a better option for a generic case, IMO. A manager implies ownership, which isn't desired. But this can be done in a different similar way. I omitted from my initial description an additional feature we have in KJob, which is job trackers. Each job can be associated with a tracker interface, which is basically the base class for something that displays job progress. I thought this could be omitted, because it's something that sits on top of QJob (if we replace job-setTracker with tracker-addJob(), so that job isn't tied to tracker). Then in your own framework, you can decide if you want a global tracker for all jobs, or one per job or per type of job, or depending on the window that created it (because each window has a different embedded progressbar)... But none of that logic has to be in QJob, so that it stays more flexible. The main difference I see between trackers and managers is that for simple cases you can do without a tracker, while a manager would be mandatory in all cases -- which is unnecessary overhead. Another major possible obstacle is a need to suspend job execution up until some additional info has taken from the user (i.e. proxy authorization, non-critical errors handling/reaction, etc.). In common case, this requires appearing some dialog in GUI thread and thus a making blocking call to some user-defined callback, if any (and possibly job re-execution after a successive call, e.g. re-try with admin privileges, where re-enqueuing is definitely not an option). There could be more elegant solution but still, implementation must not force the user to re-invent a wheel every time he needs it. To be clear: suspend() is from the outside, while this is about the job itself needing to pop up a dialog. KIO's jobs already do that just fine. It's the same job going on, the dialog is just an additional step, it can take care of re-enqueuing into its own execution manager if that's needed. I'm not saying some frameworks won't have some sort of manager object. I'm saying it doesn't need to be part of the common API, since it's framework- specific. -- David Faure | david.fa...@kdab.com | Managing Director KDAB France KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On 2013-09-12 08:07, André Somers wrote: Op 11-9-2013 17:19, David Faure schreef: Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Can we forget about threads for a second? No, I'd rather not forget that huge pink elephant in the room... In this age of multi-core systems being the norm rather than exception, we simply cannot ignore threads when designing a generic API for asynchronous jobs. The main point of event-driven jobs is to have them use the event loop, NOT threads. That sounds pointless to me. Why would you design an API for asynchronous jobs, but limit that to those jobs that use the eventloop? What does the user of the API really care what the API does to pull of the async-ness? Compare with QNAM: it uses threading too in the background, doesn't it? To me, a generic job API only makes sense if the API makes sense for all async operations, no matter if the job itself uses threading or the eventloop to work. Both should work equally well, and have equal support IMHO. Seconded. I, too, have written Job-classes, but I wouldn't do so anymore. That concept is so 90's. I'd use a future. A future isn't (necessarily) the result of a thread. It's a placeholder for an asynchronously calculated result. That could be a UDP message. And Qt's future can even transport progress, and supports cancellation and stop/resume. It's not a QObject, except if you want it to (QFutureWatcher). All that's needed is to wrap QFutureInterface into a QPromise (and wrap that one into a QPackagedTask). And the future needs .then() support. Thanks, Marc ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
Thanks Marc, that's exactly what I was saying about. The only difference is (seems to be) a user-visible manager that owns (or let's say locks) the task up until it gets processed so no deletion may occur in the middle of execution, i.e. What the manager is and where/how it executes the tasks is probably up to the user but having a standard interface with some guaranteed functionality is for good, IMO. Regards, Konstantin 2013/9/13 Marc Mutz marc.m...@kdab.com On 2013-09-12 08:07, André Somers wrote: Op 11-9-2013 17:19, David Faure schreef: Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Can we forget about threads for a second? No, I'd rather not forget that huge pink elephant in the room... In this age of multi-core systems being the norm rather than exception, we simply cannot ignore threads when designing a generic API for asynchronous jobs. The main point of event-driven jobs is to have them use the event loop, NOT threads. That sounds pointless to me. Why would you design an API for asynchronous jobs, but limit that to those jobs that use the eventloop? What does the user of the API really care what the API does to pull of the async-ness? Compare with QNAM: it uses threading too in the background, doesn't it? To me, a generic job API only makes sense if the API makes sense for all async operations, no matter if the job itself uses threading or the eventloop to work. Both should work equally well, and have equal support IMHO. Seconded. I, too, have written Job-classes, but I wouldn't do so anymore. That concept is so 90's. I'd use a future. A future isn't (necessarily) the result of a thread. It's a placeholder for an asynchronously calculated result. That could be a UDP message. And Qt's future can even transport progress, and supports cancellation and stop/resume. It's not a QObject, except if you want it to (QFutureWatcher). All that's needed is to wrap QFutureInterface into a QPromise (and wrap that one into a QPackagedTask). And the future needs .then() support. Thanks, Marc ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
Op 11-9-2013 17:19, David Faure schreef: Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Can we forget about threads for a second? No, I'd rather not forget that huge pink elephant in the room... In this age of multi-core systems being the norm rather than exception, we simply cannot ignore threads when designing a generic API for asynchronous jobs. The main point of event-driven jobs is to have them use the event loop, NOT threads. That sounds pointless to me. Why would you design an API for asynchronous jobs, but limit that to those jobs that use the eventloop? What does the user of the API really care what the API does to pull of the async-ness? Compare with QNAM: it uses threading too in the background, doesn't it? To me, a generic job API only makes sense if the API makes sense for all async operations, no matter if the job itself uses threading or the eventloop to work. Both should work equally well, and have equal support IMHO. André -- You like Qt? I am looking for collegues to join me at i-Optics! ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
AFAIK, nesting event loops should be used with a great carefulness and in very limited set of cases, since it may break the expected order of event handling, i.e. when used as a signal waiter. Dunno if this changed since Qt 4.x, though. Regards, Konstantin 2013/9/12 André Somers an...@familiesomers.nl Op 11-9-2013 17:19, David Faure schreef: Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Can we forget about threads for a second? No, I'd rather not forget that huge pink elephant in the room... In this age of multi-core systems being the norm rather than exception, we simply cannot ignore threads when designing a generic API for asynchronous jobs. The main point of event-driven jobs is to have them use the event loop, NOT threads. That sounds pointless to me. Why would you design an API for asynchronous jobs, but limit that to those jobs that use the eventloop? What does the user of the API really care what the API does to pull of the async-ness? Compare with QNAM: it uses threading too in the background, doesn't it? To me, a generic job API only makes sense if the API makes sense for all async operations, no matter if the job itself uses threading or the eventloop to work. Both should work equally well, and have equal support IMHO. André -- You like Qt? I am looking for collegues to join me at i-Optics! ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Tuesday 10 September 2013 21:09:55 Peter Kümmel wrote: On 06.09.2013 19:52, David Faure wrote: connect(job, SIGNAL(result(QJob*)), this, SLOT(handleResult(QJob*))); This looks so old-school like in times of futures and monads. I'm an old-timer, clearly ;-) Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Can we forget about threads for a second? The main point of event-driven jobs is to have them use the event loop, NOT threads. -- David Faure | david.fa...@kdab.com | Managing Director KDAB France KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
Full agreement with Konstantin. It's two weeks before the feature freeze and we haven't seen any more then a draft. I am against any new classes going into Qt essential modules that do not have direct and proven use cases. Develop it in a playground project, show why it makes sense and once you have a stable API let's discuss into which module it should go. Cheers, Lars From: Konstantin Ritt ritt...@gmail.commailto:ritt...@gmail.com Date: tirsdag 10. september 2013 00:40 To: development@qt-project.orgmailto:development@qt-project.org development@qt-project.orgmailto:development@qt-project.org Subject: Re: [Development] A QtCore class for event-driven jobs 2013/9/9 Sune Vuorela nos...@vuorela.dkmailto:nos...@vuorela.dk The api has been stabilized and well tested since like forever. Let's get a QAbstractJob heavily inspired by KJob in now. A nice side effect of getting it in now is that the myriad of nice frameworks KDE is preparing for ship could be built on QAbstractJob and KDE could skip shipping KJob and move everything over to QAbstractJob now and not after we in KDE has made our first release where we promise to keep ABI and API stability. That's indeed what I was afraid of. Your goal is KJob in Qt 5.2 so you can use it by only linking to QtCore. Once released, it's API become frozen up until 6.0...and you don't really care about all others who may disagree with it's design or simply can not use it due to it's limited API and so on. All I've seen so far (following by David's links) is a piece of KIO where some API is still hardcoded to be used by KIO. I'd say this is not a 5.2 material at all. Let us see those nice QProcessJob and/or QThreadJob, that QDBusCallJob...or usable drafts at very least. Until then, I'm all in doubts about how useful would that be to the user. In example, David said job's doStart() enqueues the runnable in the thread pool; now looking at the code - it seems like we could have then started() signal emitted, say, 5 seconds earlier than the runnable gets dequeued and executed. So `for (int i = 0; i 100; ++i) (job = someoperation(params))-start();` gives us 100 started jobs where only few of them got dequeued and really started; and thus all other jobs can not report their actual execution state change because they were started and become started, once dequeued by worker thread...weird, isn't it? First of all, put the initial implementation to Qt-project as a module. Let's then gather such a use-cases and see where the current API is not sufficient, then polish the API to make QJob usable in all those cases; let's write some QJob-s based demonstration framework and polish the API once again, if needed... and only then we'll see if it is good enough for QtCore. Kind regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Mon, Sep 9, 2013 at 11:40 PM, Konstantin Ritt ritt...@gmail.com wrote: 2013/9/9 Sune Vuorela nos...@vuorela.dk The api has been stabilized and well tested since like forever. Let's get a QAbstractJob heavily inspired by KJob in now. A nice side effect of getting it in now is that the myriad of nice frameworks KDE is preparing for ship could be built on QAbstractJob and KDE could skip shipping KJob and move everything over to QAbstractJob now and not after we in KDE has made our first release where we promise to keep ABI and API stability. That's indeed what I was afraid of. Your goal is KJob in Qt 5.2 so you can use it by only linking to QtCore. Once released, it's API become frozen up until 6.0...and you don't really care about all others who may disagree with it's design or simply can not use it due to it's limited API and so on. +1 -- Laszlo ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On 06.09.2013 19:52, David Faure wrote: connect(job, SIGNAL(result(QJob*)), this, SLOT(handleResult(QJob*))); This looks so old-school like in times of futures and monads. Couldn't such a class be part of the hopefully coming QtConcurrent replacement? Peter ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On 10.09.2013 08:16, Knoll Lars wrote: Develop it in a playground project, show why it makes sense and once you have a stable API let's discuss into which module it should go. An idea I already had when I saw the QUniquePoiner implementation: Couldn't we add a new branch to dev/stable/release, which is open for very experimental code? Creating a playground project just for one class is too complicated, and most people would not give them a try, but only switching a branch lowers the barrier a lot. We could base the branch on stable, so it is a good starting point for experiments, and it would be easy to try out these new features without the hassle of checking out (multiple) other repositories, or by cherry-picking stuff. Peter Cheers, Lars From: Konstantin Ritt ritt...@gmail.com mailto:ritt...@gmail.com Date: tirsdag 10. september 2013 00:40 To: development@qt-project.org mailto:development@qt-project.org development@qt-project.org mailto:development@qt-project.org Subject: Re: [Development] A QtCore class for event-driven jobs 2013/9/9 Sune Vuorela nos...@vuorela.dk mailto:nos...@vuorela.dk The api has been stabilized and well tested since like forever. Let's get a QAbstractJob heavily inspired by KJob in now. A nice side effect of getting it in now is that the myriad of nice frameworks KDE is preparing for ship could be built on QAbstractJob and KDE could skip shipping KJob and move everything over to QAbstractJob now and not after we in KDE has made our first release where we promise to keep ABI and API stability. That's indeed what I was afraid of. Your goal is KJob in Qt 5.2 so you can use it by only linking to QtCore. Once released, it's API become frozen up until 6.0...and you don't really care about all others who may disagree with it's design or simply can not use it due to it's limited API and so on. All I've seen so far (following by David's links) is a piece of KIO where some API is still hardcoded to be used by KIO. I'd say this is not a 5.2 material at all. Let us see those nice QProcessJob and/or QThreadJob, that QDBusCallJob...or usable drafts at very least. Until then, I'm all in doubts about how useful would that be to the user. In example, David said job's doStart() enqueues the runnable in the thread pool; now looking at the code - it seems like we could have then started() signal emitted, say, 5 seconds earlier than the runnable gets dequeued and executed. So `for (int i = 0; i 100; ++i) (job = someoperation(params))-start();` gives us 100 started jobs where only few of them got dequeued and really started; and thus all other jobs can not report their actual execution state change because they were started and become started, once dequeued by worker thread...weird, isn't it? First of all, put the initial implementation to Qt-project as a module. Let's then gather such a use-cases and see where the current API is not sufficient, then polish the API to make QJob usable in all those cases; let's write some QJob-s based demonstration framework and polish the API once again, if needed... and only then we'll see if it is good enough for QtCore. Kind regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On terça-feira, 10 de setembro de 2013 21:01:51, Peter Kümmel wrote: On 10.09.2013 08:16, Knoll Lars wrote: Develop it in a playground project, show why it makes sense and once you have a stable API let's discuss into which module it should go. An idea I already had when I saw the QUniquePoiner implementation: Couldn't we add a new branch to dev/stable/release, which is open for very experimental code? Creating a playground project just for one class is too complicated, and most people would not give them a try, but only switching a branch lowers the barrier a lot. We're using Git, you can create your clone and publish it anywhere you'd like. See all my experimental code: http://qt.gitorious.org/qt/thiago-intels-qtbase/ Everything is in the master branch. This branch rebases often. If you want to publish your own class for QtCore, do that as well. People can fetch and merge/cherry-pick your change in their trees. I don't like the idea of a shared, experimental branch. How often will it get nuked and cleaned, or rebased? Who approves stuff there? It also gives the impression that the code will someday make into the release, which is not a given. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On 2013-09-09, Giuseppe D'Angelo dange...@gmail.com wrote: Indeed, but how about starting with an addon and then moving the classes inside QtCore? We can freely break API/ABI in addons -- when things get stabilized, we start including them in QTCore/QtNetwork... I'm not sure what a addon containing 1 small class doing a multiple-year long concept is going to be any good for. The api has been stabilized and well tested since like forever. Let's get a QAbstractJob heavily inspired by KJob in now. A nice side effect of getting it in now is that the myriad of nice frameworks KDE is preparing for ship could be built on QAbstractJob and KDE could skip shipping KJob and move everything over to QAbstractJob now and not after we in KDE has made our first release where we promise to keep ABI and API stability. /Sune ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On segunda-feira, 9 de setembro de 2013 19:38:20, David Faure wrote: I still think the best solution is: QtCore: QAbstractJob, and later QProcessJob and QThreadJob QtNetwork: QNetworkRequestJob (wrapping QNetworkReply) QtDBus: (later) QDBusCallJob (wrapping QDBusPendingCallWatcher) The latter is another reason why a QtJobs library wouldn't work. In order to provide an async-dbus-call job, it would have to link to QDBus, forcing a QDBus dependency onto every user of core-only jobs like QThreadJob. Looking at my initial email, I think that's all. There isn't going to be 20 more job classes in qt itself, the only async operations that happen in there are all on top of sockets, processes and threads. (and timers, but a QTimerJob isn't terribly useful Let's start with a playground to develop the idea and the API. You can freely separate the directories with corelib, network, dbus, eyeing for an eventual separation later. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On 9 September 2013 19:38, David Faure david.fa...@kdab.com wrote: I understand that you want to limit the growth of QtCore, but, hmm, a separate library/module QtJobs seems very strange. It's not like it's separate technology, the core of QAbstractJob's technology is the QtCore event loop. Can you see yourself at the next Dev Days presenting the Qt modules with Qt Core, Qt Gui, Qt Network, Qt Multimedia and ... Qt Jobs ? It doesn't fit that list, because it's not a separate technology with separate dependencies. Indeed, but how about starting with an addon and then moving the classes inside QtCore? We can freely break API/ABI in addons -- when things get stabilized, we start including them in QTCore/QtNetwork... -- Giuseppe D'Angelo ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Friday 06 September 2013 17:20:59 Thiago Macieira wrote: On sexta-feira, 6 de setembro de 2013 19:52:47, David Faure wrote: * relation to QNetworkReply? If that one didn't exist yet, we'd definitely write it as a QJob subclass. So instead, I propose to wrap QNetworkReply into a QNetworkJob or something, in order to offer the QJob interface for QNAM requests. On one hand this doesn't have to go in at the same time as QJob itself, but OTOH it could be a real- world testcase for it, proving its usefulness and correctness... Any other questions? What if we put the QJob class and convenience derived classes like QNetworkRequestJob, QProcessJob, QThreadJob, etc. in one new library. This library could be extended later with more job types. I understand that you want to limit the growth of QtCore, but, hmm, a separate library/module QtJobs seems very strange. It's not like it's separate technology, the core of QAbstractJob's technology is the QtCore event loop. Can you see yourself at the next Dev Days presenting the Qt modules with Qt Core, Qt Gui, Qt Network, Qt Multimedia and ... Qt Jobs ? It doesn't fit that list, because it's not a separate technology with separate dependencies. In addition, this prevents using these job classes within Qt itself: if QNetworkReply didn't exist yet, and if QAbstractJob was in QtCore already, then surely we'd call it QNetworkRequestJob and make it derive QAbstractJob, and we'd put that in QtNetwork. The only reason we got QNetworkReply instead is that QAbstractJob didn't exist yet. By putting the jobs on top of everything else, we will always have the QNetworkReply/QNetworkRequestJob redundancy everywhere a Qt API would benefit from returning a job class. I still think the best solution is: QtCore: QAbstractJob, and later QProcessJob and QThreadJob QtNetwork: QNetworkRequestJob (wrapping QNetworkReply) QtDBus: (later) QDBusCallJob (wrapping QDBusPendingCallWatcher) The latter is another reason why a QtJobs library wouldn't work. In order to provide an async-dbus-call job, it would have to link to QDBus, forcing a QDBus dependency onto every user of core-only jobs like QThreadJob. Looking at my initial email, I think that's all. There isn't going to be 20 more job classes in qt itself, the only async operations that happen in there are all on top of sockets, processes and threads. (and timers, but a QTimerJob isn't terribly useful ;) -- David Faure | david.fa...@kdab.com | Managing Director KDAB France KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
2013/9/9 Sune Vuorela nos...@vuorela.dk The api has been stabilized and well tested since like forever. Let's get a QAbstractJob heavily inspired by KJob in now. A nice side effect of getting it in now is that the myriad of nice frameworks KDE is preparing for ship could be built on QAbstractJob and KDE could skip shipping KJob and move everything over to QAbstractJob now and not after we in KDE has made our first release where we promise to keep ABI and API stability. That's indeed what I was afraid of. Your goal is KJob in Qt 5.2 so you can use it by only linking to QtCore. Once released, it's API become frozen up until 6.0...and you don't really care about all others who may disagree with it's design or simply can not use it due to it's limited API and so on. All I've seen so far (following by David's links) is a piece of KIO where some API is still hardcoded to be used by KIO. I'd say this is not a 5.2 material at all. Let us see those nice QProcessJob and/or QThreadJob, that QDBusCallJob...or usable drafts at very least. Until then, I'm all in doubts about how useful would that be to the user. In example, David said job's doStart() enqueues the runnable in the thread pool; now looking at the code - it seems like we could have then started() signal emitted, say, 5 seconds earlier than the runnable gets dequeued and executed. So `for (int i = 0; i 100; ++i) (job = someoperation(params))-start();` gives us 100 started jobs where only few of them got dequeued and really started; and thus all other jobs can not report their actual execution state change because they were started and become started, once dequeued by worker thread...weird, isn't it? First of all, put the initial implementation to Qt-project as a module. Let's then gather such a use-cases and see where the current API is not sufficient, then polish the API to make QJob usable in all those cases; let's write some QJob-s based demonstration framework and polish the API once again, if needed... and only then we'll see if it is good enough for QtCore. Kind regards, Konstantin ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Sun, Sep 8, 2013 at 3:06 AM, Aleix Pol aleix...@kde.org wrote: Either that or not rushing into porting everything to QJob. Having it in a separate library would mean having QtCoreJobs, QtNetworkJobs, etc. Doesn't seem to be very flexible. I also think it makes sense to have a separate playground module for experiment as Thiago and Konstantin suggested. It would be similar to qtlogger, scene graph, etc which can then later be merged back into main modules if needed. Although, having a playground module even after stabilized for a somewhat similar scenario as QIODevice: network devices, core devices, etc, seems a bit inconsistent, especially if it was also a common functionality as it seems, and not corner case. If it is not a that common functionality, then perhaps, yes... -- Laszlo ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
My two cents (with top quote apology :)
I really like it when classes get added to Qt that can be used out of the box
without sub-classing. That's been a guideline in the past, functionality that
the end users of Qt can use right away. I really really like that philosophy
(and the change to make QThread's run method non-virtual is an example of that).
I also think that it is even better when functionality gets added to Qt that is
used within, by other classes in the same module or by other modules (at least
for classes that are lower down in the dependency chain). Alternatively it is
great if API is in place that makes combining it with a specific other class
particularly easy.
With that in mind, I think that the job class represents an example that is
perhaps more interesting for a framework developer on top of Qt, not an
application developer. Consequently I suggest to let it have the term
Abstract in its name.
I also wonder if it makes sense to perhaps provide convenience API to connect
this with the state machine framework?
Lastly IMHO this should be accompanied with a documented example that shows how
to build a job based framework.
Simon
Fra: David Faure
Sendt: 19:47 fredag 6. september 2013
Til: development@qt-project.org
Emne: [Development] A QtCore class for event-driven jobs
I would like to propose the inclusion of a QJob class in QtCore, based on
years of experience with KIO::Job and KJob.
The idea is to encapsulate an event driven asynchronous operation into a job
class. Example use cases:
- a network download with QNetworkAccessManager.
- operations (command+reply) over a QLocalSocket or QTcpSocket (like akonadi).
- long async dbus calls (special case of the previous line)
- long tasks executed by external processes (e.g. make from an IDE, unrar
from an archive program)
...
At the core, QJob is really just a class with start() and kill(), calling pure
virtual methods doStart() and doKill(), and signals, most importantly the
result(QJob *) signal.
The expected use case is:
void SomeClass::methodWithAsynchronousJobCall()
{
QJob* job = someoperation(some parameters);
connect(job, SIGNAL(result(QJob*)),
this, SLOT(handleResult(QJob*)));
job-start(); // or it could just autostart, which I actually prefer...
}
(other connects, specific to the job)
And handleResult is usually at least:
void SomeClass::handleResult( QJob *job )
{
if (job-error()) {
// handle error
} else {
// handle succesful job completion, if needed
}
}
But it can and should also have the following features:
* error code, error text
* suspend/resume with doSuspend/doResume virtual methods
* capabilities Killable and Suspendable, to avoid trying these on jobs that
don't support them
* kill(Quietly) vs kill(EmitResult), for the app's convenience
* a finished signal that is emitted with both types of killing, for things
like progress dialogs
* auto-deletion (on by default, can be turned off)
* synchronous exec() using a QEventLoop, with a big fat huge warning about not
using that in GUI apps (ideally only to be used in unittests or separate
threads).
* more standard signals for progress info, messages, warnings..
The whole point of standardizing such signals is that it allows generic GUIs
to be built on top, so that your app or your desktop can show the progress of
multiple concurrent jobs, of different types (file download, CD burning, mail
checking, etc. etc.)
Finally, for the benefit of job implementors, QJob would support sub-jobs.
The job implementation would choose when to create these subjobs (all at once
initially, to have them run in parallel, or one after the other, for sequence
of operations). KDE currently does that in a subclass (KCompositeJob) but
Thiago and I (and kio, and akonadi) agree that it's better to have it all in
one class instead.
We also have a standard interface for error handling so that all jobs from a
given framework can have their error handled the same way, but thinking about
it, that part could stay separate, at least for now.
Well, that's it. So why this email? Because Thiago asked me to, and to gather
some support. I plan to make a merge request for Qt 5.2.
Thiago asked more specifically:
* API-wise, can't this be merged with QFuture?
- no, because QFuture encapsulates a value, with blocking methods for getting
the value, even available as casting-to-the-value. If we imagine a QFuture
that wraps a QJob, and the blocking method calling exec(), we'd have a lot
more nested event loops than is good for the health of our programs.
On the other hand, QJob would not be related to any value. It's really a
QObject that informs of progress via signals.
* relation to QRunnable?
A runnable is also some sort of job, but the implementation is completely
different: a runnable is one sync method, while a qjob is all signals/slots
based, with start() returning immediately. So one can't be used like the
other, a given task
Re: [Development] A QtCore class for event-driven jobs
On Sat, 7 Sep 2013, Konstantin Ritt wrote: one would probably prefer [code] job = someoperation(some parameters); manager-enqueue(job); [/code] , where someoperation could be manager's registered job factory or QJob sub-class, etc. I am not sure why everyone prefers to have a manager ;) This way, job not necessarily should derive from QObject; all required signals and methods could be provided by some kind of job watcher [or job sequence watcher] or even by the manager itself. That limits the available API to the lowest common denominator, no? No job-specific features available unless some typeless QVariant properties or similar are used. Harri.___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Friday 06 September 2013 22:22:20 Andre Somers wrote: . However, I'd hope the design would not limit itself to just event-driven jobs, if I understand correctly what that means. Some of my jobs are just heavy processing that require multi-threading. That sounds more like QRunnable than QJob then. But we could have a QThreadJob, for instance like this: it would inherit from both QJob and QRunnable, its doStart() would queue itself as a runnable into a QThreadPool. This creates issues with methods on the same class being called from two different threads though, so maybe better to have QThreadJob and QThreadJobRunnable (which inherits QRunnable and adds signals for progress and completion, optionally cancellation). All this however is experimental, so definitely not for 5.2. Would this API suport that too, or would that be hacky again to implement? Another aspect that I build into my own solution and that I find valuable, is that I have added a mechanism to prevent the same work being done twice. If different parts of the code ask for the same calculation to be performed (that can happen in my case), they actually get a pointer to the same job back. That turns out to be very convenient. Note however, that that requires a job manager, and it doesn't look like that is in this design. In my case, I submit a request to the manager, who then returns a job. You can do that on top of the QJob framework I have in mind. Instead of new CalculationJob you'd ask manager-createCalculationJob() and the manager would take care of reuse and deletion. I don't think this is necessary in the core framework. Note that in my case, I return QSharedPointerJob, so it is clear to everyone that the code that everyone that requests a job and gets one, can't treat it as if it was theirs exclusively. Yep, that can be done on top. Most use cases I've seen are about separate operations rather than calculations, so they don't need the extra syntactic overhead of QSharedPointerJob everywhere. -- David Faure | david.fa...@kdab.com | Managing Director KDAB France KDAB (France) S.A.S., a KDAB Group company Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090 KDAB - Qt Experts - Platform-independent software solutions ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On Sat, Sep 7, 2013 at 2:20 AM, Thiago Macieira thiago.macie...@intel.comwrote: On sexta-feira, 6 de setembro de 2013 19:52:47, David Faure wrote: * relation to QNetworkReply? If that one didn't exist yet, we'd definitely write it as a QJob subclass. So instead, I propose to wrap QNetworkReply into a QNetworkJob or something, in order to offer the QJob interface for QNAM requests. On one hand this doesn't have to go in at the same time as QJob itself, but OTOH it could be a real- world testcase for it, proving its usefulness and correctness... Any other questions? What if we put the QJob class and convenience derived classes like QNetworkRequestJob, QProcessJob, QThreadJob, etc. in one new library. This library could be extended later with more job types. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development Either that or not rushing into porting everything to QJob. Having it in a separate library would mean having QtCoreJobs, QtNetworkJobs, etc. Doesn't seem to be very flexible. Aleix ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
Op 6-9-2013 19:56, Matt Broadstone schreef:
On Fri, Sep 6, 2013 at 1:52 PM, David Faure david.fa...@kdab.com
mailto:david.fa...@kdab.com wrote:
I would like to propose the inclusion of a QJob class in QtCore,
based on
years of experience with KIO::Job and KJob.
The idea is to encapsulate an event driven asynchronous operation
into a job
class. Example use cases:
- a network download with QNetworkAccessManager.
- operations (command+reply) over a QLocalSocket or QTcpSocket
(like akonadi).
- long async dbus calls (special case of the previous line)
- long tasks executed by external processes (e.g. make from an
IDE, unrar
from an archive program)
...
At the core, QJob is really just a class with start() and kill(),
calling pure
virtual methods doStart() and doKill(), and signals, most
importantly the
result(QJob *) signal.
The expected use case is:
void SomeClass::methodWithAsynchronousJobCall()
{
QJob* job = someoperation(some parameters);
connect(job, SIGNAL(result(QJob*)),
this, SLOT(handleResult(QJob*)));
job-start(); // or it could just autostart, which I actually
prefer...
}
(other connects, specific to the job)
And handleResult is usually at least:
void SomeClass::handleResult( QJob *job )
{
if (job-error()) {
// handle error
} else {
// handle succesful job completion, if needed
}
}
But it can and should also have the following features:
* error code, error text
* suspend/resume with doSuspend/doResume virtual methods
* capabilities Killable and Suspendable, to avoid trying these on
jobs that
don't support them
* kill(Quietly) vs kill(EmitResult), for the app's convenience
* a finished signal that is emitted with both types of killing,
for things
like progress dialogs
* auto-deletion (on by default, can be turned off)
* synchronous exec() using a QEventLoop, with a big fat huge
warning about not
using that in GUI apps (ideally only to be used in unittests or
separate
threads).
* more standard signals for progress info, messages, warnings..
The whole point of standardizing such signals is that it allows
generic GUIs
to be built on top, so that your app or your desktop can show the
progress of
multiple concurrent jobs, of different types (file download, CD
burning, mail
checking, etc. etc.)
Finally, for the benefit of job implementors, QJob would support
sub-jobs.
The job implementation would choose when to create these subjobs
(all at once
initially, to have them run in parallel, or one after the other,
for sequence
of operations). KDE currently does that in a subclass
(KCompositeJob) but
Thiago and I (and kio, and akonadi) agree that it's better to have
it all in
one class instead.
We also have a standard interface for error handling so that all
jobs from a
given framework can have their error handled the same way, but
thinking about
it, that part could stay separate, at least for now.
Well, that's it. So why this email? Because Thiago asked me to,
and to gather
some support. I plan to make a merge request for Qt 5.2.
Thiago asked more specifically:
* API-wise, can't this be merged with QFuture?
- no, because QFuture encapsulates a value, with blocking methods
for getting
the value, even available as casting-to-the-value. If we imagine a
QFuture
that wraps a QJob, and the blocking method calling exec(), we'd
have a lot
more nested event loops than is good for the health of our programs.
On the other hand, QJob would not be related to any value. It's
really a
QObject that informs of progress via signals.
* relation to QRunnable?
A runnable is also some sort of job, but the implementation is
completely
different: a runnable is one sync method, while a qjob is all
signals/slots
based, with start() returning immediately. So one can't be used
like the
other, a given task implementation is either a blocking task for a
thread with
no event loop (QRunnable) or an async task that can actually be
used in any
thread with an event loop.
* relation to QNetworkReply?
If that one didn't exist yet, we'd definitely write it as a QJob
subclass. So
instead, I propose to wrap QNetworkReply into a QNetworkJob or
something, in
order to offer the QJob interface for QNAM requests. On one hand
this doesn't
have to go in at the same time as QJob itself, but OTOH it could
be a real-
world testcase for it, proving its usefulness and correctness...
Any other questions?
+1, or more if possible :)
Is this code up somewhere already for early review? This is very much
needed in Qt imho, I imagine it's
Re: [Development] A QtCore class for event-driven jobs
On Fri, Sep 6, 2013 at 1:52 PM, David Faure david.fa...@kdab.com wrote:
I would like to propose the inclusion of a QJob class in QtCore, based on
years of experience with KIO::Job and KJob.
The idea is to encapsulate an event driven asynchronous operation into a
job
class. Example use cases:
- a network download with QNetworkAccessManager.
- operations (command+reply) over a QLocalSocket or QTcpSocket (like
akonadi).
- long async dbus calls (special case of the previous line)
- long tasks executed by external processes (e.g. make from an IDE,
unrar
from an archive program)
...
At the core, QJob is really just a class with start() and kill(), calling
pure
virtual methods doStart() and doKill(), and signals, most importantly the
result(QJob *) signal.
The expected use case is:
void SomeClass::methodWithAsynchronousJobCall()
{
QJob* job = someoperation(some parameters);
connect(job, SIGNAL(result(QJob*)),
this, SLOT(handleResult(QJob*)));
job-start(); // or it could just autostart, which I actually prefer...
}
(other connects, specific to the job)
And handleResult is usually at least:
void SomeClass::handleResult( QJob *job )
{
if (job-error()) {
// handle error
} else {
// handle succesful job completion, if needed
}
}
But it can and should also have the following features:
* error code, error text
* suspend/resume with doSuspend/doResume virtual methods
* capabilities Killable and Suspendable, to avoid trying these on jobs that
don't support them
* kill(Quietly) vs kill(EmitResult), for the app's convenience
* a finished signal that is emitted with both types of killing, for things
like progress dialogs
* auto-deletion (on by default, can be turned off)
* synchronous exec() using a QEventLoop, with a big fat huge warning about
not
using that in GUI apps (ideally only to be used in unittests or separate
threads).
* more standard signals for progress info, messages, warnings..
The whole point of standardizing such signals is that it allows generic
GUIs
to be built on top, so that your app or your desktop can show the progress
of
multiple concurrent jobs, of different types (file download, CD burning,
mail
checking, etc. etc.)
Finally, for the benefit of job implementors, QJob would support sub-jobs.
The job implementation would choose when to create these subjobs (all at
once
initially, to have them run in parallel, or one after the other, for
sequence
of operations). KDE currently does that in a subclass (KCompositeJob) but
Thiago and I (and kio, and akonadi) agree that it's better to have it all
in
one class instead.
We also have a standard interface for error handling so that all jobs from
a
given framework can have their error handled the same way, but thinking
about
it, that part could stay separate, at least for now.
Well, that's it. So why this email? Because Thiago asked me to, and to
gather
some support. I plan to make a merge request for Qt 5.2.
Thiago asked more specifically:
* API-wise, can't this be merged with QFuture?
- no, because QFuture encapsulates a value, with blocking methods for
getting
the value, even available as casting-to-the-value. If we imagine a QFuture
that wraps a QJob, and the blocking method calling exec(), we'd have a lot
more nested event loops than is good for the health of our programs.
On the other hand, QJob would not be related to any value. It's really a
QObject that informs of progress via signals.
* relation to QRunnable?
A runnable is also some sort of job, but the implementation is completely
different: a runnable is one sync method, while a qjob is all signals/slots
based, with start() returning immediately. So one can't be used like the
other, a given task implementation is either a blocking task for a thread
with
no event loop (QRunnable) or an async task that can actually be used in any
thread with an event loop.
* relation to QNetworkReply?
If that one didn't exist yet, we'd definitely write it as a QJob subclass.
So
instead, I propose to wrap QNetworkReply into a QNetworkJob or something,
in
order to offer the QJob interface for QNAM requests. On one hand this
doesn't
have to go in at the same time as QJob itself, but OTOH it could be a real-
world testcase for it, proving its usefulness and correctness...
Any other questions?
+1, or more if possible :)
Is this code up somewhere already for early review? This is very much
needed in Qt imho, I imagine it's one of those patterns that people roll
themselves in many cases.
Matt
--
David Faure | david.fa...@kdab.com | Managing Director KDAB France
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090
KDAB - Qt Experts - Platform-independent software solutions
___
Development mailing list
Development@qt-project.org
[Development] A QtCore class for event-driven jobs
I would like to propose the inclusion of a QJob class in QtCore, based on
years of experience with KIO::Job and KJob.
The idea is to encapsulate an event driven asynchronous operation into a job
class. Example use cases:
- a network download with QNetworkAccessManager.
- operations (command+reply) over a QLocalSocket or QTcpSocket (like akonadi).
- long async dbus calls (special case of the previous line)
- long tasks executed by external processes (e.g. make from an IDE, unrar
from an archive program)
...
At the core, QJob is really just a class with start() and kill(), calling pure
virtual methods doStart() and doKill(), and signals, most importantly the
result(QJob *) signal.
The expected use case is:
void SomeClass::methodWithAsynchronousJobCall()
{
QJob* job = someoperation(some parameters);
connect(job, SIGNAL(result(QJob*)),
this, SLOT(handleResult(QJob*)));
job-start(); // or it could just autostart, which I actually prefer...
}
(other connects, specific to the job)
And handleResult is usually at least:
void SomeClass::handleResult( QJob *job )
{
if (job-error()) {
// handle error
} else {
// handle succesful job completion, if needed
}
}
But it can and should also have the following features:
* error code, error text
* suspend/resume with doSuspend/doResume virtual methods
* capabilities Killable and Suspendable, to avoid trying these on jobs that
don't support them
* kill(Quietly) vs kill(EmitResult), for the app's convenience
* a finished signal that is emitted with both types of killing, for things
like progress dialogs
* auto-deletion (on by default, can be turned off)
* synchronous exec() using a QEventLoop, with a big fat huge warning about not
using that in GUI apps (ideally only to be used in unittests or separate
threads).
* more standard signals for progress info, messages, warnings..
The whole point of standardizing such signals is that it allows generic GUIs
to be built on top, so that your app or your desktop can show the progress of
multiple concurrent jobs, of different types (file download, CD burning, mail
checking, etc. etc.)
Finally, for the benefit of job implementors, QJob would support sub-jobs.
The job implementation would choose when to create these subjobs (all at once
initially, to have them run in parallel, or one after the other, for sequence
of operations). KDE currently does that in a subclass (KCompositeJob) but
Thiago and I (and kio, and akonadi) agree that it's better to have it all in
one class instead.
We also have a standard interface for error handling so that all jobs from a
given framework can have their error handled the same way, but thinking about
it, that part could stay separate, at least for now.
Well, that's it. So why this email? Because Thiago asked me to, and to gather
some support. I plan to make a merge request for Qt 5.2.
Thiago asked more specifically:
* API-wise, can't this be merged with QFuture?
- no, because QFuture encapsulates a value, with blocking methods for getting
the value, even available as casting-to-the-value. If we imagine a QFuture
that wraps a QJob, and the blocking method calling exec(), we'd have a lot
more nested event loops than is good for the health of our programs.
On the other hand, QJob would not be related to any value. It's really a
QObject that informs of progress via signals.
* relation to QRunnable?
A runnable is also some sort of job, but the implementation is completely
different: a runnable is one sync method, while a qjob is all signals/slots
based, with start() returning immediately. So one can't be used like the
other, a given task implementation is either a blocking task for a thread with
no event loop (QRunnable) or an async task that can actually be used in any
thread with an event loop.
* relation to QNetworkReply?
If that one didn't exist yet, we'd definitely write it as a QJob subclass. So
instead, I propose to wrap QNetworkReply into a QNetworkJob or something, in
order to offer the QJob interface for QNAM requests. On one hand this doesn't
have to go in at the same time as QJob itself, but OTOH it could be a real-
world testcase for it, proving its usefulness and correctness...
Any other questions?
--
David Faure | david.fa...@kdab.com | Managing Director KDAB France
KDAB (France) S.A.S., a KDAB Group company
Tel. France +33 (0)4 90 84 08 53, Sweden (HQ) +46-563-540090
KDAB - Qt Experts - Platform-independent software solutions
___
Development mailing list
Development@qt-project.org
http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
On sexta-feira, 6 de setembro de 2013 19:52:47, David Faure wrote: * relation to QNetworkReply? If that one didn't exist yet, we'd definitely write it as a QJob subclass. So instead, I propose to wrap QNetworkReply into a QNetworkJob or something, in order to offer the QJob interface for QNAM requests. On one hand this doesn't have to go in at the same time as QJob itself, but OTOH it could be a real- world testcase for it, proving its usefulness and correctness... Any other questions? What if we put the QJob class and convenience derived classes like QNetworkRequestJob, QProcessJob, QThreadJob, etc. in one new library. This library could be extended later with more job types. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center signature.asc Description: This is a digitally signed message part. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] A QtCore class for event-driven jobs
I think it is a good idea to introduce a playground module until we have the API clean and stable, and then we might want to include it right into QtCore. Regards, Konstantin 2013/9/7 Thiago Macieira thiago.macie...@intel.com On sexta-feira, 6 de setembro de 2013 19:52:47, David Faure wrote: * relation to QNetworkReply? If that one didn't exist yet, we'd definitely write it as a QJob subclass. So instead, I propose to wrap QNetworkReply into a QNetworkJob or something, in order to offer the QJob interface for QNAM requests. On one hand this doesn't have to go in at the same time as QJob itself, but OTOH it could be a real- world testcase for it, proving its usefulness and correctness... Any other questions? What if we put the QJob class and convenience derived classes like QNetworkRequestJob, QProcessJob, QThreadJob, etc. in one new library. This library could be extended later with more job types. -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
