Re: [Development] Request for a sandbox area: Replicant
Hi all I am very interested to discuss and contribute to this project. Our motivation: Our qt application is split to modules written in qt and every module runs in its own process. This gives us robustness but our calls become IPC. All modules are updated at once, so there is no need for stable IPC. Any IPC / RPC mechanism requires a lot of boiler plate code. We develop server and client code in qt. We want small, easily maintainable qt code talking from one process to another. We don't care about particular IPC and serialization being used. Any reasonably efficient IPC is good enough, more options could be supported. The target is to let the Prime QObject class specify the IPC details and keep the boiler plate behind the scene. Not to specify an IPC and then write and maintain boiler plate qt code matching that IPC. I want to highlight that the proposal is usable also for monolitic applications. It introduces new synchronization primitive : One thread provides Prime object, other threads create thread local Replicants. * Slot invocation on replicant is synchronized to prime. Like a type safe wrapper above QMetaObject::invokeMethod to prime. * Replicant properties change only synchronously inside replicant's event loop. No locking needed. The proposal mentions template file that generates abstract prime object and replicant object. Can we generate it directly from Prime object (e.g. in moc)? That would make the framework far more usable. Looking forward to see some code posted. There are so many details. Unfortunately I am not attending the Qt Developer Summit. Maybe I could, but all this is at the last moment. Kind regards Brano ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
Thiago wrote (in a different thread): I think the answer is pretty clear: because there are 10 other RPC solutions and this needs to be harmonised. And, to be honest, a Qt-only solution is not the most ideal. I think we should use an RPC that is available in other toolkits. Oswald wrote (in a different thread): maybe something more task-oriented like QtRpc would be a better idea. My first thought is you're missing the point. But my second thought is - that is certainly my fault. My hope is that when someone thinks of Replicant (or whatever it ends up being called), they think of what it enables, not how it enables it. I don't want the focus to be on the transport, I think that should be a hidden detail. Look at QueuedConnections and everything that happens (behind the scenes) to make them work. If you connect a Signal of one QObject to a Slot on another, but they happen to be in different threads, Qt automatically makes a QueuedConnection. When the Signal is emitted the arguments are marshalled and put into the eventloop of the other thread. When that eventloop reaches the Signal, the marshalled arguments are passed to the Slot in the 2nd thread. Does that mean we should rename QObject to QThreadMarshalObject? Certainly not. Replicant works the same way. You are just dealing with QObjects in both (or all) processes. The fact that there is IPC going on behind the scenes is something you don't have to worry about, it is handled for you. Or look at it another way - look at RPC as an API for remote calls. So the task is usually to define that API and write the boilerplate code. But now look at QObject. What do the Q_PROPERTY, Signal and Slot elements provide? An API. So rather than defining a new API for RPC on top of the QObject (or without it), I'm simply automating the boilerplate code generation to make the QObject RPC-able. Properties (and changes) are automatically propagated. Signals are automatically propagated. Slots are the return API, allowing the remote process to request local changes - which can then be propagated. It is different. Different in a (MHO) cool way. The coolness is Qt only, yes. So is the coolness of Signals and Slots, and QueuedConnections for that matter. ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
Stottlemyer, Brett (B.S.) schreef op 31-5-2014 00:57: I'd like to officially request a sandbox for: Replicant What is it? Replicant is a Qt library for Inter Process Communication (IPC). It is meant as an alternate to QtDBus, but with a very different usage pattern. Are you aware of QxtRPCService as well? http://libqxt.bitbucket.org/doc/0.6/qxtrpcservice.html André ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
Are you aware of QxtRPCService as well? http://libqxt.bitbucket.org/doc/0.6/qxtrpcservice.html André, Nope. Didn't know about that one. It is Signals only, not Properties and Slots as well, though. Thanks! Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
On Monday, June 2, 2014, Stottlemyer, Brett (B.S.) bstot...@ford.com wrote: Are you aware of QxtRPCService as well? http://libqxt.bitbucket.org/doc/0.6/qxtrpcservice.html André, Nope. Didn't know about that one. It is Signals only, not Properties and Slots as well, though. Thanks! Brett There's also qjsonrpc (http://bitbucket.org/devonit/qjsonrpc), if you're interested for cross platform rpc solutions. Cheers, Matt ___ Development mailing list Development@qt-project.org javascript:; http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
I am *HUGELY* interested in this. We do lots of IPC, and across physical devices, and his design issues really hit home for us. Part of the reason for our interest is we've implemented similar things, and the design-approach-overlap is uncanny. (That's a CAN bus joke.) … This is plumbing. It should be in a library. Sitting on top of Qt. Providing simple type-safe cross-process messaging. Signals/slots transparent across processes... drool... --charley \o/ Thanks, Charley! No pressure, though, right? Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
This cross process stuff is starting to feel like 1996 and remote procedure RPC calls, now using QT signals and slots. drool again for effect. One could review the history of microsoft and the fine RPC mechanisms that turned out to be mostly unusable, or maybe just unused. Keep the optimism in check folks. We have a lot of devices now in the mix, not just Win32. md On 6/1/2014 6:50 PM, Stottlemyer, Brett (B.S.) wrote: I am *HUGELY* interested in this. We do lots of IPC, and across physical devices, and his design issues really hit home for us. Part of the reason for our interest is we've implemented similar things, and the design-approach-overlap is uncanny. (That's a CAN bus joke.) … This is plumbing. It should be in a library. Sitting on top of Qt. Providing simple type-safe cross-process messaging. Signals/slots transparent across processes... drool... --charley \o/ Thanks, Charley! No pressure, though, right? Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
This cross process stuff is starting to feel like 1996 and remote procedure RPC calls, now using QT signals and slots. drool again for effect. One could review the history of microsoft and the fine RPC mechanisms that turned out to be mostly unusable, or maybe just unused. Keep the optimism in check folks. We have a lot of devices now in the mix, not just Win32. LOL. I understand this is a proposal and is code you've never seen. So it is impossible for you to tell, based on my description, if it actually does any of the stuff I say it does. It could be the killer app Charley is looking for, or a total who cares as you seem to. I am going to QtCS to talk about it, and I'm sure I will be addressing concerns like yours. /me wonders what Thiago's first question will be BUT - unless you are going to be at QtCS, I'd appreciate it if you took the time to express what you think Replicant needs to do to be successful, or at least what you think caused other RPC mechanisms to fail While I understand the sentiment, it would be helpful to understand what you think Replicant can do to keep falling into the same trap. Or are you of the opinion that an easy/sound RPC mechanism isn't possible? Thanks, Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
Bret: I am certainly open to Ford Motor Company sponsoring my flight to wherever this summit is taking place and we can dispose of the problem of my failure to attend. My talk would be titled, How Bill Gates dreamed of using RPC to court his future yet unseen wife Hell, that might even be qualify for a TED talk as well. I promise I will buy you a beer when I land. My usage of the term HELL probably indicates that I am a crude American without the fine polish of our European counterparts, like our allies at Digia, with whom I have true respect. To put it into automotive terms, I likely drive a F-150 and not a Prius. But I will continue with this monologue until they cut me off. There is the local case and the remote case across the myriad number of devices. There are just a lot of extra variations in IPC across Desktop, Android, and IOS. For example on Local: http://www.slideshare.net/jserv/android-ipc-mechanism For Remote, I think you are left with UDP broadcast/multicast for discovery/reconnection purposes. The reason why RPC died and HTTP/REST dominates is the simple fact that HTTP/REST is SIMPLE. Not 10 million bloody incomprehensible parameters like RPC. Whey does SOAP continue to decline? Extra complexity for little or no gain. Who wants to worry about WSDL? With HTTP/REST, If you do not have network connectivity, the ergonomics are dictated by a simple timeout and an HTTP/404 error or something like that. WIth RPC, you could NEVER tell why it was not working. But you were SURE it was NOT working. Sometimes, you were not sure it WAS working. Let alone a human being operator getting meaningful feedback and the library supporting such simple feedback. What does error WM_ERRORBASE+IPCERRORBASE+CUSTOMIPCERROR mean? So be very careful in getting distracted by all the IPC unification work just to make replication function across device platforms. There may be an easy alternative for both local and remote cases. I know, I use UDP broadcast for both Android and IOS because it just works. And the code is SIMPLE. Cheers, md San Vito Costa Rica On 6/1/2014 8:30 PM, Stottlemyer, Brett (B.S.) wrote: This cross process stuff is starting to feel like 1996 and remote procedure RPC calls, now using QT signals and slots. drool again for effect. One could review the history of microsoft and the fine RPC mechanisms that turned out to be mostly unusable, or maybe just unused. Keep the optimism in check folks. We have a lot of devices now in the mix, not just Win32. LOL. I understand this is a proposal and is code you've never seen. So it is impossible for you to tell, based on my description, if it actually does any of the stuff I say it does. It could be the killer app Charley is looking for, or a total who cares as you seem to. I am going to QtCS to talk about it, and I'm sure I will be addressing concerns like yours. /me wonders what Thiago's first question will be BUT - unless you are going to be at QtCS, I'd appreciate it if you took the time to express what you think Replicant needs to do to be successful, or at least what you think caused other RPC mechanisms to fail While I understand the sentiment, it would be helpful to understand what you think Replicant can do to keep falling into the same trap. Or are you of the opinion that an easy/sound RPC mechanism isn't possible? Thanks, Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
This cross process stuff is starting to feel like 1996 and remote procedure RPC calls, now using QT signals and slots. drool again for effect. One could review the history of microsoft and the fine RPC mechanisms that turned out to be mostly unusable, or maybe just unused. Keep the optimism in check folks. We have a lot of devices now in the mix, not just Win32. I understand your assertion. By 1996 I'd already completed multiple CORBA projects for the telecom and aviation industries. You're right -- it was largely unusable. Hence, those technologies are pretty much gone now (too expensive and problematic). That era had problems: (1) Too complicated (security, platforms, versioning, etc.) (2) Really big foot print. (3) Really expensive tooling (pricing, and hardware requirements) (4) Profound invasiveness/coupling across the tiers (5) Non-scalable (real-world designs could not evolve without profound cost or rewrite, despite promises to the contrary) (6) Inability of the standard to be implemented (it was massive) (7) Inability of the tools/libraries/APIs/standards to evolve (the standard and industry variants were too big/complicated) So, everybody shifted to lighter stuff, like RPC/SOAP/RMI/REST. However, these don't solve the fundamental issue: Asynchronous type-safe messaging across processes. Fundamentally, you can't have it. (Not today, with current approaches; you have to roll-your-own.) I agree that today's lighter fare (like HTTP/REST) is better than back-then. However, that means you have to serialize to XML or JSON, and do your own marshalling to/from your higher-order types. Yes, it's simple. But, it's tedious. The first thing any sane person would do is wrap (hide) that detail so the system doesn't see it. That's why I want the library. Today's assertion is more specific: Cross-process and cross-hardware scaling requires: (a) Asynchronous/non-contention (i.e., messaging designs) (b) Type-safe APIs (beyond POD, handling data marshalling) The signals/slots approach is a new variation over the past, and over what's widely used today. IMHO it is an elegant approach (Qt users already design for (a) (b) when we do signal/slot). Don't make me define a schema. If I pass a QVariant to another process, it had better show up as a QVariant. If you make me define an XML/JSON schema, I'll do it as an implementation detail, but I won't tell anyone about it (because I own both ends). If I want to give a customer access to one end, I'll give them a library that lets them work with real-types and which handles internal control flow/timing details, and not an out-of-date JSON schema description. That's why this should be a library. A generalized solution is possible here. That's why I'm saying killer app. Yes, the downside is that I need Qt on both ends -- so a QVariant is sent and then received (through signal/slot). However, this is *also* why it offers benefits that today's other technologies cannot match (transparent serialization and notification). But, since we already use Qt as our cross-platform porting layer, IMHO this is a feature. True, large distributed systems may need other things like network topology management (nodes come-and-go, security, etc.), but IMHO that's pretty secondary and not terribly difficult to scale on top of the premise/approach, which is type-safe cross-process messaging. The reason why RPC died and HTTP/REST dominates is the simple fact that HTTP/REST is SIMPLE. Not 10 million bloody incomprehensible parameters like RPC. Whey does SOAP continue to decline? Extra complexity for little or no gain. Who wants to worry about WSDL? With HTTP/REST, If you do not have network connectivity, the ergonomics are dictated by a simple timeout and an HTTP/404 error or something like that. WIth RPC, you could NEVER tell why it was not working. But you were SURE it was NOT working. Sometimes, you were not sure it WAS working. Let alone a human being operator getting meaningful feedback and the library supporting such simple feedback. What does error WM_ERRORBASE+IPCERRORBASE+CUSTOMIPCERROR mean? These are fair points. I separate the two: (1) Ease-to-use: Signals/slots across processes is hands-down superior in every way than anything listed above. (Simpler, more correct, more elegant, easier to scale.) -- If it works. (2) Ease-to-setup/administer: I don't know yet. This will be an important detail. I think this is consistent with your concern above, which is one of the reasons HTTP/REST is so popular now (it's easy to setup and diagnose). So, I concede you raise good points. However, if it is easy to setup-and-administer (if we can properly address [2]), then I continue to stand by the Killer App assertion. --charley ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
What we really want to know Charley is whether you drive a F-150 or a Prius. That will decide whether this is a Killer App. md On 6/1/2014 9:43 PM, charleyb123 . wrote: This cross process stuff is starting to feel like 1996 and remote procedure RPC calls, now using QT signals and slots. drool again for effect. One could review the history of microsoft and the fine RPC mechanisms that turned out to be mostly unusable, or maybe just unused. Keep the optimism in check folks. We have a lot of devices now in the mix, not just Win32. I understand your assertion. By 1996 I'd already completed multiple CORBA projects for the telecom and aviation industries. You're right -- it was largely unusable. Hence, those technologies are pretty much gone now (too expensive and problematic). That era had problems: (1) Too complicated (security, platforms, versioning, etc.) (2) Really big foot print. (3) Really expensive tooling (pricing, and hardware requirements) (4) Profound invasiveness/coupling across the tiers (5) Non-scalable (real-world designs could not evolve without profound cost or rewrite, despite promises to the contrary) (6) Inability of the standard to be implemented (it was massive) (7) Inability of the tools/libraries/APIs/standards to evolve (the standard and industry variants were too big/complicated) So, everybody shifted to lighter stuff, like RPC/SOAP/RMI/REST. However, these don't solve the fundamental issue: Asynchronous type-safe messaging across processes. Fundamentally, you can't have it. (Not today, with current approaches; you have to roll-your-own.) I agree that today's lighter fare (like HTTP/REST) is better than back-then. However, that means you have to serialize to XML or JSON, and do your own marshalling to/from your higher-order types. Yes, it's simple. But, it's tedious. The first thing any sane person would do is wrap (hide) that detail so the system doesn't see it. That's why I want the library. Today's assertion is more specific: Cross-process and cross-hardware scaling requires: (a) Asynchronous/non-contention (i.e., messaging designs) (b) Type-safe APIs (beyond POD, handling data marshalling) The signals/slots approach is a new variation over the past, and over what's widely used today. IMHO it is an elegant approach (Qt users already design for (a) (b) when we do signal/slot). Don't make me define a schema. If I pass a QVariant to another process, it had better show up as a QVariant. If you make me define an XML/JSON schema, I'll do it as an implementation detail, but I won't tell anyone about it (because I own both ends). If I want to give a customer access to one end, I'll give them a library that lets them work with real-types and which handles internal control flow/timing details, and not an out-of-date JSON schema description. That's why this should be a library. A generalized solution is possible here. That's why I'm saying killer app. Yes, the downside is that I need Qt on both ends -- so a QVariant is sent and then received (through signal/slot). However, this is *also* why it offers benefits that today's other technologies cannot match (transparent serialization and notification). But, since we already use Qt as our cross-platform porting layer, IMHO this is a feature. True, large distributed systems may need other things like network topology management (nodes come-and-go, security, etc.), but IMHO that's pretty secondary and not terribly difficult to scale on top of the premise/approach, which is type-safe cross-process messaging. The reason why RPC died and HTTP/REST dominates is the simple fact that HTTP/REST is SIMPLE. Not 10 million bloody incomprehensible parameters like RPC. Whey does SOAP continue to decline? Extra complexity for little or no gain. Who wants to worry about WSDL? With HTTP/REST, If you do not have network connectivity, the ergonomics are dictated by a simple timeout and an HTTP/404 error or something like that. WIth RPC, you could NEVER tell why it was not working. But you were SURE it was NOT working. Sometimes, you were not sure it WAS working. Let alone a human being operator getting meaningful feedback and the library supporting such simple feedback. What does error WM_ERRORBASE+IPCERRORBASE+CUSTOMIPCERROR mean? These are fair points. I separate the two: (1) Ease-to-use: Signals/slots across processes is hands-down superior in every way than anything listed above. (Simpler, more correct, more elegant, easier to scale.) -- If it works. (2) Ease-to-setup/administer: I don't know yet. This will be an important detail. I think this is consistent with your concern above, which is one of the reasons HTTP/REST is so popular now (it's easy to setup and diagnose). So, I concede you raise good points. However, if
Re: [Development] Request for a sandbox area: Replicant
(I'm just jumping in here...) snip, Replicant for IPC Bret spaketh: In trying to address your points, I fear it sounds like I think D-Bus is bad. That's not what I'm trying to say. I'm saying D-Bus/QtDBus didn't work *for my use-case*. So I created something that worked better *for my use-case*. There are cases where shared memory is better than sockets for IPC. There is a single perfect solution. I have a HUGE simplifying assumption that all processes use Qt. That cuts down where Replicant is useful quite a bit. But with this assumption, I can make IPC pretty easy. And that was valuable to me, especially in the churn of early development. Thanks, Brett I am *HUGELY* interested in this. We do lots of IPC, and across physical devices, and his design issues really hit home for us. Part of the reason for our interest is we've implemented similar things, and the design-approach-overlap is uncanny. (That's a CAN bus joke.) We'd really prefer to not grow our own, because all of this is mere plumbing. We want signals/slots across processes, and across devices (physical hardware). The Prime/Replicant model concept really does work nicely for this (I haven't looked at Brett's code, I'm just nodding a lot as I hear him talk about it). Did you guys catch the VoCore a few days ago (small Linux stack module)? This kind of thing is going to be *everywhere* with the new internet-of-things. After attending QtDevDays for the last couple years, it's already clear that (today) you can ssh into your vacuum cleaner, stove, and refrigerator. IMHO, transparent signals/slots across hardware stacks will be a killer app for Qt (in addition to its cross-platform and declarative interface). (VoCore: http://linuxgizmos.com/tiniest-linux-com-yet/ ) Brett mentions: (1) Assume processes are not on the same hardware (e.g., can't do shared memory) (2) Assume processes have different target OS (e.g., Linux, Windows) (3) Assume processes have Qt We *also* are willing to make these, HUGE simplifying assumption[s] to make IPC pretty easy. Effectively, if all IPC were expressed as mere signals/slots across processes, that would be *awesome*. Unbelievably Awesome. Killer App Awesome. I estimate this would effectively cover 80+% (maybe 90+%) of the IPC use-cases, with the remainder being dedicated sockets for volume-data-transfer. We do both, but I'd guess *most* applications are nearly 100% status-and-control across processes. (That's the whole point of RESTful interfaces, and why it is increasingly popular.) Recall that today, signals/slots are thread-safe bridges to enable messaging across loosely coupled objects. That's exactly what we're talking about here, but also across-processes (and across hardware). Making it work across processes is already design-consistent with today. IMHO, that's a Killer Application. This is also the *whole point* of a microkernel such as QNX that merely enables messaging-across-processes. Simple. Reliable. Bullet-proof. Scales nicely. No locks/contention. One design implication is the new need for some type of code generation (e.g., extending moc), which Brett mentions he did. We did this too. It's necessary to manage the adapter code that no human should write, based on some kind of protocol or interface specification for the Prime/Replicant bridge. Things like IDLs were used in the past, and CORBA-like systems added functionality like enabling versioning, security, and lots of complicated things on top, but IMHO simpler-is-better. This is plumbing. It should be in a library. Sitting on top of Qt. Providing simple type-safe cross-process messaging. Signals/slots transparent across processes... drool... --charley ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
[Development] Request for a sandbox area: Replicant
Hi list, My name is Brett Stottlemyer, here with my 2nd sandbox request. I work for Ford Motor Company, and Ford has graciously agreed to let us contribute some of the cool stuff we've developed back to Qt. We aren't quite done with everything yet (still waiting on the Corporate CLA), but I will be at Qt Contributors Summit in a week and a half to discuss this and a 2nd project (previously posted QQSM), and answer (hopefully a bunch) of good questions. I'd like to officially request a sandbox for: Replicant What is it? Replicant is a Qt library for Inter Process Communication (IPC). It is meant as an alternate to QtDBus, but with a very different usage pattern. What need does this module solve? There were a number of reasons that made QtDBus unattractive for my use case. 1) Desire to work on Windows as one of the target platforms. This required building the module/dependencies from source, by every developer working on the project. 2) Complexity in initialization vs. notification. There did not seem to be a good way to get the desired properties of an object and enable change notification from that instant forward. Getting properties and getting notifications are separate requests, so there is the possibility of a race condition that would cause the client to be out of sync with the server. 3) Using QtDBus as an IPC mechanism between multiple physical computers does not seem to be a recommended use-case. 4) QtDBus makes it easy to incorporate an existing D-Bus implementation (often not in Qt) and incorporate it into Qt. However, QtDBus requires a fair amount of refactoring if all programs are Qt programs, to marshal to/from D-Bus types and connect the event handler. Detailed description At an overly simple level, the Replicant module is like Qt's QueuedConnection mechanism extended into an IPC mechanism. In a QueuedConnection, when a Signal is emitted, the arguments are copied into an event, which is placed in the event queue of the receiving thread. When that event is reached in the receiving thread, the Slot connected to the Signal is passed the argument copies to act on. This is what prevents the emitting thread from having to block waiting for queued Slots to run. In Replicant, you have the same QObject in multiple processes, and everything needed to handle Property changes and Signal/Slots is copied between processes. Replicant handles almost all of the details behind the scenes. While the intention is for Replicant to be easy for a user to work with, the description provided here is intended more for the developers of Qt itself, or anyone who wants to know how Replicant works internally. Replicant is implemented using QIODevice to marshal information between processes. It is implemented through copies of objects, not sharing via shared memory. In Replicant speak, there is a Prime (the real object) and Replicants (one or more copies). The Replicants are proxies to the real object, in the sense that property changes are forwarded to the Prime, and when the Prime changes, those changes are propagated to every Replicant. Replicants are latent copies. That is, all changes to the Prime will be received by every Replicant, in time-order, although there is a non-deterministic delay for that to happen. Replicant relies on Qt's existing eventloop (time) serialization to make its guarantees. The first issue is initialization, the second is synchronization. When a Replicant is requested, the process that owns the Prime gets the request via the QIODevice. When the event is processed, the current values of every property are gathered, and sent in a reply to the Replicant process. This is not an atomic operation, so it assumes all property changes will be handled in the eventloop as well. In this way, any future changes to object properties will be forwarded over the QIODevice as well, but they will be handled in the Replicant process in the same order they are handled in the Prime process. The actual mechanism for handling Signals/Slots and Property (Q_Property) changes is tied into the internals of qt_metacall. In effect, the parameters to qt_metacall themselves are marshalled over the QIODevice, and passed into qt_metacall for each object within the Replicant process. Note that in order to achieve this, you need 1) a custom qt_metacall implementation (not generated by MOC) and 2) strict versioning to ensure that there isn't a mismatch between the Prime and Replicant. Both of these are handled by providing a Replicant Compiler, repc. The repc program reads a template file that describes the properties, signals and slots of a type, and generates a Prime and Replicant version of a header file for that type. This is built as a custom Qt compiler, so your templates can be added to your .pro file and automatically built for you. As mentioned, Replicant handles Properties, Signals and Slots. But there
Re: [Development] Request for a sandbox area: Replicant
Em sex 30 maio 2014, às 22:57:02, Stottlemyer, Brett escreveu: I'd like to officially request a sandbox for: Replicant Hello Brett I again support the creation of the repository. We'll have to discuss whether this can become part of the Qt standard release because of the overlap in requirements with QtDBus as well as the Publish Subscribe framework that used to exist in Qt Mobility and is still present in qtsystems. What need does this module solve? There were a number of reasons that made QtDBus unattractive for my use case. 1) Desire to work on Windows as one of the target platforms. This required building the module/dependencies from source, by every developer working on the project. One of the goals for QtDBus is to replace the need to have libdbus-1 installed. We discussed this at QtCS last year and we all agreed this is the future, but work has not happened. This is required for kdbus support on Linux. See the notes from last year: http://qt-project.org/groups/qt-contributors-summit-2013/wiki/QtDBus_CS If libdbus-1 is gone, peer-to-peer D-Bus communication should just work on any platform that Qt can run, including Windows, without external dependencies. A bus-type communication will still require a server: either dbus-daemon or a replacement of your choosing. 2) Complexity in initialization vs. notification. There did not seem to be a good way to get the desired properties of an object and enable change notification from that instant forward. Getting properties and getting notifications are separate requests, so there is the possibility of a race condition that would cause the client to be out of sync with the server. That is correct. I've opposed so far any use of the PropertyChange signal that other D-Bus bindings use. The property is very useful for the receiver side, as it could get the change notification from the remote. But it's very clumsy from the sender side, as you don't know whether there's anyone listening for the notifications. Without that piece of information, a generic framework like QtDBus could end up spamming the bus with unnecessary signals. In order to implement it, I will require a counterpart method to enable the emission of signals. The same goes for the proposed ObjectManager interface that would notify of the creation and destruction of objects in the service. 3) Using QtDBus as an IPC mechanism between multiple physical computers does not seem to be a recommended use-case. Correct, but only because libdbus-1 does not offer native support for encryption and authentication. It does offer a d-tube functionality, which is simply getting and setting the bytes that correspond to one D-Bus message, so that the upper layer could send over their preferred communication method. By using this, we'd be able to put the bytes over a QSslSocket provided by the user and thus execute RPC securely. Converting QtDBus to use d-tube is one of the steps in getting rid of the lidbus-1 dependency altogether. See the QtCS link above. Once we get QtDBus independent of libdbus-1, it should be possible to simply provide an opened, bidirectional QIODevice* to a QDBusConnection and would do the communication for you. 4) QtDBus makes it easy to incorporate an existing D-Bus implementation (often not in Qt) and incorporate it into Qt. However, QtDBus requires a fair amount of refactoring if all programs are Qt programs, to marshal to/from D-Bus types and connect the event handler. That is true and there's little we can do about it, until at least C++ comes up with a generic reflection mechanism (which will most likely not happen until 2020) or we provide one ourselves. However, is the creation of the operator and operator really that difficult? -- Thiago Macieira - thiago.macieira (AT) intel.com Software Architect - Intel Open Source Technology Center ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
Re: [Development] Request for a sandbox area: Replicant
Sorry for the formatting guys. Corporate email is Outlook, so I'm manually trying to be polite and bottom post. Hello Brett Hi Thiago I again support the creation of the repository. We'll have to discuss whether this can become part of the Qt standard release because of the overlap in requirements with QtDBus as well as the Publish Subscribe framework that used to exist in Qt Mobility and is still present in qtsystems. Wow. Didn't even know about Publish and Subscribe... What need does this module solve? 1) Desire to work on Windows as one of the target platforms. ... One of the goals for QtDBus is to replace the need to have libdbus-1 installed. We discussed this at QtCS last year and we all agreed this is the future, but work has not happened. This would still be compatible with D-Bus, i.e., still require marshaling, right? Just want to make sure I understand the implications. 2) Complexity in initialization vs. notification. That is correct. I've opposed so far any use of the PropertyChange signal that other D-Bus bindings use. The property is very useful for the receiver side, as it could get the change notification from the remote. But it's very clumsy from the sender side, as you don't know whether there's anyone listening for the notifications. Without that piece of information, a generic framework like QtDBus could end up spamming the bus with unnecessary signals. Agreed. There is a lot of complexity under the covers (in Replicant) to make sure each process only receives data once. And because the objects are derived from QObject, I can notify the sender when they are instantiated and destroyed. That allows me to avoid this kind of flooding. But it certainly isn't perfect, and I hope you can find time to take a look at it and help make it better. 3) Using QtDBus as an IPC mechanism between multiple physical computers does not seem to be a recommended use-case. Correct, but only because libdbus-1 does not offer native support for encryption and authentication. Fair. ATM, I don't either. Google doesn't seem to be as succinct on the point, unfortunately, I think it favors that's not recommended links. 4) QtDBus makes it easy to incorporate an existing D-Bus implementation (often not in Qt) and incorporate it into Qt. However, QtDBus requires a fair amount of refactoring if all programs are Qt programs, to marshal to/from D-Bus types and connect the event handler. That is true and there's little we can do about it, until at least C++ comes up with a generic reflection mechanism (which will most likely not happen until 2020) or we provide one ourselves. Replicant is Qt only, that is all it tries to be. So it can take advantage of moc. That's one of several reasons why I don't think of Replicant as going head-to-head with D-Bus. However, is the creation of the operator and operator really that difficult? I used this to get it working http://techbase.kde.org/Development/Tutorials/D-Bus/Creating_Interfaces. It wasn't easy. But... In trying to address your points, I fear it sounds like I think D-Bus is bad. That's not what I'm trying to say. I'm saying D-Bus/QtDBus didn't work *for my use-case*. So I created something that worked better *for my use-case*. There are cases where shared memory is better than sockets for IPC. There is a single perfect solution. I have a HUGE simplifying assumption that all processes use Qt. That cuts down where Replicant is useful quite a bit. But with this assumption, I can make IPC pretty easy. And that was valuable to me, especially in the churn of early development. Thanks, Brett ___ Development mailing list Development@qt-project.org http://lists.qt-project.org/mailman/listinfo/development
