Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On 08/21/2011 09:31 AM, todd rme wrote: On Sun, Aug 21, 2011 at 12:09 AM, Valentin Rusu k...@rusu.info wrote: On 08/20/2011 02:11 PM, Oswald Buddenhagen wrote: A cross-desktop specification, but we still use kwallet. There's no reason to dump it in favour of another implementation. So I see no arguments at all in favour of dropping it -- in fact, I see more in keeping it. ksecretservice is a new implementation. dunno how much code was reused. the arguments against two backends are easy: - two backends means almost twice the work, including security auditing - assuming the backends use different storage (it's not part of the spec, after all), even after switching desktops you need to stick with the now alien password manager. KSecretsService is a new implementation. It has a new backend that organizes secrets in collections and follows the freedesktop.org specification. KDE applications will use the new KSecretsService client library that'll replace KWallet API. I'm currently rewriting the KWallet class in order to get it use this new KSecretsService Client API. The backend (or daemon) will also be able to import kwallet files. So in the future the kwalletd will become useless. The KSecretsService Client API will be able to connect to whatever service will expose the freedesktop.org secrets service specification on the DBus. That means that KDE applications will be able to store passwords in gnome-keyring if present instead or ksecretsserviced (this is the name of the daemon exposing fd.o secrets specification currently under development). Great! This should be very helpful A few general questions: 1. Will it be possible to add connections to password providers, like online password management services or firefox, that allow those passwords to be integrated with secretservice? Or is this something that would need to be handled at the frontend (ksecretserviced) level? I already started a sync tool named SecretSync that'll allow password synchronization over several machines/terminals. Google Chrome works with KWallet so it'll also work with KSecretsService. Firefox KWallet plugin maintainship is stalled. All these should use KWallet API or better switch to KSecretsService Client. 2. Is secretservice integrated with PAM so that the passwords can be opened on login? Or is this also something that needs to be handled at the ksecretserviced level? This is not yet defined. 3. Will gnome-keyring and ksecretserviced by storing their passwords in the same location or file(s), or will there need to be a separate set of passwords for gnome-keyring and ksecretservicd? These are two different tools, each with it's own implementation and maintainers. 4. If you are running gnome programs in a KDE workspace, or vice versus, will they automatically connect to the right secretservice or will they try to call their own backend? KDE KSecretsService API will connect to whatever daemon provides the right interfaces on the DBus. So it'll be possible for a KDE application to use gnome-keyring to store it's secrets. Ongoing work will define how users will choose the daemon they want. Valentin -- Valentin Rusu (IRC valir, KDE vrusu) KSecretsService (former KSecretService, KWallet replacement)
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday 21 August 2011, Thiago Macieira wrote: On Sunday, 21 de August de 2011 10:19:26 Oswald Buddenhagen wrote: What instability? If kded crashes, what makes you think individual services won't crash, in addition to taking longer to load and use more memory? look at it from the perspective of the daemon, not the modules. there have been *tons* of bugs related to crashes and freezes in kded - which were module bugs actually. in-process is simply no sound architecture from a stability (and security) pov. guess which other kde component i'm thinking of ... Sure, if each of 10 modules has a certain chance of failure or MTBF, the whole process has a much greater chance of failure or smaller MTBF. But if you look from the point of view from the system that requires each of those services, it doesn't matter if the crash brings down all services or just one. In any case, kded restarts. if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. does the session systemd run privileged in the first place? I have no clue. I don't even know if there's a session systemd. It's far easier to introduce the feature we want into the prelinker and dynamic loader, such as having a stasis process: the dynamic loader could load everything as needed, then freeze the images just prior to running any code. one can have that with some ptrace trickery. but i'm not sure how this fixes the problem. do you want to exec() such pre-initialized images? This isn't a fully-formed idea. It might need kernel help too. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. [...] I don't know where you got those numbers for weight, not to mention it's completely ambiguous (did you mean memory consumption?). code size = shared memory, page faults on load. That is true. But it is not proven. Sure, QtCore is much bigger than glib: $ ls -l /usr/lib/libQtCore.so.4.8.0 /lib/libglib-2.0.so.0.2800.8 -rwxr-xr-x 1 root root 979028 Jun 8 02:53 /lib/libglib-2.0.so.0.2800.8* -rwxr-xr-x 1 root root 2980096 Jul 25 22:28 /usr/lib/libQtCore.so.4.8.0* But file sizes don't mean much. You'd have to check how much in page faults really happens to fault in those services: given comparable uses of the libraries, what is the actual memory use? What's more, you can also justify it as a trade-off in readability and maintainabilty. In a running system, where QtCore is already loaded, there aren't any page faults to load the code or marginal increase in memory usage because of it. And if we fix the above problem of prelinking, the prelinked read-only data can also be shared. (note my use of commas in where QtCore is already loaded: it's not a restrictive definition of system, it's an explanation of it; for me, there are no systems where QtCore isn't already loaded) - introducing a qt or even kde dependency into a so far gnome-only system can be a very real resource problem. that's not likely to apply to kglobalaccel (as it is pretty desktop-specific), but i'm assuming you are making a general statement Right. And if we accept glib dependencies, I don't see why they shouldn't accept QtCore dependencies. Yes. Requiring QtCore as a dependency should be seen the same way as requiring glib. It's not a huge library, it has few dependencies, and it is there on many many systems anyway (e.g. all systems I have access to). We should work on that. Being able to use QtCore is great, I want to have it available on every system. There's no reason for us to say yeah, we know it's big, so it's ok if you rewrite everything using glib instead. QtCore is fine. Alex
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday 20 August 2011, Thiago Macieira wrote: On Saturday, 20 de August de 2011 10:02:31 Oswald Buddenhagen wrote: On Tue, Aug 16, 2011 at 08:58:02PM +0200, Thiago Macieira wrote: frameworks (qt-based), applications and workspace, that sounds pretty much what the KDE Platform is. What are you excluding in your definition? kded, klauncher, kdeinit, kglobalaccel, kwallet? yes, among other things. kded's module activation is redundant with systemd. But not at all the same impact. Starting an application and negotiating its connection to D-Bus is hardly comparable to loading a plugin in a daemon that is already connected. as it happens, lennart also made a mini-daemon dealing with host name changes, which is another thing kded does. kdontchangethehostname klauncher is conceptually redundant with systemd as well. the backend it uses is kdeinit, which *should* be in systemd - it's not like it would not benefit gnome as well (otherwise there would be no maemo booster) - we just need some fresh numbers to justify the hack, and some arguments why the hack isn't *that* bad after all. That I agree: klauncher is systemd for KDE only, so we should see about getting the same benefits from systemd instead. There are two drawbacks with that, though: 1) systemd will not likely ever run on non-Linux systems, not even the BSDs. Lennart simply isn't interested on ensuring compatibility and might even reject patches which introduce differences to Linux-specific behaviour. Wouldn't that also make us require Linux systems which actually run systemd ? I guess Slackware would be out then. Alex
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sat, Aug 20, 2011 at 02:43:49PM +0200, Thiago Macieira wrote: On Saturday, 20 de August de 2011 14:11:32 Oswald Buddenhagen wrote: yeah, at the cost of rather considerable instability. we should see whether regular service activation isn't the better option, now that we have it. What instability? If kded crashes, what makes you think individual services won't crash, in addition to taking longer to load and use more memory? look at it from the perspective of the daemon, not the modules. there have been *tons* of bugs related to crashes and freezes in kded - which were module bugs actually. in-process is simply no sound architecture from a stability (and security) pov. guess which other kde component i'm thinking of ... One problem with a no-exec scenario is the inability to switch security contexts and to control them on a per-binary case. Today, that is. There's nothing preventing additional code to be introduced to do that, indeed, that's what i suggested as well. if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. does the session systemd run privileged in the first place? [...] The end result is that the launcher has a lot more COW operations that aren't its fault and its pages are duplicated everywhere. If there's a privilege boundary in the process, then this might even be a security risk [...] yeah, both of these concern result from systemd being a bit more than kdeinit. they could be mitigated by forking off the actual launcher process very early, but this may turn out ugly. It's far easier to introduce the feature we want into the prelinker and dynamic loader, such as having a stasis process: the dynamic loader could load everything as needed, then freeze the images just prior to running any code. one can have that with some ptrace trickery. but i'm not sure how this fixes the problem. do you want to exec() such pre-initialized images? i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. [...] I don't know where you got those numbers for weight, not to mention it's completely ambiguous (did you mean memory consumption?). code size = shared memory, page faults on load. But I don't care what they like or dislike: if the implementation is good, it should stay. there are two problems with that: - history has shown that it doesn't matter what the pragmatic solution would be, on both sides (just remember the utterly ridiculous bike shedding when aRts dared to add a glib dependency) - introducing a qt or even kde dependency into a so far gnome-only system can be a very real resource problem. that's not likely to apply to kglobalaccel (as it is pretty desktop-specific), but i'm assuming you are making a general statement I could even accept the alternate representation to be optional, so an UTF-8- to-UTF-16 coversion needs to happen when Qt-based code reads the config. on-demand conversion could lead to some nasty encoding ping-pong when the data is accessed concurrently. The problem I found is the same as QUrl today: implicit sharing of lazily constructed data. When you need to finish your lazy work, do you need to detach or not? If you detach, only one side will get the non-lazy output. If you don't detach, you need to somehow make the setting thread-safe -- which QUrl doesn't do. not detaching needs to do atomic pointer replacement to avoid memory leaks, but other than that it is relatively uncritical. a problem is of course that the converted data would need a separate allocation, as opposed to the embedded raw string data. This also carries the burden of checking the flags at many points in the code, probably adding overhead to timing-sensitive code which many already find slow. when i was pondering this with andre, we came up with the idea to make character access possible only through some kind of reference object, so checking the need for detaching and conversions would have to be done only when obtaining the reference. but that was long before qt5 actually materialized, including its mostly source compatible mandate.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday, 21 de August de 2011 10:19:26 Oswald Buddenhagen wrote: What instability? If kded crashes, what makes you think individual services won't crash, in addition to taking longer to load and use more memory? look at it from the perspective of the daemon, not the modules. there have been *tons* of bugs related to crashes and freezes in kded - which were module bugs actually. in-process is simply no sound architecture from a stability (and security) pov. guess which other kde component i'm thinking of ... Sure, if each of 10 modules has a certain chance of failure or MTBF, the whole process has a much greater chance of failure or smaller MTBF. But if you look from the point of view from the system that requires each of those services, it doesn't matter if the crash brings down all services or just one. In any case, kded restarts. if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. does the session systemd run privileged in the first place? I have no clue. I don't even know if there's a session systemd. It's far easier to introduce the feature we want into the prelinker and dynamic loader, such as having a stasis process: the dynamic loader could load everything as needed, then freeze the images just prior to running any code. one can have that with some ptrace trickery. but i'm not sure how this fixes the problem. do you want to exec() such pre-initialized images? This isn't a fully-formed idea. It might need kernel help too. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. [...] I don't know where you got those numbers for weight, not to mention it's completely ambiguous (did you mean memory consumption?). code size = shared memory, page faults on load. That is true. But it is not proven. Sure, QtCore is much bigger than glib: $ ls -l /usr/lib/libQtCore.so.4.8.0 /lib/libglib-2.0.so.0.2800.8 -rwxr-xr-x 1 root root 979028 Jun 8 02:53 /lib/libglib-2.0.so.0.2800.8* -rwxr-xr-x 1 root root 2980096 Jul 25 22:28 /usr/lib/libQtCore.so.4.8.0* But file sizes don't mean much. You'd have to check how much in page faults really happens to fault in those services: given comparable uses of the libraries, what is the actual memory use? What's more, you can also justify it as a trade-off in readability and maintainabilty. In a running system, where QtCore is already loaded, there aren't any page faults to load the code or marginal increase in memory usage because of it. And if we fix the above problem of prelinking, the prelinked read-only data can also be shared. (note my use of commas in where QtCore is already loaded: it's not a restrictive definition of system, it's an explanation of it; for me, there are no systems where QtCore isn't already loaded) - introducing a qt or even kde dependency into a so far gnome-only system can be a very real resource problem. that's not likely to apply to kglobalaccel (as it is pretty desktop-specific), but i'm assuming you are making a general statement Right. And if we accept glib dependencies, I don't see why they shouldn't accept QtCore dependencies. I could even accept the alternate representation to be optional, so an UTF-8- to-UTF-16 coversion needs to happen when Qt-based code reads the config. on-demand conversion could lead to some nasty encoding ping-pong when the data is accessed concurrently. You misunderstand dconf: reading settings is done via mmap()ed read-only memory. If the UTF-16 encoding isn't there, then the reader needs to create a QString from the UTF-8 data. Writing is done via D-Bus. I don't see a ping-pong. The problem I found is the same as QUrl today: implicit sharing of lazily constructed data. When you need to finish your lazy work, do you need to detach or not? If you detach, only one side will get the non-lazy output. If you don't detach, you need to somehow make the setting thread-safe -- which QUrl doesn't do. not detaching needs to do atomic pointer replacement to avoid memory leaks, but other than that it is relatively uncritical. a problem is of course that the converted data would need a separate allocation, as opposed to the embedded raw string data. If you don't detach and you do replace the pointers, when do you delete the old data? You need a shared data pointer of some kind. I don't want to introduce a second reference count in QStringData. when i was pondering this with andre, we came up with the idea to make character access possible only through some kind of reference object, so checking the need for detaching and conversions would have to be done only when obtaining the reference. That's QCharRef. but that was long before qt5 actually materialized, including its mostly source compatible mandate. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 11:28:02AM +0200, Thiago Macieira wrote: Sure, if each of 10 modules has a certain chance of failure or MTBF, the whole process has a much greater chance of failure or smaller MTBF. But if you look from the point of view from the system that requires each of those services, it doesn't matter if the crash brings down all services or just one. not all kded services are required - many are more like recommended. but even if we assume that all are required, then out-of-process still improves fault isolation and recoverability. In any case, kded restarts. yes, when the faulty module crashes. not so when it deadlocks or busy-loops. also, a restart typically loses state. code size = shared memory, page faults on load. That is true. But it is not proven. [...] well, yeah, somebody has to do some real research. but the outcome is pretty predictable: qt's use of templates makes it extremely likely that significantly more code is executed, and the rather likely suboptimal code locality makes it probable that more unused code needs to be paged in from the much larger binary. What's more, you can also justify it as a trade-off in readability and maintainabilty. well, of course. but given the availability of two equivalent solutions and people willing to maintain them, the one with a lower resource consuption will always be favorable. Right. And if we accept glib dependencies, I don't see why they shouldn't accept QtCore dependencies. well, given that glib is an accepted dependency of qt (at least on the desktop), qt will always be an *additional* dependency. it's going to be interesting to push it down the stack enough that people stop thinking about it. reading settings is done via mmap()ed read-only memory. If the UTF-16 encoding isn't there, then the reader needs to create a QString from the UTF-8 data. Writing is done via D-Bus. i know. exactly this has the potential for ping-pong if you do on-demand conversion of the actual store. anyway, i really wonder whether insisting on utf-16 makes sense, assuming that the store is not abused for more than config data. KConfig and QSettings use utf-8, too, and nobody seriously complained about that afaik. the dconf client can implement caching of converted data just like the exising classes do. If you don't detach and you do replace the pointers, when do you delete the old data? the lifetime would be bound to the qstringdata, like the latin1 data in qt3-. i wouldn't bother with sharing the converted values - upon detaching, the format which is needed for the operation which caused the detach is made the primary (embedded) and only data of the clone. when i was pondering this with andre, we came up with the idea to make character access possible only through some kind of reference object, That's QCharRef. more like a qstringref, as we want bounds checking. but it should be also possible to append to the non-const variant of that thingie.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday, 21 de August de 2011 12:25:52 Oswald Buddenhagen wrote: yes, when the faulty module crashes. not so when it deadlocks or busy-loops. also, a restart typically loses state. True, but I don't remember that happening a single time to me in the past 3 years. I remember seeing people complain about kded taking 100% CPU, but it hasn't happened to me. code size = shared memory, page faults on load. That is true. But it is not proven. [...] well, yeah, somebody has to do some real research. but the outcome is pretty predictable: qt's use of templates makes it extremely likely that significantly more code is executed, and the rather likely suboptimal code locality makes it probable that more unused code needs to be paged in from the much larger binary. The extensive use of templates probably generates more code, but due to better inlining, the code is potentially faster. So we're trading off code size for code performance. I once wrote a benchmark comparing iterating over a QString to iterating over a gchar UTF-8 string using glib functions to get each UCS-4 character (ostensibly to prove that UTF-16 was better than UTF-8). The result was clear: Qt code was much faster, over 10x, compared to glib. I wouldn't be surprised if we find the same about the other containers. Our benchmark isn't glib containers, but STL ones. well, of course. but given the availability of two equivalent solutions and people willing to maintain them, the one with a lower resource consuption will always be favorable. Agreed. Which is why an existing service that we write should be in C++. We should not be required to learn glib in order to write a shared service. If the GNOME developers decide they want to rewrite it, they are welcome to do so and dedicate resources. Note I have never proposed rewriting existing shared service code in Qt. well, given that glib is an accepted dependency of qt (at least on the desktop), qt will always be an *additional* dependency. it's going to be interesting to push it down the stack enough that people stop thinking about it. Or make enough interesting services that are Qt-based. Back on the Ubuntu Developer Summit in May last year, when I was presenting some of the technologies of Qt Mobility, there was a big interest in using them. Mark Shuttleworth asked if there were bindings for Python, which is the language that Canonical likes to use. But I'm sure other people wanted to use them from C too, they just didn't ask. The point is just as above: we, the KDE and Qt communities, are not required to learn glib in order to write a new service or library. reading settings is done via mmap()ed read-only memory. If the UTF-16 encoding isn't there, then the reader needs to create a QString from the UTF-8 data. Writing is done via D-Bus. i know. exactly this has the potential for ping-pong if you do on-demand conversion of the actual store. What ping-pong? If there's no UTF-16 encoding of the read-only and unchangeable data, you use QString::fromUtf8. If there is, you return QString::fromRawData. When you write, you write with the encoding. Already-running applications must reopen the file, or scan to a later point in it anyway to see the new value. anyway, i really wonder whether insisting on utf-16 makes sense, assuming that the store is not abused for more than config data. KConfig and QSettings use utf-8, too, and nobody seriously complained about that afaik. the dconf client can implement caching of converted data just like the exising classes do. I have. Please try this: valgrind --tool=massif kwrite Then use the nice massifvisualizer to look at what consumes the most memory at startup. It's KConfig, upwards of a couple of megabytes of heap. If you don't detach and you do replace the pointers, when do you delete the old data? the lifetime would be bound to the qstringdata, like the latin1 data in qt3-. i wouldn't bother with sharing the converted values - upon detaching, the format which is needed for the operation which caused the detach is made the primary (embedded) and only data of the clone. So you're advocating the detaching method, not the convert-and-inline-replace one which QUrl tries to do and fails. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday 20 Aug 2011 13:11:32 Oswald Buddenhagen wrote: On Sat, Aug 20, 2011 at 12:20:55PM +0200, Thiago Macieira wrote: It needs a global spec too, since global shortcut grabbing with X11 libs only is sorely lacking. I think the solution we made for KDE 4 is actually quite good. Anyone wants to create an XDG spec for global accelerators? well, yeah. good luck to whoever. :P We had a bad experience last time, but we should use that experience to play smarter this time around. The big complaints that I saw from Gnome last time was that we supposedly had no clear statement of why the spec was needed and what it was meant to achieve, that they had no real involvment in the drafting of the spec, that we talked to the wrong people, and that it was something they didn't need in G3. So lets turn that on its head. Lets write a statement saying why we need a spec and what it needs to achieve. Mention we have an existing solution that would be a good starting point, but don't actually detail it. Then send that to xdg and Gnome and Unity and anyone else asking who are the right pople to talk to. Hopefully those people then decide it's a good thing and agree to work together to develop a standard. If they're not interested then we get to draft it ourselves knowing there can be no complaints. But just as before, I don't see why our code can't be cross-desktop. So no argument in favour of dropping kglobalaccel. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. seeing the reluctance to anything with g* within our community, why do you think the gnomers would embrace anything with a q* or even k*, esp. given that it usually weights in at least twice as much as the typical g* solution? We depend on a lot of g code these days, some of it even willingly, and we're looking at even more. As for Gnome never accepting any q/k code, it's mostly true, but copying is the sincerest form of flattery :-) Can we actually point to cases where Gnome rejected Qt/KDE code proposals, and how about ones that have been accepted (Poppler?). I'm not convinced it's really about the weight or otherwise of our solutions (system-config-printer is written in python!). How much is just C vs C++, or our not pushing stuff, or just not having key people employed by the distros? I'd like to see what happens when Qt5 has a nice light QtCore that we use to write something small and light that they need. John.
Re: Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
A Diumenge, 21 d'agost de 2011, John Layt vàreu escriure: On Saturday 20 Aug 2011 13:11:32 Oswald Buddenhagen wrote: On Sat, Aug 20, 2011 at 12:20:55PM +0200, Thiago Macieira wrote: It needs a global spec too, since global shortcut grabbing with X11 libs only is sorely lacking. I think the solution we made for KDE 4 is actually quite good. Anyone wants to create an XDG spec for global accelerators? well, yeah. good luck to whoever. :P We had a bad experience last time, but we should use that experience to play smarter this time around. The big complaints that I saw from Gnome last time was that we supposedly had no clear statement of why the spec was needed and what it was meant to achieve, that they had no real involvment in the drafting of the spec, that we talked to the wrong people, and that it was something they didn't need in G3. So lets turn that on its head. Lets write a statement saying why we need a spec and what it needs to achieve. Mention we have an existing solution that would be a good starting point, but don't actually detail it. Then send that to xdg and Gnome and Unity and anyone else asking who are the right pople to talk to. Hopefully those people then decide it's a good thing and agree to work together to develop a standard. If they're not interested then we get to draft it ourselves knowing there can be no complaints. But just as before, I don't see why our code can't be cross-desktop. So no argument in favour of dropping kglobalaccel. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. seeing the reluctance to anything with g* within our community, why do you think the gnomers would embrace anything with a q* or even k*, esp. given that it usually weights in at least twice as much as the typical g* solution? We depend on a lot of g code these days, some of it even willingly, and we're looking at even more. As for Gnome never accepting any q/k code, it's mostly true, but copying is the sincerest form of flattery :-) Can we actually point to cases where Gnome rejected Qt/KDE code proposals, and how about ones that have been accepted (Poppler?). You really can not cound poppler as coming from our side. Poppler was started and driven by Red Hat Desktop Team (or something similar) and it was not until they lost interest (aka Red Hat relocated people to work on a different project) that real easy collaboration started to happen. Albert I'm not convinced it's really about the weight or otherwise of our solutions (system-config-printer is written in python!). How much is just C vs C++, or our not pushing stuff, or just not having key people employed by the distros? I'd like to see what happens when Qt5 has a nice light QtCore that we use to write something small and light that they need. John.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 02:54:41PM +0200, Thiago Macieira wrote: On Sunday, 21 de August de 2011 12:25:52 Oswald Buddenhagen wrote: yes, when the faulty module crashes. not so when it deadlocks or busy-loops. also, a restart typically loses state. True, but I don't remember that happening a single time to me in the past 3 years. I remember seeing people complain about kded taking 100% CPU, but it hasn't happened to me. yes, things have stabilized meanwhile. but the problem will re-appear during the kde5 cycle. it's inherent, after all. The extensive use of templates probably generates more code, but due to better inlining, the code is potentially faster. So we're trading off code size for code performance. probably. depending on the use case, this may or may not be a convincing argument. it's going to be interesting to push it down the stack enough that people stop thinking about it. Or make enough interesting services that are Qt-based. well, that's the only way to do the pushing. it's not like we have any political leverage to do it differently. The point is just as above: we, the KDE and Qt communities, are not required to learn glib in order to write a new service or library. some day maybe. but to be able to join existing projects we have to learn glib anyway. and if qt becomes a common framework for low-level things, others will have to learn qt. we are going to fragment the ecosystem! :D What ping-pong? If there's no UTF-16 encoding of the read-only and unchangeable data, you use QString::fromUtf8. uhm, ok, i think i misunderstood you. If there is, you return QString::fromRawData. uhm, no, you must make a deep copy, otherwise you get a time bomb. but yeah, no conversion, so pretty cheap. anyway, i really wonder whether insisting on utf-16 makes sense, assuming that the store is not abused for more than config data. [...] valgrind --tool=massif kwrite Then use the nice massifvisualizer to look at what consumes the most memory at startup. It's KConfig, upwards of a couple of megabytes of heap. there is a 90+ kb file named katesyntaxhighlightingrc which seems to be mostly a cache file. this use of kconfig borders on abuse. not sure if there are other files on your system or kconfig really is that inefficient. but generally speaking, a program which actually needs to query several thousand keys on startup is definitely Doing It Wrong. and if it reads only a few keys (of even a million), then the conversion cache will hold almost nothing. on top of that, the cached converted values will be to some degree shared with the objects which requested the values. So you're advocating the detaching method, not the convert-and-inline-replace one which QUrl tries to do and fails. uhm, maybe. it's not like the terminology would be self-evident. ;)
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday 20 Aug 2011 10:14:20 Oswald Buddenhagen wrote: On Tue, Aug 16, 2011 at 08:00:19PM +0100, John Layt wrote: I've certainly seen him state that he doesn't care about KDE, that we are irrelevent to anything he does, and he sees no reason to collaborate on anything with us. and he's right. show me one case where the outcome of his low-level work did not meet kde requirements. Ummm, Pulse Audio? It completely broke sound under KDE. It caused a lot of users a lot of pain before Colin come along to fix it for us and for Gnome users too. I don't want to see a repeat of that, so figuring out how to get Lennart to care becomes important. It's similar to his attitude towards *BSD. he's right with that, too. if the bsd's (or solaris, for that matter) want to stay relevant on the desktop, they need to catch up with linux. or put differently, they make up about about 0.1% of our user base, which easily could be several times bigger if we were able to provide something well-integrated (mac os x anyone?). do you think they are worth sacrificing? I have no problem with him saying he won't support BSD, I'm happy for them to write their own patches or version, but that's a bit hard for them to do if the dbus api is a moving target. If we have people willing to do the work to support KDE on BSD, then who are we to stop them provided the rest of KDE isn't negatively affected? Will we also stop supporting Windows and OSX because we're not fully integrated there and just focus on Linux? That sounds very much like the GnomeOS idea... John.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday 21 August 2011 Aug, Oswald Buddenhagen wrote: On Sun, Aug 21, 2011 at 02:54:41PM +0200, Thiago Macieira wrote: On Sunday, 21 de August de 2011 12:25:52 Oswald Buddenhagen wrote: yes, when the faulty module crashes. not so when it deadlocks or busy-loops. also, a restart typically loses state. True, but I don't remember that happening a single time to me in the past 3 years. I remember seeing people complain about kded taking 100% CPU, but it hasn't happened to me. yes, things have stabilized meanwhile. but the problem will re-appear during the kde5 cycle. it's inherent, after all. Actually kded using 100% cpu happened to me last week, when I was travelling. Took me some googling to figure out that it was some incompability with libntrack and that I needed to install that from source. I even had to use gnome as my desktop for a day before I had time to figure out what was going on... -- Boudewijn Rempt http://www.valdyas.org, http://www.krita.org, http://www.boudewijnrempt.nl
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday, 21 de August de 2011 16:40:32 Oswald Buddenhagen wrote: If there is, you return QString::fromRawData. uhm, no, you must make a deep copy, otherwise you get a time bomb. but yeah, no conversion, so pretty cheap. You can also use a QStringData with a regular refcount and just watch as it becomes 1. When all the QStrings in this segment of mmapped memory have refcount 1 or 0, then you can unmap it. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday, 21 de August de 2011 17:30:39 Oswald Buddenhagen wrote: That sounds very much like the GnomeOS idea... yes, it does. i'm fully sold on that idea. if kde is ever going to have any globally significant market share (*), then as applications and possibly an alternative shell - on top of gnome os. exclude from that specialized markets where a broad range of applications (and thus a workable 3rd party developer platform) is insignificant. Considering the audience and considering that KDE has more deployments than GNOME, why can't it be the other way around? Why do we need to drop what we already have and are ahead to gain more? You're not going to find supporters of that idea here. Therefore, we will continue to develop KDE Platform as the main desktop and require GNOME apps to integrate with us instead (and if they don't, we'll probably call their apps and frameworks badly designed -- which will be partly right and partly wrong and petty). -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
[: Thiago Macieira :]
I once wrote a benchmark comparing iterating over a QString to iterating
over a gchar UTF-8 string using glib functions to get each UCS-4 character
(ostensibly to prove that UTF-16 was better than UTF-8). The result was
clear: Qt code was much faster, over 10x, compared to glib.
[...]
The point is just as above: we, the KDE and Qt communities, are not
required to learn glib in order to write a new service or library.
Funilly, this is something that I was just intending to do for the sake of
acceptance. I started by looking for the necessary pieces in glib and how
they work, saw glib's iteration over characters, and thought this has got
to be mightily slower. And the code I want to write will mostly be doing
that.
Do you perhaps still have that benchmark code? Do you have (or know of)
similar benchmarks for XML parsing (GMarkupParser vs. QXmlStreamReader) and
JavaScript (QtScript vs. say SpiderMonkey)? Also, a QRegExp vs. GRegex
benchmark would be nice.
If the Qt-based implementation would be significantly superior in
performance, I gather you would advise just doing it and ignoring any but
C++... but another dependency... objections?
Also, with library being native C++, can there be any problem with C
bindings? I tried something like this:
// lib.h
class Foo {
public:
Foo (...);
void doSomething (...);
};
// lib.cpp
#include lib.h
Foo::Foo (...) { ... }
void Foo::doSomething (...) { ... }
// c_bindings.h
typedef struct Foo CFoo;
CFoo *foo_new (...);
void foo_dosomething (...);
// c_bindings.cpp
#include lib.h
extern C {
#include c_bindings.h
CFoo *foo_new (...) { return new Foo(...); }
void foo_dosomething (CFoo *a, ...) { a-doSomething(...); }
}
// app_main.c
#include c_bindings.h
int main ()
{
CFoo *f;
f = foo_new(...);
foo_dosomething(f, ...);
return 0;
}
I could build it with:
g++ lib.cpp
g++ c_bindings.cpp
gcc app_main.c lib.o c_bindings.o -lstdc++
but I don't know if this test covers everything.
--
Chusslove Illich (Часлав Илић)
signature.asc
Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 06:26:19PM +0200, Chusslove Illich wrote: Also, with library being native C++, can there be any problem with C bindings? once upon a time, there were (maybe still are, i dunno) more or less full c bindings for qt and kde, and they even served as a base for bindings to some script language. so it's doable, and the history of kdebindings should contain enough example code. and there are some libraries in actual use which are written in c++ but expose (only) a c api to the outside. so there is no technical reason why it wouldn't work. the only challenge is selling the qt dependency. :)
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sunday, 21 de August de 2011 18:26:19 Chusslove Illich wrote: Do you perhaps still have that benchmark code? Do you have (or know of) The code exists, but I don't have it. It's part of the QCharIterator work I did while at Nokia but never published, so I don't have rights to that code anymore. I'm sure I put it in the codereview tool before I left so it would be opensourced some day. similar benchmarks for XML parsing (GMarkupParser vs. QXmlStreamReader) and JavaScript (QtScript vs. say SpiderMonkey)? Also, a QRegExp vs. GRegex benchmark would be nice. XML parsing? no, I doubt. JS engine? Yes, a lot. That's why JavaScriptCore is being replaced with V8 now. QRegExp? don't try. The RE backend needs to be replaced. If the Qt-based implementation would be significantly superior in performance, I gather you would advise just doing it and ignoring any but C++... but another dependency... objections? C++ being a dependency is a very, very weak argument. There are lots of GNOME applications using gtkmm, pangomm, etc. C++ is present in most/all systems anyway, so it's not a dependency. The argument they may have is to Qt as a dependency, but as I said, that's not an argument for me. Qt being present and loaded into memory is baseline. Also, with library being native C++, can there be any problem with C bindings? I tried something like this: All problems with bindings can be solved. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 06:15:53PM +0200, Thiago Macieira wrote: On Sunday, 21 de August de 2011 17:30:39 Oswald Buddenhagen wrote: if kde is ever going to have any globally significant market share (*), then as applications and possibly an alternative shell - on top of gnome os. (*) exclude from that specialized markets [...]. Considering the audience and considering that KDE has more deployments than GNOME, why can't it be the other way around? this is where we started from. gnome is now serious about making a complete platform. if they succeed, they will win. big time. that's how apple did it, at the technical level. and how kde won't, due to lack of interest (commercial or otherwise) in that area. Why do we need to drop what we already have and are ahead to gain more? the point is that there isn't much to drop. it's some barely maintained code, and some informal specifications the best of which we could implement in gnome with relative ease if we actually cared to. You're not going to find supporters of that idea here. [...] i'm not sure whether you believe that this is how it should be or whether this is resignated cynicism.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. does the session systemd run privileged in the first place? I have no clue. I don't even know if there's a session systemd. I'm not sure exactly how you people are planning to make use of systemd; but hopefully: - It won't require the operating system to be running systemd as an init system. Any OS/user should be free to use a different/custom init system and still be able to use KDE. - It won't require root privileges to start KDE (including setuid programs). I don't see how dropping privileges from a user account wouldn't work (except perhaps the not implemented part, in which case the right way is to implement it). I suggest you think well whether systemd is indeed the right solution. As far as I see it, it was designed to be used for system services only, and not as a generic framework for controlling services and dependencies (hopefully I'm wrong). I've written some software which might also be used in this place. Please, read it through: http://code.google.com/p/badvpn/wiki/NCD . Yes, it doesn't have everything that would be needed to plug it into KDE right now, but it might be worth looking into because of its simplicity, compared to systemd. Regards, Ambroz
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 11:28 AM, Thiago Macieira thi...@kde.org wrote: On Sunday, 21 de August de 2011 10:19:26 Oswald Buddenhagen wrote: if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. does the session systemd run privileged in the first place? I have no clue. I don't even know if there's a session systemd. systemd can be run in session mode, and I assume this is what is being discussed for possibly replacing kded (using the system instance would not make much sense). The session instance does not have special privileges (just like the dbus session instance). It's great to see this being discussed, having the possibility of integrating systemd with kde would be truly awesome! Cheers, Tom
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sun, Aug 21, 2011 at 08:32:07PM +0200, Thiago Macieira wrote: On Sunday, 21 de August de 2011 19:13:43 Oswald Buddenhagen wrote: Considering the audience and considering that KDE has more deployments than GNOME, why can't it be the other way around? this is where we started from. gnome is now serious about making a complete platform. if they succeed, they will win. big time. that's how apple did it, at the technical level. and how kde won't, due to lack of interest (commercial or otherwise) in that area. You seem to imply that we're not serious about doing the same. it's not that we are not serious about it. we don't even try to. Your only argument so far was that we have less manpower than the GNOME team. no, my argument is the *complete* lack of manpower directed at creating a contemporary end-to-end experience for the user and 3rd party developer. we are doing a desktop and stuff on top of it. sometimes a bit of platform comes out of it as a side effect, but that's about it. If you count Canonical as the big driver for a cohesive desktop system, you should count them out already. i was actually thinking primarily of redhat. but that doesn't matter so much if the whole community buys into the idea. There's code that's barely maintained, true. I can think of KIO, for example. kio and kparts, just like qstyles and some other plugin systems we have are not *really* part of the os platform. as far as the user is concerned, only the settings which govern network behavior, widget looks, etc. and the url syntax are part of the platform; the implementations are exchangeable. from a 3rd party dev perspective a common programming platform would be desirable. but i cannot really assess how useful the ability to provide universally usable components would be. it always seemed a bit of a gimmick/niche market to me. Do you think all of Gtk and even glib is fully maintained? i don't consider the toolkits part of the os platform itself. they are available (the lsb says so) and some are used to build the os platform, but in principle they are exchangeable. Should we pull efforts together? Yeah. Not gonna happen, though. that's a rather sad conclusion.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Monday, 22 de August de 2011 00:07:54 Oswald Buddenhagen wrote: kio and kparts, just like qstyles and some other plugin systems we have are not really part of the os platform. as far as the user is concerned, only the settings which govern network behavior, widget looks, etc. and the url syntax are part of the platform; the implementations are exchangeable. from a 3rd party dev perspective a common programming platform would be desirable. but i cannot really assess how useful the ability to provide universally usable components would be. it always seemed a bit of a gimmick/niche market to me. Do you think all of Gtk and even glib is fully maintained? i don't consider the toolkits part of the os platform itself. they are available (the lsb says so) and some are used to build the os platform, but in principle they are exchangeable. Should we pull efforts together? Yeah. Not gonna happen, though. that's a rather sad conclusion. I'm sorry, I think I've only *now* got your point: your argument is that we're not trying to make system services for the platform and we don't have a master plan on how to get there. The example of KIO being that it's a great technology, but restricted to KDE and there's no effort to expand its userbase. Akonadi, Solid, Phonon and others are counter-examples: they were designed to be part of a platform. They don't have KDE dependencies in their core. Akonadi is especially an example of our trying to build a platform, since it was proposed to fd.o -- Solid and Phonon are more C++ frameworks abstracting other technologies. Also, I agree that Red Hat is pushing a lot of work into improving the platform. Can't fault their engineers for using glib when doing that. But I still think you give the GNOME team too much credit for having a master plan for a platform. From my experience, it's more like us: let's fix what we find broken. But unlike us, they're willing to go to a deeper level and fix the platform -- HAL, udisks, upower, polkit, consolekit, systemd, etc. Sometimes it seems like arrogance that they feel they own it all so they can change everything. But it can also be called boldness: if it's broken, let's redo it. So I agree we're missing a bit of that. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tue, Aug 16, 2011 at 08:58:02PM +0200, Thiago Macieira wrote: frameworks (qt-based), applications and workspace, that sounds pretty much what the KDE Platform is. What are you excluding in your definition? kded, klauncher, kdeinit, kglobalaccel, kwallet? yes, among other things. kded's module activation is redundant with systemd. as it happens, lennart also made a mini-daemon dealing with host name changes, which is another thing kded does. klauncher is conceptually redundant with systemd as well. the backend it uses is kdeinit, which *should* be in systemd - it's not like it would not benefit gnome as well (otherwise there would be no maemo booster) - we just need some fresh numbers to justify the hack, and some arguments why the hack isn't *that* bad after all. kwallet is already in the process of being replaced by a cross-desktop specification. i don't see why the backend needs to be desktop-specific, but i guess a monolithic implementation is easier, both technically and on a collaboration level. kglobalaccel would sorely need a cross-desktop solution on the backend side, too. but it gets more interesting. just look into systemsettings and see how many things are kde-specific while they should be global, often even below the desktop level. most of this doesn't need shared code; it could be covered by shared specification. but to do this, there needs to be a powerful shared configuration storage. dconf anyone?
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tue, Aug 16, 2011 at 08:00:19PM +0100, John Layt wrote: On Tuesday 16 Aug 2011 15:55:57 Oswald Buddenhagen wrote: On Tue, Aug 16, 2011 at 03:40:22PM +0200, Albert Astals Cid wrote: So you are going to let a guy that has stated publicly that hates KDE where has he done that? and i mean literally, not according to your interpretation. I've certainly seen him state that he doesn't care about KDE, that we are irrelevent to anything he does, and he sees no reason to collaborate on anything with us. and he's right. show me one case where the outcome of his low-level work did not meet kde requirements. It's similar to his attitude towards *BSD. he's right with that, too. if the bsd's (or solaris, for that matter) want to stay relevant on the desktop, they need to catch up with linux. or put differently, they make up about about 0.1% of our user base, which easily could be several times bigger if we were able to provide something well-integrated (mac os x anyone?). do you think they are worth sacrificing? Walking away from the platform [...] as it happens, i agree with all what you write. but that's very different from sticking to the kde platform. as you basically write yourself, the way to stay relevant is not by creating/maintaining redundancy, but by getting actively involved with the respective gnome(-driven) projects.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Wed, Aug 17, 2011 at 11:03:44AM +0200, Aaron J. Seigo wrote: Ossi's email demonstrates an utter lack of understanding on his part as to what kdelibs is. errrm ... right. i hope you had enough time now to rethink that statement. ;) we do not control all (or even most) of the Linux OS space. it's a little megalomaniacal to think we can or should and still be able to develop effective applications and solutions for our users. division of labour is pretty important in achieving that, particularly as F/OSS does not particularly have a huge amount of resources that we can afford to waste on duplication without serious benefit arrising from it. i'm not sure whether this was a taunt in gnome's direction or a complete misunderstanding of what i said ... but anyway: thanks for making my point better than i ever could have myself. :P imho in a more perfect world, the current efforts from the GNOME community would concentrate almost exclusively on the kind of middleware that they have shown an interest and resource readiness for and we'd concentrate almost exclusively on portable application libraries, applications and workspaces. i think about 90% of the gnome community would consider this a pretty ridiculous statement ...
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday, 20 de August de 2011 10:02:31 Oswald Buddenhagen wrote: On Tue, Aug 16, 2011 at 08:58:02PM +0200, Thiago Macieira wrote: frameworks (qt-based), applications and workspace, that sounds pretty much what the KDE Platform is. What are you excluding in your definition? kded, klauncher, kdeinit, kglobalaccel, kwallet? yes, among other things. kded's module activation is redundant with systemd. But not at all the same impact. Starting an application and negotiating its connection to D-Bus is hardly comparable to loading a plugin in a daemon that is already connected. as it happens, lennart also made a mini-daemon dealing with host name changes, which is another thing kded does. kdontchangethehostname klauncher is conceptually redundant with systemd as well. the backend it uses is kdeinit, which *should* be in systemd - it's not like it would not benefit gnome as well (otherwise there would be no maemo booster) - we just need some fresh numbers to justify the hack, and some arguments why the hack isn't *that* bad after all. That I agree: klauncher is systemd for KDE only, so we should see about getting the same benefits from systemd instead. There are two drawbacks with that, though: 1) systemd will not likely ever run on non-Linux systems, not even the BSDs. Lennart simply isn't interested on ensuring compatibility and might even reject patches which introduce differences to Linux-specific behaviour. 2) Lennart is also very much opposed to the fork-no-exec solution that kdeinit and booster use to pre-initialise. Prelinking, as we've discussed, is only a partial solution. This is something that requires rethinking everything and will involve a lot of fixing in all libraries. kwallet is already in the process of being replaced by a cross-desktop specification. i don't see why the backend needs to be desktop-specific, but i guess a monolithic implementation is easier, both technically and on a collaboration level. A cross-desktop specification, but we still use kwallet. There's no reason to dump it in favour of another implementation. So I see no arguments at all in favour of dropping it -- in fact, I see more in keeping it. kglobalaccel would sorely need a cross-desktop solution on the backend side, too. It needs a global spec too, since global shortcut grabbing with X11 libs only is sorely lacking. I think the solution we made for KDE 4 is actually quite good. Anyone wants to create an XDG spec for global accelerators? But just as before, I don't see why our code can't be cross-desktop. So no argument in favour of dropping kglobalaccel. but it gets more interesting. just look into systemsettings and see how many things are kde-specific while they should be global, often even below the desktop level. Yes. most of this doesn't need shared code; it could be covered by shared specification. but to do this, there needs to be a powerful shared configuration storage. dconf anyone? Yes. If you profile with massif any KDE application's startup, you see it consumes a lot of memory in KConfig and the QHashes it keeps. And remember that kDebug uses KConfig. So during the Summit I began thinking of a way to use memory-mapped binary config files -- dconf is the best solutoin for this. We already have a QtDConf implementation, so we can access the same data. The next step is to have a shared configuration key-set, what the gnomies call the schema (what keys store what properties and what their types are). To be honest, I've never looked into the schema in detail. One remaining issue is to have an alternate representation for certain types in the data, most importantly strings. As I provoked in the Enlightenment talk in the Summit, when Gustavo was calling for shared, binary formats, strings are UTF-16, right? -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday 20 August 2011 12:20:55 Thiago Macieira wrote: That I agree: klauncher is systemd for KDE only, so we should see about getting the same benefits from systemd instead. There are two drawbacks with that, though: 1) systemd will not likely ever run on non-Linux systems, not even the BSDs. Lennart simply isn't interested on ensuring compatibility and might even reject patches which introduce differences to Linux-specific behaviour. This is a point where we should start thinking how much sense it makes to still target non-Linux systems at all. If we can share more code and achieve more by switching to systemd and we decide to not do it and duplicate code because we want to support BSD, Solaris and $otherosnobodyuses we do IMHO a hughe mistake and harm ourselve. Systemd is not the only thing missing on non-linux, there is the complete graphics stack which is years behind (that means: Qt 5 won't support it), you still need hal, Wayland is impossible and so on and on. Given the number of bug reports and build-fix commits from non-Linux OS we receive for example in kwin there is no reason at all to target anything than Linux. Our system is fragmented enough even without supporting non-Linux systems. Summary: let's not discuss the loss of BSD support when discussing the switch to systemd. If BSD wants to still support KDE, they need to catch up fast and far. Cheers Martin signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Sat, Aug 20, 2011 at 12:20:55PM +0200, Thiago Macieira wrote: On Saturday, 20 de August de 2011 10:02:31 Oswald Buddenhagen wrote: kded's module activation is redundant with systemd. But not at all the same impact. Starting an application and negotiating its connection to D-Bus is hardly comparable to loading a plugin in a daemon that is already connected. yeah, at the cost of rather considerable instability. we should see whether regular service activation isn't the better option, now that we have it. That I agree: klauncher is systemd for KDE only, so we should see about getting the same benefits from systemd instead. There are two drawbacks with that, though: 1) systemd will not likely ever run on non-Linux systems, not even the BSDs. Lennart simply isn't interested on ensuring compatibility and might even reject patches which introduce differences to Linux-specific behaviour. this doesn't preclude system-specific forks, or even complete re-implementations. the one thing that is important is the d-bus interface. systems incapable of delivering that are simply going to be even less relevant than they already are. https://mail.gnome.org/archives/desktop-devel-list/2011-May/msg00427.html and the subthreads have all the gory details. fwiw, macosx has something similar - launchd. it is to be seen whether kde could make use of it. 2) Lennart is also very much opposed to the fork-no-exec solution that kdeinit and booster use to pre-initialise. yes, he made it pretty clear to me that he doesn't like it. but then, in retrospect, all his arguments seem pretty easy to refute, so if he is the reasonable person he appears to be, it should be possible to change his mind if we can make a strong technical case that it would also clearly benefit gnome. and in the worst case, the feature is optional and the patch to systemd wouldn't be extraordinarily big, so we could put it in the hands of distributors. A cross-desktop specification, but we still use kwallet. There's no reason to dump it in favour of another implementation. So I see no arguments at all in favour of dropping it -- in fact, I see more in keeping it. ksecretservice is a new implementation. dunno how much code was reused. the arguments against two backends are easy: - two backends means almost twice the work, including security auditing - assuming the backends use different storage (it's not part of the spec, after all), even after switching desktops you need to stick with the now alien password manager. kglobalaccel would sorely need a cross-desktop solution on the backend side, too. It needs a global spec too, since global shortcut grabbing with X11 libs only is sorely lacking. I think the solution we made for KDE 4 is actually quite good. Anyone wants to create an XDG spec for global accelerators? well, yeah. good luck to whoever. :P But just as before, I don't see why our code can't be cross-desktop. So no argument in favour of dropping kglobalaccel. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. seeing the reluctance to anything with g* within our community, why do you think the gnomers would embrace anything with a q* or even k*, esp. given that it usually weights in at least twice as much as the typical g* solution? One remaining issue is to have an alternate representation for certain types in the data, most importantly strings. As I provoked in the Enlightenment talk in the Summit, when Gustavo was calling for shared, binary formats, strings are UTF-16, right? that's a tough one. one can't have the binary dbs in utf-16, as then the g* apps would be at disadvantage. one could have the string data in both formats at once, but that would be wasteful. one could default to utf-8 and let apps switch it to utf-16 entirely. that would work for application-specific data, but it doesn't cover global settings. but then, we have the same problem with d-bus. so an entirely different approach i've been thinking about for a while: make QString a hybrid, so it stores raw data and a codec. it would transcode only on demand, so in many (possibly even most) cases, it would do 8-bit pass-through. of course, many common small operations would become more expensive. i'm not convinced that there would be an overall improvement. or even that the exercise makes sense in the first place.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Saturday, 20 de August de 2011 14:11:32 Oswald Buddenhagen wrote: On Sat, Aug 20, 2011 at 12:20:55PM +0200, Thiago Macieira wrote: On Saturday, 20 de August de 2011 10:02:31 Oswald Buddenhagen wrote: kded's module activation is redundant with systemd. But not at all the same impact. Starting an application and negotiating its connection to D-Bus is hardly comparable to loading a plugin in a daemon that is already connected. yeah, at the cost of rather considerable instability. we should see whether regular service activation isn't the better option, now that we have it. What instability? If kded crashes, what makes you think individual services won't crash, in addition to taking longer to load and use more memory? But I agree we should investigate. fwiw, macosx has something similar - launchd. it is to be seen whether kde could make use of it. Yup. 2) Lennart is also very much opposed to the fork-no-exec solution that kdeinit and booster use to pre-initialise. yes, he made it pretty clear to me that he doesn't like it. but then, in retrospect, all his arguments seem pretty easy to refute, so if he is the reasonable person he appears to be, it should be possible to change his mind if we can make a strong technical case that it would also clearly benefit gnome. and in the worst case, the feature is optional and the patch to systemd wouldn't be extraordinarily big, so we could put it in the hands of distributors. One problem with a no-exec scenario is the inability to switch security contexts and to control them on a per-binary case. Today, that is. There's nothing preventing additional code to be introduced to do that, if we start from an all-privileged daemon like systemd. It's privilege elevation that suffers. The argument Lennart has against fork-no-exec are the COW that happen inside the original launcher due to shared pages. During the forking, all of the original launcher's private pages are marked COW and then get shared with the child process. Whenever the launcher makes a new write to those pages due to its normal operations, the kernel will fault and trigger a copy. The end result is that the launcher has a lot more COW operations that aren't its fault and its pages are duplicated everywhere. If there's a privilege boundary in the process, then this might even be a security risk, as an unprivileged process gets read-only access to all of the launcher's memory and state, then it could mount a privilege escalation attack on the launcher itself. I believe a solution to this would involve unmapping all of the original launcher's private pages, but this is extremely hard to do given copy relocs (libc variables ending up in the launcher pages, for example) and other things. Not only would this be very much OS-specific, it might be architecture- specific. It's far easier to introduce the feature we want into the prelinker and dynamic loader, such as having a stasis process: the dynamic loader could load everything as needed, then freeze the images just prior to running any code. ksecretservice is a new implementation. dunno how much code was reused. the arguments against two backends are easy: - two backends means almost twice the work, including security auditing - assuming the backends use different storage (it's not part of the spec, after all), even after switching desktops you need to stick with the now alien password manager. Ok, but see below: But just as before, I don't see why our code can't be cross-desktop. So no argument in favour of dropping kglobalaccel. i think it is pretty clear that our *code* is not going to be accepted as the cross-desktop solution. seeing the reluctance to anything with g* within our community, why do you think the gnomers would embrace anything with a q* or even k*, esp. given that it usually weights in at least twice as much as the typical g* solution? I don't know where you got those numbers for weight, not to mention it's completely ambiguous (did you mean memory consumption?). But I don't care what they like or dislike: if the implementation is good, it should stay. If they afterwards want to reimplement the code using their technologies, it's their right, but they're the ones that created duplication. One remaining issue is to have an alternate representation for certain types in the data, most importantly strings. As I provoked in the Enlightenment talk in the Summit, when Gustavo was calling for shared, binary formats, strings are UTF-16, right? that's a tough one. one can't have the binary dbs in utf-16, as then the g* apps would be at disadvantage. I don't care. As long as there's a UTF-16 representation, I'm fine. That's why I'm calling this alternate representation. I didn't mean to say that it's UTF-16 exclusively. I just want UTF-16 to be there. I could even accept the alternate representation to be optional, so an UTF-8- to-UTF-16
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Wednesday, 17 de August de 2011 08:59:43 Simon Hausmann wrote: On Tuesday, August 16, 2011 09:05:53 PM ext Thiago Macieira wrote: On Tuesday, 16 de August de 2011 20:53:45 Alexander Neundorf wrote: Your main function will contain an absolute, non-relocatable address to a variable in the application's .data segment. Then, the .data segment contains an R_386_COPY (or R_ARM_COPY) relocation, that tells ld.so to get the value from libc.so.6, write it there, and then make every other ELF module refer to this symbol, not the one in libc.so.6. The same applies to another symbol familiar to us: QCoreApplication::self (a private, static variable accessed in the inline QCoreApplication::instance function). If PIE executables still have copy relocations, we cannot dlopen them. PIE and copy relocations sound fundamentally in conflict to me, given that it relies on the executable to be loaded at the correct address (to avoid a relocation in the executable itself). Otoh being loadable at any address is what PIE is all about, no? You're probably right. To make a relocation-free text, the access to variables must be made via the GOT, which in turn means it makes no difference whether the actual contents of the variable are in the application or in the library -- at least on 32-bit systems. Can you think of any other example where PIE would differ from PIC? I can come up with ideas, given that the documentation says PIE is a specialisation of PIC for the first-loaded module. One idea is that variables are moved and the compiler uses a simpler, 32-bit PC-relative relocation to access them, as opposed to a 64-bit indirect as would be expected. Another, and far more likely, is that the compiler and linker expect that none of the symbols in the executable be overridden by any library. That is, it probably applies -Bsymbolic or something equivalent. If you have an application or library that expects to be able to override non-weak symbols by interposition, this could cause issues. Fortunately, this is very rare and frowned upon. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, August 16, 2011 09:05:53 PM ext Thiago Macieira wrote:
On Tuesday, 16 de August de 2011 20:53:45 Alexander Neundorf wrote:
However, it's still not perfectly correct: the issue is the difference
between -fPIE and -fPIC. In a PIE, the compiler and linker *know* that
this ELF module is the first open loaded,
Sorry, I don't understand that sentence. Is there something missing ?
Linker as in build time linker, or the loader ?
Both.
Rephrasing: when you use -fPIE to compile and -pie to link, the compiler, the
build-time linker and the runtime linker are under the assumption that your
code is the first ELF module to be loaded. That means the compiler and the
build-time linker may make use of certain constructions or relocations that
only work in the first ELF module and break elsewhere.
One such example are copy relocations, at least in non-PIC non-PIE code. When
you write in your code:
int main()
{
errno = 0;
}
The variable errno is exported from libc.so.6. But instead of fetching the
address of that variable via GOT from libc.so.6, or even instead of leaving a
relocation for the address of that variable in your main function's code,
what
the linker does is *copy* the variable.
Your main function will contain an absolute, non-relocatable address to a
variable in the application's .data segment. Then, the .data segment contains
an R_386_COPY (or R_ARM_COPY) relocation, that tells ld.so to get the value
from libc.so.6, write it there, and then make every other ELF module refer to
this symbol, not the one in libc.so.6.
The same applies to another symbol familiar to us: QCoreApplication::self (a
private, static variable accessed in the inline QCoreApplication::instance
function).
If PIE executables still have copy relocations, we cannot dlopen them.
PIE and copy relocations sound fundamentally in conflict to me, given that it
relies
on the executable to be loaded at the correct address (to avoid a relocation in
the
executable itself). Otoh being loadable at any address is what PIE is all
about, no?
Can you think of any other example where PIE would differ from PIC?
Simon
P.S.: errno is a macro to (*errno_location()), pointing into TLS :)
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, August 16, 2011 20:58:02 Thiago Macieira wrote: frameworks (qt-based), applications and workspace, that sounds pretty muchwhat the KDE Platform is. What are you excluding in your definition? indeed; Ossi's email demonstrates an utter lack of understanding on his part as to what kdelibs is. he confuses it with trying to create an operating system user land stack as opposed to a set of application development tools which integrate with the OS and its userland. we target more than just Linux and we do not control all (or even most) of the Linux OS space. it's a little megalomaniacal to think we can or should and still be able to develop effective applications and solutions for our users. division of labour is pretty important in achieving that, particularly as F/OSS does not particularly have a huge amount of resources that we can afford to waste on duplication without serious benefit arrising from it. imho in a more perfect world, the current efforts from the GNOME community would concentrate almost exclusively on the kind of middleware that they have shown an interest and resource readiness for and we'd concentrate almost exclusively on portable application libraries, applications and workspaces. the line of cooperation and collaboration would be far clearer, our (KDE and GNOME's) resources much more efficiently used and our user and developer stories for F/OSS would be so much clearer. those who continue to cling to some mythical struggle over control of middleware when we are a portable, application-focused community are out of touch with reality. (and i would say similarly for those dealing with the complimentary issue ..) -- Aaron J. Seigo humru othro a kohnu se GPG Fingerprint: 8B8B 2209 0C6F 7C47 B1EA EE75 D6B7 2EB1 A7F1 DB43 KDE core developer sponsored by Qt Development Frameworks signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
Hi, On Wed, 17 Aug 2011, Thiago Macieira wrote: Can you think of any other example where PIE would differ from PIC? One idea is that variables are moved and the compiler uses a simpler, 32-bit PC-relative relocation to access them, as opposed to a 64-bit indirect as would be expected. Another, and far more likely, is that the compiler and linker expect that none of the symbols in the executable be overridden by any library. That is, it probably applies -Bsymbolic or something equivalent. If you have an application or library that expects to be able to override non-weak symbols by interposition, this could cause issues. Fortunately, this is very rare and frowned upon. advancing from the cave Speculations aside: the difference is that symbols defined in a DSO are interposable with PIC (when also linked with -shared) and non-interposable with PIE. There are no other differences. That happens to be the semantics that symbol resolution during loading is given for the first loaded DSO, which conventially is called 'executable', hence the name for this option. Note that linking with -shared also has the side effect of exporting all global symbols from the DSO (appropriately filtered through symbol version lists). It also is incompatible with -fPIE code, therefore by default a PIE compiled/linked executable doesn't export any symbols. See --dynamic-list. Again, that's just a side effect, the only real difference between PIC and PIE is the one above. /crawl back Ciao, Michael.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 22:56:06 Laszlo Papp wrote: Also: we need to be sure that prelinkers do prelink PIE, despite the article that Laszlo linked to. 1) prelink tries very very hard to skip PIE 2) ld.so ignores prelink information for PIE anyways so even if you force a PIE prelink you don't get anything There is no point in prelinking PIE since the prelink information is ignored. That's the opposite of what we want: We want a prelinked executable that can be dlopened. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tue, Aug 16, 2011 at 10:59:18AM +0200, Thiago Macieira wrote: In my opinion, kdeinit should stay. try to convince lennart of that. when i suggested to add kdeinit-like functionality to systemd his response was no way. and if we ignore systemd, we'll lose in the longer run. the selinux guys and some others hate us for kdeinit anyway. regarding nvidia, they know about the issue and have been enlightened about its severity? btw, why cannot non-pic libs be prelinked? works for non-pie executables, after all. Another advantage of kdeinit is to run the static constructors in libraries, [...] good luck with that... :}
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
Hi, kdeinit can be replaced by prelinking, assuming you are not a user of the NVidia binary drivers. If you are, you can't prelink, so kdeinit is a help: /usr/sbin/prelink: /usr/bin/gears: Cannot prelink against non-PIC shared library /usr/lib/nvidia-current/libGL.so.1 The nvidia driver does a lot of tricks and libGL is not a PIC library, but that does not matter. My understanding was that kdeinit cannot be replaced by prelinking in any case since they are two different things. In harmattan the preloader is called booster, and it basically does a dlopen() on important libraries and we also use prelinking. They live together. I think prelink and kdeinit could work together, but please fix me. Prelink or cross-prelink could just be used by the distributions (but it should be done after install, just like debian does, prelink is setup in cron) to make it clear. :) Unfortunately I missed this session because of the football match. I think kdeinit is a nice thing over there. PIE executables had some different purpose, mostly security related. For some reason, some people started abusing them. There was an article by Jakub explaining this. I can not seem to find it right now. In any case, the main advantage of PIE executables was that the executable address space could be randomized, and ldconfig does randomize the address. Basically, this makes buffer-overruns more difficult. One thing is that ld.so ignores prelink information for PIE executables. Other than this I really see no point in them Best Regards, Laszlo Papp
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
There was an article by Jakub explaining this. I can not seem to find it right now. Found! :) The first one: http://lwn.net/Articles/190495/ Why PIE should not be prelinked and in general about the main purpose of PIE. btw, why cannot non-pic libs be prelinked? works for non-pie executables, after all. Well, by definition, non-pic libraries cannot be prelinked since the symbols are at fixed addresses. You can not change the symbols using prelink. It is mentioned in the prelink manual [1], and prelink goes out of its way to find non-pic libraries and ignore them. You can just read the preface where it says why the libraries should be PIC, and how only PIC libraries can actually be prelinked. Oh, and one point. I am not familiar with systemd, but AFAIK too bloated as it is. It has a lot of code in it that should not be in the startup program, for instance: it has code to plymouth in it. That is a bit funny, but it might be just that I am lacking something... :) Best Regards, Laszlo Papp [1] http://docs.google.com/viewer?url=http%3A%2F%2Fpeople.redhat.com%2Fjakub%2Fprelink.pdf
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 12:50:47 Laszlo Papp wrote: Hi, kdeinit can be replaced by prelinking, assuming you are not a user of the NVidia binary drivers. If you are, you can't prelink, so kdeinit is a help: /usr/sbin/prelink: /usr/bin/gears: Cannot prelink against non-PIC shared library /usr/lib/nvidia-current/libGL.so.1 The nvidia driver does a lot of tricks and libGL is not a PIC library, but that does not matter. My understanding was that kdeinit cannot be replaced by prelinking in any case since they are two different things. They are different things, but in the end they are trying to solve the same problem: slowness at startup due to the loading of libraries and the processing of their relocations, plus sharing the otherwise non-sharable memory resulting from it. Unfortunately, comparing the results of the best that prelink can do (prelinked system, libs compiled with -Wl,-z,relro) and kdeinit, done by launching kwrite twice from the command-line and twice using kdeinit4_wrapper, I still see that kdeinit produces better results. My findings are: 1) the kdeinit executable is far larger in VSZ and just a bit larger in RSS (that's expected and is not a problem) 2) the kdeinit instance shares 32120 kB, whereas the non-kdeinit one shares 28536 kB. Again, no conclusion from here (the kdeinit instance is sharing memory that it won't need). 3) the kdeinit instance has 10120 kB non-shared, whereas the non-kdeinit one has 10296 kB. If we take away the heap and stack, we get the breakdown: kdeinit: 88 kB code, 244 kB RO data, 376 kB RW data non-kdeinit: 0 kB code, 620 kB RO data, 992 kB RW data All of the memory above is dirty, meaning in principle that it's non-sharable. That includes the 88 kB of code, which makes no sense at all to me. I'm guessing it's an effect of the prelinking, but still at loss to explain. Also note that those RO pages, due to the way that the loader and the linker work, do *not* include the .rodata sections. They are only the relro data -- that is, read-only data that requires relocations. In harmattan the preloader is called booster, and it basically does a dlopen() on important libraries and we also use prelinking. They live together. I think prelink and kdeinit could work together, but please fix me. Prelink or cross-prelink could just be used by the distributions (but it should be done after install, just like debian does, prelink is setup in cron) to make it clear. :) The booster also dlopens the plugins, I think, which further improves performance. The prelinker does not properly prelink plugins, since it doesn't know what uses them. That means a set of applications heavily dependent on plugins, like the Qt- and KDE-based apps, will still not be at ideal conditions with the prelinker. A workaround is to create a dummy binary or library that links to the plugins (violation of the definition, I know), so that the prelinker knows that it should work on them. Unfortunately I missed this session because of the football match. I think kdeinit is a nice thing over there. PIE executables had some different purpose, mostly security related. For some reason, some people started abusing them. There was an article by Jakub explaining this. I can not seem to find it right now. In any case, the main advantage of PIE executables was that the executable address space could be randomized, and ldconfig does randomize the address. Basically, this makes buffer-overruns more difficult. One thing is that ld.so ignores prelink information for PIE executables. Other than this I really see no point in them Another advantage of PIE is that the executable itself is position- independent, meaning that the code can be shared if more than one instance is loaded. For single-instance applications, this is not a gain. The disadvantage of PIE is the same of any position-independent code: one register is reserved (big problem on a register-starved CPU like the x86), data accesses are indirect, via the GOT, and function calls are also indirect via the PLT. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 13:16:44 Laszlo Papp wrote: btw, why cannot non-pic libs be prelinked? works for non-pie executables, after all. Well, by definition, non-pic libraries cannot be prelinked since the symbols are at fixed addresses. You can not change the symbols using prelink. It is mentioned in the prelink manual [1], and prelink goes out of its way to find non-pic libraries and ignore them. You can just read the preface where it says why the libraries should be PIC, and how only PIC libraries can actually be prelinked. Technically, there two dimensions here and the PIC / non-PIC naming is misleading. Code position: fixed / movable Code cleanliness: clean (no relocations) / dirty (has text relocations) -fPIC creates clean, movable code. Without it, shared libraries are dirty, but still movable. Non-PIE applications, however, are both dirty and fixed. So the NVidia libraries have text relocations, but they are still position- independent. They can be loaded anywhere. The prelinker should be able to prelink it. A prelinked, non-PIC library is just like a Windows DLL: the PIC register is free and, if the loader can load it at its preferred address, will not dirty its pages (i.e., will be sharable). -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tue, Aug 16, 2011 at 03:40:22PM +0200, Albert Astals Cid wrote: A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 10:59:18AM +0200, Thiago Macieira wrote: In my opinion, kdeinit should stay. try to convince lennart of that. when i suggested to add kdeinit-like functionality to systemd his response was no way. and if we ignore systemd, we'll lose in the longer run. So you are going to let a guy that has stated publicly that hates KDE where has he done that? and i mean literally, not according to your interpretation. decide KDE's future? it's a simple fact that gnome will determine the future of the linux desktop platform, simply because they have the people working on it and we don't. in fact, the pragmatic solution would be dropping the kde platform and concentrating on what we are good at: applications (and the underlying qt-based frameworks). and a workspace, for those 50% of our community who can believe in that plasma thingie.
Re: Re: Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 03:40:22PM +0200, Albert Astals Cid wrote: A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 10:59:18AM +0200, Thiago Macieira wrote: In my opinion, kdeinit should stay. try to convince lennart of that. when i suggested to add kdeinit-like functionality to systemd his response was no way. and if we ignore systemd, we'll lose in the longer run. So you are going to let a guy that has stated publicly that hates KDE where has he done that? and i mean literally, not according to your interpretation. decide KDE's future? it's a simple fact that gnome will determine the future of the linux desktop platform, simply because they have the people working on it and we don't. in fact, the pragmatic solution would be dropping the kde platform and concentrating on what we are good at: applications (and the underlying qt-based frameworks). and a workspace, for those 50% of our community who can believe in that plasma thingie. I can't answer to so much enthusiasm. Albert ___ Kde-buildsystem mailing list kde-buildsys...@kde.org https://mail.kde.org/mailman/listinfo/kde-buildsystem
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday 16 August 2011, Albert Astals Cid wrote: A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 03:40:22PM +0200, Albert Astals Cid wrote: A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 10:59:18AM +0200, Thiago Macieira wrote: In my opinion, kdeinit should stay. try to convince lennart of that. when i suggested to add kdeinit-like functionality to systemd his response was no way. and if we ignore systemd, we'll lose in the longer run. So you are going to let a guy that has stated publicly that hates KDE where has he done that? and i mean literally, not according to your interpretation. decide KDE's future? it's a simple fact that gnome will determine the future of the linux desktop platform, simply because they have the people working on it and we don't. in fact, the pragmatic solution would be dropping the kde platform and concentrating on what we are good at: applications (and the underlying qt-based frameworks). and a workspace, for those 50% of our community who can believe in that plasma thingie. I can't answer to so much enthusiasm. When looking at this statement carefully, applications, the underlying qt- based frameworks and a workspace is actually pretty much what we do. Alex
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday 16 August 2011, Thiago Macieira wrote: ... Another advantage of PIE is that the executable itself is position- independent, meaning that the code can be shared if more than one instance is loaded. For single-instance applications, this is not a gain. The disadvantage of PIE is the same of any position-independent code: one register is reserved (big problem on a register-starved CPU like the x86), data accesses are indirect, via the GOT, and function calls are also indirect via the PLT. ...not sure which email to reply too, so I just use this one. Maybe I didn't express clearly what I wanted to say, or I misunderstood Dirk, or Dirk was wrong. So, I try again, just to make sure there are no misunderstandings. Right now, for kdeinit-apps, we create a kdeinit module or plugin, which contains the actual application code, and additionally a tiny executable which loads this plugin and so runs the application. Beside that, we run a kdeinit instance, which can dlopen these plugins, and start the applications they contain. By doing this we save memory and time e.g. for relocations. The idea here was not to get rid of this mechanism completely. If I understood correctly, with a PIE executable it is possible to dlopen the executable and call a symbol from it. This would make it possible to simply create regular-looking, standalone executables instead of the combination of plugin+tiny executable, and at the same time keep the kdeinit instance around, which would then not dlopen the plugin, but dlopen the PIE executable, and call the symbol from the PIE executable instead of from the plugin, this way providing the same benefits as we have now. I.e. keep the mechanism, but without the necessity to split the executable into plugin+loader executable. Alex
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tue, Aug 16, 2011 at 07:24:19PM +0200, Alexander Neundorf wrote: When looking at this statement carefully, applications, the underlying qt- based frameworks and a workspace is actually pretty much what we do. we actually do a bit more, and the side threads of the recent systemsettings thread show that this is against the user's best interest (as kde failed to establish THE linux desktop platform as it was supposed to be). where exactly to draw the line between platform and just frameworks needs to be determined on a case by case basis, and depending on the cooperativeness of the respective gnomers.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 19:36:17 Alexander Neundorf wrote: The idea here was not to get rid of this mechanism completely. If I understood correctly, with a PIE executable it is possible to dlopen the executable and call a symbol from it. This would make it possible to simply create regular-looking, standalone executables instead of the combination of plugin+tiny executable, and at the same time keep the kdeinit instance around, which would then not dlopen the plugin, but dlopen the PIE executable, and call the symbol from the PIE executable instead of from the plugin, this way providing the same benefits as we have now. Ah, that makes a lot more sense. It's also closer to what the Harmattan booster does: it dlopens the actual binaries, not a kdenit module. However, it's still not perfectly correct: the issue is the difference between -fPIE and -fPIC. In a PIE, the compiler and linker *know* that this ELF module is the first open loaded, so they may take assumptions to that effect. In PIC, that isn't the case. So we're using an option that is specifically intended for stuff that is NOT dlopened. In practice, I don't know what optimisations may be attempted that would be problematic. It might just work, as Harmattan can show. Also: we need to be sure that prelinkers do prelink PIE, despite the article that Laszlo linked to. -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday 16 August 2011, Thiago Macieira wrote: On Tuesday, 16 de August de 2011 19:36:17 Alexander Neundorf wrote: The idea here was not to get rid of this mechanism completely. If I understood correctly, with a PIE executable it is possible to dlopen the executable and call a symbol from it. This would make it possible to simply create regular-looking, standalone executables instead of the combination of plugin+tiny executable, and at the same time keep the kdeinit instance around, which would then not dlopen the plugin, but dlopen the PIE executable, and call the symbol from the PIE executable instead of from the plugin, this way providing the same benefits as we have now. Ah, that makes a lot more sense. It's also closer to what the Harmattan booster does: it dlopens the actual binaries, not a kdenit module. However, it's still not perfectly correct: the issue is the difference between -fPIE and -fPIC. In a PIE, the compiler and linker *know* that this ELF module is the first open loaded, Sorry, I don't understand that sentence. Is there something missing ? Linker as in build time linker, or the loader ? Alex
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 16:55:57 Oswald Buddenhagen wrote: in fact, the pragmatic solution would be dropping the kde platform and concentrating on what we are good at: applications (and the underlying qt-based frameworks). and a workspace, for those 50% of our community who can believe in that plasma thingie. frameworks (qt-based), applications and workspace, that sounds pretty much what the KDE Platform is. What are you excluding in your definition? kded, klauncher, kdeinit, kglobalaccel, kwallet? -- Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org Software Architect - Intel Open Source Technology Center PGP/GPG: 0x6EF45358; fingerprint: E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358 signature.asc Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday 16 Aug 2011 15:55:57 Oswald Buddenhagen wrote: On Tue, Aug 16, 2011 at 03:40:22PM +0200, Albert Astals Cid wrote: A Dimarts, 16 d'agost de 2011, Oswald Buddenhagen vàreu escriure: On Tue, Aug 16, 2011 at 10:59:18AM +0200, Thiago Macieira wrote: In my opinion, kdeinit should stay. try to convince lennart of that. when i suggested to add kdeinit-like functionality to systemd his response was no way. and if we ignore systemd, we'll lose in the longer run. So you are going to let a guy that has stated publicly that hates KDE where has he done that? and i mean literally, not according to your interpretation. I've certainly seen him state that he doesn't care about KDE, that we are irrelevent to anything he does, and he sees no reason to collaborate on anything with us. It's similar to his attitude towards *BSD. For someone working on key parts of the shared stack it is a disappointing attitude to have. decide KDE's future? it's a simple fact that gnome will determine the future of the linux desktop platform, simply because they have the people working on it and we don't. in fact, the pragmatic solution would be dropping the kde platform and concentrating on what we are good at: applications (and the underlying qt-based frameworks). and a workspace, for those 50% of our community who can believe in that plasma thingie. Walking away from the platform will eventually leave us with no future other than chasing Gnome's tail-lights and patching around infrastructure that doesn't meet our needs. If we leave Gnome to define the platform on their own then we let them define the desktop, and then we might as well all pack up and go home. To remain relevent we need to not only remain engaged in defining the platform but increase our involvement, we've allowed too many of the recent platform changes to be defined without our involvement and our devs and users have suffered as a result. The DS BoFs I participated in show that in most areas we can work productively with the Gnome developers and that they can benefit from working with us, we need to push those relationship building efforts more and stay engaged. Yes, reality is a lot of the platform now comes from funded Gnome / Red Hat work that we may not have the money or manpower to match, but that just means we have to learn to work with them where appropriate and leverage off the work they do. The hard part is figuring out how to convince them that their solutions will be better with our involvement early on rather than tacked on at the end. John.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
On Tuesday, 16 de August de 2011 20:53:45 Alexander Neundorf wrote:
However, it's still not perfectly correct: the issue is the difference
between -fPIE and -fPIC. In a PIE, the compiler and linker *know* that
this ELF module is the first open loaded,
Sorry, I don't understand that sentence. Is there something missing ?
Linker as in build time linker, or the loader ?
Both.
Rephrasing: when you use -fPIE to compile and -pie to link, the compiler, the
build-time linker and the runtime linker are under the assumption that your
code is the first ELF module to be loaded. That means the compiler and the
build-time linker may make use of certain constructions or relocations that
only work in the first ELF module and break elsewhere.
One such example are copy relocations, at least in non-PIC non-PIE code. When
you write in your code:
int main()
{
errno = 0;
}
The variable errno is exported from libc.so.6. But instead of fetching the
address of that variable via GOT from libc.so.6, or even instead of leaving a
relocation for the address of that variable in your main function's code, what
the linker does is *copy* the variable.
Your main function will contain an absolute, non-relocatable address to a
variable in the application's .data segment. Then, the .data segment contains
an R_386_COPY (or R_ARM_COPY) relocation, that tells ld.so to get the value
from libc.so.6, write it there, and then make every other ELF module refer to
this symbol, not the one in libc.so.6.
The same applies to another symbol familiar to us: QCoreApplication::self (a
private, static variable accessed in the inline QCoreApplication::instance
function).
If PIE executables still have copy relocations, we cannot dlopen them.
--
Thiago Macieira - thiago (AT) macieira.info - thiago (AT) kde.org
Software Architect - Intel Open Source Technology Center
PGP/GPG: 0x6EF45358; fingerprint:
E067 918B B660 DBD1 105C 966C 33F5 F005 6EF4 5358
signature.asc
Description: This is a digitally signed message part.
Re: kdeinit (was: Summary from Buildsystem BoF at Desktop Summit)
Also: we need to be sure that prelinkers do prelink PIE, despite the article that Laszlo linked to. 1) prelink tries very very hard to skip PIE 2) ld.so ignores prelink information for PIE anyways so even if you force a PIE prelink you don't get anything There is no point in prelinking PIE since the prelink information is ignored. Best Regards, Laszlo Papp
