Re: Inter-WG coordination: Stable application runtimes
Le Lun 13 janvier 2014 01:37, Adam Williamson a écrit : On Sun, 2014-01-12 at 19:43 +0100, Reindl Harald wrote: Am 12.01.2014 19:39, schrieb Adam Williamson: Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? that has other reasons on a typical webserver you have not *the* wordpress and *the* wordpress plugins - you have 10,20,30,100,500 *independent* wordpress setups for production the only case where you can use distribution packages is if you have a dedicated machines / VM serving only one website Sure, but that just backs up my point further. Even in the case where you have an 'appliance' style setup, you _can_ use distro packages, but why would you bother? For the same reason you can plunk all sorts of out-of-tree modules in your kernel but if you want to keep sane you don't. Does not stop a lot of people for choosing the easy road to hell. That's nothing new or specific to Linux. Regards, -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Le Dim 12 janvier 2014 19:43, Reindl Harald a écrit : Am 12.01.2014 19:39, schrieb Adam Williamson: Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? that has other reasons on a typical webserver you have not *the* wordpress and *the* wordpress plugins - you have 10,20,30,100,500 *independent* wordpress setups And that's only because vm and container systems are taking time to mature. On a containerized infra with deduping storage the easiest setup will be to run single-site containers and stop worrying how all the instances interact with each other. -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Le Lun 13 janvier 2014 18:21, Colin Walters a écrit : Many upstream build/deployment systems have substantial portions of the metadata (BuildRequires/Requires) that RPM needs, it just needs to be manually maintained/duplicated in the spec. And they are usually missing substancial portions of the metadata rpm needs. Most of them are incomplete implementations that do not address all the problems distro packagers had to solve. So instead of the perenial let's drop rpm and use upstream incomplete systems I'd like to see the people working in those language communities work at adding the missing bits to those upstream systems so the mapping gets less incomplete and conversion can approach 1:1. That's the only way it can work: add the missing capabilities to the less-complete system. Regards, -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Am 14.01.2014 10:50, schrieb Nicolas Mailhot: Le Dim 12 janvier 2014 19:43, Reindl Harald a écrit : Am 12.01.2014 19:39, schrieb Adam Williamson: Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? that has other reasons on a typical webserver you have not *the* wordpress and *the* wordpress plugins - you have 10,20,30,100,500 *independent* wordpress setups And that's only because vm and container systems are taking time to mature. On a containerized infra with deduping storage the easiest setup will be to run single-site containers and stop worrying how all the instances interact with each other. this may be a solution for *some* environments but it will never be *the* solution my daily job is develop and deploy web-applications, keep them up-to-date and maintain the servers - only over my dead body i would start wrap more and more layers on top of already virtualized infrastructures one part of this all is having a central web-interface for configure the vhosts, link them to the billing and so on signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
only over my dead body i would start wrap more and more layers on top of already virtualized infrastructures Containers have little to almost no overhead, they bring more isolation (and i can't wait docker/selinux integration for more security), the FS layered approach allows to save spaces. Yeah, leveraging containers is a waste of time ... I agree with Nicolas, containers are still immature at the moment, but that's part of our mission as fedoraproject.org to make such technologies mature and bring them to rock-solid. best regards, H. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Am 14.01.2014 11:53, schrieb H. Guémar: only over my dead body i would start wrap more and more layers on top of already virtualized infrastructures Containers have little to almost no overhead, they bring more isolation (and i can't wait docker/selinux integration for more security), the FS layered approach allows to save spaces. Yeah, leveraging containers is a waste of time ... I agree with Nicolas, containers are still immature at the moment, but that's part of our mission as fedoraproject.org http://fedoraproject.org to make such technologies mature and bring them to rock-solid you stripped the most important part of my messages beause having 200 isolated containers is nice but you need to look at the big picture of a company and having a lot of vhosts is cleary company environment one part of this all is having a central web-interface for configure the vhosts, link them to the billing and so on signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
My apologies if you felt i misquoted you, i didn't intend that. I do plenty of SaaS deployments at $DAYJOB, and i can easily pack hundreds to thousands // running containers on a single machine. Remember that Fedora is on the innovative side of the distro spectrum, yes vhost is the present, but containers is the *future*. Even the enterprise distro are betting their chips on containers (SLES and Ubuntu have LXC commercial support, RHEL/Docker partnership, etc.) As part of the Cloud WG, it's something we definitively want to support and are looking on how we could leverage it (both this and Software Collections) H. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, 2014-01-14 at 11:00 +0100, Nicolas Mailhot wrote:
So instead of the perenial let's drop rpm and use upstream incomplete
systems
You might note I didn't say that.
I'd like to see the people working in those language communities
work at adding the missing bits to those upstream systems so the mapping
gets less incomplete and conversion can approach 1:1. That's the only way
it can work: add the missing capabilities to the less-complete system.
Yes - but the first thing that needs to happen is for rpm/Fedora to
support consuming any upstream metadata *at all*. Even if it was just
rpm/Fedora will assume %prep can be run with just @buildsys, and run
through /usr/share/rpm/frameworks/{autotools.sh,ruby.rb} for scripts to
extract metadata from the source.
--
devel mailing list
devel@lists.fedoraproject.org
https://admin.fedoraproject.org/mailman/listinfo/devel
Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Sun, Jan 12, 2014 at 04:39:12PM -0800, Adam Williamson wrote: You're preaching to the choir. But if in practice people really don't deploy things via the distribution packages, it doesn't matter how awesomely secure the distribution packages are. Something that you're not using is never providing you with any additional security. So for me, the question is: how can we make these things at least meet in the middle? Can we bring some of the distro benefits to the application deployment area? I think we can. Right now, I think Docker might be the most interesting approach for that (possibly future Docker with greater future OpenShift integration). This takes two things from Fedora (which are outside of the traditional distribution but can still easily fit under our umbrella). First: People can build Docker application containers with Fedora packages. But our packages (like all packages intended to be part of a whole system) are kind of badly suited for this -- they have dependencies on systemd, they expect a certain logging infrastructure, etc. We could have package variants meant for building app containers. (This'd be an interesting COPRs experiment.) Second: we could start building a library of pre-packaged application containers. Again, Docker makes an interesting and currently-hot technology to do this on (although it doesn't have to be Docker, of course). For example, an easy Fedora-based image with OwnCloud ready to go. -- Matthew Miller-- Fedora Project--mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Adam Williamson wrote: So to bring it to the context of Fedora.next - if some of the 'Fedora.next' products want to have the capability to deploy 'stable', i.e. bundled, stacks, then I think they should be 'allowed' to do so (in the sense that we can't really stop them), but the mechanisms used to do so should not be considered a part of the Fedora project, they should be upstream projects that are deployed on top of Fedora. +1, I agree with Adam's assessment. Some level of pragmatism should be included in future/next plans, finding balance between ideal/reality will be fun(1) -- rex (1) fun, hard, painful, whatever. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Mon, 2014-01-13 at 08:39 -0500, Matthew Miller wrote: On Sun, Jan 12, 2014 at 04:39:12PM -0800, Adam Williamson wrote: You're preaching to the choir. But if in practice people really don't deploy things via the distribution packages, it doesn't matter how awesomely secure the distribution packages are. Something that you're not using is never providing you with any additional security. So for me, the question is: how can we make these things at least meet in the middle? Can we bring some of the distro benefits to the application deployment area? One thing I would really like is improved tooling for mapping from upstream sources to RPMs that works *over time*. Right now tools like cpanspec exist, and you can use them one time, but Fedora currently rather insists that the spec file that lives in pkg git is canonical - it doesn't really work to attempt to rerun cpanspec. Many upstream build/deployment systems have substantial portions of the metadata (BuildRequires/Requires) that RPM needs, it just needs to be manually maintained/duplicated in the spec. (One concrete thing to make this work is that RPM needs the ability to look at the *unpacked* upstream sources before processing BuildRequires) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Adam Williamson wrote: I'm coming to the conclusion that at some point distros have to give up swimming against the tide and just say, look, if this is the way this ecosystem wants to go, then it's your problem. Fedora's job for such ecosystems would simply be to make sure their distribution mechanism works on top of Fedora - if it's one we care about - and then butt out. I'm not sure we're achieving anything practical for anyone by bending PHP / Java / nodejs / Ruby packages 120 degrees out of shape to conform to Fedora's packaging guidelines (often at the cost that we have to turn stuff off, or break stuff, or be months or years behind upstream, or be massively incomplete), and I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. At that point, we can just close shop, we are no longer fulfilling our role as a distribution. I also agree completely with Nicolas Mailhot's reply: That is NOT what our users want! Making the software conform to packaging best practices is exactly what we packagers are for. It is the only way to deploy software in a reliable, well-integrated, secure and space-efficient way. Having multiple competing deployment systems on the same machine invariably leads to integration issues (where the ones stacked on top can get surprised when a lower-level system changes or removes something like a system library, say because the user accidentally removed it, not knowing that something outside of the system package management requires it). And the security and disk space implications of bundling have been discussed many times already. So, like Matthew Miller, I think we cannot possibly punt on this issue, but I totally DISAGREE with his proposed solution of endorsing those bundling systems officially. Instead, we need to continue packaging things properly. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Sun, 2014-01-12 at 18:55 +0100, Kevin Kofler wrote: Adam Williamson wrote: I'm coming to the conclusion that at some point distros have to give up swimming against the tide and just say, look, if this is the way this ecosystem wants to go, then it's your problem. Fedora's job for such ecosystems would simply be to make sure their distribution mechanism works on top of Fedora - if it's one we care about - and then butt out. I'm not sure we're achieving anything practical for anyone by bending PHP / Java / nodejs / Ruby packages 120 degrees out of shape to conform to Fedora's packaging guidelines (often at the cost that we have to turn stuff off, or break stuff, or be months or years behind upstream, or be massively incomplete), and I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. At that point, we can just close shop, we are no longer fulfilling our role as a distribution. I also agree completely with Nicolas Mailhot's reply: That is NOT what our users want! Making the software conform to packaging best practices is exactly what we packagers are for. It is the only way to deploy software in a reliable, well-integrated, secure and space-efficient way. Having multiple competing deployment systems on the same machine invariably leads to integration issues (where the ones stacked on top can get surprised when a lower-level system changes or removes something like a system library, say because the user accidentally removed it, not knowing that something outside of the system package management requires it). And the security and disk space implications of bundling have been discussed many times already. So, like Matthew Miller, I think we cannot possibly punt on this issue, but I totally DISAGREE with his proposed solution of endorsing those bundling systems officially. Instead, we need to continue packaging things properly. Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Am 12.01.2014 19:39, schrieb Adam Williamson: Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? that has other reasons on a typical webserver you have not *the* wordpress and *the* wordpress plugins - you have 10,20,30,100,500 *independent* wordpress setups for production the only case where you can use distribution packages is if you have a dedicated machines / VM serving only one website signature.asc Description: OpenPGP digital signature -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On 10.01.2014 21:12, Matthew Miller wrote: On Thu, Jan 09, 2014 at 07:58:44PM -0800, Adam Williamson wrote: So the question becomes, what is it appropriate for a distribution to do in this situation? My personal opinion is that what's appropriate for a distribution to do is also, happily, what's easiest for a distribution to do: punt on it. Entirely punt on the whole thing. And, this ultimately makes a better experience for users, because if Fedora's included tools are aware of the native packaging system, we can do things like system auditing, security alerts (if not updates), maybe even integration with selinux policy. Basically, we don't hammer all of the possible universe into the distribution model, and we don't include all of the packages of everything in the base Fedora distribution as RPMs, but we do include those ecosystems in the Fedora _Project_, including the tools and documentation to make them feel natural. Your expression ... tools aware of the native packaging system and the Andrew comment about the pip behaviors above in the thread, encouraged me to share my big hammer of the DB plumber style :-) opinion on the problem. TL;DR: What follows is SCC: An idea for optional DB and service which caches bits from YumDB, the local state (RPMDB and /etc) plus the native systems like NPM in the unified way for the purposes of planning, resolving and performing system state transitions. Initially I meant to say just few words to mark the possibility, but obviously my English and talent for easy for reading and well focused messages are both far from good, so the whole thing become too long and I am going to split some additional notes into self replays - Excuses! RPMDB and YumDB are two rich datasets present on every Fedora instance representing respectively the local state and distribution+ state of the package universe. '+' because +Fusion*, +COPRs, +LocalOrg, etc. Unfortunately they are somewhat hidden in the dark, because lack of interfaces - we even missing SVG or other explorer for the YumDB graph. The (let think of them as virtual) Maven DB, PyPI DB, NPM DB, LuaRocks DB, etc, technically are subsets of YumDB (in sense richness of encoded logical relations between the DB nodes used in their schemes - e.g. PyPI DB, before the pip egg, do not knows which file from which module comes, nor have the concept of higher than package NV granularity of the requirement points - Provides and Requires in YumDB). Also, the depsolving of the native packaging systems is less sophisticated than both current yum and hawkey ones. These observation naturally lead to the idea of SCC: System Configuration Cache DB representing the merge of RPMDB, YumDB and e.g. local pip egs, PyPI DB (if e.g. additional python modules/versions are needed for the given deployment) where the depsolving could be shifted, somewhat in the same fashion as Data warehouse solutions are used in the enterprises for merging the significant datasets from various ERP systems into single DB for interactive time reporting and decision making. I am using the term SCC (vs. e.g. UPMC: Unified Package Metadata Cache) in attempt to cover a (paraphrased) Mirek question from the beginning of the thread - OK, we have Fedora and PyPI integrated, at one point of the time for a given instance, the Fedora packaged module has been chosen. What happens if we upgrade Fedora along with am incompatible version this Python module for given installed service - obviously we need to register in the SCC the dependencies of the installed machine roles with the same effect like Require clauses in the our packages, so the SCC machinery to validate (with negative result in this described case) the yum upgrade or to resolve the upgrade including installation of newest available compatible version from PyPI as an alternative provision during upgrade preparation. Further, I think that ideally SCC should parse/absorb as much as possible system object properties and relations from /etc (plus /lib and /var configuration areas) to allow sysadmins and devops to inject rules for effective use of these sets latter in the depsolving (and other DB functionality). That said, the integration with OpenLMI, or even implementing the whole thing under the OpenLMI umbrella, both seems natural. So, finally on that road we have: - choice for good enough DB engine for SCC, query language, compiler [*], etc. design decisions like sync protocol and plugable data sources. - Yum/RPM datasets: optional rpm, yum, hawkey hooks for syncing their DBs, alternatively just sync tools. - optional pip, npm, luarocks, etc. hooks for the same, alternatively just sync tools. - OpenLMI integration for absorbing system configuration, alternatively just Augeas import + transformation rules to sync the DB representation of the system objects. - sccd capable to: - depsolving (on top of cumulative - YumDB + native managers DBs, preferably providing interfaces to the system
Re: Inter-WG coordination: Stable application runtimes
On 12.01.2014 22:34, Alek Paunov wrote: [*] Crucial aspect of any sophisticated data management system is the data query and manipulation language. Unfortunately the choices are rather limited - Imperative approaches (recently resurrected by some NoSQL DBs) are weak and error prone; SQL and few more text prose languages have proven their incompatibility with the vast majority of the developers (these without years of specific experience around the data volumes processing). The predominant workaround seems to be ORMs, but ORMs and sophisticated/fast should not be mixed in same project :-). My personal preference leans towards rules approach (which e.g. is also adopted by at least one of the ERP innovation leaders - LogicBlox) and especially its variant of visual rules definition UIs, where the user describes dependencies (relations) between source and result trees of a operations using blocks and arrows, and then the compiler lowers the the whole transaction definition to executable (by the DB engine), procedure. IMHO, such kind of visual interface would be one of the possible adequate languages (Adequate to the DB specialization level of our target audience, including big share of the developers). My personal preferences for the lower level executable language and DB engine are LuaJIT procedures on top of SQlite4/LSM C API. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes - SCC: Fedora social
On 12.01.2014 22:34, Alek Paunov wrote: - sccd-web: WebUI exposing full functionality, alternatively Cockpit (OpenLMI WebUI) extension. ... - NTH: SCC local state inheritance between instances Fedora Social: Almost every developer or sysadmins like to demonstrate how clean and clever is his own design :-). Currently we do not have a service where Sandra, Joe and George [1] could: - show and share with the others their Fedora based setups - conveniently keep the setup for their own reuse in the future Once we have sccd-web and few NTHs, we will be nearly (at least as code) equipped to provide something like github for Fedora setups for publishing, referring in the e-mails and forking Fedora users work. [1] http://fedoraproject.org/wiki/Server/Personas -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes - SCC
On Jan 12, 2014, at 1:51 PM, Alek Paunov a...@declera.com wrote: Once we apply FS snapshotting, combined with the SCC NTHs above, there are at least two appealing use-cases: - reusing one base e.g. F20 server container image for both the host and the incompatible containers (e.g. when one application requires nodejs 0.8 and another nodejs 0.10) - easy testbed for resolving the upgrade path for an existing, possibly non pure Fedora server with care about the deployed services. I agree, there are quite a few use cases for file system snapshots. Do the snapshots need to be bootable? Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Sun, 2014-01-12 at 19:43 +0100, Reindl Harald wrote: Am 12.01.2014 19:39, schrieb Adam Williamson: Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? that has other reasons on a typical webserver you have not *the* wordpress and *the* wordpress plugins - you have 10,20,30,100,500 *independent* wordpress setups for production the only case where you can use distribution packages is if you have a dedicated machines / VM serving only one website Sure, but that just backs up my point further. Even in the case where you have an 'appliance' style setup, you _can_ use distro packages, but why would you bother? If it's a single-purpose system then you don't get much benefit from doing so. Does RH base pre-rolled openshift gears on Fedora or RHEL distribution packages? I don't think so. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Sun, 2014-01-12 at 20:58 +0100, Till Maas wrote: On Sun, Jan 12, 2014 at 10:39:19AM -0800, Adam Williamson wrote: On Sun, 2014-01-12 at 18:55 +0100, Kevin Kofler wrote: So, like Matthew Miller, I think we cannot possibly punt on this issue, but I totally DISAGREE with his proposed solution of endorsing those bundling systems officially. Instead, we need to continue packaging things properly. Have you looked at what people are installing on Fedora lately? Have you looked at how much PHP stuff there is out there vs. what we have packaged 'properly'? Java? Ruby? Do you know anyone who deploys Wordpress plugins via distribution packages? Even if people do it, it does not meant that it is the best way to do it. Mixed packaging makes it a lot harder to properly update in case of security vulnerabilities. E.g. instead of only checking/ensuring proper RPM updates one need to check each distribution method for regular updates. Is there even some tooling available to check/update all e.g. rbenv or virtualenv setups properly? You're preaching to the choir. But if in practice people really don't deploy things via the distribution packages, it doesn't matter how awesomely secure the distribution packages are. Something that you're not using is never providing you with any additional security. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, Jan 9, 2014 at 9:04 PM, Adam Williamson awill...@redhat.com wrote: I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. For what it's worth, I read over the GitHub tickets, I think you're headed in the right direction, and I appreciate the work you've done. I think the key is that these sort of packaging efforts can't be done over a Christmas vacation, or even an entire Summer of Code. For these larger apps, it's just going to take years. A lot of gentle, consistent help coming from our side, over a long time, makes the difference. - Ken -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
I'm sorry, but you're forgetting one major thing. Sure there are lots of developers that ignore best practices. There is nothing new. But there is also a lot of users that do understand best practices and do want the apps packaged in a clean way. The apps are not packaged because the developers wanted them packaged they are packaged because users wanted them. The opinion of developers on deployment issues is, frankly, 100% irrelevant. The vast mass of developers that don't want to think about deployment issues are not the people you should base your decisions about deployment on. The developers that did think about the problem, do not react as the people you take as example. They have the option to ignore distro packaging, and they are execising it, and they complain that users are not happy about it, but repainting deployment in developer colors when that is not what the users ask for is not going to make anyone happy. The user will complain about a result that does not match his expectations and think the distro sucks. The developer will complain users still complains and consider the distro still suck. End result: a lot of work for no one happy. All the energy expended on scl and making developer-friendly deployments is like asking designers to correct application code, coders to make design decisions, marketing people to make engineering decisions or engineers to behave as salespeople. All those are different jobs. All those require prioritising different elements. -- Nicolas Mailhot -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
tor 2014-01-09 klockan 20:30 -0800 skrev Andrew Lutomirski: It would be nice, at least, if there was a clean way for these stacks to be tracked and, if needed, uninstalled. Some of these things install into /usr, which is a giant mess. (Pip, the one I use the most, doesn't do that IIRC, but it's still annoying that, if I install a package with pip, that package *automatically*, *without prompting*, and (I think) without verifying signatures or any sort, will pull in dependencies from pypi that could be satisfied by yum. If I then install the yum version, I end up in a weird state. There are systems like virtualenv (python) and local::env (perl) that mirror the base distro in a separate directory and then let the user install modules and apps on top, without touching the distro-controlled directories. This is in my opinion the only sane way to use pip and CPAN, but in can be improved: What happens if you add/remove/update a distro package after creating a virtualenv? Add and update might be ok, but remove will quite likely break the app. What about apps that use more than one stack? Can a unified tool that mirror the whole distro be created? It might be as simple as combining the existing tools for each stack. Sane defaults for directories: I've found that when using virtualenv to install a web app, SELinux will complain less if you put the base directory somewhere in /var/www. What is the right place to put these stacks? Codify this in the packaging and in SELinux so that it just works. I'd like some way (maybe using something like mock) to manage these things in a somewhat sandboxed way. Docker is trying to do this, but I think it's the wrong approach for a lot of use cases, and its nowhere near ready for prime time. But once you've considered every aspect (for example my points above), you've basically reinvented Docker anyway. /Alexander -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, Jan 09, 2014 at 07:58:44PM -0800, Adam Williamson wrote: So the question becomes, what is it appropriate for a distribution to do in this situation? My personal opinion is that what's appropriate for a distribution to do is also, happily, what's easiest for a distribution to do: punt on it. Entirely punt on the whole thing. I agree with everything you write, except for your conclusion. Or, maybe it's a matter of semantics. Either way, I think the *Fedora Project* can't afford to punt. Here's my reasoning... First, here's our mission: The Fedora Project's mission is to lead the advancement of free and open source software and content as a collaborative community. That might be a bit lofty -- not to mention ultimately vague and unmeasurable -- but it clearly sets our objective as beyond just the lower levels of a base distribution. Clearly, producing a distribution is our primary output, but we shouldn't lose sight and start thinking that what we do is the goal in and of itself. So much of the interest and enthusiasm and shear time spent in the free / open source software world is in these software stack layers and applications built on them that in order approach the mission at all we need to at least have something meant to address them. Second, if we did decide to constrict the mission to something more pragmatic and distribution-focused (like the original Fedora Project mission, inherited directly from the short-lived Red Hat Linux Project), we'd be writing ourselves into obscurity. The base Linux distribution is becoming a commodity. When I was at the Amazon web services conference last year, I asked dozens of people why they were using the Linux distribution they were, and the overwhelming answer was Oh, I actually don't care. Now, we can certainly do things to improve our production of the commodity... but, really, where's the fun in limiting ourselves to that? The interesting problems are higher up. (Not to say that everything is solved at the base layer, of course. There's still a lot going on -- this year and last, for example, it's all about containers... which of course very quickly ties back into needing something to go in those containers, and this whole higher-level question.) I said I agree with you, and one particular place I agree is that we can't come up with and dictate a Single Bundled Stack Deployment Mechanism To Rule Them All (SBSDMTRTA?). That *is* something that's part of the higher-up ecosystems. However, we need to find better ways to support and interface with those ecosystems -- that's where we can make Fedora (both the distro and the project) really compelling in the future. And, this ultimately makes a better experience for users, because if Fedora's included tools are aware of the native packaging system, we can do things like system auditing, security alerts (if not updates), maybe even integration with selinux policy. Basically, we don't hammer all of the possible universe into the distribution model, and we don't include all of the packages of everything in the base Fedora distribution as RPMs, but we do include those ecosystems in the Fedora _Project_, including the tools and documentation to make them feel natural. And, if people in the project do want to keep hammering things into RPMs -- fine, no reason to stop what some people clearly believe is still useful. Likewise, if people want to come up with other novel ways of approaching the problem, like SCLs or whatever else, I think we should find a space for it. Again, it doesn't need to be in the Fedora Distribution Per Se, but there should be room in *Fedora*. -- Matthew Miller-- Fedora Project--mat...@fedoraproject.org -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Fri, 2013-12-20 at 00:03 +0100, Kevin Kofler wrote: Jóhann B. Guðmundsson wrote: In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. Personally I think we should entirely drop any notion of implementing software collection in Fedora on rpm bases ( could be implemented as standalone app image ) since we dont need it RHEL does [1] and go directly looking at implementing Alexander's proposal regarding app images. Argh, no thanks! App images are even worse than SCLs. They are a security nightmare, an immense waste of disk space and RAM, and just generally a totally broken concept. Applications need to stop thinking they are a distribution! Such applications-that-think-they're-distros are a horrible mess to deal with, and generally the only sane way to handle them is to untangle that distro-in-a-distro mess and package them as normal applications. (See e.g. what we have done to SAGE(Math) to package it for Fedora. Now it is a sane package using system versions of its dependencies rather than a huge multi-hundred-MiB tarball bundling even things like Python and Maxima!) Coming to this discussion late, but I do think it's an interesting one. There's a deep question lurking behind the scenes here, which is: what is the distro's responsibility, exactly? People are already deploying static bundled stacks on Fedora. There are large ecosystems where this has become the norm: Javaland, PHPland, Rubyland. A lot of people don't deploy their Java or PHP or Ruby apps from their distribution's repositories, they use an overlaid distribution mechanism of some kind which promotes the use of bundled dependencies to provide a 'stable' stack. You may think they're wrong, I may think they're wrong, but they're doing it. It is very very difficult to 'convert' these entire ecosystems into sets of packages that obey the traditional policies of Linux distributions regarding bundling; in practice it's probably impossible to do so in a way that people actually involved in those ecosystems are happy with, which is why they bypass distro mechanisms. We don't have anywhere near a full Ruby or PHP or Java ecosystem packaged for Fedora, and the packages we do have are frequently broken or outdated, precisely because of this major mismatch between how we do things and how those ecosystems do things. So the question becomes, what is it appropriate for a distribution to do in this situation? My personal opinion is that what's appropriate for a distribution to do is also, happily, what's easiest for a distribution to do: punt on it. Entirely punt on the whole thing. There are already multiple mechanisms for the deployment of bundled software stacks, many associated with a given ecosystem - Composer for PHPland, npm for NodeJSland, bundler for Rubyland, etc etc. We've seen that GNOMEland is apparently looking down a similar path for deploying 'app bundles', and will no doubt come up with its own mechanism. I doubt a distribution can come up with a Single Bundled Stack Deployment Mechanism To Rule Them All, or something like that, and it is a layering violation for this kind of mechanism to be owned by a distribution: they should be - and in practice *are* - owned by their ecosystems. I'm coming to the conclusion that at some point distros have to give up swimming against the tide and just say, look, if this is the way this ecosystem wants to go, then it's your problem. Fedora's job for such ecosystems would simply be to make sure their distribution mechanism works on top of Fedora - if it's one we care about - and then butt out. I'm not sure we're achieving anything practical for anyone by bending PHP / Java / nodejs / Ruby packages 120 degrees out of shape to conform to Fedora's packaging guidelines (often at the cost that we have to turn stuff off, or break stuff, or be months or years behind upstream, or be massively incomplete), and I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. Distros can work with ecosystems - like the roughly defined 'basic *nix userland' ecosystem - which are written in C and C++ and Bash and Python and Perl, and where libraries understand the value of well-defined and widely observed policies regarding API/ABI stability and library versioning and so on. I suspect the only way distros can really 'work' with ecosystems that don't do so is to stop trying and leave them to it. So to bring it to the context of Fedora.next - if some of the 'Fedora.next' products want to have the capability to deploy 'stable', i.e. bundled, stacks, then I think they should be 'allowed' to do so (in the sense that we can't really stop them), but the mechanisms used to do so
Re: Inter-WG coordination: Stable application runtimes
On Thu, 2014-01-09 at 19:58 -0800, Adam Williamson wrote: I'm coming to the conclusion that at some point distros have to give up swimming against the tide and just say, look, if this is the way this ecosystem wants to go, then it's your problem. Fedora's job for such ecosystems would simply be to make sure their distribution mechanism works on top of Fedora - if it's one we care about - and then butt out. I'm not sure we're achieving anything practical for anyone by bending PHP / Java / nodejs / Ruby packages 120 degrees out of shape to conform to Fedora's packaging guidelines (often at the cost that we have to turn stuff off, or break stuff, or be months or years behind upstream, or be massively incomplete), and I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. A further point on this: I think distros can probably provide value to these stacks if they're clever and take a strategic approach. To take the example of PHPland, which is the one I've been dealing with the most lately, there are some libraries and frameworks which are required by a *lot* of PHP project, and which do have a _reasonable_ approach to stability. It's probably the case that Fedora can provide value by, for instance, providing packaged versions of Zend and Doctrine and Symfony - major stacks/libs/frameworks which don't break their APIs every Wednesday, have reasonable distribution mechanisms, and which quite a lot of popular projects depend on. But when I'm sitting here building a package for some garbage PHP 'library' which doesn't have a release mechanism or a versioning policy or any concept of API stability - where all you have is a git repo with one or two branches into which they regularly stuff anything from a bug fix to a major refactoring - I wonder exactly who I'm helping. Probably there are three projects in the world which use that library and each one uses a different random git checkout from the others which is in no way compatible. When you're working in that kind of context, trying to apply distribution packaging rules is like showing up to a UFC fight with a copy of the Queensberry rules and attempting to convince everyone they're doing it wrong: you're not really doing anything of any use to anyone involved. -- Adam Williamson Fedora QA Community Monkey IRC: adamw | Twitter: AdamW_Fedora | XMPP: adamw AT happyassassin . net http://www.happyassassin.net -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, Jan 9, 2014 at 7:58 PM, Adam Williamson awill...@redhat.com wrote: On Fri, 2013-12-20 at 00:03 +0100, Kevin Kofler wrote: Jóhann B. Guðmundsson wrote: In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. Personally I think we should entirely drop any notion of implementing software collection in Fedora on rpm bases ( could be implemented as standalone app image ) since we dont need it RHEL does [1] and go directly looking at implementing Alexander's proposal regarding app images. Argh, no thanks! App images are even worse than SCLs. They are a security nightmare, an immense waste of disk space and RAM, and just generally a totally broken concept. Applications need to stop thinking they are a distribution! Such applications-that-think-they're-distros are a horrible mess to deal with, and generally the only sane way to handle them is to untangle that distro-in-a-distro mess and package them as normal applications. (See e.g. what we have done to SAGE(Math) to package it for Fedora. Now it is a sane package using system versions of its dependencies rather than a huge multi-hundred-MiB tarball bundling even things like Python and Maxima!) Coming to this discussion late, but I do think it's an interesting one. There's a deep question lurking behind the scenes here, which is: what is the distro's responsibility, exactly? People are already deploying static bundled stacks on Fedora. There are large ecosystems where this has become the norm: Javaland, PHPland, Rubyland. A lot of people don't deploy their Java or PHP or Ruby apps from their distribution's repositories, they use an overlaid distribution mechanism of some kind which promotes the use of bundled dependencies to provide a 'stable' stack. You may think they're wrong, I may think they're wrong, but they're doing it. It is very very difficult to 'convert' these entire ecosystems into sets of packages that obey the traditional policies of Linux distributions regarding bundling; in practice it's probably impossible to do so in a way that people actually involved in those ecosystems are happy with, which is why they bypass distro mechanisms. We don't have anywhere near a full Ruby or PHP or Java ecosystem packaged for Fedora, and the packages we do have are frequently broken or outdated, precisely because of this major mismatch between how we do things and how those ecosystems do things. So the question becomes, what is it appropriate for a distribution to do in this situation? My personal opinion is that what's appropriate for a distribution to do is also, happily, what's easiest for a distribution to do: punt on it. Entirely punt on the whole thing. There are already multiple mechanisms for the deployment of bundled software stacks, many associated with a given ecosystem - Composer for PHPland, npm for NodeJSland, bundler for Rubyland, etc etc. We've seen that GNOMEland is apparently looking down a similar path for deploying 'app bundles', and will no doubt come up with its own mechanism. I doubt a distribution can come up with a Single Bundled Stack Deployment Mechanism To Rule Them All, or something like that, and it is a layering violation for this kind of mechanism to be owned by a distribution: they should be - and in practice *are* - owned by their ecosystems. I'm coming to the conclusion that at some point distros have to give up swimming against the tide and just say, look, if this is the way this ecosystem wants to go, then it's your problem. Fedora's job for such ecosystems would simply be to make sure their distribution mechanism works on top of Fedora - if it's one we care about - and then butt out. I'm not sure we're achieving anything practical for anyone by bending PHP / Java / nodejs / Ruby packages 120 degrees out of shape to conform to Fedora's packaging guidelines (often at the cost that we have to turn stuff off, or break stuff, or be months or years behind upstream, or be massively incomplete), and I say this as someone who's spent the last couple of weeks whacking on a PHPland stack (Owncloud) with a wrench to achieve precisely that. Distros can work with ecosystems - like the roughly defined 'basic *nix userland' ecosystem - which are written in C and C++ and Bash and Python and Perl, and where libraries understand the value of well-defined and widely observed policies regarding API/ABI stability and library versioning and so on. I suspect the only way distros can really 'work' with ecosystems that don't do so is to stop trying and leave them to it. So to bring it to the context of Fedora.next - if some of the 'Fedora.next' products want to have the capability to deploy 'stable', i.e. bundled, stacks, then I
Re: Inter-WG coordination: Stable application runtimes
On 12/19/2013 05:33 PM, Michael Catanzaro wrote: On Thu, 2013-12-19 at 09:52 -0500, Orcan Ogetbil wrote: Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? Standard C++ does not specify an ABI. Compilers get to handle that themselves. We adhere to the Itanium C++ ABI. It's not what GCC happens to implement today anymore. -- Florian Weimer / Red Hat Product Security Team -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Wed, Dec 18, 2013 at 6:05 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. How would one _in practice_ do that? What tools are available? Would each third party have to individually backport security patches? Would they? Personally I think we should entirely drop any notion of implementing software collection in Fedora on rpm bases ( could be implemented as standalone app image ) since we dont need it RHEL does [1] and go directly looking at implementing Alexander's proposal regarding app images. It seems Fedora Server needs similar functionality just as well: we can't have long-term installations without either a LTS or providing stable application runtimes. Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Wed, Dec 18, 2013 at 9:45 AM, Kevin Kofler wrote: You can, for example, use C++, which is a stable standard (a new version was published 2 years ago, but almost all C++98 code compiles unchanged as C++11), and Qt, which keeps a stable API and ABI throughout a major version (the interval between the last 2 major versions having been over 7 years), Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? Orcan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On fim 19.des 2013 14:40, Miloslav Trmač wrote: On Wed, Dec 18, 2013 at 6:05 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. How would one _in practice_ do that? The configuration files as well as the application data used by the application is stored outside the container and would be accessed through a portal. What tools are available? None that I'm aware of at this point in time. Would each third party have to individually backport security patches? Would they? That depends if an container spawned on the OS or desktop layer and would reuse what already exist and is provided in the system vs what would be an isolated application stack which by itself would provide everything necessary for it to run. In the case of the latter I would think that they would have to release a new updated image to deploy It seems Fedora Server needs similar functionality just as well: we can't have long-term installations without either a LTS or providing stable application runtimes. Right. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, 2013-12-19 at 09:52 -0500, Orcan Ogetbil wrote: Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? Standard C++ does not specify an ABI. Compilers get to handle that themselves. signature.asc Description: This is a digitally signed message part -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Jóhann B. Guðmundsson wrote: In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. Personally I think we should entirely drop any notion of implementing software collection in Fedora on rpm bases ( could be implemented as standalone app image ) since we dont need it RHEL does [1] and go directly looking at implementing Alexander's proposal regarding app images. Argh, no thanks! App images are even worse than SCLs. They are a security nightmare, an immense waste of disk space and RAM, and just generally a totally broken concept. Applications need to stop thinking they are a distribution! Such applications-that-think-they're-distros are a horrible mess to deal with, and generally the only sane way to handle them is to untangle that distro-in-a-distro mess and package them as normal applications. (See e.g. what we have done to SAGE(Math) to package it for Fedora. Now it is a sane package using system versions of its dependencies rather than a huge multi-hundred-MiB tarball bundling even things like Python and Maxima!) Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Orcan Ogetbil wrote: Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? The latest C++ ABI change was with g++ 3.4 (April 18, 2004). g++ 4.0 kept the 3.4 ABI and so did all 4.x releases. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, Dec 19, 2013 at 4:31 PM, Kevin Kofler kevin.kof...@chello.at wrote: Orcan Ogetbil wrote: Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? The latest C++ ABI change was with g++ 3.4 (April 18, 2004). g++ 4.0 kept the 3.4 ABI and so did all 4.x releases. He's probably talking about this: https://lists.fedoraproject.org/pipermail/devel/2012-February/163320.html -- Jerry James http://www.jamezone.org/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Thu, Dec 19, 2013 at 11:33 AM, Michael Catanzaro wrote: On Thu, 2013-12-19 at 09:52 -0500, Orcan Ogetbil wrote: Didn't we have a mass rebuild due to a C++ ABI change 1 or 2 years ago? Standard C++ does not specify an ABI. Compilers get to handle that themselves. Correct (which can be regarded as a weakness of C++. While you can usually mix libraries compiled by other C compilers, you cannot do this with C++, in general). But for our practical purposes (since most Fedora C++ packages are compiled with gcc), we are tied to the ABI stability of gcc, e.g. https://fedorahosted.org/fesco/ticket/813 Best, Orcan -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, Dec 17, 2013 at 10:53 PM, Chris Murphy li...@colorremedies.com wrote: On Dec 17, 2013, at 5:40 PM, Kevin Kofler kevin.kof...@chello.at wrote: Miloslav Trmač wrote: a) Do we all agree that we need to solve this? No. We should not compromise our design principles (and, e.g., endorse an abominable hack like SCLs) just to allow obsolete applications to run on current versions of Fedora or the other way round. Current applications need to work with current libraries. What is this, the chupa mi pito platform? Do it my way or GTFO? Language like that is not appropriate. Please refrain from using it in the future. The fact that it is not in english does not somehow make it better. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Wed, Dec 18, 2013 at 7:10 AM, Tomasz Torcz to...@pipebreaker.pl wrote: On Tue, Dec 17, 2013 at 06:01:04PM -0500, Colin Walters wrote: On Tue, 2013-12-17 at 23:24 +0100, Miloslav Trmač wrote: b) Which WG will take on the task of solving this? We shouldn't end up with everybody agreeing that this needs to be solved, but no PRD proposing to solve this. Is it the Base WG or the Env and Stacks WG? Or is it up to Server and Cloud to do it? If the Server/Cloud groups follow the way Red Hat Enterprise Linux is currently developing, that'd be Docker (docker-io package). (These are server applications, which should be distinct from client/workstation applications, for numerous reasons) Workstation WG probably will use GNOME Containers: https://www.guadec.org/session/sandboxed-applications-for-gnome/ That's not been discussed yet, much less decided. I would advocate for using what the Base WG and/or Env and Stacks WG settles on for container technology, so as to not differentiate needlessly. josh -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On mið 18.des 2013 12:27, Josh Boyer wrote: Workstation WG probably will use GNOME Containers: https://www.guadec.org/session/sandboxed-applications-for-gnome/ That's not been discussed yet, much less decided. I would advocate for using what the Base WG and/or Env and Stacks WG settles on for container technology, so as to not differentiate needlessly. I would think there was nothing to discuss or decide it would simply be illogical not to use systemd's container technology in both cases. ( And the baseWG package sets dont contain additional container solution then what systemd provides ) And with my QA hat on we are not about to start testing every container technology out there so good luck testing this. JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, Dec 17, 2013 at 08:53:57PM -0700, Chris Murphy wrote: If you like the idea of always reinventing the wheel seemingly for no good reason, or just to use the latest flavored language of the day, then great. Uhm. Exactly because I don't like my stuff breaking every three weeks I choose libraries that live up to my expectation. This might involve assessing the capability of an particular upstream to maintain their stuff going into the future or just avoiding the latest flavored language of the day. This whole issue is for upstream projects to solve. It shouldn't be papered over in Fedora. Want stable libraries? Cool, then let's go build some. Help upstream projects to maintain stable releases and to design their APIs with an attention to stability in the first place. Many projects already get this. It might be appropriate for Fedora to document which these are to help Fedora users make an informed choice. But just freezing libraries at some random version essentially creates a fork which has to be maintained inside Fedora. Who is going to develop programs specifically for Fedora? Most developers are targeting the broader GNU/Linux type of system. Now think about Fedora supporting libA at version x while Debian froze it at version y and SUSE at z. What have we won? Really, this should be solved in upstream projects so you can expect a stable library API across distribution boundaries. Doing it in Fedora is not actually solving the problem. Lars -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Lars Seipel wrote: Uhm. Exactly because I don't like my stuff breaking every three weeks I choose libraries that live up to my expectation. This might involve assessing the capability of an particular upstream to maintain their stuff going into the future or just avoiding the latest flavored language of the day. +1 You can, for example, use C++, which is a stable standard (a new version was published 2 years ago, but almost all C++98 code compiles unchanged as C++11), and Qt, which keeps a stable API and ABI throughout a major version (the interval between the last 2 major versions having been over 7 years), and we keep old major versions available in a clean way. And the GNOME people would probably recommend their C + g* stack which satisfies similar compatibility properties. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Wed, Dec 18, 2013 at 12:19 AM, Colin Walters walt...@verbum.org wrote: Hi Andrew, On Tue, 2013-12-17 at 15:05 -0800, Andrew Lutomirski wrote: There will be a similar problem in the docker images, unless you're suggesting that everyone use Ubuntu-in-docker-on-Fedora/RHEL. True, but it becomes the responsibility of the container creator, not Fedora. That still seems like avoiding the question to me. Specifically who is the content creator? The user or the application developer? How are they supposed to do that if the OS doesn't give them any tools? Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Wed, Dec 18, 2013 at 1:39 PM, Jóhann B. Guðmundsson johan...@gmail.com wrote: On mið 18.des 2013 12:27, Josh Boyer wrote: Workstation WG probably will use GNOME Containers: https://www.guadec.org/session/sandboxed-applications-for-gnome/ That's not been discussed yet, much less decided. I would advocate for using what the Base WG and/or Env and Stacks WG settles on for container technology, so as to not differentiate needlessly. I would think there was nothing to discuss or decide it would simply be illogical not to use systemd's container technology in both cases. Containers are primarily an isolation mechanism; not a way to install, update or deploy software. Just use containers shifts the problem to how do I update the container which is at least as complex (because now the tools need to peek into the container in addition to working on the primary file system). Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On mið 18.des 2013 16:17, Miloslav Trmač wrote: Containers are primarily an isolation mechanism; not a way to install, update or deploy software. Right Just use containers shifts the problem to how do I update the container which is at least as complex (because now the tools need to peek into the container in addition to working on the primary file system). So let's break this down you have 3 types of containers 1. OS Container 2. Host application containers 3. Standalone application containers ( 3rd party apps ) In case of 1 and 2 you use the standard installation tools either directly or wrapped in chroot command In case of 3 you would never update an container you would replace it with a new container ( or App image rather ) which contains the update/upgrade and delete the old container or in case of Gnome with a new App image If I understood Alexander correctly at that BOF at guadec. Personally I think we should entirely drop any notion of implementing software collection in Fedora on rpm bases ( could be implemented as standalone app image ) since we dont need it RHEL does [1] and go directly looking at implementing Alexander's proposal regarding app images. JBG 1. http://developerblog.redhat.com/2013/01/28/software-collections-on-red-hat-enterprise-linux/ -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Dec 18, 2013, at 6:08 AM, Lars Seipel lars.sei...@gmail.com wrote: But just freezing libraries at some random version essentially creates a fork which has to be maintained inside Fedora. Who is going to develop programs specifically for Fedora? Most developers are targeting the broader GNU/Linux type of system. Now think about Fedora supporting libA at version x while Debian froze it at version y and SUSE at z. What have we won? Really, this should be solved in upstream projects so you can expect a stable library API across distribution boundaries. Doing it in Fedora is not actually solving the problem. Thanks for the response. Is it really upstream causing the problem in the first place? Or is it the distributions, who have always selected what library versions they will package, while also proscribing packaged libraries in applications, along with the insistence that it's the distribution package maintainer who decides what ships, not the developer? I don't think you get to fully blame this lack of application portability on linux on upstream. The distributions make choices that are motivated, I'd like to think, by what's best for their users. But in so doing, they also have caused a kind of fragmentation that makes the distributions effectively proprietary, and as different as Windows is to OS X. The one commonality they share is the name of the kernel. It's actually quite disconcerting for people new to linux to find out the extent of mutual incompatibility that exists. Again, I don't think that's any upstream's design goal. Conversely, differentiation is a design goal for distributions. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Chris Murphy (li...@colorremedies.com) said: Really, this should be solved in upstream projects so you can expect a stable library API across distribution boundaries. Doing it in Fedora is not actually solving the problem. Thanks for the response. Is it really upstream causing the problem in the first place? Or is it the distributions, who have always selected what library versions they will package, while also proscribing packaged libraries in applications, along with the insistence that it's the distribution package maintainer who decides what ships, not the developer? A little from column A, a little from column B. You have well used and promoted library stacks like Boost, which don't bother with ABI compatibilty at all. OpenSSL used to do this as well, so did Berkley DB. When you have library stacks like these, a distribution policy of 'always ship the latest' *is* going to exacerbate the problem for any users. Bill -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Inter-WG coordination: Stable application runtimes
Hello, Looking at the current WG outputs, it seems that nobody is taking on the problem of stable application runtimes: Primary requirement === If all Fedora Products are released at a fairly fast cadence, and with a fairly short support cycle, how do I write, deploy and run an application so that it can be used 3-5 (or more) years into the future with no or minimal changes, without running an unsupported Fedora Product? This is particularly important for the Server, which expects to be kept installed without major disruptions for a fairly long time (even if updating ot newer releases of Server), but it does also apply to running applications via Cloud. This includes custom applications that are not (or even can not) be packaged in Fedora, and even if they were packaged in Fedora, we would still want the effort to just keep them running to be minimal. The requirement not to run an unsupported Fedora Product excludes solutions like keeping an older version of Fedora running in a VM. Possible secondary requirements 1. It seems natural that the mechanism used for this should be the same between Server and Cloud at least, so that the user can take the exact same packaging of an application and deploy it on either. 2. There are many languages, even more sets of middleware/platform libraries (Boost vs. Qt vs glib, Django vs. Turbogears vs. Zope), and many ways to write an application. It would be nice if the mechanism used were the same for C/C++, Python, Ruby, Java, and anything else you can think of. 3. In addition to allowing an old application to run on a newer Fedora release, if we ever ship a Fedora product with a longer support lifecycle, we'll be faced with the mirror of this problem: how to allow running a new application on an old Fedora installation? Both might be solvable with the same mechanism. 4. As an addition to 1., it seems natural that it should be possible to develop applications for Server and Cloud using Workstation, i.e. that the same stable runtimes should be available on a Workstation and usable for application development (ideally without requiring root access to use the runtime). Solutions? == Is anything being already done for this? SCLs are working their way through the FPC, but they haven't been so far an explicit part of Fedora design/strategy (and some suggestions to move them to coprs go even further in that direction). Regardless of whether there is a specific technical solution or whether it is being worked on, who owns this problem? a) Do we all agree that we need to solve this? b) Which WG will take on the task of solving this? We shouldn't end up with everybody agreeing that this needs to be solved, but no PRD proposing to solve this. Is it the Base WG or the Env and Stacks WG? Or is it up to Server and Cloud to do it? (I'm afraid this conversation should probably have started earlier.) Mirek -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, 2013-12-17 at 23:24 +0100, Miloslav Trmač wrote: b) Which WG will take on the task of solving this? We shouldn't end up with everybody agreeing that this needs to be solved, but no PRD proposing to solve this. Is it the Base WG or the Env and Stacks WG? Or is it up to Server and Cloud to do it? If the Server/Cloud groups follow the way Red Hat Enterprise Linux is currently developing, that'd be Docker (docker-io package). (These are server applications, which should be distinct from client/workstation applications, for numerous reasons) -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, Dec 17, 2013 at 3:01 PM, Colin Walters walt...@verbum.org wrote: On Tue, 2013-12-17 at 23:24 +0100, Miloslav Trmač wrote: b) Which WG will take on the task of solving this? We shouldn't end up with everybody agreeing that this needs to be solved, but no PRD proposing to solve this. Is it the Base WG or the Env and Stacks WG? Or is it up to Server and Cloud to do it? If the Server/Cloud groups follow the way Red Hat Enterprise Linux is currently developing, that'd be Docker (docker-io package). (These are server applications, which should be distinct from client/workstation applications, for numerous reasons) There will be a similar problem in the docker images, unless you're suggesting that everyone use Ubuntu-in-docker-on-Fedora/RHEL. (Also, docker is *far* from ready for prime time, especially in its LVM incarnation.) --Andy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Hi Andrew, On Tue, 2013-12-17 at 15:05 -0800, Andrew Lutomirski wrote: There will be a similar problem in the docker images, unless you're suggesting that everyone use Ubuntu-in-docker-on-Fedora/RHEL. True, but it becomes the responsibility of the container creator, not Fedora. Anyways, I of course come from the perspective of glib maintainer, and we're already committed to Compatibility level 1 starting from Red Hat Enterprise Linux 6, meaning there will be no ABI breaks for 3 major releases. (See https://access.redhat.com/site/articles/119073 - this is transitive from the gtk2 stability ). I think it would also make sense to enforce these in Fedora (or really, enforce them upstream, which is where I do it). (Also, docker is *far* from ready for prime time, especially in its LVM incarnation.) I actually haven't used it myself, so I can't comment on that; I'm just observing where the Red Hat Enterprise Linux engineering work is going. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, Dec 17, 2013 at 3:19 PM, Colin Walters walt...@verbum.org wrote: Hi Andrew, On Tue, 2013-12-17 at 15:05 -0800, Andrew Lutomirski wrote: There will be a similar problem in the docker images, unless you're suggesting that everyone use Ubuntu-in-docker-on-Fedora/RHEL. True, but it becomes the responsibility of the container creator, not Fedora. Anyways, I of course come from the perspective of glib maintainer, and we're already committed to Compatibility level 1 starting from Red Hat Enterprise Linux 6, meaning there will be no ABI breaks for 3 major releases. (See https://access.redhat.com/site/articles/119073 - this is transitive from the gtk2 stability ). I think it would also make sense to enforce these in Fedora (or really, enforce them upstream, which is where I do it). IMO this is made considerably more complicated by the fact that libtool's version-info doesn't really work on ELF systems -- there are lots of libraries where, for example, .so.6 would be a perfectly fine replacement for .so.7, but the system doesn't know that. In principle, it would be possible for new versions to package symlinks from the old .so.n files. --Andy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Tue, 2013-12-17 at 15:27 -0800, Andrew Lutomirski wrote: there are lots of libraries where, for example, .so.6 would be a perfectly fine replacement for .so.7, but the system doesn't know that. I think the authors of these libraries screwed up. Namely, libudev and libffi were both mistakes; for the latter, see https://lists.fedoraproject.org/pipermail/devel/2012-April/166357.html for my opinion. -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On þri 17.des 2013 22:24, Miloslav Trmač wrote: Hello, Looking at the current WG outputs, it seems that nobody is taking on the problem of stable application runtimes: Probably because no maintainer has been asked for how long release cycle they considered they could/would maintain their component for but hey go ahead and decide that for them... JBG -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
Miloslav Trmač wrote: a) Do we all agree that we need to solve this? No. We should not compromise our design principles (and, e.g., endorse an abominable hack like SCLs) just to allow obsolete applications to run on current versions of Fedora or the other way round. Current applications need to work with current libraries. Kevin Kofler -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
Re: Inter-WG coordination: Stable application runtimes
On Dec 17, 2013, at 5:40 PM, Kevin Kofler kevin.kof...@chello.at wrote: Miloslav Trmač wrote: a) Do we all agree that we need to solve this? No. We should not compromise our design principles (and, e.g., endorse an abominable hack like SCLs) just to allow obsolete applications to run on current versions of Fedora or the other way round. Current applications need to work with current libraries. What is this, the chupa mi pito platform? Do it my way or GTFO? If you like the idea of always reinventing the wheel seemingly for no good reason, or just to use the latest flavored language of the day, then great. And if you want Fedora to be a highly questionable development platform where applications appear and disappear quickly, and are at best migrated to more stable platforms because, wow, they actually have more users who have better things to do than update their OS every week, then great. But I'm thinking that the human race hasn't yet innovated the solution to the problem of how to have an aggressively evolving platform that's also stable. So far we have Apple's two OS's, which drive developers nuts because of how many APIs are deprecated every cycle, but hey many are making money so they tolerate it. And then on the opposite spectrum we have crusty Windows with such ABI/API stability that you can run 25 year old applications on it, with a commensurate platform innovation that almost excites the typical house plant. I think there's another option to the FU new is inherently good even if it's unstable approach to creating a development platform. Chris Murphy -- devel mailing list devel@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/devel Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct
