Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Mon, 17 Dec 2007 16:24:01 +0100 Bo Ørsted Andresen [EMAIL PROTECTED] wrote: Which would add an awful lot of complexity and require major design changes in order to gain anything. The beauty of the ebuild format is its simplicity. I don't really think it's worth it. I agree. I have noticed that for general emerge operations, about 1GHz of processor power will be able to hit the I/O bandwith cap on my fastest drives. In other words, I don't think lots of the time spent withh portage -- on modern systems -- is on the processor. You'd have better results -- and an easier time -- if you optimized the FS for Portage, I would imagine. -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
Hi everyone, I don't think the programming language is the problem here. The problem is that some of Portage architectural decisions have a negative impact on performance. Probably because the developers were focused on minimizing dependencies (i.e. file system based persistence) and bandwidth consumption (i.e. using rsync for updating packages). So, even if Portage was recoded in C++, performance improvements would be marginal and the cost in man-hours would be too high. It would take months before reaching the maturity level Portage has now and all this time could be better spent trying to find solutions to its architectural bottlenecks. I believe that a good solution would be evolving Portage to use different forms of storage, like databases or even LDAP. In a home desktop, you could use SQLite, which is light weight. In a Office enviroment, you could use a larger database, like MySQL or PostgreSQL. In this second case, it would even make sharing the package list faster, since the only current method is sharing it over NFS. I understand that doing so could bloat Portage dependencies, but it is, IMHO, a good way to improve its speed. Regards, Raphael -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
*removedlotsofideas* your ideas sound nice on paper. But one strenght of portage and its structures: no matter how hosed your 'data', you can repair it with cp, mv, an emerge sync and a text editor. Which is all not true, if you start using some database crap. Go, look at /var/db/pkg - you can read and repair that stuff easily. Or the files in /var/lib/portage. Damaged world-file? nano FTW! -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Monday 17 December 2007 14:38:30 Raphael wrote: So, even if Portage was recoded in C++, performance improvements would be marginal and the cost in man-hours would be too high. It would take months before reaching the maturity level Portage has now and all this time could be better spent trying to find solutions to its architectural bottlenecks. I believe that a good solution would be evolving Portage to use different forms of storage, like databases or even LDAP. In a home desktop, you could use SQLite, which is light weight. In a Office enviroment, you could use a larger database, like MySQL or PostgreSQL. In this second case, it would even make sharing the package list faster, since the only current method is sharing it over NFS. I understand that doing so could bloat Portage dependencies, but it is, IMHO, a good way to improve its speed. This post is hilarious for several reasons. Firstly there already exist a package manager for Gentoo which is written in C++. Paludis. And it has a lot of features that Portage has been missing for five years. And it's way more flexible than Portage. Secondly if you just put ebuilds in a database you gain nothing. I.e. other than the added bloat. I/O is still going to be the major bottleneck. :P -- Bo Andresen signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Dec 17, 2007 11:55 AM, Hemmann, Volker Armin [EMAIL PROTECTED] wrote: *removedlotsofideas* ?? your ideas sound nice on paper. But one strenght of portage and its structures: no matter how hosed your 'data', you can repair it with cp, mv, an emerge sync and a text editor. Which is all not true, if you start using some database crap. Go, look at /var/db/pkg - you can read and repair that stuff easily. Or the files in /var/lib/portage. Damaged world-file? nano FTW! The Portage files are easy to maintain. But I honestly never _had_ to read them. I just did out of curiosity. And I never needed to repair them either. When I got hosed data, I just replaced everything with the latest backup copy and never looked back. Using a database follows the same principle. If you make backups, you don't need to think about how the system tools store their data. If your system gets in an unstable state, you just recover it. And backing up a database is, IMHO, simpler than backing the Portage data. Regards, Raphael -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Montag, 17. Dezember 2007, Raphael wrote: On Dec 17, 2007 11:55 AM, Hemmann, Volker Armin [EMAIL PROTECTED] wrote: *removedlotsofideas* ?? your ideas sound nice on paper. But one strenght of portage and its structures: no matter how hosed your 'data', you can repair it with cp, mv, an emerge sync and a text editor. Which is all not true, if you start using some database crap. Go, look at /var/db/pkg - you can read and repair that stuff easily. Or the files in /var/lib/portage. Damaged world-file? nano FTW! The Portage files are easy to maintain. But I honestly never _had_ to read them. I just did out of curiosity. And I never needed to repair them either. When I got hosed data, I just replaced everything with the latest backup copy and never looked back. which won't help you much if the last backup is 7 days old and you just did a big fat update/cleanup circle. I had to repair stuff in /var - and I was glad that everything was nice, readable text files. Made the whole thing very easy. -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Dec 17, 2007 12:20 PM, Bo Ørsted Andresen [EMAIL PROTECTED] wrote: On Monday 17 December 2007 14:38:30 Raphael wrote: So, even if Portage was recoded in C++, performance improvements would be marginal and the cost in man-hours would be too high. It would take months before reaching the maturity level Portage has now and all this time could be better spent trying to find solutions to its architectural bottlenecks. I believe that a good solution would be evolving Portage to use different forms of storage, like databases or even LDAP. In a home desktop, you could use SQLite, which is light weight. In a Office enviroment, you could use a larger database, like MySQL or PostgreSQL. In this second case, it would even make sharing the package list faster, since the only current method is sharing it over NFS. I understand that doing so could bloat Portage dependencies, but it is, IMHO, a good way to improve its speed. This post is hilarious for several reasons. Firstly there already exist a package manager for Gentoo which is written in C++. Paludis. And it has a lot of features that Portage has been missing for five years. And it's way more flexible than Portage. Secondly if you just put ebuilds in a database you gain nothing. I.e. other than the added bloat. I/O is still going to be the major bottleneck. :P Hey, I made someone laugh today. Good deed of the day: check! :P I was unaware of Paludis. Re-reading the thread now, I saw that someone mentioned it. After googling for it, seems a lot of people are fond of it. Why is it not the default package manager yet? As for the second part, yes, using a database wouldn't get rid of the I/O problem, but could diminish it, since database data isn't spread across several directories and files. And I'm not proposing to store the entire ebuild within, but a representation of it that could be easily queried. -- Bo Andresen z� ��(��j)b� b�
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Monday 17 December 2007 16:14:24 Raphael wrote: Hey, I made someone laugh today. Good deed of the day: check! :P :) I was unaware of Paludis. Re-reading the thread now, I saw that someone mentioned it. After googling for it, seems a lot of people are fond of it. Why is it not the default package manager yet? It's not my decision so I'm not going to answer this. I can, however, say that Paludis still lacks a couple of features that Portage has (most notably binary package support and pkg/slot-move support). As for the second part, yes, using a database wouldn't get rid of the I/O problem, but could diminish it, since database data isn't spread across several directories and files. And I'm not proposing to store the entire ebuild within, but a representation of it that could be easily queried. Which would add an awful lot of complexity and require major design changes in order to gain anything. The beauty of the ebuild format is its simplicity. I don't really think it's worth it. -- Bo Andresen signature.asc Description: This is a digitally signed message part.
[gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sun, 16 Dec 2007 06:20:38 -0600 Dale wrote: ...[snip]... I read a link provided earlier about Plaudis, (sp?). It seems that Portage has a lot of hacks in it, according to what I read anyway. Is that true? Also, is it being wrote with python hurting portage as for as the program itself? If it is, why are they not trying to switch to something else? If C++ is better, then putting off changing is only going to get harder as time goes on. IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
David Relson wrote: On Sun, 16 Dec 2007 06:20:38 -0600 Dale wrote: ...[snip]... I read a link provided earlier about Plaudis, (sp?). It seems that Portage has a lot of hacks in it, according to what I read anyway. Is that true? Also, is it being wrote with python hurting portage as for as the program itself? If it is, why are they not trying to switch to something else? If C++ is better, then putting off changing is only going to get harder as time goes on. IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. Thanks. I was curious as to how a language could hurt a program as long as the end result is the same. I take what you wrote as, it is not the rules that makes a mess but all the exceptions to the rules that makes a mess. Thanks for the reply. Dale :-) :-)
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sun, 16 Dec 2007 08:05:17 -0600 Dale wrote: David Relson wrote: On Sun, 16 Dec 2007 06:20:38 -0600 Dale wrote: ...[snip]... I read a link provided earlier about Plaudis, (sp?). It seems that Portage has a lot of hacks in it, according to what I read anyway. Is that true? Also, is it being wrote with python hurting portage as for as the program itself? If it is, why are they not trying to switch to something else? If C++ is better, then putting off changing is only going to get harder as time goes on. IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. Thanks. I was curious as to how a language could hurt a program as long as the end result is the same. I take what you wrote as, it is not the rules that makes a mess but all the exceptions to the rules that makes a mess. Thanks for the reply. Dale I don't know enough to say for sure. I _am_ a programmer, but not involved with portage. My guess is that the rules are reasonable, but evolved over time. Having read many messages about portage and ebuilds I'm lead to believe that exceptions have lead to complications and a less than ideal solution -- code that's difficult to maintain.. -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
David Relson wrote: IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. C++ is most certainly going to yield faster programs since it is a machine compiled language and python is interpreted. But that's not the idea behind portage. It's all about using the right tools for the job. I do all my research code in C++ because I need good memory management and I need speed. But python is far easier to code in, doesn't need to be compiled, and is pretty dang elegant. It's also pretty platform independent, which is also nice. -- Randy Barlow http://electronsweatshop.com -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sonntag, 16. Dezember 2007, Randy Barlow wrote: David Relson wrote: IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. C++ is most certainly going to yield faster programs since it is a machine compiled language and python is interpreted. But that's not the idea behind portage. It's all about using the right tools for the job. I do all my research code in C++ because I need good memory management and I need speed. But python is far easier to code in, doesn't need to be compiled, and is pretty dang elegant. It's also pretty platform independent, which is also nice. -- Randy Barlow http://electronsweatshop.com one reason pro phyton and contra c and c++ has always been: segfaults. And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On 16/12/2007, Randy Barlow [EMAIL PROTECTED] wrote: David Relson wrote: IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. C++ is most certainly going to yield faster programs since it is a machine compiled language and python is interpreted. But that's not the idea behind portage. It's all about using the right tools for the job. I do all my research code in C++ because I need good memory management and I need speed. But python is far easier to code in, doesn't need to be compiled, and is pretty dang elegant. It's also pretty platform independent, which is also nice. I see you haven't read the portage source-code. It isn't so elegant... And I'm saying this as someone who likes python and thinks it is generally a Good Idea. On 16/12/2007, Hemmann, Volker Armin [EMAIL PROTECTED] wrote: one reason pro phyton and contra c and c++ has always been: segfaults. And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working Hehehehe. Guess what python is linked against (It doesn't have to be linked against libstdc++, but it usually is)? =P -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
[...] Just think about this horror: gcc/libstdc++ update and your package manager stops working Hehehehe. Guess what python is linked against (It doesn't have to be linked against libstdc++, but it usually is)? =P CPython is written in C and has no C++ dependencies: $ ldd `which python` linux-gate.so.1 = (0xb7ffd000) libpython2.5.so.1.0 = /usr/lib/libpython2.5.so.1.0 (0x455b) libpthread.so.0 = /lib/libpthread.so.0 (0x421c9000) libdl.so.2 = /lib/libdl.so.2 (0x421af000) libutil.so.1 = /lib/libutil.so.1 (0x42feb000) libm.so.6 = /lib/libm.so.6 (0x42189000) libc.so.6 = /lib/libc.so.6 (0x42052000) /lib/ld-linux.so.2 (0x4100) -- Albert W. Hopkins -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sunday 16 December 2007 20:32:58 Randy Barlow wrote: C++ is most certainly going to yield faster programs since it is a machine compiled language and python is interpreted. In this case it's not really significant. The biggest performance hit for a package manager for Gentoo remains I/O no matter which language you use... -- Bo Andresen signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hemmann, Volker Armin ha scritto: On Sonntag, 16. Dezember 2007, Randy Barlow wrote: David Relson wrote: IMHO, python is a very nice object oriented language and C++ is no better (unless you need particular features of the language). I suspect C++ runs somewhat faster, but that's not the issue here. As I understand, portage needs to deal with lots of special cases and exceptions to the general rules for updating package. Special cases and exceptions always lead to complications and messy code. Switching languages doesn't help a situation like this. C++ is most certainly going to yield faster programs since it is a machine compiled language and python is interpreted. But that's not the idea behind portage. It's all about using the right tools for the job. I do all my research code in C++ because I need good memory management and I need speed. But python is far easier to code in, doesn't need to be compiled, and is pretty dang elegant. It's also pretty platform independent, which is also nice. -- Randy Barlow http://electronsweatshop.com one reason pro phyton and contra c and c++ has always been: segfaults. And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working Why don't a python upgrade break your package manager?? - -- Antonio Quartulli, @System President www.atsystem.org Student of Computer Science at University of Pisa Web: www.quartulli.org GPG Key: www.quartulli.org/aquartulli_0×41A1F50B.asc -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.7 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iQEVAwUBR2WN1JUBe9lBofULAQJOZwf/b42oKASq7T3LPChNub/L48Sdc4frpTuT zNov1laSH3Wswax1UtFg+2oGxwJga3tzd4SX6F0/E5LhQE9BBSX3YGwgKy2HANsB ban2rZssT/IADSK8PE6rGM2SmBm46iGOJNDcTfw/K014FC68ZoA+giK63KTpcQ70 j789xlfDQQKrTitR0uawwPiZAE7mQ2wqgTozInOecDxEiNXV6HOOdNeqOgofGLOM R9XfpURFsaj72cMpqlW/hnuQQAf5hHq1GDBoWDpiETC3IDZNLkXI2hgoy6oBxybB VqTCUdfInUsMTlXARJgebr96Yl6x4XXafZZTbvncN0MQTCZ1WCQaUQ== =sG2L -END PGP SIGNATURE- -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On 16/12/2007, Antonio Quartulli [EMAIL PROTECTED] wrote: snip And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working Why don't a python upgrade break your package manager?? Also possible, but less likely. Python is usually OK wrt backwards compatibility. Surprisingly enough, the thing most likely to break your package manager is a package manager update. =P pycrypto, anyway? -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
Hemmann, Volker Armin wrote: one reason pro phyton and contra c and c++ has always been: segfaults. And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working Well segfaults generally indicate bugs in your code - so hopefully you would be a good coder and ensure that you manage memory correctly. Abi changes suck big time, I agree on that point, and also the updates - hadn't thought of that one :) -- Randy Barlow http://electronsweatshop.com -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
Matan Peled wrote: I see you haven't read the portage source-code. It isn't so elegant... And I'm saying this as someone who likes python and thinks it is generally a Good Idea. No, I definitely have not, but I have done some Python coding in my days. I was referring to the language, not the use of the language by Portage :) -- Randy Barlow http://electronsweatshop.com -- [EMAIL PROTECTED] mailing list
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sunday 16 December 2007 22:04:52 Randy Barlow wrote: Bo Ørsted Andresen wrote: In this case it's not really significant. The biggest performance hit for a package manager for Gentoo remains I/O no matter which language you use... Yeah, you are right - although there is one step of an emerge that seems to hit my P3 hard: right after it reads the data from disk to determine dependencies, it goes 100% CPU for a few seconds. Just think, we could save a few seconds! ;) But seriously, I totally think Python is the tool for the package manager job - I was just trying to outline when you might choose C++ over python... Heh, I don't think Python is the right choice. Performance just isn't the reason. ;) -- Bo Andresen signature.asc Description: This is a digitally signed message part.
Re: [gentoo-user] Python vs C++ [was: Gentoo Rules]
On Sonntag, 16. Dezember 2007, Randy Barlow wrote: Hemmann, Volker Armin wrote: one reason pro phyton and contra c and c++ has always been: segfaults. And with c++ comes another one: abi changes. Just think about this horror: gcc/libstdc++ update and your package manager stops working Well segfaults generally indicate bugs in your code - so hopefully you would be a good coder and ensure that you manage memory correctly. Abi changes suck big time, I agree on that point, and also the updates - hadn't thought of that one :) sometimes a lib update is the one factor that turns a simply 'thinko' in a full grown segfault ;) -- [EMAIL PROTECTED] mailing list
