Re: [CVS] RPM: rpm/ CHANGES macros.in
On Thu, Jun 28, 2007, Mark Hatle wrote: Is TEMPORARY TABLE something special? I knew about the :memory: before, but not the other. Excerpt from http://www.sqlite.org/lang_createtable.html: If the TEMP or TEMPORARY keyword occurs in between CREATE and TABLE then the table that is created is only visible within that same database connection and is automatically deleted when the database connection is closed. Any indices created on a temporary table are also temporary. Temporary tables and indices are stored in a separate file distinct from the main database file. Otherwise this looks good to me. It may even speed things up slightly. But before I can commit I at least wish to find a better approach for the DB_PRIVATE value. The current hack is not the best since sliced bread: +#if defined(WITH_SQLITE) !defined(DB_PRIVATE) +#define DB_PRIVATE 0x020 /* shameless hack for shared option */ +#endif Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: [CVS] RPM: rpm/ CHANGES macros.in
For the case w/o db3 header being available, there is the db_emu.h file. Put that value in there, and it is supposed to be picked up if db items are not defined. --Mark Ralf S. Engelschall wrote: On Thu, Jun 28, 2007, Mark Hatle wrote: Is TEMPORARY TABLE something special? I knew about the :memory: before, but not the other. Excerpt from http://www.sqlite.org/lang_createtable.html: If the TEMP or TEMPORARY keyword occurs in between CREATE and TABLE then the table that is created is only visible within that same database connection and is automatically deleted when the database connection is closed. Any indices created on a temporary table are also temporary. Temporary tables and indices are stored in a separate file distinct from the main database file. Otherwise this looks good to me. It may even speed things up slightly. But before I can commit I at least wish to find a better approach for the DB_PRIVATE value. The current hack is not the best since sliced bread: +#if defined(WITH_SQLITE) !defined(DB_PRIVATE) +#define DB_PRIVATE 0x020 /* shameless hack for shared option */ +#endif Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: fsync()
On Friday 29 June 2007 05:37, Jeff Johnson [EMAIL PROTECTED] wrote: On Jun 28, 2007, at 2:28 AM, Russell Coker wrote: When upgrading a package with RPM version 4.4.2 in SUSE doesn't call fsync()! It creates a temporary file (without using O_SYNC), writes all the data to it, closes it, and then renames it to replace the original file. The temporary file has the /path/to/file;12345678 transaction id appended? I don't recall the name. A close should sync the data, should it not? Not necessarily. Some filesystems (such as XFS) try to deduce what a user-space program desires by the pattern of system-calls and implements it (EG a certain combination of create and rename can cause the data to be sync'd faster). Or do you mean the rpmdb files? No, I mean file data. SuSE has a very different usage case for a rpmdb, and insists on avoiding sync whenever possible for performance reasons. Then SuSE are butt-heads. Has this horrible mistake been fixed in the upstream tree? I believe the problem is a change in behavior in libio in glibc. I believe that it has nothing to do with glibc or any other user-space code. But adding an explicit fsync() is trivial as soon as I can get a reproducer. You want to be able to repeatably trigger a race-condition before you fix it? To cause this race condition you must first use a file-system that is optimised for performance (EG XFS) so that it will allow long cache write-back times and also do write-related tasks after closing the file (EG assigning disk blocks to the file after close() so that it knows the length). Then put some load on the system while installing an RPM, and then trigger a hardware reset shortly after rpm exits. -- [EMAIL PROTECTED] http://etbe.coker.com.au/ My Blog http://www.coker.com.au/sponsorship.html Sponsoring Free Software development __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
On Fri, Jun 29, 2007, Mark Hatle wrote: Taken from rpm.org rpm-4.4.2.1 tree. Any comments or may I commit. (I didn't just hit go, since I figured there might be some discussion on this issue.) :) This is for HEAD BTW. Looks good to, so go from my POV. And I think we not even have to AC_MSG_WARN here as this should't be such a major support issue as the Berkeley-DB one. Just use [] as the RPM_CHECK_LIB argument. Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
In regard to: No Neon patch..., Mark Hatle said (at 1:22pm on Jun 29, 2007): Taken from rpm.org rpm-4.4.2.1 tree. Any comments or may I commit. (I didn't just hit go, since I figured there might be some discussion on this issue.) :) neon can be a PITA, especially because it defaults to static lib only and has non-obvious dependencies so linking with it can be challenging, but I'm concerned about the proliferation of rpm flavors. Your flavor might be built without neon at all so it won't support certain functionality, someone else's rpm might have neon support but no lua, etc. Sure, it's possible to have rpm's internal dependencies such that when someone tries to install a package that Requires(interp): lua (or whatever) the install aborts immediately if the rpm on the box doesn't provide that, but that's not the point. People writing spec files, or whatever packaging recipe format eventually evolves, will write their recipes to use the functionality that they know is guaranteed to be available in the rpm command that users have available on their systems -- the minimal subset of features that every rpm command supports. Pretty soon, people won't try make use of lua or neon or anything else that falls outside of that minimal subset, and those features in rpm will become the software equivalent of my appendix -- evolutionary leftovers that have ceased to serve a useful purpose. I know I'm frequently in left field WRT rpm, but I think if you make stuff like neon or lua too easily optional, those things will fall outside of the minimal subset, and at that point one needs to decide if it wouldn't be better to just remove the evolutionary leftovers completely. Just MHO, Tim (who still has his appendix, thank you very much) -- Tim Mooney[EMAIL PROTECTED] Information Technology Services (701) 231-1076 (Voice) Room 242-J6, IACC Building(701) 231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
On Friday, 29 June 2007, at 14:05:10 (-0500),
Tim Mooney wrote:
People writing spec files, or whatever packaging recipe format
eventually evolves, will write their recipes to use the
functionality that they know is guaranteed to be available in the
rpm command that users have available on their systems -- the
minimal subset of features that every rpm command supports.
I wish that were true. But the unfortunate reality is that each
distro writes their spec files to use their own proprietary set of
capabilities and macros. RHEL RPM's may not build without
redhat-rpm-config. Mandriva RPM's frequently require custom macros,
not the least of which being %{make}. Few people actually strive to
create portable spec files.
Michael
--
Michael Jennings (a.k.a. KainX) http://www.kainx.org/ [EMAIL PROTECTED]
n + 1, Inc., http://www.nplus1.net/ Author, Eterm (www.eterm.org)
---
[Heather Graham] is the new I-would-run-over-my-best-friend-in-a-
hummer-to-get-next-to-her girl.-- Claude Nobler
__
RPM Package Managerhttp://rpm5.org
Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
Few people actually strive to create portable spec files. /me sheepishly raises his hand... :-P Jason __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: [CVS] RPM: rpm/ CHANGES configure.ac rpm/rpmdb/ Makefile.am
On Jun 29, 2007, at 12:44 PM, Tim Mooney wrote: Why don't you include an rpm-bug-report script, a la bashbug, that automates much of that, including relevant information about how rpm was configured, the environment, etc, and then update the docs and begin to encourage users to use that for bug reporting? The problem with diagnosing rpm problems is capturing sufficient data. Most bug reports go something like blah blah 800 pkgs blah blah upgrade blah blah doesn't work. But that's not a reason not to add a means to report bugs. 73 de Jeff Tim -- Tim Mooney[EMAIL PROTECTED] Information Technology Services (701) 231-1076 (Voice) Room 242-J6, IACC Building(701) 231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
XAR development starts next weekend.
I have a feature deliverable with my name attached. Next weekend I start converting *.rpm to *.xar format. Originally, I planned to do the development on a branch. However, after seeing recent breakages, I will be doing my development on HEAD, just like everyone else. I will be building rpm with the same stuff that has been in rpm for years. That includes internal db, internal lua, internal neon, and sqlite. I suggest you get HEAD in shape for roadmap development. I can/will be reverting any patches on HEAD that get in my way. 73 de Jeff __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: XAR development starts next weekend.
Jeff Johnson wrote: Originally, I planned to do the development on a branch. However, after seeing recent breakages, I will be doing my development on HEAD, just like everyone else. I will be building rpm with the same stuff that has been in rpm for years. That includes internal db, internal lua, internal neon, and sqlite. From my current testing, internal db, internal lua, internal zlib and internal file are all working ... with external neon and external sqlite. There doesn't appear to be a way to do internal neon or sqlite at the moment. If this is required, it should be a minor patch to the configure.ac. __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
db3 configure wrapper
Any objections to the attached change? I believe this will work in any POSIX shell. The idea is instead of processing all of the arguments and possibly blowing away quoting and such, we process them one at a time and use shift and set to recreate the argument list. In my testing, this fixed the configuration problem I was having. My configuration is: CC=ccache gcc \ CFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES -fomit-frame-pointer -pipe -I/home/mhatle/build/host-tools/host-cross/include \ CXX=ccache g++ \ CXXFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES -fno-rtti -fno-implicit-templates -pipe \ CXXCPP=g++ -E \ CPP=gcc -E \ LD=ld \ LDFLAGS=-s -L/home/mhatle/build/host-tools/host-cross/lib \ AR=ar \ AS=as \ NM=nm \ OBJCOPY=objcopy \ OBJDUMP=objdump \ RANLIB=ranlib \ STRIP=strip \ /home/mhatle/build/host-tools/build-tools/rpm5-20070629/configure \ --cache-file=wrs_config_cache \ --prefix=/home/mhatle/build/host-tools/host-cross \ --build=i686-pc-linux-gnu --disable-nls --without-python \ --without-perl --without-selinux \ --without-apidocs This results in: configure: running /bin/sh ./configure '--prefix=/home/mhatle/build/host-tools/h ost-cross' '--build=i686-pc-linux-gnu' '--disable-nls' '--without-python' '--wi thout-perl' '--without-selinux' '--without-apidocs' 'build_alias=i686-pc-linux-g nu' 'CC=ccache gcc' 'CFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES - fomit-frame-pointer -pipe -I/home/mhatle/build/host-tools/host-cross/include' 'L DFLAGS=-s -L/home/mhatle/build/host-tools/host-cross/lib' 'CPP=gcc -E' 'CXX=ccac he g++' 'CXXFLAGS=-O2 -march=i486 -mtune=i686 -D__USE_STRING_INLINES -fno-rtti - fno-implicit-templates -pipe' 'CXXCPP=g++ -E' --cache-file=../wrs_config_cache - -srcdir=. configure: WARNING: you should use --build, --host, --target configure: WARNING: invalid host type: g++ checking build system type... i686-pc-linux-gnu checking host system type... Invalid configuration `g++': machine `g++' not reco gnized configure: error: /bin/sh ./../db/dist/config.sub g++ failed mv: cannot stat `Makefile': No such file or directory cat: Makefile.orig: No such file or directory mv: cannot stat `db.h': No such file or directory cat: db.h.orig: No such file or directory --Mark Index: CHANGES === RCS file: /v/rpm/cvs/rpm/CHANGES,v retrieving revision 1.1418 diff -u -r1.1418 CHANGES --- CHANGES 29 Jun 2007 20:23:06 - 1.1418 +++ CHANGES 29 Jun 2007 20:57:53 - @@ -1,4 +1,5 @@ 4.5 - 5.0: +- mgh: change db3 configure wrapper to process one argument at a time - mgh: allow rpm to be built w/o neon - rse: remove unused rpmio/sexp/ sub-library - rse: add errno information to be more informative in error message when lstat(2) on verification fails. Index: db3/configure === RCS file: /v/rpm/cvs/rpm/db3/configure,v retrieving revision 1.32 diff -u -r1.32 configure --- db3/configure 27 Jun 2007 21:18:48 - 1.32 +++ db3/configure 29 Jun 2007 20:57:53 - @@ -7,7 +7,12 @@ rm -f config.cache -ARGS=`echo $* |\ +# We iterate over the argument list. Processing the arguments using +# echo $* can fail with the sed chopping out parts of unrelated +# arguments +set - $@ eoa_marker +while [ $1 != eoa_marker ]; do + ARG=`echo $1 | \ sed -e 's% [^ ]*CFLAGS=[^ ]*%%' \ -e 's% [^ ]*CC=[^ ]*ccache [^ ]*%%' \ -e 's% [^ ]*CC=[^ ]*%%' \ @@ -24,6 +29,11 @@ -e 's%--with-db-mutex%%' \ -e 's%--without-db-mutex%%' \ ` + shift + set - $@ $ARG +done +shift + # NOTICE: # 1. --enable-shared is there for enforcing GNU libtool usage only
Re: XAR development starts next weekend.
On Fri, Jun 29, 2007, Mark Hatle wrote: Jeff Johnson wrote: Originally, I planned to do the development on a branch. However, after seeing recent breakages, I will be doing my development on HEAD, just like everyone else. I will be building rpm with the same stuff that has been in rpm for years. That includes internal db, internal lua, internal neon, and sqlite. From my current testing, internal db, internal lua, internal zlib and internal file are all working ... with external neon and external sqlite. There doesn't appear to be a way to do internal neon or sqlite at the moment. If this is required, it should be a minor patch to the configure.ac. No, please just let Neon and SQLite be external. No need for local inclusion... Ralf S. Engelschall [EMAIL PROTECTED] www.engelschall.com __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: fsync()
On Saturday 30 June 2007 00:35, Andy Green [EMAIL PROTECTED] wrote: I don't think fsync() for individual files is really a fair answer, Why not? it's fine if it just uses the normal filesystem APIs per-file. But after the transaction is complete, and you walk away thinking you did complete an rpm transaction, there is a case for adding a sync() to make sure everything you think you have done is truly committed to physical storage (maybe it does it already, I dunno). On the one hand this is a relatively low probability issue for a desktop box but on the other hand it is pretty cheap. The time taken for a sync() system call can be very large when you have a system under high write load. Under some older versions of Linux the time taken for sync() appeared to be unbounded (it apparently kept looping through the list of data to write while more data was being added to the list), a brief test suggests that recent versions of Linux may have solved this. sync() is not the way to get some files committed to disk. -- [EMAIL PROTECTED] http://etbe.coker.com.au/ My Blog http://www.coker.com.au/sponsorship.html Sponsoring Free Software development __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: fsync()
On Saturday 30 June 2007 02:04, Jeff Johnson [EMAIL PROTECTED] wrote: Should I add O_SYNC when opening files on delayed write file systems? Doable, but annoying mapping the path back to a file system type to infer functionality. O_SYNC is not the correct solution. XFS likes to delay block allocation to get contiguous files. O_SYNC on XFS would either result in re-allocating file blocks (terrible for write performance) or discontigous files. Write performance will always be expected to be better from a fsync() before close() than from O_SYNC. -- [EMAIL PROTECTED] http://etbe.coker.com.au/ My Blog http://www.coker.com.au/sponsorship.html Sponsoring Free Software development __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
On Friday, 29 June 2007, at 14:28:46 (-0500), Tim Mooney wrote: I was thinking mainly of the application vendors when I replied earlier -- companies like EMC, Oracle, IBM, etc., that want to claim that their application supports at least RHEL, SuSE, and possibly other rpm-based distributions. It's their packaging people (or monkeys, as the case may be) that will focus on the minimal subset, in the interest of making their packages install on the widest range of rpm-based distributions. If that's the assumption we're operating under, should we even bother trying to allow ISV rpmdb entries? I'm more inclined to think in the opposite direction: add important features that ISV's can't live without so that they will pressure distributions to update to and support those features. Michael -- Michael Jennings (a.k.a. KainX) http://www.kainx.org/ [EMAIL PROTECTED] n + 1, Inc., http://www.nplus1.net/ Author, Eterm (www.eterm.org) --- You came from heaven to Earth to show the way; from the Earth to the cross, my debt to pay. From the cross to the grave, from the grave to the sky. Lord, I lift your name on high. -- Maranatha __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: fsync()
On Jun 29, 2007, at 6:04 PM, Russell Coker wrote: On Saturday 30 June 2007 02:04, Jeff Johnson [EMAIL PROTECTED] wrote: Should I add O_SYNC when opening files on delayed write file systems? Doable, but annoying mapping the path back to a file system type to infer functionality. O_SYNC is not the correct solution. XFS likes to delay block allocation to get contiguous files. O_SYNC on XFS would either result in re- allocating file blocks (terrible for write performance) or discontigous files. Write performance will always be expected to be better from a fsync() before close() than from O_SYNC. OK/ fsync before close, todo++. Thanks for the info, xfs is a mystery to me. 73 de Jeff __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: fsync()
On Saturday 30 June 2007 12:02, Jeff Johnson [EMAIL PROTECTED] wrote: OK/ fsync before close, todo++. Thanks for the info, xfs is a mystery to me. If you want to know any specific things about XFS then ask me. I'm currently working for SGI in a group that is associated with the XFS group. I regularly talk to the XFS experts and can get any specific questions answered. -- [EMAIL PROTECTED] http://etbe.coker.com.au/ My Blog http://www.coker.com.au/sponsorship.html Sponsoring Free Software development __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
RE: fsync()
[EMAIL PROTECTED] wrote: On Saturday 30 June 2007 12:02, Jeff Johnson [EMAIL PROTECTED] wrote: OK/ fsync before close, todo++. Thanks for the info, xfs is a mystery to me. If you want to know any specific things about XFS then ask me. I'm currently working for SGI in a group that is associated with the XFS group. I regularly talk to the XFS experts and can get any specific questions answered. This is ridiculous. Nobody should need to know anything about the internal details of a specific filesystem's implementation when they're writing a userspace application. If the filesystem has chosen to do silly optimizations which doesn't work right when using published apis and methods, it's the filesystem that is broken and not even remotely the application's fault. __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
Re: No Neon patch...
In regard to: Re: No Neon patch..., Michael Jennings said (at 9:19pm on Jun...: On Friday, 29 June 2007, at 14:28:46 (-0500), Tim Mooney wrote: I was thinking mainly of the application vendors when I replied earlier -- companies like EMC, Oracle, IBM, etc., that want to claim that their application supports at least RHEL, SuSE, and possibly other rpm-based distributions. It's their packaging people (or monkeys, as the case may be) that will focus on the minimal subset, in the interest of making their packages install on the widest range of rpm-based distributions. If that's the assumption we're operating under, should we even bother trying to allow ISV rpmdb entries? I'm more inclined to think in the opposite direction: add important features that ISV's can't live without so that they will pressure distributions to update to and support those features. I would prefer that over the scenario I'm afraid we're heading for too. The question is, what shiny features will ISVs clamor for? The concerns I've voiced could turn out to be completely unfounded, too. It would be great if were the case. Tim -- Tim Mooney[EMAIL PROTECTED] Information Technology Services (701) 231-1076 (Voice) Room 242-J6, IACC Building(701) 231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 __ RPM Package Managerhttp://rpm5.org Developer Communication Listrpm-devel@rpm5.org
