Re: Support for variable number of arguments in popt
On Sep 7, 2011, at 12:41 PM, Jeff Johnson wrote: > > On Sep 7, 2011, at 12:30 PM, Wichmann, Mats D wrote: > >> >> there's some precedent (not a lot) for using a separator other than space >> for this purpose, often a comma since it happens to be neutral to the >> shell. as in: >> >> --optionlist=foo,bar,baz >> > > Yup. A comma separated list is actually what POPT uses for its Bloom filter > implementation, where a series of comma separated strings > are parsed as keys into a set container for arbitrary opaque strings > that can be used/testsed through a set membership mechanism. > > Implemented here > #define POPT_ARG_BITSET 16U+14U /*!< arg ==> bit set * > where arg is a comma separated set of strings. > > There really isn't any guidance on a "standard" compliant solution. > It wouldn't be impossibly hard to add POPT_ARGFLAG_COMMALIOST as an option modifier permitted into POPT_ARG_ARGV/POPT_ARG_BITSET and just add a split sloop for strings passed to those options. todo++ … but likely months before releasing POPT with POPT_ARGFLAG_COMMALIST, and years before any distro bothers to upgrade POPT, judging from RHEL6's and Fedora's "Fork y'all!" insistence to never use any code I've touched (both RHEL6/Fedora continute with popt-1.13 years after release.) 73 de Je __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Support for variable number of arguments in popt
On Sep 7, 2011, at 12:30 PM, Wichmann, Mats D wrote: > > > On Wed, Sep 7, 2011 at 9:16 AM, Jeff Johnson wrote: > > On Sep 7, 2011, at 4:44 AM, Christian Schmidt wrote: > > > Hi, > > > > I'm trying to parse a command line that can contain multiple variable > > number of arguments for a parameter, e.g. > > > > -r /dev/sda /dev/sdb -m /dev/sdc /dev/sdd -b 16m … > > > > Traditionally, uglix was designed with the model >PROGRAM OPTIONS ARGUMENTS > and that is captured in strict POSIX (and still common on *BSD systems iirc). > > The default option processing enhances --options=arg to take the "next" > item, and there is the "=" that has been added to introduce some form > of "stickiness". > > What you are seeking with multiple arguments to a single option > (and I understand well why you are attempting) isn't well supported > anywhere afaik. The usual solution is to embed white space within > quotes like > > -r "/dev/sda /dev/sdb" -m "/dev/sdc /dev/sdd" -b 16m > > > there's some precedent (not a lot) for using a separator other than space > for this purpose, often a comma since it happens to be neutral to the > shell. as in: > > --optionlist=foo,bar,baz > Yup. A comma separated list is actually what POPT uses for its Bloom filter implementation, where a series of comma separated strings are parsed as keys into a set container for arbitrary opaque strings that can be used/testsed through a set membership mechanism. Implemented here #define POPT_ARG_BITSET 16U+14U /*!< arg ==> bit set * where arg is a comma separated set of strings. There really isn't any guidance on a "standard" compliant solution. Unless POSIX is mandating ',' as a separator somehow … which would be truly daft *insanity* imho ;-) > > if you use space, semicolon, etc. you do need to quote it. > And these are shell, not POPT, uglix derived restrictions of course. 73 de Jeff
Re: Support for variable number of arguments in popt
On Sep 7, 2011, at 4:44 AM, Christian Schmidt wrote:
> Hi,
>
> I'm trying to parse a command line that can contain multiple variable
> number of arguments for a parameter, e.g.
>
> -r /dev/sda /dev/sdb -m /dev/sdc /dev/sdd -b 16m …
>
Traditionally, uglix was designed with the model
PROGRAM OPTIONS ARGUMENTS
and that is captured in strict POSIX (and still common on *BSD systems iirc).
The default option processing enhances --options=arg to take the "next"
item, and there is the "=" that has been added to introduce some form
of "stickiness".
What you are seeking with multiple arguments to a single option
(and I understand well why you are attempting) isn't well supported
anywhere afaik. The usual solution is to embed white space within
quotes like
-r "/dev/sda /dev/sdb" -m "/dev/sdc /dev/sdd" -b 16m
> The reason for the space separated lists is simply shell expansion.
> Given existing device nodes you could write -r /dev/sd[ab] -m
> /dev/sd[cd] -b 16m ... to reach the same. Similar expansion could be
> done with {a..z} or wildcard patterns. However this is not a case of
> leftover arguments since the context (in the above example, -r or -m) is
> relevant.
>
Got it.
> So far I tried to use POPT_ARG_ARGV (which expects one -r/-m for each
> argument) as well as poptPeekArg from within a r/m handler to look at
> the next word in the command line. However, poptPeekArg only seems to
> work after all arguments are parsed.
>
Yes I can believe that poptPeekArg() isn't going to work for what you
intend. I _MIGHT_ be able to make poptPeekArg() "work" that way, but
its rather tricky to unravel nested coroutine recursions to get
that job done. The implementation would likely need a backtracking
parser so that all the options copuld be parsed out to determine
what exactly the "next" argument should be. POPT is already doing
that to support the rather obscure "!#:+" syntax used by popt aliases.
(aside)
The syntax choice was derived from KSH/BASH … but its dreadfully obscure
shell functionality and only "!#:+" is implemented in POPT.
Here's some usage examples from /usr/lib/rpm/rpmpopt:
# [--dbpath DIRECTORY""use database in DIRECTORY"
rpm alias --dbpath --define '_dbpath !#:+'
# [--ftpport ] "port number of ftp server (or proxy)"
rpm alias --ftpport --define '_ftpport !#:+'
# [--ftpproxy ] "hostname or IP of ftp proxy"
rpm alias --ftpproxy--define '_ftpproxy !#:+'
# [--httpport ] "port number of http server (or proxy)"
rpm alias --httpport--define '_httpport !#:+'
# [--httpproxy ]"hostname or IP of http proxy"
rpm alias --httpproxy --define '_httpproxy !#:+'
You _MIGHT_ be able to consume the next two arguments by rewriting -m/-r
with an alias, and then rewriting what is passed to the program with
a POPT_ARGFLAG_DOC_HIDDEN helper option (--mstuff/--rstuff used in example) like
program alias -m--mstuff !#:+ --mstuff !#:+
program alias -r--rstuff !#:+ --rstuff !#:+
if the number of "arguments" to -m and -r is always 2 strings.
> Is there a way to realize what I'm trying within the current version of
> popt, and if not, would there be any interest (or chance of being added)
> implementing a POP_ARG_LIST that would act like POPT_ARG_ARGV, but take
> all arguments until the next parameter starting with a "-"?
>
There's likely some other alternatives using a POPT callback (caveat: it
would require peeping into POPT internal data structures)
And its possible to rewind/restart POPT argument processing, but that
too is quite tricky (RPM used to do multiple passes on options/args
in order to extract one option that then modified how the options were
further processed: total PITA to maintain and not KISS reliable engineering …)
hth
73 de Jeff
> Regards,
> Christian
> __
> POPT Library http://rpm5.org
> Developer Communication List popt-devel@rpm5.org
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: Please remove the following content
On Jul 5, 2011, at 4:32 AM, Hulin, Gerlinde (E D EA SOL D TC1) wrote: > > > Hello, > The following two mails got in your archive without my agreeement. I cannot > accept that my complete adress and email contact is online. My memory of the series of events leading to these messages goes like this: -> You sent mail privately asking about POPT licensing. <- I replied that the concerns should be posted to . POPT is an open-source project that conducts matters "transparently" and publicly, particular for matters like license audits. -> You posted publicly here: http://rpm5.org/community/popt-devel/0244.html <- The issues were replied to here: http://rpm5.org/community/popt-devel/0245.html You did ask (and received an answer) on a public mailing list. -> You (again) requested a resolution to your issues here: http://rpm5.org/community/popt-devel/0246.html <- You received two replies here: http://rpm5.org/community/popt-devel/0247.html http://rpm5.org/community/popt-devel/0248.html And now you wish to undo actions that you initiated, with replies as requested, because you dod not "consent" to publishing your contact information on line? At a minimum, your contact information is in more than the two messages you are requesting being removed. And removing only 2 messages is _NOT_ considering that the public mailing list is reflected in many other locations on the internet. Even if I removed the messages in , you seem to have given (imho, I am not a lawyer) implied consent by initiating actions that contained your contact info and choosing to publish on a public mailing list. I can/will remove the messages in if there is better reason than that you did not realize what "public" and "transparent" means when you requested information. Otherwise, consistent with policy @rpm5.org, I'd prefer to leave the public record exactly as is: other actions, like removing messages and otherwise changing the public record, are inconsistent with the intents and goals of a FL/OSS project. 73 de Jeff >>> > Please remove the conent from the archive > > http://www.mail-archive.com/popt-devel@rpm5.org/msg00245.html > http://www.mail-archive.com/popt-devel@rpm5.org/msg00246.html > > Thank you very much > > Best regards, > Gerlinde Hulin > > -Original Message- > From: Mail Archive Support [mailto:themailarch...@gmail.com] > Sent: Wednesday, June 29, 2011 10:34 PM > To: Hulin, Gerlinde (E D EA SOL D TC1) > Subject: Re: Please remove the following content > > Gerlinde, > > All the information we have on that list can be found here: > http://www.mail-archive.com/popt-devel@rpm5.org/info.html > > Beyond that, we do not know the contact information for the mailing > list administrators. Most mailing list services have a "-owner" > address where you can reach the list admin (for example, if the list > is gos...@jab.org then the list admin can likely be reached at > gossip-ow...@jab.org). If this does not work, you will need to search > for the website that hosts the mailing list, or post a question to the > mailing list. > > Tom > Support > The Mail Archive > > On Wed, Jun 29, 2011 at 8:35 AM, Hulin, Gerlinde (E D EA SOL D TC1) > wrote: >> Hello, >> Who is the list administrator ? Since this email got published without my >> aproval I do not know who added this content. However you can see from my >> email account that my email account is in the two web pages. >> _http://www.mail-archive.com/popt-devel@rpm5.org/msg00245.html_ >>> _http://www.mail-archive.com/popt-devel@rpm5.org/msg00246.html_ >> Best regards, >> Gerlinde Hulin >> >> -Original Message- >> From: Mail Archive Support [mailto:themailarch...@gmail.com] >> Sent: Wednesday, June 29, 2011 4:53 PM >> To: Hulin, Gerlinde (E D EA SOL D TC1) >> Subject: Re: Please remove the following content >> >> Gerlinde, >> >> Thanks for writing. The Mail Archive's policy is to delete messages >> only with the approval of the list administrator. We are providing a >> free service for list administrators and the admin needs to be the >> final decision maker on archived list content. >> >> Please discuss your request with the list admin and have him contact >> us directly if he wishes to remove the posts from the archive. >> >> Tom >> Support >> The Mail Archive >> >> On Wed, Jun 29, 2011 at 12:25 AM, Hulin, Gerlinde (E D EA SOL D TC1) >> wrote: >>> Hello, >>> The following two mails got in your archive without my agreeement. I cannot >>> accept that my complete adress and email contact is online. >>> >>> Please remove the conent from the archive >>> >>> _http://www.mail-archive.com/popt-devel@rpm5.org/msg00245.html_ >>> _http://www.mail-archive.com/popt-devel@rpm5.org/msg00246.html_ >>> >>> Thank you very much >>> >>> Best regards, >>> Gerlinde Hulin >>> >>> >>> >>> >> >> >> >> -- >> Support >> The Mail Archive >> www.mail-archive.com >> > > > > -- > Support > The Mail Archive > www.mail
Re: AW: INFO: Popt 1.14 license evaluation - response appreciated
On Mar 14, 2011, at 7:41 AM, Hulin, Gerlinde (E D EA SOL D TC1) wrote: > Hello, > > I was wondering if you could respond to my question ? Are ou going to get the > issues sorted out regarding the license texts ? Is the most important issue -- highlighted in RED -- the fact that the POPT license is no longer available at ftp://ftp.rpm.org/pub/rpm/dist If that's the issue, you will have to contact the owners of the rpm.org domain (i.e. Red Hat) and tell them to put their license where oit belongs. I've left the original license text in POPT exactly as written for historical continuity. But if changing a URI is all that you want/need, I can/will change the URI. 73 de Jeff > > Best regards, > Gerlinde Hulin > > > _ > Von: Hulin, Gerlinde (E D EA SOL D TC1) > Gesendet: Thursday, March 03, 2011 10:15 AM > An: 'popt-devel@rpm5.org' > Cc: Hufnagel, Michael (E D EA SOL EN) > Betreff:INFO: Popt 1.14 license evaluation - response appreciated > > Hello, > Am doing the liecense clearing for open source components in our department. > I did the clearing your component popt1.4. > I scanned all files of popt 1.14 for license informations. Unfortunately > there are issues regarding the popt licenses. Most of the licenses does not > match with the MIT license. Some files are under the GPL (wich is > irreversible !), and 2 license informations are not correct. Jef Johnson told > me to post to this email in order to get the issues sorted out. > On the home page popt is stated as MIT style license > http://freshmeat.net/tags/mitx-consortium-license. But here is what I found: > 1) Global Licenses > > MIT style - disclaimer X Consortium for promotion Copyright (c) 1998 Red > Hat Software > > Permission is hereby granted, free of charge, to any person obtaining a copy > of this software and associated documentation files (the "Software"), to deal > in the Software without restriction, including without limitation the rights > to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > copies of the Software, and to permit persons to whom the Software is > furnished to do so, subject to the following conditions: > > The above copyright notice and this permission notice shall be included in > all copies or substantial portions of the Software. > > THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN > AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. > > Except as contained in this notice, the name of the X Consortium shall not be > used in advertising or otherwise to promote the sale, use or other dealings > in this Software without prior written authorization from the X Consortium. > <36> \artifact.unpacked\popt-1.14\COPYING > <294> \artifact.unpacked\popt-1.14\install-sh > <3070> \artifact.unpacked\popt-1.14\config.sub > <2250> \artifact.unpacked\popt-1.14\config.guess > <14434> \artifact.unpacked\popt-1.14\ltmain.sh > <125> \artifact.unpacked\popt-1.14\popthelp.c > <47> \artifact.unpacked\popt-1.14\poptint.h > <0> \artifact.unpacked\popt-1.14\test1.c > <0> \artifact.unpacked\popt-1.14\po\sv.po > <0> \artifact.unpacked\popt-1.14\po\vi.po > <0> \artifact.unpacked\popt-1.14\po\fr.po > <0> \artifact.unpacked\popt-1.14\po\pl.po > <0> \artifact.unpacked\popt-1.14\po\de.po > <44> \artifact.unpacked\popt-1.14\popt.h > <49> \artifact.unpacked\popt-1.14\poptparse.c > <2797> \artifact.unpacked\popt-1.14\popt.3 > <50> \artifact.unpacked\popt-1.14\poptconfig.c > > > 2) Other Licenses – NO NOT USE – license issue > Open Issue... /* (C) 1998-2002 Red Hat, Inc. -- Licensing details are in > the COPYING >file accompanying popt source distributions, available from >ftp://ftp.rpm.org/pub/rpm/dist. */ ... This strin occurs in many > places but the > this link here is not correct > Wrong license information # This file is distributed under the same > license as the popt package. > # This file is put in the public domain. > <0> \artifact.unpacked\popt-1.14\po\fi.po > > Dual License GPL or LGPL without any version number -> this is wrong /* > Popt Library Test Program Number Too > > --> "a real world test of popt bugs" <-- > Copyright (C) 1999 US Interactive, Inc. > This program can be used under the GPL or LGPL at your > whim as long as this Copyright remains attached. > */ > <0> \artifact.unpacked\popt-1.14\test2.c > > > > 3) Other Licenses > GPL V2 or later Reference with Autoconf Exception_cct > > This file is free software; you can redistribute it and/or modify > it unde
Re: AW: INFO: Popt 1.14 license evaluation - response appreciated
On Mar 14, 2011, at 7:41 AM, Hulin, Gerlinde (E D EA SOL D TC1) wrote: > Hello, > > I was wondering if you could respond to my question ? Are ou going to get the > issues sorted out regarding the license texts ? > I thought this reply (Mark Hatle is part of the the @rpm5.org project that maintains POPT) was sufficient: http://rpm5.org/community/popt-devel/0245.html If Mark's answer is not sufficient to get the "issues sorted out", you will have to be more specific about what the issue(s) are than mailing a highlighted web form from an automated scanner in order for me to tell what issues remain. What I see is that your automated scanner is confused about generated files from GNU tools using in POPT. 73 de Jeff > Best regards, > Gerlinde Hulin > > > _ > Von: Hulin, Gerlinde (E D EA SOL D TC1) > Gesendet: Thursday, March 03, 2011 10:15 AM > An: 'popt-devel@rpm5.org' > Cc: Hufnagel, Michael (E D EA SOL EN) > Betreff:INFO: Popt 1.14 license evaluation - response appreciated > > Hello, > Am doing the liecense clearing for open source components in our department. > I did the clearing your component popt1.4. > I scanned all files of popt 1.14 for license informations. Unfortunately > there are issues regarding the popt licenses. Most of the licenses does not > match with the MIT license. Some files are under the GPL (wich is > irreversible !), and 2 license informations are not correct. Jef Johnson told > me to post to this email in order to get the issues sorted out. > On the home page popt is stated as MIT style license > http://freshmeat.net/tags/mitx-consortium-license. But here is what I found: > 1) Global Licenses > > MIT style - disclaimer X Consortium for promotion Copyright (c) 1998 Red > Hat Software > > Permission is hereby granted, free of charge, to any person obtaining a copy > of this software and associated documentation files (the "Software"), to deal > in the Software without restriction, including without limitation the rights > to use, copy, modify, merge, publish, distribute, sublicense, and/or sell > copies of the Software, and to permit persons to whom the Software is > furnished to do so, subject to the following conditions: > > The above copyright notice and this permission notice shall be included in > all copies or substantial portions of the Software. > > THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR > IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, > FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE > X CONSORTIUM BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN > AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN > CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. > > Except as contained in this notice, the name of the X Consortium shall not be > used in advertising or otherwise to promote the sale, use or other dealings > in this Software without prior written authorization from the X Consortium. > <36> \artifact.unpacked\popt-1.14\COPYING > <294> \artifact.unpacked\popt-1.14\install-sh > <3070> \artifact.unpacked\popt-1.14\config.sub > <2250> \artifact.unpacked\popt-1.14\config.guess > <14434> \artifact.unpacked\popt-1.14\ltmain.sh > <125> \artifact.unpacked\popt-1.14\popthelp.c > <47> \artifact.unpacked\popt-1.14\poptint.h > <0> \artifact.unpacked\popt-1.14\test1.c > <0> \artifact.unpacked\popt-1.14\po\sv.po > <0> \artifact.unpacked\popt-1.14\po\vi.po > <0> \artifact.unpacked\popt-1.14\po\fr.po > <0> \artifact.unpacked\popt-1.14\po\pl.po > <0> \artifact.unpacked\popt-1.14\po\de.po > <44> \artifact.unpacked\popt-1.14\popt.h > <49> \artifact.unpacked\popt-1.14\poptparse.c > <2797> \artifact.unpacked\popt-1.14\popt.3 > <50> \artifact.unpacked\popt-1.14\poptconfig.c > > > 2) Other Licenses – NO NOT USE – license issue > Open Issue... /* (C) 1998-2002 Red Hat, Inc. -- Licensing details are in > the COPYING >file accompanying popt source distributions, available from >ftp://ftp.rpm.org/pub/rpm/dist. */ ... This strin occurs in many > places but the > this link here is not correct > Wrong license information # This file is distributed under the same > license as the popt package. > # This file is put in the public domain. > <0> \artifact.unpacked\popt-1.14\po\fi.po > > Dual License GPL or LGPL without any version number -> this is wrong /* > Popt Library Test Program Number Too > > --> "a real world test of popt bugs" <-- > Copyright (C) 1999 US Interactive, Inc. > This program can be used under the GPL or LGPL at your > whim as long as this Copyright remains attached. > */ > <0> \artifact.unpacked\popt-1.14\test2.c > > > > 3) Other Licenses > GPL V2 or later Reference with Autoconf Exception_cct > > This file is free software; you can
Re: [PATCH] popt: fix pkgconfig dir definition
On Dec 24, 2010, at 12:54 AM, Matt Turner wrote:
> Hi,
> Attached is a patch which fixes the pkgconfidir definition in popt's
> Makefile.{in,am}.
>
> They were incorrectly defined as
> pkgconfigdir = /lib/pkgconfig
>
> which is wrong since is not always lib/.
>
The problem I see with the patch is that
--prefix=/usr --libdir=/lib
is common installing popt on linux, where /usr/lib/pkgconfig,
not /lib/pkgconfig, is the desired path.
There's no clear win afaict.
Thanks for the patch!
73 de Jeff
> Thanks!
> Matt Turner
>
> http://bugs.gentoo.org/show_bug.cgi?id=349558
>
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: popt with msvc?
On Oct 29, 2010, at 11:03 AM, Matt Arsenault wrote: > > On Oct 28, 2010, at 8:43 AM, Jeff Johnson wrote: > >> >> On Oct 28, 2010, at 12:50 AM, Matt Arsenault wrote: >> >>> >>> Other than those failed tests, I do now have it working without the need to >>> revert to an older version. >>> >> >> If you send alon a patch for VS, I'll try to carry that in popt (I don't do >> VS or Windoze but ..) >> There's a patch for Xcode in popt already ... popt changes little enough >> that VS might be >> maintainable in spite of bit rot. >> > Most of the patch is just casting malloc's since MSVC demands it; there isn't > all that much besides that. > > I didn't want to figure out how to use CVS, so I found that somebody seems to > have imported it to Github. I have a patch on top of the copy I found here: > http://github.com/devzero2000/POPT Wuss ;-) CVS is _EASY_ compared to git ;-) > Will that work / is this tracking the main CVS repository or something, or is > it just some random person? Nope, not random: same person who set up POPT in git will likely be the one to figger how to integrate your VS patch ... my dance card is full atm ;-) Thanks for the patch. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt with msvc?
On Oct 28, 2010, at 12:50 AM, Matt Arsenault wrote: > > Other than those failed tests, I do now have it working without the need to > revert to an older version. > If you send alon a patch for VS, I'll try to carry that in popt (I don't do VS or Windoze but ..) There's a patch for Xcode in popt already ... popt changes little enough that VS might be maintainable in spite of bit rot. The gnulib approach used by Mingw is way way overkill for a teensy library like POPT imho. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt with msvc?
On Oct 27, 2010, at 12:25 AM, Matt Arsenault wrote: > > On Oct 26, 2010, at 11:27 PM, Jeff Johnson wrote: >> What test are failing? Like "long long" and double, the tests are quite >> complete >> over data types and It Really Doesn't Matter: hardly ant program needs/uses >> long long and double CLI options. > > test1 - 19, 20, 21, 22, 23, 60, 61 19-23 are testing alias/exec functionality from test-poptrc. If you can live without alias/exec (and you likely can, only rpm makes heavy/serious use of popt alias/exec afaik). 60-61 are just cosmetic --help comparisons. If popt alias/exec isn't functional, then the --help display will fail too. > test3 - 51, 52, 53 > > I just sort of haphazardly threw things together until it built, so I may > have broken something. I quickly added cmake to try building it since I have > no idea how to get autotools to work with VS. Yes the AutoFu chains are getting rather daunting. The basic trick with autotools, is use configure, and avoid using the autotools, on tricky platforms. I haven't a clue how to get AutoFu to work with VS either. You might try older POPT tarballs. The basic functionality of POPT hasn't changed since forever, nor has the API, nor are there significant bugs. Older tarballs might have less AutoFu. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt with msvc?
On Oct 26, 2010, at 11:20 PM, Matt Arsenault wrote: > > On Oct 25, 2010, at 11:18 PM, Jeff Johnson wrote: > >> >> On Oct 25, 2010, at 10:49 PM, Matt Arsenault wrote: >> >>> Has anyone ever gotten popt to build with visual studio? I've been using >>> MinGW, but I need to get it to work with MSVC. I'm trying to link the MinGW >>> build from VS and it's not really working. If not, is there an alternative >>> that does? >> >> What isn't working? > > Mostly I get a lot of linker errors, which seem to be looking for MinGW > stuff, such as > > LINK : warning LNK4098: defaultlib 'MSVCRTD' conflicts with use of other > libs; use /NODEFAULTLIB:library > 1>LINK : warning LNK4098: defaultlib 'LIBCMT' conflicts with use of other > libs; use /NODEFAULTLIB:library > 1>libpopt.a(popt.o) : error LNK2019: unresolved external symbol > ___mingw_vfprintf referenced in function _fprintf > 1>libpopt.a(popthelp.o) : error LNK2001: unresolved external symbol > ___mingw_vfprintf > 1>libpopt.a(poptint.o) : error LNK2001: unresolved external symbol > ___mingw_vfprintf Can't help with those symbols. > 1>libpopt.a(popt.o) : error LNK2019: unresolved external symbol _strtoll > referenced in function _poptGetNextOpt > 1>libpopt.a(popt.o) : error LNK2019: unresolved external symbol ___strtod > referenced in function _poptGetNextOpt Hmm popt mostly doesn't need use strtoll or strtod. The usage case could be finessed out if necessary. I doubt you would notice ot care. > 1>libpopt.a(popthelp.o) : error LNK2019: unresolved external symbol > ___mingw_vsprintf referenced in function _sprintf > > Since then I've tried getting popt built with visual studio. I've gotten it > actually built (mostly by replacing C99 stuff), but about halfway through the > tests they start failing. What test are failing? Like "long long" and double, the tests are quite complete over data types and It Really Doesn't Matter: hardly ant program needs/uses long long and double CLI options. > __ > POPT Library http://rpm5.org > Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt with msvc?
On Oct 25, 2010, at 10:49 PM, Matt Arsenault wrote: > Has anyone ever gotten popt to build with visual studio? I've been using > MinGW, but I need to get it to work with MSVC. I'm trying to link the MinGW > build from VS and it's not really working. If not, is there an alternative > that does? What isn't working? 73 de Jeff > __ > POPT Library http://rpm5.org > Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Parsing ASN.1 with POPT 2.0? (was Re: [CVS] RPM: popt/ configure.ac)
On Jul 23, 2010, at 1:04 PM, devzero2000 wrote: > On Fri, Jul 23, 2010 at 6:22 PM, Jeff Johnson wrote: >> Careful here ... >> >> ... popt is used on many many more platforms than RPM. >> >> The issue that I see is invariably that GCC options are often >> not present in older compilers. >> >> The right fix (imho) is to set CFLAGS outside >> of AutoFu. Otherwise you end-up chasing down >> GCC versions with logic to set options dependent >> on version which becoems quite hard to manage. > Not a problem, IHMO. The command adds the flags needed only if the > compiler understands them, compiling a test program as usual. > If I run configure on a platform with an older GCC there are no > problems (also on another operating system .e.g. AIX ). > > It's ok for you ? > My only criteria with POPT patches is The "make check" MUST succeed. (and hopefully without shortcuts and hack-o-rounds but there are times that hackery is needed too.) >> >> I'm not averse to what you are doing whatsoever, >> just, well, the "fix" for a reported problem will >> be to revert. >> > I want to participate more actively. I hope that what I do, however > trivial, its usefulness anyway. If not please tell > and i will stop. > Go for it! I may critique but I do not wish to forbid anything. FL/OSS developers are really good at finding flaws, less good at identifying features/design. So I have no worries about regressions/mistakes. > And who knows if in the second edition of Oreilly "Beautiful testing" > will be cite also popt :=) other that clamav. > ;-) >> BTW, there's -fPIE that's likely more important for >> an Elf library than almost any other security measure. > Well also the other flags are important. > >> track whether that's in popt or not, ignore my aside if -fPIE >> is present. > I will add in a moment, it is not present now and i miss it. New topic: I've been debating whether to use POPT as an ASN.1 parser in POPT-2.0. Both POPT and ASN.1 are TLV (type/length/value) table driven parsers even if the usage cases are quite different (options & args != RPC marshalling). Mostly what I would attempt is to map either the libtasn1 format (its a tree of named nodes in an array) or the NSS ASN.1 format (which is directives not unlike POPT_ARGFLAG_FOO but without named nodes) into a popt table. Then I'd stub in a method to substitute an ASN.1 blob for parsing rather that argv[arc] as POPT currently uses. I'd likely duplicate the methods used while parsing argv[argc] into a different code path before attempting to unify (gulp) ARGV and ASN.1 parsing within the same routines. The ulterior motive for attempting ASN.1 parsing would be to look carefully at vectorizing POPt with callbacks. If I can trick POPT into parsing ASN.1 I'm almost positive that POPT parsing will be quite a bit more general than currently. Opinions? __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
A better example usage case for POPT_ARG_BITSET
One of the more subtle additions in popt-1.16 is
POPT_ARG_BITSET
This a Bloom filter data type that parses option
arguments as a CSV and adds to the "bit set".
What is difficult to describe is why/how Bloom filters
"work" (wikipedia does an excellent job) and particularly
how to describe
Why would I _EVER_ need Bloom filters doing CLI option parsing?
So here's a perfectly sane usage case that I'll be implementing in RPM
shortly.
One of the mysteries in RPM is %config handling. In short, a file path
marked with %config is handled with a state machine that tries to
avoid accidentally discarding local customizations during "upgrade".
However, what is often needed is a means (through POPT options) to
disable some functionality, or also to negate the behavior (like "grep -v")
Since there are like 300K possible file paths on existing systems, this is a
rather
complex task if arrays/lists are used to attach a per-file disabler using POPT.
One can certainly fiddle up the application code to process POPT
options in any number of ways.
Or (as I'm gonna do) one can let POPT_ARG_BITSET do most of the heavy lifting
for you
handling a large estimated universe (in this case potentially 300K file paths)
with almost no additional application code needed. This POPT helper
int poptBitsChk(poptBits bits, const char * s);
will return TRUE/FALSE depending on whether a string was mentioned (or not).
There's already a a test case (popt/tdict.c) based on /usr/share/dict/words
in popt code.
Fiddle around with tdict.c if you wish to attempt using POPT_ARG_BITSET
in some application.
Wayne: RSYNC+POPT likely has zillions of usage cases for POPT_ARG_BITSET
with large unknown(a priori) populations of file paths (and enablers/disablers
attached to a specific path). Note that the "default" expected population size
in POPT is way too small for 300K+ file paths (from popt/popt.h):
#define _POPT_BITS_N1024U/* estimated population */
#define _POPT_BITS_M((3U * _POPT_BITS_N) / 2U)
#define _POPT_BITS_K16U /* no. of linear hash combinations */
but can be overridden by assigning values to
/*...@unchecked@*/
extern unsigned int _poptBitsN;
/*...@unchecked@*/
extern unsigned int _poptBitsM;
/*...@unchecked@*/
extern unsigned int _poptBitsK;
There's also the attached bit of deep dark chocolate from RPM code (where
the POPT_ARG_BITSET Bloom filter implementation in POPT was swiped from)
if interested in an "optimal" set of parameters with a "false positive"
failure rate less than specified for a given estimated population size.
See the appended routine.
(aside)
I left rpmbfParams() (the mnemonic is RPMBF == RPM Bloom Filter, duh) routine
out of POPT largely because of the math complexity and no clearly established
usage case for POPT_ARG_BITSET so far. I also hadn't implemented rpmbfParams()
yet at the time I did the port to POPT ;-).
I almost certainly will include some variant of the attached routine
to estimate "optimal" bitset parameters in POPT 2.0 before I'm done.
hth
73 de Jeff
===
void rpmbfParams(size_t n, double e, size_t * mp, size_t * kp)
{
static size_t _nmin = 10;
static size_t _n = 1;
static size_t _nmax = 1 * 1000 * 1000 * 1000;
static double _emin = 1.0e-10;
static double _e = 1.0e-4;
static double _emax = 1.0e-2;
size_t m;
size_t k;
/* XXX sanity checks on (n,e) args. */
if (!(n >= _nmin && _n <= _nmax))
n = _n;
if (!(e >= _emin && _e <= _emax))
e = _e;
m = (size_t)((n * log(e)) / (log(1.0 / pow(2.0, log(2.0 + 0.5);
k = (size_t) ((m * log(2.0)) / n);
if (mp) *mp = m;
if (kp) *kp = k;
}
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: POPT 2.0: renaming to "popt2" vs. loader maps and ELF versioned symbols
On Jun 20, 2010, at 9:14 AM, Wichmann, Mats D wrote: >> >> I can/will chase down a "straw-man" implementation using loader maps >> and versioned symbols for "compatibility" over the next month >> because I _REALLY_ think renaming to "popt2" is just wussiness. > > fwiw, I've found more than one instance of projects getting symbol > versioning religion, just adding it, and creating considerable confusion. > if libfoo 1.14 is unversioned and 1.15 suddenly is versioned then you > may get surprises. it's not an utter disaster to do both: bump to > 2.0 and use all the versioning tricks from there. > POPT added symbol versioning using loader maps years ago. No confusion has been reported (although I'm quite sure that distros are not enabling symbol versioning) If distros don't enable the mechanism to support symbol versioning that might permit both POPT 1.0 and POPT 2.0 interfaces in a single library, that's no problem a developer can solve. I don't mind bumping the soname (although that too voids all attempts at "compatibility" using symbol versioning). I personally don;y think "compatibility" is all that important. There's zillions of versions of popt around, all "work", and I don't expect too much interest in using POPT 2.0, judging from comments (and the number of members on this list). I _DO_ mind renaming the entire project to "popt2" and adding -lpopt2 for the sole semiotic purpose of messaging to lusers: Danger Will Robinson! Toriodal turbulence in the API/ABI is detected! > disclaimer: I haven't looked at what issues you may be facing, > going to 2.0 may indeed be as you characterize it :) Largely I'm just collecting "features" desired in POPT 2.0. As always, the only "feature" that everyone wants is "No incompatibilities!" Which is trivially solved by doing nothing, which is still an alternative development choice for POPT. Thanks for comments. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT 2.0: version stamping in includes
On Jun 18, 2010, at 9:03 PM, Michael Jennings wrote:
>
> Ah, okay. I guess I was envisioning for runtime checks:
>
> void *f = dlopen(NULL, RTLD_LAZY);
> if (dlsym(f, "popt_something")) {
>...
> }
>
Any idea how "portably" POSIX dlsym() is in 2010?
There's the GNUish dlvsym() that needs figgering if I attempt
"compatibility" with symbol versioning too.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: POPT 2.0: version stamping in includes
On Jun 18, 2010, at 8:13 PM, Michael Jennings wrote: > On Friday, 18 June 2010, at 14:01:05 (-0400), > Jeff Johnson wrote: > >> Should I go to the effort of a run-time "features" acquisition API? > > IMHO, no. Any time you add fluff, you get further away from the meat, > and there's room for error. The least error-prone approach is to > check for X when you use X, check for Y when you use Y, etc., or at > least check for the most recently-added of X, Y, and Z. But the > moment you start checking for symbol Q which is supposed to indicate > the presence of X but isn't actually X, gratuitous abstraction and > potential for mistakes are introduce for no (again, IMHO) gain. > But note run-time, where its not always possible to do the test, particularly if popt was built by vendors differently, or different versions or feature sets. What I was envisioning is merely a Bloom filter with an API like bool poptHasFeature(const char * s) Then during build the strings/aliases are pumped into a Bloom filter, and the resultant bitsset is just stashed into octets and fed to (already exists in popt-1.16) int poptBitsChk(/*...@null@*/poptBits bits, /*...@null@*/const char * s) /*...@*/; It would be quite a simple API, and reasonably general/extensible (depending on how the bitset was generated during build) without all the usuall "fluff" of tables of arrays of ptrs of ... ... but since POPT is unlikely to accumulate 32 "features" this century, well, it could just be a simple enum of "features" returned as an int32. Hmmm ... I suspect I'll be lazy and do nothing until I'm forced. > Of course, in reality, whatever the documentation examples show as the > "right way to check for POPT 2.0" is precisely what will be > copied-and-pasted into code for the next half-century. But you know > this better than I. > Thanks for opinions. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT 2.0: version stamping in includes
On Jun 18, 2010, at 1:45 PM, Michael Jennings wrote: > On Friday, 18 June 2010, at 10:52:22 (-0400), > Jeff Johnson wrote: > >> (aside) >> Instead of a #define version, I typically us a "de facto" check >> for POPT in compatibilities. E.g. in order to use POPT 2.0 >> in rpm-5.3.2 I'll likely do >> #if defined(POPT_ARGFLAG_CALCULATOR) >> ... this is POPT 2.0 ... >> #else >> ... this is NOT POPT 2.0 ... >> #endif > > I've always preferred this technique myself. Versioning macros and > such often fail to account for things like SVN snapshots or human > error. Always better to look for what you're actually trying to > *use*. > Should I go to the effort of a run-time "features" acquisition API? (aside) For a "toy" library like POPT, all of this is horrendous overkill engineering. OTOH, horrendous overkill engineering is _EXACTLY_ what has made both POPT/RPM successful. Less maintenance -> more usage. YMMV. But that's MHO on the matter. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Disallowing args
On Jun 18, 2010, at 12:26 PM, Wayne Davison wrote: > On Thu, Jun 17, 2010 at 9:21 AM, Jeff Johnson wrote: > Meanwhile in the future, if you could send a patch against some POPT release > that I can find > > Sorry about that. I sent you a patch for the rsync popt by mistake. Here's > one I just made against 1.16 that now includes a couple tests for the arg > rejection. > > http://opencoder.net/arg-error.patch > Oooh, test cases too. Thank you! (aside) I'm gonna need some help from non-English speakers in order to get test cases for --help column alignment (and generally handling wide characters correctly) This dumb 'merikkan cannot be expected to type up reproducers for problems that he cannot see or understand. Sure I can see that the --help columns don't line up even with ired old bifocal encrusted eyes. That ain't what I'm asking for ... 73 de Jeff
POPT 2.0: adding setjmp/longjmp?
Related to the issue of throwing exceptions to C++ and other modern application environments on errors is providing some sort of restart handling within POPT itself. In C this is usually done using setjmp(3)/longjmp(3). Note that POPT has always had a rather primitive method to restart implemented in /** \ingroup popt * Reinitialize popt context. * @param con context */ /*...@unused@*/ void poptResetContext(/*...@null@*/poptContext con) /*...@modifies con @*/; so it would not be very hard to tie the already implemented "reinitialize" method into a setjmp(3)/longjmp(3) exception handling framework. (aside) With a little thought, I believe I can likely add other restart points within option/arg handling, its just another stack (and POPT already has 2 stacks now in the process of being unified into one. Adding a dynamic run-time element such as implied by setjmp(3)/longjmp(3) is largely adding another token to a union "item" which is pushed/popped accirding to well-defined rules. (aside to the aside) There's other automagic doing, say, cleanup using atexit(3) that would simplify using POPT's API as well (and get POPT off of everyone's bleeping widdle valgrind exercise hit lists without forcing me to debug everyone's POPT bugs ad nauseum. I can go many ways here. Some exception handling framework is needed for POPT to be easily used in modern languages like C++/Java/... and (in C) this means setjmp(3)/longjmp(3)/atexit(3). Opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT 2.0: version stamping in includes
More administrivia related to "incompatibilities" There is no current means (because I've always thought versioning is a crock) to detect what POPT API version is being used. (aside) Instead of versioning, I've always been ultra-careful not to introduce any incompatibilities into POPT's API/ABI. All that changes with POPT 2.0, a major release with _PLANNED_ incompatibilities. So the RFE for some stoopid #define that carries a version stamp that can be tested is quite predictable. (aside) Instead of a #define version, I typically us a "de facto" check for POPT in compatibilities. E.g. in order to use POPT 2.0 in rpm-5.3.2 I'll likely do #if defined(POPT_ARGFLAG_CALCULATOR) ... this is POPT 2.0 ... #else ... this is NOT POPT 2.0 ... #endif while idly waiting the necessary 2-3 years for the opportunity to use POPT code I wrote in RPM code I'm writing. Isn't it ironic? But I have a distinct edge "knowing" how to use POPT "portably" and so some silly version stamp is gonna be needed. I'll likely just steal the RPM versioning and push into POPT unless I hear other suggestions. Other suggestions might include a run-time API (like pcre/neon/openssl to mention 3 off the top of my head) that could be implemented so that one could track "features" which can be enabled/disable specifically into applications. Its all a crock and all unneded for a toy library like POPT (jmho, ymmv), but the RFE is predictable no matter what. Opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT 2.0: LCOV/GCOV and mccabe metrics
I use many automated tools to guide my hacking. 1) LCOV/GCOV coverage tests I've run lcov on POPT head yesterday, the results are here: http://jbj.org/files/popt/lcov-1.17/ Note that function coverage at 100% is an immediate benefit of doing ISPRAS "shallow" testing. But click through /X/src/popt if you want to see what code is actually exercised by "make check". 2) mccabe metrics I use mccabe metrics mostly to confirm what I already know, where the "cesspools" are located. Here are the 4 worst offenders from popt-1.16. (the 1st column is the metric, lower is better, a metric of 30 tends to have a 50% chance of having a "bug" through real world correlation testing): 80 80 160 1299253 popt.c(1299): poptGetNextOpt 53 63 138 307 205 popthelp.c(307): singleOptionHelp 48 48 103 263 127 poptconfig.c(263): poptConfigLine 29 30 37 578 69 popt.c(579): findOption And the same for POPT head: 74 76 171 1422273 popt.c(1422): poptGetNextOpt 53 63 140 314 207 popthelp.c(314): singleOptionHelp 48 48 103 263 127 poptconfig.c(263): poptConfigLine 29 30 37 601 69 popt.c(602): findOption The reduction from 80 -> 74 is due to a rewrite to handle POPT_ERROR_UNWANTEDARG. I'll have the mccabe "cholestrol" count down below 50 before I'm done. Meanwhile I'm going yo have to move from POPT -> RPM hacking for a bit, I have rpm-5.3.2 to get out on 7/1, and I also need to try and see what damage I've done by using POPT head under RPM for a bit. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT 2.0: renaming to "popt2" vs. loader maps and ELF versioned symbols
There's a thread on gcrypt-devel that is directly
relevant to POPT library "compatibility" and symbol handling:
http://lists.gnupg.org/mailman/listinfo/gcrypt-devel
(You will get a "403 Forbidden" for the gcrypt-devel archives, go find a mirror)
And for reference, there's Drepper's deep dark chocolate DSO guidance here:
http://people.redhat.com/drepper/dsohowto.pdf
(aside)
And gcrypt-devel is unrelated to generating 64bit RSA keypairs using
a RPN calculator in POPT. ;-)
There are (at least) two approaches to adding symbol versioning
and visibility to a DSO library.
1) use a loader map
already implemented in POPT, just "opt-in" and so
never used.
2) use GCC __attribute__(("visibility")) voo-doo
Using __attribute__ can be used to do "opt-out" symbol hiding
without the modest overhead of loader maps.
The other (and different) issue is symbol versioning. The connection
to "visibility" is largely because a loader map can also be used
to add symbol versions quite easily.
(aside)
There's also the distantly related/coupled issue (if I choose to
add ELF symbols to the RPN calculator) of using
dlsym() # in POSIX and should be "portable"
versus
dlvsym()# GNU extensions necessary with versioned symbol lookup
I can/will chase down a "straw-man" implementation using loader maps
and versioned symbols for "compatibility" over the next month
because I _REALLY_ think renaming to "popt2" is just wussiness.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
POPT 2.0: another malloc issue
This is another ancient (circa 1997) in POPT.
The argv returned through certain interfaces ('d have
to check, but the issue bites me every couple months)
has everything allocated in one contiguous blob so
that an experienced/lazy programmer just has to do
const char * s = "bing bang boom";
int ac = 0;
const char ** av = NULL;
int rc = poptParseArgvString(s, &ac, &av_;
if (av)
free(av);
i.e. no loop over that strings in the array is needed.
The issue is that there are corner cases that do
not conform to general expectations, where a string
can be reallocated, or otherwise manipulated freely
without considering whether the memory is part of
an argv array returned from POPT.
(As an experienced/lazy programmer I use this POPT (and RPM)
functionality a fair amount).
The problem is that in order to attempt "tuples" by parsing
a CSV attached to a single option like
--option bing,bang,boom
uniformly with POPT_ARG_ARGV (and I can think of several other
usage cases for "tuples"), how the memory was allocated
affects how "tuple" merges are implemented.
I'm likely to change the behavior in POPT 2.0 to allocate
all the strings independently to simplify "tuple" merging.
Which means that applications will have to explicitly write the
loop (and its trivial to add a helper function in POPT) to do
for (i = 0; i < ac; i++)
if (av[i]) free(av[i])
free(av);
But I can be convinced otherwise, and perhaps *shudder* arrange
for a "Have it your own way!" selector logic if I absolutely MUST
do that.
Personally I'd rather see KISS even if incompatible. It just
ain't that big a deal to write a loop where needed.
But -- as always -- this means I'm forced to debug everyone's
widdle valgrind exercises if/when POPT is abused and used incorrectly.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
POPT 2.0: C89 or C99?
There are various tedious "portability" issues in need of sorting out, particularly if I'm gonna start chasing more aggressive functionality like pattern matching and mutexes/threads/C++/SWIG "stuff" in POPT 2.0. The names of integers is always the silly start of the discussion. Up till now I've largely targeted C89 (and haphazardly verified conformance if/when issues were reported). I personally don't care whether I type unsigned long long or uint64_t (I prefer "uint64_t" if that matters. My fetishes don't matter ...) What I do care is "consistency" in POPT, and clearly stated goals. So I'm gonna assume #include (with typedef retrofits for the C99 challenged because that's easy to fix with AutoFu if needed) OTOH, I will continue adding declarations in the usual places unlike C99 (because that's a much more invasive change that cannot be fixed with AutoFu). I also see little use for other C99 arcana like bool etc and so will avoid using. The above is just an statement of de facto coding practice in POPT, but now clearly and explicitly stated. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Disallowing args
On Jun 5, 2010, at 11:39 AM, Wayne Davison wrote: > Here's something that was recently fixed for the popt that is included > with rsync: rejecting an arg to an option that doesn't take an arg. > > Attached is a patch. A new error code, POPT_ERROR_UNWANTEDARG, was > created to make the error message nice. This handles both -l=value > and --long-arg=value where neither one is supposed to take a value. > > ..wayne.. > Hmmm, I'll get this "fix" worked in down the road. Certainly a bug. (aside) All this code is actively being simplified for POPT 2.0 so that I can more easily follow the logic path. Meanwhile in the future, if you could send a patch against some POPT release that I can find (I tried all the way back to the popt-1_11-release tag, patch doesn't apply), or at least provide some hint about what code repository the patch applies to (the patch itself seems to have been edited), that would certainly help. Please also note that I won't accept any patch that doesn't pass "make check" regression tests. If you _REALLY_ want to make sure that odd POPT corner cases like args given to an option return an error, adding a reproducer to the test cases too. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 17, 2010, at 4:27 AM, Markdv wrote: > > I just don't see why you wouldn't want to "fix" this. Seems like all you'd > have to do is add > >con->os->nextArg = _free(con->os->nextArg); > > to poptFreeContext(poptContext con) and be done with it. > There's a double free with your suggested "fix" if/when the application has also free'd the memory returned. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
> > What? A bug - albeit not a very serious one - that's been there for ever is, > therefore, not a bug? > That isn't what I said, in fact quite the opposite. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 16, 2010, at 4:59 PM, Markdv wrote:
> Jeff Johnson wrote:
>> I happen to have a valgrind trace on my screen that shows the issue
>> ==25160== ==25160== 49 bytes in 1 blocks are still reachable in loss record
>> 2 of 2
>> ==25160==at 0x4005BDC: malloc (vg_replace_malloc.c:195)
>> ==25160==by 0x314D9A: poptGetNextOpt (popt.c:1203)
>> ==25160==by 0x40697CD: rpmcliInit (poptALL.c:751)
>> ==25160==by 0x804AC45: main (rpmqv.c:385)
>> ==25160== The "memory leak" is actually a POPT design feature. Every string
>> returned from POPT is malloc'd so that an application
>> can do whatever it wishes with the string without
>> worrying about side effects of fiddling with the memory.
>> Unfortunately, POPT is mostly not used correctly, and the expectation is
>> Hey POPT handles argv strings, I shouldn't _HAVE_ to manage those!?!
>
> I don't mind having to manage those, as long as it is documented.
>
> I just started using libpopt for the very first time today and checked my
> code with valgrind which showed a leak. The minimal program below always
> leaks an amount of memory equal to the length of the argument of the
> "hostname" option +1. Despite the fact that I think I'm free-ing everything
> I'm supposed to free... After skimming the libpopt source I inserted the two
> commented lines that, when uncommented (duh), seem to fix the leak... lucky
> shot I guess. :)
>
> Does this not demonstrate that there actually is a memory leak in popt?
I use popt & valgrind daily in all sorts of configurations.
So no, it does _NOT_ demonstrate that there's a leak in popt.
The whole point of my e-mail is/was that the arg is malloc'd by
intent so that the application can do whatever it wishes
with the returned memory.
And its not sufficient to document the behavior (which has been this
way for a decade). Inquiring minds want to know even if the
amount of memory is <100b.
>
> If not, I would appreciate someone explaining how I'm using libpopt wrongly.
>
> #include
> #include
> #include
>
> /** #include "/tmp/popt-1.14/poptint.h" **/
>
You should not ever need/want to include "poptint.h" which
contains routines and data structures internal to popt.
> struct opts {
>char*hostname;
>int verbose;
> } opts;
>
> struct poptOption optTable[] = {
>{"hostname", 'h', POPT_ARG_STRING, &opts.hostname,
>0, "hostname", "HOST" },
>{"verbose", 'v', POPT_ARG_NONE, &opts.verbose,
>0, "be verbose", 0 },
>POPT_AUTOHELP
>POPT_TABLEEND
> };
>
> int main( int argc, const char **argv )
> {
>
>poptContext optCon;
>
>optCon = poptGetContext(NULL, argc, argv,
>optTable, POPT_CONTEXT_POSIXMEHARDER);
>
Do you _REALLY_ want/need POSIXMEHARDER? The issue is
largely whether options must strictly proceed args or not.
IIRC (I don't use anything but popt for option processing)
getopt_long() default behavior in linux is _NOT_ POSIXly correct
but rather the more convenient process --foo where ever found.
>poptGetNextOpt(optCon);
>
There's usually a loop over poptGetNextOpt. E.g. here's what is in RPM
/* Process all options, whine if unknown. */
while ((rc = poptGetNextOpt(optCon)) > 0) {
const char * optArg = poptGetOptArg(optCon);
optArg = _free(optArg);
switch (rc) {
default:
fprintf(stderr, _("%s: option table misconfigured (%d)\n"),
__progname, rc);
exit(EXIT_FAILURE);
/*...@notreached@*/ /*...@switchbreak@*/ break;
}
}
The free in the loop is what is needed for "squeaky clean".
But truly, I wouldn't fuss too much about 10-20 bytes of "leak".
glibc leaks far more than that, but the leaks are masked in valgrind.
And technically none of these are "leaks" because its one-time
allocation, not continually increasing unfree'd memory.
>printf("Options:\n"
> "hostname: %s\n"
> "verbose: %d\n",
> opts.hostname,
> opts.verbose);
>
>free(opts.hostname);
> /**free(optCon->os->nextArg); **/
>
>poptFreeContext(optCon);
>
See if the loop fixes, then try using poptGetOptArg()
and free the result.
Whether that is obvious or convenient or correct or documented is a different
matter.
But whatever behavior is implemented in POPT has been largely unchanged this
century.
>return 0;
> }
>
>
> Regards,
RPN calculator wire-up
My last check-in parses (approx.) dc(1) like
syntax to extend pre-existing functionality
for handling short/int/long/longlong bit
operations (like OR/AND/XOR) to most arithmetic
operations.
Whether you like RPN syntax (RPN is unnatural to those
used to infix parentheses as taught in 3rd grade), or
think that a RPN calculator is daft (its a means to an end imho)
are rather peripheral issues.
The extension in POPT 2.0 from bit masking to all arithmetic
operations was achieved _WITHOUT_ all this baggage in
the API:
/** \ingroup popt
* \name Arg modifiers
*/
/*...@{*/
#define POPT_ARGFLAG_ONEDASH0x8000U /*!< allow -longoption */
#define POPT_ARGFLAG_DOC_HIDDEN 0x4000U /*!< don't show in
help/usage */
#define POPT_ARGFLAG_STRIP 0x2000U /*!< strip this arg from
argv(only applies to long args) */
#define POPT_ARGFLAG_OPTIONAL 0x1000U /*!< arg may be missing */
#define POPT_ARGFLAG_OR 0x0800U /*!< arg will be or'ed */
#define POPT_ARGFLAG_NOR0x0900U /*!< arg will be nor'ed */
#define POPT_ARGFLAG_AND0x0400U /*!< arg will be and'ed */
#define POPT_ARGFLAG_NAND 0x0500U /*!< arg will be nand'ed */
#define POPT_ARGFLAG_XOR0x0200U /*!< arg will be xor'ed */
#define POPT_ARGFLAG_NOT0x0100U /*!< arg will be negated */
#define POPT_ARGFLAG_LOGICALOPS \
(POPT_ARGFLAG_OR|POPT_ARGFLAG_AND|POPT_ARGFLAG_XOR)
#define POPT_BIT_SET(POPT_ARG_VAL|POPT_ARGFLAG_OR)
/*!< set arg bit(s) */
#define POPT_BIT_CLR(POPT_ARG_VAL|POPT_ARGFLAG_NAND)
/*!< clear arg bit(s) */
In fact (IIRC), about the only needed change was to add a validation
bit that sez' that opt->argDescrip does indeed contain a recipe
for option value transform parsing (including auto- inccrement/decrement
as trivial sub-cases):
#define POPT_ARGFLAG_CALCULATOR 0x0010U /*!< argDescr has RPN
string */
While there's many details that I'm not yet happy with using a RPN
caclulator, I'm rather interested in the KISS properties of
parsing a string rather than ornamenting an API/ABI with
all sorts of speshulness that makes POPT too hard to use.
Note that there's enough functionality implemented already
(integer exponentiation is 3 lines) that I can easily
see how to generate 64bit RSA keypairs using the RPN calculator.
I'll likely implement 64bit RSA keypair generation as a testcase
even if there's hardly a usage case.
(aside)
Please also note that most of the existing RPN implementation
was swiped from the BeeCrypto Calcultor, a JavaScript implementation
with MPI's that is most definitely able to generate cryptographic quality RSA
(and ECDSA) keypairs. The limitation of int64_t arithmetic in POPT
might well be removed by making the RPN calculator functionality
in POPT 2.0 "pluggable".
It's the pluggable API and the extensibility that I'm interested in seeing
in POPT 2.0, not a RPN calculator or arithmetic per-se.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: Zero tolerance for errors (like malloc returning NULL)
On Jun 14, 2010, at 12:52 PM, R P Herrold wrote: > On Mon, 14 Jun 2010, Jeff Johnson wrote: > >> The other approach is "Punish the moron. NOW!" LART 'em w a ruler and >> add assert's throughout the code on "can't happen" conditions. > > A kinder, gentler person might say: > die loudly and early when an error is detected, rather > than guess after an unknown state transition > and remove the value judgments of 'punish' and 'moron' ;) > > you know -- refusenik style > Well I'm speaking largely solipsistically, and the moron == me, when it comes to developing POPT 2.0. I'm just tightening my straitjacket before climbing into the Torquemada. But otherwise noted. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Exception handling for POPT 2.0?
Please note that I've never used _ANY_ exception in my entire professional life. However C is a dying language, and most modern languages like C++ and Java and ... handle error's on exception paths rather than returning error's. I'd like to see some reasonable exception handling so that POPT bindings become "drop-in" for C++/Java/... or anything else that SWIG wraps. I've looked multiple times at what needs doing for reasonable POPT bindings and always resorted to a barf bag instead. Anyone want to design an exception handling framework for POPT? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Wiring up pthread_create() like POPT exec
POPT already has exec aliases to do execv(2) with filtered/substituted argument lists. Why not threads? I see no reason why pthread_create() isn't a reasonable generalization of the existing POPT exec functionality. Yes there will be --without-pthreads AutoFu for the PTHREAD haters who do not wish -lpthreads as a POPT library dependency. Other opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Zero tolerance for errors (like malloc returning NULL)
There's basically two schools of thought for handling programmer/runtime errors in libraries. The "Have it your own way!" elitists claim that libraries should _NEVER_ exit and _ALWAYS_ return an error code. The other approach is "Punish the moron. NOW!" LART 'em w a ruler and add assert's throughout the code on "can't happen" conditions. For development of POPT 2.0 I prefer being punished, and so am going to wire up assert's throughout POPT to simplify the code. For "release" of POPT 2.0, I'm perfectly happy to change to the ... _NEVER_ ... _ALWAYS_ approach if that really matters. Hint: When was the last time you saw any application undertake a sane/reasonable corrective action and continue running when malloc happened to return NULL? I've never seen such an application (but my experience is limited). Meanwhile, assert's for POPT 2.0 development are about to be checked in so that I can see what needs to be done. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Dynamic symbol resolution for POPT tables
This will take some thought, but looks easy enough to do. Somehow with all the AutoFu layered in, POPT is already testing checking for dlfcn.h... yes which includes a run-time ELF symbol lookup /* Find the run-time address in the shared object HANDLE refers to of the symbol called NAME. */ extern void *dlsym (void *__restrict __handle, __const char *__restrict __name) __THROW __nonnull ((2)); which means -- if __handle on the current application can be achieved -- that POPT could do the symbol lookup at run-time, not through traditional linkage. That would add the ability to, say, do arithmetic on any ELF symbol in an application using a RPN calculator. Whether POPT 2.0 _SHOULD__ permit this type of functionality is a very different question. Opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 2:38 PM, Jeff Johnson wrote:
>
> AFAIK, all POPT args are either returned by value (like POPT_ARG_INT) or
> through malloc'd memory _ALWAYS_. The rules on table callbacks are different
> than the rules for the no-brainer, in-line loop based getopt(3) like
> processing.
>
> OTOH, some of the code paths in POPT are quite twisty, perhaps there's
> someplace
> where another strdup/malloc is needed.
>
Here is poptPeekArg() (I happen to have on my screen):
const char * poptPeekArg(poptContext con)
{
const char * ret = NULL;
if (con && con->leftovers != NULL && con->nextLeftover < con->numLeftovers)
ret = con->leftovers[con->nextLeftover];
return ret;
}
So no malloc with poptPeekArg() returns atm, largely for hysterical
reasons.
Its easy enough to add strdup (or not) to poptPeekArg(). What isn't clear is
whether a consistent rule like
All memory is malloc'd before return.
is preferred to the mysteriously inconsistent current behavior in POPT.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Handling CSV's as a POPT option argument type
The POPT_ARG_BITSET Bloom filter's in popt-1.16 already implicitly are assuming that option values are a (literally) CSV. And POPT_ARG_ARGV in popt-1.13 (when an option is used multiple times) already are storing a tuple. So I think its time to commit to some conventions (like CSV with ',' or more generally a set of characters for splitting a string into a tuple) so that a tuple can be passed as a single argument and/or with multiple occurrences of an option in a unified and transparent fashion. The pending event is how to deal with a "foo=bar" data mapping, like a symbol table or a python dict. Note that the Bloom filter implementation _ALREADY_ uses Bob Jenkins lookup3.c hash function, all that would remain to implement a mapping in POPT 2.0 is to add hash buckets and a poptHash data type. But I will proceed slowly and carefully to "mappings" in POPT 2.0. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: RPN calculator with +/- operations
On Jun 13, 2010, at 3:30 PM, Jeff Johnson wrote: > > todo++ still early yet for POPT 2.0 > Well after some thought, I believe that a RPN calculator (nominally dc(1) syntax with changes as needed) will help eliminate the litter of data typing that POPT has accumulated over the years. E.g. POPT has methods/types for all the usual int/short/long/longlong/float/double data types without any clear abstractions. All that's needed is coercion when import/exporting option values from an application. These days an internal int64_t (or uint64_t, signedness with bit operations is mostly useless and I don't expect much of a usage case for a RPN calculator in POPT). The one notable exception is RPM, which uses POPT quite extensively for parsing not only CLI options, but also config/macro values. RPM usage of POPT was what caused methods like poptSaveInt() et al to be added. But I can easily change those interfaces @rpm5.org at will and the need(s) of RPM have changed over the years so that, say, parsing Berkeley DB configuration for an rpmdb just isn't important any more (has never really been used/useful afaik). For discussion purposes, assume 1) an RPN calculator implementation for binary arithmetic operations. 2) the RPN calculator stack is initialized with 2 values (I'll call the values "X" and "Y" while describing) 3) The "X" value is pushed first and is essentially the "option value", either --foo=1234 or through the pointer in the 4th field of the table 4) The "Y" value is pushed second from the integer (and eventually long/ptr) from the 5th field of the table. 5) The RPN "program" (i.e. same as you would type to dc(1)) will come from the argDescrip 7th field in the table. That's enough of a convention for me to re-target RPM calls into POPT and eliminate most of the type-based methods like poptSaveInt() in POPT. I'll wire up the RPN binary arithmetic while muddling how strings and bit masks/sets (more useful for POPT) could/should be added to a RPN calculator abstraction. The next goal will be to attempt a per-item (not per-table) callback signature. For the short term, I'll assuma as starting point a no-brainer int main(int argc, char ** argv[]) calling signature and extend with POPT/application context arguments as needed. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
RPN calculator with +/- operations
On Jun 7, 2010, at 1:41 PM, Jeff Johnson wrote:
>
> (aside)
> I almost added a RPN calculator to popt-1.16 this weekend past.
> The implementation would have involved a table entry like
>
> int counter = 0;
>
> { "autoincrement", '\0', POPT_ARG_INT|POPT_ARGFLAG_RPN, &counter, 0,
> N_("Demonstrate autoincrmenting using a RPN calculator"), "1 +" },
>
Well I finally implemented autoincrement using a RPN calculator with a table
entry
(from popt/test1.c) like
{ "verbose", 'v',
POPT_ARG_VAL|POPT_ARGFLAG_CALCULATOR|POPT_ARGFLAG_DOC_HIDDEN, &aCounter, 1,
"POPT_ARGFLAG_CALCULATOR: increment a counter", "+" },
I almost certainly will change my mind as I proceed wiring up RPN "calculator"
functionality somehow.
There are several things I dislike with what I implemented,
basically lack of generality and too arcanely obscure to be useful (imho).
Specifically the implementation:
1) "works" only with longs/ints/shorts, not floats/doubles.
2) has a delta increment that is limited to 32 bits and must be
specified as "1"
(a variant of the need for an additional "incompatible" pointer field)
3) no clue how to display RPN calculator +/- in --help. Note that the
implementation already does more than "++" and "--", that would be easy
to add to --help spewage somehow.
4) will force the 7th table field to be used for POPT operations, not
just information in --help.
todo++ still early yet for POPT 2.0
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 4:40 PM, Danny Sung wrote: > > On 6/7/2010 11:29 AM, Jeff Johnson wrote: >> >> On Jun 7, 2010, at 2:00 PM, Danny Sung wrote: >> >>> I agree. I have no problems with breaking binary&source compatibility >>> especially if that change will make popt better/easier in the future. But a >>> change in name seems necessary for app developers to make that choice. eg. >>> what happens if you're building mulitiple different software packages... >>> both dynamically linking popt, but one uses 1.x and the other 2.x? The >>> header files would also need something like #include or >>> something like that. Can't say I particularly like it... but it does those >>> issues somewhat cleanly. >>> >> >> I respectfully and strongly _DISAGREE_. The issue can be best be >> dealt with by using ELF symbol versioning (once I get that it place >> again in popt-1.17) where a single library can provide _BOTH_ >> POPT 1.0 and POPT 2.0 functionality _TRANSPARENTLY_ (one the >> necessary pre-requsite of adding ELF symbol versioning is achieved). > > Doesn't this only work if you're maintaining source compatibility? (whiihc is > good, I just thought you said you weren't aiming for that). > Sorry, not trying to be difficult... just trying to understand. Also does > ELF symbol versioning work for both dynamically and statically linked apps? > ELF symbol versioning is what glibc uses to have a forward looking upgrade path. If it "works" for glibc, that's all I (and perhaps you) need to know. I do know more about ELF symbol versioning, its not very hard, if interested. Its largely a conscious design choice releasing a library, the rest is just "stuff". >> If I rename "popt" to "tdod" (or "popt2" that precludes even the attempt at >> doing better engineering using ELF symbol versioning. While I'm >> not averse to renaming, I've seen the bleary package churn from >> libxml -> libxml2 and GNOME 1.0 -> GNOME 2.0. No way Jose! > > Yeah, I thought it was ugly too. But it makes sense... in an ugly way. We agree "pugly". FWIW, months of my life has been spent staring at splint traces. If you think the code is ugly, wait until you see 10K lines of spewage (and attempt to "fix"). These days scan.coverity.com is better technology than splint (and POPT is already a registered project with scan.coverity.com, I signed up almost instantly when scan.coverity.com was announced). However, scan.coverity.com takes quite some time (months and years) to scan a new release, and is an exorbitanly prohibitively expensive proprietary tool for FLOSS development. Note that scan.coverity.com still thinks that rpm-2.5.x released in August, 1998 is still "current" software that is usefully scanned (because rpm-2.5.x is _STILL_ in NetBSD 12 years later). MHO differs considerably, I have no need to examine "bugs" in 12 year old releases for any purpose whatsoever. But I digress ... Which makes splint attractive for POPT development even if the annotations are quite pugly and intrusive. hth 73 de Jeff > __ > POPT Library http://rpm5.org > Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 4:16 PM, Danny Sung wrote: > > > On 6/7/2010 11:38 AM, Jeff Johnson wrote: >> >> On Jun 7, 2010, at 2:00 PM, Danny Sung wrote: >> >>> My personal choice in things I write is to expect the caller to strdup(). >>> But I understand the reentrancy issue here. (though why you'd be using >>> popt in a thread is beyond me at this time.. and it does have a poptContext >>> handle anyway). >>> >> >> popt does not use threads, applications do. so its applications, not popt, >> that force -lpopt into multi-threaded environments. All that POPT can >> do is to try and work transparently in both environments. > > Yeah, I meant as an app developer... I can't see why you'd want to use popt > outside of a single thread. That's the only context where reentrancy would > be an issue while using the popt library, no? > Ah ... There's lots of usage cases for popt arg processing. E.g. RPM has multiple embeddings and is just starting to explore the mysteries of multi-threaded development (its taken most of the last year to get posix mutexes stabilized and attached transparently to ~40 RPM objects) POPT is used for all option processing and I want "thread safe" engineering. > >>> If this is going to continue to be the behavior, I'd suggest a >>> poptFreeOptArg() call just to highlight the necessity (and deal with type >>> and NULL pointer checks). >>> >> >> Why add a new method for what is already a well known operation called >> free(3)? >> >> I mean I can add poptFreeArg in 1 #define to popt.h, but why bother? > > Because it's not just "free()" It's: > if(optarg) free((void *)optarg); > (aside) Hehe. FWIW, I deal with "free" issues (and the pesky GCC const warnings) using a static inline _free() routine. I can will expose my _free() (its already used pretty heavily internally to POPT) as poptFree() if you wish. So far I've just tried not to inflict my C coding fetishism on anyone else. It is *in fact* just laziness on the part of the developer to test for NULL and add the silly cast. Developer laziness is not a POPT problem per se. > Not a huge deal, but a bit of an inconvenience. Plus like I said the added > function just higlights the need to free the return from the Get call. I > actually have a str_delete() call in my standard library that encapsulates > the if. But I still have to call it like: str_delete((char *)optarg) just to > break the const to satisfy the compiler warnings. > > >> AFAIK, all POPT args are either returned by value (like POPT_ARG_INT) or >> through malloc'd memory _ALWAYS_. The rules on table callbacks are different >> than the rules for the no-brainer, in-line loop based getopt(3) like >> processing. > > Yeah, it's fine. As I said it was just a little unexpected when I first > started using popt. I didn't realize it until I ran a memory checker. And > from the 'const' type in the Get call I had thought it was a bug in popt. > > Looking over the man page right now (popt 1.16), I don't really see where it > talks about freeing this memory either. It's compounded by the fact that > there is a poptFreeContext and poptDupArg() calls. Those both imply to me > that popt is doing memory management for me. > > There's also a poptPeekArg() call. Is the app developer expected to free > that one as well? If so, then the example seems to lead to a memory leak as > well: > > portname = poptGetArg(optCon); > if((portname == NULL) || !(poptPeekArg(optCon) == NULL)) > usage(optCon, 1, "Specify a single port", ".e.g., /dev/cua0"); > My guess (not looked, just from memory) is that poptPeekArg _SHOULD_ return malloc'd arguments. The underlying issue is that POPT has a ar filtering construct based on the gawd awful bash syntax that looks like (here's an example from /usr/lib/rpm/rpmpopt-X.Y) # [--dbpath DIRECTORY""use database in DIRECTORY" rpm alias --dbpath --define '_dbpath !#:+' The --dbpath option is quite important for RPM and the ^%$^#^$^%#^%#$^% "!#:+" syntax was quite useful in ripping out _LOTS_ of tedious code in RPM setting what is essentially just a file path string argument to an option. The relation to "memory leaks" is that POPT had to start actively parsing argv[0] arrays in order to pull the next argument, and that turned out to be easier to do if arguments were malloc'd before return to caller. So poptPeekArg() _SHOULD_ return malloc'd memory (but may not do
Re: POPT code review URL
On Jun 7, 2010, at 3:34 PM, Jeff Johnson wrote: > > The bottom line was: excellent code > > ANd credit where it is due: > Erik Troan wrote most of POPT, I was just his packaging "bitch" ;-) > But I've long since re-written both POPT and RPM several times over, and > mostly > adiabatically ... > And last I offend inadvertently: Wayne Davisson has sent more useful POPT patches than anyone else on the planet earth. (aside) I do occaisionally receive encoded neutrino streams from SomeWhereElseInstead but I haven't a clue what it all means ... Yet. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT code review URL
I've mentioned several times, so I might as well post the URI to the POPT code review http://rusty.ozlabs.org/ Rusty is also strongly advocating POPT 2.0 (when I asked privately) and believes that "incompatibility" in an ABI isn't as important an issue as future usage cases (my paraphrase of his words). There's very little I personally disagree with in the review, all the wartlets in existing POPT code are well covered IMHO. The --help handling is quite gross and (currently) is 30-50% of POPT code with little (imho) justification or benefit. The "fix" in POPT 2.0 involves dealing with wide character encoding issues directly, atm POPT 1.0 is a confusing mess of octet and character (possibly wide) display indexing to get --help columns aligned. The coding to fix --help display column is quite easy _IFF_ I can achieve some "real world" reproducers and an acceptable API/ABI design. The issues have been mentioned repeatedly over the last couple of years on with no response. FOr better or wiorse, I'm just a dumb 'merikkan with only en_US as a native language. But that does not mean that I don't know the C wide character interfaces quite well. The splint annotations mentioned in the review are a different matter. While the annotations are immensely pugly & intrusive, it is weeks of staring at splint spewage that in fact led to POPT code in its existing state. The bottom line was: excellent code ANd credit where it is due: Erik Troan wrote most of POPT, I was just his packaging "bitch" ;-) But I've long since re-written both POPT and RPM several times over, and mostly adiabatically ... Just in case: adiabatically == legacy compatible in my private jargon. hth 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Teaching POPT 2.0 about HTML/XML/YAML?
On Jun 7, 2010, at 2:44 PM, Jeff Johnson wrote: > > On Jun 7, 2010, at 2:24 PM, Danny Sung wrote: > >> >> >> Actually I really like this idea. This would effectively introduce what >> could be a plugin architecture for popt. Ideally this should be done in a >> way where someone could create say an XML output and release a separate >> library called popt_help_xml or something. >> > > But the issue becomes > Where do I get an additional "void *" from? > in order to do > rc = (*callback) (..., callback_context); > One can always hash up (or otherwise add a "helper" table) to transform an integer back to an array, its called "table lookup", taught in CS 102 (or at least used to be before Python dicts and perl ties became all the Newer! Better! Bestest! rage). An API cpuld be insturmented to retrieve Yet More Goop from the existing "legacy compatible" 7-tuple table using additional "helper" tables and methods. Let's not go here _PLEASE_, my barf bag is already too close to overflowing, and when it does overfloweth, yours truly has to clean up the mess. But it is certainly possible to design API's for table lookup pointer retrieval if absolutely necessary and deemed important in POPT 2.0. Personally, I think "incompatibility" in POPT 2.0 is less important than designing in proper data structures that include an "obvious" and "simple" means for a callback context pointer. YMMV, everyone's does. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Teaching POPT 2.0 about HTML/XML/YAML?
On Jun 7, 2010, at 2:24 PM, Danny Sung wrote: > > > Actually I really like this idea. This would effectively introduce what > could be a plugin architecture for popt. Ideally this should be done in a > way where someone could create say an XML output and release a separate > library called popt_help_xml or something. > But the issue becomes Where do I get an additional "void *" from? in order to do rc = (*callback) (..., callback_context); The existing --help callbacks are per-table, not per-item, and so that forces the insanity of 1 option per-table in order to use. There are other, --help i18n insanities, with the exisiting POPT 1.0 callbacks too. > One of the concepts I've been playing around with is having the help system > double as a mechanism for programs to figure out what options other programs > have. > > For example, it seems common place these days to write GUI shells on top of > command-line utilities. However, it's difficult to know what exact > parameters are available for a specific build without a run-and-test > operation. > > So this mechanism would allow someone intrepid developer (or perhaps GUI > framework people like gnome/kde) to establish a standard for doing this sort > of thing. > > > > The plugin/callback thing goes even further though. Even your optionTable > could support something like this. So in the case of your auto-increment > feature... this could actually be a "plugin" to popt, allowing someone else > (either the app developer) or a plugin author to write. > > If done well, it's possible you'll start getting developers join you by > submitting plugins to the "default" installation. > > Also, I don't know how others feel about this. But if it makes things any > easier (esp. if it makes it easier for the app developer), I'm actually sort > of okay with an external utility generating the options table, and just > allowing me to #include it in my source. > Generating the options table form *WHAT*? If you have well defined markup from which a popt table can be generated, I'd rather see that utility internal to POPT itself, not external where noone else benefits from its existence. Nothing in the previous statement precludes anyone from generating POPT tables however they wish externally. Just internal can/will be maintained and tested and ... whatever else is needed. ATM, I know of no well-defined markup, nor any attempt to automatically generate POPT tables using an external utility. Translation: Show me the code. __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 2:00 PM, Danny Sung wrote: > My personal choice in things I write is to expect the caller to strdup(). > But I understand the reentrancy issue here. (though why you'd be using popt > in a thread is beyond me at this time.. and it does have a poptContext handle > anyway). > popt does not use threads, applications do. so its applications, not popt, that force -lpopt into multi-threaded environments. All that POPT can do is to try and work transparently in both environments. > But yeah, I guess I'd opt for consistency as well. The thing that surprised > me was that while poptGetOptArg() expects the caller to free it, it's still > returning a const. I use 'const' not as memory protection (we know it > doesn't do that), but as a signifier that the caller doesn't need to worry > about freeing the contents. But whatever, I just put a /* popt requires this > to be freed */ comment every time I use it. > Yes. I've added const's wherever possible and necessary in popt. Which is why I already know (from 10+ years of experience) that only PROT_READ segfaults will communicate to developers that its _NOT_ a POPT bug. > If this is going to continue to be the behavior, I'd suggest a > poptFreeOptArg() call just to highlight the necessity (and deal with type and > NULL pointer checks). > Why add a new method for what is already a well known operation called free(3)? I mean I can add poptFreeArg in 1 #define to popt.h, but why bother? > > If all Get's in popt allocate, that's fine (I actually hadn't noticed that > because I wasn't really expecting it). But if there's ever a time when you > have a mix, I'd also suggest function names containing "New" (or "Dup" or > "Alloc" or something of the sort) to drive home the point that something's > being allocated. "Get" to me typically means retrieve, not allocate. But > like I said, it's not a huge issue. Just add the poptFreeOptArg() and I > think it'll be clear for everyone. > AFAIK, all POPT args are either returned by value (like POPT_ARG_INT) or through malloc'd memory _ALWAYS_. The rules on table callbacks are different than the rules for the no-brainer, in-line loop based getopt(3) like processing. OTOH, some of the code paths in POPT are quite twisty, perhaps there's someplace where another strdup/malloc is needed. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 2:00 PM, Danny Sung wrote: > I agree. I have no problems with breaking binary&source compatibility > especially if that change will make popt better/easier in the future. But a > change in name seems necessary for app developers to make that choice. eg. > what happens if you're building mulitiple different software packages... both > dynamically linking popt, but one uses 1.x and the other 2.x? The header > files would also need something like #include or something > like that. Can't say I particularly like it... but it does those issues > somewhat cleanly. > I respectfully and strongly _DISAGREE_. The issue can be best be dealt with by using ELF symbol versioning (once I get that it place again in popt-1.17) where a single library can provide _BOTH_ POPT 1.0 and POPT 2.0 functionality _TRANSPARENTLY_ (one the necessary pre-requsite of adding ELF symbol versioning is achieved). If I rename "popt" to "tdod" (or "popt2" that precludes even the attempt at doing better engineering using ELF symbol versioning. While I'm not averse to renaming, I've seen the bleary package churn from libxml -> libxml2 and GNOME 1.0 -> GNOME 2.0. No way Jose! For libraries and projects of XML/GNOME size and magnitude, renaming may have been the only viable way forward. OTOH, popt is _TINY_ and I'm quite prepared for ELF symbol versioning (and I also know that I can figger out whether a pointer is to a POPT 1.0 or a Yet-To-Be-Devised POPT 2.0 table without too much pain. Won't be pretty code, and I am a lazy schmuck maintainer, but I _KNOW_ that I can solve the problem). The only other reason for renaming is to signal developers _EVERYWHERE_ that a change just happened. But that assumes that I am willing to describe the changes ad nauseam for the next couple of years, which is very much not the case. I much prefer "drop-in" transparent "compatibility" and am unwilling to preclude that engineering path by simply wussing out and renaming "popt" to "tdod" (or "popt2" or "getopt106" or ...) which doesn't begin to solve _ANY_ "compatibility" issue at all. In fact, renaming just sanitizes "incompatibility" without solving any problem. > I don't think it's that bad for developers to switch over. Old apps may take > a while. But people will know (especially if you put "#warning popt-1.x is > deprecated" in the headers for 1.x. =) ). And will move new software over. > Look at things like libxml2. > > Ideally, however, popt2 should include mechanisms that allow for future > versions to do this sort of versioning check at runtime. So this popt2 > actually becomes popt ABI 2.0. Not popt-2.0. (eg. popt3 may still use popt2 > ABI.. but then again with all the versioning there may never need to be a > popt3 =) ). > The project name is "POPT", the version being discussed is "2.0". There is no "popt2" nor (imho) is there any need for the renaming, nor setting the precedent of "popt3" -> ... -> "popt123456789" in the 30th century. 73 de Jeff > > On 6/7/2010 9:51 AM, Wayne Davison wrote: >> On Mon, Jun 7, 2010 at 9:40 AM, Jeff Johnson > <mailto:n3...@mac.com>> wrote: >> >>The added tyrrany of forcing every application that currently has >>-lpopt >>to change to >>-ljdod >>will be rate-determining to adoption (and glacially/tectonically >>slow imho) >> >> >> For me, if popt 2 is not compatible with popt 1 then I would rather have >> to make a conscious choice to upgrade my code (testing it for >> compatibility), and having to change the libray name as a part of that >> is pretty minor. Having a possibility for my program to be run-time >> linked with a library that is not compatible with what it expects would >> be very bad, imo. >> >> ..wayne.. > > -- > Please note my new email address: da...@dannysung.com > __ > POPT Library http://rpm5.org > Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 12:51 PM, Wayne Davison wrote:
> On Mon, Jun 7, 2010 at 9:40 AM, Jeff Johnson wrote:
> The added tyrrany of forcing every application that currently has
> -lpopt
> to change to
> -ljdod
> will be rate-determining to adoption (and glacially/tectonically slow imho)
>
> For me, if popt 2 is not compatible with popt 1 then I would rather have to
> make a conscious choice to upgrade my code (testing it for compatibility),
> and having to change the libray name as a part of that is pretty minor.
> Having a possibility for my program to be run-time linked with a library that
> is not compatible with what it expects would be very bad, imo.
>
Last I checked you already had made your choice and included your own
private version of -lpopt (and the reasons are perfectly understood and have
nothing to do with POPT 2.0).
Your decision paths are all status quo ante no matter what is done (or not) in
POPT 2.0.
And I point out that your decision paths are all independent of whether POPT
2.0 is
"compatible" or "renamed" or ... anything else.
Yes, your comments are valued, you send patches ;-)
Meanwhile the discussion on is intended to
explictly extract what problems/features need to be solved in POPT 2.0
before doing any coding.
(aside)
I almost added a RPN calculator to popt-1.16 this weekend past.
The implementation would have involved a table entry like
int counter = 0;
{ "autoincrement", '\0', POPT_ARG_INT|POPT_ARGFLAG_RPN, &counter, 0,
N_("Demonstrate autoincrmenting using a RPN calculator"), "1 +" },
where a simple uint64_t stack would push the existing value (with coercion),
parse simple
digit strings onto the uint64_t stack, with the usual toy calculator arithmetic
operations of + - * / % & | ... , and then transferring the result (again with
coercion)
back to the counter variable.
The exercise of in-fix -> reverse polish is left to inquiring minds. Hint: see
wikipedia.
Alas I got (and am) side-tracked with RSA/DSA/Elgamal/ECDSA generate & sign
methods
for RPM, so I did not get around to the RPN calculator and your "autoincrement"
request.
73 de Jeff
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 12:20 PM, Wayne Davison wrote: > On Mon, Jun 7, 2010 at 9:14 AM, Jeff Johnson wrote: > And without some deterministic way to tell whether its a POPT 1.x <-> 2.x > table being fed to the POPT API/ABI, well, only a deliberately > "incompatible" POPT 2.0 data structure with some deterministic > version identifier is possible. > > One way to deal with this is to change the library name to "popt2". That > way, a popt (1.x) using program would never get run with a popt2 library. > OK, let's go down that path ... Assume I change the name of "popt" to "jdod" (ASCII art upsidedown and reversed to make a point) What "compatibility" problem is solved by renaming? Doesn't fiddling the loader map to change the symbol versioning achieve "incompatibility" a bit more transparently. (aside) Eeek, no symbol versioning in existing -lpopt, time to fix that with popt-1.17, todo++. The added tyrrany of forcing every application that currently has -lpopt to change to -ljdod will be rate-determining to adoption (and glacially/tectonically slow imho) So slow that it may not even be worth the effort of developing and releasing POPT 2.0. That too is an option. (aside) There's no easy answer and I'm deeply conflicted, but am being encouraged to attempt a POPT 2.0 release to "fix" certain issues like --help. Go Google Rusty Russell's POPT code review. He (and I) want to see callbacks used, and a callback paradigm Done Right forces a "void *" context pointer like this (AFAICT) int rc = (*callback) (, void * callback_arg) which means that I either 1) overload the i18n field pointer(s) with Yet Another Obscure Overloading (the per-table pre- and soit- and post- callbacks are already quite tricky to use correctly) 2) add another pointer field to avoid overloading and introduce "instant incompatibility" I can go either/both/neither way. My interest is in useful and simple software, nothing more. Again, I use "jdod" just to illustrate why renaming to -lpopt2 is just the tip of a large iceberg. (you know that already). 73 de Jeff
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 12:02 PM, Jeff Johnson wrote: > > Changing the soname or even (gasp!) using ELF symbol versioning > is quite doable, all the necessary precursor elements have been in place > for years. > I should point out the deep flaw using ELF symbol versioning ... POPT tables are compiled into applications, and so versioning is dictated by the application, not by POPT itself. And without some deterministic way to tell whether its a POPT 1.x <-> 2.x table being fed to the POPT API/ABI, well, only a deliberately "incompatible" POPT 2.0 data structure with some deterministic version identifier is possible. But perhaps there's some insanely obscure way to disambigauate POPT tables that I haven't been able to devise. Have at and send a patch, I really loathe being out guessed (because I'm out numbered by zillions of POPT lusers to essentially 1 developer) since forever. I have thought long and hard for many years ... "incompatible" POPT 2.0 seems the easiest (though rocky) path into the future. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 11:50 AM, Mark Hatle wrote: > Jeff Johnson wrote: >> On Jun 7, 2010, at 10:37 AM, Mark Hatle wrote: >>> The way I've usually addressed this is to either add an option to the >>> library that changes the default behavior from strdup to passing the >>> address with the expectation of const. >>> >> I'd rather _NOT_ go the "Have it your own way!" route in >> a API/ABI becuase it adds yet another datum that needs to >> be extracted from lusers doing POPT support. > > Runtime not build time. For a library like popt, build time would be really > bad. > I was thinking "run-time" ... > For runtime, it should be a simple 'switch' for the strdup call. > ... and there's nothing "simple" about insturmenting choice in POPT underneath applications. Its either Yet Another Arcane flag that noone uses or a goosey-loosey envvar that makes debugging run-time dependent, see the tedium of POSIX_ME_HARDER that is already in POPT since forever. But sure I know the "Have it your own way!" run-time drill quite well too. Just I'm hoping not to go there, I do like KISSy-poo. >>> Either by adding const style prototypes/functions, or by using some >>> mechanism to change the behavior of all of the functions. >>> >> (aside) >> Well I've gone multiple ways with the C borkage of "const char **" >> vs "char *const *" in POPT and RPM. These days there's so much spewage >> from GCC that I don't believe that compilers or language hints solve >> any "real world" issue. But I can/will go to PROT_READ mandatory hardware >> enforcement using mmap(2) if that is what is desired. Its easier >> to implement the code than it is to discuss the various religion's >> coding fetishism's Yet Agin. But I digress ... >> Note that it _WAS_ GCC's writable strings that forced the malloc >> into POPT in the first place, where POPT sometimes returned >> strings from RO memory, and sometimes from argv, and the morons were >> confused. >> Adding the strdup() to POPT was 1 less issue to worry about. Another >> digression ... > > Ya, I don't doubt it. I believe the issue comes down to consistency. Either > everything popt returns should be duplicated so that the user has to clear > them, or nothing should be so the user knows that it's expected to be > read-only. (Enforcing read-only is another issue entirely, and one that I > don't really think is popt's responsibility. Set the prototype properly on > the functions and if the user violates them so be it.. they live with the > consequences.) > What is currently implemented is the former, malloc everything returned, and those with clue will figger it out. That's general, just surprising. But I can/will make the audit's easier if that is what is desired. Its a bleeping string, deal with it, stop fussing POPT "leaks" is MNSHO privately. >>> My biggest concern is the potential retrofit of existing apps that expect >>> the current behavior.. but I agree with many of the submitters.. popt >>> really should be sending const points and then the app needs to strdup. >>> >> And so "legacy compatibility" again again again. POPT 2.0 is a major change >> with no "compatibility" implied or intended. Meanwhile I will strive to >> make the change as painless as possible. But there are issues (aka "deep >> hacks") >> that have been in POPT for a decade that aren't the right thing to do. And >> even >> though I've overloaded just about everything possible in the 7-tuple of a >> POPT table item, there are these issues: > > If both popt 1 and popt 2 can live on the same machine and have different > sonames, then I vote for no strdup.. it's the users problem to duplicate what > they want. > Changing the soname or even (gasp!) using ELF symbol versioning is quite doable, all the necessary precursor elements have been in place for years. > IMHO libraries should not protect the user from themselves. If they are dumb > enough to modify memory or expect that memory will live beyond a certain > pre-defined lifecycle.. then it's their problem to fix... (Note of course > reentrancy and such could be an issue.. but thats only a problem if it's > allowed by popt.) > Hehe, so PROT_READ LART'ing is where you want to go ;-) That would be fine with me except I'm also trying to avoid any POPT "support" headaches. Note that C macros could be used to ease the pain of "long" <-> "void *" punning. But I _REALLY_ need another pointer field somehow without overloading the two i18n --help fields which are already way way way too complicated for my taste. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: POPT's API has designed in memory leaks. What to do?
On Jun 7, 2010, at 10:37 AM, Mark Hatle wrote: > The way I've usually addressed this is to either add an option to the library > that changes the default behavior from strdup to passing the address with the > expectation of const. > I'd rather _NOT_ go the "Have it your own way!" route in a API/ABI becuase it adds yet another datum that needs to be extracted from lusers doing POPT support. > Either by adding const style prototypes/functions, or by using some mechanism > to change the behavior of all of the functions. > (aside) Well I've gone multiple ways with the C borkage of "const char **" vs "char *const *" in POPT and RPM. These days there's so much spewage from GCC that I don't believe that compilers or language hints solve any "real world" issue. But I can/will go to PROT_READ mandatory hardware enforcement using mmap(2) if that is what is desired. Its easier to implement the code than it is to discuss the various religion's coding fetishism's Yet Agin. But I digress ... Note that it _WAS_ GCC's writable strings that forced the malloc into POPT in the first place, where POPT sometimes returned strings from RO memory, and sometimes from argv, and the morons were confused. Adding the strdup() to POPT was 1 less issue to worry about. Another digression ... > My biggest concern is the potential retrofit of existing apps that expect the > current behavior.. but I agree with many of the submitters.. popt really > should be sending const points and then the app needs to strdup. > And so "legacy compatibility" again again again. POPT 2.0 is a major change with no "compatibility" implied or intended. Meanwhile I will strive to make the change as painless as possible. But there are issues (aka "deep hacks") that have been in POPT for a decade that aren't the right thing to do. And even though I've overloaded just about everything possible in the 7-tuple of a POPT table item, there are these issues: 0) overloading with bit masks and long <-> void * punning is quite mysterious to most. 1) bits in certain fields are _ALREADY_ ambiguous for certain values, with subtle "conventional" contextual tests of other entries needed while using POPT. 2) there's a need for Yet Another Pointer for callbacks with context (changing item #5 from "int" to "long" would suffice but is "instantly incompatible" so I'd rather do something less mysterious). Thanks for comments however. And ignore my scarcasm if it offends. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT's API has designed in memory leaks. What to do?
I happen to have a valgrind trace on my screen that shows the issue ==25160== ==25160== 49 bytes in 1 blocks are still reachable in loss record 2 of 2 ==25160==at 0x4005BDC: malloc (vg_replace_malloc.c:195) ==25160==by 0x314D9A: poptGetNextOpt (popt.c:1203) ==25160==by 0x40697CD: rpmcliInit (poptALL.c:751) ==25160==by 0x804AC45: main (rpmqv.c:385) ==25160== The "memory leak" is actually a POPT design feature. Every string returned from POPT is malloc'd so that an application can do whatever it wishes with the string without worrying about side effects of fiddling with the memory. Unfortunately, POPT is mostly not used correctly, and the expectation is Hey POPT handles argv strings, I shouldn't _HAVE_ to manage those!?! I get a tedious bug report every couple of months from otherwise honest attempts to use valgrind for application "squeaky clean" memory auditing. Should this behavior be changed in POPT 2.0? It's a 1-liner change to remove a strdup() somewhere, but the change does have profound (but minor, who actually cares about a 49b 1-time memory leak these days) ramifications. Meanwhile I'm way tired of explaining why its _NOT_ a memory leak, but rather buggy use of POPT. Opinions welcomed. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Teaching POPT 2.0 about HTML/XML/YAML?
Since --help is the the topic de jure ... ... adding a popt table processor callback to transform --help spewage opaquely into whatever form is desired, with some simple transforms like HTML tables or DocBook XML markup or YAML used for configuring persistent option default settings, or the existing --help spewage, is likely useful. Most --help display formats aren't /dev/tty these days ... its more painful to do the /dev/tty plumbing than it is to just pass along popt table entries to output methods. Note that a opaque (*help_transform) pluggable callback is what I propose ... no way do I want to see POPT start acquiring heavyweight linkages like -lxml2/-lexpat/-lsyck/-lwhatever. But almost certainly a reasonable API could be added so that --help transforms, with some reasonably simple (like the existing --help CLI display) transform vectors within -lpopt explicitly. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Disallowing args
On Jun 5, 2010, at 12:42 PM, Danny Sung wrote:
> On 06/05/2010 8:56 AM, Jeff Johnson wrote:
>
>> (aside)
>> Anything you want to see in POPT 2.0? I'm collecting features ...
>>
>
> Since you're collecting features... =)
>
> One thing I'd like is to extend the help/usage capability just a little.
>
Well reworking --help is one of the primary motivations for POPT 2.0.
Hysterically, encoding was rammed into POPT by GNOME way back when.
While the scheme is workable, the original change _HAD_ to be done
as an addition to preserve ABI, and so all of the i18n for --help
is rather awkwardly done.
To make matters worse, another patch from GNOME wanted POPT to undertake
localization transforms from UTF8 to whatever was in LC_ALL etc.
While iconv(3) makes the character encoding transform as "trivial" as deciding
the to <-> from locales, its hardly trivial to make that decision reliably
based solely on a desired LC_ALL environment variable (the from locale cannot
be determined reliably, and hackery like trying various from locales like
glib does is hardly sound engineering).
The killer is that the data needed to find --help alignment reproducers is
application resident,
and application chosen, and the GNOME developers who forced the iconv(3) into
POPT just aren't helping with reproducers. I have no interest in supporting
POPT functionality that noone is willing to help maintain any more.
So the short answer is:
All of the --help handling needs to be scrapped and reworked in POPT
2.0.
For starters, 30-40% of the code is --help related, and _ALL_ of the bug reports
I've heard for years are
But --help columns don't align!
Enough already ... bring on the --help bulldozers!
> So I'd like to be able to have more descriptive usage parameters (eg. to
> cover left-over arguments). I addition it'd be nice to have a little
> description/summary of what the program to do. I realize you can do this
> with a custom help function. But it'd be nice if these were standard
> elements.
>
> Other niceties might be:
> - a way to indicate parameters enabled by default (eg. having a '*' next to
> them in the help)
There is already
#define POPT_ARGFLAG_SHOW_DEFAULT 0x0080U /*!< show default value
in --help */
as well as
#define POPT_ARGFLAG_OPTIONAL 0x1000U /*!< arg may be missing */
wired into --help output.
>
> - An additional structure that could provide detailed help on argDescription
> elements. For example, rpm has an option:
> --queryformat=QUERYFORMATuse the following query format
> It'd be nice if there were a section of help that could describe what
> QUERYFORMAT could be. So obviously it's not going to be a full man page, but
> perhaps it could just show supported % format options or something like that.
This is a different issue. There is already the 6th/7th fields in POPT tables,
the problem is really "information overload" from --help. At some point man(1)
or info(1) is a better approach than --help, particularly for RPM which has
an entire eco-system of option processing and far too many options
to be reasonably displayed with --help (because the info is not very helpful).
(aside)
Note that RPM is _ROUTINELY adding
#define POPT_ARGFLAG_DOC_HIDDEN 0x4000U /*!< don't show in
help/usage */
there's zillions of options in RPM that noone knows about. My guess is that
there's
more "hidden" than displayed options these days, usually disablers that noone
ever
really needs to use.
> I use something like this in my code, but I have specific keys like
> "[replaceme]" that I convert. And I put just the acceptable keys in the help
> cause I just need a quick reminder of what they are. But it clutters the
> option help a little. I'd be fine with specifying "FORMATSTRING" in the
> option help. Then have perhaps an arg help down below that shows what values
> FORMATSTRING understands.
>
>
> I'm not sure exactly how you could support these without breaking
> compatability with existing apps. Perhaps a new datatype something like:
>
FYI: POPT 2.0 is all about breaking compatibility. There's only so much
that can be done with obscure overloadings of the existing 7-tuple in
popt table entries.
> enum poptOptionType {
> POPT_OPTION,
> POPT_ARG,
> POPT_USAGE,
> };
>
> union poptDetailedOption {
> poptOptionType optType;
> struct poptShortOption;
> struct poptArgHelp;
> struct poptUsageHelp;
> }
>
> struct poptShortOption { /* same as poptOption but with a type field */
>poptOptionType optType;
>const char * longName;
>char shortName;
>int argInfo;
>
Re: Disallowing args
On Jun 5, 2010, at 12:48 PM, Wayne Davison wrote: > On Sat, Jun 5, 2010 at 8:56 AM, Jeff Johnson wrote: > but >--foo bar > returns bar in the argument list? > > Yes. The user may well have wanted it to be in the arg list. There's no way > for the program to know that the user didn't just toss some options in the > middle of some args (which I do all the time these days, like starting my > rsync source/dest args, and then tossing in a --remove-source-files, > --backup, or what-not at wherever I am in the list), so I wouldn't want to > see an error for something like this: > > rsync -aiv --del file --remove-source-args some/dir host:/dest/dir > > ... just because --del doesn't take an arg. > > The other way to "fix" the error is to morph "--foo=bar" behavior to be > identical to "--foo bar", i.e. an extra argument failure. > > I don't see how that would work for something like rsync that takes any > number of command-line args outside the options. > Well determistic behavior "works", just not very useful ;-) I'm not arguing, just there's issues like POSIX_ME_HARDER that get factored into "--foo=bar" error handling, all mind numbingly tedious. Your patch is likely what I will add, I'm a very lazy schmuck. > Anything you want to see in POPT 2.0? I'm collecting features ... > > A couple ideas off the top of my head: > An incrementing option -- repeated use adds 1 to the variable instead of > setting it to the same value. How about a full blown RPN calculator to handle not just increment/decrement, but all arithmetic operations, on option values. infix <-> postfix just isn't that hard, and the whole mess is just a teensy stack and a switch. I've got several RPN calculators floating around in RPM code these days that could be dropped into POPT without any effort whatsoever. Nor is a RPM calculator very hard to implement. > Multiple long names separated by "|" in the long-name string (though that > could really just be defined as an alias, it might be nice to auto-gen the > alias). There's already Bloom filters fin POPT 1.16 for opaquely handling multiple option value strings (the "bar" in "--foo bar"). That partly addresses the need for fewer popt table entries/aliases/execs. (aside) The usage case is for RPM which has >100 hash algorithm names buried into popt tables, and I need to collapse down to a single popt table entry for my own maintenance sanity. Full pattern matching, either *RE or fnmatch(3), would not be hard. The pdksh extensions to fnmatch globs would handle your specific alternation request. And POPT is already linking fnmatch(3). If you have any other RFE's just drop a note here ... and blame Rusty Russell for encouraging POPT 2.0 ;-) 73 de Jeff
Re: Disallowing args
On Jun 5, 2010, at 11:39 AM, Wayne Davison wrote: > Here's something that was recently fixed for the popt that is included > with rsync: rejecting an arg to an option that doesn't take an arg. > > Attached is a patch. A new error code, POPT_ERROR_UNWANTEDARG, was > created to make the error message nice. This handles both -l=value > and --long-arg=value where neither one is supposed to take a value. > Hmm, so --foo=bar returns POPT_ERROR_UNWANTEDARG but --foo bar returns bar in the argument list? The other way to "fix" the error is to morph "--foo=bar" behavior to be identical to "--foo bar", i.e. an extra argument failure. Lemme muddle a bit about whether the two forms should be treated differently or identically, that's purely a design issue. Both cases are an error no matter what. There's nothing wrong with your patch whatsoever, just that it introduces instant legacy incompatibility. *shrug* Thanks for the patch. (aside) Anything you want to see in POPT 2.0? I'm collecting features ... 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt-1.16: patch for building on NetBSD-current
On Jun 5, 2010, at 9:29 AM, Thomas Klausner wrote: > Hi! > > Since NetBSD-5.99.11, glob_pattern_p is included in libc. > > The attached patch fixes the resulting compilation problem. Please > include it in the next release. > I'd rather see a HAVE_GLOB_PATTERN_P test in AutoFu, with all the bleary platform tests isolated in "system.h", than having a Sysiphean litter of +#if defined(__NetBSD__) +#include +#endif platform tests everywhere in the code. No worries, I'll adjust your patch to my fetishism ... Thanks for the report. Holler at me if the issue slips, I won't be working on POPT 2.0 quite yet, a few more weeks to clear some space. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
POPT 2.0: any feature requests?
After some thought (and some prodding), I think its time for a major release of 2.0 with the (probability, dunno yet) of changing both the API and ABI of POPT. So I'm looking for feature requests -- good/bad/ugly/whatever -- in order to get a ROADMAP and a timeline together for POPT 2.0. In general, I'd personally like to see POPT remain "minimalistic" and as simple as possible to use anywhere and everywhere. But I can/will try to fit in any reasonable feature requests in POPT 2.0. Any RFE's? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Test failure for popt 1.16
Actually the previous patch is bogus. Right idea, just
the wrong way to fix.
Try this patch instead:
Index: popthelp.c
===
RCS file: /v/rpm/cvs/popt/popthelp.c,v
retrieving revision 1.94
diff -p -u -w -r1.94 popthelp.c
--- popthelp.c 28 Aug 2009 00:06:33 - 1.94
+++ popthelp.c 11 May 2010 19:12:28 -
@@ -652,6 +652,8 @@ static size_t showHelpIntro(poptContext
const char * fn = (os->argv ? os->argv[0] : NULL);
if (fn == NULL) return len;
if (strchr(fn, '/')) fn = strrchr(fn, '/') + 1;
+ if (fn[0] == 'l' && fn[1] == 't' && fn[2] == '-')
+ fn += sizeof("lt-") - 1;
/* XXX POPT_fprintf not needed for argv[0] display. */
fprintf(fp, " %s", fn);
len += strlen(fn) + 1;
Index: testit.sh
===
RCS file: /v/rpm/cvs/popt/testit.sh,v
retrieving revision 1.31
diff -p -u -w -r1.31 testit.sh
--- testit.sh 26 Jul 2009 16:25:23 - 1.31
+++ testit.sh 11 May 2010 19:12:29 -
@@ -115,7 +115,7 @@ run test1 "test1 - 56" "arg1: 0 arg2: (n
run test1 "test1 - 57" "arg1: 0 arg2: (none) aBits: foo,baz" --bits
foo,bar,baz,!bar
run test1 "test1 - 58" "\
-Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG]
+Usage: test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG]
[-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val]
[-i|--int=INT] [-s|--short=SHORT] [-l|--long=LONG]
[-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE]
@@ -124,7 +124,7 @@ Usage: lt-test1 [-I?] [-c|--cb2=STRING]
[--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|--inc]
[-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [--simple=ARG]"
--usage
run test1 "test1 - 59" "\
-Usage: lt-test1 [OPTION...]
+Usage: test1 [OPTION...]
--arg1 First argument with a really long
description. After all, we have to test
argument help wrapping somehow, right?
And if you can confirm the patch "works" in the next 24 hours,
I'll re-roll the (unannounced) popt-1.16.tar.gz.
Otherwise, I'll figger better for popt-1.17.
hth
73 de Jeff
On May 11, 2010, at 2:54 PM, Jeff Johnson wrote:
>
> On May 10, 2010, at 7:57 PM, Pieter Bowman wrote:
>
>> I did builds of popt 1.16 on a number of our systems here. Test 59
>> failed on a number of the systems with the following output:
>>
>
> The "fix" for the failure in popt-1.16 "make check" is likely (I have
> easy no easy means of testing across all the platforms that you have)
> as simple as the atached patch.
>
> If you can confirm that the patch "works", I'll re-roll the popt-1.16
> tarballs, and re-release (it hasn't been announced anywhere)
> the popt-1.16 tarball.
>
> hth
>
> 73 de Jeff
> ==
> Index: test1.c
> ===
> RCS file: /v/rpm/cvs/popt/test1.c,v
> retrieving revision 1.44
> diff -p -u -w -r1.44 test1.c
> --- test1.c 26 Jul 2009 16:38:25 - 1.44
> +++ test1.c 11 May 2010 18:50:39 -
> @@ -252,7 +252,7 @@ int main(int argc, const char ** argv)
> resetVars();
> /*...@=modobserver@*/
> /*...@-temptrans@*/
> -optCon = poptGetContext("test1", argc, argv, options, 0);
> +optCon = poptGetContext("test1", argc, argv, options,
> POPT_CONTEXT_KEEP_FIRST);
> /*...@=temptrans@*/
> (void) poptReadConfigFile(optCon, "./test-poptrc");
> (void) poptReadDefaultConfig(optCon, 1);
>
>
> __
> POPT Library http://rpm5.org
> Developer Communication List popt-devel@rpm5.org
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: Test failure for popt 1.16
On May 10, 2010, at 7:57 PM, Pieter Bowman wrote:
> I did builds of popt 1.16 on a number of our systems here. Test 59
> failed on a number of the systems with the following output:
>
The "fix" for the failure in popt-1.16 "make check" is likely (I have
easy no easy means of testing across all the platforms that you have)
as simple as the atached patch.
If you can confirm that the patch "works", I'll re-roll the popt-1.16
tarballs, and re-release (it hasn't been announced anywhere)
the popt-1.16 tarball.
hth
73 de Jeff
==
Index: test1.c
===
RCS file: /v/rpm/cvs/popt/test1.c,v
retrieving revision 1.44
diff -p -u -w -r1.44 test1.c
--- test1.c 26 Jul 2009 16:38:25 - 1.44
+++ test1.c 11 May 2010 18:50:39 -
@@ -252,7 +252,7 @@ int main(int argc, const char ** argv)
resetVars();
/*...@=modobserver@*/
/*...@-temptrans@*/
-optCon = poptGetContext("test1", argc, argv, options, 0);
+optCon = poptGetContext("test1", argc, argv, options,
POPT_CONTEXT_KEEP_FIRST);
/*...@=temptrans@*/
(void) poptReadConfigFile(optCon, "./test-poptrc");
(void) poptReadDefaultConfig(optCon, 1);
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: Test failure for popt 1.16
This is largely a cosmetic issue introduced by using libtool (i.e. "test1" != "lt-test1"in argv[0]). But yes, could/should be fixed. 73 de Jeff On May 10, 2010, at 7:57 PM, Pieter Bowman wrote: > I did builds of popt 1.16 on a number of our systems here. Test 59 > failed on a number of the systems with the following output: > > Test "test1 --usage" failed with: "Usage: test1 [-I?] [-c|--cb2=STRING] > [--arg1] [-2|--arg2=ARG] >[-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] >[-i|--int=INT] [-s|--short=SHORT] [-l|--long=LONG] >[-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] >[--randint=INT] [--randshort=SHORT] [--randlong=LONG] >[--randlonglong=LONGLONG] [--argv=STRING] [--bitset] [--bitclr] >[--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|--inc] >[-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [--simple=ARG]" != > "Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG] >[-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] >[-i|--int=INT] [-s|--short=SHORT] [-l|--long=LONG] >[-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] >[--randint=INT] [--randshort=SHORT] [--randlong=LONG] >[--randlonglong=LONGLONG] [--argv=STRING] [--bitset] [--bitclr] >[--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|--inc] >[-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [--simple=ARG]" > > The operating systems which failed were: > > gentoo/Linux, Alpha, PowerPC, PowerPC64, SPARC > MacOS X 10.5, PowerPC, Intel > FreeBSD 8, x86 > MirOS (aka. MirBSD), x86 > NetBSD 5.0.2, x86 > OpenBSD 4.7, x86 > OpenSolaris snv_134 X86 > OpenSUSE 11.2, x86_64 > Solaris 10 x86, x86_64, SPARC > > The operating systems which succeeded were: > > RedHat Enterprise Linux 5.5, x86, x86_64, IA64 > Ubuntu 9.10, x86 > Fedora 12, x86 > > Thanks for looking into this. > > Pieter > __ > POPT Library http://rpm5.org > Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: test1-55 failes on Solaris
On Sep 25, 2009, at 3:34 AM, Dagobert Michelsen wrote: Hi, I am currently trying to build libpopt 1.15 on Solaris 8 with Sun Studio 11. After successfull compilation I have a failing test: I doubt that --help formatting is a serious issue in popt. Its likely a cosmetic issue. Comparing large multiline strings in shell with escaped characters is annoying. It looks like the test is failing because "test1" != "lt-test1" in argv[0]. argv[0] is/was changed by libtool. 73 de Jeff Running test test1 - 55. Test "test1 --usage" failed with: "Usage: test1 [-I?] [-c|-- cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG]" != "Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG]" FAIL: testit.sh Further inspection shows + run test1 test1 - 55 Usage: lt-test1 [-I?] [-c|--cb2=STRING] [-- arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG] --usage prog=test1 + shift name=test1 - 55 + shift answer=Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|-- arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG] + shift + echo Running test test1 - 55. Running test test1 - 55. + /home/dam/mgar/pkg/libpopt/trunk/work/build-isa-sparcv8/popt-1.15/ test1 --usage HOME=/home/dam/mgar/pkg/libpopt/trunk/work/build-isa-sparcv8/ popt-1.15 result=Usage: test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG] + [ Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG] != Usage: test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|-- arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG] ] + echo Test "test1 --usage" failed with: "Usage: test1 [-I?] [-c|-- cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG] [-f|--float=FLOAT] [-d|--double=DOUBLE] [--randint=INT] [--randlong=LONG] [--randlonglong=LONGLONG] [--argv] [-- bitset] [--bitclr] [--bitxor] [--nstr=STRING] [--lstr=STRING] [-I|-- inc] [-c|--cb=STRING] [--longopt] [-?|--help] [--usage] [-- simple=ARG]" != "Usage: lt-test1 [-I?] [-c|--cb2=STRING] [--arg1] [-2|--arg2=ARG] [-3|--arg3=ANARG] [-onedash] [--optional=STRING] [--val] [-i|--int=INT] [-l|--long=LONG] [-L|--longlong=LONGLONG]
Breakage with --help
I just noticed this flaw while teaching FreeBSD cp(1) to use popt:
LANG=C ./cp --help
Usage: lt-cp [OPTION...]
-H -L -
P -R -a, --archive
-f, --force -i, --interactive -l, --
link -n, --noclobber -p, --preserve -
r -v, --verbose
Help options:
-?, --helpShow this help message
--usage Display brief usage message
Usage: cp [-R [-H | -L | -P]] [-f | -i | -n] [-alpv] source_file
target_file
cp [-R [-H | -L | -P]] [-f | -i | -n] [-alpv] source_file ...
target_directory
The popt table looks like this (i.e. "" for description):
...
{ NULL,'H', POPT_ARG_NONE,NULL, (int)'H',
N_(""), NULL },
...
Somebody make sure I fix this flaw please.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: [CVS] RPM: popt/ popt.pc.in
On Aug 12, 2009, at 10:47 AM, devzero2000 wrote: Sorry if i disgress, but i want take advantage of your attention. It might be useful to migrate the autofu popt to version 2 of autoconf / automake? It is something I can certainly do, if you agree. I prefer to ask first. Feel free to do whatever. All I expect is that this sequence of operations continues to "work": ./autogen.sh# <-- this already adds config.rpath multiple times, # and introduces a dependency on perl for no important reason make clean make check Note that my definition of "work" is rather *ahem* loose. But yes popt needs to build flawlessly everywhere. My guess is that upgrading to Newer! Better! Bestest! AutoFu in popt is going to run afoul of someone's build system choices with Older! Reliable! Unchanging! AutoFu down the road. 73 de Jeff
Finalizing an API for POPT_ARGS_BITSET
Attached is a toy program to illustrate the new POPT_ARG_BITSET API in popt. The toy program uses /usr/share/dict/words as a attribute universe, and checks arguments to see if they are in the attribute dictionary. Here's some usage examples $ wc -l /usr/share/dict/words 479827 /usr/share/dict/words $ ./tdict XyZZy = No words are in /usr/share/dict/words : NO XyZZy: NO $ ./tdict foo bar baz XyZZy = Some words are in /usr/share/dict/words foo:YES bar:YES baz:YES : NO XyZZy: NO = poptBits N:959654 M:1439481 K:2 (176Kb) total(5) = hits(3) + misses(2) $ The API doesn't look too bad imho. Other opinions? Choosing a reasonably sane default for the Bloom filter's N/M/K parameters in popt will still need some adjustment. 73 de Jeff tdict.c Description: Binary data
POPT_ARG_BITSET to handle arbitrary attribute-like option strings
I've just checked in a proof-of-concept implementation to use Bloom filters to handle arbitrary option string values with popt. The problem that I am trying to solve in popt (and for RPM and for option processing in general) is to devise a means to handle arbitrary (as in unknown when code is compiled) string options and values and argument lists. Bloom filters, because they partition the space, not the data, are the best means to that end that I'm aware of. (aside) All that the above means is that a Bloom filter bit set with all 1's: 111 will match *everything* that is fed to it. Consider the alternative, an explicit table of "known" values, which will fail if/when a Newer! Better! Bestest! string is added but someone fergits to add it to the "known" value table. The cost (as in no. of bytes required) is quite modest: Assuming a population of an option attribute set of ~1024 arbitrary strings (>2 orders of magnitude more than the number of fingers that I have ;-), uisng 16 linearly combined hashes (to reduce the probability of false positives to < 0.01%), popt will lazily alloc a bit set of approx. (3 * 1024) / 2 which is 1536 bits (or 192 bytes). *shrug* (aside) All of the above parameters are tunable, the only hard question is What should the default pramaters in popt be? I won't know whether I like the popt implementation (Bloom filters are not at all bloaty) until I try using it at an application level. todo++ Other opinions welcomed, particularly on the API. There's still some polishing that remains. Enjoy! 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Fwd: Adding option bit sets using Bloom filters?
Hmmm, this message never made it to popt-devel.
I'm attempting a Bloom filter implementation in popt now, will likely
also
attempt choosing 1-of-N exclusively as described here:
http://rpm5.org/community/popt-devel/0112.html
and perhaps finish up "option groups" for feature parity with
getopt_long()
(but perhaps not, option groups would be very annoying to do correctly).
So last chance to voice an opinion ...
73 de Jeff
Begin forwarded message:
From: Jeff Johnson
Date: July 10, 2009 11:24:46 AM EDT
To: popt-devel@rpm5.org
Subject: Adding option bit sets using Bloom filters?
One of the problems that I frequently encounter in RPM using
POPT is the tedium of keeping track of bits and names and
options and functionality, i.e. all the bleeping baggage.
The usual problem that I have in RPM is that I end up running out of
bits.
One solution would be to use a Bloom filter for a open-ended class
of options
that map to bits. One Bloom filter would be needed to establish
whether an option is in the permitted set, another Bloom filter would
be needed to return values that were explicitly mentioned.
The nice thing about Bloom filters is that they partition all
possible option strings without the tedium of committing to
an explicit mapping onto bits.
I'd likely do the implementation slightly differently than
most option processing, marking a subtable (which carries the strings
necessary to initialize the permitted option Bloom filter) in
this field in struct poptOption:
/*...@observer@*/ /*...@null@*/
const char * longName;
and returning which (possibly multiple) options were seen (as always
in popt) in
/*...@shared@*/ /*...@null@*/
void * arg; /*!< depends on argInfo */
with the {m,k} parameters used for the Bloom filter overloaded into
char shortName; /*!< may be NUL */
int val;/*!< 0 means don't return, just update
flag */
For more on Bloom filters, see
http://en.wikipedia.org/wiki/Bloom_filters
I already have an implementation in RPM that could be moved
to POPT, the implementation is quite straight-forward.
Opinions? I'll likely proceed with an implementation the
next time I get to popt hacking.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: [PATCH] added pkgconfig support for popt
On Jul 21, 2009, at 11:21 AM, devzero2000 wrote:
Very thiny patch for integrating pkg-config(1) support for popt
library into popt itself.
Sorry for the delay.
I applied the pkgconfig patch with these changes:
1) I'd rather see "popt.pc", not "libpo...@version@.pc". That's
consistent with "neon.pc", another library used widely where
version makes a difference. It also seems kinda silly to add
pkgconfig functionality to discover popt's version with the version
in the *.pc file name itself.
2) popt is strange in that it typically installs (on linux) into /lib
or /lib64
rather than into $(libdir). Which means I had to use this construct
in Makefile.am (which likely breaks with multilib):
pkgconfigdir = $(prefix)/lib/pkgconfig
pkgconfig_DATA = popt.pc
and this construct in popt.spec (which likely works with multlib)
%{_prefix}/%{_lib}/pkgconfig/libpopt-%{version}.pc
lest popt become the very first and only package to start populating
/lib/pkconfig/*, or the first/only per-arch package that installs
into /usr/share/pkgconfig. I _REALLY_ wish not to break new
ground.
Transitive install option discovery is of course mostly insane if/when
pkgconfig
has a different prefix than $(libdir): if you know where to put
"popt.pc" when
building popt (alternatively how to configure pkgconfig differently
than default
--libdir), then you also know both of these datum-di-dums:
Libs: -L${libdir} -lpopt
Cflags: -I${includedir}
But hey, whatever needs to be done to avoid the endless bikeshed
discussions
about --prefix and The One True Way To Build Software.
w00t! popt uses pkgconfig!
73 de Jeff
popt-pkgconfig.patch
Description: Binary data
Re: [PATCH] added pkgconfig support for popt
On Jul 24, 2009, at 11:03 AM, Ralf S. Engelschall wrote: Only true if you install POPT into standard system locations. But if I install POPT with --includedir=/usr/include/popt --libdir=/usr/lib/ popt the pkg-config file still allows me to build and link against POPT without having to know its particular install locations (assuming that at least the popt.pc is in a standard location or at least in the PKG_CONFIG_PATH -- or whatever the name of this variable is). So, yes and no, it can be considered a little bit overkill for POPT, but in general it is already useful just because it encapsulates/remembers the install locations. Point taken. Good, there's a usage case at least. There's so much fussing with popt for little known purpose. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: [PATCH] added pkgconfig support for popt
On Jul 23, 2009, at 3:36 PM, Ralf S. Engelschall wrote: Well, the main point of pkg-config is not querying the version of a library, but to _transitively_ query the build-time flags required to build against a library. For this reason even our RPM_CHECK_LIB is able to use pkg-config files ;-) That's the point, transitivity doesn't apply. popt has no flags, links nothing else, all that is needed is -lpopt. But I will apply the patch, easier than describing why pkg-config is useless (and will remain useless forever) for popt. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: [PATCH] added pkgconfig support for popt
On Jul 22, 2009, at 3:05 PM, Ralf S. Engelschall wrote: On Tue, Jul 21, 2009, devzero2000 wrote: Very thiny patch for integrating pkg-config(1) support for popt library into popt itself. Exept for the hard-coded version "1.15" in configure.ac this looks good and should be taken over. I mostly agree, but pkg-config is surely a complicated way to determinbe a version for a library that never changes. But that opinion has nothing to do with the patch. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Releasing popt-1.15, last call for features
Its likely as good a time as any to release popt-1.15, as I'm never sure when I will have time to work on popt. My reason for releasing now is so that I can start using two new features in popt, and start phasing out legacy compatible retrofit code: The first "feature" is POPT_ARGFLAG_TOGGLE, which permits both --foo and --nofoo (and --no-foo for the Debian enthralled) options to be used to set or clear bit fields. The second "feature" is a bit more subtle, doing simple sanity checks, like is it a file, and is it owned by "root", on a configuration file if the path is proceeded with a '@' attention marker. I will append the changelog for popt-1.15 below. Any other RFE's? 73 de Jeff === 1.14 -> 1.15: - jbj: poptReadFile: permit NULL if return values are not desired. - jbj: poptReadFile: add routine. - jbj: trim out escaped newline(s) from file content, other fixes. - jbj: permit popt alias/exec to include content from a file. - jbj: permit glob(3) patterns in appName field of popt alias/ exec config. - jbj: add test cases for bit operations and toggles. - jbj: avoid displaying --[no]nofoo with POPT_ARGFLAG_TOGGLE. - jbj: add poptArgInfo() to get argInfo, implementing POPT_ARGFLAG_TOGGLE. - jbj: add longOptionStrcmp() to match w POPT_ARGFLAG_TOGGLE. - jbj: change singleDash arg to a bit enum, use LF_ISSET(ONEDASH) instead. - jbj: rework the glob wrappers into something more useful. portability todo++. - jbj: stub in glob(3) wrappers for popt. more useful poptGlob() API next. - jbj: add poptInit/poptFini/poptReadConfigFiles/poptSaneFile routines. - jbj: rewrite poptReadConfigFile(), styling for (i.e. my) readbility. - jbj: reserve a bit for --[no]opt prefix toggling. - jbj: fix: check/print argv[0] in --help for NULL. - jbj: permit type/group bitmasks to be changed (if needed somewhen). - jbj: snip out 8 unused bits for argument groups. - jbj: fix: eliminate dead code (CID#5). - jbj: fix: rearrange code to better hint to coverity scan (CID#9). - jbj: fix: rewrite (and simplify) strdup_locale_from_utf8() (CID#7, CID#8, CID#11, CID#12). - jbj: test/use HAVE_SRANDOM to avoid portability issues. - jbj: fix: remove AC_CHECK_VA_COPY check, va_copy is no longer used. - jbj: add eo.po and id.po (Translation Project). - jbj: updated da.po (Translation Project). - jbj: extend coverage to several additional setup routines. - jbj: add tests for --usage/--help coverage. - jbj: add lconv/gcov targets to Makefile.am. - jbj: refactor automagic (*opt->arg) option arg store to poptSaveArg(). - ldv: update INPUT tag in Doxyfile.in, fix doxygen warnings in popthelp.c. - start popt-1.15 development. __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Extending POPT_SET_BIT from first-found to all-found?
There's an odd behavior with popt that bites me every other year or so in RPM that I should describe. Note that RPM uses popt in ways no other program does (and likely in ways that no other program should use popt). E.g., RPM has 3 separate contextual meanings for -i: 1) -i as in --install 2) -qi as in --query --info 3) -bi used while building Yes, its crazy to have 3 separate meanings for -i but it was way too late in 1998, a decade later the usage cases are so locked in concrete that the *MUST* be dealt with. The flaw that bites me every other year or so is the difference in the way per-table callbacks are handled as opposed to other option/alias/exec handling in popt. Normally, popt traverses nested tables, finds an option match, performs some operation, and returns. That is "first-found" behavior. However, a popt enhancement was introduced (to accomodate GNOME) to have popt continue through all tables performing callbacks for every occurence of an option found. That is "all-found" behavior. The problem is that there's no way to handle "all-found" behavior when there are multiple occurences of identically named options __EXCEPT__ through a call back. Non-callback options are always "first-found" behavior. The "all-found" popt behavior has actually been useful in RPM because certain options have been generalized by having multiple callbacks, with different side-effects, through multiple per-table callbacks. But popt also supplies POPT_BIT_SET and POPT_BIT_CLEAR, which are also quite convenient automation __EXCEPT__ when there are multiple tables with identically named options, and the debugging is quite obscure. I dunno if I can (or should) attempt to process non-callback options with an "all-found" mode of behavior like callback options have in popt. Hmmm ... Meanwhile reverting some POPT_BIT_SET options back to callbacks in RPM has restored --nofdigests functionality which is "gud enuf" until I forget this flaw Yet Again and attempt Yet Again to use POPT_BIT_SET instead. Thanks for listening. I'm pretty sure that the need for "all-found" non-callback option processing is specific only to RPM, but I thought I'd describe the flaw. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Adding poptSecuritySaneFile to popt-1.15?
On Dec 19, 2008, at 1:11 PM, Jeff Johnson wrote: Opinions? Otherwise the patch is mostly *yawn* ... I've checked in a mostly complete "attention" API into popt. There's a couple nit-picks left to finish up: 1) '@' ends up colliding with extended '@(/A|/B)' glob patterns. No biggie, the flags are not enabling '@(...)' patterns anyways. 2) glob_pattern_p will need to be retrofitted because of _GNU_SOURCE, sigh. The whole HAVE_GLOB_H feature test framework is a bit feeble too. 3) rpm's "best effort" sloppy config and popt's POPT_ERROR_ERRNO instant return are at cross purposes. 4) the "/etc/popt.d/*" pattern initialization should use opendir(3) et al instead. I'll buff up the chrome plating as I proceed to testing & using poptGlob() et al. *yawn* 73 de Jeff smime.p7s Description: S/MIME cryptographic signature
Re: Cloning getopt (and getopt_long) into popt?
On Dec 20, 2008, at 3:06 AM, Ralf S. Engelschall wrote: Ah, yes, as long as one can still build a POPT without these two symbols (to avoid conflicts) this would be a really nice addition for POPT. I would say: go for it as long as is an _optional_ feature! I should likely state the underlying reason as well, adding persistent configurable default option settings. Both popt/getopt_long have a means to configure options persistently. However, popt alias/exec attach to executables, while getopt_long can save option values (and presence) in a state file to be parsed. wget (and iirc gnupg) have functionality captured in ~/.foo file parse that is not easily/cleanly replicated with alias/exec. todo+= (with disabler) but it will likely be a while before I get back to studying rpmwget.c needs. 73 de Jeff smime.p7s Description: S/MIME cryptographic signature
Cloning getopt (and getopt_long) into popt?
popt competes for mind share with getopt, and particularly with GNU getopt_long, the Debian added borkedness (jmnsho). Should I add a getopt(3)/getopt_long(3) wrapper onto popt to lower the barrier to converting from getopt_long(3)? I personally don't care a bit because I use popt instead of getopt_long(3) all the time and everywhere. But the wrapper (with AutoFu disablers) is trivially arranged if there is interest. (Note: one thing that popt __DOES__ need is the ability to per-executable option persistent preference settings. But that can be done in many ways, not just the way that getopt_long(3) implements). 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Adding poptSecuritySaneFile to popt-1.15?
On Dec 19, 2008, at 11:13 AM, Jeff Johnson wrote: Any other opinions? As long as the particular security check (here rpmSecuritySaneFile for RPM_VENDOR_OPENPKG) embedded into POPT can be optionally still overridden from within RPM (in case one needs some additional checks or a different error message or whatever) I'm happy. Perhaps an optional callback does the trick. Attached is a preliminary (I'm still pondering aesthetics/ maintainenance) reflecting API additions that I might find useful, mostly because I'm doing the same silly boilerplate code everywhere I use popt these days and my mouse button finger is getting arthritic. Opinions? Otherwise the patch is mostly *yawn* ... No hurry, I have some trickier implementations to accomplish, including using the newly added poptGroup() marker to sort argv[] arguments to remove order sensitivity for the RPM CLI. And there's also --opt/--noopt automation toggling, per the RFE a couple months back. And likely a few other RFE's that I will dig out of the popt-devel archives as I proceed. If you do have additional RFE's for popt, now would be a wonderful time to send them along. 73 de Jeff popt.patch Description: Binary data smime.p7s Description: S/MIME cryptographic signature
Re: Adding poptSecuritySaneFile to popt-1.15?
On Dec 19, 2008, at 11:11 AM, Ralf S. Engelschall wrote:
On Fri, Dec 19, 2008, Ralf S. Engelschall wrote:
On Fri, Dec 19, 2008, Jeff Johnson wrote:
(resent, dunno where the 1st message went)
I don't know, never seen on the list...
I kind of like the idea of using a '@' before a file path as an
"attention" marker to increase the file validation checks, and so
I'm
likely to refactor the functionality out of rpm and into popt-1.15
as
part of simplifying rpm configuration/initialization.
At the same time, I will probably add a new poptReadConfigFiles()
method whose argument will be a colon separated list of
configuration
file paths to read.
Any other opinions?
As long as the particular security check (here rpmSecuritySaneFile
for RPM_VENDOR_OPENPKG) embedded into POPT can be optionally still
overridden from within RPM (in case one needs some additional
checks or
a different error message or whatever) I'm happy. Perhaps an optional
callback does the trick.
BTW, under RPM_VENDOR_OPENPKG the "@" attention marker is not just
used
for POPT files. It is also used for RPM macro files and Lua script
files! So, please be careful that instead of a "factoring out" not a
"partial removal" happens to the functionality.
Understood. What I like is the idea of a '@' attention in-band, not
which particular
config file reading has chosen '@' attention.
I will likely drill similar into lua bootstrapping (if not there
already).
I like consistency no matter what. But there are other issues to deal
with there permitting external system lua module loading that have
to be addressed. I have no intent of changing anything, but as you know,
the embedded lua is all snarled up throughout rpm misc/rpmio/lib ...
For completeness reasons, here is how OpenPKG 4 configures RPM 5:
| [...]
| # determine POPT option, RPM macros and Lua script file paths
| RPMPOPT="%{l_prefix}/lib/openpkg/rpmpopt"
| RPMPOPT="$RPMPOPT:%{l_prefix}/etc/openpkg/rpmpopt"
| RPMPOPT="$RPMPOPT:%{l_prefix}/etc/openpkg/rpmpopt.d/*"
| RPMPOPT="$RPMPOPT:~/.openpkg/rpmpopt"
| RPMPOPT="$RPMPOPT:@../../.openpkg/rpmpopt"
| RPMPOPT="$RPMPOPT:@../.openpkg/rpmpopt"
| RPMPOPT="$RPMPOPT:@./.openpkg/rpmpopt"
| RPMMACROS="%{l_prefix}/lib/openpkg/rpmmacros"
| RPMMACROS="$RPMMACROS:%{l_prefix}/etc/openpkg/rpmmacros"
| RPMMACROS="$RPMMACROS:%{l_prefix}/etc/openpkg/rpmmacros.d/*"
| RPMMACROS="$RPMMACROS:~/.openpkg/rpmmacros"
| RPMMACROS="$RPMMACROS:@../../.openpkg/rpmmacros"
| RPMMACROS="$RPMMACROS:@../.openpkg/rpmmacros"
| RPMMACROS="$RPMMACROS:@./.openpkg/rpmmacros"
| RPMLUA="%{l_prefix}/lib/openpkg/rpmlua"
| RPMLUA="$RPMLUA:%{l_prefix}/etc/openpkg/rpmlua"
| RPMLUA="$RPMLUA:%{l_prefix}/etc/openpkg/rpmlua.d/*"
| RPMLUA="$RPMLUA:~/.openpkg/rpmlua"
| RPMLUA="$RPMLUA:@../../.openpkg/rpmlua"
| RPMLUA="$RPMLUA:@../.openpkg/rpmlua"
| RPMLUA="$RPMLUA:@./.openpkg/rpmlua"
|
| # configure program
| ./configure \
| --cache-file=./config.cache \
| --prefix=%{l_prefix} \
| --mandir="%{l_prefix}/man" \
| --includedir="%{l_prefix}/include/openpkg" \
| --with-name="OpenPKG RPM" \
| --with-path-cfg="%{l_prefix}/etc/openpkg" \
| --with-path-rpmpopt="$RPMPOPT" \
| --with-path-macros="$RPMMACROS" \
| --with-path-rpmlua="$RPMLUA" \
| [...]
As you can see, the attention markers are especially used on all paths
relative to the current working directory. This allows OpenPKG to
use a
very flexible file-system layout with RPM (based on some additional
RPM
macro hacking) while at the same time not open a security hole.
Thanks for the clear and explcit example, that will help me find what
I'm
not supposed to break ;-)
73 de Jeff
smime.p7s
Description: S/MIME cryptographic signature
Re: Adding poptSecuritySaneFile to popt-1.15?
On Dec 19, 2008, at 11:04 AM, Ralf S. Engelschall wrote: On Fri, Dec 19, 2008, Jeff Johnson wrote: (resent, dunno where the 1st message went) I don't know, never seen on the list... I kind of like the idea of using a '@' before a file path as an "attention" marker to increase the file validation checks, and so I'm likely to refactor the functionality out of rpm and into popt-1.15 as part of simplifying rpm configuration/initialization. At the same time, I will probably add a new poptReadConfigFiles() method whose argument will be a colon separated list of configuration file paths to read. Any other opinions? As long as the particular security check (here rpmSecuritySaneFile for RPM_VENDOR_OPENPKG) embedded into POPT can be optionally still overridden from within RPM (in case one needs some additional checks or a different error message or whatever) I'm happy. Perhaps an optional callback does the trick. I really like the in-band '@' "attention grabber". I 'spose I can expose a global callback vector if you wsh to override poptSecuritySaneFile() for some purpose down the road ... Basically I'm just gonna merge the code from rpm -> popt, removing RPM_VENDOR_OOPENPKG, no other change. The ulterior motive is unrelated to popt, but rather refactoring RPM's use of popt. I haven't a clue anymore of what gets read from where in RPM, and so am hoping to bury some of the functionality that I __DO__ understand while composting the rest of RPM initialization. I'd really like to not have to engineer Yet Another callback into popt, what's there is bad enough to maintain. But I can try for a table driven callback if you prefer to a global vector override, I'm not particularly fond of globals either. Thanks for comments. 73 de Jeff smime.p7s Description: S/MIME cryptographic signature
Fwd: Adding poptSecuritySaneFile to popt-1.15?
(resent, dunno where the 1st message went)
I kind of like the idea of using a '@' before
a file path as an "attention" marker to increase
the file validation checks, and so I'm likely to
refactor the functionality out of rpm and into
popt-1.15 as part of simplifying rpm configuration/initialization.
At the same time, I will probably add a new
poptReadConfigFiles() method whose argument
will be a colon separated list of configuration
file paths to read.
Any other opinions?
73 de Jeff
Begin forwarded message:
From: Jeff Johnson
Date: December 18, 2008 3:15:33 PM EST
To: popt-devel@rpm5.org
Subject: Adding poptSecuritySaneFile to popt-1.15?
I'm currently catching up with popt changes, including adding a
--[no]option toggle flag and a precedence group to options, and
perhaps attaching a table to carry named bits in a single option enum.
I see this code in rpm using popt: basically paths can be prefixed
with
an '@' attention marker, which will do stricter sanity checks
on files as in:
int rpmSecuritySaneFile(const char *filename)
{
struct stat sb;
uid_t uid;
if (stat(filename, &sb) == -1)
return 1;
uid = getuid();
if (sb.st_uid != uid)
return 0;
if (!S_ISREG(sb.st_mode))
return 0;
if (sb.st_mode & (S_IWGRP|S_IWOTH))
return 0;
return 1;
}
...
/* work-off each resulting file from the path element */
for (i = 0; i < ac; i++) {
const char *fn = av[i];
#if defined(RPM_VENDOR_OPENPKG) /* security-sanity-check-rpmpopt-and-
rpmmacros */
if (fn[0] == '@' /* attention */) {
fn++;
if (!rpmSecuritySaneFile(fn)) {
rpmlog(RPMLOG_WARNING, "existing POPT
configuration file \"%s\" considered INSECURE -- not loaded\n", fn);
continue;
}
}
#endif
(void)poptReadConfigFile(optCon, fn);
av[i] = _free(av[i]);
}
Should I drop the '@' prefix on file path "attention!" convention
into popt instead?
Its easy enough to do, although perhaps access(2) rather
than stat(2) might be slightly better owner checking.
73 de Jeff
smime.p7s
Description: S/MIME cryptographic signature
Re: [PATCH] Compile popt under Windows using gnulib
FWIW, I checked in a patch to check for srandom() and disable (by returning POPT_ERROR_BADOPERATION) if HAVE_SRANDOM was not defined when popt was built. I think we both agree that popt not building if srandom() is not present is a flaw no matter whether its solved by disabling POPT_ARGFLAG_RANDOM or by adding gnulib either internally/externally. hth 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: [PATCH] Compile popt under Windows using gnulib
(let's move this to popt-devel instead of rpm-devel) On Oct 25, 2008, at 11:42 AM, Richard W.M. Jones wrote: On Sat, Oct 25, 2008 at 11:31:49AM -0400, Jeff Johnson wrote: FYI: patches to , please. I'll take patches however they are sent however. *shrug* I can't tie builds of popt to gnulib, even for cross-compiles, there's way too many projects that depend on popt. Gnulib is a source code library, not a "library" library, so in normal usage Gnulib source files are included in the final tarball: http://www.gnu.org/software/gnulib/manual/gnulib.html#Library-vs-Reusable-Code So there's no extra dependency. Yes, I know what gnulib is. There most certainly is an extra dependency. If gnulib is internal, there is additional bloat in the tarball, and maintenenace of the internal copy. If gnulib is external, it imposes Yet Another prereqisite for building popt. As for whether you store the Gnulib files inside your version control system, that is a slightly different issue, discussed here: http://www.gnu.org/software/gnulib/manual/gnulib.html#VCS-Issues What could be done is disable the usage of glob(3) if glob is not available through configure tests. While I agree this is one way to do it, it does mean that Windows users get one less feature. AIUI globbing is used to implement the /etc/popt.d/* directory (?), hard-coded in the source unfortunately. A new port to cross-compiled MingW will never miss the feature. There's actually another portability issue which Gnulib can help with in the future - namely, random/srandom missing. I hacked around this in the patch I just sent, but Gnulib will solve it properly (and also for other platforms, not just Windows, which lack these calls). I've got a patch for Gnulib lined up to provide the random family of functions. There is nothing in the world (except bero & Ark linux) that uses or needs the ability to read /etc/popt.d/* files. The /etc/popt.d/* functionality is already conditioned on #if HAVE_GLOB_H and should quietly just drop out of -lpopt already. You already can add the feature by compiling popt with include (and perhaps library) paths that permit configure to find glob.h Meanwhile the other patches to retrofit srand instead of srandom likely need functionality disabled rather than retrofitted as well, because there is little increase in portability if srand is used instead of srandom. Yes, gnulib might provide srand but not srandom but that misses the semantic point below. Truly, when was the last time you needed a CLI option that returned a random number in a range for any executable purpose whatsoever? I cannot justify adding gnulib everywhere for obscure and undocumented and perhaps pointless -lpopt functionality. There are way way too many projects that depend on popt to justify adding gnulib to popt. OTOH, that's my POV. You are the first ever to suggest adding gnulib. Find enough support for adding gnulib to popt and I will certainly do so. ATM, you, and you alone, are the only person advocating adding gnulib to popt. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: cross-compilation of popt 1.14 fails
On Oct 9, 2008, at 2:55 PM, Paul Smith wrote: Hi all; If I try to cross-compile popt 1.14, the configure fails (I'm cross-compiling for powerpc): checking for string.h... (cached) yes checking for va_copy() function... configure: error: cannot run test program while cross compiling See config.log' for more details. I worked around this by pre-setting the variable in the environment when invoking configure: ac_cv_va_copy=C99 ./configure ... It would be nice if cross-compiling were supported. Fixed a couple of days ago. Here's the dinky patch: Index: CHANGES === RCS file: /v/rpm/cvs/popt/CHANGES,v retrieving revision 1.85 retrieving revision 1.86 diff -u -b -B -w -p -r1.85 -r1.86 --- CHANGES 18 Sep 2008 18:16:28 - 1.85 +++ CHANGES 4 Oct 2008 19:23:50 - 1.86 @@ -1,4 +1,5 @@ 1.14 -> 1.15: +- jbj: fix: remove AC_CHECK_VA_COPY check, va_copy is no longer used. - jbj: add eo.po and id.po (Translation Project). - jbj: updated da.po (Translation Project). - jbj: extend coverage to several additional setup routines. Index: configure.ac === RCS file: /v/rpm/cvs/popt/configure.ac,v retrieving revision 1.43 retrieving revision 1.44 diff -u -b -B -w -p -r1.43 -r1.44 --- configure.ac18 Sep 2008 18:16:28 - 1.43 +++ configure.ac4 Oct 2008 19:23:50 - 1.44 @@ -43,7 +43,6 @@ AC_SYS_LARGEFILE AC_ISC_POSIX AM_C_PROTOTYPES -AC_CHECK_VA_COPY AC_CHECK_HEADERS(float.h glob.h langinfo.h libintl.h mcheck.h unistd.h) 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Some umlaut issues in popt 1.15 (the old thing again)
Here's a summary from trying to sort the dueling umlaut encoding <->
--help column alignment
issue which started here:
http://rpm5.org/community/popt-devel/0061.html
what I need if you wish me to fix in popt-1.15 is a better
test. we don't disagree that popt-1.15-iso is broken. we do disagree
whether popt-1.13-iso "works" or not. I will sort the file(1)
difference, but I don't think that test is gonna be strong enough
(but I will find out tomorrow).
i expect columns to stop aligning, the "other" problem. I can
work the column alignment rather than the umlauts destruction if you
prefer. but I have dueling bugs atm.
I'll also see about adding a "make check" reproducer to popt.
your popt-devel post with rpm --help reproducer is exactly what I
needed to attempt. thank you.
np. Let's see how testing develops. A replay of one of the
previous patches is not such hard as I got you - if needed for 1.14+
or so
On May 25, 2008, at 6:34 PM, Jeff Johnson wrote:
On May 25, 2008, at 9:19 AM, Robert Scheck wrote:
Hello all,
I've claimed that I can see some umlaut issues with popt 1.14 and
I really
would like to see it solved, now. Reproducer is for me as follows.
Using
popt-1.13-3 from Fedora 8, 9 or Rawhide, I simply executed the
following:
$ [EMAIL PROTECTED] rpm --help > rpm-1.13-iso
$ LANG=de_DE.UTF-8 rpm --help > rpm-1.13-utf
Using popt 1.15 from latest CVS checkout, I also did the following:
$ [EMAIL PROTECTED] rpm --help > rpm-1.15-iso
$ LANG=de_DE.UTF-8 rpm --help > rpm-1.15-utf
Results are now as follows using file(1) to compare:
popt-1.13-iso: ISO-8859 text# as expected
popt-1.13-utf: UTF-8 Unicode text # as expected
popt-1.15-iso: Non-ISO extended-ASCII text # NOT as expected
popt-1.15-utf: UTF-8 Unicode text # as expected
Hmmm, file(1) can be tricked and is perhaps unreliable as a test.
I've written this toy reproducer script:
#!/bin/bash
sudo make -C /X/popt/popt-1.13 install >& /dev/null && echo
"==> 1.13-3.fc9 installed"
[EMAIL PROTECTED] rpm --help > /tmp/popt-1.13-iso
LANG=de_DE.UTF-8 rpm --help > /tmp/popt-1.13-utf
diff -u /tmp/popt-1.13-{iso,utf}
sudo make -C /X/src/popt install >& /dev/null && echo "==> 1.15
installed"
[EMAIL PROTECTED] rpm --help > /tmp/popt-1.15-iso
LANG=de_DE.UTF-8 rpm --help > /tmp/popt-1.15-utf
diff -u /tmp/popt-1.15-{iso,utf}
My script here was flawed, "... libdir=/lib ..." was also needed for
"make install". Now fixed.
With popt-1.13-3.fc9 installed (with the popt-1.13-popt_fprintf
patch applied), I see
differences rendering umlaut's between the output from the 2
locales for both 1.13 and 1.15.
(aside) the description column _IS_ aligned, which is/was the
rather feeble reproducer
for this issue during popt-1.14 release.
And here is what file(1) has to say on the matter:
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.13-iso
/tmp/popt-1.13-iso: ISO-8859 English text
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.15-iso
/tmp/popt-1.15-iso: Non-ISO extended-ASCII English text
reproducing your results here.
I have most certainly removed the encoding conversion in popt-1.14
so that the --help
columns are aligned. Which likely explains the file(1) output,
iconv is _NOT_ being
performed, because doing the iconv conversion visibly destroyed the
--help column alignment.
So a little more help please. What problem should I focus on?
Having only American
English kinda hampers my abilities to fix this issue, I really
don't know what to expect
with de_DE locale encoded display output (although I'm way more
familiar with
rpm --help than kudzu or random GNOME --help). I can most easily
detect column alignment
issues, duh.
And sure I want to fix _ALL_ the popt problems. Just which one first?
hth
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Re: Some umlaut issues in popt 1.15 (the old thing again)
On May 25, 2008, at 9:19 AM, Robert Scheck wrote:
Hello all,
I've claimed that I can see some umlaut issues with popt 1.14 and I
really
would like to see it solved, now. Reproducer is for me as follows.
Using
popt-1.13-3 from Fedora 8, 9 or Rawhide, I simply executed the
following:
$ [EMAIL PROTECTED] rpm --help > rpm-1.13-iso
$ LANG=de_DE.UTF-8 rpm --help > rpm-1.13-utf
Using popt 1.15 from latest CVS checkout, I also did the following:
$ [EMAIL PROTECTED] rpm --help > rpm-1.15-iso
$ LANG=de_DE.UTF-8 rpm --help > rpm-1.15-utf
Results are now as follows using file(1) to compare:
popt-1.13-iso: ISO-8859 text# as expected
popt-1.13-utf: UTF-8 Unicode text # as expected
popt-1.15-iso: Non-ISO extended-ASCII text # NOT as expected
popt-1.15-utf: UTF-8 Unicode text # as expected
Hmmm, file(1) can be tricked and is perhaps unreliable as a test.
I've written this toy reproducer script:
#!/bin/bash
sudo make -C /X/popt/popt-1.13 install >& /dev/null && echo "==>
1.13-3.fc9 installed"
[EMAIL PROTECTED] rpm --help > /tmp/popt-1.13-iso
LANG=de_DE.UTF-8 rpm --help > /tmp/popt-1.13-utf
diff -u /tmp/popt-1.13-{iso,utf}
sudo make -C /X/src/popt install >& /dev/null && echo "==> 1.15
installed"
[EMAIL PROTECTED] rpm --help > /tmp/popt-1.15-iso
LANG=de_DE.UTF-8 rpm --help > /tmp/popt-1.15-utf
diff -u /tmp/popt-1.15-{iso,utf}
With popt-1.13-3.fc9 installed (with the popt-1.13-popt_fprintf patch
applied), I see
differences rendering umlaut's between the output from the 2 locales
for both 1.13 and 1.15.
(aside) the description column _IS_ aligned, which is/was the rather
feeble reproducer
for this issue during popt-1.14 release.
And here is what file(1) has to say on the matter:
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.13-iso
/tmp/popt-1.13-iso: ISO-8859 English text
[EMAIL PROTECTED] popt]$ file /tmp/popt-1.15-iso
/tmp/popt-1.15-iso: Non-ISO extended-ASCII English text
reproducing your results here.
I have most certainly removed the encoding conversion in popt-1.14 so
that the --help
columns are aligned. Which likely explains the file(1) output, iconv
is _NOT_ being
performed, because doing the iconv conversion visibly destroyed the --
help column alignment.
So a little more help please. What problem should I focus on? Having
only American
English kinda hampers my abilities to fix this issue, I really don't
know what to expect
with de_DE locale encoded display output (although I'm way more
familiar with
rpm --help than kudzu or random GNOME --help). I can most easily
detect column alignment
issues, duh.
And sure I want to fix _ALL_ the popt problems. Just which one first?
Once my LANG is [EMAIL PROTECTED] for popt >= 1.14, the ISO-8859 text
seems not
sane to be written somehow. Please note, that popt 1.13, I was
using here
is not the vanilla one, the following patch was applied (no matter
what it
does exactly):
http://cvs.fedora.redhat.com/viewcvs/*checkout*/devel/popt/
popt-1.13-popt_fprintf.patch
Greetings,
Robertiso.gz>
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Using fnmatch(3) for popt alias/exec poptrc config
SSIA (aside) I've wanted to have the abilty to attach popt alias/exec to argv[0] from poptrc configuration using fnmatch(3) for years. Only legacy forward compatibility has stopped me ... For portability to the fnmatch(3) deprived, I'll work up a strncmp match to handle a trailing "*" splat without needing fnmatch(3). That should cover the common usage case. todo++ 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Fwd: Adding lcov/gcov coverage testing
Not sure where this msg went ... apologies if duplicated. Begin forwarded message: From: Jeff Johnson <[EMAIL PROTECTED]> Date: May 5, 2008 3:18:06 PM EDT To: popt-devel@rpm5.org Subject: Adding lcov/gcov coverage testing I'm going to add coverage testing for popt based on lcov-1.6.1 from the LTP. Here's an example of the coverage HTML: http://wraptastic.org/pub/jbj/lcov/src/popt/index.html Very useful imho. E.g. its not very hard to understand why --help columns do not align when the coverage of popthelp.c is 0/362 0%. Duh! 73 de Jeff
Handling 1-of-N mutually exclusive options within popt
A common paradigm when using CLI options is to select 1-of-N mutually exclusive options. ATM, the burden of detecting "mutually exclusive" is up to the application, not popt. There's room (if I'm careful) for adding bits (I *think* I can squeeze out 8 bits) for a "mutually exclusive" group identifier into the 32b argInfo field, and then implement "mutually exclusive" by running a counter on the group options. There are other reasons than "mutually exclusive" to add a group identifier. Consider filtering options in popt tables based on mode of execution, where some options should only be interpreted in a specific mode of operation. rpm has many examples of both "mutually exclusive" options, and per- mode options. Yes, there are certainly application specific means to handle the usage cases. My specific and immediate usage case is writing a "universal" compressor/decompressor that handles gzip/bzip2/lzma/lzo and whatever else seems useful. Note that legacy compatibility would be achieved by reserving group 0 to be always available. Yet Another reason for adding option groups is feature parity with Newer! Better! Bestest! getopt_long() processing as implemented in (iirc, I fergit where I saw) GNU tar/cpio. (aside) My personal opinion of the GNU option grouping in tar/cpio was quietly disposed of in a barf bag. However, there are certainly other, possibly valid, certainly different, opinions around. I know only my own aesthetics. Opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Permit arg units (like 100Kb), add getdate?
On May 2, 2008, at 2:24 AM, Ralf S. Engelschall wrote:
On Thu, May 01, 2008, Jeff Johnson wrote:
While using popt in another program, I came across 2 usage
cases that might be usefully internalized in popt.
One usage case was an attempt to specify a memory limit as
--memory 16777216
Adding unit scaling, e.g.
--memory 16Mb
--memory 16384Kb
would not be hard to add using something like
POPT_ARG_INT | POPT_ARGFLAG_UNITS
with the usual Kb/Kib Mb/Mib Gb/Gib tokens to prescale.
Another possible popt-1.15 usage case is getdate.y, which is
wonderfully
useful for time/date argument input. While it is
rather easy to use getdate.y through a popt callback, perhaps
its time to push getdate.y into libpopt, and add a
POPT_ARG_DATE
distinguished method to make it as easy as possible to use getdate.y.
I'm also seeing a need for a KEY=VALUE comma separated list popt
sub-table option parser (there's a mouthful ;-)
I'm starting to see
--option K1=V1,K2=V2,K3=V3
occurrences more often. I can think of a couple ways to use popt
to recursively parse KEY=VALUE settings, either by keeping track
of a chained popt table that is used iff a specific --option is
encountered,
or by overloading the pointer field to be used as a option specific
subtable parse.
Opinions?
The --option K1=V1,K2=V2,K3=V3 is generic and would be very useful
IMHO.
But the unit and date parsing is something I think should be perhaps
still better done in the application (or the POPT callback) because
where is the end of this story? If POPT provides units and dates,
someone else comes and wants URL arguments pre-parsed into its parts,
that user/group names are converted to user ids and vice versa, that
chmod(8) strings are converted to octals and vice versa, etc. Either
POPT would have to provide a really large set of those conversions are
better none at all. Additionally, if POPT provides those
conversions it
perhaps better should be at least placed into a POPT sub-library so
that
one still can have a rather small plain POPT library and if
required and
wished one can link against the additional POPT "utility" library for
addon functionalities like those conversions...
See what is implemented in LZMA lzma/src/lzma/util.c str_to_uint64()
for what motivated the unit scaling suggestion. A table like
static const struct {
const char *name;
uint64_t multiplier;
} suffixes[] = {
{ "k", UINT64_C(1000) },
{ "M", UINT64_C(100) },
{ "G", UINT64_C(10) },
{ "Ki", UINT64_C(1024) },
{ "Mi", UINT64_C(1048576) },
{ "Gi", UINT64_C(1073741824) },
{ NULL, 0 }
};
to do scaling of an integer, or handling the [min,max] range check
in str_to_uint64() with a table instead of arguments is what I
contemplated. A table
driven parse and multiply is not much different than a table driven
parse and set for "K=V" strings, particularly since the popt 7-tuple
is likely to be abused for the subtables.
Multiplying or comparing integers is quite general, and a very different
problem space than slicing URI's or converting user id's, or handling
mode
arguments (which could be parsed using %b format if tokens were fixed
length
single chars w/o ',' separators, but I digress ...)
Parsing dates is definitely tricky, which is the reason why getdate.y
was originally written.
The rationale for adding to popt is largely to make getdate.y more
widely available.
Yet Another Library to multiply/compare integers, or to handle the
addition
of a single routine like getdate.y seems overkill.
Thanks for comments.
73 de Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Named bit fields ala ancient BSD %b format
While I'm thinking about popt features, here is another:
The BSD kernel (iirc) devised a means to map bit files to names
using a %b format.
A string was used to map bit# <-> name. E.g. here's an example
from some rpmdb code that I use for output purposes
static const char * dbtFlags =
"\20\1APPMALLOC\2ISSET\3MALLOC\4PARTIAL\5REALLOC\6USERMEM
\7DUPOK";
The same technique (of nerdy embedded octal in a string) can be used
to assign names to bits in a set,
permitting option parsing (using the format string above):
--option PARTIAL,DUPOK
to save the value 0x48 (i.e. (1 << 3) | (1 << 6)) into the target
storage.
OTOH, generating the %b format string can be tedious, and perhaps
a more general table like (this from LZMA):
typedef struct {
const char *name;
uint64_t id;
} name_id_map;
with name = NULL as table end sentinel should be done.
Opinions?
73 De Jeff
__
POPT Library http://rpm5.org
Developer Communication List popt-devel@rpm5.org
Permit arg units (like 100Kb), add getdate?
While using popt in another program, I came across 2 usage cases that might be usefully internalized in popt. One usage case was an attempt to specify a memory limit as --memory 16777216 Adding unit scaling, e.g. --memory 16Mb --memory 16384Kb would not be hard to add using something like POPT_ARG_INT | POPT_ARGFLAG_UNITS with the usual Kb/Kib Mb/Mib Gb/Gib tokens to prescale. Another possible popt-1.15 usage case is getdate.y, which is wonderfully useful for time/date argument input. While it is rather easy to use getdate.y through a popt callback, perhaps its time to push getdate.y into libpopt, and add a POPT_ARG_DATE distinguished method to make it as easy as possible to use getdate.y. I'm also seeing a need for a KEY=VALUE comma separated list popt sub-table option parser (there's a mouthful ;-) I'm starting to see --option K1=V1,K2=V2,K3=V3 occurrences more often. I can think of a couple ways to use popt to recursively parse KEY=VALUE settings, either by keeping track of a chained popt table that is used iff a specific --option is encountered, or by overloading the pointer field to be used as a option specific subtable parse. Opinions? 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Q: libpopt.vers
On Apr 26, 2008, at 6:40 PM, Dmitry V. Levin wrote: Hi, I wonder why the libpopt.vers version script exports _init and _fini symbols? Not so much exports the symbols as permits exposure. Dynamic linker calls _init/_fini anyway. Regular application can use these symbols only via dlsym(3). The likeliest reason is that gnutls (which I used as a model for drilling loader maps into rpm/popt) exported the symbols several years ago, and its seemed like a good idea. No other reason I remember. I'm more interested in tracking the symbols in the ABI, and preventing accidental leakage of other symbols in libpopt (and rpm libraries) than anything else. Neither popt (with its POPT_AUTOALIAS table) and similar static arrays exposed in the rpmlib ABI have symbols that qualify as an ABI. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt-1.14 release
On Apr 2, 2008, at 1:46 PM, Jeff Johnson wrote: I've built popt-1.14 for release in the next couple of days. The SRPM for popt-1.4 that I will be releasing is at http://wraptastic.org/pub/i386-linux/SRPMS/popt-1.14-1.src.rpm Enjoy! I've released popt-1.14. You can now find the filnal packages and tar/ xar balls at http://rpm5.org/files/popt 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
popt-1.14 release
I've built popt-1.14 for release in the next couple of days. The SRPM for popt-1.4 that I will be releasing is at http://wraptastic.org/pub/i386-linux/SRPMS/popt-1.14-1.src.rpm Enjoy! 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: A little more stpcpy() tweaking + a "..." bonus
On Mar 26, 2008, at 11:13 AM, Wayne Davison wrote: I made a couple more places use stpcpy() in the code, optimizing away a couple strlen() calls in the process. I also tweaked the help code that was abbreviating a long value using "..." to allow a string to fit if it can, rather than always reserving space for the "..." chars. I.e. if we had room for 4 word chars, instead of outputting "w...", it would allow a full "word" to be output, since it fits. Just in case: this patch was added in the last 2 days, will be in popt-1.14 when released. I've also fixed "make check" now that the root cause, broken Fedorable /bin/echo, has beeen repaired. Still no word from the umlaut's, or the UTF8 -> current locale, problems. I'm likely to release popt-1.14 in spite of the b0rkage, popt-1.14 is fully functional in spite of whether --help columns align (or not) with the current patch set. No help == no fixing, as always in OSS. Visual display issues wrto wide character encoding are nasty reproducers, extremely specific to envvar's and other reproducer peculierarities ... FWIW, I'd really really like to fix the problems. Alas, I'm just a dumb 'merikkkan, ASCII is always gud enuf for me. popt-1.14 is not broken if --help columns don't align ... 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: popt-1.13 fix for uclibc builds
Added, will be in popt-1.14. Thanks for the patch. 73 de Jeff On Mar 27, 2008, at 12:01 PM, Stanislav Brabec wrote: Hallo. Attached patch allows to build popt-1.13 on both glibc and uclibc based systems. Henning Heinold pointed to the problem and added @[EMAIL PROTECTED] Adding AM_ICONV_LINK fixed on glibc systems. Index: popt-1.13/Makefile.am === --- popt-1.13.orig/Makefile.am 2007-11-14 20:42:27.0 + +++ popt-1.13/Makefile.am 2008-03-26 22:14:05.0 + @@ -39,7 +39,7 @@ usrlib_LTLIBRARIES = libpopt.la libpopt_la_SOURCES = popt.c findme.c poptparse.c poptconfig.c popthelp.c poptint.c -libpopt_la_LDFLAGS = -no-undefined @LTLIBINTL@ +libpopt_la_LDFLAGS = -no-undefined @LTLIBINTL@ @LTLIBICONV@ if HAVE_LD_VERSION_SCRIPT libpopt_la_LDFLAGS += -Wl,--version-script=/libpopt.vers Index: popt-1.13/configure.ac === --- popt-1.13.orig/configure.ac 2007-11-14 20:42:27.0 + +++ popt-1.13/configure.ac 2008-03-26 22:20:16.0 + @@ -95,6 +95,7 @@ AC_CHECK_FUNCS(getuid geteuid mtrace __secure_getenv setregid strerror iconv) AM_GNU_GETTEXT([external]) +AM_ICONV_LINK popt_sysconfdir="" eval "popt_sysconfdir=\"\"" # expand contained -- Best Regards / S pozdravem, Stanislav Brabec software developer - SUSE LINUX, s. r. o. e-mail: [EMAIL PROTECTED] Lihovarská 1060/12 tel: +420 284 028 966, +49 911 740538747 190 00 Praha 9 fax: +420 284 028 951 Czech Republichttp://www.suse.cz/ __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Question about the xmalloc() et al functions
On Mar 9, 2008, at 10:00 PM, Wayne Davison wrote: On Sun, Mar 09, 2008 at 08:27:25PM -0400, Jeff Johnson wrote: Using xmalloc just opens up a can-of-worms while lusers fuss about non-gcc compiler extension portability. Aha, I had failed to notice that the "? :" bit was a gcc extension. What I don't like is that normal users get some memory functions that are fatal if they use gcc, while all others get non-fatal functions. Perhaps the default should be for all normal compiles to get the non- fatal functions, and then anyone that wanted to use the current gcc defines could set something extra (such as -DFATAL_MEM). Very good point. OTOH, I don't understand why some lusers are permitted vasprintf and strlcpy, while others have to hobble along with alternatives. One other reason I care about this is that I think the various bits of code that are doing a strdup()-like action should be using a strdup()- like call, not each one having their own set of strlen/malloc/strcpy calls. It's clearer and a bit safer. Yup. Way too much C developer time is spent fussing with malloc'ing strings. I'm attaching a patch that adds the extra FATAL_MEM requirement, and then uses xstrdup() in a few places (which would mean that strdup() is being used for normal popt users). I'm still considering switching to xstrdup, likely used inconsistently atm. I was also working on changing some code to use your newly-added stpcpy() function, so I included that work too. Some of the changes are optimizations to avoid a strlen() call (when we can easily compute the length via subtraction) and a fix for a potential memory leak if realloc() returns NULL. Nice. I optimized away a few variables, and rewrote findProgramPath, while adding your patch. See what you think. I sometimes regret later what seems clever atm. Off to waste some more development time malloc'ing strings ;-) 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Question about the xmalloc() et al functions
On Mar 9, 2008, at 7:59 PM, Wayne Davison wrote: On Sun, Mar 09, 2008 at 07:26:31PM -0400, Jeff Johnson wrote: There are some subtleties however with xstrdup. [... mtrace related issues ...] Right, that's why I preserved the HAVE_MCHECK_H version using its own malloc() call (it's unchanged from your version) and just made the non- HAVE_MCHECK_H version add failure checking. All the other functions are the same for both (with exit-on-failure handling). I'd love to use xmalloc everywhere. Alas, there's still lots of dain bread portability issues, the patches to "fix" portability are usually incorrect and limited in scope. See e.g. __secure_getenv and the setresgid/ setresuid "stuff", to name just two. The va_copy() hackery entered popt through the same Newer! Better! Bestest! window, where popt is now undertaking iconv conversions and bind_textdomain_codeset() retrofits that _REALLY_ need to be solved in applications, not in popt. Using xmalloc just opens up a can-of-worms while lusers fuss about non-gcc compiler extension portability. Again, I'd *love* to use xmalloc and xstrdup, I use daily, worksforme. But at some point, popt should just process options, no fuss, no muss, no problems. 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
Re: Question about the xmalloc() et al functions
On Mar 9, 2008, at 7:13 PM, Wayne Davison wrote: I'm curious why the xmalloc() function (and its brethren) exits with a fatal error on only some systems and not on all systems? I would think that the code should use the exiting functions in all cases, and just provide differing versions of xstrdup() so that the HAVE_MCHECK_H version does a malloc() (rather than strdup() doing it). xmalloc() et al is mostly a fetish of mine. I personally believe that its silly wasting time checking _ALL_ code paths for malloc failures. 99.999% of the time returning an error rather than aborting is pointless anyways (imho). If I'm not off track, you can apply the attached patch to implement this. There are some subtleties however with xstrdup. If strdup(3), rather than what I use with rpm #define xstrdup(_str) (strcpy((malloc(strlen(_str)+1) ? : vmefail(strlen(_str)+1)), (_str))) is done, then line numbers reported by glibc mtrace(3) will be wrto strdup, not the caller of strdup. All that stops me from using everywhere is the GNU extension that is needed by xstrdup so that the xstrdup macro is mostly side-effect free (I don't care so much that the (_str) macro argument is used several times on the error paths, its used only once on the success paths. And sure a static inline function avoids all the side-effects, but static inline functions were unknown to me last century when the hackery was first devised. valgrind dinna exist, and Rational Purify cost the Very Big $$$ back then too ... 73 de Jeff __ POPT Library http://rpm5.org Developer Communication List popt-devel@rpm5.org
