Re: units issues

2017-05-23 Thread Brian Inglis
On 2017-05-23 17:55, Doug Henderson wrote:
> On 23 May 2017 at 15:49, Brian Inglis wrote:
> 
>> Updating the currencies only when setup is run seems to me to be
>> insufficient if users want to use current currency conversions.
> 
> Currencies needs to be split to a different package from non-currency
> units. Non-currency units is very static, whereas most currencies
> changes daily. In the absence of updates, non-currency units are still
> useful; but currencies are incorrect in most cases, and increasingly
> so as time pass.
> 
> It is difficult to predict user requirements based on current package
> dependencies as non-currency units and currencies are likely used by
> almost disjoint sets of packages. Users of non-currency units should
> not be burdened by the complexities of currencies. But if they do need
> currencies, they should be up to date.
> 
> A possible solution is to check the currency of the currency data on
> library initialization or each use by comparing the current date with
> the data's date of update. When the data is stale, automatically
> update it if possible, or notify the user of the problem.

It's a command line utility from GNU with currency conversion factors in
a separate definition file included from the main file, updated by a
Python script, which downloads an RSS XML file of current (Euro) rates
from a free source with a permissive licence, and converts it to
definitions acceptable to the utility, overwriting the existing file.

The main issues are that, as currently implemented, currency rates are
updated automatically by a postinstall script only when setup is run;
setup may be running in an environment without external access, so the
postinstall script will generate an error; users may not want or care
about currency updates; and the postinstall script uses find to avoid
updating if there is no currency file, or it has been updated recently.

One option to deal with this is update the package to install a zero
length currency definitions file, so currency conversions are not
defined, but the program has no issues, and drop the permanent
postinstall script to perform updates.
Then announce and document that users who want updated currency
conversion rates need to run the update script from the command line, a
profile script, cron job, or Windows Scheduled Task, as is desirable if
they use currency conversions.

There are also issues with the Python update script, as the currency
source site cert expired recently, causing the update and postinstall
scripts to fail, with no workaround other than a replacement or patch;
and a poster has problems using the update script with Windows proxies,
which I have addressed to the upstream maintainer for discussion about
approaches.
If there is no response, I will create a Cygwin update patch and submit
it upstream, but there has been no visible response to issues raised on
the GNU site.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada


Re: units issues

2017-05-23 Thread Doug Henderson
On 23 May 2017 at 15:49, Brian Inglis wrote:

> Updating the currencies only when setup is run seems to me to be
> insufficient if users want to use current currency conversions.

Currencies needs to be split to a different package from non-currency
units. Non-currency units is very static, whereas most currencies
changes daily. In the absence of updates, non-currency units are still
useful; but currencies are incorrect in most cases, and increasingly
so as time pass.

It is difficult to predict user requirements based on current package
dependencies as non-currency units and currencies are likely used by
almost disjoint sets of packages. Users of non-currency units should
not be burdened by the complexities of currencies. But if they do need
currencies, they should be up to date.

A possible solution is to check the currency of the currency data on
library initialization or each use by comparing the current date with
the data's date of update. When the data is stale, automatically
update it if possible, or notify the user of the problem.

Doug

-- 
Doug Henderson, Calgary, Alberta, Canada - from gmail.com


Re: units issues

2017-05-23 Thread Brian Inglis
On 2017-05-23 11:28, Achim Gratz wrote:
> Brian Inglis writes:
>> The easiest approaches to this would be:
>> - rename or delete postinstall script which might upset cygcheck or
>> setup remove
>> - null /usr/share/units/currency.units, as it is required and produces
>> an error message if not available, but if it is empty, everything works.
> 
> Nope.  The user should not have to muck with packaged files at all.
> 
>> It is not very useful if not up to date (I run the update daily), and if
>> not up to date, might as well be empty, but some opinions may differ.
> 
> The issue is that you cannot assume that postinstall scripts are able to
> access the network at all.  Where this isn't possible the administrator
> will have to find a different way of keeping those files up-to-date
> (which should also not be packaged), but that's something to maybe just
> document.  But you will have to provide some way of letting the user
> specify if that updating is OK.  If you want this to be possible during
> the initial install you might even need to provide another sub-package
> whose only purpose is to confer this decision (it could be empty and
> just doing a simple postinstall action).
> 
>> In the postinstall script find which ensures updates happen at most daily:
> 
> I've asked specifically to structure any perpetual postinstall script in
> a way that it doesn't do any non-necessary work, be it forking or
> otherwise unless it is going to actually do something useful.  A simple
> file test that exits the postinstall script when the file is or isn't
> there satisfies that constraint, running commands and scripts doesn't.

Updating the currencies only when setup is run seems to me to be
insufficient if users want to use current currency conversions.

Would the best approach be to punt on running the update script at all,
install a null /usr/share/units/currency.units file, announce and
document that if currency conversions are desirable, the user should
arrange to run the update script, either from the command line, profile,
cron job, or Task Scheduler?

If we kept the postinstall script, we could change it to run only if the
currency.units file is non-null and drop the time check:
[ -s /usr/share/units/currency.units ] && /usr/bin/units_cur
or keep the time check, using find -mtime +0, or date and stat.

I have followed up with the upstream maintainer AdrianM at GNU dot org
about the Python currency update script issues with no response yet.
Does anyone have contact or know a better address to try? PM if so.

-- 
Take care. Thanks, Brian Inglis, Calgary, Alberta, Canada


Re: calm: mksetupini doesn't allow non-empty source but empty install files for an obsolete package

2017-05-23 Thread Ken Brown

On 5/23/2017 12:55 PM, Jon Turney wrote:

On 23/05/2017 14:44, Ken Brown wrote:
I've created an obsolete package (as discussed starting at 
https://sourceware.org/ml/cygwin-apps/2017-05/msg00084.html).  But 
mksetupini doesn't like it:


mksetupini: package 'texlive-collection-htmlxml' version '20170520-1' 
source has no non-empty install tarfiles

mksetupini: package set has errors, not writing setup.ini

I thought the following would fix it:

--- a/calm/package.py
+++ b/calm/package.py
@@ -636,6 +636,9 @@ def validate_packages(args, packages):
  if 
packages[p].tars[packages[p].vermap[v]['source']].is_empty:

  continue


This should be being caught by this case (source is empty), but it 
isn't, I guess because it's got a .keep file due to other problems 
previously discussed.


The source contains a .cygport file and a fake upstream source tarball.



+if '_obsolete' in packages[p].vermap[v].get('category', ''):
+continue


I think this maybe needs to be 
packages[p].version_hints[v].get('category', '') ?


Yes, that fixes it, thanks.  Patch attached.

Ken
From 53014620d8c1c54f0fc8bede458af9709c62142d Mon Sep 17 00:00:00 2001
From: Ken Brown 
Date: Tue, 23 May 2017 14:13:29 -0400
Subject: [PATCH] Allow an obsolete package with source to have empty install
 tarfiles

---
 calm/package.py | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/calm/package.py b/calm/package.py
index 6ddb4a5..b0ec360 100755
--- a/calm/package.py
+++ b/calm/package.py
@@ -636,6 +636,9 @@ def validate_packages(args, packages):
 if packages[p].tars[packages[p].vermap[v]['source']].is_empty:
 continue
 
+if '_obsolete' in packages[p].version_hints[v].get('category', ''):
+continue
+
 if not packages[p].tars[packages[p].vermap[v]['source']].is_used:
 logging.error("package '%s' version '%s' source has no 
non-empty install tarfiles" % (p, v))
 error = True
-- 
2.12.3



[Attn Maintainer] procps-ng

2017-05-23 Thread Achim Gratz

It appears that the procps command is missing the manpage, most likely
because Cygwin has its own manpage for ps and got updated more
recently.  Can you please arrange for a manpage "procps" to coincide
with the command name so that the two files no longer collide?

Also, it appears that there's an upstream bug in top that shows the main
memory utilization numbers in % of total, but the unit is shown as "GB",
which can be very confusing...


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Wavetables for the Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#BlofeldUserWavetables


Re: [PATCH setup 08/15] Fix comments and indentation in check_for_cached

2017-05-23 Thread Achim Gratz
Jon Turney writes:
> +  /* Note that the cache dir is represented by a mirror site of 
> file://local_dir */
>std::string prefix = "file://" + local_dir + "/";
> -  /* FIXME: Nullness check can go away once packagesource is properly
> -   * std::string-ified, and doesn't use overcomplex semantics. */
> -  std::string fullname = prefix + 
> -(pkgsource.Canonical() ? pkgsource.Canonical() : "");
> +  std::string fullname = prefix + pkgsource.Canonical();
> +

That part of the patch was a bit premature, but I see you've already
fixed it.  I found that interestingly it only crashed on 64bit, which
seems a bit odd, but maybe that's Windows for you.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds


Re: units issues

2017-05-23 Thread Achim Gratz
Brian Inglis writes:
> The easiest approaches to this would be:
> - rename or delete postinstall script which might upset cygcheck or
> setup remove
> - null /usr/share/units/currency.units, as it is required and produces
> an error message if not available, but if it is empty, everything works.

Nope.  The user should not have to muck with packaged files at all.

> It is not very useful if not up to date (I run the update daily), and if
> not up to date, might as well be empty, but some opinions may differ.

The issue is that you cannot assume that postinstall scripts are able to
access the network at all.  Where this isn't possible the administrator
will have to find a different way of keeping those files up-to-date
(which should also not be packaged), but that's something to maybe just
document.  But you will have to provide some way of letting the user
specify if that updating is OK.  If you want this to be possible during
the initial install you might even need to provide another sub-package
whose only purpose is to confer this decision (it could be empty and
just doing a simple postinstall action).

> In the postinstall script find which ensures updates happen at most daily:

I've asked specifically to structure any perpetual postinstall script in
a way that it doesn't do any non-necessary work, be it forking or
otherwise unless it is going to actually do something useful.  A simple
file test that exits the postinstall script when the file is or isn't
there satisfies that constraint, running commands and scripts doesn't.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Blofeld:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds


[PATCH setup 10/10] Remove OR from grammar

2017-05-23 Thread Jon Turney
Unused since removal of complex dependency rules in 60b4f6ca
---
 inilex.ll   | 1 -
 iniparse.yy | 2 +-
 2 files changed, 1 insertion(+), 2 deletions(-)

diff --git a/inilex.ll b/inilex.ll
index b454f0f..13422b1 100644
--- a/inilex.ll
+++ b/inilex.ll
@@ -141,7 +141,6 @@ B64 [a-zA-Z0-9_-]
 "<"return LT;
 "=" return EQUAL;
 \, return COMMA;
-"|"return OR;
 "@"return AT;
 
 {STR}  { yylval = new char [strlen(yytext) + 1];
diff --git a/iniparse.yy b/iniparse.yy
index c540146..18ebe2a 100644
--- a/iniparse.yy
+++ b/iniparse.yy
@@ -42,7 +42,7 @@ extern int yylineno;
 %token MD5 SHA512
 %token SOURCEPACKAGE
 %token PACKAGENAME
-%token COMMA OR NL AT
+%token COMMA NL AT
 %token OPENBRACE CLOSEBRACE EQUAL GT LT GTEQUAL LTEQUAL 
 %token BUILDDEPENDS
 %token MESSAGE
-- 
2.12.3



[PATCH setup 08/10] Fix infinite recursion in grammar for depends

2017-05-23 Thread Jon Turney
Allowing listseparator to be empty allows an infinite recursion in the
versionedpackagelist rule

Also make the comment documenting versionedpackageentry is non-empty
consistent with all other similar comments

Also allow lower-case depends:
---
 inilex.ll   | 2 +-
 iniparse.yy | 6 +++---
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/inilex.ll b/inilex.ll
index 798a04b..b454f0f 100644
--- a/inilex.ll
+++ b/inilex.ll
@@ -121,7 +121,7 @@ B64 [a-zA-Z0-9_-]
 
 "category:"|"Section:" return CATEGORY;
 "requires:"return REQUIRES;
-"Depends:" return DEPENDS;
+[dD]"epends:"  return DEPENDS;
 
 ^{STR}":"  ignore_line ();
 
diff --git a/iniparse.yy b/iniparse.yy
index d768400..8ee7dc3 100644
--- a/iniparse.yy
+++ b/iniparse.yy
@@ -132,12 +132,12 @@ versionedpackagelistsp /* non-empty */
  | versionedpackagelistsp versionedpackageentry
  ;
 
-listseparator: /* empty */
- | COMMA
+listseparator /* non-empty */
+ : COMMA
  | COMMA NL
  ;
 
-versionedpackageentry /* empty not allowed */
+versionedpackageentry /* non-empty */
  : STRING { iniBuilder->buildPackageListNode($1); } versioncriteria
  ;
 
-- 
2.12.3



[PATCH setup 09/10] Improve error recovery in setup.ini parsing

2017-05-23 Thread Jon Turney
Following the error token with a NL allows the parser to discard tokens
until a NL is found to resynchronize, rather than aborting.

This doesn't help hugely, as *any* parse errors are considered fatal by
do_remote_ini()/do_local_ini() and won't let us proceed.
---
 iniparse.yy | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/iniparse.yy b/iniparse.yy
index 8ee7dc3..c540146 100644
--- a/iniparse.yy
+++ b/iniparse.yy
@@ -104,7 +104,7 @@ singleitem /* non-empty */
  | REQUIRES { iniBuilder->buildBeginDepends(); } versionedpackagelistsp NL
  | BUILDDEPENDS { iniBuilder->buildBeginBuildDepends(); } versionedpackagelist 
NL
  | MESSAGE STRING STRING NL{ iniBuilder->buildMessage ($2, $3); }
- | error   { yyerror (std::string("unrecognized line ") 
+ | error NL{ yyerror (std::string("unrecognized line ")
  + stringify(yylineno)
  + " (do you have the latest setup?)");
}
-- 
2.12.3



Re: calm: mksetupini doesn't allow non-empty source but empty install files for an obsolete package

2017-05-23 Thread Jon Turney

On 23/05/2017 14:44, Ken Brown wrote:

I've created an obsolete package (as discussed starting at 
https://sourceware.org/ml/cygwin-apps/2017-05/msg00084.html).  But mksetupini 
doesn't like it:

mksetupini: package 'texlive-collection-htmlxml' version '20170520-1' source 
has no non-empty install tarfiles
mksetupini: package set has errors, not writing setup.ini

I thought the following would fix it:

--- a/calm/package.py
+++ b/calm/package.py
@@ -636,6 +636,9 @@ def validate_packages(args, packages):
  if packages[p].tars[packages[p].vermap[v]['source']].is_empty:
  continue


This should be being caught by this case (source is empty), but it 
isn't, I guess because it's got a .keep file due to other problems 
previously discussed.




+if '_obsolete' in packages[p].vermap[v].get('category', ''):
+continue


I think this maybe needs to be 
packages[p].version_hints[v].get('category', '') ?



+
  if not packages[p].tars[packages[p].vermap[v]['source']].is_used:
  logging.error("package '%s' version '%s' source has no non-empty 
install tarfiles" % (p, v))
  error = True


But I must have something wrong, because I still get the error after this 
change.


[PATCH setup 07/10] Fold build(Install|Source)(MD5|SHA512) into buildPackage(Install|Source)

2017-05-23 Thread Jon Turney
Fold build(Install|Source)(MD5|SHA512) into buildPackage(Install|Source), so
the (pathname, size, hash) information from an install: or source: line is
all processed together.
---
 IniDBBuilderPackage.cc | 76 --
 IniDBBuilderPackage.h  | 20 +
 iniparse.yy| 16 +++
 3 files changed, 60 insertions(+), 52 deletions(-)

diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc
index c90bfe3..ad1cc88 100644
--- a/IniDBBuilderPackage.cc
+++ b/IniDBBuilderPackage.cc
@@ -120,15 +120,37 @@ IniDBBuilderPackage::buildPackageLDesc (const 
std::string& theDesc)
 
 void
 IniDBBuilderPackage::buildPackageInstall (const std::string& path,
-  const std::string& size)
+  const std::string& size,
+  char *hash,
+  hashType type)
 {
   process_src (*cbpv.source(), path);
   setSourceSize (*cbpv.source(), size);
+
+  switch (type) {
+  case hashType::sha512:
+if (hash && !cbpv.source()->sha512_isSet)
+  {
+memcpy (cbpv.source()->sha512sum, hash, 
sizeof(cbpv.source()->sha512sum));
+cbpv.source()->sha512_isSet = true;
+  }
+break;
+
+  case hashType::md5:
+if (hash && !cbpv.source()->md5.isSet())
+  cbpv.source()->md5.set((unsigned char *)hash);
+break;
+
+  case hashType::none:
+break;
+  }
 }
 
 void
 IniDBBuilderPackage::buildPackageSource (const std::string& path,
- const std::string& size)
+ const std::string& size,
+ char *hash,
+ hashType type)
 {
   packagedb db;
   /* get an appropriate metadata */
@@ -168,6 +190,24 @@ IniDBBuilderPackage::buildPackageSource (const 
std::string& path,
   spec.setVersion (cbpv.Canonical_version());
 
   setSourceSize (*cspv.source(), size);
+
+  switch (type) {
+  case hashType::sha512:
+if (hash && !cspv.source()->sha512_isSet)
+  {
+memcpy (cspv.source()->sha512sum, hash, 
sizeof(cspv.source()->sha512sum));
+cspv.source()->sha512_isSet = true;
+  }
+break;
+
+  case hashType::md5:
+if (hash && !cspv.source()->md5.isSet())
+  cspv.source()->md5.set((unsigned char *)hash);
+break;
+
+  case hashType::none:
+break;
+  }
 }
 
 void
@@ -200,38 +240,6 @@ IniDBBuilderPackage::buildBeginDepends ()
 }
 
 void
-IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512)
-{
-  if (sha512 && !cbpv.source()->sha512_isSet) {
-memcpy (cbpv.source()->sha512sum, sha512, sizeof cbpv.source()->sha512sum);
-cbpv.source()->sha512_isSet = true;
-  }
-}
-
-void
-IniDBBuilderPackage::buildSourceSHA512 (unsigned char const *sha512)
-{
-  if (sha512 && !cspv.source()->sha512_isSet) {
-memcpy (cspv.source()->sha512sum, sha512, sizeof cspv.source()->sha512sum);
-cspv.source()->sha512_isSet = true;
-  }
-}
-
-void
-IniDBBuilderPackage::buildInstallMD5 (unsigned char const *md5)
-{
-  if (md5 && !cbpv.source()->md5.isSet())
-cbpv.source()->md5.set(md5);
-}
-
-void
-IniDBBuilderPackage::buildSourceMD5 (unsigned char const *md5)
-{
-  if (md5 && !cspv.source()->md5.isSet())
-cspv.source()->md5.set(md5);
-}
-
-void
 IniDBBuilderPackage::buildBeginBuildDepends ()
 {
 #if DEBUG
diff --git a/IniDBBuilderPackage.h b/IniDBBuilderPackage.h
index 1dab41b..766a5ef 100644
--- a/IniDBBuilderPackage.h
+++ b/IniDBBuilderPackage.h
@@ -18,10 +18,13 @@
 
 #include 
 #include "package_version.h"
+
 class IniParseFeedback;
 class packagesource;
 class packagemeta;
 
+enum class hashType { none, md5, sha512 };
+
 class IniDBBuilderPackage
 {
 public:
@@ -34,16 +37,21 @@ public:
   void buildPackageVersion (const std::string& );
   void buildPackageSDesc (const std::string& );
   void buildPackageLDesc (const std::string& );
-  void buildPackageInstall (const std::string&, const std::string&);
-  void buildPackageSource (const std::string&, const std::string&);
+  void buildPackageInstall (const std::string&, const std::string&,
+char *, hashType);
+  void buildPackageSource (const std::string&, const std::string&,
+   char *, hashType);
+
+  // helpers for ScanFindVisitor
+  void buildPackageInstall (const std::string& path, const std::string& size)
+  { buildPackageInstall(path, size, NULL, hashType::none); }
+  void buildPackageSource (const std::string& path, const std::string& size)
+  { buildPackageSource(path, size, NULL, hashType::none); }
+
   void buildPackageTrust (int);
   void buildPackageCategory (const std::string& );
 
   void buildBeginDepends ();
-  void buildInstallSHA512 (unsigned char const[64]);
-  void buildSourceSHA512 (unsigned char const[64]);
-  void buildInstallMD5 (unsigned char const[16]);
-  void buildSourceMD5 (unsigned char 

[PATCH setup 06/10] Fold IniDBBuilderPackage::buildInstallSize() into buildPackageInstall()

2017-05-23 Thread Jon Turney
As mentioned in 5a3799dc, this ripples through into ScanFindVisitor as well.
---
 IniDBBuilderPackage.cc | 10 +++---
 IniDBBuilderPackage.h  |  3 +--
 ScanFindVisitor.cc |  4 ++--
 iniparse.yy|  2 +-
 4 files changed, 7 insertions(+), 12 deletions(-)

diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc
index b5b5f4c..c90bfe3 100644
--- a/IniDBBuilderPackage.cc
+++ b/IniDBBuilderPackage.cc
@@ -119,9 +119,11 @@ IniDBBuilderPackage::buildPackageLDesc (const std::string& 
theDesc)
 }
 
 void
-IniDBBuilderPackage::buildPackageInstall (const std::string& path)
+IniDBBuilderPackage::buildPackageInstall (const std::string& path,
+  const std::string& size)
 {
   process_src (*cbpv.source(), path);
+  setSourceSize (*cbpv.source(), size);
 }
 
 void
@@ -198,12 +200,6 @@ IniDBBuilderPackage::buildBeginDepends ()
 }
 
 void
-IniDBBuilderPackage::buildInstallSize (const std::string )
-{
-  setSourceSize (*cbpv.source(), size);
-}
-
-void
 IniDBBuilderPackage::buildInstallSHA512 (unsigned char const *sha512)
 {
   if (sha512 && !cbpv.source()->sha512_isSet) {
diff --git a/IniDBBuilderPackage.h b/IniDBBuilderPackage.h
index 4df1bdb..1dab41b 100644
--- a/IniDBBuilderPackage.h
+++ b/IniDBBuilderPackage.h
@@ -34,13 +34,12 @@ public:
   void buildPackageVersion (const std::string& );
   void buildPackageSDesc (const std::string& );
   void buildPackageLDesc (const std::string& );
-  void buildPackageInstall (const std::string& );
+  void buildPackageInstall (const std::string&, const std::string&);
   void buildPackageSource (const std::string&, const std::string&);
   void buildPackageTrust (int);
   void buildPackageCategory (const std::string& );
 
   void buildBeginDepends ();
-  void buildInstallSize (const std::string& );
   void buildInstallSHA512 (unsigned char const[64]);
   void buildSourceSHA512 (unsigned char const[64]);
   void buildInstallMD5 (unsigned char const[16]);
diff --git a/ScanFindVisitor.cc b/ScanFindVisitor.cc
index 02cd6e8..14c3722 100644
--- a/ScanFindVisitor.cc
+++ b/ScanFindVisitor.cc
@@ -46,8 +46,8 @@ ScanFindVisitor::visitFile(const std::string& basePath,
   if (!f.what.size())
 {
   //assume binary
-  _Builder.buildPackageInstall (basePath + theFile->cFileName);
-  _Builder.buildInstallSize(stringify(theFile->nFileSizeLow));
+  _Builder.buildPackageInstall (basePath + theFile->cFileName,
+stringify(theFile->nFileSizeLow));
 }
   else
 // patch or src, assume src until someone complains
diff --git a/iniparse.yy b/iniparse.yy
index 2480cc6..98b51bb 100644
--- a/iniparse.yy
+++ b/iniparse.yy
@@ -96,7 +96,7 @@ singleitem /* non-empty */
  | T_OTHER NL  { iniBuilder->buildPackageTrust (TRUST_OTHER); }
  | SOURCEPACKAGE source NL
  | CATEGORY categories NL
- | INSTALL STRING STRING { iniBuilder->buildPackageInstall ($2); 
iniBuilder->buildInstallSize($3);} installchksum NL
+ | INSTALL STRING STRING { iniBuilder->buildPackageInstall ($2, $3);} 
installchksum NL
  | SOURCE STRING STRING sourcechksum NL {iniBuilder->buildPackageSource ($2, 
$3);}
  | DEPENDS { iniBuilder->buildBeginDepends(); } versionedpackagelist NL
  | REQUIRES { iniBuilder->buildBeginDepends(); } versionedpackagelistsp NL
-- 
2.12.3



[PATCH setup 04/10] Make PrereqChecker::setTrust() a static method

2017-05-23 Thread Jon Turney
Rather than instantiating PrereqChecker just to call an accessor method
which changes a static data member, make that method static as well.
---
 choose.cc | 3 +--
 prereq.h  | 2 +-
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/choose.cc b/choose.cc
index 4ac5aae..1bc4c0b 100644
--- a/choose.cc
+++ b/choose.cc
@@ -373,8 +373,7 @@ ChooserPage::changeTrust(trusts aTrust)
   SetBusy ();
   chooser->defaultTrust (aTrust);
   chooser->refresh();
-  PrereqChecker p;
-  p.setTrust (aTrust);
+  PrereqChecker::setTrust (aTrust);
   ClearBusy ();
 }
 
diff --git a/prereq.h b/prereq.h
index 2aed63a..163af6e 100644
--- a/prereq.h
+++ b/prereq.h
@@ -45,7 +45,7 @@ public:
   void selectMissing ();
   
   // notes the current trust (for use in selectMissing)
-  void setTrust (trusts t) { theTrust = t; };
+  static void setTrust (trusts t) { theTrust = t; };
 
 private:
   
-- 
2.12.3



[PATCH setup 05/10] Move and rename dumpAndList()

2017-05-23 Thread Jon Turney
---
 IniDBBuilderPackage.cc |  4 ++--
 Makefile.am|  1 +
 package_depends.cc | 33 +
 package_depends.h  |  3 +++
 package_meta.cc|  2 +-
 package_version.cc | 18 --
 package_version.h  |  3 ---
 7 files changed, 40 insertions(+), 24 deletions(-)
 create mode 100644 package_depends.cc

diff --git a/IniDBBuilderPackage.cc b/IniDBBuilderPackage.cc
index 7f500d6..b5b5f4c 100644
--- a/IniDBBuilderPackage.cc
+++ b/IniDBBuilderPackage.cc
@@ -77,7 +77,7 @@ IniDBBuilderPackage::buildPackage (const std::string& name)
{
  Log (LOG_BABBLE) << "Version " << cbpv.Canonical_version() << endLog;
  Log (LOG_BABBLE) << "Depends:";
- dumpAndList (cbpv.depends(), Log (LOG_BABBLE));
+ dumpPackageDepends (cbpv.depends(), Log (LOG_BABBLE));
  Log (LOG_BABBLE) << endLog;
}
 }
@@ -191,7 +191,7 @@ IniDBBuilderPackage::buildBeginDepends ()
 #if DEBUG
   Log (LOG_BABBLE) << "Beginning of a depends statement for " << cp->name
 << endLog;
-  dumpAndList (currentNodeList, Log (LOG_BABBLE));
+  dumpPackageDepends (currentNodeList, Log (LOG_BABBLE));
 #endif
   currentSpec = NULL;
   currentNodeList = cbpv.depends();
diff --git a/Makefile.am b/Makefile.am
index af89af5..d4c8472 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -203,6 +203,7 @@ inilint_SOURCES = \
package_db.cc \
package_db.h \
package_depends.h \
+   package_depends.cc \
package_meta.cc \
package_meta.h \
package_source.cc \
diff --git a/package_depends.cc b/package_depends.cc
new file mode 100644
index 000..e288c0b
--- /dev/null
+++ b/package_depends.cc
@@ -0,0 +1,33 @@
+/*
+ * Copyright (c) 2017 Jon Turney
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * A copy of the GNU General Public License can be found at
+ * http://www.gnu.org/
+ *
+ */
+
+#include 
+#include 
+
+void
+dumpPackageDepends (PackageDepends const *currentList,
+std::ostream )
+{
+  if (currentList)
+{
+  Log (LOG_BABBLE) << "( ";
+  PackageDepends::const_iterator i = currentList->begin();
+  while (true)
+{
+  if (i == currentList->end()) break;
+  Log (LOG_BABBLE) << **i << " ";
+  ++i;
+}
+  Log (LOG_BABBLE) << ")";
+}
+}
diff --git a/package_depends.h b/package_depends.h
index 05e0dc6..af3fa01 100644
--- a/package_depends.h
+++ b/package_depends.h
@@ -14,8 +14,11 @@
 #ifndef PACKAGE_DEPENDS_H
 #define PACKAGE_DEPENDS_H
 
+#include 
 #include 
 
 typedef std::vector  PackageDepends;
 
+void dumpPackageDepends (PackageDepends const *currentList, std::ostream &);
+
 #endif // PACKAGE_DEPENDS_H
diff --git a/package_meta.cc b/package_meta.cc
index 208b948..b35b554 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -564,7 +564,7 @@ packagemeta::logAllVersions () const
{
  std::ostream & logger = Log (LOG_BABBLE);
  logger << "  depends=";
- dumpAndList(i->depends(), logger);
+ dumpPackageDepends(i->depends(), logger);
  logger << endLog;
}
   }
diff --git a/package_version.cc b/package_version.cc
index 6a903c5..f022e70 100644
--- a/package_version.cc
+++ b/package_version.cc
@@ -412,21 +412,3 @@ _packageversion::scripts()
 {
   return scripts_;
 }
-
-void
-dumpAndList (PackageDepends const *currentList,
- std::ostream )
-{
-  if (currentList)
-  {
-Log (LOG_BABBLE) << "( ";
-PackageDepends::const_iterator i = currentList->begin();
-while (true)
-{
-  if (i == currentList->end()) break;
-  Log (LOG_BABBLE) << **i << " ";
-  ++i;
-}
-Log (LOG_BABBLE) << ")";
-  }
-}
diff --git a/package_version.h b/package_version.h
index ff16eb8..d70eda0 100644
--- a/package_version.h
+++ b/package_version.h
@@ -196,7 +196,4 @@ protected:
   std::vector 

[PATCH setup 02/10] Correctly calculate total data to checksum when using IncludeSource

2017-05-23 Thread Jon Turney
Correctly account for source packages installed due to IncludeSource in the
total amount of data to checksum.

The fact that this obvious bug is unreported kind of suggests that no-one is
actually using this option...
---
 install.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/install.cc b/install.cc
index 3721047..79ddd20 100644
--- a/install.cc
+++ b/install.cc
@@ -745,7 +745,7 @@ do_install_thread (HINSTANCE h, HWND owner)
   md5sum_total_bytes += pkg.desired.source()->size;
 }
 
-if (pkg.desired.sourcePackage ().picked())
+if (pkg.desired.sourcePackage ().picked() || IncludeSource)
 {
   md5sum_total_bytes += pkg.desired.sourcePackage ().source()->size;
 }
-- 
2.12.3



[PATCH setup 03/10] Rename category "Misc" to "Orphaned"

2017-05-23 Thread Jon Turney
Packages which have no category are placed into this category.  Since a
category is mandatory for all packages in setup.ini, this effectively means
packages which are "orphaned" in the sense that they are installed, but
don't appear in any setup.ini.

Usually it's safe to uninstall such packages (with --delete-orphans), unless
you are actually using them, or have locally built packages which rely on
them.
---
 choose.cc   | 2 +-
 package_db.cc   | 2 +-
 package_meta.cc | 4 ++--
 3 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/choose.cc b/choose.cc
index 2016caa..4ac5aae 100644
--- a/choose.cc
+++ b/choose.cc
@@ -260,7 +260,7 @@ ChooserPage::OnInit ()
   bool wanted= pkg.isManuallyWanted();
   bool deleted   = pkg.isManuallyDeleted();
   bool basemisc  = (pkg.categories.find ("Base") != pkg.categories.end ()
-|| pkg.categories.find ("Misc") != pkg.categories.end ());
+|| pkg.categories.find ("Orphaned") != pkg.categories.end 
());
   bool upgrade   = wanted || (!pkg.installed && basemisc)
 || UpgradeAlsoOption || !hasManualSelections;
   bool install   = wanted  && !deleted && !pkg.installed;
diff --git a/package_db.cc b/package_db.cc
index 3978421..dbec17e 100644
--- a/package_db.cc
+++ b/package_db.cc
@@ -435,7 +435,7 @@ packagedb::defaultTrust (trusts trust)
   packagemeta & pkg = *(i->second);
   if (pkg.installed
 || pkg.categories.find ("Base") != pkg.categories.end ()
-|| pkg.categories.find ("Misc") != pkg.categories.end ())
+|| pkg.categories.find ("Orphaned") != pkg.categories.end ())
 {
   pkg.desired = pkg.trustp (true, trust);
   if (pkg.desired)
diff --git a/package_meta.cc b/package_meta.cc
index b1db191..208b948 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -464,7 +464,7 @@ packagemeta::set_action (_actions action, packageversion 
const _version)
 {
   if (installed
  || categories.find ("Base") != categories.end ()
- || categories.find ("Misc") != categories.end ())
+ || categories.find ("Orphaned") != categories.end ())
{
  desired = default_version;
  if (desired)
@@ -688,7 +688,7 @@ packagemeta::hasNoCategories() const
 void
 packagemeta::setDefaultCategories()
 {
-  add_category ("Misc");
+  add_category ("Orphaned");
 }
 
 void
-- 
2.12.3



[PATCH setup 01/10] isBinary() should return true for orphaned packages

2017-05-23 Thread Jon Turney
Test added in c23d96d6 is incorrect and results in orphaned packages being
omitted from picker.
---
 package_meta.cc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/package_meta.cc b/package_meta.cc
index cffb5b7..b1db191 100644
--- a/package_meta.cc
+++ b/package_meta.cc
@@ -546,7 +546,7 @@ packagemeta::isBinary () const
 {
   for (set::iterator i=versions.begin();
i != versions.end(); ++i)
-if ((i->Type() == package_binary) && i->accessible())
+if ((i->Type() == package_binary) && (i->accessible() || (*i == 
installed)))
   return true;
 
   return false;
-- 
2.12.3



[PATCH setup 00/10] Various setup patches

2017-05-23 Thread Jon Turney
Oh, I think I can see a bit of floor over there!

Jon Turney (10):
  isBinary() should return true for orphaned packages
  Correctly calculate total data to checksum when using IncludeSource
  Rename category "Misc" to "Orphaned"
  Make PrereqChecker::setTrust() a static method
  Move and rename dumpAndList()
  Fold IniDBBuilderPackage::buildInstallSize() into
buildPackageInstall()
  Fold build(Install|Source)(MD5|SHA512) into
buildPackage(Install|Source)
  Fix infinite recursion in grammar for depends
  Improve error recovery in setup.ini parsing
  Remove OR from grammar

 IniDBBuilderPackage.cc | 88 ++
 IniDBBuilderPackage.h  | 21 
 Makefile.am|  1 +
 ScanFindVisitor.cc |  4 +--
 choose.cc  |  5 ++-
 inilex.ll  |  3 +-
 iniparse.yy| 26 ++-
 install.cc |  2 +-
 package_db.cc  |  2 +-
 package_depends.cc | 33 +++
 package_depends.h  |  3 ++
 package_meta.cc|  8 ++---
 package_version.cc | 18 ---
 package_version.h  |  3 --
 prereq.h   |  2 +-
 15 files changed, 118 insertions(+), 101 deletions(-)
 create mode 100644 package_depends.cc

-- 
2.12.3



calm: mksetupini doesn't allow non-empty source but empty install files for an obsolete package

2017-05-23 Thread Ken Brown
I've created an obsolete package (as discussed starting at 
https://sourceware.org/ml/cygwin-apps/2017-05/msg00084.html).  But mksetupini 
doesn't like it:

mksetupini: package 'texlive-collection-htmlxml' version '20170520-1' source 
has no non-empty install tarfiles
mksetupini: package set has errors, not writing setup.ini

I thought the following would fix it:

--- a/calm/package.py
+++ b/calm/package.py
@@ -636,6 +636,9 @@ def validate_packages(args, packages):
 if packages[p].tars[packages[p].vermap[v]['source']].is_empty:
 continue

+if '_obsolete' in packages[p].vermap[v].get('category', ''):
+continue
+
 if not packages[p].tars[packages[p].vermap[v]['source']].is_used:
 logging.error("package '%s' version '%s' source has no 
non-empty install tarfiles" % (p, v))
 error = True


But I must have something wrong, because I still get the error after this 
change.

Ken