Hi Sebb, Thanks so much for taking care of this. I was really stuck and your help is invaluable.
Craig > On Feb 15, 2019, at 2:36 AM, sebb <[email protected]> wrote: > > The updated keyserver seems to have helped. > > I added code to log the keyserver debug output in the server logs (it appears > in error.log) for later review if necessary. > > On Fri, 15 Feb 2019 at 09:54, sebb <[email protected] > <mailto:[email protected]>> wrote: > It looks like Whimsy uses gpg2 (2.1.1) rather than gpg. > The gpg2 man page says: > > verbose > This option has no more function since GnuPG 2.1. Use > the dirmngr configuration options instead. > > debug This option has no more function since GnuPG 2.1. Use > the dirmngr configuration options instead. > > > However I don't yet know how to pass options to dirmngr. > > I noticed that pgpkeys.mit.edu <http://pgpkeys.mit.edu/> is very slow to > respond, so I tried hkps.pool.sks-keyservers.net > <http://hkps.pool.sks-keyservers.net/> > instead. That was much quicker, so I think it is worth trying it. > > On Fri, 15 Feb 2019 at 03:49, Craig Russell <[email protected] > <mailto:[email protected]>> wrote: > Looks like the version of gpg that workbench uses does not support > --keyserver-options debug. > > Any ideas how to get more information from the failure? > > The man page on whimsy-vm4 would indicate that debug is a suitable > keyserver-option. > > Any ideas: > > gpg: Signature made Fri Feb 15 01:02:20 2019 UTC using RSA key ID 7B061396 > gpg: Can't check signature: No public key gpg: keyserver option 'verbose' is > unknown gpg: keyserver receive failed: No data > >> On Feb 13, 2019, at 2:56 AM, sebb <[email protected] >> <mailto:[email protected]>> wrote: >> >> BTW, to distinguish stdout and stderr one can redirect them to different >> files: >> >> $ command 2>stderr.out >stdout.out >> >> [Some IDEs colour them differently] >> >> On Tue, 12 Feb 2019 at 23:55, Craig Russell <[email protected] >> <mailto:[email protected]>> wrote: >> Tldr; the debug output starting with "< HTTP/1.1 " >> Seems to be the most useful. Here is some of the debug output: >> < HTTP/1.1 502 Proxy Error >> < HTTP/1.1 200 OK >> < HTTP/1.1 404 Not found >> < HTTP/1.1 503 Service Temporarily Unavailable >> >> >> The 200 is a good key retrieval. The 400 is definitely key not found. The >> 502 and 503 can be retried. >> >> Again, I'm keen to try adding "debug" to the --recv-keys command and see >> what happens. In parallel, if someone can figure out how to extract just the >> HTTP/ part of the debug output, we can programmatically distinguish between >> the key not found and the temporary errors. >> >> Craig >> >>> On Feb 12, 2019, at 3:44 PM, Craig Russell <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> I just read the code again and I'll copy it here: >>> >>> # run gpg verify command >>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>> attachment.path >>> >>> # if key is not found, fetch and try again >>> if >>> err.include? "gpg: Can't check signature: No public key" or >>> err.include? "gpg: Can't check signature: public key not found" >>> then >>> # extract and fetch key >>> keyid = err[/[RD]SA key (ID )?(\w+)/,2].untaint >>> >>> out2, err2, rc2 = Open3.capture3 gpg, '--keyserver', 'pgpkeys.mit.edu >>> <http://pgpkeys.mit.edu/>', >>> '--recv-keys', keyid >>> >>> # run gpg verify command again >>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>> attachment.path >>> >>> # if verify failed, concatenate fetch output >>> if rc.exitstatus != 0 >>> out += out2 >>> err += err2 >>> end >>> end >>> >>> >>>> On Feb 11, 2019, at 12:25 PM, Craig Russell <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> Hi Shane, >>>> >>>>> On Feb 9, 2019, at 5:07 AM, Shane Curcuru <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Can;'t type enough for code yet, but miss coding, so... ideas: >>>>> >>>>> When verifying zig, you want workbench to: >>>>> >>>>> - IF we get here: (i.e. there was an error gettign key) >>>>> https://github.com/apache/whimsy/blob/master/www/secretary/workbench/views/actions/check-signature.json.rb#L40 >>>>> >>>>> <https://github.com/apache/whimsy/blob/master/www/secretary/workbench/views/actions/check-signature.json.rb#L40> >>>> >>>> First, the entire error message should be displayed instead of just "no >>>> data". >>> >>> Actually, if we get here, there was an error verifying the signature, *not* >>> getting the key. Getting the key is done after the #extract and fetch key >>> line, using the command >>> gpg, '--keyserver', 'pgpkeys.mit.edu <http://pgpkeys.mit.edu/>', >>> >>> '--recv-keys', keyid >>> The code then tries to verify the signature again, and if it fails, >>> concatenates the failure to obtain the key with the failure to verify. >>> >>> I've never seen anything except "no key data" so I believe that the real >>> problem here is that the gpg --recv-keys command just doesn't give much >>> data if it doesn't successfully receive the key. >>> >>> So perhaps all we really need to do here is to give more options on the >>> recv-keys command. Perhaps using --keyserver-options would help, with >>> options of "verbose" or "debug". >>> >>> Using verbose does not seem to help: >>> gpg --keyserver pgpkeys.mit.edu <http://pgpkeys.mit.edu/> >>> --keyserver-options verbose --recv-keys 91CDDE15 >>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>> <http://pgpkeys.mit.edu/> >>> gpgkeys: key 91CDDE15 can't be retrieved >>> gpg: no valid OpenPGP data found. >>> gpg: Total number processed: 0 >>> gpg: keyserver communications error: keyserver helper general error >>> gpg: keyserver communications error: Invalid public key algorithm >>> gpg: keyserver receive failed: Invalid public key algorithm >>> >>> Using debug gives more information: >>> >>> bash-3.2$ gpg --keyserver pgpkeys.mit.edu <http://pgpkeys.mit.edu/> >>> --keyserver-options debug --recv-keys 91CDDE15 >>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>> <http://pgpkeys.mit.edu/> >>> gpgkeys: curl version = libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 >>> nghttp2/1.24.0 >>> * Trying 18.9.60.141... >>> * TCP_NODELAY set >>> * Connected to pgpkeys.mit.edu <http://pgpkeys.mit.edu/> (18.9.60.141) port >>> 11371 (#0) >>> > GET /pks/lookup?op=get&options=mr&search=0x91CDDE15 HTTP/1.1 >>> Host: pgpkeys.mit.edu:11371 <http://pgpkeys.mit.edu:11371/> >>> Accept: */* >>> Pragma: no-cache >>> Cache-Control: no-cache >>> >>> < HTTP/1.1 503 Service Temporarily Unavailable >>> < Date: Tue, 12 Feb 2019 23:31:23 GMT >>> < Content-Length: 323 >>> < Connection: close >>> < Content-Type: text/html; charset=iso-8859-1 >>> < >>> * Closing connection 0 >>> gpgkeys: key 91CDDE15 can't be retrieved >>> gpg: no valid OpenPGP data found. >>> gpg: Total number processed: 0 >>> gpg: keyserver communications error: keyserver helper general error >>> gpg: keyserver communications error: Invalid public key algorithm >>> gpg: keyserver receive failed: Invalid public key algorithm >>> >>> I'm not clear from the console output here which output is part of the >>> stdout and which is part of errout. But I'd be willing to just try adding >>> debug to the command and see what happens. >>> >>> WDYT? >>> >>> Craig >>> >>>>> >>>>> - And if >>>>> err.include? "gpg: Can't check signature: No public key" or >>>>> err.include? "gpg: Can't check signature: public key not found" >>>>> >>>>> - Then Do: enable the action that sends pubkey.erb (so you can push the >>>>> button - or do you want it to automatically send that upload email)? >>>> >>>> The action to send pub key.erb is always enabled. But the message should >>>> have the command that failed, e.g. >>>> http://pgpkeys.mit.edu/pks/lookup?search=91CDDE11&op=index >>>> <http://pgpkeys.mit.edu/pks/lookup?search=91CDDE11&op=index> >>>> >>>> That way, the sender has some idea of which key was being requested. >>>>> >>>>> - ELSE: (other error like proxy/timeout) >>>>> - Disable the pubkey.erb >>>> >>>> No need. >>>> >>>>> - Allow the Secretary to re-process the check sig action. >>>>> >>>>> Do you want it to simply give you the button to immediately re-chceck >>>>> for the key, >>>> >>>> That would be a nice option. Just clicking on the .pdf file doesn't >>>> automatically try to download the public key from what I can tell. >>>> >>>>> or do you want to leave this whole msg and come back later? >>>> >>>> Just having the entire error message displayed with no further >>>> editing/analysis is fine. From the message, secretary can decide whether >>>> to send the "upload pub key" message or not. And in fact, sometimes it >>>> takes a couple of upload messages before the sender gets around to >>>> uploading the key. And having the command as part of the message might >>>> allow the sender to just click on the link and see if they are successful >>>> accessing their own public key. >>>> >>>> Craig >>>>> >>>>> -- >>>>> >>>>> - Shane >>>>> Director & Member >>>>> The Apache Software Foundation >>>>> >>>>> Craig Russell wrote on 2/8/19 8:18 PM: >>>>>> Hi, >>>>>> >>>>>> When a document has an associated .asc file, whimsy automatically tries >>>>>> to download the public key in order to verify the signature. >>>>>> >>>>>> But the key server is often busy and just cannot service the request >>>>>> timely. >>>>>> >>>>>> The current behavior of whimsy does not distinguish among at least three >>>>>> cases: >>>>>> >>>>>> No key found >>>>>> Timeout >>>>>> Proxy error >>>>>> >>>>>> No key found is the only case where the secretary should send the >>>>>> "upload public key" message to the submitter. >>>>>> >>>>>> In the other two cases, the secretary should retry (later) until a >>>>>> definitive No key found error is received or the public key is >>>>>> downloaded. >>>>>> >>>>>> Can the actual error be extracted from the message to avoid confusion? >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Craig >>>>>> >>>>>> Craig L Russell >>>>>> Secretary, Apache Software Foundation >>>>>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>>>>> <http://db.apache.org/jdo> >>>>>> >>>>> >>>> >>>> Craig L Russell >>>> Secretary, Apache Software Foundation >>>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>>> <http://db.apache.org/jdo> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>> <http://db.apache.org/jdo> >> Craig L Russell >> Secretary, Apache Software Foundation >> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >> <http://db.apache.org/jdo> > > Craig L Russell > Secretary, Apache Software Foundation > [email protected] <mailto:[email protected]> http://db.apache.org/jdo > <http://db.apache.org/jdo> Craig L Russell Secretary, Apache Software Foundation [email protected] <mailto:[email protected]> http://db.apache.org/jdo <http://db.apache.org/jdo>
