Thanks again, Sebb. Checking signatures is so much faster now! Craig
> On Feb 15, 2019, at 3:40 PM, Craig Russell <[email protected]> wrote: > > Hi Sebb, > > Thanks so much for taking care of this. I was really stuck and your help is > invaluable. > > Craig > >> On Feb 15, 2019, at 2:36 AM, sebb <[email protected] >> <mailto:[email protected]>> wrote: >> >> The updated keyserver seems to have helped. >> >> I added code to log the keyserver debug output in the server logs (it >> appears in error.log) for later review if necessary. >> >> On Fri, 15 Feb 2019 at 09:54, sebb <[email protected] >> <mailto:[email protected]>> wrote: >> It looks like Whimsy uses gpg2 (2.1.1) rather than gpg. >> The gpg2 man page says: >> >> verbose >> This option has no more function since GnuPG 2.1. Use >> the dirmngr configuration options instead. >> >> debug This option has no more function since GnuPG 2.1. Use >> the dirmngr configuration options instead. >> >> >> However I don't yet know how to pass options to dirmngr. >> >> I noticed that pgpkeys.mit.edu <http://pgpkeys.mit.edu/> is very slow to >> respond, so I tried hkps.pool.sks-keyservers.net >> <http://hkps.pool.sks-keyservers.net/> >> instead. That was much quicker, so I think it is worth trying it. >> >> On Fri, 15 Feb 2019 at 03:49, Craig Russell <[email protected] >> <mailto:[email protected]>> wrote: >> Looks like the version of gpg that workbench uses does not support >> --keyserver-options debug. >> >> Any ideas how to get more information from the failure? >> >> The man page on whimsy-vm4 would indicate that debug is a suitable >> keyserver-option. >> >> Any ideas: >> >> gpg: Signature made Fri Feb 15 01:02:20 2019 UTC using RSA key ID 7B061396 >> gpg: Can't check signature: No public key gpg: keyserver option 'verbose' is >> unknown gpg: keyserver receive failed: No data >> >>> On Feb 13, 2019, at 2:56 AM, sebb <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> BTW, to distinguish stdout and stderr one can redirect them to different >>> files: >>> >>> $ command 2>stderr.out >stdout.out >>> >>> [Some IDEs colour them differently] >>> >>> On Tue, 12 Feb 2019 at 23:55, Craig Russell <[email protected] >>> <mailto:[email protected]>> wrote: >>> Tldr; the debug output starting with "< HTTP/1.1 " >>> Seems to be the most useful. Here is some of the debug output: >>> < HTTP/1.1 502 Proxy Error >>> < HTTP/1.1 200 OK >>> < HTTP/1.1 404 Not found >>> < HTTP/1.1 503 Service Temporarily Unavailable >>> >>> >>> The 200 is a good key retrieval. The 400 is definitely key not found. The >>> 502 and 503 can be retried. >>> >>> Again, I'm keen to try adding "debug" to the --recv-keys command and see >>> what happens. In parallel, if someone can figure out how to extract just >>> the HTTP/ part of the debug output, we can programmatically distinguish >>> between the key not found and the temporary errors. >>> >>> Craig >>> >>>> On Feb 12, 2019, at 3:44 PM, Craig Russell <[email protected] >>>> <mailto:[email protected]>> wrote: >>>> >>>> I just read the code again and I'll copy it here: >>>> >>>> # run gpg verify command >>>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>>> attachment.path >>>> >>>> # if key is not found, fetch and try again >>>> if >>>> err.include? "gpg: Can't check signature: No public key" or >>>> err.include? "gpg: Can't check signature: public key not found" >>>> then >>>> # extract and fetch key >>>> keyid = err[/[RD]SA key (ID )?(\w+)/,2].untaint >>>> >>>> out2, err2, rc2 = Open3.capture3 gpg, '--keyserver', 'pgpkeys.mit.edu >>>> <http://pgpkeys.mit.edu/>', >>>> '--recv-keys', keyid >>>> >>>> # run gpg verify command again >>>> out, err, rc = Open3.capture3 gpg, '--verify', signature.path, >>>> attachment.path >>>> >>>> # if verify failed, concatenate fetch output >>>> if rc.exitstatus != 0 >>>> out += out2 >>>> err += err2 >>>> end >>>> end >>>> >>>> >>>>> On Feb 11, 2019, at 12:25 PM, Craig Russell <[email protected] >>>>> <mailto:[email protected]>> wrote: >>>>> >>>>> Hi Shane, >>>>> >>>>>> On Feb 9, 2019, at 5:07 AM, Shane Curcuru <[email protected] >>>>>> <mailto:[email protected]>> wrote: >>>>>> >>>>>> Can;'t type enough for code yet, but miss coding, so... ideas: >>>>>> >>>>>> When verifying zig, you want workbench to: >>>>>> >>>>>> - IF we get here: (i.e. there was an error gettign key) >>>>>> https://github.com/apache/whimsy/blob/master/www/secretary/workbench/views/actions/check-signature.json.rb#L40 >>>>>> >>>>>> <https://github.com/apache/whimsy/blob/master/www/secretary/workbench/views/actions/check-signature.json.rb#L40> >>>>> >>>>> First, the entire error message should be displayed instead of just "no >>>>> data". >>>> >>>> Actually, if we get here, there was an error verifying the signature, >>>> *not* getting the key. Getting the key is done after the #extract and >>>> fetch key line, using the command >>>> gpg, '--keyserver', 'pgpkeys.mit.edu <http://pgpkeys.mit.edu/>', >>>> >>>> '--recv-keys', keyid >>>> The code then tries to verify the signature again, and if it fails, >>>> concatenates the failure to obtain the key with the failure to verify. >>>> >>>> I've never seen anything except "no key data" so I believe that the real >>>> problem here is that the gpg --recv-keys command just doesn't give much >>>> data if it doesn't successfully receive the key. >>>> >>>> So perhaps all we really need to do here is to give more options on the >>>> recv-keys command. Perhaps using --keyserver-options would help, with >>>> options of "verbose" or "debug". >>>> >>>> Using verbose does not seem to help: >>>> gpg --keyserver pgpkeys.mit.edu <http://pgpkeys.mit.edu/> >>>> --keyserver-options verbose --recv-keys 91CDDE15 >>>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>>> <http://pgpkeys.mit.edu/> >>>> gpgkeys: key 91CDDE15 can't be retrieved >>>> gpg: no valid OpenPGP data found. >>>> gpg: Total number processed: 0 >>>> gpg: keyserver communications error: keyserver helper general error >>>> gpg: keyserver communications error: Invalid public key algorithm >>>> gpg: keyserver receive failed: Invalid public key algorithm >>>> >>>> Using debug gives more information: >>>> >>>> bash-3.2$ gpg --keyserver pgpkeys.mit.edu <http://pgpkeys.mit.edu/> >>>> --keyserver-options debug --recv-keys 91CDDE15 >>>> gpg: requesting key 91CDDE15 from hkp server pgpkeys.mit.edu >>>> <http://pgpkeys.mit.edu/> >>>> gpgkeys: curl version = libcurl/7.54.0 LibreSSL/2.0.20 zlib/1.2.11 >>>> nghttp2/1.24.0 >>>> * Trying 18.9.60.141... >>>> * TCP_NODELAY set >>>> * Connected to pgpkeys.mit.edu <http://pgpkeys.mit.edu/> (18.9.60.141) >>>> port 11371 (#0) >>>> > GET /pks/lookup?op=get&options=mr&search=0x91CDDE15 HTTP/1.1 >>>> Host: pgpkeys.mit.edu:11371 <http://pgpkeys.mit.edu:11371/> >>>> Accept: */* >>>> Pragma: no-cache >>>> Cache-Control: no-cache >>>> >>>> < HTTP/1.1 503 Service Temporarily Unavailable >>>> < Date: Tue, 12 Feb 2019 23:31:23 GMT >>>> < Content-Length: 323 >>>> < Connection: close >>>> < Content-Type: text/html; charset=iso-8859-1 >>>> < >>>> * Closing connection 0 >>>> gpgkeys: key 91CDDE15 can't be retrieved >>>> gpg: no valid OpenPGP data found. >>>> gpg: Total number processed: 0 >>>> gpg: keyserver communications error: keyserver helper general error >>>> gpg: keyserver communications error: Invalid public key algorithm >>>> gpg: keyserver receive failed: Invalid public key algorithm >>>> >>>> I'm not clear from the console output here which output is part of the >>>> stdout and which is part of errout. But I'd be willing to just try adding >>>> debug to the command and see what happens. >>>> >>>> WDYT? >>>> >>>> Craig >>>> >>>>>> >>>>>> - And if >>>>>> err.include? "gpg: Can't check signature: No public key" or >>>>>> err.include? "gpg: Can't check signature: public key not found" >>>>>> >>>>>> - Then Do: enable the action that sends pubkey.erb (so you can push the >>>>>> button - or do you want it to automatically send that upload email)? >>>>> >>>>> The action to send pub key.erb is always enabled. But the message should >>>>> have the command that failed, e.g. >>>>> http://pgpkeys.mit.edu/pks/lookup?search=91CDDE11&op=index >>>>> <http://pgpkeys.mit.edu/pks/lookup?search=91CDDE11&op=index> >>>>> >>>>> That way, the sender has some idea of which key was being requested. >>>>>> >>>>>> - ELSE: (other error like proxy/timeout) >>>>>> - Disable the pubkey.erb >>>>> >>>>> No need. >>>>> >>>>>> - Allow the Secretary to re-process the check sig action. >>>>>> >>>>>> Do you want it to simply give you the button to immediately re-chceck >>>>>> for the key, >>>>> >>>>> That would be a nice option. Just clicking on the .pdf file doesn't >>>>> automatically try to download the public key from what I can tell. >>>>> >>>>>> or do you want to leave this whole msg and come back later? >>>>> >>>>> Just having the entire error message displayed with no further >>>>> editing/analysis is fine. From the message, secretary can decide whether >>>>> to send the "upload pub key" message or not. And in fact, sometimes it >>>>> takes a couple of upload messages before the sender gets around to >>>>> uploading the key. And having the command as part of the message might >>>>> allow the sender to just click on the link and see if they are successful >>>>> accessing their own public key. >>>>> >>>>> Craig >>>>>> >>>>>> -- >>>>>> >>>>>> - Shane >>>>>> Director & Member >>>>>> The Apache Software Foundation >>>>>> >>>>>> Craig Russell wrote on 2/8/19 8:18 PM: >>>>>>> Hi, >>>>>>> >>>>>>> When a document has an associated .asc file, whimsy automatically tries >>>>>>> to download the public key in order to verify the signature. >>>>>>> >>>>>>> But the key server is often busy and just cannot service the request >>>>>>> timely. >>>>>>> >>>>>>> The current behavior of whimsy does not distinguish among at least >>>>>>> three cases: >>>>>>> >>>>>>> No key found >>>>>>> Timeout >>>>>>> Proxy error >>>>>>> >>>>>>> No key found is the only case where the secretary should send the >>>>>>> "upload public key" message to the submitter. >>>>>>> >>>>>>> In the other two cases, the secretary should retry (later) until a >>>>>>> definitive No key found error is received or the public key is >>>>>>> downloaded. >>>>>>> >>>>>>> Can the actual error be extracted from the message to avoid confusion? >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> Craig >>>>>>> >>>>>>> Craig L Russell >>>>>>> Secretary, Apache Software Foundation >>>>>>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>>>>>> <http://db.apache.org/jdo> >>>>>>> >>>>>> >>>>> >>>>> Craig L Russell >>>>> Secretary, Apache Software Foundation >>>>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>>>> <http://db.apache.org/jdo> >>>> Craig L Russell >>>> Secretary, Apache Software Foundation >>>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>>> <http://db.apache.org/jdo> >>> Craig L Russell >>> Secretary, Apache Software Foundation >>> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >>> <http://db.apache.org/jdo> >> >> Craig L Russell >> Secretary, Apache Software Foundation >> [email protected] <mailto:[email protected]> http://db.apache.org/jdo >> <http://db.apache.org/jdo> > > Craig L Russell > Secretary, Apache Software Foundation > [email protected] <mailto:[email protected]> http://db.apache.org/jdo > <http://db.apache.org/jdo> Craig L Russell Secretary, Apache Software Foundation [email protected] <mailto:[email protected]> http://db.apache.org/jdo <http://db.apache.org/jdo>
