gpg-agent password prompt hangs since r1794166+r1795087

[Daniel Shahaf]

When I run:
.
% ./subversion/svn/svn mkdir -qmm https://dist.apache.org/repos/dist/test
.
it hangs with no output, until I «pkill pinentry» from another terminal,
at which point I get the built-in password prompt.

I bisected this to the gpg-agent changes in r1794166+r1795087 [1]: in r1794165
I get a password prompt immediately, whereas in r1794166 + 'svn merge -c 
r1795087'
I get the hang.

My system has gpg-agent version 2.1.18-8~deb9u2 (OS package, Debian stretch).

Haven't looked beyond this.

Cheers,

Daniel 

[1] 
> 
> r1794166 | jamessan | 2017-05-06 17:36:16 + (Sat, 06 May 2017) | 13 lines
> Changed paths:
>M /subversion/trunk/subversion/libsvn_subr/gpg_agent.c
> 
> Find gpg-agent socket using gpgconf, if possible.
> 
> This allows detection of socket with Gnupg >= 2.1.13, which changed the 
> default
> socket path to (/var)/run/user/UID/gnupg
> 
> * subversion/libsvn_subr/gpg_agent.c
>   (find_gpgconf_agent_socket): new function to find gpg-agent socket using
>gpgconf
>   (find_running_gpg_agent): use find_gpgconf_agent_socket to detect socket 
> when
>possible.
> 
> Patch by: Lukas Jirkovsky 
> 
> 
> r1795087 | jamessan | 2017-05-14 05:33:51 + (Sun, 14 May 2017) | 15 lines
> Changed paths:
>M /subversion/trunk/subversion/libsvn_subr/gpg_agent.c
> 
> Search for gpg-agent sockets in (/var)/run.
> 
> This reverts executing gpgconf to find the socket, as implemented in r1794166.
> Instead, follow mostly the same procedure for socket lookup as GPG.  GPG's
> hashing of non-default $GNUPGHOME to use as a portion of the path under
> (/var)/run is not (yet) done, since it relies on implementing z-base-32
> encoding.
> 
> * subversion/libsvn_subr/gpg_agent.c
>   (find_gpgconf_agent_socket): Removed function which ran gpgconf ...
>   (find_gpg_agent_socket): ... and added one that performs the various checks
>to determine where the user's gpg-agent socket resides.
>   (find_running_gpg_agent): Replace the socket lookup with a call to
>find_gpg_agent_socket.
> 
> 

Re: Ruby bindings failure, perhaps related to __?

[Julian Foad]

Branko Čibej wrote:
> On 27.09.2018 19:02, Julian Foad wrote:
> > [...] maybe it would be better to just not create the "XShelf" short name 
> > [...]
> >
> > But also I want to remember that the priority right now is to get 1.11 out 
> > with something that works, first, and then improve it later.
> 
> If we keep the XName form in, we have to document that these classes are
> experimental and all the usual disclaimers apply.

Now I've changed it to not create a short name like "XShelf" at all. A long 
name like Svn::Ext::Client::Svn_client__shelf_t is still available.

I think that will be sufficient for 1.11 now, won't it?

Can someone review the updated backport nomination please?

-- 
- Julian

Re: SVN binaries: build linking process

[Doug Robinson]

Julian:

Just one quick note:

On Fri, Sep 28, 2018 at 10:42 AM Julian Foad  wrote:

> >> I used to have such a tool in a past environment.  It was WONDERFUL.  I
> haven't seen hide nor hair of one for Linux... that would be ideal.
>
> >See the most-voted answer here:
> https://stackoverflow.com/questions/13769141/can-i-change-rpath-in-an-already-compiled-binary
> -- "patchelf"https://nixos.org/patchelf.html is available in the >Ubuntu
> 18.04 repositories at least.
>

Nice!  I searched for that a while back and didn't find it.  I'll give it
some fun-time!


>  The background is that we have a customer who wants to be able to
> relocate our stuff and one of the first things we'd need to do is to either
> rebuild with a specific path just for them, or rebuild with just the
> leafnames so taht LD_LIBRARY_PATH will work.
> >>>
> >>> The leafnames, plus perhaps some appropriate "rpath"-y configuration
> if you want a default location, sounds like the right way to go, unless
> there's already a tool that can modify .
> >>
> >> It's only the "right way to go" if the community does not have reason
> not to do so.  That's really why I asked.  And that's the key question.
>

> >Anyone?
>

That's really the question I'm after: I'd rather NOT go down a path that's
known to be "problematic"!

Cheers.

Doug
-- 
*DOUGLAS B ROBINSON* SENIOR PRODUCT MANAGER

T +1 925 396 1125
*E* doug.robin...@wandisco.com

-- 


* *

**The LIVE DATA Company
*Find out more 
*wandisco.com *



 

*


THIS MESSAGE 
AND ANY ATTACHMENTS ARE CONFIDENTIAL, PROPRIETARY AND MAY BE PRIVILEGED

If 
this message was misdirected, WANdisco, Inc. and its subsidiaries, 
("WANdisco") does not waive any confidentiality or privilege. If you are 
not the intended recipient, please notify us immediately and destroy the 
message without disclosing its contents to anyone. Any distribution, use or 
copying of this email or the information it contains by other than an 
intended recipient is unauthorized. The views and opinions expressed in 
this email message are the author's own and may not reflect the views and 
opinions of WANdisco, unless the author is authorized by WANdisco to 
express such views or opinions on its behalf. All email sent to or from 
this address is subject to electronic storage and review by WANdisco. 
Although WANdisco operates anti-virus programs, it does not accept 
responsibility for any damage whatsoever caused by viruses being passed.

Re: svn commit: r1838813 - in /subversion/trunk/subversion: libsvn_fs_fs/recovery.c tests/cmdline/svnadmin_tests.py

[Julian Foad]

Daniel Shahaf wrote:
> julianf...@apache.org wrote on Fri, 24 Aug 2018 10:48 +:
> > Author: julianfoad
> > Date: Fri Aug 24 10:48:37 2018
> > New Revision: 1838813
> > 
> > URL: http://svn.apache.org/viewvc?rev=1838813=rev
> > Log:
> > Let 'svnadmin recover' prune the rep-cache even if it is disabled.
> > 
> > Part of issue #4077: "FSFS recover should prune unborn revisions from
> > rep-cache.db".
> > 
> > Also add tests for both cases (enabled, disabled).
> > 
> > Pruning the rep-cache even if disabled was included as r1213716 in the
> > original fix for issue #4077, first released in Subversion 1.7.3, but was
> > reverted in r1367674 (issue #4214, recovery should not create rep-cache.db)
> > and so omitted from Subversion 1.8 and later series of releases.
> > 
> > * subversion/libsvn_fs_fs/recovery.c
> >   (recover_body): Prune the rep-cache no matter whether it's in use.
> > 
> > * subversion/tests/cmdline/svnadmin_tests.py
> >   (read_rep_cache,
> >check_recover_prunes_rep_cache): New functions.
> >   (recover_prunes_rep_cache_when_enabled,
> >recover_prunes_rep_cache_when_disabled): New tests.
> >   (test_list): Run them.
> > 
> > +++ subversion/trunk/subversion/tests/cmdline/svnadmin_tests.py
> > @@ -53,6 +53,23 @@ Wimp = svntest.testcase.Wimp_deco
> >  SkipDumpLoadCrossCheck = svntest.testcase.SkipDumpLoadCrossCheck_deco
> >  Item = svntest.wc.StateItem
> >  
> > +def read_rep_cache(repo_dir):
> > +  """Return the rep-cache contents as a dict {hash: (rev, index, ...)}.
> > +  """
> > +  db_path = os.path.join(repo_dir, 'db', 'rep-cache.db')
> 
> Should this function assert that db/format <= 8?  When we introduce
> format 9 it might not be compatible with this function's expectations.

I suppose that would be a way to enforce that we revisit this logic then. On 
the other hand we don't generally do that for anything else (FSFS formats, WC 
formats, etc.).

No strong opinion.

> > +  db1 = svntest.sqlite3.connect(db_path)
> > +  schema1 = db1.execute("pragma user_version").fetchone()[0]
> > +  # Can't test newer rep-cache schemas with an old built-in SQLite.
> > +  if schema1 >= 2 and svntest.sqlite3.sqlite_version_info < (3, 8, 2):
> > +raise svntest.Failure("Can't read rep-cache schema %d using old "
> > +  "Python-SQLite version %s < (3,8,2)" %
> > +   (schema1,
> > +svntest.sqlite3.sqlite_version_info))
> > +
> > +  content = { row[0]: row[1:] for row in
> > +  db1.execute("select * from rep_cache") }
> > +  return content

-- 
- Julian

Subversion 1.11.0-rc2 up for testing/signing

[Julian Foad]

The 1.11.0-rc2 release artifacts are now available for testing/signing.
Please get the tarballs from
  https://dist.apache.org/repos/dist/dev/subversion
and add your signatures there.

Thanks!

-- 
- Julian

please review outstanding backports for 1.10.x

[Stefan Sperling]

Julian and myself would like to start preparing another 1.10.x
release over the coming week, to be relased close to 1.11.0.

There are 9 pending nominations in 1.10.x/STATUS currently, and 4 bug
fixes have already been merged to the 1.10.x branch since 1.10.2.

All these fixes would provide good substance for a 1.10.3 release.

Thanks!

Re: SVN binaries: build linking process

[Julian Foad]

Can anyone shed more light on Doug's question below. Basically, svn binaries 
being installed such that they find their libs at a fixed location that cannot 
be overridden by LD_LIBRARY_PATH; is there a particular reason why this is so?

Doug Robinson wrote:
> Julian Foad wrote:
>> Doug Robinson wrote:
>>> Looking at our build process it appears that we go out of our way to link 
>>> such that the Subversion binaries (e.g. "svn") are all bound with full 
>>> paths, for example:
>>> 
>>> # ldd /usr/bin/svnadmin
>>> libsvn_repos-1.so.0 => /usr/lib64/libsvn_repos-1.so.0
>> 
>> Hi, Doug.
>> 
>> First I wrote the rest of this reply below.
>> 
>> Then something made me look twice at 'ldd' and it appears to me that the 
>> interpretation may be wrong.
>> 
>> I see the same sort of result as you for "ldd /usr/bin/svnadmin" but 
>> "objdump" shows:
>> [[[
>> $ objdump -p /usr/bin/svnadmin | grep -C2 NEEDED
>> 
>> Dynamic Section:
>>   NEEDED   libsvn_repos-1.so.1
>>   NEEDED   libsvn_fs-1.so.1
>>   NEEDED   libsvn_delta-1.so.1
>>   NEEDED   libsvn_subr-1.so.1
>>   NEEDED   libapr-1.so.0
>>   NEEDED   libpthread.so.0
>>   NEEDED   libc.so.6
>>   RPATH    /usr/lib
>>   INIT 0x00402ab8
>> 
>> ]]]
> 
> Not sure what to make of that.  Wish it were so.  Empirically, the inability 
> to use LD_LIBRARY_PATH along with the output of ldd seems conclusive.
>  
>> I'll leave the rest of the reply below in case it is of any use.
>>> Since those paths are absolute, the LD_LIBRARY_PATH environment variable 
>>> won't help if the libraries are relocated:  I found I needed to specify 
>>> them all with LD_PRELOAD as noted on NV-6950.  And looking at the standard 
>>> SVN provided by some distros (e.g. SUSE 12), they've built them the same 
>>> way.
>>> 
>>> My question is if you know the background as to this decision?  Or was it 
>>> even a decision?
>> I'm sure there's nothing specific to Subversion behind this. AFAIK that's 
>> just a typical way of installing dynamically linked executables.

On further investigation -- e.g. these were helpful: 
http://blog.tremily.us/posts/rpath/ and https://wiki.debian.org/RpathIssue -- 
it seems "RPATH" in the executable is searched before $LD_LIBRARY_PATH and so 
any libs found in RPATH take precedence, whereas if one sets "RUNPATH" in the 
executable instead then $LD_LIBRARY_PATH overrides that.

Of course different package distros can do things different ways, but should we 
be doing or enabling something different?

> Ok, so there's no historical thought process that requires it?  Might be good 
> to check with the other committers?

Anyone want to comment on that perspective?

>> The paths are written in at the 'install' phase, not any earlier in the 
>> build process. Variations are possible -- notably writing a lib search path 
>> called "rpath" into the executable which can be overridden at run time by 
>> LD_LIBRARY_PATH, or something like that. Fairly recently for some reason or 
>> other I read a bit about it and found there were a few variations on the 
>> theme, and it wasn't as simple as that and I don't have the complete scheme 
>> clear in my head.
> 
> Never is "simple".  I've done explicit things with linking and paths on other 
> (non-Linux) toolsets.  So I "get it".  And the docs are very spotty on Linux 
> as I've been doing research.
> And then there's the fact that the LD_LIBRARY_PATH is explicitly disabled 
> when in the context of a set-id program.  So dynamically finding your shared 
> library seems doomed in the most general case.
>  
>> I half recall it may be possible to update the library path information in 
>> an already-installed executable. I think this was definitely possible for 
>> some kind of "rpath" included in the executable; not sure if it's possible 
>> to update the paths to individual .so files.
> 
> I used to have such a tool in a past environment.  It was WONDERFUL.  I 
> haven't seen hide nor hair of one for Linux... that would be ideal.

See the most-voted answer here: 
https://stackoverflow.com/questions/13769141/can-i-change-rpath-in-an-already-compiled-binary
 -- "patchelf"https://nixos.org/patchelf.html is available in the Ubuntu 18.04 
repositories at least.

>>> The background is that we have a customer who wants to be able to relocate 
>>> our stuff and one of the first things we'd need to do is to either rebuild 
>>> with a specific path just for them, or rebuild with just the leafnames so 
>>> taht LD_LIBRARY_PATH will work.
>> 
>> The leafnames, plus perhaps some appropriate "rpath"-y configuration if you 
>> want a default location, sounds like the right way to go, unless there's 
>> already a tool that can modify .
> 
> It's only the "right way to go" if the community does not have reason not to 
> do so.  That's really why I asked.  And that's the key question.

Anyone?

- Julian

Re: svn commit: r1838813 - in /subversion/trunk/subversion: libsvn_fs_fs/recovery.c tests/cmdline/svnadmin_tests.py

[Daniel Shahaf]

julianf...@apache.org wrote on Fri, 24 Aug 2018 10:48 +:
> Author: julianfoad
> Date: Fri Aug 24 10:48:37 2018
> New Revision: 1838813
> 
> URL: http://svn.apache.org/viewvc?rev=1838813=rev
> Log:
> Let 'svnadmin recover' prune the rep-cache even if it is disabled.
> 
> Part of issue #4077: "FSFS recover should prune unborn revisions from
> rep-cache.db".
> 
> Also add tests for both cases (enabled, disabled).
> 
> Pruning the rep-cache even if disabled was included as r1213716 in the
> original fix for issue #4077, first released in Subversion 1.7.3, but was
> reverted in r1367674 (issue #4214, recovery should not create rep-cache.db)
> and so omitted from Subversion 1.8 and later series of releases.
> 
> * subversion/libsvn_fs_fs/recovery.c
>   (recover_body): Prune the rep-cache no matter whether it's in use.
> 
> * subversion/tests/cmdline/svnadmin_tests.py
>   (read_rep_cache,
>check_recover_prunes_rep_cache): New functions.
>   (recover_prunes_rep_cache_when_enabled,
>recover_prunes_rep_cache_when_disabled): New tests.
>   (test_list): Run them.
> 
> +++ subversion/trunk/subversion/tests/cmdline/svnadmin_tests.py
> @@ -53,6 +53,23 @@ Wimp = svntest.testcase.Wimp_deco
>  SkipDumpLoadCrossCheck = svntest.testcase.SkipDumpLoadCrossCheck_deco
>  Item = svntest.wc.StateItem
>  
> +def read_rep_cache(repo_dir):
> +  """Return the rep-cache contents as a dict {hash: (rev, index, ...)}.
> +  """
> +  db_path = os.path.join(repo_dir, 'db', 'rep-cache.db')

Should this function assert that db/format <= 8?  When we introduce
format 9 it might not be compatible with this function's expectations.

> +  db1 = svntest.sqlite3.connect(db_path)
> +  schema1 = db1.execute("pragma user_version").fetchone()[0]
> +  # Can't test newer rep-cache schemas with an old built-in SQLite.
> +  if schema1 >= 2 and svntest.sqlite3.sqlite_version_info < (3, 8, 2):
> +raise svntest.Failure("Can't read rep-cache schema %d using old "
> +  "Python-SQLite version %s < (3,8,2)" %
> +   (schema1,
> +svntest.sqlite3.sqlite_version_info))
> +
> +  content = { row[0]: row[1:] for row in
> +  db1.execute("select * from rep_cache") }
> +  return content

How to get feedback on experimental features

[Julian Foad]

In the 1.11 release notes I wanted to write "We would like to hear your 
feedback on these experimental features. Please share your thoughts with us 
" ...

What would make a good landing page for that link?

More generally, how can we better handle getting feedback?

Currently we have almost no user-facing mention of the IRC channels, and only 
our "mailing-lists.html"[1] which is not a friendly interface to giving 
feedback through the users@ mailing list.

How can we do this better?

Some of my thoughts:

  * need a dedicated landing page for that link
  * needs to be inviting and very easy for drive-by users (anyone already 
engaged with the Subversion project will already know how to give feedback)
  * some will just want to send a message to us; others will want to ask a 
question or engage in conversation; the latter is probably more useful feedback 
for us.
  * offer options for how to communicate, such as: IRC, email, Jira ticket
  * need to give a bit more guidance than just "these methods are available" -- 
suggest something, for the drive-by user who doesn't have their own preferences
  * for each option, try to have an easy web interface
- starting points: ASF Pony Mail [4]?, IRC web interface [3]?, a Jira 
"issue collector" [2]?
- but [4] requires a login before posting
  * not for this purpose, but while we're talking of such thing, we should also 
have a page describing the details of all the ways to communicate, that covers 
IRC and Jira as well as email, in the same way that [1] currently details the 
email lists.

Other ideas or starting points, anyone?

[1] http://subversion.apache.org/mailing-lists.html
[2] 
https://issues.apache.org/jira/secure/ViewCollectors!default.jspa?projectKey=SVN
[3] https://webchat.freenode.net/?channels=#svn
[4] https://lists.apache.org/list.html?us...@subversion.apache.org

-- 
- Julian

Re: How to get feedback on experimental features

[Daniel Shahaf]

Julian Foad wrote on Fri, 28 Sep 2018 16:52 +0100:
> Other ideas or starting points, anyone?

Would it help to have a non-publicly-archived feedback channel?  I'm
thinking of, say, some feedback@subversion.a.o mailing list, privately
archived and with moderated subscriptions.

What do other ASF projects do?