Subversion should merge with Mercurial (reverse takeover)

2014-02-03 Thread Paul Hammant
I've been nudged by nice people to post this here - honest !! http://paulhammant.com/2014/02/03/facebook-scaling-mercurial-for-trunk-based-development - (scroll down half way). I'm a user of Subversion from about 0.7 onwards (via Codehaus). I was at the 1.0 party in Brisbane, CA many tears ago.

SVN-4635: Cherry-pick merge scenario causes Svn to choke

2016-06-11 Thread Paul Hammant
http://subversion.apache.org/docs/community-guide/issues.html amongst other things, says " if you're pretty sure about the bug, go ahead and post directly to our development list". OK, so here's a bug with merge tracking: https://issues.apache.org/jira/browse/SVN-4635 Synopsis: Series of

Re: Files with identical SHA1 breaks the repo

2017-02-24 Thread Paul Hammant
Linus weighs on on Git's use of SHA1 (may be interesting) http://marc.info/?l=git=148787047422954=2

Re: Files with identical SHA1 breaks the repo

2017-02-26 Thread Paul Hammant
Why don't y'all take the same tactic as Git does - SHA1 the contents of the file *and a prepended a type/length field* ?. That and a tool to back convert SHA1s for existing repos. Linus weighed in again: https://plus.google.com/+LinusTorvalds/posts/7tp2gYWQugL Svn is more likely to be used as a

New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-09-24 Thread Paul Hammant
In order to be able to do some Merkle-tree style functions on sets of files canonically held in Subversion, it would be great to ask Svn for a SHA1 for the files, or collections thereof from that node downwards. I would raise a new feature request direct into Svn, but the JIRA notes says to not

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-09-24 Thread Paul Hammant
Isn't 'ls --xml' a machine interface? > > Using the Subversion API directly would be the best way to do this. The > checksum is available at the API level, but wouldn't serve any useful > purpose in the output of 'svn ls' -- which is, after all, a user, not > machine interface. > > -- Brane >

Svn capability for SparkleShare

2016-08-25 Thread Paul Hammant
SparkleShare: https://www.sparkleshare.org/ TL:DR: Like DropBox but your own SCM as the backing store, and not as-a-service. There's many non-DropBox host-it-yourself solutions, but none use a formal SCM as the backing store. So of course, SparkleShare is *super-stable for Git only*, but the

Re: Svn capability for SparkleShare

2016-08-29 Thread Paul Hammant
My fork of sparkleshare now does svn-up and svn-commit well enough, in anyone is interested. It's only working on the svn:// protocol, and even then only for allow-anon-write repos. - Paul Sent from my iPhone > On Aug 25, 2016, at 6:18 PM, Paul Hammant <p...@hammant.org&

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-09-24 Thread Paul Hammant
com> wrote: On 24 September 2016 at 15:36, Paul Hammant <p...@hammant.org> wrote: > In order to be able to do some Merkle-tree style functions on sets of files > canonically held in Subversion, it would be great to ask Svn for a SHA1 for > the files, or collections thereof fro

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-09-26 Thread Paul Hammant
Merkle trees / hashes can help a server maintained graph of objects survive "Bitrot" (http://arstechnica.com/information-technology/2014/01/bitrot-and-atomic-cows-inside-next-gen-filesystems/ - data in SSD or HD being corrupted by (say) nutrinos over time. See also a guy/gal lamenting their

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-09-26 Thread Paul Hammant
Replying to various. I'm making a Dropbox-a-like client that uses Svn/WebDav/AutoIncrement as the server. Critical design goal - to *not* have a classic Svn working tree locally. Think 50GB of binary files sync'd down to a client, and a wish to not have that take 100GB of local storage. > What

Re: Svn perf curiosity

2016-11-20 Thread Paul Hammant
to the same USB hard drive - different low-level libc calls to openssl? 2. I made a mistake about the built in SSD - it was fast regardless, despite previous advice. - Paul On Mon, Oct 31, 2016 at 8:27 AM, Paul Hammant <p...@hammant.org> wrote: > I'm curious as to what could make a checkin of

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-11-01 Thread Paul Hammant
With a maintained server-side merkle tree for repos/users y'all would be a little closer to the 'have set' fu of Perforce :) Sent from my iPhone > On Nov 1, 2016, at 6:29 AM, Julian Foad <julianf...@apache.org> wrote: > > Branko Čibej wrote: >> Paul Hammant wrote: >&

Svn perf curiosity

2016-10-31 Thread Paul Hammant
I'm curious as to what could make a checkin of a single 10MB file take 3 mins on a Linux machine, when it is 2 seconds on a Mac. I'm able to reproduce it easily: svnadmin create --fs-type fsx fsxRoot svn checkout file://$(pwd)/fsxRoot testLocalCheckout cd testLocalCheckout openssl

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-10-31 Thread Paul Hammant
van Zhakov <i...@visualsvn.com> wrote: > >> On 12 October 2016 at 14:03, Paul Hammant <p...@hammant.org> wrote: >> It's very exciting to hear that Subversion already calculates shas somewhere >> in the backend :) > I noted this multiple times on this thread: SHAs a

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-10-11 Thread Paul Hammant
n 11.10.2016 14:09, Paul Hammant wrote: >> For the entry of directory that contains mod_dav_svn.c, I'd hope for the >> SHA1 to be a function of the SHA1s of the files within. That's Merkle-tree >> style - a super important feature generally as well as specifically to my >> use

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-10-11 Thread Paul Hammant
> > Thanks for these details, they clarify the picture. > :) > > I've read the ?kw=1 section of the release notes. My use case would not > > need keyword replacement. In fact it would need it to be off. > Are you sure? The only situations in which you'd need keywords > expansion *off* is if

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-10-11 Thread Paul Hammant
> > > As a test, I'm using openssl to make huge files that > > change wholly with every revision, and trying to find the top limits of > > Subversion. Sadly I've only found the top limits of Docker on the mac so > > far - 60GB. > > 60GB being the size of each revision of a single versioned file? >

Re: New SHA1 property for nodes returned 'svn ls --xml' invocations.

2016-10-12 Thread Paul Hammant
You're right - and in the fullness of time, I'll replace all the Svn uses with their wire equivalents. If Shas were implemented at some future date, I'd be happy for them to be available via PROPFIND. I's be even more happy for them to be passed back to me in the response of a PUT. Or are you

Re: Progress on SHA-1 fixes in patch releases?

2017-03-30 Thread Paul Hammant
> Eh? Patch release changelogs get written when the patch is released. I'm not following what you're saying there.

Re: Progress on SHA-1 fixes in patch releases?

2017-03-30 Thread Paul Hammant
No patch releases though, right? http://svn.apache.org/repos/asf/subversion/trunk/CHANGES Is that because, "the server can store the offending PDFs" isn't enough of a releasable in its own right? Or because the "the server can store the offending PDFs" specifically is imperfect and

Re: Progress on SHA-1 fixes in patch releases?

2017-03-30 Thread Paul Hammant
My apologies. When I wrote "No patch releases though, right?" my intention was to communicate "No patch releases YET though, right? -ph On Thu, Mar 30, 2017 at 6:28 AM, Branko Čibej <br...@apache.org> wrote: > On 30.03.2017 12:26, Paul Hammant wrote: > >>

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-01 Thread Paul Hammant
If I can manage to build the LZW compression version of the forthcoming v1.11 Subversion, I'll put that in the mix as a permutation to test. Is anyone interested in the Python harness for these integration tests? - Paul

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-02 Thread Paul Hammant
> > > Here it is - http://imgur.com/a/vBeKi - a more interesting graph. > Note that was 4 iterations, not 6 (as before). Server side Storage Used (MB): Default: 129 No_Compression_No_Deltification: 257 No_Deltification_No_Rep_Sharing: 513 No_Compression_No_Deltification_No_Rep_Sharing: 513

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-03 Thread Paul Hammant
Markus, Yes, leaving rep sharing enabled is better for another reason that a file sync agent is *likely* to encounter: someone renames a directory. Remember I'm not using Subversion client tools to effect a commit. But... >. Rep sharing brings its real benefits when equal files are stored >

Re: Proposal - option to store unzipped office documents on server side.

2017-08-03 Thread Paul Hammant
> > > You are not the first to ask for this, but it is significantly more > complex than just a backend setting. [] > Yup, I didn't think about the SHA1 being different. I'll implement it client-side, just ignore this request. -ph

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
On Thu, Jul 13, 2017 at 06:34:17AM -0400, Paul Hammant wrote: > >> Or am I really wanting Svn's backend compression and deltification to be > >> out of band ? > >> > >> 1. compression-strategy = defer-to-idle-time-even-if-days-later > >> 2. de

Re: Proposal: new fsfs.conf properties

2017-07-14 Thread Paul Hammant
Corveleyn <jcor...@gmail.com> wrote: > On Fri, Jul 14, 2017 at 1:18 PM, Evgeny Kotkov > <evgeny.kot...@visualsvn.com> wrote: > > Paul Hammant <p...@hammant.org> writes: > > > >> 1. compression-exempt-suffixes = mp3,mp4,jpeg > >>

Re: Proposal: new fsfs.conf properties

2017-07-14 Thread Paul Hammant
Well anecdotally, it wasn't oscillating like that in earlier phases. I'll do p4 obliterate and start over. I mean svn obliterate. Err :-P > > In fact it might be. A full filesystem may have a harder time finding > contiguous empty space, which may translate to more separate > (non-contiguous)

Proposal - Optional JavaScript include in "Powered by Apache Subversion..." pages

2017-07-17 Thread Paul Hammant
The page I'm talking about is the skinny one that presents directory listings: svnRepo1 - Revision 195: / svnRepo1 - Revision 195: / .. a/ Powered by http://subversion.apache.org/;>Apache Subversion version 1.9.5 (r1770682). It would be super awesome to be able to administrators of

Re: Proposal: new fsfs.conf properties

2017-07-17 Thread Paul Hammant
Philip, > Ah, it's not running out of space in tmp, rather it is failing to find > tmp. There is a behaviour difference between Subversion 1.9 and 1.10 > here: each Apache child process using 1.9 will create two zero-length > files in tmp, the first is APR finding tmp the second is Subversion >

Re: Proposal: new fsfs.conf properties

2017-07-17 Thread Paul Hammant
I think as this thread draws towards a close, that the lz4 modifictions Evgeny made in part address my speed concerns for gigabyte sized files. And that if any more speed for PUT is wanted, that one would set: max-deltification-walk = 0 compression-level = 0 It also looks like the there is

Re: Proposal - Optional JavaScript include in "Powered by Apache Subversion..." pages

2017-07-17 Thread Paul Hammant
> > See the SVNIndexXSLT directive. You can do pretty much whatever you like > with the directory listing. > Near perfect. I'll blow the dust off my XLS skills (unused for 16 years).

Re: LZ4 compression

2017-07-17 Thread Paul Hammant
Evgeny wrote: Speaking of only using it with compression level 1, that's not a starting > point. While LZ4 offers superior speeds, it is not a substitute for any > zlib compression level > 1, including our current default of 5, as the > latter gives better compression ratio. > > I was thinking

Re: Expected speed of commit over HTTP?

2017-07-07 Thread Paul Hammant
mistake. What I'm using is in fact: dd if=/dev/zero bs=1M count=500 2>/dev/null | openssl enc -rc4-40 -pass pass:weak > six - Paul On Fri, Jul 7, 2017 at 2:39 PM, Stefan Fuhrmann <stefanfuhrm...@alice-dsl.de > wrote: > On 07.07.2017 01:10, Paul Hammant wrote: > >

Proposal: new fsfs.conf properties

2017-07-07 Thread Paul Hammant
1. compression-exempt-suffixes = mp3,mp4,jpeg 2. deltification-exempt-suffixes = mp3,mp4,jpeg Regardless of the setting of 'compression-level', #1 above two mean certain things can skip the compression attempt. It must give up at a certain point right? Same for deltification re #2 I'm

Re: Big files PUT into Subversion - encountering

2017-07-09 Thread Paul Hammant
I think I have tracked it to root cause: boot partition didn't have 10GB free at the time of the upload. There's a /etc/apache2/envvars file where I can change the location of TMP, I think. Which mean would mean that somewhere in the chain of Apache and it's modules the file system is used for the

Re: Proposal: new fsfs.conf properties

2017-07-11 Thread Paul Hammant
com> wrote: > Hi, > > (Sorry, it seems my previous message was sent _very_ prematurely :-( > > From: Pavel Lyalyakin [mailto:pavel.lyalya...@visualsvn.com] > > Hello Paul, > > > > On Sat, Jul 8, 2017 at 2:51 AM, Paul Hammant <p...@hammant.org> wrote: > &g

Re: Proposal: new fsfs.conf properties

2017-07-11 Thread Paul Hammant
I'm perfectly happy for the solution to be mime-type based. Maybe we can take the mime-type to *suffix table* from Apache itself to do the translation :- https://svn.apache.org/repos/asf/httpd/httpd/trunk/ docs/conf/mime.types :-P I used it (implicitly) in a Subversion backed wysi-wiki *ten

Re: Proposal: new fsfs.conf properties

2017-07-11 Thread Paul Hammant
So I'm after a time saving. I'm perfectly happy for the backend to waste space (in my configuration), I just don't want it to take 15 mins to transfer a single 15GB file into Subversion. In my configuration, I'd like to pre-advise Subversion to save as much time as possible for uploads, by

Re: Big files PUT into Subversion - encountering

2017-07-11 Thread Paul Hammant
Reporting back - The /etc/apache2/envvars file is the *perfect* place to change the temp folder that Apache uses on load, and avoid an out of space error for bg files(see log above). I thought there might also be a speed boost for having the temp folder set to the same driver as the

Re: Big files PUT into Subversion - encountering

2017-07-08 Thread Paul Hammant
Interesting - the first restart of that sequence failed again for the same reason. I didn't try it a third time, I just rebooted the box (instead of apachectl restart), so I don't know if that was a *persistent* state for the implicit Svn server or not. On Sat, Jul 8, 2017 at 9:36 AM, Paul

Big files PUT into Subversion - encountering

2017-07-08 Thread Paul Hammant
Same system as before - 4GB RAM Intel Atom Linux box , with attached 4TB USB3 driver (yes in a USB3 port) where the svn repo's root is. AutoIncrement is turned on (or whatever it's called), and

Re: Proposal: new fsfs.conf properties

2017-07-12 Thread Paul Hammant
Jul 12, 2017 at 10:40 PM, Paul Hammant <p...@hammant.org> wrote: > > I'd be fine with that too if it is also settable via curl --header > > "svn:compression: no" for non-client auto-increment operations. > > I'm wondering whether you'll still need this. You ended

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
On rationale ... This is a file sync agent. 600 lines of Python that I'm going to open source. The same thing I was working on in 2016 when I chatting on this list. Goals: 1) *not have two copies of everything on the clien*t, 2) have different semantics to pushing back changed files. Secondary

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
> > > > Mmm ... interesting proposition. Also huge -1 because it'd be a really > awesome abstraction leak. :) > Is there any way to implement the (reasonable in my opinion) feature request, without transgressing on programming standards? I think turning off all deltification is a bit brute

Re: Big files PUT into Subversion - encountering

2017-07-13 Thread Paul Hammant
Markus - you may be right on hopes for perf improvements. I'm reevaluating what I said a couple of days ago in this thread. The best case PUT times for that 15GB random resource at *7 mins*, but about 1/5 of them are at *15 mins*. I'm going to try to undo the TMPDIR change and see if it goes back

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
Sweet - I'll operationalize that ASAP. The client side of the sync agent knows whether the item is new or to be changed, already. - Paul > > > Sorry I'm gunna be a bit slow here, Philip, but you're saying that > sequence > > of four is a single atomic commit in Subversion? > > Yes. > >

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
> > > The delete+recreate takes the same 5.2s as the original create and is > faster than curl's 6.7s update. > > New fsfs config prop ? delete_and_recreate_instead_of_change_threshold = 50MB -ph

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
Sorry I'm gunna be a bit slow here, Philip, but you're saying that sequence of four is a single atomic commit in Subversion? - Paul On Thu, Jul 13, 2017 at 7:48 AM, Philip Martin wrote: > Philip Martin writes: > > > The fact that the create

Re: Proposal: new fsfs.conf properties

2017-07-13 Thread Paul Hammant
wrote: > On 13.07.2017 13:07, Paul Hammant wrote: > > > > > > The delete+recreate takes the same 5.2s as the original create and is > > faster than curl's 6.7s update. > > > > > > New fsfs config prop ? > > > > delete_and_recreate_in

Re: Proposal: new fsfs.conf properties

2017-07-14 Thread Paul Hammant
Philip, > I don't believe Apache/mod_dav_svn uses TMPDIR when processing a PUT. > I can prove that either Apache or ModDavSvn (or the OS) uses TMPDIR during a PUT of a very large resource. Test 1: leave 3GB free on system drive, try to PUT 15GB file thru DAV into it's ultimate destination on a

Re: Big files PUT into Subversion - encountering

2017-07-15 Thread Paul Hammant
4.10.0-26-generic More than you asked for, but more specs: OS: Ubuntu 17.04, server Processor: 2.4 GHz Intel Atom RAM: 4 GB - DDR3 Hard Drive: 64 GB (SSD of some sort - poss USB) Checking the read speed of the boot partition: $ sudo hdparm -tT /dev/mmcblk0 /dev/mmcblk0: Timing cached reads:

Re: Proposal: new fsfs.conf properties

2017-07-15 Thread Paul Hammant
f 64GB SSD. - Paul On Fri, Jul 14, 2017 at 4:42 PM, Philip Martin <phi...@codematters.co.uk> wrote: > Paul Hammant <p...@hammant.org> writes: > > >> I don't believe Apache/mod_dav_svn uses TMPDIR when processing a PUT. > >> > > > > I can prove that e

Re: Proposal: mod_authz_svn to get an option to inform clients that resource 'cannot be written to'

2017-07-21 Thread Paul Hammant
> > > What is the story? How is this going to help Subversion users? > > The idea sounds interesting. However, I'm not sure how it will help > the users. I don't mind whether I have Read Only or Read / Write > access to an FTP resource or another kind of remote data. I just know > whether I have

Re: Proposal: mod_authz_svn to get an option to inform clients that resource 'cannot be written to'

2017-07-25 Thread Paul Hammant
May I go ahead and raise the feature request in Jira now? I believe the approved process was discuss on mail-list, then raise in Jira. - Paul On Fri, Jul 21, 2017 at 5:40 AM, Julian Foad <julianf...@gmail.com> wrote: > Paul Hammant wrote:> It will help the owners of subv

Re: Proposal: new fsfs.conf properties

2017-07-27 Thread Paul Hammant
> > A couple of tests show that it's possible to get up to 2x speed improvement > when importing large binary files, although the actual improvement will > vary, depending on the configuration of the server and the client. And, > apart from this, such imports won't be using the free space in

Re: Proposal: mod_authz_svn to get an option to inform clients that resource 'cannot be written to'

2017-07-27 Thread Paul Hammant
OK, I've added https://issues.apache.org/jira/browse/SVN-4691 I've tried to focus on the meat of the ask, and cover previous questions. Also I've covered previous suggestions, though while welcome, would not solve the same problem.

Re: Proposal: new fsfs.conf properties

2017-07-12 Thread Paul Hammant
You know, in all seriousness I think the (empty by default) list of exempted files suffixes the the best way forward. If suffixes is good enough for Apache itself to use (link provided earlier), it is good enough in this scenario on the server side of Svn. If the function in question doesn't know

Re: Proposal: new fsfs.conf properties

2017-07-12 Thread Paul Hammant
I'd be fine with that too if it is *also* settable via curl --header "svn:compression: no" for non-client auto-increment operations. On Wed, Jul 12, 2017 at 4:10 PM, Mark Phippard wrote: > I cannot find it in archives so maybe this happened in IRC, but I remember > one time

Re: [ANNOUNCE] Apache Subversion 1.9.6 released

2017-07-06 Thread Paul Hammant
Someone owns the consequential updates to Homebrew - right ? Ref: https://github.com/Homebrew/homebrew-core/blob/master/Formula/subversion.rb - Pull-Requests are the usual model, but there's easily room for duplicated effort, here. - Paul

Re: [ANNOUNCE] Apache Subversion 1.9.6 released

2017-07-06 Thread Paul Hammant
Great work, all round :)

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
e on the end server of any > size. HTTP/mod_dav and RTT is usually the culprit in our cases, on a LAN > that issues collapses. Likely server side processing of the delta's? > > -jacek > > On Thu, Jul 6, 2017 at 1:54 PM, Paul Hammant <p...@hammant.org> wrote: > >> For

Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
For something that's 500MB in size (random binary data) I'm experiencing commits taking 10x longer than a straight copy to the drive the Svn repo is on. Both timings are on the same Ubuntu 17.04 machine, with the boot drive being the starting position of the 512MB file and a USB3 mounted 4TB

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
Jul 6, 2017 at 8:03 AM, Branko Čibej <br...@apache.org> wrote: > On 06.07.2017 13:54, Paul Hammant wrote: > > For something that's 500MB in size (random binary data) I'm > > experiencing commits taking > > 10x longer than a straight copy to the drive the Svn

Re: Expected speed of commit over HTTP?

2017-07-06 Thread Paul Hammant
With autorevision set to 'on' and curl: Reverence speed for boot drive to USB3 spinning platter 4TB thing: paul@paul-HiBox:~$ time cp /home/paul/clientDir/seven /media/paul/sg4t/sevenb real 0m1.539s Create a new 501MB file on Svn Server: paul@paul-HiBox:~$ time curl -u paul:myPassword

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-01 Thread Paul Hammant
I'll re-run for smaller sizes (and yes that 128GB was a typo) Things won't be not so separable that way, IIRC.

Benchmarks for PUT for various fsfs config settings.

2017-07-31 Thread Paul Hammant
Chart: http://imgur.com/cKe2l4J In Python I made two 128GB random files, then uploaded them alternately six ties three each to the same endpoint in Svn over DAV. That would be a run. After wiping the server repo and repeating that, I performed four runs all in all. For each datapoint, I

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-02 Thread Paul Hammant
Alternate file sizes - still pretty much every byte different between the two alternately uploaded files: 24MB - http://imgur.com/a/kPyY8 4MB - http://imgur.com/a/uRaW8 Next up - larger files where the second has a mid-section that changes for a few bytes, vs the first.

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-02 Thread Paul Hammant
> > Next up - larger files where the second has a mid-section that changes for > a few bytes, vs the first. > Here it is - http://imgur.com/a/vBeKi - a more interesting graph.

Proposal - option to store unzipped office documents on server side.

2017-08-03 Thread Paul Hammant
I understand the process to propose feature requests is to first post the idea on this forum, then if everyone agrees, post an item to Jira. I previously wrote a blog entry on Git storing MsOffice document unzipped (.docx and .xlsx are zips XML and a few things) -

Re: Proposal - option to store unzipped office documents on server side.

2017-08-04 Thread Paul Hammant
le zip itself compresses all files independently. Not related to Svn, but elsewhere I was working towards Maven Jars in Git, too .. *"Github releases as a Maven repo" * So I have 27 releases of XStream unzipped and pushed to https://github .com/paul-hammant/mc-xs-classes In terms o

Re: [RFC] Using LZ4 compression by default

2017-08-04 Thread Paul Hammant
Wouldn't the svn client just speculatively specify a HTTP "Accepts" header with requests up to the server? You'd be able to do back/forwards compatibility with that, and not have to change any other wire spec ?

Re: Benchmarks for PUT for various fsfs config settings.

2017-08-06 Thread Paul Hammant
http://imgur.com/a/vVKSx This one is the original 128MB PUT test but on a Raspberry Pi Zero W (client and server). Raspbian is based on Debian Jesse which uses Svn 1.8.10. The Zero has 512MB RAM and a 1Ghz ARM chip (the same spec as the original Pi). Performance was 1/6 of the CHUWI HiBox Smart

Re: Sparse checkouts suggestion

2017-09-14 Thread Paul Hammant
Thanks. Here's what I expect to land - https://github.com/apache/subversion/pull/5 Ten tests - ten passing in less than 0.1s I don't know if there is a clean workflow from Git commits back into Subversion so, I'll make the same commit through the Svn HTTPS interface, if code review comments are

[GitHub] subversion pull request #5: pytest tests for svn-viewspec

2017-09-14 Thread paul-hammant
GitHub user paul-hammant opened a pull request: https://github.com/apache/subversion/pull/5 pytest tests for svn-viewspec You can merge this pull request into a Git repository by running: $ git pull https://github.com/paul-hammant/subversion trunk Alternatively you can

Re: Sparse checkouts suggestion

2017-09-14 Thread Paul Hammant
> > Yes, that one's pretty nice, but as a rule we try not to burden our > users with dependencies outside the standard library. The script uses > os.system, what, twice? Hardly worth replacing. > > On the other hand, we have wrappers around subprocess.Popen in >

Re: Sparse checkouts suggestion

2017-09-14 Thread Paul Hammant
I ran pep8 over the sources and thought I'd remediated all the training spaces already. I'll finish the job. My alternative to the four globals is to pass the function pointers as parameters through one or two intermediates to the functions that would use them. Would that be acceptable? I can't

Re: Sparse checkouts suggestion

2017-09-14 Thread Paul Hammant
Would I retain the previous value of os.system and set it back after any assertions (pass or fail) so that following tests would not affected? Or does it und itself naturally when Pytest recurses in the next test in its suite ? As I hinted I couldn't find such an example online, with an emphatic

Re: Building Subversion.

2017-09-14 Thread Paul Hammant
The needs of the book are different to the needs of the Svn project. And the needs of the project are far more important than one of a few Svn books. I'm going to stand by what I said. -ph

Re: Sparse checkouts suggestion

2017-09-14 Thread Paul Hammant
My aim was to minimally impact the prod source. The context object is a good idea. I've been using https://amoffat.github.io/sh/ and quite enjoying it for my other python/svn project.

Building Subversion.

2017-09-14 Thread Paul Hammant
The tests made for https://github.com/apache/subversion/pull/5 didn't even require me to have Svn on my system. That's because I'm asserting against a series of expected shell operations, because that's how the author implemented svn-viewspec Has anyone built Subversion on a Mac? If yes, do you

Re: Building Subversion.

2017-09-14 Thread Paul Hammant
> > > > If yes, do you have the homebrew installables you needed handy? > > #LemmeKnowPls > > "brew install subversion"? > > I've done't that already, but I'm switching hats from 'user' to 'developer' of subversion. The brew installables I was thinking of were the development pre-requisites for

Re: Sparse checkouts suggestion

2017-09-13 Thread Paul Hammant
OK, so I am up at 93% coverage with a PyTest script. As an Apache Member I'm already CLA'd. I could start to write tests that break the script in ways that it should handle more gracefully, but I need assurance that I'm not wasting my time. I'm not expecting committership (especially since my

Re: Sparse checkouts suggestion

2017-09-16 Thread Paul Hammant
> > > foo.py: > > import os > def my_mocked_os_system(*args, **kwargs): > print('a mockery') > os.system = my_mocked_os_system > > > bar.py: > > import os > os.system("echo bar") > > import foo > os.system("echo foo") > > > Then: > > $ PYTHONPATH=. python bar.py > bar > a mockery > > I've

Re: Sparse checkouts suggestion

2017-09-19 Thread Paul Hammant
> On 19.09.2017 00:59, Paul Hammant wrote: > > I like the `svn co svn://vcs/trunk --view foo`. Well maybe if in a > > following `svn up` it would *remember the current view*, it would be > good. > > > I don't think there's any need to remember the "current view&q

Re: Sparse checkouts suggestion

2017-09-21 Thread Paul Hammant
> > > I'm not sure if you agree or not, but I think the fine-grained globbing > include/exclude capability as Perforce has it. is required. > > > "Required" implies "working copy format change". That's always hard. > > "Optional" means "get some of the features sooner." > I'm a big advocate for

Re: Sparse checkouts suggestion

2017-09-18 Thread Paul Hammant
I like the `svn co svn://vcs/trunk --view foo`. Well maybe if in a following `svn up` it would *remember the current view*, it would be good. > I'm not sure about listing the "available views" in a default > property, but okay, it's a possibility. I think it would be nice to be > able to read it

Re: Sparse checkouts suggestion

2017-09-21 Thread Paul Hammant
> > [...] > > if /foo does not exist at checkout time -- and if we'd want /* to mean > "include anything new that appears in the repository", not "include > everything else that's in the repository at the time of the checkout". > I'm not sure if you agree or not, but I think the fine-grained

Re: Sparse checkouts suggestion

2017-09-21 Thread Paul Hammant
> > > > 2. Previously I forked a medium-sized Monorepo on Github, and did the the > complete expand/contract work for it - https://github.com/paul-hamm > ant-fork/jooby-monorepo-experiment - in Python. > > LOGIBALL's Tim Krüger has just written about a deployment of an expanding/contracting

MKCOL question

2017-10-09 Thread Paul Hammant
Hi gang, *(non subversion-client usage warning, also BDD ahead warning)* Given I have a directory /path/to/missing/directory/ that does not exist on the Svn server at all When I want to put a file in there (say foo.mp3) Then I have to MKCOL path/ And I have to MKCOL path/to/ And I have to MKCOL

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

2017-10-10 Thread Paul Hammant
As my forthcoming multi-user app that uses Subversion as a backing store is going to kill Svn it with Depth-∞ PROPFINDs from root, I really want to see this implemented. Because I can't wait I will implement something that calculates SHA1s for the directory in question, and drops it into a

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

2017-10-10 Thread Paul Hammant
Yikes! I'll have to run through the build setup for Subversion - 52 pages of requirements *before* launching ./config, right? - including wind-of-newt, left handed screwdriver, *two* four-leaf clovers ;) :-P Seriously: sounds good - and I appreciate it :)

Re: MKCOL question

2017-10-10 Thread Paul Hammant
Well pipelining into Svn's HTTP 1.1 interface does work as you suggested. There is a library in Python called 'hyper' that does it - http://hyper.readthedocs.io/en/latest/quickstart.html#streams though the example get_response signature is not current (I raised a bug). Also, if I change

Re: How to manage PR's (was: [GitHub] subversion pull request #6: VC should be 14.1 and not 15.0)

2017-10-10 Thread Paul Hammant
xpect patch submissions on dev@ instead)? > > If you ask people comfortable with donating code via PR to email patches (or similar) to dev@ instead, you'll get a 1/100th contribution rate. Or worse. There's no easy answer. Partially related - I'd previously whined about GitHub's lock-in for some dat

Re: Quick question about the sha1-checksum for directories in svn.

2017-10-05 Thread Paul Hammant
> Correct. Thanks Julian. Observation: If it did then Subversion would qualify as a *full* Merkle tree implementation. Albeit through APIs designed for other purposes :-p ;)

Quick question about the sha1-checksum for directories in svn.

2017-10-05 Thread Paul Hammant
While a _directory_ has a revision number, it does not have a SHA1 does it? If so there's no point asking how it's calculated, is there ? - Paul

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

2017-10-05 Thread Paul Hammant
Not that my vote counts for much, but I'd prefer w/o props, obeying read permissions.

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

2017-10-05 Thread Paul Hammant
> "Obeying read permissions" means that the directory hashes would have to > be computed dynamically for each user. Yes I know. I've a frankenstein project over here - https://github.com/paul-hammant/merkle-rust - There's a solid Rust merkle tree that watches for changes at nod

Re: Merkle trees in svn [was: Quick question about the sha1-checksum for directories in svn.]

2017-10-05 Thread Paul Hammant
Agree. For those that are super interested in this, I did a series of four blog entries on Merkle trees and Blockchains. - Sept 28th » Choosing Between Blockchains And Vanilla Merkle Trees -

  1   2   >