Re: [fossil-users] OT: security/entropy (was Re: New Fossil user experiences)

[Joerg Sonnenberger]

On Fri, Jul 13, 2018 at 02:49:52PM -0700, Jungle Boogie wrote:
> On Fri 13 Jul 2018  4:22 PM, David Mason wrote:
> > So I guess this is what Warren had in mind.  Posting this in case it helps
> > somebody on the list.
> > 
> 
> Taking this offtopic a little bit more...let's talk about VPNs.
> 
> Don't use PPTP and don't get tangled up in ipsec configuration hell.
> 
> Be happy with wireguard! https://www.wireguard.com
> 
> It runs on everything but Windows (Linux, *BSD, MacOS, Andriod, iphone). 
> There's
> no passwords to share and the public keys are easily, easily distributed.

While I know that the underlaying cryptographic primites are sound, a
lot of the wireguard description is overhyped and misguided. Code
doesn't become more secure by moving it into the kernel and a VPN daemon
does a lot more than just dealing with key exchange. I'd strongly advice
to look at the ingredience list of the KoolAid...

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] OT: security/entropy (was Re: New Fossil user experiences)

[Joerg Sonnenberger]

On Fri, Jul 13, 2018 at 03:27:14PM -0600, Warren Young wrote:
> On Jul 13, 2018, at 3:13 PM, Warren Young  wrote:
> > 
> > Now paste in an equivalent number of ‘a’ characters, and you get 0 bits of 
> > entropy.  Strictly speaking, you get 1 bit of entropy for the whole 
> > message, but it shows 0 because the calculator is rounding the result off 
> > to 3 significant figures.
> 
> Hmmm…we also need something like a run-length prefix to reconstruct the 
> message, so this calculator is undershooting slightly.
> 
> For example, 100 a’s requires a 7-bit run-length plus zero bits for our
> only code point, so we should get 0.07 bpc, within this calculator’s
> apparent precision even without dealing with roundoff errors.

You need more than zero bits to encode the original a though. Frankly,
this seems like a bit of pseudo-math to me. The very definition of
entropy depends on the context. So a question of "how much entropy does
this string have" is seriously underdefined. If you can take the output
of any modern CPRNG as hex and don't get 4bpc, the entropy estimator is
broken.

Joeg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] What should email notifications look like?

[Joerg Sonnenberger]

On Fri, Jun 22, 2018 at 02:18:35PM -0400, Richard Hipp wrote:
> Shown below is what I have at the moment.  It is more succinct than
> other examples I've seen, but you can easily follow the link for
> details.

BTW, the main mercurial list uses the following mail format for "batch"
changes:

https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-June/117551.html

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] What should email notifications look like?

[Joerg Sonnenberger]

On Fri, Jun 22, 2018 at 09:09:11AM -0400, Richard Hipp wrote:
> When an event occurs (such as a check-in, or wiki-page edit) and a
> user wants a notification of that event, what should the email look
> like?

What I am using for the Mercurial tests is the following:

https://mail-index.netbsd.org/source-changes-hg/2018/06/21/msg002146.html

Subject contains the common directory prefix of all changed files and
the start of the commit message, up to 80 characters in total. The user
on the web page is anti-spam fudging. Patch itself is truncated after
300 lines.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Perception of Fossil

[Joerg Sonnenberger]

On Thu, Jun 14, 2018 at 10:44:24PM +0100, Thomas wrote:
> On 2018-06-14 22:37, Joerg Sonnenberger wrote:
> > How do I develop a patch locally and send it to someone for review? The
> > pull request model is kind of stupid and works only for a centralized
> > system (the irony...), but integration of something like "patchbomb" or
> > even just bundles is quite handy for this.
> 
> The pull request is exactly this. Sending a patch via mail or mailing list
> like the dinosaurs did is not going to make it more appealing.

Can you please just stop trolling? Everyone else, please ignore
"Thomas".

Thanks.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Perception of Fossil

[Joerg Sonnenberger]

On Thu, Jun 14, 2018 at 04:51:08PM -0400, Ron W wrote:
> In another forum I follow,a commented claims that Fossil is designed for
> "cathedral development" not "bazaar development", so would be of little
> interest to anyone. Unfortunately, the poster did not elaborate on why.
> 
> Except maybe possible issues scaling to a large number of contributors, I
> don't see how Fossil is less suitable for  "bazaar development" than git or
> Hg.
> 
> Thoughts?

How do I develop a patch locally and send it to someone for review? The
pull request model is kind of stupid and works only for a centralized
system (the irony...), but integration of something like "patchbomb" or
even just bundles is quite handy for this.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] [sqlite] Mailing list shutting down...

[Joerg Sonnenberger]

On Thu, Jun 14, 2018 at 09:23:14PM +0100, Thomas wrote:
> On 2018-06-14 20:59, Warren Young wrote:
> > On Jun 14, 2018, at 1:36 PM, Thomas  wrote:
> > > 
> > > no one wants to see all those in their inbox.
> > 
> > Mailing list messages are easily filtered.
> > 
> > I have one mailbox for each mailing list I subscribe to, and I read through 
> > the messages in list order, which makes it easy to mentally switch gears 
> > from one project to the next.
> 
> Most people only have one mailbox. I presume you're referring to folders.

OK, I guess that makes it pretty clear that your knowledge of mail
handling is limited. That's unsurprising, given that the majority of
fori proponents never really managed basic things like mail filtering...

> > If one project gets out of hand for a while, I can mark only that one 
> > mailbox as “read” without declaring email bankruptcy on all my other email.
> 
> Forum software offers the very same functionality but that's not the direct
> purpose of it. In a mailing list you're either "in" or "out". A forum
> provides all possible options.

I've had to deal with my share of fori. Frankly, they all suck for power
users, often badly. While mailing lists do tend to be a bit more
annoying than newsgroups, they nevertheless share the majority of
advantages. Offline access, decent filtering etc. Heck, a lot of fori
programs still hasn't even managed good threading.

You can't search a mailing list? Stop using Outlook.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Mailing list shutting down...

[Joerg Sonnenberger]

On Wed, Jun 13, 2018 at 09:10:48AM -0400, Richard Hipp wrote:
> Other issues with GNU MailMan:
> 
> (1) Only works with Apache.  Or, at least, I have only been able to
> get it to work with apache.  That means I have to run a separate
> apache server just to operation MailMan, whereas the rest of SQLite
> and Fossil uses a non-apache solution.

I never used it with Apache. All my past installations where done with
FastCGI.

> (2) We keep having problems with evil subscribers harvesting the email
> addresses of innocent posters and send them porn-spam via private
> email.  Since the porn-spam contain the subject line of the original
> posting, it often makes it through spam filters.  MailMan has not
> effective solution to this.

This is nothing mailman can do anything about though. The same issue
exist with any other mailing list.

> (3) GNU MailMan is a pile of Python, spread out across many
> directories in magical places all over the filesystem.  It is sparsely
> documented (that I have been able to find) and difficult to work on.

Is it? At least the pkgsrc version is fully contained under lib/mailman.
I don't comment on the source quality.

> For all of the above reasons, I think the time has come to abandon GNU
> MailMan for something better - something that I have more control
> over.

From past experience, I would generally recomment against rewriting mail
handling software. It tends to just not be worth the trouble.

That said, which version are you currently using -- the latest? I can
try to spend some time on checking how much work the captcha patch
needs in terms of updating.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Mailing list shutting down...

[Joerg Sonnenberger]

On Wed, Jun 13, 2018 at 07:28:02AM -0400, Richard Hipp wrote:
> Unfortunately, I'm going to need to shut down this mailing list due to
> robot harassment.  I am working to come up with a fix or an
> alternative now.  Your suggestions are welcomed.

Does the patch from
https://www.dragonsreach.it/2014/05/03/adding-recaptcha-support-to-mailman/
work?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fast-import crash (mark not declared)

[Joerg Sonnenberger]

On Fri, Jun 08, 2018 at 05:06:23PM +0300, Petr Ovtchenkov wrote:
> During attempt to export fossil's repo (tcl, http://core.tcl.tk/tcl) to git I 
> face with
> fast-import (fossil export --git --export-marks ../tcl-fossil/fossil.marks 
> ../tcl.fossil | git
> fast-import --export-marks=../tcl-fossil/git.marks) crash:

Hm. That repo has timewraps.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] HTTP caching, again

[Joerg Sonnenberger]

On Fri, May 18, 2018 at 08:39:15AM +0200, Florian Balmer wrote:
> Also, with "Vary: Cookie", there may be issues with caching proxies,
> depending on whether they receive and evaluate all the cookies, but this
> may not be a problem for Fossil.

Such a proxy would be pretty broken. It has to parse the request to find
the URL already and the header will tell it the client cookie. Varying
on cookies is also one of the most common instances.

> For clients that do not understand or support "Vary: Cookie", I would still
> suggest to perform the Last-Modified checks only if no ETag was included
> with the request (so that ETag misses can not be outdone by Last-Modified
> hits).

Again, such a client is pretty much broken already under the caching
model. But it would likely not care about the login details in that
case.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] HTTP caching, again

[Joerg Sonnenberger]

On Thu, May 17, 2018 at 04:08:18PM -0400, Richard Hipp wrote:
> On 5/17/18, Joerg Sonnenberger <jo...@bec.de> wrote:
> > On Thu, May 17, 2018 at 07:02:17PM +0200, Florian Balmer wrote:
> >> So I tried to to generate a "login-time-sensitive" ETag. This worked well
> >> with the "cexpire" field from the "user" table (which is actually the
> >> login
> >> time, shifted to the future by one unit of the "cookie-expire" setting).
> >
> > Would a vary-on-cookie solve this already?
> >
> 
> Fossil uses two cookies:  One for the login and a separate "display
> cookie" to remember display preferences.  The ETag values already
> reset based on changes to the display cookie.  I suppose they could
> change again based on the login cookie.  The question is, would this
> solve Florent's problem?

I don't think you need to reset it, just sending the vary header should
be enough?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] HTTP caching, again

[Joerg Sonnenberger]

On Thu, May 17, 2018 at 07:02:17PM +0200, Florian Balmer wrote:
> So I tried to to generate a "login-time-sensitive" ETag. This worked well
> with the "cexpire" field from the "user" table (which is actually the login
> time, shifted to the future by one unit of the "cookie-expire" setting).

Would a vary-on-cookie solve this already?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Why Fossil is so fast?

[Joerg Sonnenberger]

On Mon, Mar 26, 2018 at 09:50:22AM -0600, Warren Young wrote:
> On Mar 24, 2018, at 5:11 PM, Svyatoslav Mishyn  
> wrote:
> > 
> > just noticed that page generation of many of my repos is 0.001s.
> 
> The calculation for that is in the skin’s Footer code:
> 
> [expr {([utime]+[stime]+1000)/1000*0.001}]
> 
> In TH1, the utime and stime functions return microseconds, so + 1000
> means “add one millisecond”, so that when dividing microseconds down
> to seconds with the subsequent arithmetic, the resulting value can never be 
> lower than 0.001s.

That's a very complicated way to just add 0.001 if that is the
intention. If the goal is to round up, it would be better to add 999 :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Replacing subversion revision number... by what?

[Joerg Sonnenberger]

On Wed, Dec 27, 2017 at 11:56:31PM +0100, Olivier Mascia wrote:
> > Le 27 déc. 2017 à 23:24, Joerg Sonnenberger <jo...@bec.de> a écrit :
> > 
> > On Wed, Dec 27, 2017 at 10:10:21PM +, bch wrote:
> >> On Wed, Dec 27, 2017 at 2:06 PM Olivier Mascia <o...@integral.be> wrote:
> >> 
> >>> Hello,
> >>> 
> >>> Coming from subversion where there is a revision number, incremented by
> >>> one by each commit,
> >> 
> >> 
> >> 
> >> Let me be the first of many to say that those centrally controlled
> >> increments are not possible in a *distributed* source control system.
> > 
> > That's not completely true. You could use the length of the commit chain
> > to the root for most of the purposes of the CVS/SVN revision number.
> > It's just not necessarily a unique property of a commit.
> 
> Thanks Joerg.
> 
> >> I'm considering replacing the subversion revision ID, for the purpose of 
> >> defining the file version ID (as above) at release-external build time, by 
> >> the count of check-ins in the root repository.  That is the count returned 
> >> by 'fossil info' in one of the multiple lines of output (for instance 
> >> 'check-ins: 8801').
> 
> My 'count of check-ins' is your 'length of the commit chain to the root', or 
> are we talking of something else here?

If you have a commit graph like:

A
|
B
| \
C D
| |
E F

Both E and F have a LoCC of 4, but the count f check-ins would depend on
the order of commits?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Replacing subversion revision number... by what?

[Joerg Sonnenberger]

On Wed, Dec 27, 2017 at 10:10:21PM +, bch wrote:
> On Wed, Dec 27, 2017 at 2:06 PM Olivier Mascia  wrote:
> 
> > Hello,
> >
> > Coming from subversion where there is a revision number, incremented by
> > one by each commit,
> 
> 
> 
> Let me be the first of many to say that those centrally controlled
> increments are not possible in a *distributed* source control system.

That's not completely true. You could use the length of the commit chain
to the root for most of the purposes of the CVS/SVN revision number.
It's just not necessarily a unique property of a commit.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Weird behavior on Chisel, perhaps a version mismatch?

[Joerg Sonnenberger]

On Wed, Nov 29, 2017 at 03:36:07PM -0700, Warren Young wrote:
> On Nov 29, 2017, at 3:10 PM, Joerg Sonnenberger <jo...@bec.de> wrote:
> > 
> > On Wed, Nov 29, 2017 at 12:02:11PM -0500, Richard Hipp wrote:
> >> On 11/29/17, Jacob MacDonald <jaccar...@gmail.com> wrote:
> >>> Ah, I see. It seems strange to me that the policy didn't get set when I
> >>> cloned. How can I avoid the same thing in the future and is there any way
> >>> to change the artifacts in the local repository to SHA3?
> >> 
> >> There is no good way to change historical artifacts from SHA1 to SHA3.
> > 
> > There could be a loss-less one-time migration of the whole repo
> 
> We discussed that back around March when all of this was happening.  The 
> simple cases are easy and the hard cases are really hard.
> 
> For example, if checkin [1234abcd] has a comment that refers to ticket
> [bcdef522] which in turn refers to artifact ID [6543cfe], is the
> migration tool expected to chase and re-point  all those links when all
> the hashes change?

That can be handled by adding optional aliases. Old links will remain
valid even though the integrity of the repository depends on the
stronger hash.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Weird behavior on Chisel, perhaps a version mismatch?

[Joerg Sonnenberger]

On Wed, Nov 29, 2017 at 12:02:11PM -0500, Richard Hipp wrote:
> On 11/29/17, Jacob MacDonald  wrote:
> > Ah, I see. It seems strange to me that the policy didn't get set when I
> > cloned. How can I avoid the same thing in the future and is there any way
> > to change the artifacts in the local repository to SHA3?
> 
> There is no good way to change historical artifacts from SHA1 to SHA3.

There could be a loss-less one-time migration of the whole repo and even
an incremental mirror mode that supports keeping SHA1 and SHA3 repos in
sync. It's certainly much simpler than keeping a git mirror in-sync :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Repository size - Fossil v. Git

[Joerg Sonnenberger]

On Mon, Nov 27, 2017 at 08:53:06PM -0500, David Mason wrote:
> Does it stay that size with moderate activity, or does it start growing
> significantly?

Incremental fastimport isn't that bad, but occassional repacks would
still help. Of course, github doesn't allow triggering those remotely
and the only option is to delete the repo and repush from scratch.
...which in turn breaks any forks. Basically, if you convert a larger
existing repo with fastimport, remember to repack aggressively before
pushing to github.

> Does the pack format slow it down, or speed it up?

Neither compared to fossil. I don't think either storage format is
optimized for reducing seeking (which matters for spinning disks) and
I'm not sure how well sqlite benefits from mmap. Everything else is
mostly a function of the delta chain length.

> Given that the Git version only has 93% of what the Fossil repo has, I'd
> say Fossil is doing quite well.

The sqlite tree is also both flat and shallow, which is one of the
biggest costs for the NetBSD repos.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Repository size - Fossil v. Git

[Joerg Sonnenberger]

On Mon, Nov 27, 2017 at 03:28:37PM -0500, Richard Hipp wrote:
> I didn't try any repacking.  I merely ran "git clone" then looked at
> the packfile in .git/objects/pack.  You would think that the server
> would want to do an aggressive repack before sending the packfile
> across a clone, to save bandwidth.  But maybe GitHub values CPU cycles
> more than bandwidth...

git import is writing pretty dumb packs. Lots of redundancy in it,
so it's not really that surprising. It's kind of similar to the effect
of avoiding "fossil rebuild --compress" or Mercurial's generic delta.
Cloning IIRC will mostly use the deltas as recorded, it doesn't
recompute them. GitHub in generally naturally avoids CPU load as much as
possible, since it is one of the more expensive parts of running in the
cloud.

> Your git-foo is much greater than mine, Joerg.  Can you please clone
> https://github.com/mackyle/sqlite.git and see if you can get the
> packfile to come out smaller?

git repack -A -d --depth=500 --window-memory 4g -f -F
gives me around 43MB for .git.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Repository size - Fossil v. Git

[Joerg Sonnenberger]

On Mon, Nov 27, 2017 at 02:28:37PM -0500, Richard Hipp wrote:
> TL;DR:  A Git packfile for SQLite is about 52% larger than the
> equivalent content in a Fossil repository.

Did you run repack with aggresive settings? I.e. with -A -d -f and large
--depth and --window-size settings? Especially if the original migration
wasn't done well, the pack files are often quite redundant.

Your numbers really don't match my experience, i.e. what I see is about
a factor of 2 to 2.5 larger Fossil repos.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil-NG ideas

[Joerg Sonnenberger]

On Wed, Nov 22, 2017 at 07:01:31PM -0500, Richard Hipp wrote:
> > (2) Store true differential manifests.
> 
> I'm thinking that Fossil-NG will probably do like Git and store
> separate artifacts holding the content of each directory.  (Git calls
> these "Tree Objects").  I need to do more research, but I'm thinking
> that for most check-ins, only one or two directory artifacts will
> actually change, even if there are dozens or hundreds (or multiple
> thousands as in NetBSD) of directory artifacts in a single check-out.

I'm not sure splitting it up on a per-directory level well help that
much. If you change a number of files in the same directory, they will
have the same prefix and compress well (even without doing any adhoc
prefix compression schemes). Changes are quite high that larger commits
will scatter around many directories, it's a common theme i.e. for API
changes in larger projects. The primary reason for storing tree objects
in something like git is the lack of better indexing structures, I would
say. In short, I would strongly distinguish between storage format and
working format, optimizing the former for compactness and building up
the latter when necessary.

> >
> > (3) Make cluster manifests non-permanent artifacts.
> >
> 
> I'm thinking of doing away with cluster artifacts entirely, and
> managing sync in some other way.

Having something like cluster artifacts can be useful, i.e. consider the
typical case of one central main server and many clients pulling from
it. If the server builds a ephemeral cluster ever so often and the
clients remember the last cluster they got, they can just ask the server
"what changed since this point". If the implementation of the
communication process is actually asynchronous, adding a few round trips
is not that bad as long as the waiting time is minimized. If the first
request from the client starts with the last "cluster", the server can
answer with the commits added on top. Client can filter out those it
already has and request the new ones. While it gets the new commits, it
can start requesting the file changes. Non-commit artifacts would
essentially work like new leaf commits. A new "cluster" can either
reference an existing one as delta or be send from scratch. It would
still allow the server to do regular garbage collection.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil-NG ideas

[Joerg Sonnenberger]

On Mon, Nov 20, 2017 at 01:33:11PM -0700, Warren Young wrote:
> I see a new wiki article:
> 
> https://www.fossil-scm.org/index.html/wiki?name=Fossil-NG

There are two central design flaws in Fossil that affect larger
repositories and those are the repos that primarily benefit from
narrow/shallow clones. Properly addressing them is kind of a requirement
for either.

(1) The need to parse all artifacts on clone. Artificates should be
strongly typed, i.e. the system should at the very least distinguish
fully between "content" blobs and "meta data" blobs. Only the latter
have and should be parsed. This has a number of important implications,
but the easiest is that the number of artificates a rebuild or even just
a sync has to look at goes down by a factor of 2 at the very least. For
something like NetBSD src or pkgsrc, more like a factor of 10 (number of
blobs in total / number of commits, i.e. the average commit touches 10
files).

(2) Store true differential manifests. The current base line approach is
a somewhat crude approximation. It has the advantage that only two
manifests have to be parsed, but it makes the average manifest size much
larger for larger file trees. The same benefit could be obtained by
caching the file list, either every so often like the current base line
or on-demand. The difference is that the cached manifests are not
persistent meta data and don't have to be transfered.

I would also add a point (3) which is kind of related to (1):

(3) Make cluster manifests non-permanent artifacts. They can also
consume a good amount of space and their purpose could be served by a
Merkle tree as well. This is even more important when doing single
branch sync.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Empty file constantly being deleted

[Joerg Sonnenberger]

On Fri, Sep 08, 2017 at 02:49:16PM +0100, Thomas wrote:
> On 2017-09-08 09:48, Joerg Sonnenberger wrote:
> > On Thu, Sep 07, 2017 at 11:39:39PM +0100, Thomas wrote:
> > > What I mean is, someone who's got that file within the checkout folder
> > > automatically causes it to be checked in again, independent of what the
> > > ignore-glob says.
> > 
> > You have to mark it as deleted. Then it will only be checked in (again),
> > if someone actually adds it. Which normally honors the ignore-glob.
> 
> How do I mark a file as deleted?

fossil delete ...

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Empty file constantly being deleted

[Joerg Sonnenberger]

On Thu, Sep 07, 2017 at 11:39:39PM +0100, Thomas wrote:
> What I mean is, someone who's got that file within the checkout folder
> automatically causes it to be checked in again, independent of what the
> ignore-glob says.

You have to mark it as deleted. Then it will only be checked in (again),
if someone actually adds it. Which normally honors the ignore-glob.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Two easy questions

[Joerg Sonnenberger]

On Tue, Aug 15, 2017 at 08:18:08PM +0200, Stephan Beal wrote:
> On Tue, Aug 15, 2017 at 8:09 PM, Steve Schow  wrote:
> 
> > well I’m hoping to have a version that is stamped into the comments of the
> > actual file as well.
> 
> 
> Stamping that version would change the hash. To the best of my knowledge
> that feature cannot be implemented with DVCS (which requires hashing
> (non-serial IDs) for versioning).

It can be done, just replace the keyword expansion with a canonical form
whenever hashing the content. Keeping the original hash and the hash of
the expanded form allows faster working copy modification checks.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] What is the best way to update to a new upstream version

[Joerg Sonnenberger]

On Tue, Jun 13, 2017 at 02:44:31PM -0500, The Tick wrote:
> Thanks, that is what I was looking for. I've set up a test respository and
> things look good until I get to the final "merge" step:
> 
> $ f init F.fossil
> $ f open F.fossil
> $ f add project
> $ f commit --tag 1.5 --branch Official
> $ f tag add --propagate Official 0dd97cd973
> 
> $ f co trunk
> $ f add project
> $ f commit

This part is the problem. You are missing the merge of 1.5 unto trunk.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] What is the best way to update to a new upstream version

[Joerg Sonnenberger]

On Tue, Jun 13, 2017 at 12:17:46PM -0500, The Tick wrote:
> Given a repository based on a previously fetched version of some software
> package and having been modified with local changes, what is the best way to
> update to a more recent version of the upstream package?

The best approach is still to kind of emulate what CVS is doing with
vendor branches. It works best if you start from scratch, but with some
work it can be fixed up later:

- Create a branch off the initial empty commit for this package.
- Put the sources into the correct subdirectory, addremove.
- Commit, possibly with a tag including the package name and version.
- Now merge that branch into trunk.
- Commit local changes on top.

If you want to update:

- Checkout the branch.
- Remove all files, put the sources into the correct subdirectory,
  addremove.
- Commit.
- Merge the branch into trunk.
- Fix any conflicts.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil-users Digest, Vol 113, Issue 18

[Joerg Sonnenberger]

On Sun, Jun 11, 2017 at 08:49:24PM +0100, John Pateman wrote:
> Yes, but its not sourcecode that I am wanting to manage so that won’t wash.

make doesn't care.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil-users Digest, Vol 113, Issue 18

[Joerg Sonnenberger]

On Sun, Jun 11, 2017 at 08:02:08PM +0100, John Pateman wrote:
> I want to try to find an alternative way with fossil if at all possible.

Use a Makefile and recreate the checked in copy based on the in-progress
copy by filtering?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] /dev/null and /dev/urandom not available ?

[Joerg Sonnenberger]

On Mon, May 15, 2017 at 09:15:02AM -0500, Roy Keene wrote:
> Maybe it should open /dev/null and /dev/urandom before chroot()'ing ?

At /dev/urandom doesn't need to be used on newer Linux systems,
getentropy/getrandom provide the same service as system call.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Is there a way to disable the JavaScript requirement for a hosted fossil repo wiki?

[Joerg Sonnenberger]

On Sat, Apr 29, 2017 at 04:23:50PM -0400, Richard Hipp wrote:
> You can further allow users to see hyperlinks without first logging in
> as anonymous by going to Admin/Users and editing the "anonymous" user
> to give that user "hyperlink" privilege.  If you do that, though, your
> site will become infected by robots who will download every historical
> tarball, ZIP archive, "annotation", and diff, sucking up all your
> bandwidth and CPU cycles.  A robots.txt file will not help - the
> offending bots all ignore robots.txt.  Perhaps you can keep the bots
> at bay by setting up a bandwidth-limiting proxy of some kind.

Something like

if ($http_user_agent ~* 
(360Spider|80legs|App3leWebKit|Baiduspider|EasouSpider)) {
return 403;
}
from my nginx.conf is a great help :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Can Fossil preserve file timestamps when opening a repository ?

[Joerg Sonnenberger]

On Wed, Apr 19, 2017 at 08:20:00PM -0700, Richard Hipp wrote:
> On 4/19/17, Martin Irvine  wrote:
> > I would prefer that Fossil preserved the date and time stamp that the file
> > had when it was most recently committed.
> >
> 
> That is an unusual preference, because most people when they do
> 
> fossil update some-older-verion
> 
> expect afterwards to be able to type "make" in order to build that
> older version, but if the timestamps are some point in the past, then
> that make will not work.

CVS distinguishes between initial checkout and update.  For the former,
it will use the commit time of each revision as mtime, for the latter it
will use the current time. That works well enough.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SSL on Mac

[Joerg Sonnenberger]

On Thu, Apr 13, 2017 at 09:48:59AM -0700, Ryan Dingman wrote:
> While building statically linked binaries on macOS is possible, Apple doesn’t 
> officially support it.

You don't need statically linked binaries for this purpose. You only
need to statically link OpenSSL.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Couple of beginner questions

[Joerg Sonnenberger]

On Thu, Mar 30, 2017 at 04:34:57PM +0200, Stephan Beal wrote:
> On Thu, Mar 30, 2017 at 4:30 PM, Warren Young  wrote:
> 
> > I don’t think it’s fair to notable Fossil users like Jörg Sonnenberger
> > that we misspell their names simply because we refuse to give up
> > ASCII-centrism.
> >
> 
> OTOH, Joerg can't (IMO!) expect the majority to change keyboard mappings or
> use app/OS-specific voodoo to type his name.

It's more that I prefer a decent transliteration of my name over the
dumb approach used for most French names :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil_v_2_0 Bug: Abandoning commit due to long lines in Foo

[Joerg Sonnenberger]

On Tue, Mar 21, 2017 at 08:05:33PM -0400, Richard Hipp wrote:
> Those servers are all hosed by OVH at https://www.ovh.com/us/ - to
> whom I have written multiple time and who seem utterly unconcerned.
> Be sure to say bad things about OVH on all your social media posts.

I've filled a complain with the Canadan Spam Reporting Center against
OVH Canada now. Let's see if they are more concerned about it.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Modify username after git import

[Joerg Sonnenberger]

On Fri, Mar 10, 2017 at 12:40:34PM +, Pietro Cerutti wrote:
> On 2017-Mar-10, 12:35, Joerg Sonnenberger wrote:
> > On Fri, Mar 10, 2017 at 11:30:15AM +, Pietro Cerutti wrote:
> > > I would like to change the user associated with commits from
> > > g...@gahr.ch to simply gahr. Do to that, I updated the 'user' column of
> > > the 'event' table.
> > 
> > That's not enough. You have to actually "amend" the commit.
> 
> Thanks Joerg, I understand. Still, I can't seem to figure out how fossil
> recovered the original information.

Because it is stored in the manifests. Most database tables are
effectively a decomposed cache of the manifest contents.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Modify username after git import

[Joerg Sonnenberger]

On Fri, Mar 10, 2017 at 11:30:15AM +, Pietro Cerutti wrote:
> I would like to change the user associated with commits from
> g...@gahr.ch to simply gahr. Do to that, I updated the 'user' column of
> the 'event' table.

That's not enough. You have to actually "amend" the commit.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] The SHA3 transition as firewall

[Joerg Sonnenberger]

On Thu, Mar 09, 2017 at 01:37:35PM -0700, Warren Young wrote:
> On Mar 9, 2017, at 1:03 PM, Richard Hipp  wrote:
> > 
> > If a new artifact Y' which has the
> > same SHA1 hash as Y comes along, it will be discarded, since an
> > artifact with that same hash is already in the repository.
> 
> That can be gotten around with a MITM attack, as I’ve already brought
> up several times on the list.  Many Fossil instances won’t have TLS
> protection against MITM attacks, and those that do have it may be
> weakened by some well-intentioned TLS-busting middlebox or antimalware
> package.

It still only matters if you can *introduce* objects. MITM for a given
repository with sensible content requires a second preimage attachk.
Those are not possible on any kind of (un)realistic budget.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Joerg Sonnenberger]

On Fri, Feb 24, 2017 at 03:54:56PM -0700, Warren Young wrote:
> On Feb 24, 2017, at 10:37 AM, Joerg Sonnenberger <jo...@bec.de> wrote:
> > 
> > On Thu, Feb 23, 2017 at 05:01:56PM -0700, Warren Young wrote:
> >> But now we have new data.
> >> Before, this sort of attack was theoretical only.  Now it’s not only
> >> proven possible, it is already within the ROI budget for certain
> >> specialized attacks; attacks only get cheaper over time.
> > 
> > Actually, the only thing that changed is that the attack now has a
> > decent price tag.
> 
> We also know it can happen before most of our respective careers are
> over, so it isn’t something we can boot down the road to the next generation.

The attack itself hasn't add anything significant to the SHA1 attack
knowledge AFAICT. As such, it is a good implementation of what has been
known already. Seriously, the only real surprise is the price tag.

> > Fossil had "hash collissions" issues for a long time
> 
> Really?  Fossil has a very nice status page showing what your current
> repository is doing in this regard: Admin > Reports > SHA1 Collisions.

Completely different thing.

> But this report only tells you about accidental collisions, whereas the
> SHAttered attack is about creating purposeful collisions.

Ever tried attaching an existing commit manifest to a bug report or
committing it locally and then rebuilding the repo?

> > The new stored blob should be:
> > - hash of the rest of the blob
> > - blob type
> > - content size
> > - content
> 
> What does the content size buy you?

Making it more difficult to use payload-after-random-data attacks that
exploit the block nature of many hash function constructions. This is
the mechanism that allows stripping aligned suffixes from both documents
without changing the status of the colliding hash.


> The SHAttered attack shows that if you’re only after a collision, you
> can maintain the file size while making your collision.

Creating random collisions like in that is only of limited usefulness.
More interesting attacks require tighter control.

> If you’re trying to protect against preimage attacks with this
> modification, the content size isn’t an independent variable with
> respect to the content itself.  I think if you asked a cryptographer,
> they’d tell you it adds nothing to the robustness of the resulting hash.

I didn't claim it to be. It is useful data to have access to as early as
possible and it is a pesky consistency check, even if it doesn't provide
any additional (theoretical) security. But it does help with some
classes.

> I also don’t see what hashing the blob data twice gets you.  The hash
> value changes, but again not as an independent variable.

When you can create a hash collissions for the inner hash, it doesn't
mean you have automatically a collission for the outer hash as well.
You almost certainly do not. At the same time, creating a collission of
the outer hash directly is much harder as you need to also compensate
for the structure of the data to have a useful result. At the very least
it means that the attacker has to do a full hash recomputation in every
cycle and not just update the last block of a shared prefix.

> If you want to make things more difficult, you could throw a timestamp
> in there, as Git does.

What timestamp again? Like the useless size data you just argued
against?

> > can significantly cut down in the parsing time on rebuilds.
> 
> Why?  If you’re trying to find data of a particular type in the DB,
> doesn’t event.type tell you what you want to know without parsing cards?

Well, where does that table get populated from after the initial clone?

> What became of the idea of skipping rebuilds for very large repos, by
> the way?  Fossil rebuilds are strictly optional, aren’t they?

Not at all. You have to rebuild at least once after the clone.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Joerg Sonnenberger]

On Fri, Feb 24, 2017 at 10:32:20AM -0800, bch wrote:
> Are you saing:
> 
> contenthash = sha256(content);
> identifier = sha256 (contenthash . blobtype . conentsize . content);
> 
> "blobtype" == cardtype ?

Yes.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fixing incorrect user names imported from another SCM

[Joerg Sonnenberger]

On Fri, Feb 24, 2017 at 10:27:11AM -0700, Warren Young wrote:
> What table did I miss updating?  Where is “fossil clone” getting the old user 
> names from?

From the blob table :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Joerg Sonnenberger]

On Thu, Feb 23, 2017 at 05:01:56PM -0700, Warren Young wrote:
> Second, there will be those who say we’ve covered all of this already,
> multiple times.  I know, I was there.  But now we have new data.
> Before, this sort of attack was theoretical only.  Now it’s not only
> proven possible, it is already within the ROI budget for certain
> specialized attacks; attacks only get cheaper over time.

Actually, the only thing that changed is that the attack now has a
decent price tag. That's really not all that much, contrary to all the
ranting going on right now.

Fossil had "hash collissions" issues for a long time and I'm not even
talking about this kind of issues. It might be a good idea to change the
storage format for new blobs and optionally provide a one time
conversion option. But I don't have the time to implement that.

The new stored blob should be:
- hash of the rest of the blob
- blob type
- content size
- content

The idea is that object id is the hash of everything. This should be
much more resilient to preimage attacks based on the block structure of
the hash function. I haven't had a chance to talk with a real
cryptographer about this yet, though. Including the blob type makes it
more robust and can significantly cut down in the parsing time on
rebuilds. Including the content size is just another way of making
meaningful preimage attacks somewhat harder.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Google Security Blog: Announcing the first SHA1 collision

[Joerg Sonnenberger]

On Thu, Feb 23, 2017 at 06:12:18PM -0500, Martin Gagnon wrote:
> Seems that Git can store both of them, I beleive it calculate the sha1
> on a combination of the filename and the content or something like that.

No, it stores the object type first, which effectively creates a
different block structure. It doesn't mean that the same type of
computation wouldn't create a conflict to hide objects in git. It just
needs slightly different content.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fast-import crash (mark not declared)

[Joerg Sonnenberger]

On Thu, Feb 16, 2017 at 02:15:25PM -0800, Ross Berteig wrote:
> I'd argue the problem here is with Git.

Actually, git is not at fault here. We do the export chronologically as
it is much easier to do than computing a topologically sorted output.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil export to git broken for duplicate tags

[Joerg Sonnenberger]

On Thu, Feb 09, 2017 at 10:13:21AM -0500, Richard Hipp wrote:
> On 2/9/17, Joerg Sonnenberger <jo...@bec.de> wrote:
> >
> > I disagree. The problem is not on the fossil side and I am not even sure
> > the "specification" of fastexport disallows it. It's really a git issue.
> >
> 
> I agree with you, Joerg.  But if the problem is going to be fixed, it
> will probably have to be fixed on the Fossil end.  I'm guessing the
> git people will not be receptive to fixing it on the git side.

ACK. It would be nice to have an option for only exporting open leaves
(and their parents) for similar reasons, but it is not something I have
time to implement myself.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Branch and Tag names in export

[Joerg Sonnenberger]

On Thu, Feb 09, 2017 at 01:47:43PM +, Roy Marples wrote:
> Hi List!
> 
> In fossil I have this: branch-1.7, tag-1.2
> But when I export I get this: branch_1_7, tag_1_2
> 
> Why is the name mangled like so? Surely the original names should be
> preserved.

It's currently preserving only alphanumeric characters. Noone ever
implemented the full sanitation requirement of git.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil export to git broken for duplicate tags

[Joerg Sonnenberger]

On Thu, Feb 09, 2017 at 01:15:51PM +, Roy Marples wrote:
> Hi List!
> 
> I have a few repos which I sync to git. However, one of them I noticed
> is no longer syncing tags. The fossil repo is here:
> https://roy.marples.name/projects/dhcpcd
> 
> Using fossil-1.37
> 
> fossil export --git dhcpcd.fossil > dhcpcd.export
> git init dhcpcd-git
> cd dhcpcd-git
> git fast-import <../dhcpcd.export
> error: multiple updates for ref 'refs/tags/dhcpcd_6_11_3' not allowed.
> git tag
> # No tags are shown, but it did import a few of them according to stats
> 
> The clue is in the error.
> Now, fossil probably doesn't know which tag to use (first? last?) which
> might be the case, but in that instance fossil should error out, or at
> least print a warning during the export phase.

I disagree. The problem is not on the fossil side and I am not even sure
the "specification" of fastexport disallows it. It's really a git issue.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Question about the file formats.

[Joerg Sonnenberger]

On Tue, Dec 20, 2016 at 08:48:27PM +0200, John Found wrote:
> What makes the binary files different from the text files? The presence or 
> absence of
> 0 bytes does not seems to make serious difference for processing by the same 
> algorithms.

Many text formats allow merging changes from one version to another with
minimal context. E.g. let's say you start from a C file and modify a line
in the middle in your checkout and then update your tree. Someone else
added a new function at the beginning of the file. This creates a
conflict and Fossil will try to resolve it by finding the context of the
line you modified in a similiar place and then readd that change. While
this doesn't work all the time for text files, it has a high chance of
working. Even if it doesn't work i.e. because the changes overlap, it
provides enough information that a user can typically do the same.

The same kind of tooling could be provided for binary formats, but it is
rarely exist directly.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Bug report: Terrible Performance, when Checking in LLVM Source

[Joerg Sonnenberger]

On Sun, Dec 04, 2016 at 09:23:37PM +0100, Karel Gardas wrote:
> On Sun, Dec 4, 2016 at 8:57 PM, Joerg Sonnenberger <jo...@bec.de> wrote:
> > On Sun, Dec 04, 2016 at 08:50:44PM +0100, Karel Gardas wrote:
> >> Otherwise as Nikita recommended, switching off repo checksums helps a
> >> lot, but then make sure you are on the filesystem like ZFS/btrfs which
> >> does that for you transparently and you do not need to do that on the
> >> fossil side.
> >
> > Eh, no. You do not need a file system with automatic hashing. Every
> > single file is still recorded by checksum in Fossil. It is not what the
> > repo checksum option does.
> 
> Errhmm, thanks for correction. Am I right that repo checksum switched
> on means that modified files will be those where checksum stored and
> checksum computed from the file on fs is different? And once you
> switch that off, you rely purely on comparison of modification time on
> file in fs and I guess stored modif time in repo db? If so, then
> indeed I've been completely mistaken and thank you very much for your
> kind correction. If however I'm still off, I would appreciate
> reference to some material explaining repo/chksums business in fossil.

No. What repo checksum does is compute a separate checksum over the
concatenation of all files.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Bug report: Terrible Performance, when Checking in LLVM Source

[Joerg Sonnenberger]

On Sun, Dec 04, 2016 at 08:50:44PM +0100, Karel Gardas wrote:
> Otherwise as Nikita recommended, switching off repo checksums helps a
> lot, but then make sure you are on the filesystem like ZFS/btrfs which
> does that for you transparently and you do not need to do that on the
> fossil side.

Eh, no. You do not need a file system with automatic hashing. Every
single file is still recorded by checksum in Fossil. It is not what the
repo checksum option does.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Linux binary downloads

[Joerg Sonnenberger]

On Sun, Nov 20, 2016 at 05:13:56PM +0200, Emil Totev wrote:
> Furthermore, even the old 1.35 binary does not completely work on
> CentOS 7 - 'fossil clone' returns  "bad hostname lookup" for otherwise
> perfectly resolvable and pingable hosts.

This part is moderately easy to explain. "Statically" linked glibc
binaries don't include pretty much all the NSS functionality, but
instead try to hook up the normal dynamic versions. So if your glibc
mismatches whichever version was used to build the binary, you are out
of luck.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Download v1.36 Linux x86 tar.gz actually contains 64bit binary

[Joerg Sonnenberger]

Dear "K. Fossil user",
it's been said before. Please keep your aggressive attitude and
platitudes to yourself. You are just annoying and distracting people.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Why we should NEVER use inetd/xinetd

[Joerg Sonnenberger]

On Mon, Oct 24, 2016 at 02:36:23PM -0600, Warren Young wrote:
> On Oct 24, 2016, at 2:16 PM, Joerg Sonnenberger <jo...@bec.de> wrote:
> > 
> > On Mon, Oct 24, 2016 at 09:56:45AM -0600, Warren Young wrote:
> >> The only common exception is this recent trend of replacing old,
> >> bloated software that grew organically over decades with well-focused
> >> fresh alternatives.  (e.g. BIND vs nsd/unbound, LibreSSL vs OpenSSL,
> >> Postfix vs Sendmail, etc.)
> > 
> > Bad examples. BIND was rewritten from scratch on a regular base
> 
> Really?  The only time BIND was ever completely rewritten to my
> knowledge was for BIND 9, which is now 16 years old.

Either BIND4 or BIND5 was a complete rewrite as well.

> More to the point, nsd + unbound still isn’t as functional as BIND 9,
> meaning there are fewer places for bugs to hide.

Well, the primary difference is that nsd by design is not caching. That
rules out all the cache poisoning attacks, one of the biggest problems
in bad BIND deployments. Of course, if you followed DNS best practises,
you would have been using authoritive-only servers with BIND as well...

> > LibreSSL doesn't fix any of the fundamental issues of OpenSSL
> 
> It fixes at least one, being the OpenSSL had turned into a kind of
> crypto dumping ground, so that the library supports virtually every
> weird crypto idea that’s ever been tried out around the SSL space for
> the past couple of decades.

So? The crypto primitives rarely have any issues at all. There is a
somewhat questionable recent trend to make all kinds of timing "attacks"
a major thing, but that's about it.

> LibreSSL strips a whole lot of that out, so that it only supports
> modern TLS, no legacy SSL or nonstandard extensions, and then only the
> parts that are currently well-regarded, so that a program linked
> against it is not vulnerable to any of the bugs in those rarely-used
> parts of OpenSSL.

I know the marketing language as well. It completely forgets that the
primary reason why a lot of software was still using OpenSSL until
recently was exactly this compatibility. SSLv3 hasn't even decomposed
properly yet. The reality is that if your customer facing web server
rejects a non-trivial number of potentially paying customers, people are
going to hang the admin. It doesn't matter whether there are some
theoretical security issues if a company is losing enough revenue. There
are still a lot of devices installed that only support SSLv3. There is a
reason why the deprecation process took so long.

> There have been cases where a program linked against OpenSSL was vulnerable 
> but not when linked to LibreSSL:
> 
>   https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3566
>   https://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2014-3567

...and there have been cases where LibreSSL riped things out and broke
things by that.

> If you simply mean that there is a certain amount of horridness to the
> OpenSSL API and that LibreSSH shares this, then yes, that is true. 
> The only fix is a redesign, which means you break compatibility with
> all the programs that currently depend on OpenSSL or LibreSSL.

It is not a certain amount. The API is one of the worst thing ever done.
LibreSSL hasn't improved things at all by claiming to be OpenSSL 2.x or
some other random junk.

> Ideally, LibreSSL is just a bridge to something better, but knowing the
> way software inertia works, I wouldn’t bet on us getting to that
> something-better any year soon.

Better alternatives exist. Many of them were not feasible for a long
time due to the SSLv3 constraint, but that is finally gone. One of the
problems is that dealing with complex highly conditional data structures
is a mess in C. It gets worse when people assume that char arrays are
NUL-terminated etc. At the same time, library developers want to
constraint the input without understanding the way how X509 is used in
practise. Correct TLS use is more than just "use this certificate and
encrypt that data"...

> > Postfix is more secure than (old) sendmail due to a different
> > architecture. :)
> 
> Yes, Postfix is a pile of much smaller cooperating programs rather than
> a monolithic program as with sendmail, each of which may be debugged
> and privilege separated from the rest, which is exactly my point.
> (“…well-focused fresh alternative…”)

sendmail has also been reworked to fix some of the implementation
problems. It is telling that the last CVE is from 2014 and the one
before was from 2009. That's a lot better than some of the replacements
designed for "security". Which brings back the original point of
"mature" != "insecure".

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] features I'd like to have in fossil

[Joerg Sonnenberger]

On Mon, Oct 24, 2016 at 10:35:46AM -0600, Warren Young wrote:
> As far as I can tell, syncing to a remote repository locks the entire
> remote repository, making it *worse* than Fossil, not better.  (SQLite
> concurrency locks only single tables, not the whole database, and then
> only for writes, not reads.)  

If you use WAL mode, you can have a single writer locking the whole
database for the transaction, but pure readers can continue fine. I
would have to check how this interacts with the temporary tables needed
by the xfer code for pulls, e.g. whether pulls need a write transaction
or not.

The only scalability issue on the hosting side I really hit more than a
couple of times is related to certain Chinese spiders following every
link they can and ignoring robots.txt. That's an issue for any version
control system though. I can assure you that repository hosting is the
smallest issue, even for very large repositories.

I have some old presentations still online talking about the NetBSD
repository and the fossil conversion, that's likely still one of the
biggest fossil repositories around:
http://www.sonnenberger.org/archive/presentations/fossil/
http://www.sonnenberger.org/archive/presentations/fossil2/
(content mostly the same, from 2011)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Why we should NEVER use inetd/xinetd

[Joerg Sonnenberger]

On Mon, Oct 24, 2016 at 09:56:45AM -0600, Warren Young wrote:
> The only common exception is this recent trend of replacing old,
> bloated software that grew organically over decades with well-focused
> fresh alternatives.  (e.g. BIND vs nsd/unbound, LibreSSL vs OpenSSL,
> Postfix vs Sendmail, etc.)

Bad examples. BIND was rewritten from scratch on a regular base,
LibreSSL doesn't fix any of the fundamental issues of OpenSSL and
Postfix is more secure than (old) sendmail due to a different
architecture. :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] disabled due to excessive bounces (again?)

[Joerg Sonnenberger]

On Sun, Oct 16, 2016 at 03:27:02AM +, K. Fossil user wrote:
> Try to convince people to use Fossil after that : very hard.

Can you please take your negative and hostile attitude somewhere else?
It seems clear now that you have a number of problems on your side.
From a mail server that doesn't work properly for whatever reason, over
a build environment you don't want tell us about to whatever random FUD
you are repeating based on unnamed coworkers. I'd kindly request you to
follow basic rules of productive communication. If you have a problem
and you don't want it solved, don't bother telling us about it. If you
do want it to be improved, it doesn't help if you don't tell us what it
actually is.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] autosetup: hidden autoconf/automake compatibility

[Joerg Sonnenberger]

On Sat, Oct 15, 2016 at 01:01:17PM +0900, Osamu Aoki wrote:
> As I see output of "./configure --help" on autoconf/automake based code,
> I see the following:
> 
>   --enable-silent-rules   less verbose build output (undo: "make V=1")
>   --disable-silent-rules  verbose build output (undo: "make V=0")

Frankly, I find those rules supremely annoying. They just add overhead
to debugging build problems for no good reason.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] mark offeset: fossil export and import and re-export

[Joerg Sonnenberger]

On Sat, Oct 15, 2016 at 05:59:50AM +0900, Osamu Aoki wrote:
> These 2 files should be the same ... I hoped but reality is a bit
> complicated.  They are very similar but the mark field is strange.

No, the marks are derived from the rid and those again depend on the
order in which blobs are added. I'm not at all surprised if the
export/import cycle does change the order. If you want to compare two
fossil repositories for being equivalent, the easiest approach would be
to take all non-cluster entries in the blob table, print the uuid and
sort the result.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Files named "AUX" on Windows

[Joerg Sonnenberger]

On Tue, Oct 04, 2016 at 12:15:58PM -0400, Richard Hipp wrote:
> See https://www.fossil-scm.org/aux-test/doc/trunk/aux.md
> 
> Apparently if a Fossil repository contains a file whose basename is
> "aux", then an attempt to open or check-out that repo fails with an
> error.  Only the basename needs to be "aux".  The full name can be
> things like "src/aux.c" or "doc/aux.txt".

aux is a device name, nul and a couple of other things should trigger
the same behavior. I can't remember if there was ever a way to
workaround it. One of the funny ways to kill a Win9x installation was to
patch the io.sys and rename aux to windows...

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Preferred way of dealing with upstream projects maintained in different VCSes?

[Joerg Sonnenberger]

On Wed, Jun 15, 2016 at 09:02:54PM -0500, lvh wrote:
> I’m interested in exploring the possibilities of moving to a Fossil
> monorepo at $dayjob. One of the things I’m not sure how to deal with
> appropriately is dealing with projects managed upstream in some other
> VCS (which, de facto, appears to be just git).

The primary question is whether you want to apply local changes and how
often you want to update. If there is any chance for the former, it is
often a good idea to follow the vendor branch approach from CVS to a
degree. That is, create a branch from the initial (empty) revision with
the unmodified upstream state. Merge the new branch head into your
normal development branch (e.g. trunk) and adjust as necessary. For
updaptes, go back to the vendor branch, commit the new version and merge
it again to your development branch.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] HOWTO: TLS-protected Fossil with nginx and Let's Encrypt

[Joerg Sonnenberger]

On Wed, Jun 01, 2016 at 10:26:17PM -0700, David Simmons wrote:
> 1. The fossil website information on using nginx with fossil is not
> "helpful" in that scgi had numerous problems, and ultimately serving via
> proxy configuration was straightforward and reliable.

This is absolutely not my experience. I found SCGI to be much more
reliable than using HTTP proxy. That might partially be a side effect of
having GB+ sized repositories.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] multiple open leaf on closed branches?

[Joerg Sonnenberger]

On Tue, May 31, 2016 at 01:50:01PM +0200, Luca Ferrari wrote:
> So the branch 'feature-installazioni' is closed, and is merged.
> I'm a little confused.

A merge doesn't automatically close a branch, it must be instructed so.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] multiple open leaf on closed branches?

[Joerg Sonnenberger]

On Mon, May 30, 2016 at 11:49:47AM -0400, Richard Hipp wrote:
> On 5/30/16, Luca Ferrari  wrote:
> > Hi all,
> > I suspect I did not get right the multiple open leaf warning:
> >
> 
> Maybe the LEAF table in the database file is somehow out of sync.
> Have you tried running "fossil rebuild" to see if that clears the
> problem?

Or the more light-weight "fossil leaves --recompute" for starters.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil 2.0: rethinking extras, addremove, and clean

[Joerg Sonnenberger]

On Tue, May 24, 2016 at 07:06:18AM -0600, Warren Young wrote:
> On May 22, 2016, at 2:49 PM, Andy Goth  wrote:
> > 
> > On 5/22/2016 3:22 PM, Ron W wrote:
> >> The build
> >> systems "knows" how to clean up files it creates. Any other files not
> >> managed by the VCS are the responsibility of the user.
> > 
> > It's also my preference that the build system account for 100% of its
> > generated files and know how to clean them up.  But I rarely get the
> > chance to do the build system or beat it into shape.  So I'm happy to
> > have a version control system that can also handle the job.
> 
> I wouldn’t mind seeing all but “extras” go away.  You should be able
> to run your build system’s clean command, then run “fossil extras” to
> learn what files your build system still needs to be told about.

Having to clean the build before finding out what potential new files
exist is extremely unfriendly...

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Workaround for the missing 'squash' command?

[Joerg Sonnenberger]

On Thu, May 12, 2016 at 09:43:34AM +0200, Joerg Sonnenberger wrote:
> On Thu, May 12, 2016 at 12:24:20AM +0200, Marko Käning wrote:
> > How to achieve a git'ish squash when merging a (private) branch into trunk?
> 
> You can use update+commit to get the effect.

checkout+commit I meant :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Workaround for the missing 'squash' command?

[Joerg Sonnenberger]

On Thu, May 12, 2016 at 12:24:20AM +0200, Marko Käning wrote:
> How to achieve a git'ish squash when merging a (private) branch into trunk?

You can use update+commit to get the effect.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Feature Request: Automatic vacuuming of .fslckout (or _FOSSIL_)

[Joerg Sonnenberger]

On Tue, May 10, 2016 at 10:34:29AM -0700, Christopher M. Fuhrman wrote:
> I've noticed that the .fslckout file can grow exceptionally large,
> especially for larger repositories such as pkgsrc[1].

I think the real problem is that update is rewriting the mtime table,
isn't it?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] update vs checkout

[Joerg Sonnenberger]

On Sat, Apr 30, 2016 at 11:30:08PM -0600, Scott Robison wrote:
> Note that you have to merge mybranch back to trunk, otherwise all the
> changes made in mybranch will live happily over there forever. Once you
> updated to trunk, you abandoned all committed work from the other branch.

You can also mark the leaf as closed.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Colored output on console

[Joerg Sonnenberger]

On Mon, Apr 25, 2016 at 05:22:54PM +0200, Stephan Beal wrote:
> Yes, colorization is commonly-request feature, but is also not trivial to
> do portably, which is probably why relatively few people are interested in
> implementing it.

Uses terminfo is not that difficult and there are even separate
implementations :)

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Robustness checks

[Joerg Sonnenberger]

Hi all,
while working on an update for my cvs2fossil tool, I was doing some
mistakes ^W^W^W fuzzing the output and able to reliably crash fossil.
In rebuild_step, blob_delta_apply is called without checking the return
value. If the delta is for some reason valid compressed data, but not a
valid delta, it returns directly and leaves next uninitialized. The
reassignment in the tail recursion part then corrupts pBase. The only
part I'm not sure is how to best deal with this.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] HOWTO: TLS-protected Fossil with nginx and Let's Encrypt

[Joerg Sonnenberger]

On Thu, Apr 14, 2016 at 03:32:48PM -0600, Warren Young wrote:
> STEP 1: Split your “server” configurations

I don't think this is necessary at all.

> STEP 2: Prepare for the Let’s Encrypt challenge/response sequence

This part can just statically go into the same server block, no need for
a separate file.

> STEP 3: Write the wrapper script

Personally, I would recomment just using the SSH FUSE binding and doing
the dance from a separate machine. No need to have letsencrypt and all
dependencies running on a server.

> STEP 5: Create the base SSL/TLS configuration
> --
> 
> We extracted the site configuration from our server { } blocks above because 
> we now need to create a second such block for each site that nginx serves.
> 
> server {
> include local/ssl;
> include local/site;
> }
> 
> That is, instead of including the letsencrypt-challenge file — since we only 
> serve the Let’s Encrypt challenge/response sequence via HTTP — we include the 
> following SSL configuration file:
> 
> listen 443 ssl;
> 
> ssl on;

No need for "ssl on" in combination with the ssl on the listen line --
means all the config can be shared with plain HTTP.

> Let’s Encrypt certs only last for 90 days, which means it’s an ongoing
> task to keep this up-to-date.  Until Let’s Encrypt learns about safe
> nginx configuration file modification, it’s a manual process.  (With
> Apache, letsencrypt-auto sets up a background auto-renewal process so
> you can’t forget to renew.  You could script this manually for nginx,
> if you wanted.)

Given that it is *not* supposed to change the configuration on renewal
at all, that's a non-issue.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil mentioned on HN

[Joerg Sonnenberger]

On Thu, Dec 17, 2015 at 06:57:06PM +0300, Konstantin Khomoutov wrote:
> On Thu, 17 Dec 2015 13:16:43 +0100
> Joerg Sonnenberger <jo...@britannica.bec.de> wrote:
> 
> > Now the tricky part is this can be done *without rewriting history*.
> > Essentially, you can (semi-automatically) reapply all changes on top
> > of the new commit and record which commit they were originally. This
> > allows three things:
> > (1) Tight main line with keeping incremental stages.
> > (2) Preservation of what commits originally happened in case you need
> > the full audit trail because there was a subtil merge fault or the
> > like.
> > (3) It allows automated follow-up rebases for 3rd parties that already
> > got the initial changes. I.e. you can safe push your work-in-progress
> > and still transplant it later.
> > 
> > "git rebase" only really allows the first point.
> 
> Not quite.  Well, `git rebase` does indeed replace the tip of the
> branch it operated on with a set of changed commits, but this counts as
> a so-called "drastic head movement" in the Git parlance.  Such
> movements are recorded in the so-called "reflog" ("reference log" --
> with refs or references being heads (branches) and tags), so as soon as
> a rebase operation completes successfully (as opposed to having been
> aborted using `git rebase --abort` where the branch being operated on
> is left intact), you can retreive the previous tip of the affected
> branch from the reflog.  The reflog keeps about 30 days worth of such
> changes by default which is more than enough to cover the "oops!"
> situations.

The very need for reflog and the 30days recovery frame is one of the
saddest part of the git design. It is an arbitrary implementation detail
and not really related to the way "git rebase" works.

> As to "without rewriting history" bit, this might be a terminological
> issue but in a (typical) DVCS you simply cannot cherry-pick or
> otherwise (re-)apply an existing commit to some tip to produce another
> commit without essentially rewriting it because the commit hash includes
> the commit creation date.  And once you've changed a commit's hash, the
> next commit being applied on top of this one will have its hash changed
> also because it will refer to the changed hash-name of its parent
> commit, and this "link" is hashed as well.

The part you are still missing is that the original commit never
got lost. That's why am I talking explicitly about *recreating* and not
*rewriting*. It's like kind of like editing the commit message -- that
doesn't change the original commit either, but creates an update entry
in fossil that just says "well, take that commit, but show *this* commit
message".

> Another point is that when you rebase (or "linearize"), the new upstream
> tip might actually contain changes which will make some or all of the
> commits in the series being rebased/linearized be apply with some
> fuzz, and in complicated cases they might even fail to apply.  In this
> case, even "the contents" (as opposed to metadata) of the
> rebased/linearized commits will be re-written quite literally.  In the
> sense that if you had a commit A on top of upstream's U, and then
> rebased A on top of the updated U' to produce A', the differences A':U'
> and A:U might be substantial.

Yes, some commits might have been getting obsoleted. That's why it can't
be a fully automated process. So you start transplanting the other
commits and either include a dummy entry for nothing at all. Unless the
3rd party was explicitly basing the changes on that now irrelevant
commit, the transplating of their changes can still continue.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Is fossil export known to be broken?

[Joerg Sonnenberger]

On Sun, Dec 13, 2015 at 09:57:44PM +0100, Piotr Orzechowski wrote:
> I guess it would be better to change help message about rebuild then.
> As of now, it says "Run this command after updating the fossil
> executable in a way that changes the database schema." So I would
> expect to see some information about such need in release notes.
> Perhaps it would be better to add to help that it is a good practice
> to run this command after each update. But then, why wouldn't rebuild
> happen automatically?

I wouldn't advise to run it after every update, especially for large
repos. On something like NetBSD's src.fossil, it can take hours even on
a fast machine.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] openBSD package download request: zip to tar.gz

[Joerg Sonnenberger]

On Fri, Dec 11, 2015 at 01:54:21PM -0700, Warren Young wrote:
> On Dec 11, 2015, at 1:38 PM, jungle Boogie  wrote:
> > 
> > Man page for tar:
> > http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man1/tar.1?query=tar
> 
> Scroll down to AUTHORS and HISTORY: it’s not bsdtar.
> 
> Pity.
> 
> NetBSD doesn’t ship bsdtar in base, either.

Yeah, it is on my list of things I need to fix :) NetBSD does have
unzip(1) though.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Password prompt with SSH protocol on Windows?

[Joerg Sonnenberger]

On Thu, Dec 10, 2015 at 03:59:29PM +0100, Daniel Dumitriu wrote:
> Still there seems to be another problem with fossil: it does not pass
> the password to plink when it was given on the command line as in
> user:pass@host:port. Maybe something along these lines ("-p pass")?

I would call that a security nightmare. Unless I am missing some option
for plink.exe to say "use the password from this file descriptor",
that's a big N.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil sync doesn't sync

[Joerg Sonnenberger]

On Wed, Dec 02, 2015 at 05:54:06AM -0500, Richard Hipp wrote:
> On 12/2/15, j. van den hoff  wrote:
> >
> > thank you. sorry if this has been discussed/explained before: this means,
> > there still is demand for that option? why? is there still a bug out
> > there? because if not, it seems whatever `--verily' is doing here should
> > be done all the time in order to avoid incomplete syncs in the first place?
> >
> 
> There are no known bugs.  --verily is a defensive measure in case some
> currently unknown bugs appear in the future.

The problem for the NetBSD repo was if the local repository on a pull
has a large list of phantoms and the server would decline all of them as
unknown, the client wouldn't continue with the next set of phantoms, but
just give up.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] VCS Theory

[Joerg Sonnenberger]

On Thu, Nov 19, 2015 at 11:51:41AM -0800, Scott Doctor wrote:
> I am looking for information about the theory of VCS that is being used for
> systems such as Fossil, Git... Not so much the how-to-use, but the concepts
> and issues.

Both TLA and Darcs had quite a bit of write up when I looked a decade
ago.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] git LFS

[Joerg Sonnenberger]

On Tue, Oct 27, 2015 at 05:01:09PM -0400, Ron W wrote:
> On Tue, Oct 27, 2015 at 4:29 PM, Abilio Marques  wrote:
> ...
> 
> > As far as I can see, binary files get uploaded into a web server using
> > PUT, and then a reference is made into the repo. That way git doesn't deal
> > with useless DIFFs of such files. Then, when a checkout is made, it
> > downloads the file over http/https. Seems like a neat solution.
> >
> > I guess that fossil users with a bunch of local repos (like myself)
> > wouldn't benefit if they had to load a web server. But for other people,
> > perhaps... Perhaps if instead of a web server, the file just gets copied in
> > a directory... I don't know...
> >
> 
> Fossil's internal diffing algorithm handles binary files quite well. Also,
> I recall reading somewhere that Fossil is able to determine when just
> storing the new content would be more efficient.

Same for git really.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Css not loading when adding a '/' to url

[Joerg Sonnenberger]

On Sat, Oct 03, 2015 at 03:36:03PM +0200, Eduardo Morras wrote:
> When I go to http://www.sqlite.org//src/doc/trunk/README.md instead of
> http://www.sqlite.org/src/doc/trunk/README.md, the page loads without
> css and link I follow don't load it neither and the additional '/' is
> not corrected.

Well, you are asking for a completely different resource, so why do you
expect it to work?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Out-of-order timestamps cause invalid export files

[Joerg Sonnenberger]

On Mon, Sep 28, 2015 at 10:12:25PM +0200, David Given wrote:
> There ought to be a cleverer way to do this than just relying on the
> timestamp (which can be wrong). Isn't there an SQL example which uses a
> recursive query to walk the commit tree? That way we'd be guaranteed to
> get a correct order.

Sure, but those ways are significantly more expensive :(

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Why Hash

[Joerg Sonnenberger]

On Fri, Sep 11, 2015 at 04:57:46PM -0600, Warren Young wrote:
> I wonder if this is an implementation detail leaking through into the
> UI, though.  Under what conditions, except for Noam’s contrived example
> with hardcoded dates, is there a useful distinction between “hash” —
> implying a number that you could reliably recompute given all the input
> data — and “random number”?

The SHA1 hash is just a deterministic random number for this purpose.
Making it deterministic just means that the identifier can also double
as checksum.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] [possible bug] pull from empty repo to empty 'open' dir deadlocks repo

[Joerg Sonnenberger]

On Sun, Jul 19, 2015 at 10:45:29AM +0300, Alexey V Gorshkov wrote:
 sorry, missformatted command sequence text
 
 mkdir fos
 cd fos
 fossil init
 test.fossil
 mkdir t
 cd t fossil
 open ../test.fossil
 fossil pull
 ../test.fossil

Assuming this is fossil open ../test.fossil; fossil pull
../test.fossil, you are trying to pull from the repository itself,
which is naturally a dead lock. Why do you expect that to work (or even
provide something useful)?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Automatically put version / checkout infomation into source code on commit

[Joerg Sonnenberger]

On Tue, Jul 14, 2015 at 09:51:20AM +0200, j. van den hoff wrote:
 incidentally I'm using nearly exactly the same approach including the call
 to `status' (as I was under the same misconception that it should be the
 most efficient way to get at the sha1 hash). so I have just switched to
 `info' as well. but calling `status' two times in a row, one sees that the
 second call is much faster, so I presume `status' is reading a lot of stuff
 which then is cached. since the information displayed by `info' and `status'
 seems virtually identical: what is the reason for the high latency of a
 first `status' call?

status checks whether any files are changed. As such, it has to stat(2)
everything first.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Tue, Jun 23, 2015 at 08:32:13AM -0700, bch wrote:
 Thanks Joerg. I'll re-sync shortly. Did you happen to test if it had any of
 the 0 byte blobs I had before (and the 1 I still have)?

Likely. As I said, I don't really know why it sometimes creates those :(

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Tue, Jun 23, 2015 at 11:33:31AM -0700, bch wrote:
 Still no dice:
 
 kamloops$ fossil pull --verbose
 Pull from http://netbsd.sonnenberger.org
 Bytes  Cards  Artifacts Deltas
 Sent:9546202  0  0
 Received:  78  2  0  0
 Pull done, sent: 5082  received: 333  ip: 135.28.13.11
 kamloops$

Hm. I wonder if the problem is that your system is asking for artefacts
that no longer exist my public copy and doesn't loop when it doesn't get
an answer for any of those. --trace would help for that.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Tue, Jun 23, 2015 at 01:05:00PM -0600, Andy Bradford wrote:
 Thus said bch on Tue, 23 Jun 2015 11:58:35 -0700:
 
  Good idea (I presume you mean sqltrace):
 
 More likely he meant --httptrace which will reveal the HTTP transactions
 during the pull operation (e.g. what was sent/received).

Yeah, --httptrace.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Tue, Jun 23, 2015 at 12:31:20PM -0700, bch wrote:
 I tried that (and sent response that only went to Andy (which I think
 is not first time has happened between Andy, fossil-users, and
 myself)).
 
 
 kamloops$ fossil pull --httptrace
 Pull from http://netbsd.sonnenberger.org
 Round-trips: 1   Artifacts sent: 0  received: 0
 Pull done, sent: 9759  received: 331  ip: 135.28.13.11
 kamloops$
 
 
 But even on a repo that does pull artifacts, I get no extra output:
 
 kamloops$ fossil pull --httptrace
 Pull from http://pkgsrc.sonnenberger.org
 Round-trips: 3   Artifacts sent: 0  received: 117
 Pull done, sent: 6543  received: 65534  ip: 135.28.13.11
 kamloops$
 
 
 What am I missing ?

There should be a trace file for teach round trip.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fwd: DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Tue, Jun 23, 2015 at 01:40:04PM -0700, bch wrote:
 Ugh. Again, include fossil-users@

Right, that's the situation I meant. fossil should be retrying with the
next set of missing changes and not stop when it can no longer make
progress on the current set.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Mon, Jun 22, 2015 at 01:44:12PM -0700, bch wrote:
 W/ latest fossil from tip of [trunk], a pull now looks roughly like
 this (note, no reported errors or warnings, no looping like before,
 but still not actually working properly):

Well, I've rebuild the repository, so it should have no artifacts like
before.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SQLITE_BUSY ?

[Joerg Sonnenberger]

On Thu, Jun 18, 2015 at 01:45:24PM -0400, Ron W wrote:
 On Wed, Jun 17, 2015 at 3:17 PM, Joerg Sonnenberger jo...@britannica.bec.de
  wrote:
 
  On Wed, Jun 17, 2015 at 12:10:36PM -0400, Richard Hipp wrote:
   On 6/17/15, Jan Danielsson jan.m.daniels...@gmail.com wrote:
   Out of curiosity; why aren't pulls 100% read-only on the server?
   
  
   The server might decide to create a new cluster artifact
   (https://www.fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki#cluster
  )
   to help it with the sync.
 
  But it should not have to do that on a pull.
 
 
 See section 5.1, step 5 in
 https://www.fossil-scm.org/index.html/doc/trunk/www/sync.wiki
 
 Of course, if the read-only repo is a snap shot clone of a repo with
 fewer than 101 unclustered  artifacts, then new clusters won't be created.

I'm aware of the sync protocol. My point is that all clusters should
already exist on the public master repository as part of other sync
operations and I would strongly argue that this is the case that should
be optimised for. It is a pity that the nature of the sync protocol
doesn't make it possible to enforce this more strongly.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SQLITE_BUSY ?

[Joerg Sonnenberger]

On Wed, Jun 17, 2015 at 12:10:36PM -0400, Richard Hipp wrote:
 On 6/17/15, Jan Danielsson jan.m.daniels...@gmail.com wrote:
  On 17/06/15 11:38, Joerg Sonnenberger wrote:
  [---]
  Sadly, a plain pull is not 100% read-only, so WAL doesn't help avoiding
  such problems.
 
 Out of curiosity; why aren't pulls 100% read-only on the server?
 
 
 The server might decide to create a new cluster artifact
 (https://www.fossil-scm.org/fossil/doc/trunk/www/fileformat.wiki#cluster)
 to help it with the sync.

But it should not have to do that on a pull.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SQLITE_BUSY ?

[Joerg Sonnenberger]

On Tue, Jun 16, 2015 at 11:04:07PM -0700, Matt Welland wrote:
 We see these (or similar) occasionally when the filesystem gets slow. The
 problem is exacerbated with large repos.

It's possible to hit one of the hidden master - public repo pushes,
they can take a bit as the disks are generally busy in the machine with
other conversions :)

Sadly, a plain pull is not 100% read-only, so WAL doesn't help avoiding
such problems.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Mon, Jun 08, 2015 at 02:07:36PM -0700, bch wrote:
 unclear what I should check -- you mean measure blob sizes in the
 sqlite db, or tcpdump or some fossil option w/ another pull attempt

Right, the entry in the blob table.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] DB corruption and error msg string mis-handling.

[Joerg Sonnenberger]

On Mon, Jun 08, 2015 at 11:52:20AM -0700, bch wrote:
 There's either a corrupted database or transport corruption occurring
 w/ netbsd-src, but additionally apparent string mis-handling in the
 fossil binary:

For unknown reasons, sometimes empty artefacts appear. Can you check
your repo copy?

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Version 1.33 - /reports failing

[Joerg Sonnenberger]

On Tue, Jun 02, 2015 at 11:55:39AM -0600, Warren Young wrote:
 On Jun 2, 2015, at 2:21 AM, Jan Nijtmans jan.nijtm...@gmail.com wrote:
  
  It turns out not to be a gcc optimization bug after all: the optimization
  is very valid
 
 According to what standard??  What I see in 30af11d4 should be legal even in 
 C89.

It is syntactically correct, but UB. The variable is going out of scope
and the associated storage is therefore recycled.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Version 1.33 - /reports failing

[Joerg Sonnenberger]

On Tue, Jun 02, 2015 at 12:11:55PM -0600, Warren Young wrote:
 On Jun 2, 2015, at 12:02 PM, Joerg Sonnenberger jo...@britannica.bec.de 
 wrote:
  
  On Tue, Jun 02, 2015 at 11:55:39AM -0600, Warren Young wrote:
  On Jun 2, 2015, at 2:21 AM, Jan Nijtmans jan.nijtm...@gmail.com wrote:
  
  It turns out not to be a gcc optimization bug after all: the optimization
  is very valid
  
  According to what standard??  What I see in 30af11d4 should be legal even 
  in C89.
  
  It is syntactically correct, but UB.
 
 “Undefined Behavior”?

Yes.

  The variable is going out of scope
 
 The patch changes only the scope of azView, so if it is out of scope, then 
 the use on line 725 won’t compile.
 
 The only way you can refer to a variable that has gone out of scope is to 
 pass pointers around, which isn’t going on here.

No, it is exactly what is happening here via style_submenu_multichoice.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] problem migrating from git

[Joerg Sonnenberger]

On Fri, May 22, 2015 at 02:00:41PM +0200, Luca Ferrari wrote:
 Hi all,
 today I tried to migrate one of my git repository to fossil, but I'm
 getting a disk full error that is unlikely to happen.

Try with TMPDIR and TEMPDIR set to /sviluppo too. I forgot which one is
actually used.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Two trunks?

[Joerg Sonnenberger]

On Sat, Apr 25, 2015 at 01:33:09PM -0700, Matt Welland wrote:
 If a fork happens,  merge it, change it into a branch or close it. There is
 no need for a forks page.  All that is needed is to keep developers
 informed so the fork doesn't lie undetected and cause confusion.

I fully agree and I don't see how the leaves page with maybe a filter
option isn't already good enough for this purpose.

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Two trunks?

[Joerg Sonnenberger]

On Sat, Apr 18, 2015 at 09:53:53AM -0400, Richard Hipp wrote:
 Other proposed changes include more frequent nagging about forks.  The
 issue is less clear-cut, but I still worry that it might contribute to
 warning fatigue.

I think the most reasonable approach is to mirror Mercurial. Before a
pull/sync operation, remembering the number of leaves per branch.
Compare this to the number of leaves per branch afterwards and report
changes. If you skip new/closed branches, it would result in something
like:

+1 leaf for trunk
-1 leaf for hot-development

Joerg
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users