Re: [fossil-users] Fossil in a chroot jail. Was: Chiselapp status

[Eduard]

As a defense against DoS attacks, Fossil has a feature were it refuses
to run certain expense web pages (ex: creating new tarballs) if the
system load averages is too high.  Fossil uses the getloadavg()
interface to compute this.  On Linux, getloadavg() requires that /proc
be mounted.  So, if you want to use the rate limiting feature on
Linux, you will need /proc mounted in your chroot jail.  I wish there
were a better way...


A Linux-specific hack here could be to open the `/proc/loadavg` file 
before entering the chroot and dropping privileges. You can then seek to 
zero and read as many times as you want.

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] branch assistance needed

[Eduard]

1) See the `--cherrypick` switch to `fossil merge`.

2)

a. undo the tags you've fiddled with, so that branch=trunk is original 
trunk and side-branch is branch=side-branch.


b. fossil checkout trunk

c. fossil merge --cherrypick 

d. (optional) move the erroneous commit (I'm assuming the last commit on 
`side-branch`) to branch `mistake` and set to hidden


Best,
Eduard

On 07/03/2018 05:09 PM, Dewey Hylton wrote:
I've used fossil for years now with lots of commits to trunk and very 
few very simple branches which tend to get merged right into trunk 
after only a few commits. I have managed to get myself in a confusing 
place with one of my projects.


Essentially what I did was to commit a change to a new branch, forget 
I was in that branch and committed another unrelated change which 
should have gone back to trunk but went into the branch. I then 
attempted to change that latest commit by adding a 'trunk' tag and 
cancelling the new branch tag. I'm pretty sure that was wrong, but I'm 
unsure now where to go with this. I've tried a few other things to no 
avail, but now even after executing 'fossil update trunk' the 'fossil 
branch' command shows I'm still in the new branch.


So I have two problems to solve:
1) how do I properly move commits between branches (and to trunk, in 
my case)?

2) how do I unfudge my current condition and get back to trunk?


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Perception of Fossil

[Eduard]

A lot of people allow wiki append by anonymous on their repos. You may choose 
not to. Maybe PR should get its own capability so you may restrict to 
authenticated or particular users (or not).


On June 18, 2018 8:39:59 AM EDT, Karel Gardas  wrote:
>On Mon, 18 Jun 2018 00:01:33 +0300
>John Found  wrote:
>
>> > Please no, this would be real security nightmare. Anyone can attack
>any fossil public repo then by simple DoS. Do not ever allow anonymous
>to play with your pristine repository! If anon needs to "push"
>something, then he/she needs to make his/her repo public and *you* can
>investigate the patch of her/him first.
>> > 
>> > Thanks,
>> > Karel
>> 
>> At first it seems you underestimate the ability of fossil to
>withstand high load. But then, there are many ways to overload web
>server without pushing bundles. My experience is that fossil is pretty
>hard to be overloaded, even on very lightweight servers.
>
>I've not been talking about DoS using CPU consumption, but rather about
>DoS based on disk size consumption. Is it that hard to create a bundle
>automatically and then push that to the remote server and do that in
>loop to consume all the drive space? Let's see then how underlying OS
>stops logging into /var/log due to partition shared with /fossil data.
>Will all the important daemons survived 0 available space etc. etc.
>
>By openning option to upload data somewhere for anyone, you put
>yourself on very danger land indeed. IMHO!
>___
>fossil-users mailing list
>fossil-users@lists.fossil-scm.org
>http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Using bundles for pull requests

[Eduard]

A simpler alternative to having a dedicated pull-request feature in
fossil is to just have a second "free-for-all" repository where commit
capability is granted to pretty much any passerby willing to fill out
a CAPTCHA and register an account. Said passerby can then just use the
ticket system (or a mailing list, or the built-in forum that drh seems
to have started working on) to notify the main repository developers
that a pull-request exists on some particular branch in the
free-for-all repository. A developer on the main repository then uses
`fossil bundle` (or some improved version thereof) to import the
changes on that specific branch into the main repository, possibly
altering them in the process (i.e. by importing the bundle as a
private branch, then "rebasing" or compressing the changes onto a
different public branch).

The comments Florian and I made about `bundle import` security still
apply, though. There needs to be some way for a developer to ensure
the bundle they're importing will not "spray" checkins and control
artifacts all over the repository.

By the way, the current `bundle ls` implementation just trusts the
`notes` column of the `bblob` table; it doesn't actually check the
blob contents.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Using bundles for pull requests

[Eduard]

>> ... it's possible to introduce unwanted contents (and random
>> control artifacts) ...

I can foresee another attack that could be used to inject malicious 
control artifacts into the repository (depending on the bundle 
implementation). The attacker creates a delta control artifact 
referencing an as-yet unpublished innocuous artifact. Once that has been 
inserted, they later submit that artifact as well (which resolves the 
delta, and makes the malicious control artifact available). The way to 
prevent this would be to reject any unresolved deltas when importing a 
bundle.


In general, I think `bundle import` should allow only:
- manifests
- regular files referenced by in-bundle manifests
- control artifacts referencing in-bundle manifests

My comments about `bundle import` also apply to any new "pull 
request"-like feature.


Regards,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Hydra registrations open (open beta)

[Eduard]

Hi all,

I have opened registration on Hydra. Feel free to try it out at
[ https://hydra.ecd.space/ ], and give me your feedback. Features:

- Single sign-on; same username on all repositories.
- Extra security features
  - Separate subdomains to limit cross-repository XSS attacks.
  - Each Fossil process runs in a chroot, with unique UID/GID.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] "remember this password (Y/n)?" and `isatty()`

[Eduard]

The '--interactive' switch is only for the sqlite shell I think. It is 
not accepted for the remote-url command.


Simple test case:

$ fossil remote-url https://a:b...@example.com/
remember password (Y/n)? y
https://a...@example.com/
$ yes | fossil remote-url https://a:b...@example.com/
https://a...@example.com/

The problem is that `save_password_prompt` never gets called if the 
stdin is not a tty. There are two code paths leading to it, and both are 
guarded by `isatty(fileno(stdin))`:


https://fossil-scm.org/index.html/artifact?udc=1=61b100991bd4e137=279-280
https://fossil-scm.org/index.html/artifact?udc=1=61b100991bd4e137=552+561

I think one fix is to factor out the `isatty(fileno(stdin))` in a 
separate function, which checks the environment for a 
`FOSSIL_FORCE_INTERACTIVE` variable (or a new global command-line switch).


That, and/or add a `--remember-password` option to `remote-url`.

On 06/08/2018 10:24 AM, Warren Young wrote:

On Jun 8, 2018, at 2:47 AM, Eduard  wrote:

Maybe it would be possible to add a command-line switch to force it to remember 
the password, or a switch to make it pretend stdin is a tty?


It looks like that last is already available: -interactive.

I’m not aware of any place that all of the global options to Fossil are 
documented.  I had to dig through the command line parser in main() within 
src/shell.c to find it.

I’m not certain that this option will affect your symptom.  If not, let us know 
exactly what method you’re trying and what symptom you get when it fails, if it 
does.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] "remember this password (Y/n)?" and `isatty()`

[Eduard]

Hi,

I would like to be able to non-interactively set `remote-url` and make
Fossil remember the password. The `isatty()` check makes that far more
difficult than it should be. Maybe it would be possible to add a 
command-line switch to force it to remember the password, or a switch to 
make it pretend stdin is a tty?


Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Show time...

[Eduard]

I finished implementing UID-based segregation. You can view the
resulting code here[1]. In particular, you may find the suid wrapper
at `fossil/hydra_fossil_chroot_wrapper.c` useful for your own
purposes.

I've added a patch for safe Tcl (controlled by configure switch
`--force-tcl-makesafe`). I've however left Tcl disabled altogether for
two main reasons:
- Tcl provides far too large an attack surface.
- The Tcl interpreter needs access to its function library in
  `/usr/share/tcltk/tcl8.6/`, and I don't feel like messing with bind
  mounts right now to make it available.

I might enable public registration 'soon'. Now all I need is a catchy 
name, like `chiselapp` :p


[1] 
https://hydra.ecd.space/f/hydra/info/c34b243efda4fe2775d7c46f1d9b3cd1b2eb7190eb22933e469c7eb08fee5636


Cheers,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Show time...

[Eduard]

> 1. Enable Safe interpreters for Tcl
How did you do that? The flint repository doesn't seem to include the
fossil build script.

> 2. Enforce that the SSH program cannot be run (by patching popen2()
> to return an error)
Are you not using chroot to protect repositories from each other?

> 3. (Not complete, but started) run each instance of Fossil as a
> different UID based on their Flint UID+131072
That's actually the next thing on my TODO list -- I think it's a good
idea!

> 1. Need to add the domain to the Public Suffix List (otherwise,
> you haven't mitigated the issue completely)
I think the only security issue left vis-a-vis untrusted subdomains is
that a malicious repository can insert thousands of junk cookies,
displacing the login cookie on the secure subdomain and thus logging
the user out. I think it's a mild annoyance at most. Feel free to
contradict me on this.

> 2. Getting a wildcard cert
I also put it off for the longest time. It turned out to be
surprisingly easy to do, and in fact less annoying than http
validation.

Cheers,
Eduard

On 06/04/2018 10:37 PM, Roy Keene wrote:

Other things we do at ChiselApp:
 1. Enable Safe interpreters for Tcl
 2. Enforce that the SSH program cannot be run (by patching
    popen2() to return an error)
 3. (Not complete, but started) run each instance of Fossil as a
    different UID based on their Flint UID+131072

I thought about putting each repo under their own domain, but doing so 
requires a bit more work:

 1. Need to add the domain to the Public Suffix List (otherwise,
    you haven't mitigated the issue completely)
 2. Getting a wildcard cert

On Mon, 4 Jun 2018, Eduard wrote:


I was planning on making a more official announcement, but here goes.

I'm the developer of Hydra, a single-sign-on and manager for fossil 
repositories. https://hydra.ecd.space/f/hydra/wiki/hydra


I think this is relevant as people may be looking to GitHub 
alternatives for multiproject hosting.


I've recently fixed the XSS/CSRF vulnerabilities inherent to hosting 
multiple repositories on the same domain (which also affect 
chiselapp), when setup privilege is given to malicious users (for the 
repositories they create) and they convince other people to visit 
their malicious repository while logged in. I've done this by using a 
separate subdomain for each repository, and by patching Fossil itself 
to receive the CSRF token from Hydra. More details here: 
https://static.ecd.space/x/hydra/doc/build/html/subdomains.html


I've also done some security hardening by dropping each repository in 
a separate chroot (to contain damage from a potential arbitrary code 
execution vulnerability in fossil itself).


(Sorry drh, I accidentally replied only to you instead of the mailing 
list.)


On 06/03/2018 09:28 PM, Richard Hipp wrote:

There is suddenly a big uptick in traffic to fossil-scm.org,
apparently due to the recent GitHub rumor.  Unlike that traditional
"slashdot effect", though, the referrals seem to be coming for a large
variety of sources.

So, if anybody sees any last minute tidying up that we need to do to
the website in anticipation of a huge influx of first-time visitors,
please speak up.  Quickly.



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Show time...

[Eduard]

I was planning on making a more official announcement, but here goes.

I'm the developer of Hydra, a single-sign-on and manager for fossil 
repositories. https://hydra.ecd.space/f/hydra/wiki/hydra


I think this is relevant as people may be looking to GitHub alternatives 
for multiproject hosting.


I've recently fixed the XSS/CSRF vulnerabilities inherent to hosting 
multiple repositories on the same domain (which also affect chiselapp), 
when setup privilege is given to malicious users (for the repositories 
they create) and they convince other people to visit their malicious 
repository while logged in. I've done this by using a separate subdomain 
for each repository, and by patching Fossil itself to receive the CSRF 
token from Hydra. More details here: 
https://static.ecd.space/x/hydra/doc/build/html/subdomains.html


I've also done some security hardening by dropping each repository in a 
separate chroot (to contain damage from a potential arbitrary code 
execution vulnerability in fossil itself).


(Sorry drh, I accidentally replied only to you instead of the mailing list.)

On 06/03/2018 09:28 PM, Richard Hipp wrote:

There is suddenly a big uptick in traffic to fossil-scm.org,
apparently due to the recent GitHub rumor.  Unlike that traditional
"slashdot effect", though, the referrals seem to be coming for a large
variety of sources.

So, if anybody sees any last minute tidying up that we need to do to
the website in anticipation of a huge influx of first-time visitors,
please speak up.  Quickly.



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Is it possible to merge repository's?

[Eduard]

On 05/29/2017 11:35 PM, The Tick wrote:
> On 5/29/2017 10:30 PM, Andy Bradford wrote:
>> Thus said Stephan Beal on Tue, 30 May 2017 02:57:38 +0200:
>>
>>> However, there is  _hypothetically_ a way to completely  merge 2 repos
>>> into one while keeping all commits, but i'm not at all certain if this
>>> would work...
>>
>> I think it actually will work for some definition of ``work''. I've done
>> it before.
>>
>> But... it depends on what one expects out of it.
>>
>> There will  be 2 separate  and independent timelines  once reconstructed
>> and there  will not be a  relationship between artifacts except  for the
>> fact that they all live in the same Fossil file.
>>
>
> From this I gather that there would be no way to connect the imported
> repository onto the main trunk. That was not what I was hoping for.
>
> If I decide to do it, I'll probably look into a shell script to
> checkout each commit, then add and commit to the target repository.
>
Have you considered the "reparent" subcommand?

Usage: fossil reparent [OPTIONS] CHECK-IN PARENT 
*
**Create a "parent" tag that causes CHECK-IN to be interpreted as a**
**child of PARENT.*  If multiple PARENTs are listed, then the first is
the primary parent and others are merge ancestors.

This is an experts-only command.  *It is used to patch up a repository*
that has been damaged by a shun or *that has been pieced together from**
**two or more separate repositories**.*  You should never need to reparent
during normal operations.

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil hosting platforms

[Eduard]

Hi Damien,

Full disclosure: I am the Hydra author.

Hydra repositories are not tied to particular users. Administrators can freely 
modify the repository permissions, and can make them 'private' by removing view 
permissions for the nobody users.

I don't think either flint or hydra supports anything similar to github 
organizations. All they do is provide a thin wrapper around fossil to automate 
repository creation (and, in the case of Hydra, a global single sign-on across 
all repositories, which is really the only major difference between the two; 
Hydra also uses a long random token as the fossil password instead of the login 
password). That said, it shouldn't be very difficult to implement, thanks for 
the suggestion.

I haven't ever set up a flint instance so I can't compare. Hydra is no harder 
to set up than any other django app (setting up a http server and an app 
server, and maybe an actual database although sqlite should probably be fine 
unless you have lots of traffic). That being said, I should probably have added 
a setup guide somewhere... sorry...

IMPORTANT SECURITY NOTICE: Note that both flint and hydra suffer from a major, 
easily *exploitable* vulnerability (XSS attack), and is exploitable by any user 
who can admin or create repositories. For more detail see 
http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25070.html and 
http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25075.html . I 
strongly advise against setting up a flint/hydra instance with open 
registration.

Best regards,
Eduard

On April 23, 2017 12:59:31 PM EDT, Damien Sykes-Lindley 
<dam...@dcpendleton.plus.com> wrote:
>Hey there,
>I currently know of two Fossil hosting systems, Flint and Hydra.
>1. I have seen Flint in action and so have an idea what it’s about. It
>separates repositories into users and makes them either public or
>private. Is Hydra any different, or is it just an alternative? Do
>either of them support anything similar to Github’s organisations?
>2. What are the pros and cons for each?
>3. Which one is easiest to set up if you’re new to server
>administration?
>I’ve followed Flint’s readme (can’t find any other docs) and I’m
>getting the following error from PHP:
>PHP Fatal error:  Uncaught Error: Class 'Nano_View' not found in
>/var/www/fossil/public/index.php:3
>I can’t find any documentation whatsoever on Hydra, and have no
>experience whatsoever with Python servers.
>Any help appreciated.
>Thanks.
>Damien.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] XSS attack and fossil hosting services

[Eduard]

On 04/03/2017 02:16 AM, Warren Young wrote:
> On Apr 2, 2017, at 9:28 PM, Eduard <eduard.c.dumitre...@gmail.com> wrote:
>>
>> An attacker can place malicious javascript at the top of every page
> 
> Certainly.
> 
>> they could, for example, change the victim's password
> 
> Doesn’t the login cookie prevent the hosted user from doing that to any but 
> their own repository, at least through normal HTTP URLs?  (As you say, TH1 
> and SQL may well be a different matter.)
> 
> Setup privilege isn’t given to all users of a repository, only to the most 
> trusted administrator(s) of that repository.
> 
> Unless Fossil erroneously lets you log into repo A and then have the same 
> privileges on repo B hosted by the same Fossil server, I don’t see the 
> problem.  Any wounds would be self-inflicted.

Not true. Suppose that good-user registers
"https://example.com/f/goodproject; and evil-user registers
"https://example.com/f/evilproject; (both have setup privileges on their
respective repositories but don't trust each other). Good-user spends
most of their time logged into goodproject. Evil-user then convinces
good-user to visit evilproject while logged into goodproject; javascript
on evilproject (running on good-user's browser instance) can make
queries to and receive results for all pages on https://example.com/
(according to same-origin policy), including goodproject; it of course
also has access to all of the good-user's cookies. That JS code can do
anything that good-user can do on https://example.com/.

The usage scenario to consider here is that of a large repository host
(like github/sourceforge) which doesn't trust its users (since anyone
with an email address can register an account).

>> (Another way to fix it is by giving each repository a separate subdomain
> 
> Since ChiselApp is using HTTPS, they must be using some kind of front-end 
> proxy, which could redirect each repo URL to a different Fossil instance.  
> That in turn could be run as a chroot or jail, which would mitigate most of 
> the problems you bring up, if not all.
> 
> It’s not hard.  I do this on for my own public repositories.
> 
> It does mean you’re limited in the number of Fossil instances you can run on 
> a single server by RAM and private TCP port numbers, but I shouldn’t think 
> you’d want more than a few tens of thousands of Fossil instances on a single 
> machine anyway.

Sorry, by "it" in "Another way to fix it" I meant the XSS vulnerability.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] XSS attack and fossil hosting services

[Eduard]

Hi,

I recently realized that fossil repository hosting websites (such as
chiselapp <http://chiselapp.com/> and hydra
<https://hydra.ecd.space/f/hydra/>) are vulnerable to arbitrary HTML
injection (XSS) as soon as they give (untrusted) users the 'setup'
capability to the repositories they create. An attacker can place
malicious javascript at the top of every page by supplying a custom
fossil theme; alternatively they could configure the server to serve an
arbitrary html documentation file as the home page. Said malicious
javascript (allowed by same-origin policy) can perform requests and read
their results; they could, for example, change the victim's password
and/or give the attacker maximal privileges on all repositories the
victim has access to.

One way to prevent this would be by creating a cap between 'setup' and
'admin'. This cap would be able to configure all fossil settings that
aren't an XSS risk. In particular, the ability to run arbitrary TH1 and
SQL may also need to be restricted/disallowed (because a user could just
modify the config table using SQL, for example).

For now I'm considering patching fossil itself to prevent users from
modifying dangerous settings (and outright removing the run-SQL/TH1
pages) and by changing the recognized mimetype for html files.

(Another way to fix it is by giving each repository a separate
subdomain, but letsencrypt doesn't do wildcard subdomains and real TLS
certs are expensive.)

As a related and more general question, what damage can a "fossil http
-R $repo" command do to surrounding files/other repositories? In
particular, using TH1/SQL or using the JSON interface?

Best,
Eduard

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] openssl 1.1 compatibility issue in http_ssl.c

[Eduard]

Hi,

When upgrading to openssl 1.1, I noticed that repositories fail to sync
with "SSL: cannot connect to host hydra.ecd.space:443 (unsupported ip
family)". I found a Debian bug describing this issue[1], and Sergei
Golovan released a fix (see patch at bottom of their message in [1]).

[1] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=847556#10

Best regards,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] setting hash-policy via web UI

[Eduard]

Hello,

Are there plans to expose the 'hash-policy' configuration option for
modification in the fossil web interface?

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] tracking changes relatively to root directory on Unix

[Eduard]

Hi Peter,

> Would anyone suggest possible workaround for Unix with no way to open
> repository placed in root of filesystem?
If your sysadmin allows FUSE, you can use `bindfs` to bind-mount the
relevant directories as you wish inside your home directory (as an
unprivileged user).

Best,
Eduard

On 02/21/2017 09:07 PM, Petr Ferdus wrote:
> I am struggling to find a way to track changes in specific directories
> under Unix root directory hierarchy. My r/w access is limited to some
> subdirectories, as illustrated below.
> I have r/w access in
> /home/user/and/below
> /user/anywhere/below
> an read only access in
> /some/directory/with/read/only/acces
> /other/directory/with/read/only/acces
> 
> I am not allowed to open fossil repo in root '/' directory.
> 
> On widows machine, under my administration, I can open repository in
> root directory of a drive and add any tracked file relatively to this
> root. (I use this approach for several years now, with great success)
> 
> Would anyone suggest possible workaround for Unix with no way to open
> repository placed in root of filesystem?
> 
> Thanks,
> Peter
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Adding binary files to fossil

[Eduard]

On 01/03/2017 01:25 PM, Richard Hipp wrote:
> I have entire projects consisting of *only* binary files - for example
> my collection of OpenOffice presentations.
Actually you can use the flat XML representation (simply by using the
.fodt/fodp/fods extension when saving), which turns them into (quite
large) text files, more amenable to merging.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] announcing a new repository manager webapp

[Eduard]

Hi,

I'd like to announce a piece of (free open source) software I wrote
called Hydra. It is a single sign-on gateway to and manager for fossil
repositories, written using Django. In the hopes that it'll be useful to
at least one person on this mailing list, the source code is available
at https://hydra.ecd.space/f/hydra/

Some features:
- integrates with django user system so you can set up your own
registration/authentication policy and so on
- user passwords aren't fossil passwords; instead, the app gives the
user a long random token that is used as a fossil password instead

Of course, a Hydra instance is up and running at https://hydra.ecd.space/

Best regards,
Eduard

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Release 1.35 checksums?

[Eduard]

On 07/05/2016 02:56 PM, jungle Boogie wrote:
> On 1 July 2016 at 09:39, Warren Young  wrote:
>> If you’re expecting the checksum to protect you against someone hacking the 
>> web site and uploading malware, they can modify the checksums on the web 
>> site at the same time.
> Absolutely.
> 
> As a small request, maybe when Dr. Hipp makes a release, he can also
> include the hash in the email. As Andy indicated, this can be archived
> by search engines and even available on the archive of the mailing
> list.

As a related small request, it would be very much appreciated if more
people (including D. R. Hipp) signed their commits with PGP (in addition
to the build hashes on the site). After all we already have the fossil
'clearsign' setting, it's just a matter of generating a key (gpg
--gen-key) and using it.




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil bundle export of all artifact types ...

[Eduard]

Hi Jousef,

If you want everything, then

fossil clone input.fossil output.fossil

You can also have an intermediary that you periodically sync with:

fossil sync -R input.fossil intermediary.fossil

Use the "--private" option in either case to also sync private branches.

Best,
Eduard

On 11/25/2015 02:47 PM, Jousef Lofstrom wrote:
> How can I export file *and* non-file artifacts (tickets, wiki-pages)
> into a bundle for import into another repo (offline synchronization) ?
>
> Many thanks,
>
> Jousef
>
>




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] diff for undo

[Eduard]

Hi,

It seems I can't open a feature request ticket anonymously, and I don't
have an account on fossil-scm.org (I don't know whether drh has received
my contributor's agreement). I would like to request two features:
(1) the addition of a "undo" checkin alias; and
(2) the modification of the "ckout" checkin alias to allow it within the
"fossil diff" command.

This would allow users to run "fossil (g)diff --from undo --to ckout" in
order to see what the last undo-able command did
(update/merge/stash-apply). Newbies would find this extremely useful,
and it would allow one to review exactly what changes they just updated
past (when in autosync mode). As an alternate (and possibly simpler)
solution, one could instead add an "--undo" argument to (g)diff which
would do what I just described.

Best,
Eduard




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] diff before update

[Eduard]

Hi Stephan,

On 11/21/2015 04:38 AM, Stephan Beal wrote:
> On Sat, Nov 21, 2015 at 12:24 AM, Eduard
> <eduard.c.dumitre...@gmail.com <mailto:eduard.c.dumitre...@gmail.com>>
> wrote:
>
> The problem lies not with 'update'. The problem is that there's no
> way to refer within other commands (specifically 'diff') to the
> commit that 'update' (with no VERSION argument) would update to.
>
>
> Not even update knows which version it will pick until it is run.
> Therefore i don't foresee other commands being able to predict that.
> When update is run, it will sync (if enabled), the apply whatever the
> latest is (unless told otherwise, in which case it _does_ know in
> advance).
>
>  
>
> In the above example, I would like there to be some alias
> "update-target" that would resolve to
> d83fc58dead2d03428a763b0890b8b5fbffb7957 (not actually suggesting
> that as a name, there's probably a better name for it), so that I
> could do "fossil diff --from current --to update-target".
>
>
> That update-target value is unknown at that point, and there's no way
> for diff to know it, in particular if autosync is on (which it
> normally is).
You make an excellent point/!/ (I forgot that most people do use
autosync.) To make everyone happy, I think it would be very useful
instead if there was a checkin alias "undo" that would refer to the
state of the local source tree if 'undo' were to be run. This way after
doing an 'update' (or even a 'merge'), one could run "diff --from undo
--to current" to see exactly what the last undo-able command changed. I
think it would also be pretty useful when doing more than 2-way merges.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] diff before update

[Eduard]

Hi,

I would like to have a way to review the diff of all changes that have
been made between the current checkout and the tip of the current branch
before running "fossil update".

Is there a way of specifying the commit to which "fossil update" would
update to? The documentation to "update" says "If the VERSION argument
is omitted, then the leaf of the subtree that begins at the current
version is used, if there is only a single leaf.", so is there an alias
for that (like "current" or "next")? Note that this is not the same as
running "fossil update branch", because that would bring you to the
latest commit tagged with "branch".

Thanks,
Eduard




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] diff before update

[Eduard]

Hi Stephan,

Thanks for your reply!

On 11/20/2015 02:42 PM, Stephan Beal wrote:
>
> Is there a way of specifying the commit to which "fossil update" would
> update to? 
>
>
> fossil update UUID-or-TAG
Sorry, I think I haven't been very clear. My question is rather, is
there an alias with the meaning "whatever commit 'fossil update' would
update to if I ran it right now"?

I could of course do this by parsing the output of "fossil update
--dry-run" and then running "fossil diff --from current --to $UUID", but
I'm wondering if there's a built-in way of doing it without resorting to
shell scripts.

Thanks,
Eduard



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] diff before update

[Eduard]

On 11/20/2015 03:10 PM, Stephan Beal wrote:
> On Fri, Nov 20, 2015 at 8:55 PM, Eduard <eduard.c.dumitre...@gmail.com
> <mailto:eduard.c.dumitre...@gmail.com>> wrote:
>
> On 11/20/2015 02:42 PM, Stephan Beal wrote:
>>
>> Is there a way of specifying the commit to which "fossil
>> update" would
>> update to? 
>>
>>
>> fossil update UUID-or-TAG
> Sorry, I think I haven't been very clear. My question is rather,
> is there an alias with the meaning "whatever commit 'fossil
> update' would update to if I ran it right now"?
>
>
> That alias is the name of the current branch (which you seem to want
> to avoid, though i'm not sure why).
I'm trying to avoid it because the two disagree if there are multiple
independent branches that share the same name. Example:

$ f status
repository:   snip/fossil.fossil
local-root:   snip/
config-db:snip/.fossil
checkout: f0de41fd5ed81cf91953e30fe5ca9e057aed2d34 2014-01-09
05:56:05 UTC
parent:   cbcd7056b6c3cba36c8cac8385d23517222116f3 2014-01-08
11:38:20 UTC
child:d83fc58dead2d03428a763b0890b8b5fbffb7957 2014-01-09
13:17:23 UTC
tags: pending-review
comment:  Attempts to add files outside the open checkout tree
result in fatal error.
  (user: andybradford)
$ f up -n   
UPDATE src/add.c
---
updated-to:   d83fc58dead2d03428a763b0890b8b5fbffb7957 2014-01-09
13:17:23 UTC
tags: pending-review
comment:  Add comments to better explain what it happening for the
out-of-tree check
  in the "add" command. (user: drh)
changes:  1 file modified.
$ f up -n pending-review
(snip)
---
updated-to:   2a1d7e0efc0f048b495815bac577d3c0f5c4d9a6 2015-10-20
08:14:11 UTC
tags: pending-review
comment:  Docker build "release" by default. Meant for upcoming
Fossil release. (user:
  jan.nijtmans)
changes:  322 files modified.
$

> fossil pull
>
> will sync any remote changes without applying them, then you can:
>
> fossil diff [--branch TAG]
>
> will show you changes against the tip of that branch (which is what
> 'update' would update you to).
I'm actually not exactly sure what that does. (The documentation isn't
very clear about it.) But it's again not equivalent to what update would
do in the above scenario:
$ f diff --from current --to d83fc58dead2d03428a763b0890b8b5fbffb7957 |
grep '+++'
+++ src/add.c
$ f diff --branch pending-review | grep '+++'  
+++ Dockerfile
+++ src/info.c
+++ src/timeline.c

Again, I should have explained better. I want the diff between the
current checkout (or even whatever commit the current checkout is based
on) and what "fossil up" would update me to. The point is to have a
quick glance at whatever everyone else did before updating (because
maybe they introduced a really obvious bug, or maybe they inserted
malicious code). This is the main reason why I don't use autosync;
because then I would have to remember what the last commit I trust was,
and then "fossil diff --from ${last_trusted_commit} --to current".
> If, however, autosync is on, then there is a window of time where
> 'diff' and 'update' may show different things because someone can
> commit to the remote repo between your 'diff' and 'update'.
>  
>
> I could of course do this by parsing the output of "fossil update
> --dry-run" and then running "fossil diff --from current --to
> $UUID", but I'm wondering if there's a built-in way of doing it
> without resorting to shell scripts.
>
>
> 'update' will always take the tip unless told otherwise, which seems
> to me to be what you want to diff against? If you'll disable autosync
> (which isn't generally recommended) you can avoid any sort of race
> condition between diff and update, and then simply rely on the
> 'default to tip' behaviour.
It depends what you mean by "tip" (whether it's the latest commit with a
certain tag, or whether it's the latest commit with a certain tag /that
is a direct descendant of the current commit/).

Best,
Eduard


signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] diff before update

[Eduard]

On 11/20/2015 04:06 PM, Stephan Beal wrote:
>  
>
> $ f status
> tags: pending-review
>   (user: andybradford)
> $ f up -n   
> UPDATE src/add.c
> 
> ---
> updated-to:   d83fc58dead2d03428a763b0890b8b5fbffb7957 2014-01-09
> 13:17:23 UTC
> tags: pending-review
> comment:  Add comments to better explain what it happening for
> the out-of-tree check
>   in the "add" command. (user: drh)
> changes:  1 file modified.
> $ f up -n pending-review
> updated-to:   2a1d7e0efc0f048b495815bac577d3c0f5c4d9a6 2015-10-20
> 08:14:11 UTC
> tags: pending-review
> comment:  Docker build "release" by default. Meant for
> upcoming Fossil release. (user:
>   jan.nijtmans)
> changes:  322 files modified.
>
>
> Aha, now i see the problem. i don't have a solution. In such a crazy
> case, using the UUID is really the only sane choice.
So uh is there any chance of adding an alias for this (@drh)? It's easy
to write a shell script that does it, but it seems to me that this is
something that should really be within fossil itself (since not every
fossil user necessarily knows how to write shell scripts).

Best,
Eduard


signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] diff before update

[Eduard]

On 11/20/2015 04:56 PM, Stephan Beal wrote:
> On Fri, Nov 20, 2015 at 10:52 PM, Eduard
> <eduard.c.dumitre...@gmail.com <mailto:eduard.c.dumitre...@gmail.com>>
> wrote:
>
> On 11/20/2015 04:06 PM, Stephan Beal wrote:
>>  
>>
>> $ f status
>> tags: pending-review
>>   (user: andybradford)
>> $ f up -n   
>> UPDATE src/add.c
>> 
>> ---
>> updated-to:   d83fc58dead2d03428a763b0890b8b5fbffb7957
>> 2014-01-09 13:17:23 UTC
>> tags: pending-review
>> comment:  Add comments to better explain what it
>> happening for the out-of-tree check
>>   in the "add" command. (user: drh)
>> changes:  1 file modified.
>> $ f up -n pending-review
>> updated-to:   2a1d7e0efc0f048b495815bac577d3c0f5c4d9a6
>> 2015-10-20 08:14:11 UTC
>> tags: pending-review
>> comment:  Docker build "release" by default. Meant for
>> upcoming Fossil release. (user:
>>   jan.nijtmans)
>> changes:  322 files modified.
>>
>>
>> Aha, now i see the problem. i don't have a solution. In such a
>> crazy case, using the UUID is really the only sane choice.
> So uh is there any chance of adding an alias for this (@drh)? It's
> easy to write a shell script that does it, but it seems to me that
> this is something that should really be within fossil itself
> (since not every fossil user necessarily knows how to write shell
> scripts).
>
>
> i don't think an alias is needed - simply leave out the branch name
> when you do 'update' (or pass the name 'current'). Then, according to
> the code-level docs, if you leave out a name or use 'current', it will
> look for a direct ancestor. Try it with 'current' and see if that
> solves your case.
The problem lies not with 'update'. The problem is that there's no way
to refer within other commands (specifically 'diff') to the commit that
'update' (with no VERSION argument) would update to. In the above
example, I would like there to be some alias "update-target" that would
resolve to d83fc58dead2d03428a763b0890b8b5fbffb7957 (not actually
suggesting that as a name, there's probably a better name for it), so
that I could do "fossil diff --from current --to update-target".

Best,
Eduard


signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil repo on a network share?

[Eduard]

And even if you don't have a static IP and you're behind a stupid NAT,
you can still set up a tor hidden service.

On 11/18/2015 12:12 PM, Scott Robison wrote:
>
> On Nov 18, 2015 1:28 AM, "Stephan Beal"  > wrote:
> >
> > On Wed, Nov 18, 2015 at 9:19 AM, Dömötör Gulyás  > wrote:
> >>
> >> And unfortunately not all devs sit in the same building. Anyway,
> I'll just upload a repo, probably most people will just read it, and
> when changes are to be made, it'll be on me to manage merging anyway.
> This really isn't a software company :/
> >
> >
> > You have been warned.
> >
> > Why not host the repo on an external site via CGI? A hoster capable
> of this costs $5/month, and anyone with internet access will be able
> to reach it (provided they know where it is).
>
> And if one is concerned about the price of such a hoster, or the
> potential security ramifications of keeping repos on a server you
> don't own or control, it can be pretty easy to setup a VM on a home
> computer if you have anything remotely close to a static IP.
>
>
>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] server returned an error - clone aborted

[Eduard]

Hi,

Not sure if this is a bug. But you may want to try cloning/pulling from
"ssh://nrb@101.102.103.104//path/to/repo.fossil" (instead of a directory
with an open checkout). My first guess is that it's a difference between
Windows and UNIX fossil, in particular that the file containing the
current checkout information is named differently in the two cases
("_FOSSIL_" vs ".fslckout").

I may be completely wrong though.

Best,
Eduard

On 11/18/2015 11:59 PM, fossil@9ox.net wrote:
> Hi,
>
> I am stuck getting Windows to work with my repo (88KB).  I have it syncing
> between two Debian environments (one a VPS) both using Stunnel.
>
> But to pull it onto a windows desktop seems to be evading me.  I tried three
> flavours so far. Cygwin, Windows, Windows + Cygwin's ssh.
>
> The last one that is OK is from the Debian box.
>
> I seem to be a bit sort of further ideas.  Perhaps I can ask for some help.
>
> BR/ Nigel
>
>
> Cygwin:
> nrb@Lenovo-PC ~
> $ fossil version
> This is fossil version 1.33 [9c65b5432e] 2015-05-23 11:11:31 UTC
> nrb@Lenovo-PC ~
> $ fossil clone ssh://nrb@101.102.103.104//var/repos/test C:\test.fossil
> nrb@101.102.103.104's password:
> Fossil internal error: repository does not exist or is in an unreadable
> directory: /var/repos/test
> server did not reply
> Clone done, sent: 283  received: 0  ip: 101.102.103.104
> server returned an error - clone aborted
>
> Windows:
> C:\Users\nrb>"c:\Program Files (x86)\Fuel\fossil.exe" version
> This is fossil version 1.34 [62dcb00e68] 2015-11-02 17:35:44 UTC
>
> nrb@Lenovo-PC ~
> $ fossil clone ssh://nrb@101.102.103.104//var/repos/test C:\test.fossil
> nrb@101.102.103.104's password:
> Fossil internal error: repository does not exist or is in an unreadable
> directory: /var/repos/test
> server did not reply
> Clone done, sent: 283  received: 0  ip: 101.102.103.104
> server returned an error - clone aborted
>
> nrb@Lenovo-PC ~
> C:\Users\nrb>"c:\Program Files (x86)\Fuel\fossil.exe" clone
> --ssh-command=C:\cyg
> win\bin\ssh.exe ssh://nrb@101.102.103.104//var/repos/test C:\test.fossil
> cannot create child process
>
> # fossil clone ssh://nrb@101.102.103.104//var/repos/test test2
> nrb@101.102.103.104's password:
> Round-trips: 2   Artifacts sent: 0  received: 31
> Clone done, sent: 585  received: 25479  ip: 101.102.103.104
> Rebuilding repository meta-data...
>   100.0% complete...
> Extra delta compression...
> Vacuuming the database...
> project-id: 41431e30848dacce9667441f278e53832d83cdde
> server-id:  e07c5bcf7b48e3f3dfc86a8ae005f156a39762ee
> admin-user: nrb (password is "097efc")
>
>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil does not generate https link prefix

[Eduard]

Hi Taras,

I've had a very similar problem. I fixed it by setting the "HTTPS"
environment variable (for CGI execution) to "on" when the request comes
in through https, i.e.


  SetEnv HTTPS on


You might want to remove the "" part if you're only accepting https
anyway.

I'm not sure whether this is the truly correct way to do it, but it sure
solved my problem.

Cheers,
Eduard

On 11/04/2015 11:48 PM, Taras Zakharko wrote:
> Dear all, 
>
> I am serving repositories on our internal server via the Fossil GCI
> mechanism behind Apache2 with SSL. The system is OS X. Everything was
> running perfectly, but today I have updated the OS X Server app to the
> new version and while everything is still running, Fossil now fails to
> generate the https:// prefixes for the links, instead using http://.
> As a result, security features in the modern browser ignore the CSS
> stylesheets (because there is a protocol mismatch). So while pushing,
> pulling as well as website do work — the later looks quite ugly. I was
> looking for the way to make Fossil generate https:// prefix, but I
> couldn’t find any definite information. 
>
> I am sure that Fossil itself is not to blame, because everything was
> working perfectly before the server upgrade. But maybe someone has
> encountered a similar problem and can point me into a direction where
> to look. The server migration script did change the conf file header from:
>
> 
>   ServerName URL
>
> to
>
> 
>   ServerName https://URL:443 <https://url:443>
>
> There were no other changes. Other services (some Rails applications)
> are running without any issues. 
>
> Any ideas?
>
> Best, 
>
>  Taras
>
>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] meeting tomorrow at 10:00?

[Eduard]

Hi,

Would meeting Wednesday (tomorrow) at 10:00 be okay with you? Please
reply to let me know (unless you've filled in the scheduling poll
<http://doodle.com/poll/sqaibsinpk8feiww> or already told me in person).

Cheers,
Eduard



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] handling backports

[Eduard]

Hi,

I have some questions as to the 'correct' way of handling backports.

Scenario 1: Suppose a really terrible bug (e.g. an exploitable
vulnerability) is discovered. Although the latest release is version
2.47, it affects every version from 2.12 onwards. Trunk gets quickly
patched and a new minor version 2.47.1 is released. However, most users
are still using versions 2.35 and 2.43, and they can't necessarily
upgrade to 2.47 (e.g. because of some new library dependency that was
introduced, or some breaking change). What is the correct way of
handling this in Fossil (or in a DVCS in general)?

Scenario 2: Consider a repository containing a scientific paper's latex
sources. It is discovered that every single checkin since the very
beginning has a bug that used to cancel out with another bug in latex
itself. However that latex bug has been recently patched, and now every
older version fails to build. The fix itself is just the replacement of
one line that hasn't changed since the bug was introduced. Again, what
is the correct way of handling this?

Best,
Eduard



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] impersonating users

[Eduard]

On 11/03/2015 06:38 AM, Richard Hipp wrote:
> On 11/3/15, Eduard <eduard.c.dumitre...@gmail.com> wrote:
> 
> The artifact receipt log (/rcvfromlist) shows *all* artifacts that
> have been received.  But it does not filter by artifact type or user -
> it shows everything.
> 
> The list-of-artifacts pages (/bloblist) show all artifacts and
> describes each one.  But it also omits the user and where it was
> received.
That's actually amusing.

> Perhaps you could use those two pages as a guide to devise a new page
> that shows what you want, then send in patches?
I would love to! I doubt I'll have the time over the next month though.
I should probably fill out and send a contributor agreement just in case
I do, though.

Best,
Eduard




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] meeting tomorrow at 10:00?

[Eduard]

Hi,

I posted to the wrong mailing list. I apologize. Serves me right for
naming them so similarly (and for not getting enough sleep). You may
safely hide/delete/clean up
http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg21825.html
(as well as this message) as it was not intended for this list.

Sorry,
Eduard



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SHA1 and security

[Eduard]

Hi Scott,

Thank you for your reply!

On 10/29/2015 01:40 AM, Scott Robison wrote:
> On Wed, Oct 28, 2015 at 6:37 PM, Eduard <eduard.c.dumitre...@gmail.com
> <mailto:eduard.c.dumitre...@gmail.com>> wrote:
> 
> If fossil didn't say it used SHA1 to generate artifact IDs, I don't
> think anyone would care how it generated IDs.
> 
> An artifact ID is a way of assigning a fixed length identifier to an
> artifact with good distribution of IDs in the fixed length space
> provided. It is not intended to be a cryptographic.
> (...)
> In this case, it's just an identifier, and the odds of a
> non-malicious collision are so close to zero that those odds might as
> well be zero.

That's the thing, an artifact ID is *not* just an identifier! I don't
know whether this is an intentional feature or not, but a Fossil
repository is structured as a Merkle tree (or a "hash tree"). If I know
that a single commit is genuine (because I wrote it down on a piece of
paper or because it is PGP signed), then that guarantees that all of its
ancestors (and all of their files) have not been tampered with. This is
a very powerful property, assuming that the hash function is
cryptographically secure.

In contrast, suppose that instead artifacts were identified by their
CRC160 (like CRC32, but longer) or by some sort of GUID. Accidental
collisions would be extremely unlikely, but intentional collisions or
preimages would be trivial to forge. Knowing that a particular commit
manifest is genuine would be pointless; anyone could make up a different
ancestor tree that matches the parent ID in the manifest. Worse yet,
that would not even guarantee the 'goodness' of the files in the commit
(anyone would be able to make up a different file with the same artifact
ID).

> You can't create a collision in advance based on not knowing who is
> going to commit what to the repository in advance.
> 
> Let's say you do, after the fact, manage to create a collision. If you
> try to upload it to the repository it will be ignored because fossil
> believes (correctly) it already has the artifact in question.

That's not a collision, that's a second-preimage. My point is rather
that an attacker can serve the clean version of the file to the
auditors, and the malicious version to the users (who trust the auditors
to tell them whether a commit is okay to use). This doesn't require a
second-preimage attack against the hash, only the ability to generate
collisions (which is possible right now, but expensive).

> As you observe, one could in theory mount a MITM attack. At this point
> what is to stop them from serving a completely alien repository that
> they've specially crafted? No collisions required.

It would then be obvious that the repository is completely alien since
none of the artifact IDs match. If the user knows at least one top-level
genuine artifact ID (for example, because they trust a developer and
that developer signs their commits), then that guarantees the integrity
of all the ancestors.

> In fact, the "easiest" way to getting people to use malicious software
> is to host a compromised repository and convince people to use it
> instead of the "blessed" repository.

I agree, that's the way it usually works in practice. But no amount of
checking (save for re-hashing every single artifact in a different
database and signing *that*) can protect against SHA1 collision (or
second-preimage) attacks.

> If you want to change the way fossil does things to limit the
> possibility of fraudulent artifacts, that's fine. Perhaps prefixing the
> blob data with a length (ala git) might help mitigate the possibility of
> hash collisions. Perhaps creating a hash of the complete commit (vs just
> the manifest) and storing it in the manifest might help.

If the hash function itself is broken, neither of those would work. The
first fix would only work if SHA1 collisions are only possible with
different-length inputs (from what I understand, they're usually on
equal-length inputs), and the second fix is essentially the R-card
(which has been deprecated for efficiency reasons, and because it is
redundant with the files' artifact IDs which (are supposed to) guarantee
their contents' genuineness).

> Ultimately, one can chase hash algorithms forever trying to create some
> ultimately secure ideal. In the case of actual security software, I can
> see the point.

A DVCS *is* a security software. It is perhaps the most important
security software, since the security and integrity of all other
software ultimately depends on it. The integrity of the sqlite
repository (which is probably used by over a billion people right now)
depends on it.

> one can chase hash algorithms forever

I know that it may appear so (I thought so too for a long time), but
it's not a question of choosing better hash algorithms forever either.
>From t

Re: [fossil-users] SHA1 and security

[Eduard]

Hi Warren,

Thanks for replying!

On 10/29/2015 02:46 PM, Warren Young wrote:
> On Oct 28, 2015, at 6:37 PM, Eduard <eduard.c.dumitre...@gmail.com> wrote:
>>
>> I wish to discuss the issues surrounding the use of SHA1 in Fossil
> 
> Have you read the prior discussions on this?
> 
>   
> http://www.mail-archive.com/fossil-users%40lists.fossil-scm.org/msg18053.html
>   
> http://www.mail-archive.com/fossil-users%40lists.fossil-scm.org/msg05970.html
>   
> http://www.mail-archive.com/fossil-users%40lists.fossil-scm.org/msg21423.html

I had read 2/3 of them, yes. Thanks for the third one!

> 
>> First I propose that the use of SHA1 in Fossil is a serious problem.
> 
> The known attacks on SHA-1 are still computationally expensive, and will 
> remain so for years.  Not impossible, but still very difficult.  We have time 
> to move, if we need to.

I agree. I also believe that the best time to think about it is right
now. The number of Fossil users will only increase with time (in fact
I'm about to introduce four new people to Fossil), and so will the
number of people potentially annoyed by a non-backwards compatible
change in the specification.

> But also, and much more importantly, most of the attacks on SHA-1 only apply 
> to standalone blob cases such as binary package validation, X.509 certificate 
> signing, etc.  In Fossil, most of the SHA-1 checksummed artifacts are chained 
> in some way, so that you can only modify the leaves of branches.

And individual files (that are part of commits). That won't show up in
the timeline.

>> If the attacker can intercept
>> communications between the server and a developer
> 
> …then you did not run Fossil over TLS, like you should if MITM is a 
> legitimate risk in your situation. :)
> 
>> If the attacker is in control of the server
> 
> …then he can serve you any content he likes, no matter how good your hash 
> algorithm is.

True, but he shouldn't be able to convince me that ID "abcdef"
corresponds to something other than the original artifact created with
ID "abcdef". Again, I might know (through some other source, e.g.
PGP-signed email) that artifact "abcdef" is genuine, and it shouldn't
matter where I download it from. If artifact "abcdef" refers to "xyzzy",
trusting the genuineness of "abcdef" should imply trusting that of "xyzzy".

I also don't believe that the users and developers should have to trust
the Fossil server (including mirrors) and its operator; I don't have to
trust my Debian mirror to download packages (and their sources) from it.
That would avoid happenings like the XcodeGhost incident.

> The correct solution here is something like TLS with certificate pinning, GPG 
> signing, etc.
That's the thing, GPG signing covers the contents of the manifest, which
itself refers to the files inside it only by their SHA1 hash. If someone
substitutes a file with a malicious file that hashes the same, it won't
change anything in the manifest and the GPG signature will remain valid.

>> The third solution is to change the Fossil specification to redefine the
>> artifact ID to be the concatenation of the SHA1 and BetterHash
> 
> A fourth solution is to use Modular Crypt Format to declare the hash for each 
> artifact, and for future Fossil versions to tolerate SHA-1 only in existing 
> artifacts, accepting new ones using only known-good algorithms:
> 
>   https://pythonhosted.org/passlib/modular_crypt_format.html
> 
> This could be done without breaking the DB, simply because a 20-byte hash 
> must be SHA-1, since even a 160-bit BetterHash will have the MCF wrapper on 
> it, making it more than 20 bytes.  
> 
> The SQLite card format parser would have to be made more flexible, to make it 
> understand that if it sees a leading dollar sign, the following hash can be 
> variable-width.

That is a great (and extensible!) solution! There are a few issues though:
- Every artifact must be hashed by every known algorithm. The database
size grows linearly with the number of hashing algorithms.
- There must be an additional mechanism for upgrading the older hash
version artifacts. Consider a checkin manifest from 3 years ago. It is
very likely that no new checkin/branch will ever refer to it directly,
so nobody will ever refer to it by new-hash. Worse yet, it is likely
nobody will ever refer to the files inside that checkin by new-hash. If
a preimage attack on old-hash becomes possible (or even easy), one could
mess with the artifacts that are only referred to using old-hash.

I don't believe the first issue will ever be a problem, though, since I
personally don't think we'll ever need to go past BetterHash-512.

As for the second issue, one solution is to rehash all of the older
artifacts using new-hash and

Re: [fossil-users] SHA1 and security

[Eduard]

Hi Richard,

Thanks for replying!

On 10/29/2015 06:13 PM, Richard Hipp wrote:
> On 10/29/15, Scott Robison <sc...@casaderobison.com> wrote:
>>  Why is the R card optional?
> 
> Because it is expensive to compute on large repos (ex: NetBSD) with
> hundreds of megabytes of content.  Some projects elect to omit it.

Therefore large projects have to choose between having
order-of-magnitude slower security checks and being liable to SHA1
collision attacks. Moreover, it is precisely those large projects that
suffer significantly from the slowdown that need additional protection
the most (since it is easier to hide a malicious needle in a bigger
haystack).

Best,
Eduard




signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SHA1 and security

[Eduard]

On 10/29/2015 04:26 PM, Christopher M. Fuhrman wrote:
> On Wed, 28 Oct 2015 at 5:37pm, Eduard wrote:
> 
> What kind of speed hit would using the BLAKE2b algorithm on 32-bit
> machines such as i386, vax, or m68k?  Yes, there's the BLAKE2s
> algorithm for 8-32 bit architectures but that produces different
> hashes than BLAKE2b.  Is it even possible to use BLAKE2b on a 32-bit
> CPU?

Yes it is possible, and not much slower either! Here's what I get on my
Intel L2300 CPU (which is 32-bit only):

blake2s-256
268435456 bytes (268 MB) copied, 3.8387 s, 69.9 MB/s
blake2b-512
268435456 bytes (268 MB) copied, 3.97249 s, 67.6 MB/s
sha1
268435456 bytes (268 MB) copied, 2.23538 s, 120 MB/s
sha2-256
268435456 bytes (268 MB) copied, 5.03352 s, 53.3 MB/s
sha2-512
268435456 bytes (268 MB) copied, 19.9417 s, 13.5 MB/s

And here's what I get on a 64-bit CPU (Intel i5-3470):

blake2s-256
1073741824 bytes (1.1 GB) copied, 2.09602 s, 512 MB/s
blake2b-512
1073741824 bytes (1.1 GB) copied, 1.6206 s, 663 MB/s
sha1
1073741824 bytes (1.1 GB) copied, 2.35279 s, 456 MB/s
sha2-256
1073741824 bytes (1.1 GB) copied, 5.14392 s, 209 MB/s
sha2-512
1073741824 bytes (1.1 GB) copied, 3.47958 s, 309 MB/s

I also remember trying this on an Allwinner A10 ARM (32 bit) CPU, and
the NEON-optimized version was actually faster than SHA1.

Best,
Eduard


Script:
#!/bin/sh
do_hash() { echo "$1"; shift; { dd if=/dev/zero bs=16M count=$n | "$@"
/dev/stdin >/dev/null; } 2>&1 | grep 'copied'; }
n=16
do_hash blake2s-256 ./b2sum -a blake2s
do_hash blake2b-512 ./b2sum -a blake2b
do_hash sha1 sha1sum
do_hash sha2-256 sha256sum
do_hash sha2-512 sha512sum



signature.asc
Description: OpenPGP digital signature
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] SHA1 and security

[Eduard]

Hi Warren,

On 10/29/2015 06:50 PM, Warren Young wrote:
> On Oct 29, 2015, at 3:40 PM, Eduard <eduard.c.dumitre...@gmail.com> wrote:
>> On 10/29/2015 02:46 PM, Warren Young wrote:
>>> (...)
>> I had read 2/3 of them, yes. Thanks for the third one!
> 
> The third one’s the mother lode.  Don’t be fooled by the mail-archive.com UI, 
> which presents only 10 or so results at a time.  That thread went on and on 
> and on.  Hopefully we can avoid retreading some of the same ground in this 
> one.

Thanks!

>>> most of the attacks on SHA-1 only apply to standalone blob cases
>>
>> And individual files (that are part of commits). That won't show up in
>> the timeline.
> 
> Do you mean newly-added files?  They’re shown on the Check-in details screen, 
> the most likely thing you’re going to click on on the Timeline page.
> 
> For a newly-added file to have an effect on a repo, it’ll probably also 
> require modification to an existing file, such as the Makefile.
> 
> (Exceptions are files included by wildcard.)

I'm talking about generating collisions for non-control artifacts
(actual files), not control artifacts. This has a higher chance of being
successful if the repository has the R-card disabled (for efficiency
reasons).

>>>> If the attacker is in control of the server
>>>
>>> …then he can serve you any content he likes, no matter how good your hash 
>>> algorithm is.
>>
>> True, but he shouldn't be able to convince me that ID "abcdef"
>> corresponds to something other than the original artifact created with
>> ID "abcdef”.
> 
> How are you going to know that the legitimate file has ID abcdef?  
> Cross-reference to another repo?  What if there is only one central repo?

I'll check the PGP signature of a checkin manifest that has a checkin in
its ancestry that contains this file.

> If an evildoer has taken over the central server, they are just providing a 
> pile of artifacts, and you are trusting that those artifacts are legitimate.

But if you do know that some subset of the (control) artifacts are
legitimate (because you checked the PGP signature or because the lead
dev told you the tip artifact ID in-person), it is so extremely useful
to be able to infer that all of their ancestors must also be legitimate.

>> I might know (through some other source, e.g.
>> PGP-signed email) that artifact "abcdef" is genuine, and it shouldn't
>> matter where I download it from.
> 
> How many people will be doing such cross-checking?
> 
> Again I bring up the XcodeGhost example.  People do foolish things in the 
> name of expediency.

Well, I know I will be doing such cross-checking. Hopefully I'm not the
only one. Right? ...right?

>> I don't have to trust my Debian mirror to download packages (and their 
>> sources) from it.
> 
> You’re referring to the fact that DEBs are GPG-signed, I assume?
> 
> That works because the Debian gatekeepers can sign the packages on an 
> assumed-secure box.  (Such central package repos have been compromised in the 
> past.)  The distro includes a copy of the central source’s public key, so if 
> the package signature doesn’t decrypt correctly, it isn’t legitimate.

You're right; the binary packages are automatically signed on an
assumed-secure box. But the source packages are also signed by the
individual packagers, whose private keys reside only on their computers
(and not on the server).

> Where can you put such a root of trust in the Fossil case?
> There is no central presumed-secure site with Fossil.  Remember, you
were just positing that the central repo’s server got rooted.

There is more than one answer, but one is that the root of trust are the
PGP private keys on the individual developers' personal computers. The
developers' private keys should never ever reach the public central repo
server.

> You also can’t solve it by having people with checkin bits submit a GPG 
> public key to the repo along with their login creds and sign checkins, 
> because those keys live on the same compromised server.  The evildoer can 
> just generate a new set of keys, re-sign the compromised artifacts, and store 
> the new keys in the user table instead of the original keys.
> 
> It’s the problem with all PKIs: who do you trust?

That's exactly correct! This is why it's important to set up a web of
trust in PGP (sign the keys of the people you trust, etc).

https://www.gnupg.org/gph/en/manual.html#AEN554

> because those keys live on the same compromised server

(Sorry, I just couldn't let this one pass.) The entire point of PGP is
that it's public key cryptography, so (in that scenario) the developers
would only be submitting their *public* keys to the server (and would
keep the private bits, well, 

[fossil-users] SHA1 and security

[Eduard]

Hi,

I wish to discuss the issues surrounding the use of SHA1 in Fossil and
their consequences, as well as propose several possibilities to deal
with them.

I would like to take a moment to define collision resistance and
second-preimage resistance. A hash H is collision-resistant if it is
infeasible to come up with x1 and x2 such that H(x1)=H(x2). A hash H is
second-preimage resistant if given some x1, it is infeasible to come up
with x2 such that H(x1)=H(x2). Of course, collision resistance implies
second-preimage resistance (but not the other way around).

First I propose that the use of SHA1 in Fossil is a serious problem.
Even if no second-preimage attack is ever successful against it, a
collision attack is currently considered possible (although expensive) [1].

How much damage can be done given the capacity to generate collisions?
Suppose the attacker generates two versions of file "main.c" that share
the same SHA1 hash, one which is malicious ("main-malicious.c") and one
which is clean ("main-clean.c"). If the attacker can intercept
communications between the server and a developer, the attacker can push
"main-malicious.c" to the server, intercept the sync between the server
and developer and substitute "main-clean.c" for "main-malicious.c", then
wait until the developer tags and/or signs the change. Moreover, it will
appear as if the developer has PGP signed the malicious version!

If the attacker is in control of the server, then this becomes even
easier; push the clean version, tell the developer to tag/sign/approve
their checkin, then shun the clean version and replace it with the
malicious one.

If a project is hosted on multiple mirrors that periodically sync with
each other and the attacker knows that the main developer tends to use
only one of them, the attacker can push the clean version to the mirror
that is used by the main developer, and simultaneously push the
malicious one to the other servers.

These concerns are only amplified as the price of generating full SHA1
collisions drops (by further cryptanalytic advancement or by
technological improvements in computing).

Hoping that I have convinced you that this is a serious problem, I would
like to discuss the ways to tackle it.

The first solution is to do nothing and just tell users not to sync with
untrusted repositories. Given the distributed nature of software (and
otherwise) development, I believe it is a difficult burden to impose
upon developers that all contributors always be carefully vetted, and
that third-party (web) hosting never be trusted. I feel that this also
breaks the "eternally incorruptible" promise of Fossil.

The second solution is to incompatibly change the Fossil specification
and replace SHA1 hashes with BetterHash (for some value of BetterHash;
discussion below) in the definition of an artifact ID. This is a
*breaking* change, and requires the *modification* of artifacts (which I
believe is frowned upon in the fossil community to say the least). This
would break older hyperlinks (which would be easy to fix automatically
just by replacement when porting the artifacts to the new format), and
most definitely breaks older PGP clearsigned checkins (which would have
remained secure as long as SHA1 second-preimage attacks are infeasible).
The main advantage to this approach is that it is the most elegant and
easy to understand and deal with. The main disadvantage is that porting
artifacts to the new format requires their modification (which breaks
the "artifacts never change" promise; I would like to note that that
promise would also be broken as soon as an attacker inserts an artifact
for which a SHA1 collision is known).

The third solution is to change the Fossil specification to redefine the
artifact ID to be the concatenation of the SHA1 and BetterHash hash
digests, and allow 40 hexadecimal digit IDs as prefixes. One can show
that the preimage- and collision-resistance of this combination is at
least as good as the strongest of the two. The main advantage of this
approach is that it is not a breaking change, and does not require the
modification of older artifacts (hyperlinks stay the same too). The main
disadvantage is that if SHA1 preimages become feasible, an attacker can
definitely go back and mess with the pre-change SHA1-only artifacts (and
thus corrupt repositories, or worse). Another disadvantage is that the
SHA1 part of the ID takes up extra room and extra computing time with no
benefit in security.

As for the exact value of BetterHash, I would like to nominate
BLAKE2b-512 [2]. It is faster than both MD5 and SHA1, it is based upon
BLAKE which has received a lot of cryptanalytic attention during the
SHA3 competition, and it retains a large security margin (the best
(academic) attack to date is on a reduced version that does only 2.5
rounds instead of 10, and even then only downgrades the security from
512 to 481 bits).

P