Re: [fossil-users] REST API and client for same

[Stephan Beal]

On Sun, Apr 2, 2017 at 11:38 PM, Warren Young  wrote:

> On Apr 2, 2017, at 2:48 PM, Stephan Beal  wrote:
> >
> > a) that's essentially what the JSON API is
>
> …minus the lightweight Subversion-like client, of course.
>
> But, it’s good to know that most of the work is already done.
>

A good deal of it is. There are certainly a few holes in the API.


> > with the notable exception of missing blob support (since JSON has no
> definition for blobs).
>
> When I said that the default return representation should be JSON, I did
> not mean that all data must be in JSON format.  I just mean that, when no
> better representation exists, the API should use JSON.
>

i meant more for pushing new data, e.g. commits. Commits can't be done
without a checkout, though (not without significant changes, anyway).
Fetching binary blobs directly with JSON can't be done without an
intermediary format (base64 or whatever) or (my preferred solution) the
JSON sends back links which, when visited, return the raw blobs. That would
be one solution for such cases as downloads, as you mention here:


> Contrast the Fossil HTTP API here, where sometimes pulling a specific file
> shows it in the browser as-is, sometimes it displays it inline with other
> material, and sometimes it downloads it.





> With this REST API, I’d expect to get a file download every time, because
> the server cannot make any assumptions about the nature of the client,
> particularly about the display semantics.  Web applications like Fossil UI
> often make such assumptions.  The REST API would be more policy-free, in
> this sense.
>

Agreed. The current JSON API makes no assumptions about how the data will
be consumed/rendered.

> A strictly-REST-compliant API
>
> Who specifies “strict” REST?


> All I see are a bunch of cliques and camps, no actual standards.
>

Good point. AFAIK there is no official definition (or even RFC). i was
going by past experience with at-work projects, where my colleagues are
sticklers for differentiating between PUT/POST (insert vs. update).


That said, if some of this didn’t work with CGI, I don’t see a problem with
> that, since it is merely one alternative way to access Fossil, not the main
> one.


CGI accounts for 100% of my fossil access :). (That was the killer feature
for me when i very first looked at fossil.)


>   If the requirement is that you use “fossil server” with this, that seems
> fine to me.
>

But that immediately excludes everyone who doesn't have their own root
server (or otherwise has access to run server processes full-time). The
majority (i suspect) of us have $5/month hosters which support PHP and CGI,
but have no way to run servers jobs full-time.

(Apologies for my brevity - my left hand is offline again.)

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] XSS attack and fossil hosting services

[Stephan Beal]

On Mon, Apr 3, 2017 at 5:28 AM, Eduard 
wrote:

> Hi,
>
> I recently realized that fossil repository hosting websites (such as
> chiselapp  and hydra
> ) are vulnerable to arbitrary HTML
> injection (XSS) as soon as they give (untrusted) users the 'setup'
> capability to the repositories they create.
>
Giving an untrusted user setup rights is kind of like giving them your bank
card and its PIN. Setup rights are only intended for the person (or people)
who physically maintain a repository, not untrusted users.


> As a related and more general question, what damage can a "fossil http -R
> $repo" command do to surrounding files/other repositories? In particular,
> using TH1/SQL or using the JSON interface?
>
-R $repo limits that server process's access to that one repo. There are no
services (other than "login groups") which span repos. TH1, if you add TCL
support to it, can run any arbitrary commands which TCL allows, so all bets
are off in that case ("rm -fr /etc"). Only setup user(s) should be able to
add TH1/TCL code to a repo, though, and if you have "untrusted" setup users
then (again) all bets are off. That's not a technical/fossil problem, but
an organizational one.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] SSL on Mac

[J. Cameron Cooper]

I had Fossil installed on a seldom-used Mac and it worked fine. Trying it
again today, it failed to connect to Chisel::

$ fossil version
This is fossil version 1.33 [9c65b5432e] 2015-05-23 11:11:31 UTC
MacBook-Pro:bottlemarkapp hercooper$ fossil up
Autosync:  https://jccoo...@chiselapp.com/user/jccooper/repository/bott
lemark
SSL: cannot connect to host chiselapp.com:443 ()
Pull done, sent: 0  received: 0  ip:
Autosync failed.
Cannot proceed with update

Maybe it's old. So I grab the latest version::

$ sudo mv ~/Downloads/fossil /usr/local/bin/fossil
$ fossil up
dyld: Library not loaded: /usr/local/opt/openssl/lib/libssl.1.0.0.dylib
  Referenced from: /usr/local/bin/fossil
  Reason: image not found
Trace/BPT trap: 5

Same message with any command and all three versions available on the
website. Perhaps it's something changed on the OS, but it seems like that
shouldn't be a problem with a standalone executable. Though perhaps I'm
missing something; I don't know much about OS X internals.

Any idea?
-- 
J Cameron Cooper
jccoo...@gmail.com
jccoo...@cooperdigital.com 
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] REST API and client for same

[Warren Young]

On Apr 3, 2017, at 1:29 AM, Stephan Beal  wrote:
> 
> Commits can't be done without a checkout

Given a way to ask Fossil over HTTP for the set of artifacts that makes up 
$reference, where the latter is anything Fossil currently accepts in “fossil up 
$reference” you’ll then have a way to check out that version:

$ frapi checkout https://m...@fossil-scm.org
Password, remember, etc.
…fossil-scm/ created and populated with tip-of-trunk
$ cd fossil-scm
…hack, hack, hack…
$ frapi ci
…diff checkout against pristine copies, send diffs up to server

This hypothetical lightweight client (which I’m calling frapi) would then 
record the version it checked out, so that on checkin, it can run the diffs 
against the pristine copies it squirreled away (like Subversion does) and 
submit the diffs to the backend to be checked in or rejected.

On success, it gets back a new checkin ID, which it records in the same place 
it recorded the initial checkin ID, ready to go again.

I think a local JSON file would make sense for that place, again a lightweight 
alternative to .fslckout and ~/.fossil.  This client already has to have a JSON 
parser built in, but does *not* have a reason for an embedded SQLite DB.

The use case that prompted all of this — and it is admittedly silly — is 
thinking about what it would take to port Fossil to a PDP-11.  Whether anyone 
actually does that is not the point.  The point is that such constraints should 
guide the design of this lightweight client.  

Such a client would necessarily be different from the current Fossil 
implementation.  We must get something new and different out of it, else why 
bother?

Incidentally, I don’t mean with all of this to disrespect Fossil itself.  I’m 
not proposing replacing it because it’s bad.  I’m just proposing a…shuttle 
craft, for use when we can’t fly the Enterprise itself on the mission.  The 
shuttlecraft always comes back home, and we can’t always use them, as when 
Ubering around Lwaxana Troi.  Gotta use the whole Enterprise for such things, 
you know. :)

> Fetching binary blobs directly with JSON

While that might be useful, I’m proposing that there be a way to get artifact 
data raw, directly over HTTP, just like downloading a file.

This is what I meant by JSON only being a default, when no better 
representation exists.  For “frapi up $version” and such, that better 
representation is “raw.”  Content-Length, binary data, the whole bit.

I suppose it could optionally be delta-compressed against the version the 
client claims to have, to save bandwidth.

> (my preferred solution) the JSON sends back links which, when visited, return 
> the raw blobs.

While that sounds like a useful API, a two round-trip solution introduces the 
possibility of race conditions.

For “frapi up”, you’d want a way to say “I have version $uuid, give me the tip 
of that artifact’s branch, optionally delta-compressed.”

This is also what I meant by this REST API being its own thing, designed with 
stateless HTTP in mind: you give everything in the request that tells the 
server what it needs to know to process your entire request in one go.  
Ideally, every operation takes only one round-trip.  The server carries the 
bulk of the processing workload.

That not only avoids race conditions, it means you pay for only one HTTP 
round-trip delay.

> i was going by past experience with at-work projects, where my colleagues are 
> sticklers for differentiating between PUT/POST (insert vs. update).

Yes, all experienced web devs know exactly how it should be done…and no one 
agrees. :)

It’s spaces vs tabs and emacs vs vi all over again.

The point here, though, is that doing everything with GET and POST is entirely 
doable and legal within the REST concept.

> CGI accounts for 100% of my fossil access :). (That was the killer feature 
> for me when i very first looked at fossil.)

I’m not suggesting that we change anything that already exists.  I’m just 
saying that if, for some reason, the person implementing this decided that they 
absolutely positively had to use the PUT verb for one new API that only frapi 
used, that would be fine by me.

> The majority (i suspect) of us have $5/month hosters which support PHP and 
> CGI, but have no way to run servers jobs full-time.

$5 gets you a root access VPS these days, plenty big enough to run Fossil.  
Shop around.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] REST API and client for same

[Stephan Beal]

On Mon, Apr 3, 2017 at 3:25 PM, Warren Young  wrote:

> On Apr 3, 2017, at 1:29 AM, Stephan Beal  wrote:
> >
> > Commits can't be done without a checkout
>
> Given a way to ask Fossil over HTTP for the set of artifacts that makes up
> $reference, where the latter is anything Fossil currently accepts in
> “fossil up $reference” you’ll then have a way to check out that version:
>

Unfortunately my aching hands won't let me respond nearly as fully as i'd
like, so i'm just gonna leave it at:

emacs

;)

(Today the doc finally passed me on to a neurosurgeon, meaning the way for
an operation is paved. With any luck, i'll be able to type full-time again
within a few months.)

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] XSS attack and fossil hosting services

[Eduard]

On 04/03/2017 02:16 AM, Warren Young wrote:
> On Apr 2, 2017, at 9:28 PM, Eduard  wrote:
>>
>> An attacker can place malicious javascript at the top of every page
> 
> Certainly.
> 
>> they could, for example, change the victim's password
> 
> Doesn’t the login cookie prevent the hosted user from doing that to any but 
> their own repository, at least through normal HTTP URLs?  (As you say, TH1 
> and SQL may well be a different matter.)
> 
> Setup privilege isn’t given to all users of a repository, only to the most 
> trusted administrator(s) of that repository.
> 
> Unless Fossil erroneously lets you log into repo A and then have the same 
> privileges on repo B hosted by the same Fossil server, I don’t see the 
> problem.  Any wounds would be self-inflicted.

Not true. Suppose that good-user registers
"https://example.com/f/goodproject; and evil-user registers
"https://example.com/f/evilproject; (both have setup privileges on their
respective repositories but don't trust each other). Good-user spends
most of their time logged into goodproject. Evil-user then convinces
good-user to visit evilproject while logged into goodproject; javascript
on evilproject (running on good-user's browser instance) can make
queries to and receive results for all pages on https://example.com/
(according to same-origin policy), including goodproject; it of course
also has access to all of the good-user's cookies. That JS code can do
anything that good-user can do on https://example.com/.

The usage scenario to consider here is that of a large repository host
(like github/sourceforge) which doesn't trust its users (since anyone
with an email address can register an account).

>> (Another way to fix it is by giving each repository a separate subdomain
> 
> Since ChiselApp is using HTTPS, they must be using some kind of front-end 
> proxy, which could redirect each repo URL to a different Fossil instance.  
> That in turn could be run as a chroot or jail, which would mitigate most of 
> the problems you bring up, if not all.
> 
> It’s not hard.  I do this on for my own public repositories.
> 
> It does mean you’re limited in the number of Fossil instances you can run on 
> a single server by RAM and private TCP port numbers, but I shouldn’t think 
> you’d want more than a few tens of thousands of Fossil instances on a single 
> machine anyway.

Sorry, by "it" in "Another way to fix it" I meant the XSS vulnerability.

Best,
Eduard
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] XSS attack and fossil hosting services

[Warren Young]

On Apr 2, 2017, at 9:28 PM, Eduard  wrote:
> 
> An attacker can place malicious javascript at the top of every page

Certainly.

> they could, for example, change the victim's password

Doesn’t the login cookie prevent the hosted user from doing that to any but 
their own repository, at least through normal HTTP URLs?  (As you say, TH1 and 
SQL may well be a different matter.)

Setup privilege isn’t given to all users of a repository, only to the most 
trusted administrator(s) of that repository.

Unless Fossil erroneously lets you log into repo A and then have the same 
privileges on repo B hosted by the same Fossil server, I don’t see the problem. 
 Any wounds would be self-inflicted.

> In particular, the ability to run arbitrary TH1 and SQL may also need to be 
> restricted/disallowed (because a user could just modify the config table 
> using SQL, for example).

I can only see that being possible if embedded SQL is allowed to attach to 
arbitrary other SQLite DB files.  Without that ability, all SQL executes within 
the same repo that had the JS code added to it, which isn’t a problem per se, 
because you’ve given the user who added the JS code Setup privilege.

As for TH1, I’ve not looked into it, and so don’t know how close it is to Tcl 
in terms of capability.  It may be sufficiently restricted that it can’t do 
anything evil outside of the particular repo it runs atop.

> (Another way to fix it is by giving each repository a separate subdomain

Since ChiselApp is using HTTPS, they must be using some kind of front-end 
proxy, which could redirect each repo URL to a different Fossil instance.  That 
in turn could be run as a chroot or jail, which would mitigate most of the 
problems you bring up, if not all.

It’s not hard.  I do this on for my own public repositories.

It does mean you’re limited in the number of Fossil instances you can run on a 
single server by RAM and private TCP port numbers, but I shouldn’t think you’d 
want more than a few tens of thousands of Fossil instances on a single machine 
anyway.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] XSS attack and fossil hosting services

[Warren Young]

On Apr 3, 2017, at 11:15 AM, Eduard  wrote:
> 
> Evil-user then convinces
> good-user to visit evilproject while logged into goodproject

Ah, I see.  Yes, I agree now.

>>> (Another way to fix it is by giving each repository a separate subdomain
>> 
>> ...run as a chroot or jail, which would mitigate most of the problems you 
>> bring up, if not all.
> 
> Sorry, by "it" in "Another way to fix it" I meant the XSS vulnerability.

Yes, I’m just giving another way to solve the problem, until the ones you want 
solved get some developer attention.

Running these repos in individual jails or chroots is a good idea anyway.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users