Re: [fossil-users] No rule to make target 'src/email.c', ...

[Eric Dillon]

Nice quick fix. Compiles now. Thanks.

On Thu, Jun 21, 2018, 10:03 AM Richard Hipp  wrote:

> On 6/21/18, Eric Dillon  wrote:
> > Fails to compile on Win32 VS2013 (12.0)
> >
> > email.obj : error LNK2019: unresolved external symbol _popen referenced
> in
> > function _email_send
>
> Thanks for the report.  Fixed now.
>
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] No rule to make target 'src/email.c', ...

[Eric Dillon]

Fails to compile on Win32 VS2013 (12.0)

email.obj : error LNK2019: unresolved external symbol _popen referenced in
function _email_send

On Wed, Jun 20, 2018, 3:51 PM Florian Balmer 
wrote:

> Richard Hipp:
>
> > It is reassuring to know that so many people routinely build Fossil
> > from the trunk sources :-)
>
> À propos build:
>
> I have accidentally set FOSSIL_ENABLE_MINIZ=1 (in win/Makefile.msc),
> and this doesn't seem to work, probably because src/shell.c includes
> zlib.h without examining FOSSIL_ENABLE_MINIZ.
>
> Not sure if this is important, or if it's even possible for
> src/shell.c to support FOSSIL_ENABLE_MINIZ, but I'd rather let you
> know.
>
> --Florian
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil-users Digest, Vol 121, Issue 9

[Eric Dillon]

>From "fossil ui", go to "admin > settings". In diff-command put the path
and executable name for winmerge.

My winmerge is a portable installation, so my diff-command is set as ...

C:\bin\winmerge-2.14.0\winmergeu.exe

Works great.

On Tue, Feb 13, 2018, 1:30 PM  wrote:

> I love WinMerge and was unable to trigger the diff from fossil(many
> versions ago) as you suggest?
> Can you describe how WinMerge is installed and fossil settings that worked
> for you?
> For now, I cut and paste, which is a drag.
>
> Thanks!
>
> On Tue, Feb 13, 2018 at 1:11 PM, Ron W  wrote:
>
>> On Tue, Feb 13, 2018 at 7:00 AM, <
>> fossil-users-requ...@lists.fossil-scm.org> wrote:
>>>
>>> Date: Tue, 13 Feb 2018 11:42:59 +0100
>>> From: Gilles 
>>> Subject: Re: [fossil-users] Windows GUI that allows diff between two,
>>> revisions?
>>>
>>> Thanks for the tip. Fuel does have an internal web browser:
>>>
>>>
>>> https://s14.postimg.org/efgcfmwj5/Fossil.Fuel.history.internal.browser.png
>>>
>>> If there's no Windows alternative to Fuel, it looks like it's either the
>>> web UI or the command line.
>>>
>>> Is there an easy way to compare two revisions of a file while the
>>> repository is closed (ie. not a comparison between a revision in the
>>> repo and a live file in the workspace), possibly using WinMerge, so I
>>> can easily see what the changes are, check out the latest verison, and
>>> copy/paste ?
>>>
>>
>> In the Fossil "web UI", from the timeline, select a commit. Scroll down
>> to the file of interest and click on it to show that file's history. In the
>> "revision graph" (on the left), click the version you want to compare from
>> (the node will turn red), then the version you want to compare to. This
>> will display (in the web browser), a differ of the 2 versions.
>>
>>  To use WinMerge, you need to configure Fossil to use that as the
>> external diff (or gdiff) tool. then from the command line:
>>
>>fossil diff --from VERSION --to VERSION
>>
>> This will launch WinMerge (with temporary copies) to display the
>> differences between the 2 versions. However, Fossil will ignore any changes
>> you make in WinMerge.
>>
>> Until either Fuel is updated (possibly by some one forking it) or some
>> one creates a new GUI front end for Fossil (possibly using Electronic or
>> similar), this is the best you can get.
>>
>>
>> ___
>> fossil-users mailing list
>> fossil-users@lists.fossil-scm.org
>> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>>
>>
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] strange problem with graphical diff

[Eric Dillon]

On Wed, Aug 23, 2017, 12:19 PM Richard Hipp  wrote:

And it should now be fixed on trunk.
>

Verified. Thanks.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] strange problem with graphical diff

[Eric Dillon]

Here's a bisect of the error on Windows.

11:39:32 $ fo bisect good
bisect complete
  2 BAD 2017-08-21 12:18:49 810dd031ec3ee1d6
  5 BAD 2017-08-12 18:24:29 9eea719af68932ad
  6 BAD 2017-08-12 18:20:26 9690d370e0714a43
  7 BAD 2017-08-12 18:15:36 3b191c984b831571
  8 GOOD2017-08-12 04:19:56 45a3d4b1670f123e CURRENT
  4 GOOD2017-08-11 16:00:55 3ebbe7bcaa8f2221
  3 GOOD2017-07-28 18:41:13 04de083ec842c3f2
  1 GOOD2017-07-10 18:19:00 b1a7527b73a906a0


On Wed, Aug 23, 2017, 11:20 AM Richard Hipp  wrote:

> On 8/23/17, j. van den hoff  wrote:
> > unable to create directory /var
>
> You might be able to work around the problem by adding the
> --no-dir-symlinks command-line option.
> --
> D. Richard Hipp
> d...@sqlite.org
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] strange problem with graphical diff

[Eric Dillon]

Here is the error generated in Windows. Works correctly without the --to
parameter.

10:19:38 $ fo gdiff --from d5b015 --to 810dd0

Index: src/http_transport.c
==
the
[C:\Users\xx\AppData\Local\Temp\/src/http_transport.c-v1-6pCA3817v8hKvMt.txt]
argument to the "C:\BIN\WinMerge-2.14.0-exe\winmergeu.exe" command contains
a character (ascii 0x5c) that is a security risk

On Wed, Aug 23, 2017, 10:24 AM j. van den hoff  wrote:

> On Wed, 23 Aug 2017 17:14:05 +0200, Warren Young 
> wrote:
>
> > On Aug 23, 2017, at 7:21 AM, Richard Hipp  wrote:
> >>
> >> On 8/23/17, j. van den hoff  wrote:
> >>> unable to create directory /var
> >>
> >> It is trying to create a temporary file in which to store the one of
> >> the two sides of the diff.  Can you trace the problem by running in a
> >> debugger?
> >
> > This sounds like a repeat of:
> >
> >
> https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg24920.html
> >
> > Basically, it’s an artifact of the way macOS symlinks /var, /tmp, etc.
> > to their actual locations, which doesn’t happen on other *ix type
> > systems, so the bug is never tickled there.
>
> yes, this looks like the exact same problem, see my previous mail: fossil
> detects that `/var' exists while not being a directory and bails out
> (which it should not but rather resolve the link and check whether the
> resolved link is a directory...).
>
> I reiterate that I wonder what has changed recently (either on the side of
> OSX or fossil) since the problem was definitely non-existent some time
> ago? and why does it work when omitting `--to' from the `gdiff' call? no
> idea...
>
> more important: is a fix/work-around possible (apart from telling me to do
> it myself which I would have a hard time with ...)
>
>
> > ___
> > fossil-users mailing list
> > fossil-users@lists.fossil-scm.org
> > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
>
> --
> Using Opera's revolutionary email client: http://www.opera.com/mail/
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
>
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Purging data from Fossil

[Eric Rubin-Smith]

>
>
> (A)  Suggest a better name than "fossil trim"
>
> (B)  Define the syntax of ARGS.
>
> (C)  Define a safety mechanism that allows content to be restored if
> it is accidentally trimmed when there are no other repos available
> with which to sink.  Perhaps the trimmed content gets written into a
> separate "trash-can" database?
>

You might consider this from the converse perspective, allowing 'fossil
export' to a new fossil repo, but specifying some sort of complex boolean
selector on export.  Then you at least get the safety for free since the
source repo is only read from.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

[Eric Rubin-Smith]

> On May 27, 2016, at 12:26, Andy Gibbs  wrote:
> 
> Hi,
> 
> I've just had a very, very odd experience with fossil.  I'm running version 
> 1.34.
> 
> Let me first explain what I have done.
> 
> I cloned a respository off our server.  I then went into the clone's web UI 
> and disabled the auto-sync feature.  I then made 7 commits, the first of 
> which caused a trunk fork which I then "repaired" by branching the old 
> tip-of-trunk that caused the fork.  I then committed two commits to the new 
> branch, then realised that I'd committed the wrong code, so shunned the last 
> two commits, rebuilt the respository, unshunned the sha1 signatures, rebuilt, 
> then recommitted the last two commits using the correct code.
> 
> Ok, a little complicated but nothing too out-of-the-ordinary, I hope.
> 
> Except ... now I have two utterly random commits in my cloned repository - 
> both seemingly from the sqlite repository:
> 
> 2014-12-09 [f66f7a17b7] Version 3.8.7.4 (user: drh, tags: release, 
> version-3.8.7.4)
> 2011-05-19 [ed1da510a2] Version 3.7.6.3 release candidate 1. (user: drh)

If you commit a file that looks like a fossil manifest, then fossil treats it 
as such and displays the commits listed therein as if they happened in your 
actual repository.  Or something. I had this happen to me too a while back when 
I did the same thing, importing the SQLite tree wholesale into another fossil 
repo.  Drh said at the time that there's no "real" corruption.  

Sent from my iPhone


> 
> Now, it is true that I have a clone of the sqlite respository on my server 
> too ... but I haven't yet synced with the server.  Absolutely no server 
> communication has happened since my initial clone.  And these odd artifacts 
> were definitely not there (or, rephrased, definitely not showing) when I was 
> mid-way through all of this work, and are not there on the server's copy of 
> the repository either.
> 
> Unfortunately, I cannot say exactly when these artifacts appeared, but my 
> hunch would be somepoint around the shunning/rebuilding process?  Does this 
> even make sense?!?
> 
> Worse, I am left not sure whether to simply shun the two random artifacts and 
> then push the changes to the repository up to the server.  It has taken the 
> best part of a day to get all these commits in and it represents about 6GB of 
> source files (the repository has doubled from ~700 MB to ~1.5 GB) requiring a 
> lot of manual "fossil mv"s to synchronise many moved files (fossil doesn't 
> yet support a git-like auto-mv-detection sadly).
> 
> Any idea how I can easily check the validity of my repository?  I have done 
> the obvious which is to check out each of the recent check-ins and compare 
> them with the original source folders, but what I don't know is whether the 
> structure of my respository is damaged in some way...
> 
> Help :o)
> 
> Thanks!
> Andy
> 
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Conversation with a CM guy

[Eric Rubin-Smith]

>
>
> So far, none of the IDEs I've used seem to support VCS merges from within
> the IDE. I've always had to go to the VCS itself (or, when using Hg or SVN
> on Windows, TortoiseHg or TortoiseSVN), so lack of merging in libfossil
> might not be a big issue for creating Fossil plug-ins for IDEs.
>
> A GUI for Fossil would benefit from merge support in libfossil, but could
> work around it.
>
> I'll think about taking over libfossil. My main concern is that my
> employer's legal department won't allow me to sign the Fossil copyright
> assignment document.
>

EGit, the git plugin for Eclipse, does merges.

The existence of the Eclipse plugin is the only reason we went with Git for
the project rather than Fossil -- some of the devs in my team seem only be
able to do work via Eclipse and have no experience with the command line
etc.

The merge feature works, until it doesn't.

A week or two ago EGit worked together with git, Eclipse, and a coworker to
lose some of the coworker's code.  I'm not really sure how it happened.
You just look into the VCS history tree and you see a commit with the
message "Inconsistent merge state" (a message generated by EGit).  The
commit tree is hard to understand from the available GUIs, so that doesn't
help.

The message would more rightly have said "I'm deleting some of your code
not telling u which files haha sux 2 B U".

I think it had something to do with making extra changes on top of an
uncommitted merge that had had conflicts, which then confused git and/or
EGit down the line.  But I don't really know.

In any case, it wasn't until yesterday that the coworker noticed a bunch of
his code was missing.  I had to go perform some git-fu (at which I'm a
mid-ranking white belt at best) to figure out what had been lost to some
measure of certainty.  Then the coworker just applied the patch I had
generated by hand to the latest commit in his branch, and we were off to
the races again.  Pretty horrible.

So, there's a market of at least 1 team that would love a Fossil plugin for
Eclipse.  It would be ironic if Fossil's killer use case wound up being its
support of a best-of-breed IDE plugin.  Fossil's original author is
definitely not a big IDE guy. :-)

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Conversation with a CM guy

[Eric Rubin-Smith]

>
>  employability.


It takes less than a day to pick up git if you're used to fossil.  So I
don't really think it makes a huge difference as to future employability
unless the hiring manager is looking for the wrong things.

I grant that most hiring managers *do* look for the wrong things, but let's
not base decisions in our current project around the industry's silly
hiring practices.

My $0.02,
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Semi-annual drumming-up-of-support for libfossil

[Eric Rubin-Smith]

On Thu, Jan 21, 2016 at 11:36 AM, Stephan Beal <sgb...@googlemail.com>
wrote:

> On Thu, Jan 21, 2016 at 5:27 PM, Christopher M. Fuhrman <
> cfuhr...@pobox.com> wrote:
>
>> Is your Caps-Lock key to the left of the 'A' key on your keyboard?  If so,
>> I've had good luck swapping the Caps-Lock key with the Control key so my
>> left pinky doesn't have to do gymnastics to hit Cntrl all the time.
>>
>
> i've tried remapping those but just can't deal with. My fingers refuse to
> make that switch. Because i do so much typing with one hand now, i actually
> use the caps lock (for the first time since i started typing, in 1983!).
> It's generally a useless key for 2-handed people, but it's useful when
> you're limited to 1 hand.
>

I saw a talk some years ago by a temporarily-disabled programmer who
learned how to code by speech, eventually becoming faster that way than he
had been with two healthy hands.  Perhaps you'd be interested in checking
out his methods.

https://www.youtube.com/watch?v=8SkdfdXWYaI

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Rewrite of fossil-v-git.wiki. Was: Fossil mentioned on HN

[Eric Rubin-Smith]

>
> > >> Would be nice if there was a "fossil ticket export" command that would
> > >> produce a "proper" CSV file. {snip}
> > > A problem with CSV is that there really isn't a clear definition of it
> at
> > > its edge cases other than testing what Excel will import correctly.
> >
> > Which is a sorry state of things, in itself. Reading an article about
> support
> > of CSV files in PostgreSQL {snip}
>
> If you want to get files with newlines etc into Excel, your best bet is
> always the libreoffice round. Alternatively, creating Office XML might
> also be an option.{snip}


Maybe JSON is a better bet.  Fossil could support a JSON ticket dump in
some standard form.  And hey, we can reuse someone's work who has already
figured out a decent way to represent a ticket database in JSON:

https://confluence.atlassian.com/jira/importing-data-from-json-495976468.html

So if you follow that form, then you get your Jira export capability for
free. :-)

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Rewrite of fossil-v-git.wiki. Was: Fossil mentioned on HN

[Eric Rubin-Smith]

On Wed, Dec 16, 2015 at 11:16 AM, Richard Hipp <d...@sqlite.org> wrote:

> Based on comments on HN and on this mailing list, I have attempted to
> rewrite the fossil-v-git.wiki document
> (https://www.fossil-scm.org/fossil/doc/trunk/www/fossil-v-git.wiki) to
> better summarize the differences between the two products.
>
> Your feedback on the rewrite, and especially suggestions for improving
> it, are welcomed.
>

The doc says:

> If you start out using one DVCS and later decide you like the other
better, it is easy to change
<https://www.fossil-scm.org/fossil/doc/trunk/www/inout.wiki>.

DRH and crew have defined "easy to change" (meaning easy to migrate away
from) in a way that is misleadingly narrow for many people.  The reason is
that if you are using Fossil with its ticket tracker, then there is no
simple way to export those tickets to any current member of the Git
ecosystem that does ticket tracking (e.g. Jira).

Providing an outbound export interface to at least one such tool would
further strengthen the claim that it is easy to migrate away from Fossil.
Otherwise, users will be "locked in" in the sense that their tickets are
stuck in Fossil, where it is hard to properly cross-reference them with the
ongoing stream of check-ins after export to Git.

In the past, DRH has criticized my above observations by saying that since
Fossil is backed by an SQLite database which the user may query at his or
her leisure, this should suffice for an export capability for tickets.
This criticism does not hold much water, however, since we can observe the
following:

(1) The raw check-in data is *also* backed by an SQLite database.  But the
Fossil dev team saw fit to write an export-to-git capability on top of
that.  This is an internal inconsistency in DRH's argument, since
ostensibly the devs don't think the git export capability was a mistake.

(2) The reasons for having written an export-to-git capability are good
reasons.  Analogously to Word's ability to export to PDF, the application
in question knows both the source and target data formats and how to do the
relevant translations.  Indeed, the whole purpose of export can be seen as
relieving the user of the burden of understanding the source and target
data formats and performing the conversions.  (In a prior thread on this
topic, someone criticized this point by saying that Microsoft has more
developer power than does Fossil.  This, while true, is irrelevant to
questions about whether such an export feature would be useful.)  And in
particular, merely having a well-understood data format does not suffice:
Word cannot claim to have the ability to export to PDF *merely* because its
internal data format is public.

(3) The reasons in (2) generalize to the case of tickets.

DRH has made another criticism of my observations which was that no other
ticket manager permits easy export of its tickets to a 3rd-party ticket
manager.  Whether this is true or not, it is again irrelevant to my claim
that Fossil's ticket system is not, in fact, easy to migrate away from.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] allow downloading wiki page attachments for users without Read permission

[Eric Rubin-Smith]

A user with the following permission flags:

bcfhjkmnprtw

for a site in which the virtual users Reader, Developer, Anonymous and
Nobody have no default privileges, cannot download an attachment directly
from a wiki page attachments list.  The list points to URIs such as this
one:

https://www.example.com/artifact/66a9b2083b1ad09a

This is because the Read permission is needed to download a raw artifact.

However, the same user may access the attachment via a different URI:

https://www.example.com/attachdownload?file=file_name.exe=Wiki+Page+Name

This bounces the user through a code path that does this (attach.c):

  g.perm.Read = 1;
  cgi_replace_parameter("name",zUUID);
  if( fossil_strcmp(g.zPath,"attachview")==0 ){
artifact_page();
  }else{
cgi_replace_parameter("m", mimetype_from_name(zFile));
rawartifact_page();
  }

i.e. pretend the user has read permission for this request, and press on to
download the thing.

So it would be more convenient for the wiki page to link to the permissive
URI rather than the restrictive one.

Note that a similar issue exists with tickets.  A user permitted to view
tickets should be able to conveniently download the objects attached to the
ticket.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Fossil does not generate https link prefix

[Eric Rubin-Smith]

On Thu, Nov 5, 2015 at 6:53 AM, Richard Hipp <d...@sqlite.org> wrote:

> On 11/4/15, Eduard <eduard.c.dumitre...@gmail.com> wrote:
> > Hi Taras,
> >
> > I've had a very similar problem. I fixed it by setting the "HTTPS"
> > environment variable (for CGI execution) to "on" when the request comes
> > in through https, i.e.
> >
> > 
> >   SetEnv HTTPS on
> > 
> >
> > You might want to remove the "" part if you're only accepting https
> > anyway.
> >
> > I'm not sure whether this is the truly correct way to do it, but it sure
> > solved my problem.
> >
>
> Fossil depends on the HTTPS environment variable to know whether or
> not the inbound request was over http or https.  Without that
> environment variable, Fossil has no way of knowing, and hence has no
> way of knowing whether to generate http: or https: on generated
> hyperlinks.
>


In my Fossil server config (running as an inetd-style program behind
stunnel), I pass the '--https' flag and the '--host' parameter to fossil
when forking an instance.  This may give OP the desired effect.

Full stunnel config:

$ cat stunnel.conf
pid = /home/fossil/stunnel.pid
output = /home/fossil/stunnel.log
RNDfile = /home/fossil/randfile

[fossil-https]
accept = 10443
cert = /home/fossil/www.example.com.pem
key = /home/fossil/www.example.com.key
exec = /usr/local/bin/fossil
execargs = fossil http /home/fossil/my.fossil.db --https --host
www.example.com

For completeness, here is the redirect rule to expose stunnel at port 443
(allowing stunnel to run as user 'fossil' rather than 'root'):

# iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 31728 packets, 1886K bytes)
 pkts bytes target prot opt in out source
destination
 5342  309K REDIRECT   tcp  --  *  *   0.0.0.0/0
0.0.0.0/0tcp dpt:443 redir ports 10443


HTH,
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

On Thu, Nov 5, 2015 at 1:56 AM, Stephan Beal <sgb...@googlemail.com> wrote:
>
> Thanks to Joe for stepping in to stop the bikeshedding :).
>

Yeah.  In that spirit, I will abstain from addressing your other points
from this morning, since I think most of the useful arguments are already
on the table.

Instead I'll just hope that you realize the depth of your error before the
next time the topic comes back up. :-)

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

>
> This issue was more subtle than it originally appeared.  I think the
> current
> trunk
> changes should make it work right for both versioned and non-versioned
> allow-symlinks
> settings.


Thanks so much for looking at that.  I was trying to get started writing
some unit test cases around this so I could submit them back here.  I note
that 69 tests fail in version [2186f0f0e7] (see below).  Though it seems
pretty unlikely (from their names) that your symlink changes caused this.

* End of utf: 69 errors so far **
* Final result: 69 errors out of 27300 tests
* Failures: amend-comment-5.1 amend-comment-5.2 amend-comment-5.3
amend-comment-5.4 clean-6 clean-12 clean-18 relative-tree-name-100.1
relative-tree-name-100.2 relative-tree-name-100.3 relative-tree-name-100.4
relative-tree-name-101.1 relative-tree-name-101.2 relative-tree-name-101.3
relative-tree-name-102.1 relative-tree-name-102.2 relative-tree-name-102.3
relative-tree-name-103.1 absolute-tree-name-100.1 absolute-tree-name-100.2
absolute-tree-name-100.3 absolute-tree-name-100.4 absolute-tree-name-101.1
absolute-tree-name-101.2 absolute-tree-name-101.3 absolute-tree-name-102.1
absolute-tree-name-102.2 absolute-tree-name-102.3 absolute-tree-name-103.1
merge-utf-27-23 merge-utf-27-32 merge_multi-4 merge_renames-5
mv-soft-relative-2 mv-soft-relative-4 mv-hard-relative-2 mv-hard-relative-4
mv-soft-absolute-2 mv-soft-absolute-4 mv-hard-absolute-2 mv-hard-absolute-4
rm-soft-relative-4 rm-soft-relative-6 rm-hard-relative-4 rm-hard-relative-6
rm-soft-absolute-4 rm-soft-absolute-6 rm-hard-absolute-4 rm-hard-absolute-6
revert-1-1 revert-1-2 revert-1-3 revert-1-4 revert-1-5 revert-1-6
revert-1-7 revert-2-1 th1-checkout-1 th1-checkout-2 th1-header-2
th1-footer-2 th1-footer-3 th1-artifact-3 th1-artifact-5 th1-artifact-7
th1-artifact-9 th1-globalState-1 th1-globalState-8 th1-encode64-3

In any case, the three scenarios I am personally aware of behave as
expected now.  Thanks again!

For completeness, Warren Young seems to be complaining about Fossil's path
resolution logic when symlinks are present.  This is (from my perspective)
a very different issue, but you nevertheless may want to track it.  Also, I
did not read every detail in the thread ("xkcd on git") that led to this
subtopic, so perhaps people aired other symlink-related complaints there.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

On Tue, Nov 3, 2015 at 1:33 AM, Stephan Beal <sgb...@googlemail.com> wrote:

>
> On Mon, Nov 2, 2015 at 6:32 PM, Eric Rubin-Smith <eas@gmail.com>
> wrote:
>
>> the user when trying to move a tarball from one OS to another.  In other
>> words, I believe that you perceive a dichotomy that is false (between (a)
>> not implementing symlinks at all and (b) implementing them while having
>> fossil perfectly and automatically solve all complications that may arise).
>>
>
> That sounds like a fair summary of my feelings on SCM support for
> symlinks, but i'd argue that if a system (SCM) cannot do (b) for the vast
> majority of use cases, then it probably shouldn't be implemented at all.
> Why not? Because it's the _displeased_ users who complain the loudest when
> something doesn't work how they want/expect it do, and that percentage of
> users is relatively high in the case of symlinks.
>
> A user who only ever uses fossil on unix should get unix symlink semantics
>> on unix, without quirks or surprises.  Surely you and DRH would agree with
>> that?
>>
>
> i can't speak for Richard, but if i had my way, fossil wouldn't support
> symlinks at all. They are inherently platform-dependent and, IMHO, belong
> in the build process, not the SCM. They are a big can of worms, as we can
> see by large amount of emails on the topic. To be clear: i use symlinks all
> over the place on my systems, i just refuse to use them in an SCM. Call me
> old fashioned.
>
>

:-) Fossil creates a problem by not supporting symlinks properly, and you
use the volume of complaints about the problem to support your claim that
the problem was inevitable.

Not implementing them at all, or implementing them poorly as Fossil has, is
what maximizes complaints -- hence the large amount of emails.  I wouldn't
dream of complaining about Git's support of symlinks, because it just works.

I just don't think this is a deep conundrum.  Lots of cross-platform
programs have to deal with symlinks, have done so, and the universe has not
collapsed from the logical implications.  If the link is broken just fix
it.  If the OS doesn't support symlinks then those see some reasonable
degradation, possibly coupled with warnings.  And those of us who use OSes
with symlink support don't have to suffer needlessly because of the poor
decision-making of other OS authors.


>
> Absolute paths in an SCM are "just plain wrong" (IMO). Even the innocuous
> link to /etc might be wrong in certain build environments (and won't work
> on Windows). Why should fossil assist in doing the wrong thing?
>

Here you give away the game by saying "IMO".  Whether it's wrong is a
question of policy, as you seem to admit.  And such examples have clearly
led you down a path of arguing that your policy is the right one -- to the
detriment of the tool.

Your particular example is actually great for my case, because it applies
with equal force to the case of filesystems.  A filesystem can be mounted
at any mount point, so an absolute symlink to '/etc' that works fine when
the FS is mounted at '/' will not be correct when the FS is mounted at
'/foo/bar'.  So absolute symlinks are also wrong in this case.  Why should
Linux assist in doing the wrong thing?

Well, because it's a really useful tool.  Sometimes the tool can be misused
or uncover some corner cases or whatever, but overall it's really useful.
So let's support it and let the user deal with those weird cases if they
come up (or just wallow in the pure bliss of a useful tool if the cases
don't come up).


> Stated yet another way: we don't expect the SCM to solve all problems that
>> users create.
>>
>
> But if it sets out to solve them, then it should solve them, not provide a
> half-solution to philosophically unsolvable problems. (IMO.)
>

Your argument is analogous to an argument that compilers can't detect and
correct all program bugs, so we may as well not write compilers at all.
This is an enormous problem, you could argue, because almost all programs
have bugs, and so the compiler is doomed to fail almost every time!

Of course you wouldn't say that, because you would rightly notice that this
is a false dichotomy: it's not the compiler's job to detect all program
bugs.

We can support symlinks without setting out to solve all problems that
arise.  It's not a half-solution -- it's a full solution to a narrower
(and, in fact, pretty simple) problem.


>
> For context, my particular use case: we need the openssl source tree in
>> our project, and that tree contains internal symlinks.  Again, people have
>> to jump through silly hoops to get new repos set up properly, because
>> fossil breaks those symlinks by populating new repos with flat text files
>> (and this goes undiscovered til the openssl build fails in myst

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

>
>
> Just to clarify, what are the behavioral changes needed on the Unix side to
> make
> things work seamlessly?
>

(1) Default allow-symlinks to true
(2) Fix bug in which the allow-symlinks setting is not honored while
opening a repository (requires manual clean-up of symlinks after opening a
repo).  I pasted a transcript session a few emails ago -- let me know if
you would like me to re-paste.  I also submitted a patch to this forum
something like a year ago and then detected a bug with it -- let me know if
you would like me to also re-raise that.


> Are there differing opinions on the changes needed (i.e. and not just
> whether or not there should be any changes in the current behavior)?
>

So far this thread has not reached such a stage, afaik.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

On Tue, Nov 3, 2015 at 4:51 PM, Joe Mistachkin <sql...@mistachkin.com>
wrote:

>
> Eric Rubin-Smith wrote:
> >
> > (1) Default allow-symlinks to true
> > (2) Fix bug in which the allow-symlinks setting is not honored while
> > opening a repository
> >
>
> Please try the latest Fossil trunk and let us know if that fixes all the
> issues you were seeing.
>
>
Version [aa92270fe9] seems to have regressed the case of opening a
repository with a .fossil-checkout/allow-symlinks file set to 'on'.  See
the transcript below.  Note that I had created the repository earlier (I
assume this does not matter for the purposes of this test).

[little:test] $ fossil version
This is fossil version 1.34 [aa92270fe9] 2015-11-03 22:09:23 UTC
[little:test] $ fossil set
access-log
admin-log
allow-symlinks
auto-captcha
{snip}
[little:test] $ mkdir sandbox
[little:test] $ cd sandbox/
[little:sandbox] $ fossil open ../foo.db
.fossil-settings/allow-symlinks
foo
foo.lnk
project-name: 
repository:   /home/eas/tmp/test/sandbox/../foo.db
local-root:   /home/eas/tmp/test/sandbox/
config-db:/home/eas/.fossil
project-code: 430c12926d87d5e8e5517f89d786fbd1a5dde989
checkout: 727acad4494a443864efdf56bef74fbdc3787fef 2015-11-03 19:49:14
UTC
parent:   389a88be2cef9727974da9ecc30d10d3df1bae79 2015-11-03 19:48:58
UTC
tags: trunk
comment:  Add files. (user: eas)
check-ins:3
[little:sandbox] $ lh
total 24K
-rw-r--r-- 1 eas eas 3 Nov  3 17:59 foo.lnk
-rw-r--r-- 1 eas eas 4 Nov  3 17:59 foo
[little:sandbox] $ rm -f foo.lnk
[little:sandbox] $ fossil update
UPDATE foo.lnk
---
updated-to:   727acad4494a443864efdf56bef74fbdc3787fef 2015-11-03 19:49:14
UTC
tags: trunk
comment:  Add files. (user: eas)
changes:  1 file modified.
 "fossil undo" is available to undo changes to the working checkout.
[little:sandbox] $ lh
total 16K
-rw-r--r-- 1 eas eas 4 Nov  3 17:59 foo
lrwxrwxrwx 1 eas eas 3 Nov  3 17:59 foo.lnk -> foo
[little:sandbox] $ cat .fossil-settings/allow-symlinks
on
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

On Tue, Nov 3, 2015 at 2:44 PM, Joe Mistachkin <sql...@mistachkin.com>
wrote:

>
> Eric Rubin-Smith wrote:
> >
> > (2) Fix bug in which the allow-symlinks setting is not honored while
> > opening a repository
> >
>
> Did the following changes (a while back) not address this?
>
> https://www.fossil-scm.org/fossil/vinfo/010451e7a5fe116a?sbs=0
>
> If not, in what way are they not adequate?


They seem to address the case where the setting is created via
.fossil-settings/allow-symlinks.  Very cool!

But the transcript I pasted uses a "global" setting because of the order in
which I issued the command for that test.  That global setting does not
seem to be honored on open, even in v1.34.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

> - Symlinks. Now we're getting into file system specifics. Some users
> may want to track them because they find them useful. What about users
> that find FIFOs or block devices or character device useful? Should
> fossil attempt to save enough information to recreate them?
>

Support for FIFOs and block devices is a separate discussion.  My defense
of symlink support does not force me to also defend FIFO and block device
support.


> - Extended attributes. Many file systems now have extended attributes.
> Should Mac users complain because fossil doesn't support HFS+ extended
> attributes or resource forks?
>

They may do so if they think it's reasonable, and be subject to all the
usual considerations around developers' time and attention, etc.  But
again, for the purposes of arguing whether good symlink support is
warranted, that is a separate topic.

> I think fossil can support the permissions level. Anything further
> down in the previous list should be part of the project's build cycle.

My biggest complaint about this discussion is that some folks seem to be
coming at it like fossil is the first tool to confront this issue.  It
isn't.  There are real examples of programs in the wild (SCMs, no less!)
that have solved it in ways that are more coherent, more user-friendly, and
just as easy to implement as what fossil already has.  For Git, our
poster-child for difficult tools, this is just a total non-issue.

Fossil already sort-of attempts to do what Git does, it just has some bugs
and incorrect default config values.  So the distance to fixing Fossil is
quite small.  But closing the gap (and keeping it closed) is much harder
when a vocal subset of the community argues that supporting symlinks is
impossible or ill-advised (it is neither, by my lights).

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] symlinks (was Re: xkcd on git)

[Eric Rubin-Smith]

> My problem is not the decision itself, but that, in terms of how fossil
>>> should behave, it's a philosophical question. Those have no right/wrong
>>> answer, and i dislike seeing software pretend to know the answer to such
>>> questions.
>>>
>>
>> Isn't that essentially confirming my point? Fossil merely stores the
>> pointer. It need not waste time analysing the link to make a judgement call
>> in any way. Just store it and move on.
>>
>
> But if it only stores a pointer, and requires the user to reconstruct the
> link, it's not terribly useful/friendly. The user would potentially have to
> replace fossil's placeholder pseudosymlink file with a link of his own
> (which he could point somewhere else than fossil thinks it "should" be). He
> might has well simply have a "post-checkout" script which sets up his
> symlinks for him. To me, that's the "proper"/"safest" way to handle
> symlinks in a repo (but i'm willing to accept being in the minority on that
> point).
>

Fossil's poor handling of symlinks is a severe knock against it, both in my
opinion and in my experience with proselytizing it to people who are not
already on the fossil kool-aid.

Every time a user opens our repo on unix, the symlinks are initially
populated as regular files whose contents are the link target.  This is
causes me significant embarrassment for recommending fossil when others on
my project have to deal with it.  (I tried pushing through a code patch a
year or so ago to fix this, but I detected some bug or another with it and
I don't think the devs ever accepted the patch (not sure).)

Note that this a Fossil-specific quirk.  Not all cross-platform programs
are equally awkward in their symlink handling -- in fact, most are better.

Stephan: I think that you are overcomplicating the matter, in the sense
that the complications you note can (and should) be passed on to the user
-- in the same way that the program tar(1) might pass on such complications
to the user when trying to move a tarball from one OS to another.  In other
words, I believe that you perceive a dichotomy that is false (between (a)
not implementing symlinks at all and (b) implementing them while having
fossil perfectly and automatically solve all complications that may arise).

A user who only ever uses fossil on unix should get unix symlink semantics
on unix, without quirks or surprises.  Surely you and DRH would agree with
that?

The cases you are worried about:

* absolute paths -- fossil can preserve the absolute path, it's the user's
fault if that's the wrong thing to do.
* broken links -- fossil preserves the original link, it's the user's fault
if the link is incorrectly broken.
* cross-platform semantics -- implement the proper semantics where you can,
and don't where you can't.  A user who only ever uses fossil on unix should
get unix symlink semantics on unix.  If the fossil devs are able to
implement proper symlink handling on next year's Windows, then awesome.  If
not, then degrade to whatever and document it for the user (emitting
warning messages as appropriate or so on).

Stated yet another way: we don't expect the SCM to solve all problems that
users create.  An example is naming branches incorrectly.  If the user
names a branch "" when the project has a rule that branch names should
be descriptive, we don't expect Fossil to flag it and deal with it -- we
expect the user's coworkers to yell at them til they fix it.  So it should
be with symlinks -- each project has its own rules, Fossil doesn't know
about those rules, Fossil just faithfully syncs the target around, and if a
link gets broken in violation of project rules then a user gets yelled at
by another user.  (Note that the project-local rules might be "never use
symlinks because our Windows doesn't handle them how we would like" or "we
think symlinks are always perverse".  Fossil should not IMO be constrained
by such subsets of the user base.)

For context, my particular use case: we need the openssl source tree in our
project, and that tree contains internal symlinks.  Again, people have to
jump through silly hoops to get new repos set up properly, because fossil
breaks those symlinks by populating new repos with flat text files (and
this goes undiscovered til the openssl build fails in mysterious ways).  So
their first experience with Fossil winds up being a pretty big "WTF".

My $.02,
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] xkcd on git

[Eric Rubin-Smith]

I suspect Fossil folks will appreciate this :-)

http://xkcd.com/1597/

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] merge after cherrypick plus edit does not identify GCA as I would like

[Eric Rubin-Smith]

> The merge algorithm does *not* consider cherry-picks.  It looks for
> the most recent common ancestor without taking cherry-picks into
> account.

Another popular version control tool whose name I won't mention (hint: rhymes 
with "zit") behaves identically to fossil in this scenario. Is there some 
deeper reason for not using the cherry pick "arrow", or is it simply that yall 
haven't had a need to improve the behavior here yet so haven't bothered?

> On the other hand, the merge algorithm will usually recognize when the
> same edit is applied twice, and do the right thing.
> -- 
> D. Richard Hipp
> d...@sqlite.org
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] merge after cherrypick plus edit does not identify GCA as I would like

[Eric Rubin-Smith]

> Merge is done by a classic 3-way diff.  It looks at all the changes
> that occurred on the path from A to B and applies those same changes
> to C.  (A in this case would be the most recent common ancestor of B
> and C).
>
> How would cherry-picks factor into this?
>

Sorry, maybe I'm confused.  I'm not at the moment talking about the 3-way
diff algorithm, but the step before that in which the GCA (your 'A') is
determined.  You said that the GCA calculation does not consider
cherry-pick merges.  I'm wondering whether there is a deep reason for that,
or if it's just because no one has yet had a need for it.

Perhaps it is because if cherry-picks are considered, then you'd have to
determine a GCA for every file rather than for the check-ins as a whole,
and you believe this will be too slow?  Or maybe there is some deeper
logical problem that I haven't considered.

In my initial example, if the cherry-picks were considered then the 'beta'
leaf would be the GCA for my test file, so the merge would be a trivial one
(simply take the current 'trunk' leaf as the new file data).

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] merge after cherrypick plus edit does not identify GCA as I would like

[Eric Rubin-Smith]

See the transcript below for gory details.  The summary is:

1. create a new file on trunk and check it in.
2. edit the file and check in on a branch (let's call it "beta")
3. trunk decides it wants that particular change set from step (2), so
cherrypick it (assume in this example that other stuff is happening on the
beta branch that we don't want in trunk at the moment, so a normal merge is
not appropriate).
4. edit the same file on trunk and check it in
5. the "beta" branch now wants to merge the latest from trunk to continue
work

==> It's noted as a merge conflict

This is because the GCA calculation does not seem to incorporate the
cherrypick info (at least in this case).

Perhaps there is some deeper reason for this that I'm unaware of, but for
this case the behavior is suboptimal.

Transcript illustrating the above:

eas@little:~$ fossil version
This is fossil version 1.33 [9c65b5432e] 2015-05-23 11:11:31 UTC
eas@little:~$ mkdir /tmp/fossil
eas@little:~$ cd !$
cd /tmp/fossil
eas@little:/tmp/fossil$ fossil new test.db
project-id: c3037e9c81eb4c3279dfc24f07579bfbe604ddee
server-id:  ba2bb96bf830fa680389b425fa09c5dcfd5370c2
admin-user: eas (initial password is "dc73fc")
eas@little:/tmp/fossil$ mkdir sandbox
eas@little:/tmp/fossil$ cd !$
cd sandbox
eas@little:/tmp/fossil/sandbox$ fossil open /tmp/fossil/test.db
project-name: 
repository:   /tmp/fossil/test.db
local-root:   /tmp/fossil/sandbox/
config-db:/home/eas/.fossil
project-code: c3037e9c81eb4c3279dfc24f07579bfbe604ddee
checkout: 6bb0b6577411bd798631d137bf5b2d7d8fc3ac12 2015-09-14 15:28:16
UTC
tags: trunk
comment:  initial empty check-in (user: eas)
check-ins:1
eas@little:/tmp/fossil/sandbox$ echo 1.0 > VERSION
eas@little:/tmp/fossil/sandbox$ fossil add VERSION
ADDED  VERSION
eas@little:/tmp/fossil/sandbox$ fossil commit -m "Add version file"
New_Version: b6f302b927b0289feae9831c80f8b066f6e87d70
eas@little:/tmp/fossil/sandbox$ echo 1.1 > h^CRSION
eas@little:/tmp/fossil/sandbox$ set -o vi
eas@little:/tmp/fossil/sandbox$ echo 1.1b1 > VERSION
eas@little:/tmp/fossil/sandbox$ fossil commit --branch beta -m "Start beta
branch."
New_Version: ac64ec791f7be8601848e4c50a87dc54262cc659
eas@little:/tmp/fossil/sandbox$ fossil update trunk
UPDATE VERSION
---
updated-to:   b6f302b927b0289feae9831c80f8b066f6e87d70 2015-09-14 15:28:52
UTC
tags: trunk
comment:  Add version file (user: eas)
changes:  1 file modified.
 "fossil undo" is available to undo changes to the working checkout.
eas@little:/tmp/fossil/sandbox$ fossil merge --cherrypick
ac64ec791f7be8601848e4c50a87dc54262cc659
UPDATE VERSION
 "fossil undo" is available to undo changes to the working checkout.

eas@little:/tmp/fossil/sandbox$
eas@little:/tmp/fossil/sandbox$ fossil commit -m "Cherrypick version number
change."
New_Version: 8a4693e6ce2faa5cf3cd1e5a839b33ba7c590d02
eas@little:/tmp/fossil/sandbox$ echo 1.1b1.01 > VERSION
eas@little:/tmp/fossil/sandbox$ fossil commit -m "More work on the trunk."
New_Version: cee15c31915298ecce84eb0b5aa9b7520e3c8b61
eas@little:/tmp/fossil/sandbox$ fossil update beta
UPDATE VERSION
---
updated-to:   ac64ec791f7be8601848e4c50a87dc54262cc659 2015-09-14 15:29:19
UTC
tags: beta
comment:  Start beta branch. (user: eas)
changes:  1 file modified.
 "fossil undo" is available to undo changes to the working checkout.
eas@little:/tmp/fossil/sandbox$ fossil merge trunk
MERGE VERSION
* 1 merge conflicts in VERSION
WARNING: 1 merge conflicts
 "fossil undo" is available to undo changes to the working checkout.
eas@little:/tmp/fossil/sandbox$ cat VERSION
<<< BEGIN MERGE CONFLICT: local copy shown first <<<
1.1b1
=== COMMON ANCESTOR content follows 
1.0
=== MERGED IN content follows ==
1.1b1.01
>>> END MERGE CONFLICT >
eas@little:/tmp/fossil/sandbox$ exit
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] please compile official fossil builds with https support

[Eric Rubin-Smith]

On Wed, Jun 10, 2015 at 3:46 PM, Richard Hipp d...@sqlite.org wrote:

 On 6/10/15, Eric Rubin-Smith eas@gmail.com wrote:
  On Wed, Jun 10, 2015 at 3:30 PM, Richard Hipp d...@sqlite.org wrote:
 
  On 6/10/15, Eric Rubin-Smith eas@gmail.com wrote:
  
   If you are worried that some people don't want the bloat of openssl in
   their base fossil, perhaps provide both options on the site?
  
 
  It's not a question of bloat, its a question of whether or not we
  require the user to have previously done apt-get install openssl
  libssl (or whatever other magic incantation is required to get the
  right shared libraries running).
 
 
  I guess I had assumed you could bake it in statically.  Is that a
  non-option for yall?
 

 Can you send me the apt-get command for installing the necessary
 static libraries?


I believe you should be able to say:

# apt-get install libssl-dev

yielding /usr/lib/x86_64-linux-gnu/libssl.a and libcrypto.a on my arch.

This came up before.  Seems like somebody said that there are issues
 with openssl that prevent it from being statically linked.  But I
 don't recall the details.


I've been building my own openssl and statically linking it into programs
running on little things like home routers for around a year now.  I assume
you don't want to fiddle with building your own openssl, but I'm happy to
share my makefile magic if so.

I also don't know whether a static build has any legal/licensing
implications on fossil that you'd find undesirable.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] please compile official fossil builds with https support

[Eric Rubin-Smith]

$ fossil pull
Pull from https://eas@.../
HTTPS: Fossil has been compiled without SSL support
Pull done, sent: 0  received: 0  ip:
$

This burdens adoption, since now I have to build my own fossil and
distribute that to people on my team internally, rather than just pointing
them at the web site.

If you are worried that some people don't want the bloat of openssl in
their base fossil, perhaps provide both options on the site?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] please compile official fossil builds with https support

[Eric Rubin-Smith]

On Wed, Jun 10, 2015 at 8:50 PM, Eric Rubin-Smith eas@gmail.com wrote:



 Eric:  Can you discover what apt-get is needed in order to statically
 link libssl using -m32?


 Perhaps this?

 # apt-get install libssl-dev:i386

 Warning: I just got that command line from google and verified apt-get
 accepted it -- didn't actually try to link against the libs in that package.


I poked at this some more and it seems to work:

[little:fossil-src-1.33] $ make distclean
rm -rf bld/* fossil
rm -f autoconfig.h config.log Makefile
[little:fossil-src-1.33] $ export CFLAGS=-m32  LDFLAGS=-m32
[little:fossil-src-1.33] $ ./configure --with-openssl
Host System...x86_64-unknown-linux-gnu
Build System...x86_64-unknown-linux-gnu
C compiler... cc -m32
C++ compiler... c++ -m32
Build C compiler...cc
Checking for stdlib.h...ok
Checking for uint32_t...ok
Checking for uint16_t...ok
Checking for int16_t...ok
Checking for uint8_t...ok
Checking for pread...ok
Checking for tclsh...ok
Checking for ssl in 1...ok
HTTPS support enabled
Checking for zlib.h...ok
Checking libs for inflateEnd...-lz
Checking for readline/readline.h...not found
Checking for editline/readline.h...not found
Checking libs for gethostbyname...none needed
Checking libs for socket...none needed
Checking libs for iconv...none needed
Checking for utime...ok
Checking for usleep...ok
Checking for strchrnul...ok
Checking for getloadavg...ok
Checking for getpassphrase...not found
Checking libs for getpass...none needed
Checking libs for dlopen...-ldl
Checking libs for sin...-lm
Checking libs for fuse_mount...no
Created Makefile from Makefile.in
Created autoconfig.h
[little:fossil-src-1.33] $ vi Makefile
... make this edit:
--- Makefile.old2015-06-10 21:53:54.376496452 -0400
+++ Makefile2015-06-10 21:54:02.664496097 -0400
@@ -38,7 +38,7 @@
 #
 TCLSH = tclsh

-LIB =  -m32 -L1/lib -lssl -lcrypto -lz -ldl -lm
+LIB =  -m32 -L1/lib /usr/lib/i386-linux-gnu/libssl.a
/usr/lib/i386-linux-gnu/libcrypto.a -lz -ldl -lm
 TCCFLAGS = -I1/include  -m32 -DHAVE_AUTOCONFIG_H
-D_HAVE_SQLITE_CONFIG_H
 INSTALLDIR = $(DESTDIR)/usr/local/bin
 USE_SYSTEM_SQLITE = 0

[little:fossil-src-1.33] $ make -j10
...
[little:fossil-src-1.33] $ file ./fossil
./fossil: ELF 32-bit LSB  executable, Intel 80386, version 1 (SYSV),
dynamically linked (uses shared libs), for GNU/Linux 2.6.24,
BuildID[sha1]=6ed3d241209ed598c1ac326e169ff40239ad5f5f, not stripped
[little:fossil-src-1.33] $ ldd ./fossil
linux-gate.so.1 =  (0xf76f9000)
libz.so.1 = /lib/i386-linux-gnu/libz.so.1 (0xf76bd000)
libdl.so.2 = /lib/i386-linux-gnu/libdl.so.2 (0xf76b8000)
libm.so.6 = /lib/i386-linux-gnu/libm.so.6 (0xf7671000)
libc.so.6 = /lib/i386-linux-gnu/libc.so.6 (0xf74c3000)
/lib/ld-linux.so.2 (0xf76fa000)
[little:aterlo] $ ../Downloads/fossil-src-1.33/fossil pull
Pull from https://eas@redacted/
Round-trips: 1   Artifacts sent: 0  received: 0
Pull done, sent: 390  received: 553  ip: redacted
[little:aterlo] $

i.e. success on all fronts, I think.  You'd then just need to fix up your
Makefile to figure out dynamically whether to include the static or shared
libssl and libcrypto.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] please compile official fossil builds with https support

[Eric Rubin-Smith]

 Eric:  Can you discover what apt-get is needed in order to statically
 link libssl using -m32?


Perhaps this?

# apt-get install libssl-dev:i386

Warning: I just got that command line from google and verified apt-get
accepted it -- didn't actually try to link against the libs in that package.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] please compile official fossil builds with https support

[Eric Rubin-Smith]

On Wed, Jun 10, 2015 at 9:56 PM, Eric Rubin-Smith eas@gmail.com wrote:

 -LIB =  -m32 -L1/lib -lssl -lcrypto -lz -ldl -lm
 +LIB =  -m32 -L1/lib /usr/lib/i386-linux-gnu/libssl.a
 /usr/lib/i386-linux-gnu/libcrypto.a -lz -ldl -lm


I suppose this is sexier:

 LIB =   -m32 -L1/lib -Wl,-static -lssl -lcrypto -Wl,-Bdynamic -lz -ldl -lm

anyway, you get the drift.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] minor: broken hyperlink in fossil web site

[Eric Rubin-Smith]

From the download page for v1.33, Improved ability to customize the
timelime graph https://www.fossil-scm.org/customgraph.md is a broken
link.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] fossil http -h causes a segfault

[Eric Tsau]

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] How to fix parallel timeline

[Eric Rubin-Smith]

 Are you sure your students didn't shun something or try to use
 reconstruct?


What would happen if the student tried to push a repo that they had created
with 'fossil init' to the central clone?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] How to fix parallel timeline

[Eric Rubin-Smith]

... unless the students used raw SQL to hack there project-id to make
 it match the repository into which they were pushing.  But I'm
 thinking that is not what happened here.



Little anecdote.  When I was a student we were using CVS for our big
project.  One teammate couldn't understand why his check-ins were not
showing up in CVSTrac (thanks Richard:-)... investigation revealed that he
was trying to 'cp' his files onto the CVS repo tree rather than using 'cvs'
commands.

Not sure which is a better stress tester, a randomized fuzz program or an
inexperienced student with 5 volts of coffee onboard at 2am...

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Justification for two-step mv and rm

[Eric Rubin-Smith]

I fwiw have always found Fossil's mv and rm semantics odd.  The following
semantics are basically what I expected when I first started using Fossil,
but extended to preserve backward compatibility.  They basically do what
the user intended in all cases, do they not?

* fossil rm FILE:
* If FILE is unmanaged, then error out (without deleting FILE).
* if FILE is managed but does not exist, then apply current semantics
(i.e. stage for removal at commit time).
* If FILE is managed and exists, then do a filesystem remove of FILE
and then apply current semantics.[1]

* fossil mv A B:
(1) if A is unmanaged, then error out.
(2) if A and B are both managed, then apply current semantics (i.e.
error out).
(3) if A is managed, A does not exist, and B exists but is not managed,
then then apply current semantics.
(4) if A is managed and A exists:
* if B does not exist, do a filesystem rename of A to B and then
apply current semantics (3)
* if B exists, then error out.  (I am not a fan at all of the
current semantics, in which B is irrevocably deleted if initially unmanaged
after 'fossil mv A B; fossil revert', so am trying to smuggle a fix for
that in here:-)

Maybe I'm missing sub-cases, but the key thing is that I expect Fossil to
do the filesystem 'rm' or 'mv' for me if it's safe to do so.  And I think
Fossil can preserve backward compat even if extended in this way, right?

Eric

[1] Consider first checking for changes against current check-in and
erroring out if file is 'dirty' (recommending revert first), which I think
Git does.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Larger diff possible?

[Eric Rubin-Smith]

Jungle Boogie wrote:

 It absolutely is a side-by-side. I'm asking if its possible to
 maximize or display the diffs in a larger format with larger font.

Try holding down the Control key and then hitting the Equals key =
on your keyboard a few times.  In most browers these days, that zooms 
in the whole page while trying to keep things in relative proportion.

You can zoom back out by holding down Control and hitting the Minus
key -.

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] proposed patch: symlinks appear as regular files even when allow-symlinks is on

[Eric Rubin-Smith]

Richard Hipp wrote:

 sqlite3_snprintf() is guaranteed to be available.  Note, though, that the
 first two parameters are reversed.  :-\

Well, I only really want to copy up to 3 bytes, so we can keep it
simple, stupid and just not make a function call.  The revised patch is
below.  

But in further testing, I noticed that there is a series of
fossil calls involving a sequence of on/off settings to allow-symlinks
intermixed with 'fossil update', 'rm', etc that would result in fossil 
starting to return errors on 'update'.  I'm not convinced yet that it 
is a regression, but the below should be treated with caution until I've 
had a chance to look at it further.

I probably won't get back to that for at least 10 days, though.  Someone 
else can feel free to pick it up from here if they would like.

To Stephan's earlier point about just looking specifically for on and
off: I would prefer not to do that, since it duplicates logic from 
is_truth().  (Maybe the fossil devs will want to allow y in the
future or something, and I wouldn't want a little gotcha lurking in the
present patch when that happens.  And in any case just looking for yes
would skip the present valid values of 1, on, and true.)

[miami:src] $ fossil diff
Index: src/vfile.c
==
--- src/vfile.c
+++ src/vfile.c
@@ -268,21 +268,29 @@
   int promptFlag /* Prompt user to confirm overwrites */
 ){
   Stmt q;
   Blob content;
   int nRepos = strlen(g.zLocalRoot);
+  char bSawSymlinkSetting = 0;
 
+  /* In order to properly write out symlinks rather than regular files,
+  ** we must first observe the .fossil-settings/allow-symlinks file if it
+  ** exists.  If it does, we want to make sure we see it prior to any
+  ** symlink files.  This is why we sort ascending by 'islink'.
+  */
   if( vid0  id==0 ){
 db_prepare(q, SELECT id, %Q || pathname, mrid, isexe, islink
  FROM vfile
-WHERE vid=%d AND mrid0,
+WHERE vid=%d AND mrid0
+ORDER BY islink ASC,
g.zLocalRoot, vid);
   }else{
 assert( vid==0  id0 );
 db_prepare(q, SELECT id, %Q || pathname, mrid, isexe, islink
  FROM vfile
-WHERE id=%d AND mrid0,
+WHERE id=%d AND mrid0
+ORDER BY islink ASC,
g.zLocalRoot, id);
   }
   while( db_step(q)==SQLITE_ROW ){
 int id, rid, isExe, isLink;
 const char *zName;
@@ -291,10 +299,21 @@
 zName = db_column_text(q, 1);
 rid = db_column_int(q, 2);
 isExe = db_column_int(q, 3);
 isLink = db_column_int(q, 4);
 content_get(rid, content);
+if( !bSawSymlinkSetting 
+zName[nRepos]=='.'  zName[nRepos+1]=='f'  zName[nRepos+2]=='o' 
+strcmp(zName[nRepos], .fossil-settings/allow-symlinks)==0
+){
+  char zValue[4] = {0,0,0,0};
+  int i=0;
+  bSawSymlinkSetting = 1;
+  const char* zCont = blob_str(content);
+  while( i3  zCont[i]!=0  zCont[i]!='\n' ){zValue[i]=zCont[i]; ++i;}
+  g.allowSymlinks = is_truth(zValue);
+}
 if( file_is_the_same(content, zName) ){
   blob_reset(content);
   if( file_wd_setexe(zName, isExe) ){
 db_multi_exec(UPDATE vfile SET mtime=%lld WHERE id=%d,
   file_wd_mtime(zName), id);


--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] proposed patch: symlinks appear as regular files even when allow-symlinks is on

[Eric Rubin-Smith]

Another iteration.  We need to add an extra byte in case the true
value is the string true.  I've re-expressed the loop as a 'for' loop
while I was at it.

Sorry for the spam.  Again, please treat the below with caution until 
I have (or someone else has) a chance to exercise it better.

Index: src/vfile.c
==
--- src/vfile.c
+++ src/vfile.c
@@ -268,21 +268,29 @@
   int promptFlag /* Prompt user to confirm overwrites */
 ){
   Stmt q;
   Blob content;
   int nRepos = strlen(g.zLocalRoot);
+  char bSawSymlinkSetting = 0;
 
+  /* In order to properly write out symlinks rather than regular files,
+  ** we must first observe the .fossil-settings/allow-symlinks file if it
+  ** exists.  If it does, we want to make sure we see it prior to any
+  ** symlink files.  This is why we sort ascending by 'islink'.
+  */
   if( vid0  id==0 ){
 db_prepare(q, SELECT id, %Q || pathname, mrid, isexe, islink
  FROM vfile
-WHERE vid=%d AND mrid0,
+WHERE vid=%d AND mrid0
+ORDER BY islink ASC,
g.zLocalRoot, vid);
   }else{
 assert( vid==0  id0 );
 db_prepare(q, SELECT id, %Q || pathname, mrid, isexe, islink
  FROM vfile
-WHERE id=%d AND mrid0,
+WHERE id=%d AND mrid0
+ORDER BY islink ASC,
g.zLocalRoot, id);
   }
   while( db_step(q)==SQLITE_ROW ){
 int id, rid, isExe, isLink;
 const char *zName;
@@ -291,10 +299,21 @@
 zName = db_column_text(q, 1);
 rid = db_column_int(q, 2);
 isExe = db_column_int(q, 3);
 isLink = db_column_int(q, 4);
 content_get(rid, content);
+if( !bSawSymlinkSetting 
+zName[nRepos]=='.'  zName[nRepos+1]=='f'  zName[nRepos+2]=='o' 
+strcmp(zName[nRepos], .fossil-settings/allow-symlinks)==0
+){
+  char zValue[5] = {0,0,0,0,0};
+  int i;
+  bSawSymlinkSetting = 1;
+  const char* zCont = blob_str(content);
+  for(i=0; i4  zCont[i]!=0  zCont[i]!='\n';++i ){ zValue[i]=zCont[i]; 
}
+  g.allowSymlinks = is_truth(zValue);
+}
 if( file_is_the_same(content, zName) ){
   blob_reset(content);
   if( file_wd_setexe(zName, isExe) ){
 db_multi_exec(UPDATE vfile SET mtime=%lld WHERE id=%d,
   file_wd_mtime(zName), id);


--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] proposed patch: symlinks appear as regular files even when allow-symlinks is on

[Eric Rubin-Smith]

:  Test (user: eas)
checkins: 2
[miami:sandbox2] $ ls -l
total 4
-rw-r--r--. 1 eas eas 0 Oct 30 20:25 a
-rw-r--r--. 1 eas eas 1 Oct 30 20:25 b
[miami:sandbox2] $ cat b
a[miami:sandbox2] $ rm b
rm: remove regular file `b'? y
[miami:sandbox2] $ fossil update
UPDATE b
---
updated-to:   3aee795224e735d8b6c08f3bcd10c8acf98b15c0 2014-10-31 00:25:31 UTC
leaf: open
tags: trunk
comment:  Test (user: eas)
changes:  1 file modified.
 fossil undo is available to undo changes to the working checkout.
[miami:sandbox2] $ ls -l
total 0
-rw-r--r--. 1 eas eas 0 Oct 30 20:25 a
lrwxrwxrwx. 1 eas eas 1 Oct 30 20:26 b - a

TRANSCRIPT - AFTER

[miami:fstest] $ mkdir sandbox3
[miami:fstest] $ cd !$
cd sandbox3
[miami:sandbox3] $ ~/fossil-scm/bld/fossil open ../test
.fossil-settings/allow-symlinks
a
b
project-name: unnamed
repository:   /home/eas/tmp/fstest/sandbox3/../test
local-root:   /home/eas/tmp/fstest/sandbox3/
config-db:/home/eas/.fossil
project-code: 981a44330154ce354fc1407af51b2ccaaacf1440
checkout: 3aee795224e735d8b6c08f3bcd10c8acf98b15c0 2014-10-31 00:25:31 UTC
parent:   c8b4245419520d30cc360295cfa22d28c814e586 2014-10-31 00:24:42 UTC
leaf: open
tags: trunk
comment:  Test (user: eas)
checkins: 2
[miami:sandbox3] $ ls -l
total 0
-rw-r--r--. 1 eas eas 0 Oct 30 21:26 a
lrwxrwxrwx. 1 eas eas 1 Oct 30 21:26 b - a

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Stephan Beal wrote: 

 FYI: i committed one on top of that.  The advantage is that it's 
 centralized, the disadvantage is that it hashes every manifest before 
 parsing (to get the UUID, since parsing modifies it).  Might be considered 
 too expensive, considering how rare broken manifests are.  

Follow-up.  The investigation led to the following patch: 

http://fossil-scm.org/index.html/vinfo/bf3db3d16e15ee98f5c72ebdc4ff2f2ed1609ff5?sbs=0
 

which, if I understand correctly, only makes corrections to the error 
reporting, and does not change any of the core program logic.  After 
that change, Fossil now only reports errors out of the Tcl core that I 
am using to implement one of my Transfer hooks.  

My current suspicion is that something about the PKI stuff on my 
server box has changed, which causes Tcl to have trouble connecting to a 
remote HTTPS server from my TH1 hooks.  I.e. Fossil itself is no longer 
implicated as a contributing cause.

I'll keep pulling on that thread in private, since it now seems it 
is specific to my particular deployment.  Many thanks to Richard and 
Stephen for looking into it and making the improvement to the error
reporting.

Eric 

-- 
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Stephan Beal wrote: 

 We'd be interested in hearing back if you discover how an error on your 
 end (if indeed it is) is confusing fossil into trying to read non-manifest 
 files as manifests.  

Actually, your question sort of confuses me.  From my (very tenuous) 
understanding of the code, it tries to read *every* incoming file during 
a sync as a manifest.  It only takes further action if the file is 
successfully parsed as a manifest.  

This sort of thing leads to other kinds of confusion.  For example, 
I took a snapshot of a 3rdparty repository.  That 3rdparty repository 
itself includes a snapshot of the SQLite source tree, which had a 
manifest file in it.  When I committed my snapshot, I suddenly saw an 
SQLite commit, complete with 'drh' as the committer, show up in own my 
project timeline!  It corresponded to the data that was found in the 
manifest file.  This supports my read of the code, I think.  

So the parse error is generated regardless -- the only question is
whether it gets reported back to the pushing remote 'client'.  That I
guess is stimulated by my failing TH1 script.  So Richard's change
caused the true underlying cause of the error to propagate back to the
client, not a 'harmless' parse error on a file that is not a manifest.

But that is just my vague understanding: obviously Richard can comment 
more intelligently on that than I can.

-- 
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

I haven't changed much recently about my repository topology.  One central
master repo that I push to from my dev repo.  No one else is doing any
pushing.  The master repo has hooks set up for publishing my commits etc to
a remote HTTP REST API.

The only change I made recently is to open up my same local repo a second
time into a new sandbox.  That worked fine for many commits.  But today I
suddenly see this:

$ fossil version
This is fossil version 1.29 [3e5ebe2b90] 2014-06-12 17:25:56 UTC
$ uname -a
Linux miami 3.6.11-4.fc16.x86_64 #1 SMP Tue Jan 8 20:57:42 UTC 2013 x86_64
x86_64 x86_64 GNU/Linux

$ fossil commit --branch test -m Test check-in.
New_Version: 81ba47a887f005b11fe14aedb9c8c25f0bc23ffe
$ fossil push
Push to https://eas@snip:10444/
Round-trips: 1   Artifacts sent: 2  received: 0
Error: push script failed: syntax error in manifest
Round-trips: 1   Artifacts sent: 2  received: 0
Push finished with 817830 bytes sent, 330 bytes received
$ fossil push
Push to https://eas@snip:10444/
Round-trips: 1   Artifacts sent: 0  received: 0
Push finished with 817459 bytes sent, 286 bytes received

The only interesting thing I did recently was (a) make a commit from my old
sandbox on a dev branch; (b) switch to my new sandbox, make two new
unrelated commits to trunk; (c) cherrypick the commit I made in (a) to
trunk.  That's when the issue seems to have started.

The data appears to have landed on the server properly.  The error now
happens consistently every time I try a check-in + push sequence (twice in
a row, anyway).  My Transfer hooks on the server don't seem to have fired.

I tried this, which doesn't give much useful info:

$ fossil test-parse-manifest ./manifest
$ echo $?
0
$

I assume it's the remove fossil complaining, though -- not the local one.

How can I gather more info?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Stephan Beal wrote:

 According my debugger, that uuid is 41 bytes long, including a
 carriage-return character. How on earth that happend is a mystery to me, as
 fossil does not use \r characters anywhere unless it's prescribed by a
 standard (e.g. HTTP headers).
 
 The C-card also contains a trailing \r character, as does the D-card. i
 don't know how you've done it, but your system is writing the manifest with
 Windows newlines in it.

How dare you suggest it!  :)  I don't think that's what's happening.  
Maybe that was your mail client when you saved my attachment, or mine
when I initially sent it (though I doubt it, since I am using a
Linux + mutt).  

Here's what I see in 'hexedit':

   43 20 43 68  65 63 6B 2D  69 6E 5C 73  66 6F 72 5C  C Check-in\sfor\
0010   73 6F 66 66  69 63 69 61  6C 5C 73 62  75 69 6C 64  sofficial\sbuild
0020   2E 0A 44 20  32 30 31 34  2D 31 30 2D  31 35 54 31  ..D 2014-10-15T1
0030   36 3A 33 38  3A 35 37 2E  39 34 37 0A  46 20 2E 66  6:38:57.947.F .f
0040   6F 73 73 69  6C 2D 73 65  74 74 69 6E  67 73 2F 61  ossil-settings/a
0050   6C 6C 6F 77  2D 73 79 6D  6C 69 6E 6B  73 20 61 30  llow-symlinks a0
0060   39 61 39 36  38 62 66 30  35 61 35 30  30 35 38 66  9a968bf05a50058f
0070   33 61 64 35  30 31 33 32  37 33 30 62  37 31 39 62  3ad50132730b719b
0080   63 33 39 65  37 36 0A 46  20 33 72 64  70 61 72 74  c39e76.F 3rdpart
0090   79 2F 4D 61  6B 65 66 69  6C 65 20 34  31 61 32 65  y/Makefile 41a2e

No 0Ds in there. 

Let me re-send you the manifest as a 'gz' file.  Hopefully no code
between me and you will be too clever for its own good this time.

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Richard Hipp wrote:

 Are you sure that is the manifest that is causing the problem?  Perhaps we
 should enhance the error message to include the UUID of the problem
 manifest as part of the error message?

In case you find it useful for your debugging, I have continued plodding
along with my normal work today, and continue seeing the error:

$ fossil commit 
vim ../../../ci-comment-0CB216CD5263.txt
New_Version: efc2f0ca8089b41f9aa144480d530737fa38e755
$ fossil push
Push to https://eas@snip:10444/
Round-trips: 1   Artifacts sent: 4  received: 0
Error: push script failed: syntax error in manifest
Round-trips: 1   Artifacts sent: 4  received: 0
Push finished with 819175 bytes sent, 330 bytes received

My commits continue showing up in the remote repo, despite the error.
Might have something more to do with the Transfer hooks TH1 stuff
('push script failed') than the sync logic itself?  Let me know if 
you would like to see that TH1 script.

Eric

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Stephan Beal wrote:

 Working on a patch now.

Sorry, I'm realizing that the Fossil version I mentioned (1.29) is
tainted with my own private changes to the C code (to get my TH1 hooks
working).  I submitted them to this list, and Joe Mistachkin accepted 
them -- but with significant changes.  It may be that this is my fault, 
not Fossil's.  I would like to eliminate that possibility myself, for
fear that I am wasting your time on an issue that does not affect anyone
but me.

What is a good fossil trunk version for me to re-test with?

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Richard Hipp wrote:

 Can you try compiling the tip of the better-error-msgs branch and put
 that on your server, then let us know what the new error message is?

Our emails crossed paths.  I'll do as you suggest and let you know what
happens.

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Richard Hipp wrote:

 Can you try compiling the tip of the better-error-msgs branch and put
 that on your server, then let us know what the new error message is?

Here we go:

$ fossil push
Push to https://eas@snip:10444/
Round-trips: 1   Artifacts sent: 2  received: 0
Error: push script failed: syntax error in manifest 
[449bc674c3fc9c036db0d4dc71281b7cb900fe7d]
Round-trips: 1   Artifacts sent: 2  received: 0
Push finished with 818173 bytes sent, 385 bytes received
$ 

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Eric Rubin-Smith wrote:

 $ fossil push
 Push to https://eas@snip:10444/
 Round-trips: 1   Artifacts sent: 2  received: 0
 Error: push script failed: syntax error in manifest 
 [449bc674c3fc9c036db0d4dc71281b7cb900fe7d]
 Round-trips: 1   Artifacts sent: 2  received: 0
 Push finished with 818173 bytes sent, 385 bytes received
 $ 

Looks like that hex value is the artifact ID for the file I edited for
the test check-in.  The relevant bytes from the manifest seem 
fine, to my eye.  'F', space, filename, space, 40 hex digits, newline.

The fossil server is sitting behind stunnel.  I'll see if I can take a 
quick capture of the unencrypted traffic -- maybe it'll shed light on
something.

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] push fails with complaint about manifest syntax error

[Eric Rubin-Smith]

Stephan Beal wrote:

 You mention custom changes with regards to push scripts, which make me
 curious about that error message: i can find push script failed nowhere
 in fossil. Where is it coming from (if you know)?

The strings are escaped :)

eric@dev:~/Fossil-5ff4e33617/src$ grep -r 'push\\s' .
./xfer.c:@ error not\sauthorized\sto\spush\sconfiguration
./xfer.c:@ error push\sscript\sfailed:\s%F(g.zErrMsg)

I compiled your better-error-messages Fossil, and that's what
I'm running now.  So the currently-running code is no longer tainted by
my private edits, though of course if I managed to screw up my repo with
my private changes then we still might be seeing the result of that.

Not that I think that is particularly likely, but I'm a fossil-hacking
noob so what do I know.

--
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] minor bug on export to git

[Eric Rubin-Smith]

When exporting to git, the check-in comments that are exported are the
original comments.  If you had subsequently edited the comments, then those
edits are not retained during the export.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Getting configure to find openssl on FreeBSD

[Eric Rubin-Smith]

org.fossil-scm.fossil-us...@io7m.com wrote:

 'Lo.
 
 The fossil build scripts seem to be unable to find openssl on FreeBSD
 9.2. It has a choice of the version included with the base system
 (in /usr) or the version available from FreeBSD ports (/usr/local),
 but it can't seem to find either of them.
 
 Is there any way to get it to give more information about why it's
 failing to find them?

I skimmed through the configure code.  Looks like you need to have 
the subdir and file 'openssl/ssl.h' beneath the dir you specify, and 
the subdir and files 'lib/libssl.so' and 'lib/libcrypto.so' beneath the 
dir you specify.  Furthermore, one of those libraries has to provide
the symbol SSL_new.  

My (very quick and possibly wrong) read of the code is that those are 
exactly the pass criteria for deciding that SSL is supported on the 
build box using the dir your specify.

Did you confirm that all those are true in your case?

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] fossil - JIRA hooks

[Eric Rubin-Smith]

Does anyone have any experience with hooking up fossil to JIRA?

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil - JIRA hooks

[Eric Rubin-Smith]

Ron W wrote: 

  Does anyone have any experience with hooking up fossil to JIRA?  
  
 
 Do you mean automatic ticket sync between Fossil and an external issue 
 tracking system?  Or using an external issue tracking system instead of 
 Fossil's?  

I guess I meant the former, though the latter is also interesting if it 
can be made as nice as Fossil's internal tracker (in the sense that it 
is easy to drill from the issue down to the check-ins addressing it).  

 My generic experience with the latter has been to either use pre and 
 post commit hooks, or create a wrapper around the VCS, to interact with 
 the issue tracking system.  
 
 FWIW, I originally found Fossil while I was looking for an easy to use, 
 easy to setup issue tracking system that would not require involvement of 
 the IT people.  

I'm in the middle of a debate about whether to use Fossil or git+JIRA.  
If I can hook Fossil up to JIRA then any points about the superiority of 
JIRA will experience significant erosion.  

As a separate question along those lines, have people had other sorts of 
trouble (IT, data sanity or otherwise) with git+JIRA?  

 That said, there have been times when it would be nice if I could setup 
 Fossil to receive new tickets from an external system and send updates to 
 those tickets back.  I just haven't had the time to truly look into this.  
 Off hand, it looks to me like this could reasonably be accomplished with 
 some combination of the JSON API and the TCL integration.  

I guess there are complexities around reconciliation with those 
simpler approaches.  If the external system and Fossil are incommunicado 
for some period of time (during which they are diverging), then some 
process will need to have some way of determining the deltas on both 
sides and doing the sync.  The TCL hooks will not suffice for that, 
unless you extend the scope of what you mean to encompass those 
full-sync semantics.  (This is analogous to the thinking involved in 
implementing 'fossil sync'.)  

This in turn will have much to do with the vagaries how to get data into 
and out from the particular external bug tracker -- JIRA in my case.  
The tool appears to support a relatively featureful HTTP/REST/JSON 
API.  I guess one of my questions is whether anyone has tried to bite 
off hooking up a fossil-jira feed using that API yet, whether they 
experienced any robustness problems or other gotchas in the JIRA API, 
etc.  

-- 
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil - JIRA hooks

[Eric Rubin-Smith]

Stephan Beal wrote: 

 That said: ticket integration with the JSON API turned out to be a lot 
 more work than anticipated (and thus it is quite castrated) because of 
 the customizability of the ticket subsystem and its close relationship to 
 the scripting world.  Thus, i suspect, the TCL/TH1 support might be (out 
 of the box) more use in that area.  (And i see no inherent reason why we 
 couldn't generate JSON out of the those worlds, it just has never happened 
 so far.)  

Okay interesting -- thanks for the response.  Did you capture any
more specific notes or code anywhere that you can point me to?

-- 
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil - JIRA hooks

[Eric Rubin-Smith]

 As a separate question along those lines, have people had other sorts of
 trouble (IT, data sanity or otherwise) with git+JIRA?


Sorry to self-reply.  I meant for the sense of this question to be
constrained
specifically to the git-to-JIRA hooks or JIRA itself -- not to git as a
stand-alone
entity.  Fossil devs'/users' criticisms of git in itself are pretty well
documented
on this forum and elsewhere, and my intention was not to rehash that
thinking
on this thread.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil - JIRA hooks

[Eric Rubin-Smith]

Stephan Beal wrote: 

 In principal, plain old transport is not the problem.  The problem (as i 
 recall it, though it's been a while and i've got the memory of a goldfish) 
 was that i could not define a concrete structure for JSON/Ticket I/O 
 because tickets are customizable.  Suggestions are of course welcomed, and 
 if there's a concrete case you or Eric have in mind, i can probably say 
 whether it's currently workable or estimate what it would take to make it 
 so.  

If we assume for a moment that the user is willing to customize the 
Fossil ticket system to exactly match whatever is needed to integrate 
with JIRA, does the problem become easy overall?

-- 
Eric A. Rubin-Smith

Aterlo Networks, Inc.
http://aterlo.com

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] late-night Fossil hacks for competition with Git: arranging an active feed of commits and tickets to a remote HTTP API

[Eric Rubin-Smith]

 delta, then send
 ** nothing and return zero.
 **
@@ -853,24 +880,26 @@

 /*
 ** Run the specified TH1 script, if any, and returns 1 on error.
 */
 int xfer_run_script(const char *zScript, const char *zUuid){
   int rc;
   if( !zScript ) return TH_OK;
   Th_FossilInit(TH_INIT_DEFAULT);
   if( zUuid ){
 rc = Th_SetVar(g.interp, uuid, -1, zUuid, -1);
-if( rc!=TH_OK ){
-  fossil_error(1, %s, Th_GetResult(g.interp, 0));
-  return rc;
-}
+  }else{
+rc = Th_SetVar(g.interp, uuid, -1, , -1);
+  }
+  if( rc!=TH_OK ){
+fossil_error(1, %s, Th_GetResult(g.interp, 0));
+return rc;
   }
   rc = Th_Eval(g.interp, 0, zScript, -1);
   if( rc!=TH_OK ){
 fossil_error(1, %s, Th_GetResult(g.interp, 0));
   }
   return rc;
 }

 /*
 ** Runs the pre-transfer TH1 script, if any, and returns its return code.
@@ -911,20 +940,22 @@
   int isPush = 0;
   int nErr = 0;
   Xfer xfer;
   int deltaFlag = 0;
   int isClone = 0;
   int nGimme = 0;
   int size;
   int recvConfig = 0;
   char *zNow;
   int rc;
+  char* zUuidList = 0;
+  int nUuidList = 0;

   if( fossil_strcmp(PD(REQUEST_METHOD,POST),POST) ){
  fossil_redirect_home();
   }
   g.zLogin = anonymous;
   login_set_anon_nobody_capabilities();
   login_check_credentials();
   memset(xfer, 0, sizeof(xfer));
   blobarray_zero(xfer.aToken, count(xfer.aToken));
   cgi_set_content_type(g.zContentType);
@@ -963,42 +994,42 @@
 **
 ** Accept a file from the client.
 */
 if( blob_eq(xfer.aToken[0], file) ){
   if( !isPush ){
 cgi_reset_content();
 @ error not\sauthorized\sto\swrite
 nErr++;
 break;
   }
-  xfer_accept_file(xfer, 0);
+  xfer_accept_file(xfer, 0, zUuidList, nUuidList);
   if( blob_size(xfer.err) ){
 cgi_reset_content();
 @ error %T(blob_str(xfer.err))
 nErr++;
 break;
   }
 }else

 /*   cfile UUID USIZE CSIZE \n CONTENT
 **   cfile UUID DELTASRC USIZE CSIZE \n CONTENT
 **
 ** Accept a file from the client.
 */
 if( blob_eq(xfer.aToken[0], cfile) ){
   if( !isPush ){
 cgi_reset_content();
 @ error not\sauthorized\sto\swrite
 nErr++;
 break;
   }
-  xfer_accept_compressed_file(xfer);
+  xfer_accept_compressed_file(xfer, zUuidList, nUuidList);
   if( blob_size(xfer.err) ){
 cgi_reset_content();
 @ error %T(blob_str(xfer.err))
 nErr++;
 break;
   }
 }else

 /*   gimme UUID
 **
@@ -1269,29 +1300,32 @@
 */
 {
   cgi_reset_content();
   @ error bad\scommand:\s%F(blob_str(xfer.line))
 }
 blobarray_reset(xfer.aToken, xfer.nToken);
 blob_reset(xfer.line);
   }
   if( isPush ){
 if( rc==TH_OK ){
-  rc = xfer_run_script(xfer_push_code(), 0);
+  rc = xfer_run_script(xfer_push_code(), zUuidList);
   if( rc==TH_ERROR ){
 cgi_reset_content();
 @ error push\sscript\sfailed:\s%F(g.zErrMsg)
 nErr++;
   }
 }
 request_phantoms(xfer, 500);
   }
+  if( zUuidList ){
+Th_Free(g.interp, zUuidList);
+  }
   if( isClone  nGimme==0 ){
 /* The initial clone message from client to server contains no
 ** gimme cards. On that initial message, send the client an igot
 ** card for every artifact currently in the repository.  This will
 ** cause the client to create phantoms for all artifacts, which will
 ** in turn make sure that the entire repository is sent efficiently
 ** and expeditiously.
 */
 send_all(xfer);
 if( xfer.syncPrivate ) send_private(xfer);
@@ -1631,31 +1665,31 @@
   fflush(stdout);
 }
   }

   /*   file UUID SIZE \n CONTENT
   **   file UUID DELTASRC SIZE \n CONTENT
   **
   ** Receive a file transmitted from the server.
   */
   if( blob_eq(xfer.aToken[0],file) ){
-xfer_accept_file(xfer, (syncFlags  SYNC_CLONE)!=0);
+xfer_accept_file(xfer, (syncFlags  SYNC_CLONE)!=0, 0, 0);
 nArtifactRcvd++;
   }else

   /*   cfile UUID USIZE CSIZE \n CONTENT
   **   cfile UUID DELTASRC USIZE CSIZE \n CONTENT
   **
   ** Receive a compressed file transmitted from the server.
   */
   if( blob_eq(xfer.aToken[0],cfile) ){
-xfer_accept_compressed_file(xfer);
+xfer_accept_compressed_file(xfer, 0, 0);
 nArtifactRcvd++;
   }else

   /*   gimme UUID
   **
   ** Server is requesting a file.  If the file is a manifest, assume
   ** that the server will also want to know all of the content files
   ** associated with the manifest and send those too.
   */
   if( blob_eq(xfer.aToken[0], gimme)
===

--
Eric Rubin-Smith
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] late-night Fossil hacks for competition with Git: arranging an active feed of commits and tickets to a remote HTTP API

[Eric Rubin-Smith]

Ron W wrote: 

 This enhancement to Fossil will certainly make integration with other 
 tools easier, not just Slack 

I was thinking of an example: git.  Right now there's a bulk import 
from git, and a bulk export to git.  The present feature might be used 
along with git's hooks features to permit an automated, bidirectional 
incremental sync between git and fossil repositories, which might help 
ease some users' transitions from git to fossil (possibly encouraging
more fossil uptake).  

-- 
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] edits to ticket table not synchronized with 'fossil config pull all'

[Eric Rubin-Smith]

In one of my fossil repos, I edited the tickets table by adding a column.

I wanted to pull the new column definition into a repo clone, so I said

fossil config pull all

from the other repo, to no avail.  New reports, the New Ticket and Edit
Ticket pages and so on were properly synchronized, but the table definition
itself was not.

This leads to this error message in one of the reports I have defined that
uses the new column:

SQLITE_ERROR: no such column: releaseGate

Other commands that did not work:

fossil config pull ticket

fossil config pull ticket --overwrite

fossil pull

I have seen this every time I have tried to add a column to the tickets
table.  I normally work around it by copy-pasting the ticket definitions,
but that is probably not going to be acceptable for my other users.

Am I doing something wrong?

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] edits to ticket table not synchronized with 'fossil config pull all'

[Eric Rubin-Smith]

Andreas Kupries wrote:

 You have to run
 
   fossil rebuild
 
 on _all_ repositories with the new definition.
 
 This rebuilds the derived tables (TICKET, TICKETCHNG) from the actual
 tickets, and ensures that they actually have the new column you
 defined.

Works like a charm -- thank you sir!

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] symlinks

[Eric Rubin-Smith]

Any plan to support symlinks any time soon?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] symlinks

[Eric Rubin-Smith]

On Thu, Aug 28, 2014 at 10:55 AM, Stephan Beal sgb...@googlemail.com
wrote:


 On Thu, Aug 28, 2014 at 4:51 PM, Eric Rubin-Smith eas@gmail.com
 wrote:

 Any plan to support symlinks any time soon?


 ???

 [stephan@host:~]$ f help set | grep -C3 sym
access-log   If enabled, record successful and failed login attempts
 in the accesslog table.  Default: off

allow-symlinks   If enabled, don't follow symlinks, and instead treat
 (versionable)   them as symlinks on Unix. Has no effect on Windows
 (existing links in repository created on Unix become
 plain-text files with link destination path inside).
 Default: off



D'oh.  I had searched the forum + google and found threads in which the
devs described why there was no support, and then tested to see if there
was support by just checking in a symlink (which didn't work by default).
So I missed the existence of the feature.  Sorry for the noise! :-(

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] crash while running 'fossil pull' on Linux

[Eric Rubin-Smith]

Stephan Beal wrote:

 On Tue, Aug 19, 2014 at 7:31 PM, Warren Young war...@etr-usa.com wrote:
 
  On 8/18/2014 19:39, Eric Rubin-Smith wrote:
 
 
  warning: Can't read pathname for load map: Input/output error.
 
  That looks like a corrupted stack to me.
 
 
  Try running it under Valgrind.
 
 
 +1

I ran it under valgrind already and mentioned the results in the OP -- 
you probably missed that in my wall of text :-).  The upshot is that 
valgrind's imposed slow-down causes us not to reach the bad path.  So no 
help there.

If you follow my post through to the end, though, you will see that 
the call to 'xfer_accept_file' in xfer.c line 1641 is strongly
implicated by my gdb session.  If you go and look at that function,
you will see a call to blob_extract on line 139 that passes 'n', where
'n' was obtained above from the incoming data (xfer.c line 130).  It seems 
very likely that we are placing too much trust in this 'n' during 
'blob_extract'... though this is just my suspicion of course, not a 
certainty.

 and while i'm here: thank you for the tremendously detailed bug report.

no problem.  I've been on the receiving end of tickets like teh
software doesn't work 4 me can u plz fix it enough times to try to 
avoid inflicting similar pain. :-)

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] crash while running 'fossil pull' on Linux

[Eric Rubin-Smith]

I configured fossil to use openssl (for https) and built it for Linux
(kernel 3.11.0-12-generic, Ubuntu 13.10).  Fossil crashes during the 'pull'
portion of a 'fossil update' or just while running 'fossil pull'.  The pull
implies the transfer of a few large artifacts (~60MB range) as well as lots
of small ones.

The crash happens when the client speaks SSL to the fossil server via
stunnel, and also when the client speaks in the clear directly to the
fossil server.

I believe that the server is running out of memory during the transaction,
and the resulting server response causes the client to fail.

$ uname -a
Linux snip 3.11.0-12-generic #19-Ubuntu SMP Wed Oct 9 16:12:00 UTC 2013
i686 i686 i686 GNU/Linux

$ ~/fossil-src-20140612172556/fossil version
This is fossil version 1.29 [3e5ebe2b90] 2014-06-12 17:25:56 UTC

I am unfortunately not able to get much useful information out of the core
file, apart from the fact that we seem to have been in the openssl lib when
we crashed:

$ ~/fossil-src-20140612172556/fossil pull
Pull from https://e...@dev.packetup.net:10444/
Segmentation fault (core dumped) 0  received: 0

$ gdb ~/fossil-src-20140612172556/fossil ./core
GNU gdb (GDB) 7.6.1-ubuntu
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later http://gnu.org/licenses/gpl.html

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type show copying
and show warranty for details.
This GDB was configured as i686-linux-gnu.
For bug reporting instructions, please see:
http://www.gnu.org/software/gdb/bugs/...
Reading symbols from /home/eric/fossil-src-20140612172556/fossil...done.

warning: exec file is newer than core file.
[New LWP 7065]

warning: Can't read pathname for load map: Input/output error.
Core was generated by `/home/eric/fossil-src-20140612172556/fossil update'.
Program terminated with signal 11, Segmentation fault.
#0  0xb75bffb0 in ?? () from /lib/i386-linux-gnu/libcrypto.so.1.0.0
(gdb) where
#0  0xb75bffb0 in ?? () from /lib/i386-linux-gnu/libcrypto.so.1.0.0
#1  0xefcdab89 in ?? ()
#2  0x98badcfe in ?? ()
#3  0x10325476 in ?? ()
#4  0xc3d2e1f0 in ?? ()
#5  0x402a84b0 in ?? ()
#6  0x in ?? ()

This happens every time given my current clone pairs.

I was not able to reproduce it under valgrind, because the slow-down
imposed by valgrind caused the initial network transaction to time out.

I've reproduced it under strace, but I'm not sure how helpful that output
will be.

I then recompiled fossil as a static executable and re-ran my test.  Fossil
usually crashes still, but one time it did not crash.  Instead, it produced
the following output:

$ ../fossil-src-20140612172556/fossil pull
Pull from https://e...@dev.packetup.net:10444/
Round-trips: 1   Artifacts sent: 0  received: 0
server replies with HTML instead of fossil sync protocol:
file 010dfdc0db6ff3cf63e0da8f90681fef5e80ee56 5032
/*
 * Copyright (C) 1995, 1996, 1997, 1998, and 1999 WIDE Project.
{snip a bunch of file data}
file 25fc6f1d7ddb3d0c6f812a397752b83472776e56
8d0ef3a2aee8895877014a1b273cc4dfd20e5f34 272
AWP
1d:C {snipped my check-in comment}
D 2014-08-18T16:46:56.6919j2@22,q:nvram_read.c
83e2de86fe92201e7662fc9db14387f4d4d79a0fjd@9kq
,1I:8d0ef3a2aee8895877014a1b273cc4dfd20e5f34
U eas
Z a2e9f38c4965a46f1e4f925f9e9a8a07
3GbccP;file 267f9bd5f5b347d26ede6b0c20c332d8af1b3bc0 5212

Round-trips: 2   Artifacts sent: 0  received: 51
server replies with HTML instead of fossil sync protocol:
p class=generalErrorout of memory/p
Round-trips: 2   Artifacts sent: 0  received: 51
Pull finished with 4250 bytes sent, 99232 bytes received
$

So I guess a hypothesis is that the server is running out of memory and is
barfing back some response that the client cannot handle, and the client is
crashing.

I have taken a wire capture of the cleartext client-server interaction
and can share that with the devs privately if they would like.  There is
not a lot of data transferred in this scenario -- only 12 packets.  The
server works for 80 seconds, sends back a small response, and the
connection is terminated.

I ran fossil and stepped it through the debugger starting from the
blob_uncompress.  Here's the contents of the uncompressed blob:

(gdb) where
#0  blob_uncompress (pIn=pIn@entry=0xb434, pOut=pOut@entry=0xb434)
at ./src/blob.c:933
#1  0x0807bf60 in http_exchange (pSend=pSend@entry=0xb420,
pReply=pReply@entry=0xb434, useLogin=1,
maxRedirect=maxRedirect@entry=20) at ./src/http.c:390
#2  0x080ca4b5 in client_sync (syncFlags=optimized out,
configRcvMask=optimized out, configSendMask=configSendMask@entry=0)
at ./src/xfer.c:1561
#3  0x080acd1c in pull_cmd () at ./src/sync.c:172
#4  0x0804d402 in main (argc=4, argv=0xb6c4) at ./src/main.c:674
(gdb) print pOut-aData
$8 = 0x844aa08 file 59cdb30316cbb0d1d35f248ac3bcabcde7a419c2
988f3554814eeae25a87235c9c8e3a4f84b29a75 64937242\np
class=\generalError\out of memory/p

Re: [fossil-users] How to export ala CVS?

[Eric Rubin-Smith]

Andy Bradford wrote:

 Does Fossil  have an option  that exports  a particular revision  of the
 repository  similar to  how CVS  export  works? CVS  export will  export
 either HEAD or an explicit revision to a named directory without all the
 CVS control directories/files in the export.
 
 As far as I can tell, the only way to get it is:
 
 fossil open REPO VERSION
 fossil close

In case you didn't see it, it looks like from the command line you can 
use this:

==
$ fossil help tarball
Usage: fossil tarball VERSION OUTPUTFILE [--name DIRECTORYNAME] 
[-R|--repository REPO]

Generate a compressed tarball for a specified version.  If the --name
option is used, its argument becomes the name of the top-level directory
in the resulting tarball.  If --name is omitted, the top-level directory
named is derived from the project name, the check-in date and time, and
the artifact ID of the check-in.
==

Or if you are talking to the web service, you can do this:

wget localhost:8080/tarball/foo.tgz?uuid=VERSION

(where I think VERSION can be a tag name too).

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil vs git-based arrangements. code review and ticket export

[Eric Rubin-Smith]

 More seriously, you're comparing a small project like Fossil's

with the capabilities of behemoths like Microsoft.


No, I'm really not.  drh was making a claim that users will ALWAYS have to
convert between two database schemas when exporting tickets from one system
to another.  He was making a technical claim, and I was disagreeing with it
by analogizing the situation to other situations in which export is useful
and in which a product knows about the accepted import formats of other
products.

I could just as easily have said that there is this cool little gem of a
DVCS called fossil.  The devs know about Git of course so they wrote a
Git-specific export tool.  I don't need to know what fossil's schema is,
nor the import format of git, to use the tool.


 I don't think you're going to find the tool you're looking for.


Okay, thanks.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] fossil vs git-based arrangements. code review and ticket export

[Eric Rubin-Smith]

I'm starting a company with some folks.  Their notion of the default choice
for SCM is git + JIRA for bug tracking + some other tool we'd pick for code
review.  This is probably quite common.  Since the answers to my questions
will probably be interesting to a relatively wide audience, I hope the
forum will forgive the length of my post.

By 'code review' here I mean a web-based tool that has a formalized state
machine for (a) uploading code deltas (b) filing and fixing defects against
the uploaded code and (c) having the right set of people sign off.  Like
Code Collaborator.  I guess JIRA has an integrated tool that is free for
small teams too.  Whatever.  It just needs to eventually support reviewing
every line of code that gets written, in case we decide to go in that
direction.

I'm trying to convince my co-founders to use fossil.  I won't address all
of fossil's strengths, since you will all agree with me.  I want you to
give me some ammunition on the weaker points:

 * Export of tickets.

If we want to move away from fossil to a git-based thing in the future, we
can export the source version history, though that is not as trivial as the
Fossil page on the matter makes it seem (see below).

Exporting the tickets is more of a PITA -- AFAIK I'd have to generate CSVs
right from the SQLite database, massage them, and import them into JIRA (or
whatever tool).  (In reality, it looks like JIRA supports JSON import, so
I'd probably wind up writing code to read the SQLite database and emit JSON
instead of CSV -- but whatever, similar idea.)  I'm guessing this would be
like a 1-day project for a relatively large set of tickets.

Furthermore, I'd need to massage all of the check-in comments themselves in
order to reference the new JIRA IDs for tickets rather than the Fossil
IDs.  Is there a better story than this?

 * Code review!

DRH has said on this forum (and I'm paraphrasing here) that he prefers to
review humans more than code.  If he decides he trusts the human, then he
mostly trusts the human's output.  So he didn't really care to write a
native tool for code review for Fossil.

This is what I think I need the most help with.  While I feel mostly the
same as DRH on the matter, I also would like to have a story in case one
day the team decides they want to have every line go through a review
process.  If I don't have a story up front for that then it will be a
biasing factor against Fossil, since it will be more likely we'd need to go
through a transition (one they would worry is onerous or error-prone) to
some other SCM in the future.

I've come up with some work-flows that sort of simulate code reviews right
from within fossil.  For example, you might make each of your check-ins on
its own branch, and when that check-in is accepted it gets merged into the
feature branch.  The existence of a pending review is signaled by a naming
convention on the branches; devs are trained to do an 'lsbranch | grep
conventional name' every X days or whatever.  Defects are filed by
writing comments right into the code under a certain convention and
checking in the commented code against the review branch.  This still
leaves a lot to be desired (e.g. it's very awkward at best for reviewers to
write defects, assign severities to the defects and so on).

My major questions here are:

 (1) I know the fossil devs had considered writing a native proper code
review feature.  Is that anywhere on the near-term road map?

 (2) Are there 3rd-party code review tools that (a) are good and (b)
support fossil?

 (3) Are there clever work-flows using native fossil features that more
closely resemble proper code review tools than the sort of bad one that I
sketched above?

Thanks!
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil vs git-based arrangements. code review and ticket export

[Eric Rubin-Smith]

Richard Hipp wrote: 

 Is there a better story for moving between any two bug tracking 
 systems?  Do there exist any two bug tracking systems in the world were 
 you can move from one to the other without having to write some scripts to 
 transform the data?  

I can't tell whether you're asking rhetorically or in earnest.  I
think rhetorically.

To which I reply that I make no claim that I am being fair. :-)  
I just want to use fossil, and am looking for talking points to 
convince others that the two points I initially enumerated are not
going to bite us.

 Fossil can give you the ticket data as SQL.  I think that is probably 
 about as portable as ticket data is going to get.  

... says the top SQL expert between here and the Romulan Neutral Zone. :-)

From an information theoretic perspective, yes, all the data is 
available.  But that doesn't necessarily mean that it's easy for a poor 
user, unfamiliar with both the Fossil database schema and the import 
requirements of the target tool, to move the data between them.

 You can use the fossil sql command to run .dump to get the ticket 
 content and then load that content into *any* SQL database (MySQL, 
 PostgreSQL, Oracle, SQL Server - name your poison) You will then have 
 to transform the schema into whatever your target system expects.  But I 
 think that is always going to be the case, no?  

I hope not.  Microsoft Office products can import/export to/from 
different formats.  Lots of products know about the particular file 
formats required by other specific products for the purposes of export.

Fossil *could* support export to JIRA+git in particular, for example, by 
providing a tool that (a) exports to JIRA's supported JSON import format,
(b) collects the mapping from the fossil ticket IDs to the JIRA ticket
IDs, then (c) does a git export but massages the check-in comments 
according to the data collected in (b).  

Such a tool that is written and tested by the fossil devs would 
obviously be preferable to whatever my sad little user-brain could 
generate.  

I make no claim that writing such a thing would be the best use of the 
fossil devs' time -- again, just wondering what is out there.  

 We do low-ceremony code review on SQLite.  
{snip} 

 We use more ceremony for testing.  
{snip} 

Thanks for sharing that, Richard.  

-- 
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] slow commit

[Eric Rubin-Smith]

Gour wrote:

 Stephan Beal sgb...@googlemail.com writes:
 
  i don't have any more ideas off-hand, but i've never worked with repos
  having anywhere near that many files. Maybe a list-member who has can
  suggest something. Maybe it's something as simple as changing the sqlite3
  write mode (and maybe it's not).
 
 OK.

Maybe try this?

  fossil setting repo-cksum off

Then re-test.

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] committing to a *new* branch from a closed branch not currently possible?

[Eric Rubin-Smith]

Stephan Beal wrote:

 On Fri, Jul 18, 2014 at 6:06 PM, Matt Welland estifo...@gmail.com wrote:
 
  It seems it is not possible to commit to a new branch from a closed
  branch. this is version 1.28.
 
  I think this should be allowed. Closing a branch only implies to me that
  no more commits are to be made to that branch.
 
 
 The offending code is in checkin.c:
 
   /*
   ** Do not allow a commit against a closed leaf
   */
   if( db_exists(SELECT 1 FROM tagxref
  WHERE tagid=%d AND rid=%d AND tagtype0,
 TAG_CLOSED, vid) ){
 fossil_fatal(cannot commit against a closed leaf);
   }
 
 i unfortunately cannot say how that query needs to be changed to catch
 that, but i'm betting that Jan can.

From http://www.fossil-scm.org/index.html/doc/tip/www/branching.wiki:

==
Closed Leaf

A closed leaf is any leaf with the closed tag. These leaves are
intended to never be extended with descendants and hence are omitted
from lists of leaves in the command-line and web interface.
==

The docs appear quite clear that the behavior OP complains about is
intentional: regardless of whether the new check-in goes to a
different branch, the new check-in is a direct descendant of the closed
leaf.  Probably the OP should just re-open the leaf and commit against 
it.

--
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] committing to a *new* branch from a closed branch not currently possible?

[Eric Rubin-Smith]

Matt Welland wrote: 

  From http://www.fossil-scm.org/index.html/doc/tip/www/branching.wiki: 
  
  == 
  Closed Leaf 
  
  A closed leaf is any leaf with the closed tag.  These leaves are 
  intended to never be extended with descendants and hence are omitted 
  from lists of leaves in the command-line and web interface.  
  == 
  
  The docs appear quite clear that the behavior OP complains about is 
  intentional: regardless of whether the new check-in goes to a 
  different branch, the new check-in is a direct descendant of the closed 
  leaf.  Probably the OP should just re-open the leaf and commit against 
  it.  
  
 
 In that case I'd suggest a new mechanism - locked with the behavior I 
 described.  

I dunno.  In my mind one of fossil's big advantages is that stays out 
of your way because it has a limited number of concepts you have to 
deal with.  This leaves your brain free to write code.  

Absent some stronger argument to the contrary, I'd personally hate 
to see the devs add a new concept to address your relatively narrow 
use case.  Especially since your new concept has significant semantic 
overlap with the existing thing, and since getting the behavior you want 
is pretty darn simple with the existing feature set (just re-open the 
branch).  

That's just $.02 from another user though.  

If you will allow me to guess the reason for your request -- is it 
because you have released to customers against the head of some branch, 
and wish to leave that leaf locked so that you have a clear idea of what 
you have released, may return to it, may write patches against it, etc?  

If so, then FWIW the SQLite and Fossil devs deal with this case by 
merely tagging the version.  E.g. released-8.6.1b3 or whatever.  You 
can always get back to that version (fossil update released-8.6.1b3), 
can always make a new branch from it.  

This is orthogonal to the closed-ness of a branch.  Under this model, 
you'd picture every check-in you've tagged under your versioning 
scheme as a locked thing, since you're being careful not to reuse 
your version tags and since in Fossil the check-ins themselves are 
immutable.  

-- 
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] committing to a *new* branch from a closed branch not currently possible?

[Eric Rubin-Smith]

Matt Welland wrote: 

 The scenario I'd like to see supported is roughly as follows: 
 
 Work on a release is done.  The release manager closes the branch so no 
 one can accidentally merge or commit to it.  
 
 A bug fix is needed.  The developer branches off the closed node and 
 fixes the bug and runs QA.  
 
 The release manager can examine the bug fix, assess the QA results and 
 then open the closed branch, merge the fix and re-close.  
 
 I think this is a useful release flow.  

With only slight adjustments, that flow is supported by the tagging 
scheme that I outlined and that is used to much success and fanfare by 
the SQLite and Fossil folks, as far as I can tell.  

It is also supported by doing what you described, and just having your 
bugfix dev open the leaf before making their new branch.  

I am starting to wonder whether your issue is primarily around the 
simplicity (or coarse-grained-ness) of the user authorization primitives 
in Fossil.  For example, perhaps you wish that you could say user A can
create check-ins but cannot open or close branches.

If so, then it is worth noting a few things about that:

 (a) Adding your own permission primitive is probably not necessary.  
 If you trust user A to make check-ins, why don't you trust them 
 to open and close branches?

 (b) If you are only worried about inadvertent check-ins then you can 
 always correct them after the fact.  Keep in mind that
 the branch on which a check-in was made is super easy to change
 in Fossil.  Also, your release manager will be policing these
 things and will send nasty emails to developers who don't follow
 the local rules.

 (c) If you are worried about developers that are so confused that they
 are doing this often, and your training has consistently failed,
 and this is causing an overall productivity problem...  then fossil 
 is not your primary problem.

 (d) If, despite all that you, *really* want a new permission primitive, 
 it's probably not too hard to add yourself.  And DRH's code is 
 pretty fun to hack against, as many here will attest to. :-)

-- 
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil CLI tricks: interrupting a commit message

[Eric Rubin-Smith]

This thread is hilarious.  I thought I was pretty old-school -- I use 
vi, xterm, fvwm2, and other tools written by my forebears around the 
time when I was born.  I get made fun of by people twice my age for my 
dev toolkit.  

But even *I* will have two terminals up concurrently -- so that I can 
write my check-in comment in terminal 1 while looking at my diff in 
terminal 2.  

I must be one of those millennials with their newfangled contraptions 
and their damn music.  

-- 
Eric A. Rubin-Smith

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Sometimes fossil doesn't sync properly

[Eric Rubin-Smith]

I believe I have seen this issue.  It's been a while, but here is the
scenario as far as I can recollect:

   1. Assume there are three repo copies in a master/client topology: M,
   C1, and C2.  M is the master, and C1/C2 are clones of the master (meaning
   that C1 and C2 don't know about each other; they always push to and pull
   from M).  No proxies anywhere.
   2. M, C1, and C2 are all separated by low-bandwidth, high-latency links,
   which causes syncs to take a long time.
   3. C1 creates a giant file, checks it in, and pushes it to M.  The push
   completes.
   4. C2 begins a pull.
   5. During C2's pull, C1 creates a small file, checks it in, and pushes
   it to M.
   6. A long time later, C2's pull completes.
   7. C2 initiates another pull.  But M does not report the new small file,
   so C2 does not see the new file.

In my case, I worked around the issue by creating yet another artifact from
C1.  This seems to have caused M and/or C2 to sort themselves out.

I have no idea how the sync code works, but at the time I suspected that
there is some sort of optimization involving timestamps, and a slow sync
can cause that code to get confused and miss some artifacts.  May or may
not have an interaction with sqlite's WAL setting (which allows reads
concurrent with one write, if memory serves).

Anyway, hopefully that won't send you on too big of a goose chase.


On Thu, Jun 12, 2014 at 1:54 PM, Stephan Beal sgb...@googlemail.com wrote:

 On Thu, Jun 12, 2014 at 6:00 PM, Andy Bradford amb-fos...@bradfords.org
 wrote:

 This has been mentioned before a few  times, but so far, nobody has been
 able to  provide enough details to  reproduce it and none  of the Fossil
 devs have seen it.


 It's come up 3 or 4 times the past year, IIRC. i _might_ have seen it
 once, but i might have just forgotten to pull in that case. Nobody's been
 able to consistently reproduce it, nor come up with a hint about where it
 may lie. i _think_ (but my memory's far from perfect) we ruled out a
 caching proxy server in the middle in at least one case.


 If it happens again will you provide  the output of both the commit from
 one machine  and the update  from the second? At  least that would  be a
 start.


 +1

 --
 - stephan beal
 http://wanderinghorse.net/home/stephan/
 http://gplus.to/sgbeal
 Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
 those who insist on a perfect world, freedom will have to do. -- Bigby Wolf

 ___
 fossil-users mailing list
 fossil-users@lists.fossil-scm.org
 http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Sometimes fossil doesn't sync properly

[Eric Rubin-Smith]

It may be that you need to replace the one giant file in the below
scenario with a great many files that as a whole take up a lot of bytes.
I don't remember.  Sorry. :-/


On Thu, Jun 12, 2014 at 2:46 PM, Eric Rubin-Smith eas@gmail.com wrote:

 I believe I have seen this issue.  It's been a while, but here is the
 scenario as far as I can recollect:

1. Assume there are three repo copies in a master/client topology:
M, C1, and C2.  M is the master, and C1/C2 are clones of the master
(meaning that C1 and C2 don't know about each other; they always push to
and pull from M).  No proxies anywhere.
2. M, C1, and C2 are all separated by low-bandwidth, high-latency
links, which causes syncs to take a long time.
3. C1 creates a giant file, checks it in, and pushes it to M.  The
push completes.
4. C2 begins a pull.
5. During C2's pull, C1 creates a small file, checks it in, and pushes
it to M.
6. A long time later, C2's pull completes.
7. C2 initiates another pull.  But M does not report the new small
file, so C2 does not see the new file.

 In my case, I worked around the issue by creating yet another artifact
 from C1.  This seems to have caused M and/or C2 to sort themselves out.

 I have no idea how the sync code works, but at the time I suspected that
 there is some sort of optimization involving timestamps, and a slow sync
 can cause that code to get confused and miss some artifacts.  May or may
 not have an interaction with sqlite's WAL setting (which allows reads
 concurrent with one write, if memory serves).

 Anyway, hopefully that won't send you on too big of a goose chase.


 On Thu, Jun 12, 2014 at 1:54 PM, Stephan Beal sgb...@googlemail.com
 wrote:

 On Thu, Jun 12, 2014 at 6:00 PM, Andy Bradford amb-fos...@bradfords.org
 wrote:

 This has been mentioned before a few  times, but so far, nobody has been
 able to  provide enough details to  reproduce it and none  of the Fossil
 devs have seen it.


 It's come up 3 or 4 times the past year, IIRC. i _might_ have seen it
 once, but i might have just forgotten to pull in that case. Nobody's been
 able to consistently reproduce it, nor come up with a hint about where it
 may lie. i _think_ (but my memory's far from perfect) we ruled out a
 caching proxy server in the middle in at least one case.


 If it happens again will you provide  the output of both the commit from
 one machine  and the update  from the second? At  least that would  be a
 start.


 +1

 --
 - stephan beal
 http://wanderinghorse.net/home/stephan/
 http://gplus.to/sgbeal
 Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
 those who insist on a perfect world, freedom will have to do. -- Bigby Wolf

 ___
 fossil-users mailing list
 fossil-users@lists.fossil-scm.org
 http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] autosync from GUI

[Eric Rubin-Smith]

Is it reasonable to ask that the 'autosync' setting cause artifacts created
from the GUI to also be autosynced?

I know this would likely increase the latency of the GUI, and would
possibly create a series of error cases and/or user interactions that do
not need to be handled in the GUI today.  But I believe that today's
behavior nevertheless violates the principle of least astonishment.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] autosync from GUI

[Eric Rubin-Smith]

Stephan Beal wrote: 

  I know this would likely increase the latency of the GUI, and would 
  possibly create a series of error cases and/or user interactions that do 
  not need to be handled in the GUI today.  But I believe that today's 
  behavior nevertheless violates the principle of least astonishment.  
  
 
 OTOH, since you've obviously thought through the long tail of problems 
 which come with having such a feature, you are probably not surprised that 
 Fossil does not currently have that feature ;).  

Okay :) but I don't think the tail should be *that* long.  

Two of Fossil's self-proclaimed major benefits are: (a) its GUI and 
(b) its autosync feature.  That the GUI does not attempt to implement 
autosync at all *is* surprising.  

Note that in non-tiny teams, there is often a project manager 
type whose job includes defining ticket reports, categorizing and 
prioritizing tickets, editing the wiki, and so on.  This type of person 
might exclusively use the GUI.  Forcing them to go use the CLI (even 
after they use the GUI Admin page to choose the 'autosync' setting) 
feels stranger still, since they would have no reason to use the CLI 
otherwise.  

We poor geeks who would like to pitch Fossil to our project teams will 
encounter less resistance as the number of such surprises shrinks.  

Perhaps a middle ground, instead of autosync, would be to have an 
indicator on all (or some) of the GUI pages that a push is necessary.  
I believe you can detect this using just the local repo clone in most 
of the key circumstances.  (You could take it a step further and poll 
the remote clone to see if it has artifacts that the local clone does 
not have, so that you could also report to the user that a pull is 
necessary.  But the devs will likely find that idea distasteful?)  

Eric 

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Version 1.28 release?

[Eric Rubin-Smith]

I have a request.  Can you guys do the official builds SSL-enabled?


On Sat, Jan 11, 2014 at 2:22 PM, Jan Nijtmans jan.nijtm...@gmail.comwrote:

 2014/1/11 Richard Hipp d...@sqlite.org:
  I don't think this should hinder the release.

 That's great news. So the valgrind error in the /tar page and
 the two failing test-cases (which simply could be disabled) are
 the only things which should be handled? I wouldn't know
 anything else. Thanks!

 Regards,
  Jan Nijtmans
 ___
 fossil-users mailing list
 fossil-users@lists.fossil-scm.org
 http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] commit signing

[Eric Rubin-Smith]

If you guys are going to get into this more deeply, you should probably
also consider revocation issues.  That is, what happens when it is
discovered that a contributor's private key has been compromised?

The discovery date of the compromise is obviously = the compromise date.
As such, some set of prior check-ins were signed while the key was
compromised.  So you need to figure out how to deal with those check-ins.
Do you display them differently, or shun them, or so on.

On the other hand, you're not going to address every possible threat model
with your system.  For example, you are not going to prevent rubber-hose
attacks on contributors.

If you haven't already, you should probably enumerate exactly what threat
models you care about.  Only then can you reason clearly about whether you
are protecting against such threats.

I'd imagine, since one of fossil's primary purposes seems to revolve around
preserving a clear and unimpeachable chain of intellectual property
ownership, that you'll want to come up with specific potential attacks on
that chain and figure out if you are preventing them.

See here for an example of a well-defined model, arranged as a tree:
http://tldp.org/HOWTO/Disk-Encryption-HOWTO/introduction.html#ThreatModel

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Ticket system

[Eric Rubin-Smith]

On Thu, Aug 29, 2013 at 6:02 PM, Jan Jurak yan.ju...@gmail.com wrote:

 Dear developers,

 First thank you for nice piece of software. I am using fossil for some
 of my projects and some users wants more featured ticket system. For
 example spent time for solving the issue. What is your opinion on
 that. I am trying avoid installing another software.


You can easily extend the info tracked by fossil to include that:
http://www.fossil-scm.org/index.html/doc/trunk/www/custom_ticket.wiki
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] commit signing

[Eric Rubin-Smith]

 [507ee45f25] http://localhost:8080/info/507ee45f25 Fix an off-by-one
 bug in the network protocol handler so that it can accept a zero-length
 file. (*PGP SIGNED*) (user: 
 drhhttp://localhost:8080/timeline?u=drhc=2007-08-25+12%3A31%3A55nd,
 tags: 
 trunkhttp://localhost:8080/timeline?r=trunkndc=2007-08-25+12%3A31%3A55
 ) 04:02
 [9b30224db7] http://localhost:8080/info/9b30224db7 Closed-Leaf: Merging
 formatting changes to timeline and concepts documentation (*PGP SIGNED*)
 (user: akuhttp://localhost:8080/timeline?u=akuc=2007-08-25+04%3A02%3A27nd,
 tags: 
 trunkhttp://localhost:8080/timeline?r=trunkndc=2007-08-25+04%3A02%3A27
 )


You should be careful how you render things like that.  I think now a
malicious user Mallory can easily subvert your scheme by appending the text
 (*PGP SIGNED*) to the end of his unsigned check-in comment.  People will
think he has signed the check-in when he really hasn't.

It gets worse if Mallory can masquerade as DRH during a check-in, and you
are relying solely on PGP signatures for authentication.  Then you will
think that Mallory's code has DRH's blessing when it really does not.
Mayhem will surely ensue. :-)

This is analogous to a consideration given by the authors of Mutt (an
emailer) in which by default they did not render ANSI color escape
sequences in messages -- again because it could be used to subvert their
PGP rendering scheme.  See http://www.mutt.org/doc/manual/manual-6.html and
search for allow_ansi.

You might be able to cure the issue by rendering the the indicator in a way
that a user cannot affect directly.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] getting fossil timeline to show time offsets

[Dillon, Eric W - Norman, OK - Contractor]

In TCC (a CMD replacement for windows), I use the following where %_unixtime is 
a variable representing the current time.

echo insert into config (name,value,mtime) values 
(timeline-utc,0,%_unixtime); | fossil sql

From: fossil-users-boun...@lists.fossil-scm.org 
[mailto:fossil-users-boun...@lists.fossil-scm.org] On Behalf Of Stephan Beal
Sent: Tuesday, August 20, 2013 11:45
To: Fossil SCM user's discussion
Subject: Re: [fossil-users] getting fossil timeline to show time offsets

On Tue, Aug 20, 2013 at 6:25 PM, j. van den hoff 
veedeeh...@googlemail.commailto:veedeeh...@googlemail.com wrote:
On Tue, 20 Aug 2013 17:59:11 +0200, Benedikt Ahrens 
benedikt.ahr...@gmx.netmailto:benedikt.ahr...@gmx.net wrote:
Hello,

the fossil timeline command shows UTC time rather than observing the
system setting (UTC+02:00, in my case).
Is there anything I can do to change this?

you can switch off UTC (so that it uses local time) in the web interface (admin 
- timeline). don't know whether there is a command line way to do this.

in parallel i've been looking for one and haven't found it. i assume there 
isn't one.

TODO += add 'localtime' config setting (boolean)

IIRC the internal infrastructure is there, it just seems to be missing from the 
CLI settings comment (?).

--
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] fossil silently ignores capitalized ticket table field names

[Eric Rubin-Smith]

I spent a couple hours hunting for a bug in which fossil does not properly
apply changes to my custom fields to the ticket table.  The root cause
appears to be these lines from cgi.c's add_param_list():


if( fossil_islower(zName[0]) ){
  cgi_set_parameter_nocopy(zName, zValue);
}


... i.e. silently ignore capitalized POST argument names.  Is there a
reason for that?

It appears to have been like that since the very first check-in of the
file.  I can't possibly be the only person in six years to have been hit by
this.

Fossil should have rejected my attempt to create capitalized ticket table
column names if it was planning on ignoring edits to those fields later.

And it should warn me on the ticket table edit page that capitalized names
do not work.

Or, better, it should just allow capitalized ticket table column names.

Also, I managed to hose my repo by changing my ticket column names from
upper case to lower case, hoping that fossil would have some heuristics to
carry my data over, or else would warn me that it was going to blow away my
values.  I hoped in vain.  There should be a warning about that, too.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil silently ignores capitalized ticket table field names

[Eric Rubin-Smith]

 ... i.e. silently ignore capitalized POST argument names.  Is there a
 reason for that?


 Yes there is.


Thanks Richard.  So I was initially planning on doing a brute-force
migration to the new ticket table schema with the lowercase letters.  But
it sounds like that is a pointless exercise, since the real point of truth
is the artifacts (and I should have known that, since I read all your
start-up docs).

So I guess I have two options:

 * Blow out all the artifacts into a directory, munge them to have the new
column names, and then rebuild the database from the munged artifacts.
 * Generate new artifacts indicating additions of tags that are the new
column names.  Leave the old tags alone, and just adjust the ticket view
and edit pages' code.

I assume you will find the former distasteful and will recommend the
latter.  Any easy way to accomplish that, or should I start hacking up a
script?

Thanks,
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] admin pages are empty and have bad titles

[Eric Rubin-Smith]

I've resolved this.  I'll share my outcome for future folks who want to get
a low-maintenance fossil HTTPS server up quickly.

The initial scheme was to use 'stunnel' as a reverse proxy to terminate
SSL, and forward the request on to the fossil web server daemon that is
listening on the same box.

Here's the stunnel config I initially came up with:

=
pid = /home/fossil/stunnel.pid

[fossil-https]
accept = 10443
cert = /home/fossil/mysite.com.pem
connect = localhost:10080
=

And, again, here is how I was running fossil:

/usr/local/bin//fossil server /home/fossil/repo.fossil -P 10080 --baseurl
https://mysite.com:10443/

The baseurl here is required under this scheme.  If it were not given, then
when fossil sends a redirect page it would send urls like
http://mysite.com:10080/, because fossil thinks it is speaking HTTP and
that the server is on port 10080.  So the remote browser would follow the
redirect to the wrong L4 endpoint.

The thing mostly works, except that it does not handle the extra slash it
receives in some URLs, e.g. on the links from the Admin page.  (I believe
fossil's name resolution behavior here is defensible under RFC2616, by the
way -- the RFC says that you compare URLs octet-for-octet with one narrow
exception that does not apply here.  So, in other words, adding a slash to
a URL path changes the URL.  So the problem was the URL my browser is
sending in the first place, or in the URL that fossil was putting in the
HREF in the HTML it was serving.)

I considered chasing this further by hacking in the code or looking at
getting a real industrial web server up.  But I saw that DRH had
responded to some previous question that the official fossil web site
itself is also served by HTTPS, which made me think I was overcomplicating
things.

So I tried going for a different scheme in which stunnel behaves a bit more
like xinetd, and used fossil's supporting feature for that:

=
pid = /home/fossil/stunnel.pid
output = /home/fossil/stunnel.log

[fossil-https]
accept = 10443
cert = /home/fossil/mysite.com.pem
exec = /usr/local/bin/fossil
execargs = fossil http /home/fossil/repo.fossil --https --host
mysite.com:10443
=

I.e. fork one fossil process per request.  This appears to Just Work and is
probably what the fossil devs had initially intended.

From a look at the code, by the way, the --https argument there is
required to prevent fossil from thinking that it should not authenticate
the user.

Eric



On Wed, Jul 24, 2013 at 10:06 PM, Andy Bradford 
amb-sendok-137730.cdeapjlkdpclmgfol...@bradfords.org wrote:

 Thus said Eric Rubin-Smith on Wed, 24 Jul 2013 10:55:24 -0400:

  Am  I doing  something wrong  with  my configs,  or is  a code  change
  warranted?

 That's hard to say since I don't know under what conditions --baseurl is
 intended to be  used (I know the  docs say reverse proxy,  but I haven't
 ever set  one up so  I don't understand all  the fine details).  The one
 time I tried to use it, I was  doing it wrong:

 http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg12107.html

 In my case,  however, I was using ``fossil http''  not ``fossil server''
 and when I got rid of the --baseurl option, things worked as expected.

 In your case,  you are trying to  setup a reverse proxy...  if you could
 provide some  details about  to setup  a reverse  proxy similar  to your
 configuration (perhaps it  is done with Apache), it might  be easier for
 someone to reproduce.

 Have you tried using it without --baseurl?  If so, what happened?

 Thanks,

 Andy
 --
 TAI64 timestamp: 400051f08852



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] cvs to git to fossil does not preserve user names on check-ins

[Eric Rubin-Smith]

I exported my CVS repo to git using cvs2git (version 2.4.0-dev), and
ingested the resulting git repo into fossil according to the instructions
on the fossil web site, using fossil 1.26.  My git version is 1.7.7.6.

This failed to preserve the original CVS user names.

The reason is that git assumes a null email address for the incoming CVS
user names, so that the git export file has records like this:

author eas  1368997852 +
committer eas  1368997852 +

where eas is my CVS user name, and the null email address is between the
angle brackets.

The fossil importer ignores the first user name, going only for whatever's
in the angle brackets:

if( memcmp(zLine, tagger , 7)==0 || memcmp(zLine, committer ,10)==0
){
  sqlite3_int64 secSince1970;
  for(i=0; zLine[i]  zLine[i]!=''; i++){}
  if( zLine[i]==0 ) goto malformed_line;
  z = zLine[i+1];
  for(i=i+1; zLine[i]  zLine[i]!=''; i++){}
  if( zLine[i]==0 ) goto malformed_line;
  zLine[i] = 0;
  fossil_free(gg.zUser);
  gg.zUser = fossil_strdup(z);
  secSince1970 = 0;
  for(i=i+2; fossil_isdigit(zLine[i]); i++){
secSince1970 = secSince1970*10 + zLine[i] - '0';
  }
  fossil_free(gg.zDate);
  gg.zDate = db_text(0, SELECT datetime(%lld, 'unixepoch'),
secSince1970);
  gg.zDate[10] = 'T';


I made a quick-and-dirty hack to use the first user name in the case where
the email address is empty.  I thought I'd post it in case someone in the
future finds it helpful.

The patch comes with a long list of caveats*, and I'm not suggesting it be
adopted into the fossil trunk at this time.  Just posting it for the narrow
purpose of possibly helping other users who are having the same issue
getting their CVS repos into fossil.

--- import.c.orig   2013-07-22 16:24:26.305215527 -0400
+++ import.c2013-07-27 10:49:29.894610274 -0400
@@ -471,21 +471,21 @@
   zName[i] = 0;
 }


 /*
 ** Read the git-fast-import format from pIn and insert the corresponding
 ** content into the database.
 */
 static void git_fast_import(FILE *pIn){
   ImportFile *pFile, *pNew;
-  int i, mx;
+  int i, j, k, mx;
   char *z;
   char *zUuid;
   char *zName;
   char *zPerm;
   char *zFrom;
   char *zTo;
   char zLine[1000];

   gg.xFinish = finish_noop;
   while( fgets(zLine, sizeof(zLine), pIn) ){
@@ -569,27 +569,41 @@
 }else
 if( memcmp(zLine, author , 7)==0 ){
   /* No-op */
 }else
 if( memcmp(zLine, mark , 5)==0 ){
   trim_newline(zLine[5]);
   fossil_free(gg.zMark);
   gg.zMark = fossil_strdup(zLine[5]);
 }else
 if( memcmp(zLine, tagger , 7)==0 || memcmp(zLine, committer
,10)==0 ){
+  /* Try first to use the email address that is in angle brackets.  If
+  ** that is empty, then use the user name that precedes it.
+  */
+  j = (zLine[0]=='t' ? 7 : 10); /* Index the first char of the user
name. */
   sqlite3_int64 secSince1970;
   for(i=0; zLine[i]  zLine[i]!=''; i++){}
   if( zLine[i]==0 ) goto malformed_line;
+  k = i-1; /* Index the character just after the user name. */
   z = zLine[i+1];
   for(i=i+1; zLine[i]  zLine[i]!=''; i++){}
   if( zLine[i]==0 ) goto malformed_line;
-  zLine[i] = 0;
+  if( z[0]==zLine[i] ){
+/* The email address is empty.  Use the user name instead of the
+** email address.
+*/
+if( kj ) goto malformed_line;
+z=zLine[j];
+zLine[k] = 0;
+  }else{
+zLine[i] = 0;
+  }
   fossil_free(gg.zUser);
   gg.zUser = fossil_strdup(z);
   secSince1970 = 0;
   for(i=i+2; fossil_isdigit(zLine[i]); i++){
 secSince1970 = secSince1970*10 + zLine[i] - '0';
   }
   fossil_free(gg.zDate);
   gg.zDate = db_text(0, SELECT datetime(%lld, 'unixepoch'),
secSince1970);
   gg.zDate[10] = 'T';
 }else

Eric

* This solution is not the right approach overall -- it's probably better
to permit command-line options altering the behavior here.  I tested it
only one time, on my one well-formed repo export file, using a different
version of the patch.  All my CVS usernames are just lowercase alphabet
characters (with no spaces or funny characters that one of the tools in
question might want to escape or parse differently).  I had been using git
and fossil for a total of one calendar day when I made the hack.  I have no
idea if git promises to retain their export format.  I was born yesterday,
as a fledgling greenhorn tenderfoot newbie fish-face bush leaguer, on the
back of a turnip truck.  Etc etc etc.  YMMV.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Ingest CVS repo + cvstrac tickets into fossil?

[Eric Rubin-Smith]

On Mon, Jul 22, 2013 at 3:05 PM, Stephan Beal sgb...@googlemail.com wrote:

 On Mon, Jul 22, 2013 at 9:04 PM, Richard Hipp d...@sqlite.org wrote:

 The stumbling block is that the ticket text is Wiki, but the format for
 Fossil Wiki and CVSTrac Wiki is different, which would require a tricky
 translator.


 But the ticket system now allows one (IIRC) to change the text type to
 plain, which could(?) be used to import them in some halfway usable form?


Just importing them as plain text would totally be sufficient for my use
case -- my brain has a CVSTrac wiki rendering engine in it anyway. :-) Or,
frankly, just importing them in whatever fashion would be fine for now --
if the wiki renderings are wrong for some of them, I can just fix them by
hand.  If later on I'm so burdened that I need to write a translator, I can
do that.

Basically, your wiki consideration does not seriously concern me -- mostly
right is good enough here (just need to preserve severities, headlines etc
in the right places).  I'm quickly moving in the direction of adopting
fossil, and the need to ingest my old tickets is I think the one remaining
serious issue.

Any recommendations or code you can share before I go start hacking on the
sql tables directly (which of course I'd prefer to avoid)?

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] fossil commit is extremely slow

[Eric Rubin-Smith]

I have a largish repo I ingested from CVS (via git, as I previously
described on this list).  I'm using fossil 1.26.

A tiny commit to a single file takes 63 seconds:

[monk:code] $ fossil diff
Index: {snip}/test-file
==
--- {snip}/test-file
+++ {snip}/test-file
@@ -11,5 +11,7 @@
 Test6

 test7

 test8
+
+test9

[monk:code] $ time fossil commit -m Test check-in
New_Version: c46175729e936137f58ef302308d1e95b62e6a61

real1m2.767s
user0m15.090s
sys 0m7.227s

I.e. ~22 seconds of CPU usage, and presumably the rest is on the disk.

The box is pretty old (see below for /proc/cpuinfo), and I know that fossil
is not written to be a speed demon -- but this still seems pretty
ridiculous.

Any way to speed it up for the very common case of making small commits?

How can I gather some more useful data so experts can help me?

Thanks,
Eric

===

processor   : 0
vendor_id   : AuthenticAMD
cpu family  : 15
model   : 127
model name  : AMD Athlon(tm) Processor LE-1640
stepping: 2
cpu MHz : 1000.000
cache size  : 512 KB
fdiv_bug: no
hlt_bug : no
f00f_bug: no
coma_bug: no
fpu : yes
fpu_exception   : yes
cpuid level : 1
wp  : yes
flags   : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca
cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx mmxext fxsr_opt rdtscp
lm 3dnowext 3dnow up extd_apicid pni cx16 lahf_lm svm extapic cr8_legacy
3dnowprefetch lbrv
bogomips: 1999.50
clflush size: 64
cache_alignment : 64
address sizes   : 40 bits physical, 48 bits virtual
power management: ts fid vid ttp tm stc 100mhzsteps
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil commit is extremely slow

[Eric Rubin-Smith]

On Sat, Jul 27, 2013 at 3:23 PM, Richard Hipp d...@sqlite.org wrote:


 That is ridiculous.  Most commits take less than a second, even on archaic
 machines, such as my 15-year-old PPC iBook clocked at 400MHz.

 How many files are in your check-out?


[monk:repo.fossil] $ find .|wc -l
8095

What's the total size of all those files (how big is the checkout)?


[monk:repo.fossil] $ du -sch .
392M.
392Mtotal


 Is the repository or the check-out on a network filesystem?


No and no.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil commit is extremely slow

[Eric Rubin-Smith]

On Sat, Jul 27, 2013 at 4:15 PM, Richard Hipp d...@sqlite.org wrote:



 On Sat, Jul 27, 2013 at 3:41 PM, Eric Rubin-Smith eas@gmail.comwrote:

 On Sat, Jul 27, 2013 at 3:23 PM, Richard Hipp d...@sqlite.org wrote:

 What's the total size of all those files (how big is the checkout)?


 [monk:repo.fossil] $ du -sch .
 392M.
 392Mtotal



 That would be the culprit.  As one of several self-checks (see
 http://www.fossil-scm.org/fossil/doc/trunk/www/selfcheck.wiki), Fossil
 always computes an MD5 checksum over the entire check-out and compares that
 to the content being checked in, to make sure they are identical.  With a
 392MB checkout on an older machine, that might easily take a minute.



I tested this basic claim and do not believe it holds:

[monk:~] $ head -c $(echo 392*1024*1024|bc) /dev/zero  foo
[monk:~] $ du -sch foo
392Mfoo
392Mtotal
[monk:~] $ time md5sum foo
c6d8f8fc5c75fd6ecceb4edf42f3ac4d  foo

real0m1.324s
user0m0.998s
sys 0m0.247s

So just over a second to calculate that hash on the same box.  I retried
this after dropping kernel caches to test whether it's the disk, and it
still only took 3.6 seconds to calculate the hash.

Of course, that's just the time it takes to calculate the hash.  Obviously
it does not include the time spent concatenating the world together to send
to your MD5 function.  Perhaps there's a super-linear algorithm in that
concatenation stuff?

Turning off repo-cksum* **did* address the issue, at least by an order of
magnitude:

[monk:code] $ fossil setting repo-cksum off
[monk:code] $ time fossil commit -m test commit
New_Version: 4d3b92dca8a617d6004bbe4e9c158fc11882720d

real0m7.365s
user0m0.627s
sys 0m0.398s

Does this leave any serious gaps in fault-tolerance?

The new performance is acceptable, though I'm still happy to keep digging
around if you're still curious (either about what was taking so long, or
about what is still taking 7 seconds, or both).

Thanks Richard.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] fossil commit is extremely slow

[Eric Rubin-Smith]

On Sat, Jul 27, 2013 at 4:58 PM, Andy Bradford 
amb-sendok-1377550706.oeilkncbciakkppah...@bradfords.org wrote:

 Thus said Eric Rubin-Smith on Sat, 27 Jul 2013 16:31:46 -0400:

  I tested this basic claim and do not believe it holds:
 
  [monk:~] $ head -c $(echo 392*1024*1024|bc) /dev/zero  foo
  [monk:~] $ du -sch foo
  392Mfoo
  392Mtotal
  [monk:~] $ time md5sum foo
  c6d8f8fc5c75fd6ecceb4edf42f3ac4d  foo
 
  real0m1.324s
  user0m0.998s
  sys 0m0.247s

 I  believe  this test  is  slightly  flawed.  You  have 8095  files  and
 directories for a total  of 392M. This is not at all the  same as 1 file
 that totals 392M.  So your test doesn't account for  the distribution of
 the data  on the  disk and  the file system  slowness that  could result
 therefrom.


Good point!  Not to mention duplicated syscall overhead etc.  I ran a riff
on your idea and got a very different result:

[monk:repo.fossil] $ time find . -type f -exec cat {} \; | md5sum -
3abe8f411181a328c7b64946ff6a9c7a  -

real0m37.631s
user0m2.973s
sys 0m11.543s

As you predicted, most of that time is spent on disk I/O, not e.g. in
forking 'cat'.  So that explains over half of the run-time for my fossil
command.

For the other half, I ran fossil under callgrind and found that at least
44% of its instruction reads were inside zlib, and at least 34% were spent
updating the MD5 sum:


Ir

41,797,779,918  PROGRAM TOTALS


Ir  file:function


18,101,410,264   /usr/src/debug/zlib-1.2.5/inflate.c:inflate (55531x)
[/lib64/libz.so.1.2.5]
18,101,410,264  *  /usr/src/debug/zlib-1.2.5/inffast.c:inflate_fast
[/lib64/libz.so.1.2.5]

13,824,797,833   /home/eas/Fossil-c9cb6e72932fefbe/./src/md5.c:MD5Update
(24296657x) [/usr/local/bin/fossil-1.26-eas-built]
 3,983   /home/eas/Fossil-c9cb6e72932fefbe/./src/md5.c:MD5Final
(7x) [/usr/local/bin/fossil-1.26-eas-built]
13,824,801,816  *
/home/eas/Fossil-c9cb6e72932fefbe/./src/md5.c:MD5Transform
[/usr/local/bin/fossil-1.26-eas-built]

(and those are just the top two functions).

All that uncompressing seems to come from blob_uncompress.  So I guess the
only remaining question is whether all those blob uncompresses are really
necessary.  I assume yes -- and in any case I have my answers. :-)

Thanks again.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] merging and file names: help text is wrong and conflicts are not reported

[Eric Rubin-Smith]

From 'fossil help merge':

===
Only file content is merged.  The result continues to use the
file and directory names from the current checkout even if those
names might have been changed in the branch being merged in.
===

This struck me as very odd.  If the file name only changed on the branch
being merged in (called M in the code) since the nearest common ancestor,
then the file name on M should be chosen.  So I did some digging.  It turns
out that this documentation is not correct.

The code in merge.c and my own hand-testing confirm that the behavior is
what you would expect.

So I think the help text needs an update.

And, BTW, the fact that renaming conflicts are not raised to the user
(fossil silently chooses the target version's name) should probably be
considered a serious bug.

I skimmed through open tickets on the ticket tracker and didn't find
anything open for that.  I noticed a related ticket here[1], however, which
I think is no longer valid and can be closed.

Eric

[1] http://www.fossil-scm.org/index.html/tktview?name=67176c3aa4
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] admin pages are empty and have bad titles

[Eric Rubin-Smith]

I've started a fossil repo by importing a git repo to my local laptop, and
then cloning the repo over to my target production web server.

For testing, I've exposed the fossil server like this:

/usr/local/bin/fossil server /home/fossil/myrepo.fossil --th-trace -P 10080
--baseurl http://localhost:10080/

and I'm accessing the server from my laptop using an ssh tunnel:

ssh -L 10080:localhost:10080 mycompany.com

so that I can point my browser to localhost:10080 and see the site exposed
by fossil.  Works fine, including login etc, until I try to access an admin
page.

By an admin page, I mean any admin page.  They all behave the same.

I click Admin and that works fine.  I click any of the links, and I'm
taken to a web page that has all of fossil's normal headers (so the name of
my project, who I'm logged in as, links for Admin, the Timeline etc) but
the body of the web page says Empty page.  And the page title is e.g.
xfersetup instead of Transfer Setup.  The other pages' titles are
analogous.

The TH tracer shows no indication that there is a capabilities issue:

=
BEGIN_HEADER
th1-setup {} = TH_OK
set project_name {My project name}
set title {xfersetup}
set baseurl {http://localhost:10080/}
set home {}
set index_page {/home}
set current_page {?name=xfersetup}
set release_version {1.26}
set manifest_version {[c9cb6e7293]}
set manifest_date {2013-06-18 21:09:23}
set compiler_name {gcc-4.4.5 20101112 (Red Hat 4.4.5-2)}
set login {eas}
BEGIN_HEADER_SCRIPT
eval {

 if {[info exists login]} {
   puts Logged in as $login
 } else {
   puts Not logged in
 }


}
eval {

html a href='$home$index_page'Home/a\n
if {[anycap jor]} {
  html a href='$home/timeline'Timeline/a\n
}
if {[hascap oh]} {
  html a href='$home/dir?ci=tip'Files/a\n
}
if {[hascap o]} {
  html a href='$home/brlist'Branches/a\n
  html a href='$home/taglist'Tags/a\n
}
if {[hascap r]} {
  html a href='$home/reportlist'Tickets/a\n
}
if {[hascap j]} {
  html a href='$home/wiki'Wiki/a\n
}
if {[hascap s]} {
  html a href='$home/setup'Admin/a\n
} elseif {[hascap a]} {
  html a href='$home/setup_ulist'Users/a\n
}
if {[info exists login]} {
  html a href='$home/login'Logout/a\n
} else {
  html a href='$home/login'Login/a\n
}

}
[hascap jor] = 1
[hascap oh] = 1
[hascap o] = 1
[hascap r] = 1
[hascap j] = 1
[hascap s] = 1
END_HEADER
BEGIN_FOOTER
eval {

puts [expr {([utime]+[stime]+1000)/1000*0.001}]

}
END_FOOTER
=


Any hints?

Thanks,
Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] admin pages are empty and have bad titles

[Eric Rubin-Smith]

Yes, that works for the test case.  But I think I'll need --baseurl for
when I put fossil behind an SSL-terminating reverse proxy and want to
access it using the company FQDN.


On Tue, Jul 23, 2013 at 10:22 PM, Andy Bradford 
amb-sendok-1377224557.emjjjkijcgiknbipb...@bradfords.org wrote:

 Thus said Eric Rubin-Smith on Tue, 23 Jul 2013 22:02:11 -0400:

  /usr/local/bin/fossil server /home/fossil/myrepo.fossil --th-trace -P
 10080
  --baseurl http://localhost:10080/

 Try removing the --baseurl option.

 It works for me when I do:

 fossil server /tmp/test.fossil

 ssh -L 10080:localhost:10080 remote

 Andy
 --
 TAI64 timestamp: 400051ef3a90



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Ingest CVS repo + cvstrac tickets into fossil?

[Eric Rubin-Smith]

I've seen some traffic on this list that touches on the issue, but haven't
seen anyone offer specific scripts that will let me ingest a CVS repo *plus
all my CVSTrac tickets* into fossil.

I've successfully used cvs2git to move my repo to a git repo, and
successfully imported the git repo into fossil.  But my CVSTrac tickets are
still orphans.

I don't care about wiki pages or reports.

The ticket schemas look similar enough that I could possibly hack something
on my own, but if DRH or someone else intimately familiar with the two
schemas has already written a conversion utility, all the better.

Does anyone have such a thing?  I see elsewhere in the list that DRH
mentioned a 200-some-odd-line Tcl script that does at least part of the
task -- is that published somewhere?

Thanks to DRH and team for the continued beautiful work.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Several commands not working in Win32/MinGW

[Dillon, Eric W - Norman, OK - Contractor]

Jan, 

Updated to tip of trunk and problem is gone. Thanks for the quick fix.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Several commands not working in Win32/MinGW

[Dillon, Eric W - Norman, OK - Contractor]

 I'm getting an idea what's going on: writability of a directory on
 win32 does not
 really tell whether a file in this directory can be written. Please have a 
 look at branch home-check-too-restricted.
 https://www.fossil-scm.org/index.html/info/1e90d43d76
 This branch checks if the $HOME/.fossil file can be written in stead of the 
 $HOME directory.

Updated to 1e90d43d764 (home-check-too-restricted), compiled and I'm not seeing 
any issues on first inspection.

Eric
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] timeline bug?

[Eric]

On Wed, 16 Jan 2013 17:48:52 +0100, j. van den hoff 
veedeeh...@googlemail.com wrote:
 hi,
 
 incidentally (when calling fossil from with a script constructing the
 suitable call) I noted that
 
 +fossil timeline ''+
 
 (i.e. appending an empty string '' after `timeline') shows the timeline
 below (and including *CURRENT*) instead of below the most recent checkin.
 therefore, output is different
 if *CURRENT* is not identical to leave.
 
 is this a bug or a feature? I do hope the former since it's really bad
 behaviour when driving `fossil' with a script.

I don't get it. No-one would ever put that on the commandline, and if
you are writing a script to construct and run the command, why would you
ever construct that? If it's a consequence of having no other parameters
just make your script check for that and avoid putting in an empty string.

It makes no sense to me that you are generating something that makes no
sense.

Eric
-- 
ms fnd in a lbry
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users