Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Eric Rubin-Smith


> On May 27, 2016, at 12:26, Andy Gibbs  wrote:
> 
> Hi,
> 
> I've just had a very, very odd experience with fossil.  I'm running version 
> 1.34.
> 
> Let me first explain what I have done.
> 
> I cloned a respository off our server.  I then went into the clone's web UI 
> and disabled the auto-sync feature.  I then made 7 commits, the first of 
> which caused a trunk fork which I then "repaired" by branching the old 
> tip-of-trunk that caused the fork.  I then committed two commits to the new 
> branch, then realised that I'd committed the wrong code, so shunned the last 
> two commits, rebuilt the respository, unshunned the sha1 signatures, rebuilt, 
> then recommitted the last two commits using the correct code.
> 
> Ok, a little complicated but nothing too out-of-the-ordinary, I hope.
> 
> Except ... now I have two utterly random commits in my cloned repository - 
> both seemingly from the sqlite repository:
> 
> 2014-12-09 [f66f7a17b7] Version 3.8.7.4 (user: drh, tags: release, 
> version-3.8.7.4)
> 2011-05-19 [ed1da510a2] Version 3.7.6.3 release candidate 1. (user: drh)

If you commit a file that looks like a fossil manifest, then fossil treats it 
as such and displays the commits listed therein as if they happened in your 
actual repository.  Or something. I had this happen to me too a while back when 
I did the same thing, importing the SQLite tree wholesale into another fossil 
repo.  Drh said at the time that there's no "real" corruption.  

Sent from my iPhone


> 
> Now, it is true that I have a clone of the sqlite respository on my server 
> too ... but I haven't yet synced with the server.  Absolutely no server 
> communication has happened since my initial clone.  And these odd artifacts 
> were definitely not there (or, rephrased, definitely not showing) when I was 
> mid-way through all of this work, and are not there on the server's copy of 
> the repository either.
> 
> Unfortunately, I cannot say exactly when these artifacts appeared, but my 
> hunch would be somepoint around the shunning/rebuilding process?  Does this 
> even make sense?!?
> 
> Worse, I am left not sure whether to simply shun the two random artifacts and 
> then push the changes to the repository up to the server.  It has taken the 
> best part of a day to get all these commits in and it represents about 6GB of 
> source files (the repository has doubled from ~700 MB to ~1.5 GB) requiring a 
> lot of manual "fossil mv"s to synchronise many moved files (fossil doesn't 
> yet support a git-like auto-mv-detection sadly).
> 
> Any idea how I can easily check the validity of my repository?  I have done 
> the obvious which is to check out each of the recent check-ins and compare 
> them with the original source folders, but what I don't know is whether the 
> structure of my respository is damaged in some way...
> 
> Help :o)
> 
> Thanks!
> Andy
> 
> ___
> fossil-users mailing list
> fossil-users@lists.fossil-scm.org
> http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Ross Berteig

On 5/27/2016 3:58 PM, John P. Rouillard wrote:

Hi Richard:

Richard Hipp writes:

Just to be clear, I consider anything involving shunning to be
out-of-the-ordinary.


Perfectly reasonable.

On that note, does anybody have code for tcl hooks that can be used to
reject artifacts that have text that matches a particular pattern


IMHO, that is capability is outside the core mission of fossil. One of 
the great selling points of fossil to new users is the low ceremony of 
fossil. One executable. A repo is a single file, that can even safely 
reside in the same folder as your checkout. And very little policy is 
enforced by fossil directly.


Aside, of course, from the big one: fossil preserves everything, and 
that history is immutable.


That said, there is a hook mechanism that can be used to check 
preconditions before executing a command. It might require some 
cleverness to use it for this purpose, however.


Hooks exist if fossil is compiled with the option, are written in TH1, 
can configured to make Tcl available to TH1, and are not extensively 
documented. One of the hooks is "command_hook" which is invoked for 
every fossil command. That name can raise an error (or call break or 
continue) to prevent the fossil command from executing.


Since the hook is called early, it knows the command name, its 
arguments, its flags, and not a lot else. Hence the need for cleverness 
since you would want to learn what files are going to be committed.


This hook has to run at the client, and before the commit is performed. 
So that won't prevent a user from bypassing it, or a misconfigured 
repostory from failing to call it, or if depending on Tcl, I'm sure 
there are more failure modes since Tcl is (usually, depending on 
configuration of fossil) loaded from the system at run time.




If you really wanted to commit a file that matched that pattern, you
added a string like: BYPASSPASSWORD to the commit message and the
check would be bypassed.


The fossil -no-th-hooks option will skip all hooks for that command. 
Depending on what else you do in a hook, that might be more than you wanted.



Is there some similar way to inspect the transferred artifacts and
file contents and roll back the commit?


Nope. And there can't be. Nothing is transferred until well after the 
whole collection of artifacts that make up the commit have been created 
and safely stowed in the local repository. There is no "roll back" from 
that.


--
Ross Berteig   r...@cheshireeng.com
Cheshire Engineering Corp.   http://www.CheshireEng.com/
+1 626 303 1602
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Ron W
On Fri, May 27, 2016 at 6:58 PM, John P. Rouillard  wrote:

>
> On that note, does anybody have code for tcl hooks that can be used to
> reject artifacts that have text that matches a particular pattern.


The closest Fossil has to this is the "commit permission", which blocks the
commit based on the user ID.

Otherwise, to "roll back" a commit requires shunning the manifest.
(Shunning the files is also possible, but you have to be careful to only
shun the ones that actually changed in that commit).

Either way, the originating repo will still have the commit.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Rejected JSON tests

2016-05-27 Thread Stephan Beal
On Fri, May 27, 2016 at 1:39 PM, Kain Abel  wrote:

>
> ./src/cson_amalgamation.c: In function ‘cson_value_new_integer’:
> ./src/cson_amalgamation.c:2863:13: warning: dereferencing type-punned
> pointer will break strict-aliasing rules [-Wstrict-aliasing]
>  *CSON_INT(c) = v;
>

Hi again Kain,

i suspect that this warning might be specific to gcc 4.4. i just tried it
on the original sources with gcc 4.8 and it does not warn there (and also
didn't with the older gcc versions it was original developed with):

[stephan@host:~/fossil/cson]$ gcc -c -pedantic -Wall -Werror -fPIC
-Wstrict-aliasing -g -std=c89 cson_amalgamation.c
[stephan@host:~/fossil/cson]$ gcc --version
gcc (Ubuntu 4.8.4-2ubuntu1~14.04.3) 4.8.4

also tried with -std=c99.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Rejected JSON tests

2016-05-27 Thread Ross Berteig

On 5/27/2016 4:39 AM, Kain Abel wrote:

Dear devs, dear users,

the script tester.tcl don't like the JSON support?.

lm@um:/tmp$ tclsh /tmp/test/tester.tcl /tmp/fossil -quiet -prot
can't find package json
while executing
"package require json"
(file "/tmp/test/json.test" line 38)
invoked from within



The issue here is that you don't have the json package installed in your 
copy of Tcl with which you are running the test harness.


There is a comment in json.test that has the reminder of which json 
library I used. Specifically, I used the one from tcllib, as packaged 
and delivered by ActiveTcl's teacup utility.


https://www.fossil-scm.org/index.html/artifact/65f8333164e4?txt=1=33-38

On linux, you'll have to find your own way to get the library, IIRC I 
was able to tease it out of apt-get, but I don't recall what its name was.

--
Ross Berteig   r...@cheshireeng.com
Cheshire Engineering Corp.   http://www.CheshireEng.com/
+1 626 303 1602
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Richard Hipp
On 5/27/16, Andy Gibbs  wrote:
>
> Ok, a little complicated but nothing too out-of-the-ordinary, I hope.
>

Just to be clear, I consider anything involving shunning to be
out-of-the-ordinary.
-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Stephan Beal
On Fri, May 27, 2016 at 6:39 PM, Andy Gibbs  wrote:

>File 3rdparty/chromium/third_party/sqlite/sqlite-src-3080704/manifest -
> part of check-in [079a920cf4] at 2016-03-13 00:00:00 on branch trunk - **
> message elided **
>File 3rdparty/chromium/third_party/sqlite/src/manifest - part of
> check-in [079a920cf4] at 2016-03-13 00:00:00 on branch trunk - ** message
> elided **
>

That's the source of your... troubles is not quite the word (see below). In
fossil, any content which looks like a manifest, is a manifest and will be
treated as such. There is no marking of "that is a manifest" - it's
determined by checking "can it be parsed as a manifest?"

To quote:

http://fossil-scm.org/index.html/doc/trunk/www/fileformat.wiki

"Any artifact in the repository that follows the syntactic rules of a
manifest is a manifest. Note that a manifest can be both a real manifest
and also a content file, though this is rare."




> And the other artifact is the similar.  Could this account for the odd
> behaviour?  Fossil has somehow got confused about an actual commit and a
> manifest file that has been checked in?
>

It's not confused - this is how it works. Admittedly, a bit confusing, but
AFAIK harmless. Once, while testing libfossil's manifest parser, i
inadvertently imported a manifest from the main fossil tree into it, and
was very surprised to see a commit with the username "drh". It didn't break
anything (somewhat surprisingly).


> So, I certainly don't want to shun this artifact because then I'll lose a
> file from a valid check-in -- isn't that right?


i believe that's correct.


> But how can I fix it?
>

This is how fossil behaves, so there's nothing to fix :/.

-- 
- stephan beal
http://wanderinghorse.net/home/stephan/
http://gplus.to/sgbeal
"Freedom is sloppy. But since tyranny's the only guaranteed byproduct of
those who insist on a perfect world, freedom will have to do." -- Bigby Wolf
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


Re: [fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Andy Gibbs

Hi again,

I've just discovered something and maybe it is helpful.  I checked the 
"artifact" webpage for both of these rogue artifacts, and I get multiple 
sources for the artifact in question, e.g.:


http://localhost:8080/artifact/f66f7a17b7

Artifact f66f7a17b78ba617acde90fc810107f34f1a1f2e:

   File 3rdparty/chromium/third_party/sqlite/sqlite-src-3080704/manifest - 
part of check-in [079a920cf4] at 2016-03-13 00:00:00 on branch trunk - ** 
message elided **
   File 3rdparty/chromium/third_party/sqlite/src/manifest - part of 
check-in [079a920cf4] at 2016-03-13 00:00:00 on branch trunk - ** message 
elided **


Also Manifest of check-in [f66f7a17b7] - Version 3.8.7.4 by drh on 
2014-12-09 01:34:36.


   C Version\s3.8.7.4
   D 2014-12-09T01:34:36.054
   F Makefile.arm-wince-mingw32ce-gcc 
d6df77f1f48d690bd73162294bbba7f59507c72f

   F Makefile.in cf57f673d77606ab0f2d9627ca52a9ba1464146a
   F Makefile.linux-gcc 91d710bdc4998cb015f39edf3cb314ec4f4d7e23
   F Makefile.msc e31dee24038965fb6269d6d61073fd6b7e331dec
   F Makefile.vxworks 034289efa9d591b04b1a73598623119c306cbba0
   F README.md 64f270c43c38c46de749e419c22f0ae2f4499fe8
   F VERSION d7e46e14bd7393d54d19ad900222e5f20d41ea1b
   ...

And the other artifact is the similar.  Could this account for the odd 
behaviour?  Fossil has somehow got confused about an actual commit and a 
manifest file that has been checked in?


So, I certainly don't want to shun this artifact because then I'll lose a 
file from a valid check-in -- isn't that right?  But how can I fix it?


Thanks again for your help in my hour of need :o)

Andy


- Original Message - 
From: "Andy Gibbs" 

To: 
Sent: Friday, May 27, 2016 6:26 PM
Subject: Weird cross-contamination between two fossil repositories (and not 
even talking to server!)




Hi,

I've just had a very, very odd experience with fossil.  I'm running 
version 1.34.


Let me first explain what I have done.

I cloned a respository off our server.  I then went into the clone's web 
UI and disabled the auto-sync feature.  I then made 7 commits, the first 
of which caused a trunk fork which I then "repaired" by branching the old 
tip-of-trunk that caused the fork.  I then committed two commits to the 
new branch, then realised that I'd committed the wrong code, so shunned 
the last two commits, rebuilt the respository, unshunned the sha1 
signatures, rebuilt, then recommitted the last two commits using the 
correct code.


Ok, a little complicated but nothing too out-of-the-ordinary, I hope.

Except ... now I have two utterly random commits in my cloned repository - 
both seemingly from the sqlite repository:


2014-12-09 [f66f7a17b7] Version 3.8.7.4 (user: drh, tags: release, 
version-3.8.7.4)

2011-05-19 [ed1da510a2] Version 3.7.6.3 release candidate 1. (user: drh)

Now, it is true that I have a clone of the sqlite respository on my server 
too ... but I haven't yet synced with the server.  Absolutely no server 
communication has happened since my initial clone.  And these odd 
artifacts were definitely not there (or, rephrased, definitely not 
showing) when I was mid-way through all of this work, and are not there on 
the server's copy of the repository either.


Unfortunately, I cannot say exactly when these artifacts appeared, but my 
hunch would be somepoint around the shunning/rebuilding process?  Does 
this even make sense?!?


Worse, I am left not sure whether to simply shun the two random artifacts 
and then push the changes to the repository up to the server.  It has 
taken the best part of a day to get all these commits in and it represents 
about 6GB of source files (the repository has doubled from ~700 MB to ~1.5 
GB) requiring a lot of manual "fossil mv"s to synchronise many moved files 
(fossil doesn't yet support a git-like auto-mv-detection sadly).


Any idea how I can easily check the validity of my repository?  I have 
done the obvious which is to check out each of the recent check-ins and 
compare them with the original source folders, but what I don't know is 
whether the structure of my respository is damaged in some way...


Help :o)

Thanks!
Andy



___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


[fossil-users] Weird cross-contamination between two fossil repositories (and not even talking to server!)

2016-05-27 Thread Andy Gibbs

Hi,

I've just had a very, very odd experience with fossil.  I'm running version 
1.34.


Let me first explain what I have done.

I cloned a respository off our server.  I then went into the clone's web UI 
and disabled the auto-sync feature.  I then made 7 commits, the first of 
which caused a trunk fork which I then "repaired" by branching the old 
tip-of-trunk that caused the fork.  I then committed two commits to the new 
branch, then realised that I'd committed the wrong code, so shunned the last 
two commits, rebuilt the respository, unshunned the sha1 signatures, 
rebuilt, then recommitted the last two commits using the correct code.


Ok, a little complicated but nothing too out-of-the-ordinary, I hope.

Except ... now I have two utterly random commits in my cloned repository - 
both seemingly from the sqlite repository:


2014-12-09 [f66f7a17b7] Version 3.8.7.4 (user: drh, tags: release, 
version-3.8.7.4)

2011-05-19 [ed1da510a2] Version 3.7.6.3 release candidate 1. (user: drh)

Now, it is true that I have a clone of the sqlite respository on my server 
too ... but I haven't yet synced with the server.  Absolutely no server 
communication has happened since my initial clone.  And these odd artifacts 
were definitely not there (or, rephrased, definitely not showing) when I was 
mid-way through all of this work, and are not there on the server's copy of 
the repository either.


Unfortunately, I cannot say exactly when these artifacts appeared, but my 
hunch would be somepoint around the shunning/rebuilding process?  Does this 
even make sense?!?


Worse, I am left not sure whether to simply shun the two random artifacts 
and then push the changes to the repository up to the server.  It has taken 
the best part of a day to get all these commits in and it represents about 
6GB of source files (the repository has doubled from ~700 MB to ~1.5 GB) 
requiring a lot of manual "fossil mv"s to synchronise many moved files 
(fossil doesn't yet support a git-like auto-mv-detection sadly).


Any idea how I can easily check the validity of my repository?  I have done 
the obvious which is to check out each of the recent check-ins and compare 
them with the original source folders, but what I don't know is whether the 
structure of my respository is damaged in some way...


Help :o)

Thanks!
Andy

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users


[fossil-users] Rejected JSON tests

2016-05-27 Thread Kain Abel
Dear devs, dear users,

the script tester.tcl don't like the JSON support?.

lm@um:/tmp$ ./fossil version -v
This is fossil version 1.35 [893905c83e] 2016-05-23 01:05:08 UTC
Compiled on May 27 2016 12:36:50 using gcc-5.3.1 20160413 (64-bit)
SQLite 3.13.0 2016-05-18 10:57:30 fc49f556e4
Schema version 2015-01-24
zlib 1.2.8, loaded 1.2.8
SSL (OpenSSL 1.0.1t  3 May 2016)
TH1_DOCS
TH1_HOOKS
TCL (Tcl 8.6.0, loaded TH_OK: 8.6.5)
USE_TCL_STUBS
TCL_PRIVATE_STUBS
JSON (API 20120713)
UNICODE_COMMAND_LINE
DYNAMIC_BUILD
lm@um:/tmp$ tclsh /tmp/test/tester.tcl /tmp/fossil -quiet -prot
can't find package json
while executing
"package require json"
(file "/tmp/test/json.test" line 38)
invoked from within
"source $testdir/$testfile.test"
("foreach" body line 3)
invoked from within
"foreach testfile $argv {
  protOut "* $testfile **"
  source $testdir/$testfile.test
  protOut "* End of $testfile: [llength $bad_test] er..."
(file "/tmp/test/tester.tcl" line 716)
lm@um:/tmp$ tail prot
test glob-parse-116.1 OK
/tmp/fossil test-glob 'o*,two' one,two
test glob-parse-117.1 OK
/tmp/fossil test-glob {'o*,two three,four'} {one two three,four}
test glob-parse-118.1 OK
/tmp/fossil test-glob {'o*,two three,four'} {one,two three,four}
test glob-parse-119.1 OK
* End of glob: 0 errors so far **
* json **
/tmp/fossil test-th-eval {hasfeature json}
lm@um:/tmp$

With regards,
Kain


How the ./fossil was configured and built:
( Would it be possible to embed the flags of the compiler and also the
whole config line in the output of ./fossil version -v (or -vv)? - For
a better differentiation of different versions and a faster
reproduction of errors.)

lm@um:~/src/fossil$ uname -a
Linux um 4.4.0-22-generic #40-Ubuntu SMP Thu May 12 22:03:46 UTC 2016
x86_64 x86_64 x86_64 GNU/Linux
lm@um:~/src/fossil$ CFLAGS="-Os -pipe -Wall" ./configure --json
--with-openssl=tree --with-th1-hooks --with-th1-docs --with-tcl=1
--with-tcl-private-stubs=1
lm@um:~/src/fossil$ make clean && make 2> err4.log
lm@um:~/src/fossil$ cat err4.log
./src/import.c: In function ‘import_cmd’:
./src/import.c:1705:9: warning: ‘markfile_out’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
 fossil_fatal("cannot open %s for writing\n", markfile_out);
 ^
./src/import.c:1667:13: warning: ‘markfile_in’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
   FILE *f = fossil_fopen(markfile_in, "r");
 ^
./src/report.c: In function ‘rptview_page’:
./src/report.c:781:9: warning: ‘sState.zWikiEnd’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
 @ %s(pState->zWikiEnd)
 ^
./src/report.c:1188:25: note: ‘sState.zWikiEnd’ was declared here
 struct GenerateHTML sState;
 ^
./src/report.c:775:9: warning: ‘sState.zWikiStart’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
 @ 
 ^
./src/report.c:1188:25: note: ‘sState.zWikiStart’ was declared here
 struct GenerateHTML sState;
 ^
./src/report.c:779:9: warning: ‘sState.wikiFlags’ may be used
uninitialized in this function [-Wmaybe-uninitialized]
 wiki_convert(, 0, pState->wikiFlags);
 ^
./src/report.c:1188:25: note: ‘sState.wikiFlags’ was declared here
 struct GenerateHTML sState;
 ^
./src/search.c: In function ‘search_body_sqlfunc’:
./src/search.c:474:7: warning: ‘nHdr’ may be used uninitialized in
this function [-Wmaybe-uninitialized]
   int nHdr;
   ^
./src/search.c: In function ‘search_title_sqlfunc’:
./src/search.c:461:5: warning: ‘nHdr’ may be used uninitialized in
this function [-Wmaybe-uninitialized]
 sqlite3_result_text(context, z, nHdr, SQLITE_TRANSIENT);
 ^
./src/wiki.c: In function ‘wiki_cmd’:
./src/wiki.c:1333:27: warning: ‘rid’ may be used uninitialized in this
function [-Wmaybe-uninitialized]
 if( g.argv[2][1]=='r' && rid>0 ){
   ^
./src/cson_amalgamation.c: In function ‘cson_value_new_integer’:
./src/cson_amalgamation.c:2863:13: warning: dereferencing type-punned
pointer will break strict-aliasing rules [-Wstrict-aliasing]
 *CSON_INT(c) = v;
 ^
lm@um:~/src/fossil$
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users