Re: [fossil-users] Logout/PW/Email UX fix
Warren Young: > I’d then like to see /login change to something more generic like > /user. If someone depends on the existing name as an API, a /login > alias could be added, either in code or via the existing URL aliasing > feature. It's possible (and very convenient) to have a direct logout link: http://example.com/cgi-bin/fossil.cgi/repository/login?out Or, from (unversioned) wiki pages (with the Fossil Wiki markup flavor): [/login?out | Logout] I think this could also work for the Page Header edited through the Customize Skin interface. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] cgi_set_cookie()
It seems like the intention of the following call to cgi_set_cookie() is to generate a cookie to expire immediately, and direct the browser to remove it (at least this could make sense for login cookies): http://fossil-scm.org/index.html/artifact?name=4e7cdb5c6a=354-355 But in this case, cgi_set_cookie() creates a cookie with an empty value, and no "max-age" directive. Would it make sense to modify cgi_set_cookie() like this: lifetime<=0 → max-age=0 This seems compatible with all (6) calls to cgi_set_cookie(). It would no longer be possible to generate cookies without expiration information, but the call linked above is currently the only one to take this code path. Another approach may be to modify cgi_set_cookie() to automatically add "max-age=0" for any cookies with empty values, which could also work together with the modified "lifetime" check. Thanks, --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Delay with `fossil ui' (related to backoffice processing?)
Warren Young:
> You can open a file called NUL on Windows to get the same effect as
> /dev/null on POSIX type platforms.
A very interesting topic!
NULL
(void*)0, or nullptr (was used in a confusing manner, in my post)
\Device\Null
Windows NT (object manager) equivalent to /dev/null
\??\NUL
Windows NT global symlink to \Device\Null
Not accepted by Win32 APIs
\\.\NUL
\\?\NUL
Local Device and Root Local Device forms of \??\NUL
Works with Win32 APIs
(\\.\ and \\?\ both escape to \DosDevices, the latter without any path
name preprocessing, and on current versions of Windows NT, \DosDevices
is a is a symlink to \??, which is a virtual directory containing both
objects from the \GLOBAL?? and from the user session directories)
NUL
Win32 API magic file name mapped to \\.\NUL
\\?\C:\dev\NUL
Path to file C:\dev\NUL (must be specified as an absolute path)
Works with Win32 APIs
(\\?\ escapes to \DosDevices, which links to \??, where the symlink
named C: is found, pointing to \Device\HarddiskVolumeN, and finally
resolving to \Device\HarddiskVolumeN\dev\NUL)
> That’s no concern here, since the Fossil code will likely just say
> "NUL".
Yes, I think the Win32 variant should be:
(a) g.httpOut = fossil_fopen("NUL", "wb");
instead of:
(b) g.httpOut = fossil_fopen("/dev/null", "wb");
Code snippet (a) returns a valid (i.e. non-zero) FILE*, on Windows.
If the root drive of the fossil.exe current (working) directory has a
directory named "dev", say "C:\dev", code snippet (b) creates the file
"C:\dev\null", on Windows.
Just tested with fossil_open().
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Delay with `fossil ui' (related to backoffice processing?)
Richard Hipp: > Please test the latest trunk version and let me know if you are still > seeing issues. Thank you very much for the fix! I see no more delays with "fossil ui" on Windows. Just a thought: would it make sense to have --nodelay as a global option recognized by all commands, and maybe also through an environment variable, and a CGI control option? So that any backoffice processing could be temporarily disabled? This would probably require an explicit "backoffice" command, which could then also be set up to run as a task or cron job every few hours. This may be handy for fatal incidents that require "repository surgery", or restoring from backups, where you don't want any notifications to be sent before things are fixed. But maybe this is overengineering. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Delay with `fossil ui' (related to backoffice processing?)
Richard Hipp: > Hence, if the subprocess gets involves in backoffice work, that can > delay the return of content to the user. > The only comes up on Windows. I do not yet have a good work-around. I see, thank you very much for the detailed explanations. I only have the obvious (inefficient?) ideas, like using a command-line switch to direct the "HTTP renderer" to offload backoffice processing to another separate process, and exit immediately once the HTTP response is ready. Looking forward to see what will be your solution. I can always learn a lot of interesting things from the Fossil source code. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Delay with `fossil ui' (related to backoffice processing?)
> ... writing to a FILE* pointing to NULL may have exactly the same > effect as writing to a FILE* pointing to "/dev/null"? Well, unless there's a directory named "dev" in the root of the current drive, which would result in creation (due to the passed "wb" mode) of a file named "null", on Windows. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Delay with `fossil ui' (related to backoffice processing?)
Thank very much for your feedback! Warren Young: > Which is it? With the random appearance of the delay, and the long duration of 60 seconds, I eventually gave up on "bisecting". But with a more systematic approach, I've now got: Last good: [06507038] First bad: [947081aa] > If the problem first appeared with [76800769] ... Backing out [76800769] (and the dependent [d216ea9a]) does not fix the delay. > Also, does this behavior happen on only one machine? I can also see it on a Windows 7 (32-bit) VM. Here's what I've found out in the meantime: Waiting for 60 seconds for the delay to time out, until the default /timeline web page is loaded in the browser, instead of terminating the server with Ctrl+C, will cause another 60 seconds delay for any links clicked in the web ui. Reloading the delayed /timeline web page with F5, before the 60 seconds have elapsed, gives a speedy refresh, with no delay, so multiple requests within the 60 seconds period are not delayed. Waiting for 60 seconds before running `fossil ui' after the last server was terminated, causes the delay to always happen. The observed delay of 60 seconds seems somehow related to: [5544931c] src/backoffice.c:59 #define BKOFCE_LEASE_TIME 60 /* Length of lease validity */ On Windows, the DbgView utility is useful to trace messages from OutputDebugStringA(), handy if there's no attached console, and easier than attaching WinDbg to spawned sub-processes: https://live.sysinternals.com/Dbgview.exe By adding calls to OutputDebugStringA() before and after the following line of source code, I found that one of the fossil.exe processes seems to be stuck here during the delay: [5544931c] src/backoffice.c:240 sqlite3_sleep(1000*(x.tmCurrent - tmNow + 1)); This code path is only entered if the delay becomes manifest. I don't understand all the details of the backoffice processing code, yet, in particular not the condition to trigger the sleep. But the sleep seems to block the web ui, somehow? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Delay with `fossil ui' (related to backoffice processing?)
I've noticed something with the current tip version of Fossil (on Windows): I see random delays for the `fossil ui' command, i.e. the web browser is opened at once, but is displaying the loading/busy indicator and a blank page for some time, before the default web page finally appears: http://localhost:8080/timeline?c=current When repeatedly entering `fossil ui' from the command prompt, followed by immediately closing the web browser as soon as the /timeline web page is displayed, then hitting Ctrl+C to stop the Fossil web server, and restarting the cycle by running `fossil ui' again, the delay seems to appear with every 5th to 10th cycle. During the delay, I can see two fossil.exe processes with `tasklist | findstr fossil', whereas with `fossil ui' without the delay, there's only one fossil.exe process. The %TEMP% directory has 3 or more files appearing instantaneously at the beginning of the delay, named fossil_server_P8080_0N_xxx.txt, with N being low numbers, and xxx being 'in', 'out', and 'cmd', containing the HTTP request and response, and a file/IP address list. When terminating the delayed `fossil ui' server (responds to Ctrl+C) and immediately relaunching it, it is as fast as usual, without any delay. The delay takes approximately 60 seconds, and after that the HTTP request completes normally with the display of the /timeline web page, and the temporary files are cleared. I tried to find a "last good" version by means of "bisecting", but unfortunately no success, so far. All I can say is that the delay never appears with Fossil version 2.6 [9718f3b078] 2018-05-04 12:56:42 UTC, the binary for Windows downloaded from the Fossil web site. My tests were on Windows 10.0.17134.167 (64-bit), with MSVC 19.14.26430 (32-bit). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
Richard Hipp: > I think this problem has been addressed in a more general way > on the latest trunk. Please let me know if you find otherwise. This works fine (tested only on Windows, so far), thank you very much! --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
> There's been some changes to fossil_exit() in the meantime, I'll check > these, and report back here. I was wrong, the changes were to fossil_fatal() and fossil_panic(), and not to fossil_exit(). The current tip version of Fossil still exhibits the behavior summarized here: https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg27269.html --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
Warren Young: > Quantify “a lot.” I have some rarely committed-to but frequently web-accessed repositories (with login), and I see daily backups of the modified repository database, even though I'm sure I haven't committed anything. It's like "hey, what's going on there with my babies?" everytime, but I need to get used to it. > I’d find out why the DB client is dying early and fix that, so that > the WAL ends up being deleted entirely upon a clean DB shutdown. I think I found it: https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg27269.html There's a call to fossil_exit() from within a db_step()...db_finalize() block, and calling fossil_exit() only after db_finalize() fixed it. There's been some changes to fossil_exit() in the meantime, I'll check these, and report back here. > I find it odd that some people get so itchy over DB concurrency and > such with Fossil when highly active projects might have 40 or so > commits per day. I'm not worried by this. Stephan just wondered if a shared cookie database may be prone to locks contention, if I got him right. I'd assume the main bottleneck to be high-frequency, read-only, no-login web access (for a renowned project), in which case the cookie database doesn't even need to be attached, and not the frequency of commits. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
Just one more thought: Copying database files (vs. cloning) also includes any hand-made meta changes, for example I remember adjusting the page size and journal mode for older repositories, when the defaults for new Fossil repositories were changed. Of course `fossil rebuild --wal' after the sync can help with things like these, but the database file checksum will definitely change and trigger a complete backup, for the rebuilt repository. I think I need to come away from my traditional "copy a file and get exactly what you had" way of thinking ... --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
Richard Hipp: > ... create your backups by cloning and syncing ... Thank you for your comments. I see, this completely makes sense. The process of "restoring" a repository from backup would include copying database files, as syncing from backup → original might not work if something's gone awry with the original. My main concern here is that the cloned backup really includes everything from the original (configuration, etc.). But hearing again (haven't you already outlined the "cloning as backup strategy" recently, on this list?) that it works for the experts should give me the faith to trust it. Backing up "hot" databases is currently not a concern with my private, traditional-style CGI-served repositories. I would like to have some "rotating" backup, with a way to go back certain steps with the complete repository, i.e. day-by-day, for up to one week, so I could catch the "last good" if I notice something wrong. Copying and replacing duplicate files with hard-links is an extremely straight forward and space efficient process to achieve this. I will try the same with cloning new (some extra logic required) and syncing existing repositories. But it may not be possible to detect unchanged / duplicate repository database files, like this, as some internally stored last sync or URL last access time stamps might always result in a different database file, I assume. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Backup traffic
Stephan Beal:
> .. i'm not sure that i like it enough to justify the idea of
> maintaining two files where one file is sufficient.
The current implementation uses one single cookie database shared for
all repositories in the same directory, which can be excluded from
backups, and deleted (or, better, emptied by SQL script) to have
everybody logged off. But it's possible to modify the code to use one
single cookie database per system, or per repository.
> .. the login cookie db could become a point of locking contention ...
Would WAL mode prevent this, mostly?
Below are my current "works for me on Windows and FreeBSD" patches. I
hope we still have the same definitions of "surprisingly simple" :)
The 2nd patch is only required with my previous patch to change ETag
generation to produce a "login-time-sensitive" hash.
I'm sorry e-mail processing may insert one or two unwanted line breaks
after column 72, as Fossil seems to use a source code line length
limit of 80 chars.
Some notes are included directly with the patch file headers, but I'd
like to emphasize that I haven't bothered making things work with
login groups, so far.
I'd be happy to do more work towards a more generalized "separate
(shared) database for non-repository contents, such as 'volatile' or
'system-specific' state information" approach, should this be
considered interesting for Fossil.
--Florian
= Patch for Fossil [e08f9c04] ==
Baseline: Fossil [e08f9c0423]
Proof-of-concept to outsource login cookie information to a separate
database named "cookiestore", saved as "fossil-cookiestore.sqlite" in
the directory of the main repository database, and attached on demand.
The "cookiestore" database is left attached until shutdown; it may be
safer to have it detached explicitly as soon as possible.
HTTP cache handlers, and any other code relying on "user.cexpire", must
query "cookiestore.user.cexpire", instead.
Support to share login credentials across login groups is not
implemented by this patch; in fact, this may even break login group
features.
Admins changing their own password through the /setup_uedit page (not
through the /login page) are no longer logged out automatically.
To prevent writes to the main repository database caused by read-only
web server access, the "PRAGMA optimize" call needs to be removed, and
the "access_log" feature needs to be disabled (the logs could be
recorded to a plain text file, or outsourced to a separate database, if
required).
There may be more elegant SQL queries to work with the connected tables,
either by using JOINs, or FOREIGN KEYs (yet the latter have been
disabled by Fossil).
Windows batch file to dump or tweak the "cookiestore" database:
:: @echo off
:: setlocal
:: set c=fossil-cookiestore.sqlite
:: if not exist "%c%" goto:eof
:: (
:: echo ATTACH '%c%' AS 'c';
:: echo -- PRAGMA c.journal_mode;
:: echo -- PRAGMA c.page_size;
:: echo -- PRAGMA c.auto_vacuum;
:: echo SELECT * FROM c.user;
:: echo -- UPDATE c.user SET cexpire=0;
:: ) | fossil sql --no-repository
Index: src/login.c
==
--- src/login.c
+++ src/login.c
@@ -143,10 +143,53 @@
*/
static char *abbreviated_project_code(const char *zFullCode){
return mprintf("%.16s", zFullCode);
}
+/*
+** Attach the fossil-cookiestore.sqlite db to store login cookies.
+*/
+void attach_cookiestore()
+{
+ static int attached_cookiestore = 0;
+ char *zDBName;
+ Blob bDBFullName;
+ char *zProjCode;
+
+ if (attached_cookiestore) return;
+
+ zDBName = mprintf("%s/../fossil-cookiestore.sqlite",g.zRepositoryName);
+ file_canonical_name(zDBName,,0);
+ sqlite3_free(zDBName);
+ db_attach(blob_str(),"cookiestore");
+ blob_reset();
+
+ /* Initialize */
+ db_multi_exec(
+"CREATE TABLE IF NOT EXISTS cookiestore.user( "
+"repo TEXT, uid INTEGER, login TEXT, "
+"cookie TEXT, ipaddr TEXT, cexpire DATETIME,"
+"PRIMARY KEY (repo, uid), "
+"UNIQUE (repo, uid, login) ON CONFLICT REPLACE );");
+ /* Clear expired cookies */
+ zProjCode = db_get("project-code",NULL);
+ db_multi_exec(
+"DELETE FROM cookiestore.user WHERE "
+"repo=%Q AND cexpire 0) && "Invalid user data.");
+ attach_cookiestore();
zHash = db_text(0,
- "SELECT cookie FROM user"
- " WHERE uid=%d"
+ "SELECT cookie FROM cookiestore.user"
+ " WHERE repo=%Q AND uid=%d"
" AND ipaddr=%Q"
" AND cexpire>julianday('now')"
" AND length(cookie)>30",
- uid, zRemoteAddr);
+ zProjCode, uid, zRemoteAddr);
if( zHash==0 ) zHash = db_text(0, "SELECT hex(randomblob(25))");
zCookie = login_gen_user_cookie_value(zUsername, zHash);
cgi_set_cookie(zCookieName, zCookie, login_cookie_path(), expires);
record_login_attempt(zUsername, zIpAddr, 1);
db_multi_exec(
-"UPDATE user SET cookie=%Q, ipaddr=%Q, "
-" cexpire=julianday('now')+%d/86400.0 WHERE uid=%d",
-
[fossil-users] Backup traffic
As much as I like the simplicity of keeping the full repository history in one single SQLite database, I see a minor downside. There's a lot of backup traffic, if "non-contents changes" (such as the admin and user logs, the login cookies, but also the "PRAGMA optimize" information) are causing updates to the repository database, marking it dirty for the next backup cycle. (This was also the main reason for my complaining about the leftover WAL and SHM files, recently, which accumulated in my backup logs. Because in the end, WAL and SHM have to be kept together with the SQLite database, as they might contain valuable information?) From peeking at the Fossil timeline, my question is, will the new "backoffice processing" cause even more frequent updates to the main repository database, i.e. with the pids stored in the configuration table, and updated after each web page display? I have created a (surprisingly simple) patch to attach a separate login cookie database (shared among all repositories in the same directory), so that plain login and logout actions will no longer cause repository database writes. With admin and user logs turned off, and "PRAGMA optimize" removed, the repository database is only touched if there's new contents, or new configuration settings. What's your comments to this? Does anybody care about the repository database, holding all your valuable contents, being modified frequently with simple non-contents state information? Given the reliability of SQLite, we probably shouldn't care. But what is a good strategy to minimize backup traffic, if repository databases change that often? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] A fossil library
David Mason: > Fossil appears to be careful with memory allocation too, with very few > raw calls to malloc, so memory allocations can be unwound. SQLite has the "Zero-malloc" and "Application-supplied memory allocators" options [0], which may be helpful for cases without proper db engine shutdown? On the other hand, at least the first option may not be a good choice for Fossil's heavy-lifting work? [0] https://www.sqlite.org/malloc.html Also, I've recently come across an article to solve a problem with memory fragmentation by using private heaps [1]. This way, the library could simply dispose the entire heap when done, and also "have the system do the cleanup". Or is this too heavy-weight for a library, or not supported on all systems? [1] https://blogs.msdn.microsoft.com/ricom/2006/02/02/unmanaged-memory-fragmentation-an-old-story/ --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] No rule to make target 'src/email.c', ...
Richard Hipp: > It is reassuring to know that so many people routinely build Fossil > from the trunk sources :-) À propos build: I have accidentally set FOSSIL_ENABLE_MINIZ=1 (in win/Makefile.msc), and this doesn't seem to work, probably because src/shell.c includes zlib.h without examining FOSSIL_ENABLE_MINIZ. Not sure if this is important, or if it's even possible for src/shell.c to support FOSSIL_ENABLE_MINIZ, but I'd rather let you know. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Using bundles for pull requests
Richard Hipp: > (5) The server accepts the bundle and parks it in a separate holding > table, but does not merge it or otherwise make it available to average > passers by. The server then sends email notifications to developers > with appropriate privileges to let them know that a pull request has > arrived. > (6) Developers who receive notification of the pull request can run a > command that pulls down the bundle and applies it as a private branch > on their own personal clones of the repo. Developers can then either > approve of the pull request by publishing it (marking it non-private) > and pushing it back to the server. Or they can reject the pull request > which erases it from their clone. They might also cause the pull > request to be erased from the holding table on the server. From my experience with GitHub pull requests, it seems that they are visible to the public and open for review and comments even before they are reviewed and accepted by the upstream project, and anybody interested can take the submitted code to their own clone. It has a less "open" character if core developers need to become active to make pull requests visible to the public, more so if they can delete submissions without anybody from outside of their team ever noticing. If bundles were reworked to be "sets of patches", they could me made visible to the public more easily, before reaching the main repository store. And there would be the already mentioned greater flexibility with meta-data (allowing the receiver to decide about the merge base, the check-in comments, as well as the branch name and other tags). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Using bundles for pull requests
Another issue with bundles: "push requests" derived from local private branches. It may be not only the receiver of a "push request" to decide to import only the final "accumulated patch", without all the detailed "commit-by-commit" history. It may also be the submitter to decide to keep the development history for a "push request" in a local private branch, and only submit the final "accumulated patch", without the full history of all the trials, reverts, and minor coding style and typo fixes. So, the intuitive way for this would be to merge their local private branch into a new branch derived from the same root, and export the new branch to a bundle with the "--standalone" option (the "push request" feature will probably do this, by default). But importing this bundle into the upstream repository, lacking all the knowledge of the local private branch, may have undesired effects. In one of my tests, the /bloblist showed an "unknown" artifact of 0 bytes, with the UUID being one of those listed in the P-card of the imported check-in. Also, not sure if this might lead to problems with future merges, if the parents and merge ancestors referenced by the imported check-in cannot be found in the upstream repository. Bundles are not patches that can be applied freely and flexibly, but tend to rely on the full repository history. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Using bundles for pull requests
More thoughts: > fossil merge --bundle BUNDLE ?--cherrypick|--backout CHECK-IN? * BUNDLE could be a disk file, or something like "bundle:ID" to refer to a bundle submitted via "push request", and queued in a separate repository db table. * If the current check-out is different from the root of the branch contained in the bundle, `fossil merge --bundle' could issue a warning (even if there's no merge conflicts). * The established work-flows for code review and testing could be applied, before committing the merged-in bundle, and `fossil revert' could avoid potential havoc before anything goes to the repository store. > ... it's possible to introduce unwanted contents (and random > control artifacts) ... For example, it's possible to foist control artifacts to modify the comments of random commits already in the repository, or to rename or hide the "trunk". This may not be too easy to detect, if the modified check-ins are out of sight with usual timeline view limits, and control artifacts can also have their time stamps forged by "--date-override", or they can even be hand-crafted and picked up by `fossil reconstruct', and show up far away from their referenced check-ins. Or, if bundles would only allow control artifacts referencing check-ins from that same bundle, multiple serial changes to comments could mask unwanted contents to be sneaked into the repository. This case would be easier to shun, but detailed review of the (possibly complex) meta-data of any bundles is essential. (With the current implementation of bundles, it's also possible to import check-in manifests referencing user names not present in the target repository.) It seems to me that accepting complete bundles from untrusted sources goes near granting them commit access. In order to be merging of "push requests" a task of code review and testing, instead of tedious meta-data checks, I think there should be a way to get "just the code". --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Hydra registrations open (open beta)
Eduard: > I have opened registration on Hydra. Feel free to try it out at > [ https://hydra.ecd.space/ ], and give me your feedback. Roy Keene: > Is ChiselApp not serious enough for you ? (Quotes unconnected.) I think both ChiselApp and Hydra are very nice, and technically solid projects (and I have learned how to use PHP's PDO::SQLite from the ChiselApp source code, so thanks for this, too). I'm sure I would use one of them (or even both), were it not that easy to run your own Fossil web server, even on cheap shared hosting systems. An important point to motivate people to use these services: is it possible to move existing projects there, or is it just possible to create new (empty) repositories? Also, when it comes to attracting people on the lookout for GitHub alternatives, I believe it's important to have a "landing page" (note the non-dinosaur slang) to show the occasional visitor a list of repositories with "real-world" projects under active maintenance, and with concise descriptions. At least on chiselapp.com, the "Public Repositories" page shows (on first impression) a list of "cryptic" repository and user names, and randomly clicking some links mostly leads to empty, abandoned or one-time-testing repositories. Such a "landing page" may need to be hand-crafted, but I think it's worth the effort to have occasional visitors stay on site, browse around, and get familiar with the capabilities of Fossil. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Using bundles for pull requests
> Let me decide whether the bundle is committed "step-by-step" > ... or "all-at-once" ... I think this is what I'm looking for: fossil merge --bundle BUNDLE ?--cherrypick|--backout CHECK-IN? > But I would like to have some more flexibility and control when > "applying" (or importing) bundles by third parties: Some may call me a "dinosaur", but plain old patches offer all of this flexibility. And the patch utility is able handle differences in white space and line endings, and can even do inexact/fuzzy matching (yet, I'm sure the Fossil internal patch routines also have all this functionality). The point is, by allowing (potentially untrusted) third parties to submit complete bundles to be committed directly, it's possible to introduce unwanted contents (and random control artifacts), which may not be so easy to be shunned (as, for example, the check-in comments already mentioned, persisting in the "Original Comment:" field). This will require very detailed examination of third party "meta-patch" information. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Using bundles for pull requests
Fossil bundles are a great feature, and may be useful to implement pull (or "push") requests. But I would like to have some more flexibility and control when "applying" (or importing) bundles by third parties: * Let me decide where the bundle is committed, be it directly on trunk instead of on a separate branch, or on a different base than initially used to create the bundle (with notifications about merge conflicts, if any). * Let me decide whether the bundle is committed "step-by-step", as in the branch contained in the bundle, or "all-at-once", like an "accumulated patch", which may already be enough history to record for the upstream project (the submitted original bundle could still be archived separately, if required). * Allow me to make additional changes to the "accumulated patch" prior to committing, for minor "on-the-fly" fixes to comments and coding style. * Let me edit the check-in comments, branches and tags myself, so they can have some "standard format" for contributed contents, such as: "Patch to achieve ... contributed by ..." (tags: contribution) I'm aware that check-in comments an tags can be edited later, but even then, the original comments contained in the bundle will still be recorded, and they may contain e-mail addresses, trade secrets, passwords, or other confidential information, if third parties are allowed to introduce comments into my repositories. And rejecting a submitted bundle solely due to a check-in comment causes at least some extra effort for either side, as merely editing the comment and re-exporting the bundle won't remove the relevant comment from the history. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Incorrect arrow colors when using -bgcolor
Andy Goth:
> In my test case, the outbound arrows are white, which doesn't look
> so great against the default white background.
The timeline arrow color generated from background color #00aa00 was
not white, but #ffd6, which is an invalid HTML/CSS color value,
and happened to be displayed as white by your web browser.
Richard Hipp:
> Fixed on trunk
With the fix committed as [7ac88481a6] [0], timeline arrow colors have
been turned off completely, and all timeline arrows are now displayed
with black color, also for skins with dark background colors, see for
example "xekri" [1].
[0] http://fossil-scm.org/index.html/info/7ac88481a6
[1] http://fossil-scm.org/skins/xekri/timeline
The problem is that the function to generate the timeline arrow colors
from background colors can have an "unsigned integer underflow",
resulting in invalid HTML/CSS color values.
I have suggested a modification for this, a while ago. With the
modification, the generated timeline arrow color for background color
#00aa00 would be #008000, and not #ffd6.
I really liked the timeline arrow colors, they were very helpful to
more easily distinguish the course of different timeline rails.
Please allow me to encourage you to have a look at my modifications,
and implement something similar, to bring back the beloved timeline
arrow colors.
Thank you very much
--Florian
= Patch for Fossil [4bbd5c3e] ==
Prevent unsigned integer underflow in the code to calculate slightly
darker timeline rail colors derived from branch background colors,
resulting in invalid output for the HTML/Javascript timeline drawing
code. Example: branch color #ff00ff results in invalid timeline rail
color #80ff without the fix, and in #800080 with the patch applied.
Index: src/timeline.c
==
--- src/timeline.c
+++ src/timeline.c
@@ -651,11 +651,14 @@
static const unsigned int t = 215;
if( mxt ) for(i=0; i<3; i++) x[i] -= mx - t;
+if( mx>t ) for(i=0; i<3; i++){
+ if( x[i]>=mx - t ) x[i] -= mx - t;
+ else x[i] = 0;
+}
}
sqlite3_snprintf(sizeof(zRes),zRes,"#%02x%02x%02x",x[0],x[1],x[2]);
return zRes;
}
= Patch for Fossil [4bbd5c3e] ==
= Patch for Fossil [4bbd5c3e] ==
Check if branch background colors are in correct HTML/CSS hexadecimal
RGB representation before calculating the derived timeline rail colors,
and also allow color values in shorthand hex triplet notation #HHH.
Index: src/timeline.c
==
--- src/timeline.c
+++ src/timeline.c
@@ -631,21 +631,33 @@
/*
** Change the RGB background color given in the argument in a foreground
** color with the same hue.
*/
static const char *bg_to_fg(const char *zIn){
+ int nIn; /* length of zIn */
int i;
unsigned int x[3];
unsigned int mx = 0;
static int whiteFg = -1;
static char zRes[10];
- if( strlen(zIn)!=7 || zIn[0]!='#' ) return zIn;
+ nIn = strlen( zIn );
+ if(( nIn!=7 && nIn!=4 ) || zIn[0]!='#' ) return zIn;
+ for(i=1; imx ) mx = x[i];
+ if( nIn==4 ){
+for(i=0; i<3; i++){
+ x[i] = hex_digit_value(zIn[0])*17;
+ zIn++;
+ if( x[i]>mx ) mx = x[i];
+}
+ }else{
+for(i=0; i<3; i++){
+ x[i] = hex_digit_value(zIn[0])*16 + hex_digit_value(zIn[1]);
+ zIn += 2;
+ if( x[i]>mx ) mx = x[i];
+}
}
if( whiteFg<0 ) whiteFg = skin_detail_boolean("white-foreground");
if( whiteFg ){
/* Make the color lighter */
static const unsigned int t = 215;
= Patch for Fossil [4bbd5c3e] ==
= Patch for Fossil [4bbd5c3e] ==
Return default timeline rail color values #00 (for white background)
or #ff (for black background) for all non-hex triplet branch color
values. Note that named color values (such as 'red') and color values in
functional notation (such as 'rgba(255,128,64,0.8)') are not affected,
as only branch colors starting with '#' are passed to bg_to_fg() to
calculate derived rail colors, in the first place. So this patch only
filters color values with bad hexadecimal RGB representation, such as
'#12345' (invalid length) or '#z12345' (invalid hexadecimal character).
Index: src/timeline.c
==
--- src/timeline.c
+++ src/timeline.c
@@ -637,14 +637,15 @@
int i;
unsigned int x[3];
unsigned int mx = 0;
static int whiteFg = -1;
static char zRes[10];
+ if( whiteFg<0 ) whiteFg = skin_detail_boolean("white-foreground");
nIn = strlen( zIn );
- if(( nIn!=7 && nIn!=4 ) || zIn[0]!='#' ) return zIn;
+ if(( nIn!=7 && nIn!=4 ) || zIn[0]!='#' ) goto default_fg;
for(i=1; imx ) mx = x[i];
}
}
- if( whiteFg<0 ) whiteFg = skin_detail_boolean("white-foreground");
if( whiteFg ){
/* Make the
Re: [fossil-users] Fossil binary file size on FreeBSD (amd64)
Stephan Beal: > Try stripping the debug symbols: > strip fossil Thank you for the useful tip, now down to 3.6 MB! My MSVC/WinSDK toolset doesn't provide the equivalent utilities any more, as everything is controlled by compiler and linker flags -- and obviously for a longer time than my long-term memory span :) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] HTTP caching, again
Though I'm aware that this is not something you may consider useful
for Fossil, I'm posting a minor update, for the sake of correctness:
(0) Wrong Statement
Me:
> I think that "Vary: Cookie" is intended to work with unconditional
> HTTP requests: the browser is directed to stick to the expiry date
> and use the cached page, unless the cookies have changed.
I'm sorry I was wrong here.
"Vary: Cookie" directs clients to include cookies as part of their
"cache key", and it also works with conditional HTTP requests.
But if clients mark resources as stale due to cookie updates, they
will still "revalidate" them with conditional If-None-Match requests,
using their last-received ETag. And if the server does not consult all
the information from the cookies to generate its ETag (here, the user
login time), it won't produce a new ETag and expire the page, but
instead reply with "304 Not Modified".
(1) Introduced two Bugs
With the previous version of the patch, accessing the /uv page with no
file name specified caused an assertion failure, as doc_page()
repeatedly tries various index documents (index.html, index.wiki,
index.md, and 404.md). So the cache checks are only performed if
there's a valid hash (i.e. if "SELECT hash FROM unversioned" returns
non-empty data). Moreover, the Last-Modified response header was not
sent, if an ETag cache hit was already handled (though, at least my
Apache web server seems to purge the Last-Modified header from "304
Not Modified" responses).
(2) Updated If-Modified-Since checks
The updated patch also includes the modifications to the
If-Modified-Since checks, as suggested in a separate post.
--Florian
= Patch for Fossil [a7056e64] ==
Index: src/cgi.c
==
--- src/cgi.c
+++ src/cgi.c
@@ -249,11 +249,11 @@
iReplyStatus = 200;
zReplyStatus = "OK";
}
if( g.fullHttpReply ){
-fprintf(g.httpOut, "HTTP/1.0 %d %s\r\n", iReplyStatus, zReplyStatus);
+fprintf(g.httpOut, "HTTP/1.1 %d %s\r\n", iReplyStatus, zReplyStatus);
fprintf(g.httpOut, "Date: %s\r\n", cgi_rfc822_datestamp(time(0)));
fprintf(g.httpOut, "Connection: close\r\n");
fprintf(g.httpOut, "X-UA-Compatible: IE=edge\r\n");
}else{
fprintf(g.httpOut, "Status: %d %s\r\n", iReplyStatus, zReplyStatus);
@@ -269,10 +269,11 @@
fprintf(g.httpOut, "Cache-control: no-cache\r\n");
}
if( etag_mtime()>0 ){
fprintf(g.httpOut, "Last-Modified: %s\r\n",
cgi_rfc822_datestamp(etag_mtime()));
+fprintf(g.httpOut, "Vary: Cookie\r\n"); /* HTTP/1.0 (no ETags) */
}
if( blob_size()>0 ){
fprintf(g.httpOut, "%s", blob_buffer());
}
Index: src/doc.c
==
--- src/doc.c
+++ src/doc.c
@@ -641,17 +641,29 @@
}
}
if( isUV ){
if( db_table_exists("repository","unversioned") ){
Stmt q;
+char *zHash=0;
+sqlite3_int64 mtime=0;
db_prepare(, "SELECT hash, mtime FROM unversioned"
" WHERE name=%Q", zName);
if( db_step()==SQLITE_ROW ){
- etag_check(ETAG_HASH, db_column_text(,0));
- etag_last_modified(db_column_int64(,1));
+ zHash = fossil_strdup(db_column_text(,0));
+ mtime = db_column_int64(,1);
}
db_finalize();
+if( zHash ){
+ /* Only call etag_check() if the unversioned file was found
+ ** and has a valid hash, as doc_page() is called repeatedly
+ ** to search for index documents (index.html, index.wiki,
+ ** index.md, and 404.md), causing an assertion failure in
+ ** etag_check(), due to zETag already initialized. */
+ if( zHash[0] )
+etag_check(ETAG_HASH|ETAG_CEXP, zHash, mtime);
+ free(zHash);
+}
if( unversioned_content(zName, )==0 ){
rid = 1;
zDfltTitle = zName;
}
}
@@ -847,11 +859,11 @@
*/
void logo_page(void){
Blob logo;
char *zMime;
- etag_check(ETAG_CONFIG, 0);
+ etag_check(ETAG_CONFIG, 0, 0);
zMime = db_get("logo-mimetype", "image/gif");
blob_zero();
db_blob(, "SELECT value FROM config WHERE name='logo-image'");
if( blob_size()==0 ){
blob_init(, (char*)aLogo, sizeof(aLogo));
@@ -881,11 +893,11 @@
*/
void background_page(void){
Blob bgimg;
char *zMime;
- etag_check(ETAG_CONFIG, 0);
+ etag_check(ETAG_CONFIG, 0, 0);
zMime = db_get("background-mimetype", "image/gif");
blob_zero();
db_blob(, "SELECT value FROM config WHERE name='background-image'");
if( blob_size()==0 ){
blob_init(, (char*)aBackground, sizeof(aBackground));
Index: src/etag.c
==
--- src/etag.c
+++ src/etag.c
@@ -24,10 +24,11 @@
** (1) The mtime on the Fossil executable
** (2) The last change to the CONFIG table
** (3) The last change
[fossil-users] Fossil binary file size on FreeBSD (amd64)
When building Fossil on a recent out-of-the-box FreeBSD (amd64) virtual machine, the resulting binary file is about 10.1 MB. The binary files downloaded form [0] are at most 3.8 MB. [0] http://pkg.freebsd.org/ I have tried various ./configure options, FOSSIL_DYNAMIC_BUILD is enabled (=1), and TCL seems to be disabled by default (or, is it?). Not that it matters (these times are over), but out of curiosity, why is there such a difference? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Reversed If-Modified-Since checks
As `wget -N' adds 1 second to the If-Modified-Since stamp derived from
existing local files (maybe due to different time stamp resolutions on
various file systems), I noticed that the If-Modified-Since stamp
checks seem to give "reversed" results.
The following test commands display the HTTP response headers for the
current Fossil Source Tarball. Requests are sent with
If-Modified-Since stamps matching the time stamp of the unversioned
file ("Fri, 4 May 2018 13:06:49 GMT", at the time of this writing), as
well as 1 second earlier, and 1 second later.
(0) Matching If-Modified-Since stamp (equal to the file mtime),
returns "304 Not Modified", as expected:
wget --spider -d --header "If-Modified-Since: Fri, 4 May 2018 13:06:49
GMT" "http://www.fossil-scm.org/index.html/uv/fossil-src-2.6.tar.gz;
(1) Expired If-Modified-Since stamp (older than the file mtime),
erroneously returns "304 Not Modified", but should return "200 OK":
wget --spider -d --header "If-Modified-Since: Fri, 4 May 2018 13:06:48
GMT" "http://www.fossil-scm.org/index.html/uv/fossil-src-2.6.tar.gz;
(2) Valid If-Modified-Since stamp (more recent than the file mtime),
returns "200 OK", but should return "304 Not Modified":
wget --spider -d --header "If-Modified-Since: Fri, 4 May 2018 13:06:50
GMT" "http://www.fossil-scm.org/index.html/uv/fossil-src-2.6.tar.gz;
This "reversed" caching behavior seems to origin from the following test:
http://www.fossil-scm.org/index.html/artifact?name=0afe42f3bd=152
The second comparison should probably be changed from:
if( x<=0 || x>mtime ) return;
to:
if( x<=0 || xhttp://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] File History: "Classic View" == "Modern View"?
For the File History web page, it looks like "Classic View" is the same as "Modern View": http://fossil-scm.org/index.html/finfo?name=VERSION=x --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] HTTP caching, again
Now I see why the clever and elegant solution to use "Vary: Cookie", as suggested by Joerg, does not fix /uv page expiration after login and logout, and I can also explain the strange differences between the local Fossil built-in web server, and my remote web server. The local Fossil built-in web server uses the HTTP/1.0 protocol. On my remote web server, Apache automatically upgrades the CGI responses generated by Fossil to HTTP/1.1. HTTP/1.0 does not yet support ETags, but only Last-Modified stamps, and thus web browsers do not include If-None-Match with their requests, but just stick to If-Modified-Since [0]. Interestingly, "Vary: Cookie" works (i.e. /uv pages are expired after login and logout) with Chrome, and looks like it's working with IE and Edge (but in fact they are not caching pages with "Vary: Cookie" at all, mimicking a correct refresh triggered by a cookie update; it may be my browser settings). [0] https://stackoverflow.com/a/28033770 With HTTP/1.1, browsers include If-None-Match with their requests, but "Vary: Cookie" no longer has any effects (i.e. /uv pages are not expired after login and logout). I think that "Vary: Cookie" is intended to work with unconditional HTTP requests: the browser is directed to stick to the expiry date and use the cached page, unless the cookies have changed. But caching works differently with conditional HTTP requests (If-None-Match, If-Modified-Since): pages are always revalidated with the server, whether or not the cookies have changed, and "Vary: Cookie" has no additional effects, here. I've attached a simple PHP script to test this: The script generates two web pages, one to expire after 10 seconds (through a "Cache-Control: max-age=10" HTTP response header), the other to handle a conditional request (through a "Cache-Control: must-revalidate, private" header) that always returns "304 Not Modified" (to simulate the current Fossil "Last-Modified always wins" caching behavior). Repeatedly clicking the "Reload" links causes the first page to refresh every 10 seconds (watch the "Date" and "Cookie" entries). The second page remains unchanged (unless reloaded with Ctrl+F5). Clicking "Update cookie" modifies the test cookie (by JavaScript). Now the first page is refreshed immediately - the effect of "Vary: Cookie". The second page still remains unchanged. That's why /uv pages are not expired after login and logout, even if the login cookie has changed. The browser always revalidates the page, and whether or not Fossil detects an ETag mismatch after login and logout (currently, it doesn't, as the ETag is not "login-time-sensitive"), it is immediately undermined by a Last-Modified match. "Vary: Cookie" can't fix this. I have been using Fossil with the patch to expire /uv web pages whenever a new user is logged-in for a few days, now. With the repository index page set to a /uv page [1], I have a very smooth user experience for login and logout actions: After login, I can immediately see the Admin menu entry and the user name display in the top right corner, without the need to do a "hard" reload (Ctrl+F5) -- exactly the way it was before Fossil supported HTTP caching. My index page has a direct logout link, and when clicked, the Admin menu entry and user name display are again updated immediately: [/login?out | Logout] If the user login state does not change, Fossil sends a "304 Not Modified" response, and the web browser shows the cached page, with the Admin menu entry and the user name display in sync. If there's future plans to use HTTP caching not only for /uv, but also for /doc and /wiki pages, more people may run into the issue that they need to do "hard" reloads after login and logout. I have refactored the patch (attached) to have one single function handle either conditional request, and hide the logic to ensure that ETag mismatches won't be undermined by Last-Modified matches, so it's easier to reuse it for other page generators than /uv, at a later stage. I'm not sure if it's safe to change HTTP/1.0 to HTTP/1.1 for the local Fossil built-in web server (I think it is, as it's likely that it is silently upgraded by most web servers). "Vary: Cookie" was left in place, just in case, as a possible fallback for local HTTP/1.0 servers. Like this, it's easy to test the impact of "Vary: Cookie", as removing the ETAG_CEXP flag from the call to etag_check() changes ETag generation to be "login-time-agnostic", again. I would really like to encourage you to try the patch, and see how this changes the user experience for login and logout actions related to /uv web pages, on local and remote Fossil web servers. Thank you very much --Florian [1] A /uv repository index page can be updated by scripting and replaced for local clones, and unlike with /doc or /wiki, changes never show up in the Fossil timeline and/or file hierarchy. I'm keeping the (identical) index pages for my repositories in a separate meta-repository, so no need to archive
Re: [fossil-users] HTTP caching, again
Joerg:
> Such a proxy would be pretty broken. ... Again, such a client is
> pretty much broken already under the caching model. ...
I agree. Writing an HTTP server in a perfect world may be easy. But I feel
like a lot of programming work are efforts to make broken clients (or
"components") work?
My tests for the "Vary: Cookie" header were on Windows, with the Fossil
built-in (`fossil ui') local web server, and the "localauth" setting
enabled, so that login and logout was possible. This worked fine.
But then I noticed something strange: most major web browsers (not tested
with Firefox) do not seem to send If-None-Match request headers for
localhost connections, but only use If-Modified-Since.
So I rebuilt Fossil with the same modification to send a "Vary: Cookie"
header along with ETag and Last-Modified headers, and tested it on my
FreeBSD / Apache remote web server -- and unfortunately it doesn't work
(again tested with most major web browsers, not tested with Firefox).
Apache seems to do minimal HTTP header rearrangements compared to the
Fossil local web server, such as:
Vary: Accept-Encoding
Vary: Cookie
to:
Vary: Cookie,Accept-Encoding
and:
Connection: close
to:
Connection: Keep-Alive
Keep-Alive: timeout=2, max=99
but I think this should not affect HTTP caching.
So it looks like "Vary: Cookie" only works for clients relying on the
If-Modified-Since cache control mechanism?
The only way I found to get both the Fossil built-in local web server, and
the FreeBSD / Apache remote web server to work as expected, i.e. that /uv
web pages are are correctly expired after a new user is logged-in, was the
following combination:
* Add "user.cexpire" to the ETag ingredients.
* Skip If-Modified-Since handling after If-None-Match has already detected
an ETag cache miss.
* Send a "Vary: Cookie" header with ETag and Last-Modified headers.
--Florian
= Patch for Fossil [04190488] ==
Index: src/cgi.c
==
--- src/cgi.c
+++ src/cgi.c
@@ -242,10 +242,11 @@
/*
** Do a normal HTTP reply
*/
void cgi_reply(void){
+ int vary_cookie = 0;
int total_size;
if( iReplyStatus<=0 ){
iReplyStatus = 200;
zReplyStatus = "OK";
}
@@ -262,19 +263,24 @@
/* isConst means that the reply is guaranteed to be invariant, even
** after configuration changes and/or Fossil binary recompiles. */
fprintf(g.httpOut, "Cache-Control: max-age=31536000\r\n");
}else if( etag_tag()!=0 ){
fprintf(g.httpOut, "ETag: %s\r\n", etag_tag());
+vary_cookie = 1;
fprintf(g.httpOut, "Cache-Control: max-age=%d\r\n", etag_maxage());
}else{
fprintf(g.httpOut, "Cache-control: no-cache\r\n");
}
if( etag_mtime()>0 ){
fprintf(g.httpOut, "Last-Modified: %s\r\n",
cgi_rfc822_datestamp(etag_mtime()));
+vary_cookie = 1;
}
+ if( vary_cookie )
+fprintf(g.httpOut, "Vary: Cookie\r\n");
+
if( blob_size()>0 ){
fprintf(g.httpOut, "%s", blob_buffer());
}
/* Add headers to turn on useful security options in browsers. */
Index: src/doc.c
==
--- src/doc.c
+++ src/doc.c
@@ -641,17 +641,26 @@
}
}
if( isUV ){
if( db_table_exists("repository","unversioned") ){
Stmt q;
+char *zHash=0;
+sqlite3_int64 mtime=0;
db_prepare(, "SELECT hash, mtime FROM unversioned"
" WHERE name=%Q", zName);
if( db_step()==SQLITE_ROW ){
- etag_check(ETAG_HASH, db_column_text(,0));
- etag_last_modified(db_column_int64(,1));
+ zHash = fossil_strdup(db_column_text(,0));
+ mtime = db_column_int64(,1);
}
db_finalize();
+if( zHash==0 || etag_check(ETAG_HASH, zHash)==0 ){
+ /* Prevent the If-Modified-Since cache handler to
+ ** undermine cache misses already cleared by the
+ ** If-None-Match cache handler. */
+ if ( mtime ) etag_last_modified(mtime);
+}
+if( zHash ) free(zHash);
if( unversioned_content(zName, )==0 ){
rid = 1;
zDfltTitle = zName;
}
}
Index: src/etag.c
==
--- src/etag.c
+++ src/etag.c
@@ -20,14 +20,15 @@
** An ETag is a hash that encodes attributes which must be the same for
** the page to continue to be valid. Attributes that might be contained
** in the ETag include:
**
** (1) The mtime on the Fossil executable
-** (2) The last change to the CONFIG table
-** (3) The last change to the EVENT table
-** (4) The value of the display cookie
-** (5) A hash value supplied by the page generator
+** (2) The "user.cexpire" field for logged-in users
+** (3) The last change to the CONFIG table
+** (4) The last change to the EVENT table
+** (5)
Re: [fossil-users] HTTP caching, again
Joerg: > I don't think you need to reset it, just sending the vary header > should be enough? I was able to try this, and it works fine! Adding the following line: fprintf(g.httpOut, "Vary: Cookie\r\n"); right after printing the ETag header in src/cgi.c (and also after printing the Last-Modified header, if not already printed after the ETag header) results in correct web page expiration after login and logout. Using "user.cexpire" to calculate the ETag may give more fine-grained control, as for example a /uv page would not need a refresh if an unrelated cookie (for example, to set /timeline display options) were changed, but overall, the "Vary: Cookie" method may work well enough. Also, with "Vary: Cookie", there may be issues with caching proxies, depending on whether they receive and evaluate all the cookies, but this may not be a problem for Fossil. For clients that do not understand or support "Vary: Cookie", I would still suggest to perform the Last-Modified checks only if no ETag was included with the request (so that ETag misses can not be outdone by Last-Modified hits). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] HTTP caching, again
D. Richard Hipp: > The ETag values already reset based on changes to the display > cookie. I suppose they could change again based on the login > cookie. The question is, would this solve Florent's problem? Yes, adding "user.cexpire" to the ETag ingredients [0] would solve part of the problem: /uv pages (and hence the navigation menu entries) are not expired after a new user is logged-in. [0] http://fossil-scm.org/index.html/artifact?=9ef915be04=24-28 The other part is that whenever the client has included an ETag header with the request, the Last-Modified header should no longer be evaluated, but an ETag mismatch should already result in a cache miss, without giving the Last-Modified pathway a chance to "validate" a page previously classified as expired by the ETag handler. To test this, you can view the (unversioned) Download page on the Fossil website, then click login and logout from there, and check if you can always see the correct (i.e. the current, or no) username near the login/logout link, and the Admin menu entry (if enabled by the skin), or if you need to press Ctrl+F5. Joerg: > Would a vary-on-cookie solve this already? Aha, thanks for the input, today's bedtime reading ... I live almost in the UTC time zone, handy when using Fossil :) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] HTTP caching, again
> One possible solution may be to include the "cexpire" field in > ETag calculation, drop the If-Modified-Since handler, but still > return a Last-Modified date. Well, it may be possible to support both caching mechanisms. But then an ETag mismatch should result in a cache miss, and the If-Modified-Since route should no longer be taken thereafter. As Fossil supplies both and ETag and a Last-Modified header for the initial (non-cached) page view, web browsers will also send back both headers, and then If-Modified-Since can undermine If-None-Match cache misses. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] HTTP caching, again
Sorry for my perseveration on the topic. I'm using a /uv index page for my repositories. After login, the index page is not expired, and I can only see the Admin entry from the top navigation bar until after a "hard" reload with Ctrl+F5. So I tried to to generate a "login-time-sensitive" ETag. This worked well with the "cexpire" field from the "user" table (which is actually the login time, shifted to the future by one unit of the "cookie-expire" setting). But as there's serial cache expiration checks, an ETag cache miss is immediately caught by a Last-Modified cache hit (no matter whether or not the Fossil executable mtime is used to limit the age of the Last-Modified date). I'm not sure if this can be solved to work well for both web UI pages (ETag preferred) and "static" files downloaded via scripts (Last-Modified preferred). One possible solution may be to include the "cexpire" field in ETag calculation, drop the If-Modified-Since handler, but still return a Last-Modified date. Like this, wget and simple scripts can still do their own If-Modified-Since cache checks based on HTTP HEAD requests, and use the returned Last-Modified date to adjust file time stamps. Given that the time stamps of unversioned files can be changed arbitrarily, and may not be 100% accurate anyway, this could make a good compromise. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] The /timewarps web UI does not seem to work
Ah, now I see. I was using a script to check-in a commit to a new (empty)
repository, with the commit time older than the initial check-in time. I
aborted the script on the clock skew warning, but found an empty /timewarps
log.
It's only timewarps originating from the initial check-in that are not
shown. Timewarps not originating from the "repository root" are displayed:
fossil init --admin-user florian test.fossil
fossil open test.fossil
fossil ci -m "Older check-in date" --date-override "2018-04-14T16:30" \
--allow-empty --allow-older --user florian
fossil ci -m "Oldest check-in date" --date-override "2018-04-14T16:29" \
--allow-empty --allow-older --user florian
fossil ui --page timewarps
This displays one entry ("Older" to "Oldest", but not "Older" to "initial
empty check-in").
Sorry for the false alarm. I fear to be dropped soon from the list :(
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] The /timewarps web UI does not seem to work
I'm sorry then I didn't understand what are "timewarps", and there's none in the Fossil repository (as I had mistakenly assumed). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] The /timewarps web UI does not seem to work
Sorry for messing up the line-width for sample commands and scripts posted to this mailing list. I have noticed that the /timewarps web UI does not seem to work, any more. Test case: fossil init --admin-user florian test.fossil fossil open test.fossil fossil ci -m "Older check-in date" --date-override "2018-04-14T16:30" \ --allow-empty --allow-older --user florian fossil ui --page timewarps Or: http://www.fossil-scm.org/index.html/timewarps The "Instances of timewarp" page says "No timewarps in this repository" for either case. I think the Fossil repository had some check-ins with timewarps for testing. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Patch to fix leftover SHM and WAL files
I have now been able to reproduce the behavior on Windows, with a fake CGI request for an /uv cache hit. The following commands create a new test repository with an unversioned file, a CGI control file, and set a few environment variables. The commands can be run from a batch file, or by copy-pasting all of them at once into the command prompt. fossil init --admin-user dummy test.fossil fossil rebuild --wal -R test.fossil echo Just a text file> test.txt fossil uv add test.txt --mtime "2018-05-14T00:00" -R test.fossil --user dummy echo repository: test.fossil> test.cfg set SCRIPT_NAME=dummy set PATH_INFO=/uv/test.txt set HTTP_IF_MODIFIED_SINCE= fossil cgi test.cfg echo List SHM and WAL files with HTTP 200 OK == dir *-shm *-wal set HTTP_IF_MODIFIED_SINCE=Mon, 14 May 2018 00:00:00 GMT fossil cgi test.cfg echo List SHM and WAL files with HTTP 304 Not Modified dir *-shm *-wal And before each new test run: del test.fossil test.fossil-shm test.fossil-wal With the latest Fossil binary downloaded from fossil-scm.org, the SQLlite temporary files are removed for "200 OK", but they are left behind for "304 Not Modified". With the patch from the first post applied, the temporary files are also removed for "304 Not Modified". --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Patch to fix leftover SHM and WAL files
I have now had access to a fully equipped FreeBSD machine, and was able to
do more investigation.
I found that the patch appended to this message fixes the problem with the
leftover SHM and WAL files. They are seen each time an unversioned file is
accessed over the /uv web interface, with an IF-NONE-MATCH or
IF-MODIFIED-SINCE cache hit.
I was using the Fossil binary from here when I discovered the problem:
http://pkg.freebsd.org/FreeBSD:10:amd64/latest/All/fossil-2.6,2.txz
I have verified that a mere rebuild (without applying the patch) does not
fix the problem.
I'm using a simple CGI setup to run Fossil on my web server, where each CGI
request is answered by a newly spawned (and immediately terminated)
process. People with more sophisticated server configurations, where Fossil
keeps running in the background, may not see this problem, probably as the
database is "implicitly finalized" by subsequent SQL queries, or because
other running instances of Fossil immediately clean up the SHM and WAL
files, I don't know. Also, I could not reproduce the problem on Windows.
Anyway, this simple patch fixes my problem, and I'd be very happy if you
could incorporate it into the main Fossil code base. (I have already sent
you my signed Contributor Agreement last year, in case this matters.)
If there's cases where Fossil keeps running after serving a /doc or /uv
page, it may be worthwhile to free the allocated temporary string.
--Florian
= Patch for Fossil [04190488] ==
--- Fossil-04190488\src\doc.c Fri May 11 17:02:34 2018
+++ Fossil-patched\src\doc.c Sat May 12 10:41:00 2018
@@ -636,27 +636,31 @@
if( !file_is_simple_pathname(zName, 1) ){
goto doc_not_found;
}
}else{
goto doc_not_found;
}
}
if( isUV ){
if( db_table_exists("repository","unversioned") ){
Stmt q;
+char *zHash=0;
+sqlite3_int64 mtime=0;
db_prepare(, "SELECT hash, mtime FROM unversioned"
" WHERE name=%Q", zName);
if( db_step()==SQLITE_ROW ){
- etag_check(ETAG_HASH, db_column_text(,0));
- etag_last_modified(db_column_int64(,1));
+ zHash = fossil_strdup(db_column_text(,0));
+ mtime = db_column_int64(,1);
}
db_finalize();
+if( zHash ) etag_check(ETAG_HASH, zHash);
+if( mtime ) etag_last_modified(mtime);
if( unversioned_content(zName, )==0 ){
rid = 1;
zDfltTitle = zName;
}
}
}else if( fossil_strcmp(zCheckin,"ckout")==0 ){
/* Read from the local checkout */
char *zFullpath;
db_must_be_within_tree();
zFullpath = mprintf("%s/%s", g.zLocalRoot, zName);
= Patch for Fossil [04190488] ==
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil 2.6: shm- and wal-files left behind
Sorry for spamming the list, I try short. Just another comment about Fossil's If-Modified-Since rules: http://fossil-scm.org/index.html/artifact?name=535a623ccd=154-157 I learned a lot about If-Modified-Since when adding HTTP caching to my home-brewed wiki. All kind of information was important to compute the Last-Modified header, like the last modified time of the most recent page in the wiki (to decide whether or not to expire a page, and reparse its hyperlinks, as they were styled differently when targeting existing or missing pages), the user login time (to decide whether or not to expire a page, if another user with different permissions was logged in), or the CGI script last modified time (to expire pages rendered with an earlier version of the script, mostly useful for developing and testing) -- but the "real" modified time of the wiki page itself was not used. That's the principle Fossil uses when serving "processed" unversioned content (such as wiki pages, where the output may change with new versions of Fossil with enhanced wiki rendering capabilities, or with the menu bar entries reflecting permissions of the currently logged-in user). The Last-Modified header of an unversioned file is not important, in this case, and can never be older than the Fossil binary file mtime. But for binary files, such as zip-archives, I see the /uv web page more as a "static" store, and would like to get the "real" Last-Modified header, so that (simple) scripts can decide whether or not a re-download is necessary, and apply correct time stamps to the downloaded files. Maybe Fossil could treat "processed" content (wiki pages and the like) differently from purely "static" content? Or, an additional /uvraw (or /static) web interface could be used to download unprocessed content, as is, with the original Last-Modified time stamps? Is this the famous 2 hard problems in computer science: cache invalidation, naming things, and off-by-1 errors? :) (I'm just trying to be funny, I hope my amateurish messages are not offending to all you highly decorated computer specialists. Also, I can't reproduce the leftover SHM and WAL files on Windows, and I'm not able to build Fossil for FreeBSD and do more tests there, my shared host is lacking tools, and I don't have my virtual machine "at hand", right now, I'm sorry.) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil 2.6: shm- and wal-files left behind
With further testing, I noticed that the If-Modified-Since cache control
mechanism does not seem to work for /uv web pages, anymore. This can be
seen by repeatedly downloading the Fossil Source Tarball using wget:
wget -N http://www.fossil-scm.org/index.html/uv/fossil-src-2.6.tar.gz
...
For unmodified downloads, wget is expected to say:
HTTP request sent, awaiting response... 304 Not Modified
File 'fossil-src-2.6.tar.gz' not modified on server. Omitting download.
But for the Fossil Source Tarball, it repeatedly says:
HTTP request sent, awaiting response... 200 OK
Server ignored If-Modified-Since header for file 'fossil-src-2.6.tar.gz'.
I know that this feature used to work well, when it was added, I can still
remember my joy :) about it:
[...] If-Modified-Since [...] works like a charm! [0]
[0]
https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg27068.html
I noticed a few things:
Both etag_check() and etag_last_modified() in src/etag.c properly call
fossil_exit() to shutdown SQLite when they handle a cache hit (they are
called from doc_page() in src/doc.c).
But they are called inside a db_step() { ... } db_finalize() block [1], and
when they call fossil_exit(), db_finalize() does not seem to be called.
(When fossil_exit() calls db_close(), there's some db_finalize() action
going on, but if I understand that right, it's just for the local checkout
database, and not for the main repository database.) I'm not sure if this
might be the reason for the leftover SHM and WAL files?
[1] http://fossil-scm.org/index.html/artifact?name=dbf23864f8=648-652
I'm also not 100% sure why wget can't handle the If-Modified-Since case any
more, as the initial requests to /uv web pages correctly return both ETag
and Last-Modified headers. But, after updating the Fossil binary to version
2.6, I suspect that wget may hit the following condition: [2]
[2] http://fossil-scm.org/index.html/artifact?name=535a623ccd=154-157
This looks like unversioned files older than the Fossil binary always
generate If-Modified-Since cache misses (since the Last-Modified header
always seems to report the "real" mtime, and not a "Fossil binary"-adjusted
"pseudo mtime", as is done for ETags computation).
Therefore, I would like to make the following suggestions:
* Check if db_finalize() should be called explicitly before etag_check()
and etag_last_modified() are allowed to terminate Fossil.
* Check if the "resources older than the Fossil binary always get
If-Modified-Since cache misses" condition should be relaxed for /uv web
pages (this condition may make sense for CSS style sheets compiled into the
Fossil binary, for example).
* cgi_modified_since() in src/cgi.c seems to be unused [3].
[3] http://fossil-scm.org/index.html/artifact?name=da970218b4=1957-1970
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil 2.6: shm- and wal-files left behind
Warren Young:
> If “fossil server” or “fossil ui” is still running in the
> background, then I would fully expect the WAL and SHM files to
> still be around. The worrying case is when they’re left behind
> after the SQLite-based process that created them has died.
My observations were from CGI requests ("fossil cgi"). I think the Fossil
process is terminated after each request, as I usually don't see any
temporary files, and `ls /proc' shows no leftovers (if this is the proper
way to check this on *nix).
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] More Login/Logout web page focus issues
With Fossil 2.6, there's a new "unencrypted connection" security warning in the Login/Logout form, which also has a hyperlink to a secure login page. In some web browsers, pressing TAB to move the input focus from "User ID" to "Password" sets the focus to this hyperlink, instead of the next form control. I would like to suggest to add "tabindex" attributes to the login form controls, so that it's possible to cycle through the controls in a logical order, i.e. "tabindex=1" for "User ID", "tabindex=2" for "Password", "tabindex=3" for the "Login" button, and maybe "tabindex=4" for the secure login link. As auto-focusing the "User ID" field is unfortunately no longer possible [0], setting a "tabindex" may also have the effect that the "User ID" field can be focused with one single TAB keypress (on some web browsers, this may only work if all other controls have a higher "tabindex", or "tabindex=-1", the latter reducing accessibility). [0] https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg27053.html Also, adding "accesskey" attributes can be handy to jump to non-auto-focused form controls, for example "accesskey=u" for "User ID", and "accesskey=p" for "Password". The hotkeys could be displayed with underlined style in the form labels, to make them visible to the user. Moreover, wrapping the labels for the input fields inside "..." enables another quick way (by clicking the label) to focus an input field with all contents selected. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Fossil 2.6: shm- and wal-files left behind
With Fossil 2.6, sometimes there's *.fossil-shm and *.fossil-wal files left behind in the directory where the repository databases are stored. This happens if an unversioned file is accessed through the /uv web interface for the second time (or more). If the web interface to view an unversioned file is accessed for the first time, or if any other page of the web UI is accessed, no such files are present. All web requests seem to complete without signs of error in the web browser (in particular, no red error message on top, also not for non-cached pages other than /uv). There's no suspicious entries in the web server error logs, and no core dump files, or similar. My shared web host is running FreeBSD (64-bit) and Apache, with Fossil 2.6 [9718f3b078] through a CGI script. I haven't been able to reproduce this on Windows, but I have only tested the `fossil ui' mechanism, and not the full web server/CGI setup. I wonder if this could be related to the recently added support for the ETags, Last-Modified, and If-Modified-Since cache control mechanisms, i.e. that there's some "premature shutdown" of the SQLite engine when handling /uv cache hits? I'm aware this is not dangerous, as SQLite will recover any information from the leftover files, but nonetheless I'm reporting it. --Florian (My index-pages are set to unversioned wiki files, and they have hyperlinks to the repository list, so it's a common scenario for me to "leave" a repository through an unversioned index-page, revealing the shm- and wal-files.) ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Direct SQL dumps have modified line endings
I'm sorry the batch script from my previous message did not make it through the mailing list in a ready-to-copy-and-paste form, so it's posted here: https://pastebin.com/8StRhDHJ --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Feature request: warnings about *mixed* line endings only
The "crlf-glob" setting allows files to have CR, CR+LF or mixed line endings. The "--no-warnings" option for "fossil commit" works analogously (and also affects warnings about binary files). As a (mostly) Windows user, I would appreciate a way to silence warnings for all files with *consistent* line endings (CR, CR+LF, and LF), but still display a warning for files with *mixed* line endings, or *binary* contents. Decent text editors ensure consistent line endings, but files auto-generated from various sources, or modified using command-line tools, can still have mixed line endings. This sounds like a simple change, but from looking at the Fossil source code, it seems less trivial, as the decisions about line ending warnings are all coalesced to single "crlfOk" variables. Maybe a new "crlf-consistency" setting (to alphabetically sort near "crlf-glob", but work independently) is the best way to implement this? A new command line option for "fossil commit" may also be required, so as not to change workflow? I hope that this feature request is of enough general interest, so that you may consider adding it. Thank you very much! --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Direct SQL dumps have modified line endings
I'm using direct SQL queries on Fossil repository databases to find file blobs, similar to what Fossil itself uses to generate the /bloblist web page: http://fossil-scm.org/index.html/artifact?name=d99ecd15=837-847 With a list of file blobs, it's possible to find files from one repository missing in another repository. Or, by printing file blobs to stdout redirected to a pipe, it's possible to create scripts to perform full-text searches over the entire repository contents from the command line. For me, these are more great example of the powers of Fossil! While using SQL anyway, I dumped the file blobs with the following SQL command: [a] SELECT content(blob.uuid) FROM blob WHERE blob.rid = "RID"; I'm aware that this is not the documented way to dump a file blob, there's: [b] fossil artifact "rid:RID" [c] fossil cat filename -r VERSION (Finding VERSION from a RID or UUID requires further SQL queries.) While [b] and [c] generate output with the same line endings as the original files, method [a] generates output with completely different line endings. Following is a Windows batch script to demonstrate this. The Windows built-in `certutil' command is used to generate test files with CR+LF, LF, or CR line endings, which are then committed to a new repository. Next, the file blobs are dumped to separate files with each of the methods [a], [b], and [c]. -- file: test-eol.cmd -- @echo off echo 43 52 0d 43 52 0d 43 52 0d 43 52 0d 43 52 0d CR.CR.CR.CR.CR.> hex-cr.txt certutil -decodehex -f hex-cr.txt file-cr.txt del hex-cr.txt echo 43 52 4c 46 0d 0a 43 52 4c 46 0d 0a 43 52 4c 46 CRLF..CRLF..CRLF> hex-crlf.txt echo 0010 0d 0a 43 52 4c 46 0d 0a 43 52 4c 46 0d 0a ..CRLF..CRLF..>> hex-crlf.txt certutil -decodehex -f hex-crlf.txt file-crlf.txt del hex-crlf.txt echo 4c 46 0a 4c 46 0a 4c 46 0a 4c 46 0a 4c 46 0a LF.LF.LF.LF.LF.> hex-lf.txt certutil -decodehex -f hex-lf.txt file-lf.txt del hex-lf.txt fossil init --admin-user florian test.fossil fossil open test.fossil fossil add file-cr.txt file-crlf.txt file-lf.txt fossil ci --user florian -m "EOL Test" --no-warnings fossil artifact "rid:2"> dump-artifact-rid2.txt fossil artifact "rid:3"> dump-artifact-rid3.txt fossil artifact "rid:4"> dump-artifact-rid4.txt fossil cat file-cr.txt -r tip> dump-cat-file-cr.txt fossil cat file-crlf.txt -r tip> dump-cat-file-crlf.txt fossil cat file-lf.txt -r tip> dump-cat-file-lf.txt echo SELECT content^^^(blob.uuid^^^) FROM blob WHERE blob.rid = "2"; | fossil sql> dump-sql-rid2.txt echo SELECT content^^^(blob.uuid^^^) FROM blob WHERE blob.rid = "3"; | fossil sql> dump-sql-rid3.txt echo SELECT content^^^(blob.uuid^^^) FROM blob WHERE blob.rid = "4"; | fossil sql> dump-sql-rid4.txt fossil close -- file: test-eol.cmd -- The resulting "dump-artifact-*.txt" and "dump-cat-file-*.txt" are all identical to the original "file-*.txt", but the "dump-sql-*.txt" have different line endings: CR → CR, with an extra CR+LF at the end CR+LF → CR+CR+LF, with an extra CR+LF at the end LF → CR+LF, with an extra CR+LF at the end I don't consider this a problem or even a bug, and the work-around is to use method [b] to dump a file blob. However, I wanted to report the behavior, as I don't see the logic (but this may be just me), and I'm somewhat surprised that SQL queries return "modified data", in case somebody relies on similar SQL queries for backups. Or is it just the Fossil-specific `contents(uuid)' SQL function doing some line ending magic for internal use, which is fixed before using the blob in Fossil? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
It's not at all that I think I'm the expert you've summoned in your
last post on this thread. As you've been running your own high-traffic
"althttpd" server for many years, maybe you should explain the
Internet to us.
I was doing some (limited) research about RFCs, recommendations from
the Apache web server documentation, and blog posts by web browser
developers. It's often stated that "If-None-Match" should be given
precedence over "If-Modified-Since", but I haven't come across any
"hard facts".
I've created a small PHP script to test HTTP caching with ETags.
= etags.php =
4) $m=0;
$lmt=1519776000; // Wed, 28 Feb 2018 00:00:00 GMT
$etag='caffee'; // fixed
// handle If-Modified-Since
if (($m==1 || $m>=2 && !isset($_SERVER['HTTP_IF_NONE_MATCH'])) &&
isset($_SERVER['HTTP_IF_MODIFIED_SINCE']) &&
strtotime($_SERVER['HTTP_IF_MODIFIED_SINCE'])==$lmt) {
header('Not Modified',true,304);
exit;
}
// handle If-None-Match
if ($m>=3 && isset($_SERVER['HTTP_IF_NONE_MATCH']) &&
$_SERVER['HTTP_IF_NONE_MATCH']=='"'.$etag.'"') {
header('Not Modified',true,304);
exit;
}
// generate new data
if ($m==2)
$etag='--'; // none
else if ($m!=4)
$etag=uniqid(); // random
if ($m!=2)
header('ETag: "'.$etag.'"');
header('Last-Modified: '.gmdate('D, d M Y H:i:s',$lmt).' GMT'); // fixed
header('Cache-Control: must-revalidate, private');
header('Content-Type: text/plain; charset=utf-8');
echo
"TEST HTTP CONDITIONAL REQUEST PRECEDENCE\n".
"\n\n".
"• Always reply with fixed \"Last-Modified\" date header\n".
"• Reply with random, fixed, or no ETag (see query parameters)\n\n".
"• IF_MODIFIED_SINCE: always cache hit (send \"304/Not Modified\")\n".
"• IF_NONE_MATCH: cache hit on ETag match, cache miss otherwise\n";
echo
"\nQuery Parameters (Mode):\n\n".
"• ?m=0: random ETag, no caching (default)\n".
"• ?m=1: random ETag, favor IF_MODIFIED_SINCE (cache hit)\n".
"• ?m=2: omit ETag, favor IF_MODIFIED_SINCE (cache hit)\n".
"• ?m=3: random ETag, favor IF_NONE_MATCH (cache miss)\n".
"• ?m=4: fixed ETag, favor IF_NONE_MATCH (cache hit)\n";
echo "\nScript State:\n\n";
echo "• Mode: $m\n";
echo "• Date: ".gmdate('D, d M Y H:i:s')." GMT\n";
echo "• ETag: $etag\n";
echo "\nRequest Headers:\n\n";
foreach ($_SERVER as $k=>$v)
if (substr($k,0,5)=='HTTP_') echo "• $k: $v\n";
echo "\nResponse Headers:\n\n";
foreach (headers_list() as $k=>$v)
echo "• $v\n";
?>
= etags.php =
The script always responds with a constant "Last-Modified" header, but
generates a random ETag on each run (there's also other options,
explained in the script output).
The idea is that whenever the ETag → "If-None-Match" route is taken,
the script output in the web browser is also updated, so it's obvious
that the client preferred "If-None-Match" over "If-Modified-Since".
The script should be reloaded with F5 and Ctrl+F5 (to disable caching)
to see the effects.
The clients I've tested all favor ETags over "If-Modified-Since", even
Internet Explorer 8 on Windows XP, so no harm here. But of course
"wget -N" without the extra Perl wrapper to add ETag support favors
"If-Modified-Since".
I see that client-server clock skews, and the low resolution (1
second) of the "Last-Modified" timestamp, even more so in conjunction
with the --mtime option to set arbitrary timestamps for unversioned
files, may cause havoc.
But maybe the above script is helpful to test more "exotic" clients on
the way to the decision to support ETags AND "Last-Modified" vs. ETags
only.
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
D. Richard Hipp:
> Consider this sequence of operations:
>
> (1) User A does a "uv push"
> (2) User B does a "wget -N" against the uv.
> (3) User C does a "uv push" of different content.
>
> If all of (1), (2), and (3) happen during the same second and if
> unversioned content honors the If-Modified-Since header, then
> subsequent attempts by user B to do "wget -N" will continue to
> retrieve the obsolete version of the unversioned content that was
> uploaded by user A. User B will never see the more recent content
> uploaded by user C.
Yes, I agree.
It's the same situation as if multiple users were sharing files on a
web server space not under version control, with uploads by FTP ("uv
push") and downloads by HTTP ("wget -N").
This may work well for "inert" files, requiring small temporary local
adaptations not worth archiving, but bothersome if overwritten by an
accidental "wget". However, "mission-critical" and "fast-changing"
files, for which the risk of loosing content is unacceptable, had
better be put under version control.
As far as I remember, I've come across the recommendation to combine
ETags and Last-Modified headers, so the client could pick
If-None-Match or If-Modified-Since to validate its cached content.
And, it's already there, and works like a charm!
Thank you very much for all your efforts you're putting into this
project, this is really appreciated!
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
Ron: > Someone already made an ETag wrapper for wget: > https://www.w3.org/2001/12/rubyrdf/pack/tests/scutter/wget.pl Thanks for the interesting link. > There is no requirement or specification for the value of an ETag, > could use the Fossil artifact hash as the ETag value. Then a script > trying to fetch the file can generate the value from the current copy > of the file. I agree, but with the current implementation, at least the ingredient is hard to guess for a client script trying to generate a Fossil ETag. I think that the "Last-Modified" header is much easier to handle, as it boils down to parsing a GMT date/time string. And, as already mentioned, clobbering of locally-modified files can be avoided with the timestamp mechanism, this is not possible with an ETag alone. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
Thinking about HTTP caching twice, the following comes to my mind: A command line download tool, that unlike a web browser does not keep a cache of content and associated ETags, won't be able to calculate the ETag for a file it is going to request from a Fossil web server. ETags are not simple file checksums, and Fossil seems to use the following information to generate them: /// So command line download tools can only rely on the "Last-Modified" header. And, as already mentioned, I do see additional benefit from receiving file timestamps over the web. This is in contrast to check-out timestamps, which Fossil does not preserve, likely so that build tools like `make' know what to do. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
> Is the new ETag mechanism sufficient for your purposes? That's a great addition, thanks! However, my simple scripting tools don't support ETags, i.e. the `wget -N' example (latest version 1.19.4) mentioned earlier still carries out two full downloads of the Fossil source code archive. With the "Last-Modified" header, the timestamp of the downloaded unversioned file can be set correctly, and locally updated unversioned files waiting to be synchronized with the server would not be clobbered by accidental runs of the download script, for example, but this would fail with ETags (if I get it right, ETags can only say "different", whereas "Last-Modified" can say "newer"). Even the most basic scripting solutions, like "WinHttp.WinHttpRequest.5.1" used with WSH, are able to return the "Last-Modified" header, whereas implementing ETag support in these cases requires "less-recreational" scripting ;) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Minor Issues with Fossil 2.5
> That's because autofocus causes CSS flashes on some web-browsers.
> If you can suggest a reasonable workaround, that would be great.
Oh, I'm sorry I've missed that.
The only other way I know is by JavaScript, something like:
window.onload = function() {
document.getElementById("u").focus();
};
Nowadays, the faster "DOM ready" event may be preferred over
`window.onload', but finding a cross-browser solution here always
seems somewhat cumbersome to me, and the latter seems quick enough for
simple web pages.
However, I'm not sure what are "CSS flashes", and whether they may
appear with the JavaScript method, as well.
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Minor Issues with Fossil 2.5
I've discovered a few minor issues after updating to Fossil 2.5. As I have skipped version 2.4, I'm not sure when they were really introduced. I'd appreciate if they could be considered to be fixed in case there's some nearby work, say like an occasional appendectomy ;) 1. On the Login/Logout web page, the User ID input field used to have the keyboard "autofocus". With Fossil 2.5, the input field is no longer focused, but has to be activated by hitting Tab repeatedly (cycling through all the links in the top navigation bar, first), or by using the mouse. I would like to suggest restoring the old "autofocus" behaviour. 2. There's no "Last-modified" HTTP header when downloading unversioned files through the /uv web page, causing undesired network traffic for `wget -N' scripting scenarios, for example, and maybe also with web spiders (if they are enabled). I would like to suggest sending the "Last-modified" HTTP header corresponding to the timestamp of the requested unversioned file. Example: wget -N http://fossil-scm.org/index.html/uv/fossil-src-2.5.tar.gz Running this command twice should only download the file once, if the timestamp had not changed in between the commands. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] GOlang supports fossil
Chris: > This makes fossil a first class scm citizen together with git and others .. Depends on your point of view, you could also say that this makes Go a first class citizen among programming languages ... ;-) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Check-ins not colored on /leaves web page
Compared to version 2.3 [f7914bfdfa], current tip version 2.5 [8007cfc0db] no longer displays check-in background colors on the /leaves web page. As the /brlist web page has recently been enhanced with an option to show branch colors, I think this minor regression for /leaves was not intentional, but a side effect of the latest timeline and skin updates? At least, I liked it, and wanted to report it. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil with IPv6 support on Windows XP
Olivier Mascia: > But on XP the delay is 'intolerable' rather than unnoticeable. :) > > A small patch for that is included along a larger IPv4/IPv6 patch > I submitted today. > > I see Richard put it on trunk (see [e506ebb764]) minutes ago. > > Especially those of you using Windows XP would help by building > from that code and see how it goes for them. > > It looks good, and does not require anymore to install IPv6 > support (command 'ipv6 install') on Windows XP. The solution is > IPv4/IPv6 agnostic. If at least one of both protocols are useable, > the software should work nicely. > > As I'm responsible for asking for this support and did a fair part > of that coding, I'll stay alert on issues that would come out, in > order to quickly help, should it happen. > > You might then enjoy a fresh build out of trunk. Goes off like a rocket, thanks for the nice work! Also, using `GetStdHandle(STD_INPUT_HANDLE)!=NULL' to test whether or not running in a console session is clever, my proposed solution was to check for the `ui' and `server' commands. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil with IPv6 support on Windows XP
Thomas Schnurrenberger: > I measured the time it takes to call StartServiceCtrlDispatcherW > on my 8 year old Windows 7 64bit box: roughly 600 microseconds, I > don't think this is noticeable! > It would be interesting to known time on your XP boxes. Thanks for your interest and for the test program! My results are: Elapsed microseconds: 15000606 Elapsed seconds : 15.000607 rc : 1 This is for both my standard and tweaked physical machines mentioned earlier [0], as well as a newly setup virtual box. Note that all my systems are 32-bit. [0] https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25001.html A virtual box (32-bit) to reproduce the behavior can be setup quickly: * Download "Windows XP Mode" [1] * The downloaded file is an SFX archive containing a VHD * Start the OS from the VHD with any virtualization tool [1] https://www.microsoft.com/en-us/download/details.aspx?id=8002 I'm (still) aware that this is not an important problem, but I find it interesting. I have some old development tools only available on my XP machines (older versions of Visual Studio still providing Resource Editors, to name it). Given the portability of Fossil, I was happy it worked on Windows XP, but waiting 15 seconds for `fossil ui' cools down my joy a little bit ;-) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil with IPv6 support on Windows XP
Thanks for the interesting details about IPv6 and Windows. Richard Hipp: > I'd appreciate it if some readers could try the two versions out, > side by side, and give their opinions. Focusing on Windows XP here, the version with [c038de8d] works fine and navigates to the correct page for `fossil ui'. Hard to say anything reasonable about speed on Windows XP, as Google Chrome version 49.0.2623.110 is the latest XP-available browser to handle all the JavaScript and timeline drawings correctly. The startup delay for `fossil ui' and `fossil server' on Windows XP is more obvious than possibly sluggish browser navigation, which I *think* is due to waiting for StartServiceCtrlDispatcherW. This could be cut down by skipping the call to StartServiceCtrlDispatcherW for the `ui' and `server' commands, as Fossil always runs in a console session, and not as a service, in these cases. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Fossil with IPv6 support on Windows XP
Re: Fossil with IPv6 support on Windows XP I have several physical and virtual XP machines around, and `fossil ui' doesn't work any more with the new patch. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Automatic update when changing type on timeline?
A submit button would make it more evident that the page can be updated by hitting enter or pressing the submit button. Relying on the JavaScript "onxxx" handlers to update the page works inconsistently across web browsers. Moreover, the page is updated each time an individual setting is changed, i.e. entering a limit, choosing a type, and entering a tag filter will generate three page reloads. Of course nowadays we have enough bandwith for this, but usually Fossil strives for more network-efficiency in other areas. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Automatic update when changing type on timeline?
No, my patches were about intercepting the enter key from text input elements to submit the form and update the view. In my tests, I found that the "onchange" handler worked fine for drop-down lists, so you may have detected a new problem. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Automatic update when changing type on timeline?
As the form has no "submit" element, it works differently in different
web browsers. I've sent a set of patches directly to Richard Hipp to
make use of the "onkeydown" ("onkeyup" in the initial version) instead
of the "onchange" handler for text input elements [0], as in some web
browsers, the "onchange" handler is only fired after the control lost
focus. The "onchange" handler usually works better for drop-down
lists, but obviously not in all cases, and I'm not sure what's the
solution here.
[0] https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25061.html
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Orphaned option: "timeline-commit-format"
At the time of this writing, the "timeline-commit-format" option seems "orphaned", i.e. it is only referenced by the /setup_timeline web page to display the setting, but not queried or honored any more. As the timeline web view mode defines the display format, I'd like to suggest to change this option to define the default timeline web view (Modern View, Compact View, Verbose View or Columnar View). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Tag problems
Following patch fixes the clash of the short forms of the --limit and
--dryrun options for the `fossil tag' command.
Another form than -N|--limit might be preferred to ensure a better
distinction from -n|--dryrun, such as -c|--limit or -l|limit.
--Florian
Index: src/tag.c
==
--- src/tag.c
+++ src/tag.c
@@ -403,11 +403,11 @@
** of results to the given value.
**
** Options:
** --raw Raw tag name.
** -t|--type TYPE One of "ci", or "e".
-** -n|--limit NLimit to N results.
+** -N|--limit NLimit to N results.
**
** %fossil tag list|ls ?--raw? ?CHECK-IN?
**
** List all tags, or if CHECK-IN is supplied, list
** all tags and their values for CHECK-IN.
@@ -435,11 +435,11 @@
void tag_cmd(void){
int n;
int fRaw = find_option("raw","",0)!=0;
int fPropagate = find_option("propagate","",0)!=0;
const char *zPrefix = fRaw ? "" : "sym-";
- const char *zFindLimit = find_option("limit","n",1);
+ const char *zFindLimit = find_option("limit","N",1);
const int nFindLimit = zFindLimit ? atoi(zFindLimit) : -2000;
db_find_and_open_repository(0, 0);
if( g.argc<3 ){
goto tag_cmd_usage;
@@ -485,11 +485,11 @@
Stmt q;
const char *zType = find_option("type","t",1);
Blob sql = empty_blob;
if( zType==0 || zType[0]==0 ) zType = "*";
if( g.argc!=4 ){
- usage("find ?--raw? ?-t|--type TYPE? ?-n|--limit #? TAGNAME");
+ usage("find ?--raw? ?-t|--type TYPE? ?-N|--limit #? TAGNAME");
}
if( fRaw ){
blob_append_sql(,
"SELECT blob.uuid FROM tagxref, blob"
" WHERE tagid=(SELECT tagid FROM tag WHERE tagname=%Q)"
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Tag problems
Following patch changes all --dryrun options to the more common form
--dry-run for all commands, but this would break working scripts!
--Florian
Index: src/tag.c
==
--- src/tag.c
+++ src/tag.c
@@ -282,11 +282,11 @@
}
/*
** OR this value into the tagtype argument to tag_add_artifact to
** cause the tag to be displayed on standard output rather than be
-** inserted. Used for --dryrun options and debugging.
+** inserted. Used for --dry-run options and debugging.
*/
#if INTERFACE
#define TAG_ADD_DRYRUN 0x04
#endif
@@ -381,11 +381,11 @@
** Options:
** --raw Raw tag name.
** --propagate Propagating tag.
** --date-override DATETIMESet date and time added.
** --user-override USERName USER when adding the tag.
-** --dryrun|-n Display the tag text, but do not
+** --dry-run|-nDisplay the tag text, but do not
** actually insert it into the database.
**
** The --date-override and --user-override options support
** importing history from other SCM systems. DATETIME has
** the form '-MMM-DD HH:MM:SS'.
@@ -392,11 +392,11 @@
**
** %fossil tag cancel ?--raw? TAGNAME CHECK-IN
**
** Remove the tag TAGNAME from CHECK-IN, and also remove
** the propagation of the tag to any descendants. Use the
-** the --dryrun or -n options to see what would have happened.
+** the --dry-run or -n options to see what would have happened.
**
** %fossil tag find ?OPTIONS? TAGNAME
**
** List all objects that use TAGNAME. TYPE can be "ci" for
** check-ins or "e" for events. The limit option limits the number
@@ -452,11 +452,11 @@
if( strncmp(g.argv[2],"add",n)==0 ){
char *zValue;
int dryRun = 0;
const char *zDateOvrd = find_option("date-override",0,1);
const char *zUserOvrd = find_option("user-override",0,1);
-if( find_option("dryrun","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
+if( find_option("dry-run","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
if( g.argc!=5 && g.argc!=6 ){
usage("add ?options? TAGNAME CHECK-IN ?VALUE?");
}
zValue = g.argc==6 ? g.argv[5] : 0;
db_begin_transaction();
@@ -470,11 +470,11 @@
"Use the \"fossil branch new\" command instead.");
}else
if( strncmp(g.argv[2],"cancel",n)==0 ){
int dryRun = 0;
-if( find_option("dryrun","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
+if( find_option("dry-run","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
if( g.argc!=5 ){
usage("cancel ?options? TAGNAME CHECK-IN");
}
db_begin_transaction();
tag_add_artifact(zPrefix, g.argv[3], g.argv[4], 0, dryRun, 0, 0);
@@ -604,11 +604,11 @@
** reparenting operation, simply delete the tag.
**
**--test Make database entries but do not add the tag artifact.
** So the reparent operation will be undone by the next
** "fossil rebuild" command.
-**--dryrun | -nPrint the tag that would have been created but do not
+**--dry-run | -n Print the tag that would have been created but do not
** actually change the database in any way.
*/
void reparent_cmd(void){
int bTest = find_option("test","",0)!=0;
int rid;
@@ -615,11 +615,11 @@
int i;
Blob value;
char *zUuid;
int dryRun = 0;
- if( find_option("dryrun","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
+ if( find_option("dry-run","n",0)!=0 ) dryRun = TAG_ADD_DRYRUN;
db_find_and_open_repository(0, 0);
verify_all_options();
if( g.argc<4 ){
usage("[OPTIONS] CHECK-IN PARENT ...");
}
Index: src/unversioned.c
==
--- src/unversioned.c
+++ src/unversioned.c
@@ -183,11 +183,11 @@
*/
static int unversioned_sync_flags(unsigned syncFlags){
if( find_option("verbose","v",0)!=0 ){
syncFlags |= SYNC_UV_TRACE | SYNC_VERBOSE;
}
- if( find_option("dryrun","n",0)!=0 ){
+ if( find_option("dry-run","n",0)!=0 ){
syncFlags |= SYNC_UV_DRYRUN | SYNC_UV_TRACE | SYNC_VERBOSE;
}
return syncFlags;
}
@@ -236,11 +236,11 @@
** local repository to match the remote repository
** URL.
**
** Options:
**-v|--verbose Extra diagnostic output
-**-n|--dryrun Show what would have happened
+**-n|--dry-run Show what would have happened
**
**remove|rm|delete FILE ...
** Remove unversioned files from the local repository.
** Changes are not pushed to other repositories until
** the next sync.
@@ -250,11 +250,11 @@
** of each
Re: [fossil-users] Tag problems
The following command fails: $ fossil tag add -n --user-override username tagname checkin But this one works as expected: $ fossil tag add --dryrun --user-override username tagname checkin That's because -n is the short form for both --limit and --dryrun: http://fossil-scm.org/index.html/artifact?ln=440=24c77636171affc2 http://fossil-scm.org/index.html/artifact?ln=457=24c77636171affc2 I think the short form for --limit should be changed, because -n|--dryrun is also used by a few other commands. While at it, I noticed that --dryrun is sometimes spelled --dry-run, maybe this could be unified? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Metadata in Timeline Verbose View
Warren Young: > Switch to Modern view? :) Verbose View seems more comfortable to use for me, as the hashes are more prominent, so easier to read, easier to click, entries as a whole are easier to copy-paste (only one triple-click to select all, no mouse move required), and also easier to read when copy-pasted (now that the parenthesis are back). While talking about the views, please allow me some Compact View bashing: Compact View looks plain and elegant, I agree. But whenever I view the timeline, I'd either like to SEE THE HASHES (to check-out or merge a specific version), or CLICK THE HASHES (to view the check-in details and diffs). I never just browse my comments ;-) With Compact View, I need to do a not-so-intuitive click on the comment (that has always been plain text to me) to make the hash appear, move the mouse pointer over the unpredictable position of the now visible hash, and finally click the hash. But if I get it right (without building and trying the latest version) there's a config option to set the default view to Verbose View (without having to explicitly select it and store the option as a cookie). Verbose View definitely works best, for me. Rolf Ade: > It's mixing completely different things and levels ... I don't think so. Adding the CSS to display the parenthesis means maintaining your own custom skin? Warren Young: > $ fossil conf export skin foo > …upgrade Fossil, switch repo to skin you wish to merge… > $ fossil conf merge foo Sounds easy. But on shared web hosting, with limited shell access, and a total of 10-15 repositories, this already makes me edgy. I have also tried using a shared skin loaded from a directory using the "skin:" CGI control line, but then the skin is missing on local machines. Richard Hipp: > (2) The details of each entry are shown inside parentheses in the > Verbose and Compact views. Thank you very much for this change. With "physical" (as opposed to "spiritual-CSS") parenthesis, readability is also fine when copy-pasting timeline entries. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Metadata in Timeline Verbose View
Thanks for the instructions and demonstration. Simple to do, I agree. A minor issue for me is that I have some tasks where I copy-paste parts of the timeline view to another program, and some web browsers do not copy the CSS parenthesis, making things harder to read, again. Picking individual entries from the command line timeline interface is not so straight forward. But you got me on the wrong leg: I'm a computer hobbyist, and I "refused" to use version control systems for a long time, due to its complexity, and the amount of hobby time I would have to invest. This has changed only when I learned about Fossil, and I'm always fascinated by its simplicity. At first I maintained my own Fossil skins. But after two or three updates, even with only simple changes to the default CSS, I gave up my own skins, as I didn't want to spend my hobby time with CSS diffing. So my strategy here was to argue for a generally accepted default that would be built into Fossil ;-) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Metadata in Timeline Verbose View
Yes, Modern View and Columnar View already have a good demarcation due to spatial distance, borders, and different styles. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Metadata in Timeline Verbose View
Thanks for the recent timeline changes. Most of my check-in comments do not end with a dot. In the timeline web view, it used to look like this: [xx] This is the check-in comment (user: username, tags: trunk) For the current development version in Verbose View, it looks like this: [xx] This is the check-in comment user: username, tags: trunk I found it easier to distinguish metadata from comments in the first version. Even if check-in comments are terminated with a dot, there's not the same clear visual demarcation with lowercase letters. So please allow me to vote for metadata being put in parenthesis, again, for the Timeline Verbose View. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] enhanced-symlink branch
Andy Goth: > Does anyone know a good way to create *.lnk files from Tcl? There's an ancient SHORTCUT.EXE command line utility: https://ss64.com/nt/shortcut.html It even has an -s command line option to disable link tracking. I don't have the original. A few clones can be found in the web, but I haven't come across an up-to-date version that allows access to modern Shell Link properties, such as PKEY_AppUserModel_ID to control taskbar icon grouping, and the like. Also, the venerable Windows Scripting Host can manipulate Shell Links by simple JS or VBS scripts, but only the basic properties: https://ss64.com/vb/shortcut.html --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] enhanced-symlink branch
Warren Young: > By that reckoning, I’d rank *.lnk above Cygwin symlinks in many > regards. Why wouldn’t that work? Handling Windows Shell Links (*.lnk) can be tricky: There's built-in logic to resolve links to missing targets (called "link tracking", can be disabled). That's why portability of *.lnk files may be limited across machines, and Fossil would have to keep the link information separately, and create *.lnk files as needed. https://technet.microsoft.com/en-gb/magazine/2009.10.windowsconfidential.aspx Also, there's other *.lnk file options that may affect portability: https://msdn.microsoft.com/en-us/library/windows/desktop/bb762540.aspx --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Ignoring files/directories in automated zip files?
> To set-up aliases, visit the Setup/URL_Aliases page. Thanks, that's a very handy feature! Would you consider enabling "rewriting" of the default webpage URLs, i.e. by searching the alias list prior to the dispatch table? This could be controlled by a global setting (potential subtle user trap), or on a per-alias basis with a special syntax like !ALIAS or ALIAS! (in analogy to X! to drop CGI variables), but probably only for exact matches to avoid redirection loops. This would be useful for example to set default query parameters for the /timeline webpage without modifications to the skin: !/timeline → /timeline?y=all=50 or: /timeline! → /timeline?y=all=50 Original suggestion: https://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg25757.html --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Default timeline query parameters
Patch to patch: the mprintf() is redundant.
--
Index: src/cgi.c
==
--- src/cgi.c
+++ src/cgi.c
@@ -992,11 +992,11 @@
/* Apply default query parameters for the /timeline web page */
if( strlen(z)==0 ){
z = (char*)P("PATH_INFO");
if( fossil_strcmp(z,"/timeline")==0 ){
- z = mprintf("%s",db_get("timeline-default-query",""));
+ z = db_get("timeline-default-query","");
add_param_list(z,'&');
}
}
z = (char*)P("REMOTE_ADDR");
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Default timeline query parameters
Following is a patch (against 9e48dad49b) to add a new option to the
/setup_timeline web page to specify the default query parameters for
the /timeline web page, that works without modifications to the web UI
skins.
To keep it simple and avoid code duplication, the QUERY_STRING
variable is processed in cgi_init(), but this may not be the right
hook point. Also, I'm not sure if this covers all the possible web
server entry points.
Please ignore it if it does not meet Fossil's requirements regarding
feature set and/or code quality, but it may still be a starting point
for somebody to model a more compliant version, if it's of general
interest.
--Florian
--
Index: src/cgi.c
==
--- src/cgi.c
+++ src/cgi.c
@@ -987,10 +987,19 @@
z = (char*)P("QUERY_STRING");
if( z ){
z = mprintf("%s",z);
add_param_list(z, '&');
}
+
+ /* Apply default query parameters for the /timeline web page */
+ if( strlen(z)==0 ){
+z = (char*)P("PATH_INFO");
+if( fossil_strcmp(z,"/timeline")==0 ){
+ z = mprintf("%s",db_get("timeline-default-query",""));
+ add_param_list(z,'&');
+}
+ }
z = (char*)P("REMOTE_ADDR");
if( z ){
g.zIpAddr = mprintf("%s", z);
}
Index: src/configure.c
==
--- src/configure.c
+++ src/configure.c
@@ -94,10 +94,11 @@
{ "background-mimetype",CONFIGSET_SKIN },
{ "background-image", CONFIGSET_SKIN },
{ "timeline-block-markup", CONFIGSET_SKIN },
{ "timeline-max-comment", CONFIGSET_SKIN },
{ "timeline-plaintext", CONFIGSET_SKIN },
+ { "timeline-default-query", CONFIGSET_SKIN },
{ "adunit", CONFIGSET_SKIN },
{ "adunit-omit-if-admin", CONFIGSET_SKIN },
{ "adunit-omit-if-user",CONFIGSET_SKIN },
#ifdef FOSSIL_ENABLE_TH1_DOCS
Index: src/setup.c
==
--- src/setup.c
+++ src/setup.c
@@ -1520,10 +1520,19 @@
"timeline-max-comment", "tmc", "0", 0);
@ The maximum length of a comment to be displayed in a timeline.
@ "0" there is no length limit.
@ (Property: "timeline-max-comment")
+ @
+ entry_attribute("Default timeline query parameters", 12,
+ "timeline-default-query", "tdq", "", 0);
+ @ The default query parameters handed to the "/timeline" web
+ @ page for requests not otherwise including any query parameters.
+ @ Query parameters can be copied from the URL generated by the
+ @ timeline filter menu. The default is "y=all=50".
+ @ (Property: "timeline-default-query")
+
@
@
@
db_end_transaction(0);
style_footer();
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Minor markdown glitches
Maybe not the best idea ☹ but I was taking notes about special characters in batch files on a Fossil markdown/wiki page. It's even worse than what you can imagine, it seems like batch file syntax was designed to stress-test markdown parsers: ! " % & ( ) , ; ^ | < > I enclosed each character inside `code` markup, but I have not been able to escape the pipe symbol: `|` → ...``... (paragraph divided by table) `\|` → \| (escaping backslash not removed) Also, the following markdown construct -- * The first list: ... * The second list: `|` `<` ... -- causes a crash when fed to `fossil test-markdown-render' or pasted into a wiki page. Tested with Fossil version 2.3 [f7914bfdfa] 2017-07-21 03:19:30 UTC on Windows 7 64-bit (downloaded from the Fossil website). --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Lost "reparented" check-ins after rebuild
> Reparenting is for doing one-time major surgery operations on a Fossil > repo, not for everyday use. Thank you very much for your comments, and for your suggestions about dealing with unwanted check-ins. I take great care to keep my Fossil repositories intact, and no such thing as a "reparent" tag will ever make it there. I have multiple levels of backups, and new commits often go to some temporary repository, with a scripting solution (basically `fossil zip' from old / unzip / check-in on new) to import ranges of check-ins from the temporary to the permanent repository (so that all later amendments are already there right from the beginning). With these short-lived temporary repositories, I sometimes try new things about Fossil, and I found it would be okay to report that I have a case of the rescue "one-time major surgery operation" behaving somewhat unexpected. But I agree it's not "everyday use". After a few more tests, I still haven't been able to find a simple reproducible test case. But following is a modification to my previous (Windows batch) script that produces disjoint timeline rails in the web view, by means of branch hiding. Though, not sure if it is related to my original problem. fossil init test.fossil fossil open test.fossil echo m1.1>sample.txt @echo *** & pause fossil add sample.txt fossil ci -m m1.1 --tag m1.1 --branch br1 --no-warnings fossil amend m1.1 -m m1.1-edited echo m1.2>sample.txt @echo *** & pause fossil ci -m m1.2 --tag m1.2 --no-warnings fossil amend m1.2 -m m1.2-edited echo m1.3>sample.txt @echo *** & pause fossil ci -m m1.3 --tag m1.3 --no-warnings fossil amend m1.3 -m m1.3-edited echo m1.4>sample.txt @echo *** & pause fossil ci -m m1.4 --tag m1.4 --no-warnings fossil amend m1.4 -m m1.4-edited echo m1.5>sample.txt @echo *** & pause fossil ci -m m1.5 --tag m1.5 --no-warnings fossil amend m1.5 -m m1.5-edited fossil co m1.1 echo m2.1>>sample.txt @echo *** & pause fossil ci -m m2.1 --tag m2.1 --branch br2 --no-warnings fossil amend m2.1 -m m2.1-edited @echo *** & pause fossil merge br1 --integrate fossil ci -m m2.2 --tag m2.2 --no-warnings fossil amend m2.2 -m m2.2-edited echo m2.3>>sample.txt @echo *** & pause fossil ci -m 2.3 --tag m2.3 --no-warnings fossil amend m2.3 -m m2.3-edited rem fossil reparent m1.5 m1.3 @echo *** & pause fossil merge m1.4 fossil ci -m m2.4 --tag m2.4 --no-warnings --allow-conflict fossil amend m2.4 -m m2.4-edited fossil tag add --raw --propagate hidden m1.2 --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] User-defined config values
Thank you very much for your reply. I have found two simple ways to plant a `default-homepage' tag on unversioned wiki pages, so that the script to update the wiki pages can just search for the tag in the output of `fossil uv cat': 1. ... 2. default-homepage But according to the Fossil Wiki Formatting Rules [0], the second variant would not be allowed: "... any attributes that specify javascript or CSS are elided." [0] https://www.fossil-scm.org/index.html/wiki_rules --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] User-defined config values
I have a few scripts to maintain my repositories, and among other things, they add (and update) multiple flavors of (unversioned) wiki homepages (the remote homepages have more links, such as login/logout/index, and the local ones show a reminder about syncing). Some repositories have customized wiki homepages, and for these I would like to skip the updates of the default homepages. It seems like a simple solution to add a custom value to Fossil's `config' table to indicate whether or not the homepages are customized. Is this a safe and encouraged approach? Is it harmless but discouraged, or even harmful? Would such custom values be excluded from syncing? This task seems more complicated to do with the `fossil uv' interface, as it doesn't allow to query the hashes or time stamps of unversioned files. Also, not sure if wiki pages allow embedded comments (standard HTML comments do not seem to work?) that could be searched for custom settings? Another approach I see is keeping custom configuration data in unversioned files of their own. Or does anybody know of a better method? --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Accelerate multi-repository queries
Zakero: > Did you check the execution time of the "find | sort" versus the > "listfossil()"? Looking at your script, I think you can simplify it > a bit. The script takes between 1.2 and 1.8 seconds in the original form, and between 130 and 160 ms if the `listfossil()' function is simplified to just output the repository name (and not run the `./fossil --sql ...' part). > If using bash, every function call can spawn a sub-shell as well as > "| while read". My knowledge on this may be outdated. The above will > only spawn 1 sub-shell. By using "ls", the files will be sorted by > name so the "find | sort" is not needed. All the repositories are > passed at once, removing the need for the "| while read". Note that > this will break if the length of all the filenames goes beyond the > bash arg limit. This limit is between 1K and 8K characters, don't > remember exactly. Thanks for the interesting information. I have switched from `find -exec' to `| while read' to avoid sub-shells (with custom functions), but your approach with `for' seems to be another 10-20 ms faster (I could even omit the custom function, as the program structure is much simpler). When replacing the calls to `./fossil' with `./sqlite3' (without any further optimizations to the script), execution time drops from the initial 1.2-1.8 seconds to 300-370 ms. So it seems that Fossil has to perform more initialization work than the plain SQLite CLI program. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Accelerate multi-repository queries
On another server, where I have access to PHP, it also takes less than 100 ms to query the repository files using the built-in PDO/sqlite interface. So it seems that running ./sqlite3 instead of ./fossil could speed up the queries. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Load customized skin from subdirectory on Windows
Thank you very much for the quick fix, it also works fine on my system. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Accelerate multi-repository queries
I'm using a customized repository index page to display a link to each
repository, along with the project name and description queried
directly from the repository database files.
For 10-20 repositories, the following (skeleton) shell script takes
more than 1 second to finish:
#!/bin/sh
...
listfossil() {
echo "$(basename "$1")"
cat <<'SQL' | ./fossil sql --repository "$1"
SELECT value FROM config
WHERE name IN ('project-name', 'project-description')
ORDER BY name DESC LIMIT 2;
SQL
}
find . -name "*.fossil" -type f -print | sort | \
while IFS= read -r file; do listfossil "$file"; done
Is there any way to speed up this script, i.e. by loading a :memory:
or :temp: database instead of a real repository, attaching to each of
the repositories and reading all the desired values in one single SQL
query?
Another idea in this context might be that the built-in "repolist"
allowed some customization, i.e. to display the project name and
description, possibly even with a customized header/footer
HTML/wiki/markdown file loaded from the repository directory (similar
to the display of the README* files in the "File List" web view).
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Load customized skin from subdirectory on Windows
On Windows, the Fossil `--skin LABEL' command line option can't be used to load a custom skin from a subdirectory, probably because only skin labels containing forward slashes but not backslashes are treated as paths to subdirectories containing the skin files: http://fossil-scm.org/index.html/artifact?name=0b6ae3=101 I'm very happy with the built-in Fossil skins, and I avoid customizations as far as possible, so as not having to update the customized skins each time the original template is adapted. One reason for me to use custom skins is to be able to change the default query parameters of the "Timeline" link from the navigation bar. I would highly appreciate a configuration option (similar to `index-page') to set custom default timeline query parameters. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Suggested patch for src/user.c (password prompt)
Two other interesting cases, run from the standard console: > fossil test-prompt-password PROMPT: 0 2>&1 | more INPUT PROMPT:[INPUT] 4. Password is visible while typing. > echo PASSWORD | fossil test-prompt-password PROMPT: 0 2>&1 | more PROMPT:[PASSWORD ] 5. As expected, _isatty() detects that STDERR is not a console, but a pipe, so the input is also read from the pipe using `getc(stdin)'. No harm here either, just for completeness. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Suggested patch for src/user.c (password prompt)
I get the following results for the new `fossil test-prompt-password' command, run in the standard CMD.EXE console on Windows, typing "INPUT[ENTER]" on the keyboard if possible: > echo PASSWORD | fossil test-prompt-password PROMPT: 0 PROMPT:* [INPUT] 1. The default case. > echo PASSWORD | fossil test-prompt-password PROMPT: 0 2>nul [INPUT] 2. No prompt, but ability to type, no * placeholders while typing. Not sure if this is expected, but "nul" is considered to be a console(!), so input is read from the keyboard, and not from the piped `echo' command. > echo PASSWORD | fossil test-prompt-password PROMPT: 0 2>error.log [PASSWORD ] > type error.log PROMPT: 3. Prompt goes to redirected file instead of console (expected), no possibility to type, but password from redirected standard input (the piped `echo') is accepted (that is, "error.log" is not considered to be a console, so `getc(stdin)' is used instead of `_getch()'). I agree that my examples are all quite stupid, and I can't think of any serious harm to be caused like this. But my point was that testing whether STDERR is a console handle does not automatically imply that STDIN is also a console handle. Moreover, _isatty() (UCRT 10.0.14393.0 in this case) yields different results for the "2>nul" and "2>error.log" redirects, but neither is a console. That's why I think _isatty() is not reliable, and GetConsoleMode() should be preferred. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Suggested patch for src/user.c (password prompt)
Just because I have done some research about this topic recently, and find it interesting, please allow me to ask why you can make any assumptions regarding STDIN, after only testing STDERR? Couldn't each of the standard handles be redirected individually, as in "COMMAND nul 2>errors.log"? To distinguish console handles from files, pipes and other redirected devices on Windows, MSDN recommends using GetConsoleMode(), as for example mentioned in the documentation for WriteConsole(): https://msdn.microsoft.com/en-us/library/windows/desktop/ms687401.aspx "... determine whether the output handle is a console handle (one method is to call the GetConsoleMode function and check whether it succeeds) ..." A more detailed test from Michael Kaplan, similar to earlier versions of MSVCRT (current UCRT seems to rely on a combined test of isatty(), GetFileType() and GetConsoleMode(), now): http://archives.miloush.net/michkap/archive/2008/03/18/8306597.html A comment by Raymond Chen why GetFileType() alone (as used by some isatty() CRT implementations) is not sufficient to detect a console handle: https://blogs.msdn.microsoft.com/junfeng/2005/07/08/how-do-i-know-if-console-handle-is-redirected/ My investigations were about output handles, but GetConsoleMode() can also be used with input handles. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Submit web forms by hitting enter
I noticed it's no longer possible to submit the form on top of the
timeline web view by hitting enter if one of the text input elements
("Max" or "Tag Filter") is focused in IE.
It seems that (all?) web browsers disable this logic for forms with
multiple text input elements lacking a "submit" element. The fix was
to add an "onchange" handler [0], but unfortunately this handler is
only triggered if the text input element lost focus, and not by
hitting enter in IE.
[0] http://www.fossil-scm.org/index.html/info/2929d5fced0909b5
Adding a hidden "submit" element would work, but might raise security
warnings in some web browsers. Conditional comments to make a hidden
"submit" element only visible for IE are no longer supported [1].
[1] https://msdn.microsoft.com/en-us/library/hh801214.aspx
I would like to suggest adding a new "onkeyup" handler to the text
input elements on the timeline web view to submit the form if enter is
pressed.
I'm sending a patch (as a Fossil bundle) directly to Mr. Hipp.
--Florian
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] "CGI" command and argc
> Therefore, I would expect: > env "-S perl -e 'print qq([$_] ) for (@ARGV)' a b c" > to output: > [a] [b] [c] Your example works on my FreeBSD server if the double quotes are omitted, and the command is run from a shell: env -S perl -e 'print qq([$_] ) for (@ARGV)' a b c ==> [a] [b] [c] But for a CGI script (named ./sample.cgi), it looks like this: #!/usr/bin/env -S perl -e 'print qq([$_] ) for (@ARGV)' a b c ==> [a] [b] [c] [./sample.cgi] The CGI script name is appended as an extra command line argument. So for the Fossil CGI command, it looks like this: #!/usr/bin/env -S ./fossil cgi fossil.config ==> no such file: cgi > If you have access to a compiler (gcc, clang) on your shared host, > I would recommend that solution. Unfortunately I don't, but for a short moment I was considering to use the custom libraries trick with clang or gcc ... ;-) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] "CGI" command and argc
Thanks for the link and the additional information. I have stumbled upon a website with meticulous research about the unspecified shebang behavior across a wide range of systems: https://www.in-ulm.de/~mascheck/various/shebang/ There's indeed so many variants that it may not be a good idea to do any modifications, here. Of course the portable solution you mentioned is also fine (performance is no issue on my stone-age shared host with just a handful of personal repositories). > So, I would expect both of the following to work: > #!/usr/bin/env -S fossil2/fossil cgi fossil.config > #!/user/bin/env -S fossil1/fossil cgi fossil.config No, they don't, as the CGI script name is appended as an extra argument to the shebang command line, causing Fossil to leave the path with the explicit "CGI" command. I was suggesting that Fossil keep going the explicit "CGI" command path even if there's more than three command line arguments. http://fossil-scm.org/index.html/artifact/5105d4bc1b?ln=1834 --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] `fossil ui` startup delay
On my Windows XP machines (one is a standard installation, the other a highly tweaked system), I noticed a considerable timeout for the web browser to appear when `fossil ui` is invoked. The delay seems to origin from StartServiceCtrlDispatcherW(), called from win32_http_service() [0] inside cmd_webserver() [1], the latter of which handling both the `fossil ui` and `fossil server` commands. The timeout occurs regardless of whether the Fossil service (`fossil winsrv`) is started, stopped, or not registered at all. Modifying the `ServicesPipeTimeout` registry setting [2] as an attempt to have StartServiceCtrlDispatcherW() abort earlier has no effect. [0] http://fossil-scm.org/index.html/artifact/a3970bdc0b?ln=642-669 [1] http://fossil-scm.org/index.html/artifact/5105d4bc1b?ln=2532 [2] https://support.microsoft.com/en-us/kb/922918 Each new `fossil ui` server that is not able to bind to the default port 8080 takes the next available port. This is also true if the Fossil service (`fossil winsrv`) is up and running, in this case `fossil ui` (or `fossil server`) brings up a new server listening to a new port. So these facilities do not seem to be linked, i.e. the `winsrv` service does not seem to be "recycled" for the `ui` and `server` commands. Also, the comment to win32_http_service() [3] states that if Fossil was running in a interactive console session, the routine would fail, and if I get it right, this is most likely true if Fossil is run with either the `ui` or `server` command. So I would like to suggest to only issue the win32_http_service() call from cmd_webserver() if `isUiCmd` evaluates to false. Maybe the checks can be omitted entirely, so that `fossil server` will be unlinked from waiting for `fossil winsrv` as well. [3] http://fossil-scm.org/index.html/artifact/a3970bdc0b?ln=637-640 Do you think this makes sense, and could be changed in Fossil? Or am I missing something important? A patched version works fine on Windows XP, the web browser shows up instantly. I've seen no problems on Windows 7 and Windows 10 with the modified version, so far. A workaround for me is to start `fossil ui` only once in a separate console (preferably with a repository name indicated, so closing the repository won't interfere with the server) and leave it running in the background, then switch to the web browser and hit F5 as necessary each time I want to see new check-ins. But running `fossil ui` from the command line is a lot more straightforward, as it navigates directly to the current check-in, without the need to switch windows and refresh pages. I must admit, submitting change requests for Windows XP may seem a little odd, I agree, but it's the wide availability of Fossil that helps me archiving my projects from my old PCs, to be able to move on, one day, maybe. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] "CGI" command and argc
I'm currently stuck on a shared web host running FreeBSD 9.1 without any C compilers available. I'm still able to run the latest version of Fossil from [0] with some quirks (see bottom if you're interested in details). [0] http://pkg.freebsd.org/ I have created an extra CGI script to start playing with Fossil 2.1, and decided to offload the CGI options from the former CGI script to a separate control file (named `fossil.config`), i.e. from: #!./fossil directory: ../.data/fossil ... to: #!./fossil cgi fossil.config Like this, the two CGI scripts for Fossil 1.x and Fossil 2.1 can share the same CGI server setup. However, I noticed that my custom `fossil.config` file was not loaded with the above CGI script, but only this version worked: #!./fossil fossil.config The CGI script name is appended as an extra argument to the shebang command line, and Fossil refuses to read the control file if the "CGI" command is followed by more than one argument [1]. The fallback is to use `g.argv[1]` in this case. [1] http://fossil-scm.org/index.html/artifact/5105d4bc1b?ln=1834 If I may I would like to suggest that the control file passed to the "CGI" command be read even if there's additional command line arguments following, so it works the same as if the "CGI" command was omitted (and CGI mode was triggered by the GATEWAY_INTERFACE environment variable). I feel that explicitly passing the "CGI" command followed by the control file (as documented in the Fossil help) is better for later reference and script maintenance than relying on the "Fossil thinks it's reading its own CGI script but I tricked it" fallback. To get Fossil 2.1 running on my FreeBSD 9.1 shared host, I downloaded Fossil 2.1 and the OpenSSL libraries for FreeBSD 11 from [0], and grabbed a copy of libc.so.7 from a FreeBSD 11 ISO image (for `strchrnul`). These files were saved in a separate ./fossil2 directory, and the symlinks libcrypto.so.9 -> libcrypto.so.7 and libssl.so.9 -> libssl.so.7 created. Now Fossil 2.1 works fine with the following one-line CGI script: #!/usr/bin/env -S LD_LIBRARY_PATH=fossil2 fossil2/fossil fossil.config I'm always amazed about the simplicity and modest system requirements of Fossil, thanks for this masterpiece! --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Command-line output format
Citation from: http://www.mail-archive.com/fossil-users@lists.fossil-scm.org/msg24841.html > ... What can we do to help you move away from scripts that depend > on the details of command-line output and toward something that is > more likely to survive an update? ... Would it be better to run > SQL queries directly against the repository database? ... Are new > fixed-output-format commands needed in Fossil to extract > information that is important to scripts? I'm working with a lot of scripts to process Fossil command-line output. The JSON interface is not available with some of the precompiled binaries I'm using, and parsing JSON from shell scripts or Windows batch files seems not so trivial. Running SQL queries directly would mean to reinvent the wheel and perform complex operations in many cases, as i.e. the output of `fossil whatis [check-in]` involves parsing and integrating all amendments to a check-in (or thorough knowledge of the internal Fossil storage format if this information is cached in some table). Therefore, I would like to vote for the command-line output to remain as stable as possible, make the suggested "fixed-output-format" the default, and carefully document modifications to the command-line output format in the Fossil change logs. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Synchronizing endless loop
With the help of the excellent documentation on the Fossil Sync Protocol I have found a very simple solution to fix synchronization of unversioned files. Due to how work is distributed between the server and the client, it's correct that the server ignores "uvigot" cards, but the client should stop sending them. http://www.fossil-scm.org/index.html/doc/trunk/www/sync.wiki I would like to suggest a solution in prosaic form, so you can add it to Fossil without any copyright concerns, if you think it's useful: In the client synchronization code, merge the test for the break condition whether the send limit has been reached with the entry test for the `while' loop, so that the loop is only entered if the size of the output blob is LESS THAN the send limit AND `db_step()' returns SQLITE_ROW. http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=1934 Like this, send_unversioned_file() is only called by the client if there's room for one more file, and no "uvigot" cards will be generated. By not entering the loop, the skipped unversioned file remains in the `uv_tosend' table to be sent later, and `uvDoPush' remains 1 as `db_step()' is not called before assuring there's enough room. Testing if the output blob size is LESS THAN the send limit does not allow the output blob to grow as large as the send limit, whereas before it was allowed to grow up to the send limit. But `send_unversioned_file()' takes the "uvigot" path if the output blob is GREATER THAN OR EQUAL the send limit, so the changed `while' condition uses the same limits as `send_unversioned_file()'. I'm not sure however if "max-upload" can be assumed to always be greater than zero. Through the web interface, "max-upload" can be set to a negative number, so it may be safe to make sure "max-upload" is greater than zero when reading the value from the configuration. http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=1745 --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Synchronizing endless loop
Andy Bradford: > This error would seem to indicate that your client sync code > doesn't actually know what the uvigot card is Indeed, if I get it right from following the code paths, there are separate functions to handle synchronization for the client: http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=1681-1689 and for the server: http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=1127-1140 The client code is processing the "uvigot" card: http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=2113-2183 The counterpart for the server handler is missing (that's where the "server says: bad command" message comes from), but that's probably intentional, as inferred from this comment: http://www.fossil-scm.org/index.html/artifact/0d5ab26abb?txt=1=682-685 The `send_unversioned_file()' function should only generate "uvigot" cards on the server (that is, for the client), but the function is also called from the client sync code (generating "uvigot" cards for the server), without discrimination of client or server mode. I have rebuilt the Fossil client binary with an extra flag passed to this function to indicate whether "uvigot" cards are desired or not. The error message is gone, but unversioned files are not synchronized at all, that way. So I feel that the server should handle "uvigot" cards as well, and indicate the unversioned files that the client should transfer. Let's see if I dare diving any deeper ... ;-) --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Synchronizing endless loop
I have another observation regarding the failed synchronization of unversioned files: If the `max-upload' setting is tweaked to be large enough (I set it to the default 25*1000 on both the server and the client) so that the synchronization operation does not require multiple round-trips, unversioned files are synchronized properly. There are only two artifact receipts log entries, one containing all the committed files and check-in metadata, and another one for the unversioned file. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Synchronizing endless loop
Thank you very much for looking into my issue! I have built the server from source code with `./configure; make'. I haven't run `make clean' as I was working inside a new directory containing just the extracted Fossil source code. I have now replaced the server binary with the one matching my OS from http://pkg.freebsd.org: Fossil version 1.36 [c24373934d] 2016-10-24 14:59:33 UTC Compiled on Oct 28 2016 19:32:28 using gcc-4.2.1 20070831 patched [FreeBSD] (64-bit) The version hash is the same as for my previous server binary, and also matches that of my Windows client executable (the official download from fossil-scm.org). But duh, I need a faster Internet connection ... On my slower home line, I have always cancelled my tests after around 100 or so iterations of the synchronization operation. Today, on my ultra-fast line at work, I noticed that the `sync -u' operation completes after 130 iterations, but unversioned files are NOT synchronized. The artifact receipts log shows 130 corresponding entries, most of them consisting of only a single artifact. If the synchronization is started without `-u', there are only 18 iterations, and 18 corresponding artifact receipts log entries for the same commit. Each artifact receipts log entry has multiple (usually 5-10) artifacts. So this is a somewhat inflationary use of the term "endless loop" for my part, I'm sorry for my impatience and not waiting for my tests to finish! Updated problem summary === When adding and committing a lot of data, the synchronization operation invoked with `fossil sync -u' emits a "server says: bad command: uvigot ..." for each round-trip, and unversioned files are NOT synchronized. Moreover, the synchronization operation takes many more round-trips to complete, and most of the corresponding artifact receipts log entries consist of only one single artifact. Updated steps to reproduce == * Use Fossil version 1.36 [c24373934d] on the server and the client * Create a new empty repository on the server * Clone the empty repository from the server to the client * Add and commit a lot of files (try the Fossil /src directory) * Add an unversioned file * Run `fossil sync -u' --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
[fossil-users] Synchronizing endless loop
Hello I have just discovered a situation where Fossil seems to enter an endless loop when synchronizing a lot of commit data and some unversioned files. I'm not sure about the exact size limit, but probably large enough to trigger multiple round-trips. Steps to reproduce: Server == FreeBSD 9.1-RELEASE amd64 Fossil version 1.36 [c24373934d] 2016-10-24 14:59:33 UTC Compiled on Oct 25 2016 14:06:50 using clang-3.4.1 (tags/RELEASE_34/dot1-final 208032) (64-bit) * Create a new empty fossil repository on the server (I named it `test-repo') Client == Windows XP SP3 (32-bit) and Windows 7 SP1 (32-bit) Fossil version 1.36 [c24373934d] 2016-10-24 14:59:33 UTC Compiled on Oct 24 2016 13:12:30 using mingw32-3.18-gcc-4.5.0 (32-bit) * Clone the repository from the server to the client * Add and commit a lot of files (try the Fossil /src directory) * Add an unversioned file (I named it `home.wiki') * Run `fossil sync -u' Client Output = [Password prompt ...] Sync with http://some.fossil.server/fossil.cgi/test-repo Round-trips: 2 Artifacts sent: 3 received: 0 Error: bad command: uvigot home.wiki 1477722315 ecb1c59ef204582770184dfc6ddc3f7451224c94 416 Round-trips: 3 Artifacts sent: 3 received: 0 server says: bad command: uvigot home.wiki 1477722315 ecb1c59ef204582770184dfc6ddc3f7451224c94 416 Round-trips: 4 Artifacts sent: 4 received: 0 server says: bad command: uvigot home.wiki 1477722315 ecb1c59ef204582770184dfc6ddc3f7451224c94 416 Round-trips: 5 Artifacts sent: 5 received: 0 server says: bad command: uvigot home.wiki 1477722315 ecb1c59ef204582770184dfc6ddc3f7451224c94 416 Round-trips: 6 Artifacts sent: 6 received: 0 server says: bad command: uvigot home.wiki 1477722315 ecb1c59ef204582770184dfc6ddc3f7451224c94 416 [...] The displayed hash is the checksum of the unversioned file (verified with a separate sha1 utility). Shutting down Fossil at this point does not seem to cause any harm to the local or remote repository, and the problem can be worked around by running `fossil sync' first, and then `fossil sync -u' in a second step, so I would consider this to be only a minor issue. --Florian ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
