[fossil-users] Autosync hangs
Hi guys, I'm enjoying playing with Fossil, but I've run into a problem... I have a satellite Internet connection. Apart from the incredible latency (over 1 second, testing with ping), my connection freezes every now and again. This freezing problem has something to do with communication with the satellite, so it's not just my system. What happens is that the Internet is dead for awhile, maybe a minute, maybe longer. Anyway, I did a 'fossil commit' just as one of these freezes occurred, and I got this: # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 The freeze was temporary, a matter of seconds later the Internet was working again. However the 'fossil commit' operation was hung. I had to do a CTRL-C to terminate it. My question: Fossil should not be assuming a perfect Internet connection should it? Shouldn't a push (or pull) have a timeout, and maybe a retry? Just to hang is not very good. Then there's the question, as I had to terminate it with a CTRL-C, is there any kind of assurance that what did arrive at the remote repository, if anything, is not partial and thus corrupted? If I use a program like wget, it does have fallback code to handle this kind of situation. Regards, Barry Kauler ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Autosync hangs
On Nov 13, 2009, at 7:30 AM, Barry Kauler wrote: Continuing from my previous post: Continuing from before, I attempted a second time, this time the Internet connection is good: # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 Received: 230 5 0 0 Total network traffic: 323 bytes sent, 466 bytes received fossil: nothing has changed I found that file PKGS_MANAGEMENT was not added to the repository, so the failed 'commit' had also cleared the 'add'. There are two autosyncs on a commit. The first autosync is really just a pull. It checks to see if somebody else has already committed changes against the same version you are committing against and hence that you are about to fork. If the first autosync fails, the commit does not occur. It really shouldn't cancel your add (that seems like a bug) but at the same time, information you have previously committed should remain intact. The second autosync is a push that sends the newly committed content back up to the remote repository. The second autosync occurs after the commit was successful and so if the second autosync fails, it should be sufficient to simply rerun sync to push the contents again. So I had to do the whole thing again, and this time it worked: # fossil add PKGS_MANAGEMENT ADDED PKGS_MANAGEMENT # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 Received: 230 5 0 0 Total network traffic: 322 bytes sent, 466 bytes received New_Version: 8db611a1f647f81834a5df1b7a59959c80a25f5a Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send:8663 9 2 0 Received: 322 7 0 0 Total network traffic: 4649 bytes sent, 514 bytes received Regards, Barry Kauler On 11/13/09, Barry Kauler bkau...@gmail.com wrote: Hi guys, I'm enjoying playing with Fossil, but I've run into a problem... I have a satellite Internet connection. Apart from the incredible latency (over 1 second, testing with ping), my connection freezes every now and again. This freezing problem has something to do with communication with the satellite, so it's not just my system. What happens is that the Internet is dead for awhile, maybe a minute, maybe longer. Anyway, I did a 'fossil commit' just as one of these freezes occurred, and I got this: # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 The freeze was temporary, a matter of seconds later the Internet was working again. However the 'fossil commit' operation was hung. I had to do a CTRL-C to terminate it. My question: Fossil should not be assuming a perfect Internet connection should it? Shouldn't a push (or pull) have a timeout, and maybe a retry? Just to hang is not very good. Then there's the question, as I had to terminate it with a CTRL-C, is there any kind of assurance that what did arrive at the remote repository, if anything, is not partial and thus corrupted? If I use a program like wget, it does have fallback code to handle this kind of situation. Regards, Barry Kauler ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users D. Richard Hipp d...@hwaci.com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Autosync hangs
Dr Hipp, Thanks very much for the detailed explanation! Ok, it is probably best in my situation if I leave autosync permanently off. Your clarification about the inherent integrity of the database has greatly reassured me. One extra question: You have mentioned about possible schema changes in the database in a later version of Fossil. Ok, I have my online repository. If someone installs Fossil by downloading your latest binary executable, and then clones my repository, isn't there a potential problem, if I have been working with an older Fossil executable? Regards, Barry Kauler On 11/13/09, D. Richard Hipp d...@hwaci.com wrote: On Nov 13, 2009, at 7:30 AM, Barry Kauler wrote: Continuing from my previous post: Continuing from before, I attempted a second time, this time the Internet connection is good: # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 Received: 230 5 0 0 Total network traffic: 323 bytes sent, 466 bytes received fossil: nothing has changed I found that file PKGS_MANAGEMENT was not added to the repository, so the failed 'commit' had also cleared the 'add'. There are two autosyncs on a commit. The first autosync is really just a pull. It checks to see if somebody else has already committed changes against the same version you are committing against and hence that you are about to fork. If the first autosync fails, the commit does not occur. It really shouldn't cancel your add (that seems like a bug) but at the same time, information you have previously committed should remain intact. The second autosync is a push that sends the newly committed content back up to the remote repository. The second autosync occurs after the commit was successful and so if the second autosync fails, it should be sufficient to simply rerun sync to push the contents again. So I had to do the whole thing again, and this time it worked: # fossil add PKGS_MANAGEMENT ADDED PKGS_MANAGEMENT # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 Received: 230 5 0 0 Total network traffic: 322 bytes sent, 466 bytes received New_Version: 8db611a1f647f81834a5df1b7a59959c80a25f5a Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send:8663 9 2 0 Received: 322 7 0 0 Total network traffic: 4649 bytes sent, 514 bytes received Regards, Barry Kauler On 11/13/09, Barry Kauler bkau...@gmail.com wrote: Hi guys, I'm enjoying playing with Fossil, but I've run into a problem... I have a satellite Internet connection. Apart from the incredible latency (over 1 second, testing with ping), my connection freezes every now and again. This freezing problem has something to do with communication with the satellite, so it's not just my system. What happens is that the Internet is dead for awhile, maybe a minute, maybe longer. Anyway, I did a 'fossil commit' just as one of these freezes occurred, and I got this: # fossil commit -m added file PKGS_MANAGEMENT --nosign Autosync: http://bkhome.org/fossil/woof.cgi Bytes Cards Artifacts Deltas Send: 130 1 0 0 The freeze was temporary, a matter of seconds later the Internet was working again. However the 'fossil commit' operation was hung. I had to do a CTRL-C to terminate it. My question: Fossil should not be assuming a perfect Internet connection should it? Shouldn't a push (or pull) have a timeout, and maybe a retry? Just to hang is not very good. Then there's the question, as I had to terminate it with a CTRL-C, is there any kind of assurance that what did arrive at the remote repository, if anything, is not partial and thus corrupted? If I use a program like wget, it does have fallback code to handle this kind of situation. Regards, Barry Kauler ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users D. Richard Hipp d...@hwaci.com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Autosync hangs
On Nov 13, 2009, at 6:19 PM, Barry Kauler wrote: You have mentioned about possible schema changes in the database in a later version of Fossil. Ok, I have my online repository. If someone installs Fossil by downloading your latest binary executable, and then clones my repository, isn't there a potential problem, if I have been working with an older Fossil executable? When I said schema changes I was referring to the auxiliary schema that holds repository metadata in an easily accessible format. To put it another way, I meant the schema that holds derived data ithat is precomputed from the canonical artifacts for speedy access. The content of the database with this schema can *always* be recomputed simply by rescanning all of the underlying immutable artifacts. Call it the secondary schema if you will The secondary schema does change from time to time. When it does and you upgrade to the latest version of Fossil, simply run fossil all rebuild and the repositories will be updated with the new schema. The underlying, canonical artifact representation of the repository does not change, however. It is has been and always will be the same (or at least backwards compatible). The same goes for the wire protocol. Two different versions of Fossil, with different secondary schemas, can still sync against each other. They don't even know what secondary schema the other end is running. Really all they do is exchange artifacts until they both have the same set of artifacts, then they each independently recomputed the derived data for the secondary schema from those canonical artifacts. Fossil is designed in such a way that someone could decide to reimplement the whole thing from scratch using a different technology and the new Fossil would be able to synchronize with the old. (There are some people who have been threatening to reimplement Fossil using Tcl/Tk, for example, but no results on that front yet, at least not that I've heard of.) For that matter, there is nothing in the artifact format or wire protocol of Fossil the requires the use of SQLite, or any other SQL database engine, as the repository format. You could reimplement Fossil using a pile-of-files database (after the fashion of Git or Hg) if you wanted to and make it so that it was 100% compatible with the current SQLite-based Fossil. D. Richard Hipp d...@hwaci.com ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Autosync hangs
On 14/11/2009, at 9:38 AM, D. Richard Hipp wrote: Fossil is designed in such a way that someone could decide to reimplement the whole thing from scratch using a different technology and the new Fossil would be able to synchronize with the old. (There are some people who have been threatening to reimplement Fossil using Tcl/Tk, for example, but no results on that front yet, at least not that I've heard of.) Was that Mark or me? Actually, we were talking about creating a Tcl/Tk binding using Critcl - essentially just turning Fossil into a library that can be called from Tcl rather than by exec'ing the Fossil command for each operation. That might be the gateway to richer user interfaces (perhaps even sitting GitTk on top of Fossil, albeit that might be a bridge too far), plus also a form of megawidgets - scripted encapsulation of workflow. But, as you say, no results yet. Cheers Steve ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
