Re: [fossil-users] Unused artifact view

[Richard Hipp]

On 1/22/18, bytevolc...@safe-mail.net  wrote:
> In rebuild.c (commit 1a95af721e467a6e3321c1557c4f0cdfa353ef4a):
>
> Ln 168:
>
>   db_multi_exec(
> "CREATE VIEW IF NOT EXISTS "
> "  repository.artifact(rid,rcvid,size,atype,srcid,hash,content) AS "
> "SELECT blob.rid,rcvid,size,1,srcid,uuid,content"
> "  FROM blob LEFT JOIN delta ON (blob.rid=delta.rid);"
>   );
>
> It appears that this view is not in use at all. What is this for?

At one point I was going to try to combine the BLOB and DELTA tables
into a single table named ARTIFACT.  This was a stepping stone toward
that goal.
-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Bug Report: Fossil 2.3 runs out of Memory During Check-in

[Kees Nuyt]

[Default] On Mon, 22 Jan 2018 13:54:00 -0700, Warren Young
 wrote:

> Fossil makes some reasonable assumptions about its working
> environment: individual file sizes are a small fraction of
> available VM, the entire repository probably fits into VM,
> and if not, then it can at least get several artifacts into
> VM at once, etc.
> 
> If you’re dealing with artifact sizes a large fraction of
> the size of your virtual memory size, then you’re probably
> asking for trouble with Fossil.

I agree. In other words, Martin could increase the size
of the swap file, and try the checkin again.

-- 
Regards,
Kees Nuyt
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Unused artifact view

[bytevolcano]

In rebuild.c (commit 1a95af721e467a6e3321c1557c4f0cdfa353ef4a):

Ln 168:

  db_multi_exec(
"CREATE VIEW IF NOT EXISTS "
"  repository.artifact(rid,rcvid,size,atype,srcid,hash,content) AS "
"SELECT blob.rid,rcvid,size,1,srcid,uuid,content"
"  FROM blob LEFT JOIN delta ON (blob.rid=delta.rid);"
  );

It appears that this view is not in use at all. What is this for?
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Bug Report: Fossil 2.3 runs out of Memory During Check-in

[Warren Young]

On Jan 22, 2018, at 6:44 AM, Martin Vahi  wrote:
> 
> Fossil had an opportunity to allocate
> at least 1GiB of RAM

There are cases where Fossil needs 2-3x the size of an object to operate.  

For example, my understanding of the way Fossil does the diff operation is that 
it loads the new and old files into memory, and then it creates an output 
buffer which needs to be large enough to hold the differences.

You speak of algorithms, and indeed, you might talk about creating a sliding 
window version of the diff algorithm so that you only need something like 1.2N 
memory, where N is the size of the output buffer, the rest going to the input 
files’ sliding windows, but then we get back to the need for motivating 
examples.

With 1 GB of RAM and presumably some nonzero multiple of that for VM, the 
current Fossil diff algorithm only breaks down when you’re checking in diffs on 
files hundreds of megs in size, which begs the question, “Seriously?”

> In my opinion the correct
> case might be that Fossil should be able to run even
> on the old Raspberry Pi 1 that has 512MiB RAM

It does.  One of my public Fossil projects is based on the Pi, and the Pi B+ 
remains a major development and deployment target.

It’s not surprising that it works well there, since the largest file in this 
project’s Fossil repository is 3.3 MB and the total repository size is 37 MB, 
so 512 MB of RAM and some amount of VM on top of that is plenty for this 
particular application.

I got all of that from my repository’s /stat page.  What does your repository’s 
/stat page show?

> Fossil should just look,
> how much free RAM the computer has

Easier said than done, which is why the C Standard doesn’t have a way to get 
that number.

No single one of the answers to this similar question on Stack Overflow are 
entirely correct:

   https://stackoverflow.com/questions/2513505/how-to-get-available-memory-c-g

When it takes multiple answers which are correct within a fixed scope to come 
up with a proper cross-platform solution, it’s a good bet that you’re probably 
chasing the wrong problem.

(The accepted answer to that question is arguably even wrong.  It’s certainly 
unsuitable for Fossil’s purposes.)

Even if Fossil were to mash up all of that advice into a solution that works 
everywhere, Windows, Linux, and the BSDs (including macOS, in this case) don’t 
all agree on what “free RAM” means.  The BSDs have the concept of “wired” 
memory, which doesn’t exist on the other two.  Windows has non-pageable RAM, 
which the other two don’t, etc.

Then you add in all the other random OSes Fossil runs on, and things get even 
more complicated.

> adjust its algorithm parameters accordingly.

Fossil makes some reasonable assumptions about its working environment: 
individual file sizes are a small fraction of available VM, the entire 
repository probably fits into VM, and if not, then it can at least get several 
artifacts into VM at once, etc.

If you’re dealing with artifact sizes a large fraction of the size of your 
virtual memory size, then you’re probably asking for trouble with Fossil.  
Fossil would have been designed differently if that were the common use case.
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

Re: [fossil-users] Bug Report: Fossil 2.3 runs out of Memory During Check-in

[Richard Hipp]

On 1/22/18, Martin Vahi  wrote:
>
> citation--start---
> Fossil internal error: out of memory
> citation--end-
>
> It happened during the execution of the
>
> fossil ci

Do you have a test case that we can use for debugging?

-- 
D. Richard Hipp
d...@sqlite.org
___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users

[fossil-users] Bug Report: Fossil 2.3 runs out of Memory During Check-in

[Martin Vahi]

citation--start---
Fossil internal error: out of memory
citation--end-

It happened during the execution of the

fossil ci

Given that the Fossil had an opportunity to allocate
at least 1GiB of RAM without running out of RAM,
the issue must have something to do with the
algorithm. In my opinion the correct
case might be that Fossil should be able to run even
on the old Raspberry Pi 1 that has 512MiB RAM
in total and Fossil should just look,
how much free RAM the computer has and
adjust its algorithm parameters accordingly.

___
fossil-users mailing list
fossil-users@lists.fossil-scm.org
http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users