On Sun, Sep 08, 2013 at 02:49:31PM +0700, Henrik Sarvell wrote:
> At the same time projects that rely on the framework need to be able to do
> the same thing without being screwed if the framework needs a new "slot".

As I wrote, allocating a new "slot" not a problem. In fact, this is
normally happening quite frequently in a typical application.

Still: Is it normally not necessary to extend the number of files each
time new class or index is added.

Usually, I start up with a skeleton like:

   # Database sizes
   (dbs
      (3 +Role +User)                  # 512 Prevalent objects
      (0 )                             # A:64 Tiny objects
      (1 (+User pw))                   # B:128 Small objects
      (2 )                             # C:256 Normal objects
      (4 )                             # D:1024 Large objects
      (6 )                             # E:4096 Huge objects
      (2 (+Role nm) (+User nm))        # F:256 Small indexes
      (4 )                             # G:1024 Normal indexes
      (6 ) )                           # H:4096 Large indexes

and then, whenever necessary, add new entries to _these_ files.

For example, when I have a new class of entities where I estimate that
the these are relatively large objects, I add them to 'D'. Creating a
new file instead of that won't have any performance advantage, because
access to objects is done with 'lseek' and thus the size of the files
doesn't matter.

Opening too many files might even have disadvantages, due to the number
of open file descriptors, and may have hardly-to-predict dis/advantages
due to worse/better disk cache efficiency.

For a concrete result of that process, see the 'dbs' call at the end of
this mail [*]. Here we probably already _have_ too many files ;-)


Also, keep in mind that this whole issue is not sooo extremely critical,
as long as you don't put _everything_ into a single file. If objects are
smaller than the block size, a little space is wasted, and if they are
larger, they will occupy more than one block, resulting in slightly
longer load times (which is not critical either, because objects are
cached anyway).

A little more critical than entities are the indexes. The block size
should not be too small here, but if you go with all indexes on block
size 4, you are on the safe side (just wasting _a_little_ space for very
small indexes). In my experience, index block sizes of more than 6 show
a decrease in performance, probably because of longer loading times.


> If *Dbs is only used to point to locations where new objects are created
> how does the database logic work for objects that already exist? How does
> it know to look for X in file Y if it doesn't use the pointers in *Dbs?

The value of '*Dbs' itself is used _only_ in 'pool' (to create or open
the DB). So this list in *Dbs doesn't hold any pointers, just the
indicators for the block sizes in the corresponding file. And even these
are used only once, when the DB is created the first time. After that,
they are just used as indicators to _open_ the existing files (where the
block sizes are already kept in the file headers).


Objects are located solely by their "name", which encodes the file
number and the file-offset to the symbol's first block. Once an object
is created, it won't move (its first block, that is)

♪♫ Alex


*) Example from an existing application (just for illustration, no need
   to understand the details).

# Database sizes
(dbs
   (4)
   (1 +Note +Leg (+User pw))
   (1 +Role +Jahr +Art +ArtKat +Status +BSped +Abteilung +Inhalt +Land)
   (1 +Kfz +Log +Currency +Team +Steuer +Anlass +Branche +Carrier +Inco +Svs)
   (1 +Memo +Besuch +Anr +AbwChk +Abw +Arb)
   (2 +User +Mitarb +Sup +Ort +Dln +Ves +ZollAmt +Lag)
   (3 +Kleb +PrForm)
   (4 +Firma +Entn +Artikel +Schaden +AbhA)
   (4 +TxtKat +MacText +BrfText +prs +PGrp)
   (4 +Aust)
   (4 +Messe)
   (5 +Beleg +PosText)
   (5 +Sendung)
   (5 +Ansp)
   (6 +Person)
   (3
      (+User nm)
      (+Role nm)
      (+Kfz kz)
      (+Currency key)
      (+Team key)
      (+Jahr key)
      (+Steuer txt)
      (+ZollAmt nm)
      (+Anlass txt)
      (+Anr nm)
      (+PGrp nm prs)
      (+Ves nm ets)
      (+PrForm nm)
      (+Land de en es) )
   (4 (+Leg pos ansp dat mes))
   (4
      (+Kfz mit)
      (+Mitarb sb plz ort tel fax mob mail)
      (+AbwChk nr txt)
      (+Abw ter)
      (+Entn snd)
      (+Arb nr mes snd)
      (+Lag nr mes snd)
      (+Schaden pos btg mit dat kw) )
   (4 (+Note nm act kw) (+Log mit) (+Mitarb name) (+Memo dat mit))
   (3 (+Entn nr) (+Sup mc) (+Branche key) (+Carrier key) (+Inco key))
   (4 (+Sup nm plz ort tel fax mob))
   (4 (+Entn nm txt dat tim kfz))
   (4 (+Artikel nm))
   (4 (+Artikel sup mc kat))
   (4 (+Ort nm lnd))
   (4 (+Ort cont exp imp))
   (4 (+Branche txt) (+Carrier nm) (+Inco nm))
   (4 (+Besuch dat txt))
   (4 (+Person nr nr2))
   (4
      (+Kunde nm)
      (+Partner nm)
      (+Btg nm)
      (+Dfg nm)
      (+Org nm)
      (+Dtv nm)
      (+LkwU nm)
      (+Sonst nm) )
   (4
      (+Kunde plz ort lnd tel fax mail)
      (+Partner plz ort lnd tel fax mail)
      (+Btg plz ort lnd tel fax mail)
      (+Dfg plz ort lnd tel fax mail)
      (+Org plz ort lnd tel fax mail)
      (+Dtv plz ort lnd tel fax mail)
      (+LkwU plz ort lnd tel fax mail)
      (+Sonst plz ort lnd tel fax mail) )
   (4 (+Ansp nm))
   (4 (+Ansp bra tel fax mob mail gtxt gt))
   (4 (+Messe plan imp exp kw))
   (4 (+Mes nm) (+Arc nm))
   (4 (+Mes ort von bis bra ueb vo part) (+Arc ort von bis bra ueb vo part))
   (4 (+Sendung pos ansp mit car inc dat awb lkw))
   (4 (+Sendung aust stb ksp avo hal std ves kw))
   (4 (+Beleg ansp nr dat mit) (+Rech of) (+Guts of))
   (4
      (+Aust lst)
      (+PosText mc)
      (+Art txt)
      (+ArtKat txt)
      (+Status txt)
      (+BSped txt)
      (+Abteilung txt)
      (+Inhalt txt)
      (+TxtKat bez)
      (+MacText txt)
      (+BrfText mc) )
   (4 (+Person zd sd bse bse1 bse2 bse3 bse4))
   (0 +Doc +Calc +V)
   (6 +Tarif +Manifest +Instr)
   (4 (+Calc nr trf) (+Tarif nm)) )
-- 
UNSUBSCRIBE: mailto:picolisp@software-lab.de?subject=Unsubscribe

Reply via email to