Re: [dev-context] integrating context mkiv, luatex, and fmtutil, mktexlsr, etc
Hi Hans, hi all, On Mi, 03 Okt 2007, Hans Hagen wrote: Norbert Preining wrote: $ENGINE -ini -jobname=cont-en -progname=context -8bit *cont-en.ini [...] that won't work because luatex by itself cannot take care of the stubs it may need for starting up (how could it know -) making a format (at least formkiv, but also for plain if you want) boils down to: - making a lua stub file (lua) - compiling that for fast loading (luc) - making a format Ok, that said we have to adjust the fmtutil script. Currently the information stored in the fmtutil.cnf file is $format $engine $sometestfile $restofcmdlineoptions These lines normally trigger a call to $engine -ini -jobname=$format -progname=context $cmdline (the progname is already a special case AFAIR). Now for combinations of cont-?? $engine ... we could switch to calling texexec --engine $engine --make ?? Is this right? But this doesn't work with luatex: texexec --engine luatex --make en still runs luatex. (Strange btw, isn't it?) So, what is the *recommended* way to generate context formats for the following engines: pdftex: ini, texexec xetex: ini, texexec aleph: ini, texexec luatex: ??? (ini: calling $engine -ini ..., texexec: calling texexec --engine $engine --make ??) $ENGINE --luaonly luatools.lua --make --compile cont-en $ENGINE --luaonly luatools.lua --make --compile plain Ok, so one could call luatools --make --compile cont-en ??? For all those texexec/luatools creating of formats there is another problem: The formats are placed in quite arbitrary locations. TEXMFVAR should be the right place, but this is not honoured (in my case texexec did put it into TEXMFHOME!). Or at least it should be *fixed* to which tree the format will be written. Then there is the problem that luatools --make does not work without any cache, well, see below. - integration into mktexlsr see my other email about multiple cache merging, but to some up: . cache should be read from more then one directory, eg from TEXMFSYSCACHE and TEXMFCACHE (with usual override stuff) . single cache updates should be possible: luatools --generate $HOME/texmf i'll see what i can do should generate the cache for $HOME/texmf in $TEXMFCACHE what exactly do you mean with integration? calling luatools? Well, the idea is that if someone calls mktexlsr $tree and luatex/context is installed, the luatex cache is also updated. The idea is that we just plug in some code that is only executed if context and luatex is installed, that updates the cache. Take for example Debian: One can install texlive packages, plus independent context and luatex packages. But all the add-on packages for fonts etc only call mktexlsr. We will have a hard time forcing them to add some magic that in case of the presence of luatex/context also the cache (for system files) is updated. My idea is to have something mktexlsr.d/ directory where packages can drop scripts (same would work also in TeX Live) and mktexlsr at the end of the run calls these scripts. So if luatex drops something, and context drops something, we could organize that not only the mktexlsr db, but also the luatex cache is up2date. Other things I realized as user: - running texexec --lua test.tex generates the format in ~/luatex-cache/context/e027248d6557d124c703335e8a95ecd5/formats but it creates it again and again. strange, since texexec --lua is not supposed to generate a format at all Maybe I missed something ... - If I move the generated files from the above dir to ~/.texmf-var/web2c/luatex/ and regenerate the luatex-cache with --generate --verbose the next run does not work: (i wonder what i do with the . in .texmf-var -) Aehmmm, but the luatex cache will quite sure be set to something with a dot, at least the local one ;-) - I think it is time to discuss all this problems now, otherwise inclusion into TeX Live upstream and Debian will make problems. Ok, I can just ship the stuff and don't produce and cache/formats whatsoever for MkIV, MkII still works quite well. But I guess it would be nice to sort this out in a good way for all parties. Best wishes Norbert --- Dr. Norbert Preining [EMAIL PROTECTED]Vienna University of Technology Debian Developer [EMAIL PROTECTED] Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 --- THURNBY (n.) A rucked-up edge of carpet or linoleum which everyone says someone will trip over and break a leg unless it gets fixed. After a year or two someone trips over it and breaks a leg. --- Douglas Adams, The Meaning of Liff
Re: [dev-context] Call for test documents
On Di, 02 Okt 2007, Hans Hagen wrote: http://context.aanhet.net/svn/manuals/ Is there svn access? I have a checkout (very old) but the svn address has changed. Best wishes Norbert --- Dr. Norbert Preining [EMAIL PROTECTED]Vienna University of Technology Debian Developer [EMAIL PROTECTED] Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 --- SLUGGAN (n.) A lurid facial bruise which everyone politely omits to mention because it's obvious that you had a punch-up with your spouse last night - but which into a door. It is useless to volunteer the true explanation because nobody will believe it. --- Douglas Adams, The Meaning of Liff ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
Re: [dev-context] [NTG-context] Call for test documents
Norbert Preining wrote: On Di, 02 Okt 2007, Hans Hagen wrote: http://context.aanhet.net/svn/manuals/ Is there svn access? I have a checkout (very old) but the svn address has changed. My mirror script (that populates the above url) uses: svn://83.247.100.17:33690/manuals ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
Re: [dev-context] [NTG-context] Call for test documents
On Mi, 03 Okt 2007, Taco Hoekwater wrote: svn://83.247.100.17:33690/manuals Thanks, worked. Best wishes Norbert --- Dr. Norbert Preining [EMAIL PROTECTED]Vienna University of Technology Debian Developer [EMAIL PROTECTED] Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 --- KANTURK (n.) An extremely intricate knot originally used for belaying the topgallant foresheets of a gaff-rigged China clipper, and now more commonly observed when trying to get an old kite out of the cupboard under the stairs. --- Douglas Adams, The Meaning of Liff ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
Re: [dev-context] integrating context mkiv, luatex, and fmtutil, mktexlsr, etc
HI Hans! On Mi, 03 Okt 2007, Hans Hagen wrote: database for luatex. But it is placed NOT alongside the ls-R files, right? well, TEXMFSHARECACHE=yes makes sure that it is put in the same path as the ls-r file Ah, ok, that is with the new context you created today. No, didn't test this till now ;-) First I guess I need an updated luatex you said. no, not the trees, but if one uses a font, say lmroman10-regular.otf, then the 'converterd/prepared/whatever' datatable is cached so only that file ends up in the cache (flat structure, so, when one needs lmroman10-regular.otf, i first look in the cache (no database needed) for a tmc/a file, if not present i generate one, using the otf file from the normal tree, looked up in a kpse like way) [...] - preprocessed font data cache so that loading the stuff is done faster indeed, only at runtime, so normally in the users path someplace Hmm, I still don't get it. In all this processing, is there a job depending thing contained? I thought extracting all those tables etc can be done independently of all jobs (and will help all jobs). Anyway, I got it. So this is not a bad thing at all to have this stuff generated at job running time, so we (integrators) don't have to worry about it. Only finding all the files is important for a start, so ls-R replacement. $ ls -l ~/luatex-cache/context/f7d1b3c25487ab1e1035aff1c53b90da/formats/ -rw-r--r-- 1 norbert norbert 5669003 2007-10-03 14:07 cont-en.fmt -rw-r--r-- 1 norbert norbert 38786 2007-10-03 14:07 cont-en.log -rw-r--r-- 1 norbert norbert 159484 2007-10-03 14:07 cont-en.lua -rw-r--r-- 1 norbert norbert 112438 2007-10-03 14:07 cont-en.luc $ So the format is placed in some strange ;-) place under luatex-cache. hm, also with the new version and the env var set? here it nicely goes to texmf-linux/web2c/luatex Probably an instance of get the new beta, btw, where do I get it, from the usual download place? Or is there something else for betas? - formats??? what is saved for this a cont-en.fmt as well as a cont-en.luc file, the format and it sstub Yes of course, but what needs to be cached here but the sole information that the file exist? (ie updating ls-R/cache) But all of this is somehow static. On a normal system a normal user shouldn't have the necessity to change anything of the above. That should be done at install time of the respective stuff. no, the font cache is not pregenerated at all, think of it like this: - luatex loads otf font (takes time) - provides it as table to mkiv (takes more time) - mkiv adds a few things, preprocesses the font a bit - then saves it as table (compiled) so that loading goes in an eyeblink Ok, so after the first processing later jobs will have access to these tables in a blink. Ok, don't mind. Good for me ;-) No need to worry about this. Furthermore, the cache could be used (no idea whether this is true) for: - single job caching of data wouldn't it be better to keep generated files like this in the cwd, like .aux, etc files we're talking of megabytes here Are these megabytes kept and are they useful for later runs? If yes, then it is ok to put them in a separate cache (I assume the stuff is useful, it is the font stuff). What was the reason to do the hashing somewhere else but add the md5sums etc for these trees? lengths of paths and such; i run luatex on the unix web/etc servers and have multiple trees in parallel, so now i can with one variable changed access all data; /temp/luatex/context/treeroot/whatever /temp/luatex/context/c:/blabla/tex/blabla/whatever does not work that well, so i hash the treeroot Hmm, but if the caches would be in TEXMFTREE/lcache/ then the only thing necessary would be a change of TEXMFCNF referencing other trees. (Sorry, only trying to understand the details ...) Best wishes Norbert --- Dr. Norbert Preining [EMAIL PROTECTED]Vienna University of Technology Debian Developer [EMAIL PROTECTED] Debian TeX Group gpg DSA: 0x09C5B094 fp: 14DF 2E6C 0307 BE6D AD76 A9C0 D2BF 4AA3 09C5 B094 --- He stood up straight and looked the world squarely in the fields and hills. To add weight to his words he stuck the rabbit bone in his hair. He spread his arms out wide. `I will go mad!' he announced. --- Arthur discovering a way of coping with life on --- Prehistoric Earth. --- Douglas Adams, The Hitchhikers Guide to the Galaxy ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
Re: [dev-context] integrating context mkiv, luatex, and fmtutil, mktexlsr, etc
Norbert Preining wrote: Are these megabytes kept and are they useful for later runs? If yes, then it is ok to put them in a separate cache (I assume the stuff is useful, it is the font stuff). not sure yet, maybe later; but there is a cache cleaner (i may even thrash by date and usage) Hmm, but if the caches would be in TEXMFTREE/lcache/ then the only thing necessary would be a change of TEXMFCNF referencing other trees. (Sorry, only trying to understand the details ...) sure, all depends on what the cache var is set too (too many variants so i play safe and hash (but i can make that piece configurable if needed) the beta is on the website and on the ftp Hans - Hans Hagen | PRAGMA ADE Ridderstraat 27 | 8061 GH Hasselt | The Netherlands tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com | www.pragma-pod.nl - ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
[dev-context] xml secrets
Hello Hans, Here's an almost minimal example to reproduce the problems with trailing spaces in XML files. Mojca secret.tex Description: TeX document Secrets This is the way how secrets are done The line after space unnoticed has gone. :) ___ dev-context mailing list dev-context@ntg.nl http://www.ntg.nl/mailman/listinfo/dev-context
