Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Taco Hoekwater


Hi Lars,

Lars Huttar wrote:

Hello,

We've been using TeX to typeset a 1200-page book, and at that size, the
time it takes to run becomes a big issue (especially with multiple
passes... about 8 on average). It takes us anywhere from 80 minutes on
our fastest machine, to 9 hours on our slowest laptop.


You should not need an average of 8 runs unless your document is
ridiculously complex and I am curious what you are doing (but that
is a different issue from what you are asking).


So the question comes up, can TeX runs take advantage of parallelized or
distributed processing? 


No. For the most part, this is because of another requisite: for
applications to make good use of threads, they have to deal with a
problem that can be parallelized well. And generally speaking,
typesetting  does not fall in this category. A seemingly small change
on page 4 can easily affect each and every page right to the end
of the document.

About the only safe things that can be threaded is the reading of
resources (images and fonts) and, mostly because of the small gain,
nobody has been working on that that I know of.


parallel pieces so that you could guarantee that you would get the same
result for section B whether or not you were typesetting the whole book
at the same time?


if you are willing to promiss yourself that all chapters will be exactly
20 pages - no more, no less - they you can split the work off into
separate job files yourself and take advantage of a whole server
farm. If you can't ...

Best wishes,
Taco
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] Some ligatures cannot be copied and pasted

2008-12-16 Thread Hans Hagen

Taco Hoekwater wrote:


Yue Wang wrote:

Hi,

see the attachment


Probably because these ligatures are not in Unicode.
I do not know whether it is possible to support cutpaste
of these, I have to investigate.


maybe if we use the a_b_c naming for such ligature


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] markiv, output written on...

2008-12-16 Thread Hans Hagen

Yue Wang wrote:

and I also find that MKIV does not take terminal width into consideration.
(pdftex output has a max line length limit)


it does, but i've set it to a higher value (i prefer to let the terminal 
do the wrapping itself)


Hans


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Hans Hagen

Lars Huttar wrote:

Hello,

We've been using TeX to typeset a 1200-page book, and at that size, the
time it takes to run becomes a big issue (especially with multiple
passes... about 8 on average). It takes us anywhere from 80 minutes on
our fastest machine, to 9 hours on our slowest laptop.


often there are ways to optimize the document style

btw, in my laptop mk.tex which loads a bunch of real big fonts (esp punk 
slows down) does 10 pages/second (242 pages currently) so your setu opif 
probably not that efficient



So the question comes up, can TeX runs take advantage of parallelized or
distributed processing? As many in the computer industries are aware,
processor speed (clock rate) has plateaued; it is not going to continue
rising at the rate it had been. Hence the common move to dual-core,
quad-core, etc. machines. But applications in general cannot take
advantage of multiple cores to speed their work unless they are
architected to take advantage of them.

We googled around a bit but were surprised not to find any real
references to efforts at running TeX in parallel or on distributed
networks or clusters. Wouldn't this be something that a lot of people
would find useful? Or does everyone only use TeX for typesetting short
papers?


it all depends on what you process; for simple docs tex is rather fast:

\starttext

\setuplayout[page]

\dorecurse{1000}{\null\page}

\stoptext

such a run takes in mkiv:

5.944 seconds, 1000 processed pages, 1000 shipped pages, 168.237 
pages/second



Sure, you can use manual tricks to speed up TeX processing.
You can comment out sections of a document, or select them via modes.
But then you have to remember where you did the commenting out, so you
can reverse it. And you have no guarantees as to whether the
inclusion/exclusion of section B will affect the layout of section C or not.


often it's inefficient font calls that slow down the job (or big 
graphics that one can skip including in a first pass)



Wouldn't it be nice if TeX (or a TeX wrapper, or macro package, or
typesetting system) could take care of this for you?


mayne in the future we can do some parallization

also, i'm thinking of 'one run with prerolls' but it has no high 
priority (maybe i'll do it when i need it for a project)



What if you had a language -- or a few extensions to existing languages
-- to give your typesetting engine hints or commands about where to
split up your long document into fairly-independent chunks? What if you
designed your document specifically to be typeset in independent,
parallel pieces so that you could guarantee that you would get the same
result for section B whether or not you were typesetting the whole book
at the same time?


there are quite some dependencies of pages on each other (take cross 
refs and how they might influence a next run)



What if the typesetting system automated the stitching-back-together
process of the chunks, gathering page reference info from each chunk to
inform the next iteration of typesetting the other chunks?


this is an option when you have to reprocess parts of the documents often


Has anyone been working on this already? It seems like it must have been
discussed, but I don't know where to go to look for that discussion.


if i were you i'd first look into optimizing the style

Hans

-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] PDF page numbers

2008-12-16 Thread Zhichu Chen
Best Regards
Chen


  Zhi-chu Chen | Shanghai Synchrotron Radiation Facility
 No. 2019 | Jialuo Rd. | Jiading | Shanghai | P.R. China
 tel: 086 21 5955 3405 | zhichu.chen.googlepages.com
   | www.sinap.ac.cn




On Tue, Dec 16, 2008 at 4:33 PM, Taco Hoekwater t...@elvenkind.com wrote:


 Mohamed Bana wrote:
 Derek CORDEIRO wrote:
 In latex + book/memoir + hyperref, if \frontmatter \mainmatter are
 used, the pdf files generated use the roman numbers in the pdf itself.
 It appears as ii (2 of 500) in the pdfviewer. Is it possible to do the
 same with ConTeXt?

 It would be great if ConTeXt could.

 Doesn't sound like rocket science. Perhaps someone could dig up the
 latex code that does this ?

Does the following code work for you?
==
\pdfcompresslevel 0

\setupinteraction
  [state=start,
   focus=width,
   page=yes,
   openaction=firstpage,
   click=no,
   title=TEST,
   author=Neals,
   subtitle=Test,
   keyword={test}]

\setupcolors
  [state=start]

\newcount\FrontPageNumbers

\setupsubpagenumber[way=bytext,state=start]
\setuppagenumbering[location=]

\startsectionblockenvironment[frontpart]
 \setupfootertexts[pagenumber]
 \setuppagenumbering[conversion=romannumerals]
\stopsectionblockenvironment

\startsectionblockenvironment[bodypart]
 \global\FrontPageNumbers=\pagenumber
 \global\advance\FrontPageNumbers\minusone
 \setuppagenumber[number=1]
 \resetsubpagenumber
%  \setupfootertexts[\the\FrontPageNumbers]
 \setupfootertexts[pagenumber]
\stopsectionblockenvironment

\startsectionblockenvironment[backpart]
 \resetsubpagenumber
\stopsectionblockenvironment

\def\PageLabels{\pdfcatalog{/PageLabels  /Nums [ 0  /S /r 
\the\FrontPageNumbers  /S /D  ]   }}


\prependtoks
  \PageLabels
\to \everystoptext

\starttext



\startfrontmatter
% \chapter{Table of Contents}
\completecontent[interaction=all]
\chapter{Preface}
\dorecurse{10}{\input zapf\par}
% \chapter{Preface}
% \dorecurse{10}{\input zapf\par}
% \chapter{Preface}
% \dorecurse{10}{\input zapf\par}
\stopfrontmatter

\startbodymatter
\chapter{Test}
\dorecurse{15}{\input tufte\par}
\chapter{TestAgain}
\dorecurse{15}{\input ward\par}
\stopbodymatter

\startbackmatter
Index maybe.
\stopbackmatter


\stoptext
==





 Taco
 ___
 If your question is of interest to others as well, please add an entry to the 
 Wiki!

 maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
 webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
 archive  : https://foundry.supelec.fr/projects/contextrev/
 wiki : http://contextgarden.net
 ___

___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Hans Hagen

Aditya Mahajan wrote:

On Tue, 16 Dec 2008, Hans Hagen wrote:


Lars Huttar wrote:

Hello,

We've been using TeX to typeset a 1200-page book, and at that size, the
time it takes to run becomes a big issue (especially with multiple
passes... about 8 on average). It takes us anywhere from 80 minutes on
our fastest machine, to 9 hours on our slowest laptop.


often there are ways to optimize the document style


In Latex, there is a package called mylatex, which allows you to create 
a format consisting of your preamble. Then you can call latex with your 
own format and this speeds up things.


not on this document since the processing takes way more than the startup

in jelle's doc it's the many many many local font definitions and the 
some 800 metapost graphics that are the culprit


- define fonts beforehand
- use unique mpgraphic when possible

i changes the definitions a bit and now get 5 pages per second on my 
laptop in luatex; xetex processes the pages a bit faster but spends way 
more time on the mp part


(of course 1450 pages of columnsets and multipage bTABLE's also cost a 
bit of time)


This approach did not provide a significant speed improvement in latex 
for me, and I don't know whether it will do so in context. Hans and 
Taco, do you think that creating a personal format and possibly also 
dumping some font related info could provide a tradeoff between 
processing speed and disk space?


it depends, you might win maybe a second per run when on startup in 
cases where you have thousands of small runs but often using a small 
tree (like the minimals) pays of more


Hans

-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] Some ligatures cannot be copied and pasted

2008-12-16 Thread Yue Wang
1 (page number) also has

On Tue, Dec 16, 2008 at 6:24 PM, Hans Hagen pra...@wxs.nl wrote:
 Yue Wang wrote:

 Hi,

 see the attachment


 only Th has a problem; if you run with \nopdfcompression you can peek into
 the file and see what unicode mapping is used; however, i would not be
 surprised if there's a bug in the font's resource

 Hans

 -
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
 -
 ___
 If your question is of interest to others as well, please add an entry to
 the Wiki!

 maillist : ntg-context@ntg.nl /
 http://www.ntg.nl/mailman/listinfo/ntg-context
 webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
 archive  : https://foundry.supelec.fr/projects/contextrev/
 wiki : http://contextgarden.net
 ___

___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Aditya Mahajan

On Tue, 16 Dec 2008, Hans Hagen wrote:


Lars Huttar wrote:

Hello,

We've been using TeX to typeset a 1200-page book, and at that size, the
time it takes to run becomes a big issue (especially with multiple
passes... about 8 on average). It takes us anywhere from 80 minutes on
our fastest machine, to 9 hours on our slowest laptop.


often there are ways to optimize the document style


In Latex, there is a package called mylatex, which allows you to create a 
format consisting of your preamble. Then you can call latex with your own 
format and this speeds up things.


This approach did not provide a significant speed improvement in latex for 
me, and I don't know whether it will do so in context. Hans and Taco, do 
you think that creating a personal format and possibly also dumping some 
font related info could provide a tradeoff between processing speed and 
disk space?


Aditya
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] markiv, output written on...

2008-12-16 Thread Hans Hagen

Patrick Gundlach wrote:

Hi,

when I run a markiv file I don't get any message of the resulting pdf
file. Is this done on purpose?


probably (i overload some things)

Hans


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] about LuaTools | error unable to identify cnf file

2008-12-16 Thread Mojca Miklavec
On Mon, Dec 15, 2008 at 9:16 PM, Diego Depaoli wrote:
 On Mon, Dec 15, 2008 at 12:13 PM, Dalyoung Jeong wrote:
 Dear Mojca,

 texmfstart texexec --make (--xtx) --all run smoothly, but luatools

 still has a cnf error.
 Stupid question... did you run/source setuptex before?

In pure theory it should not be needed.



I told you to update TL, but I forgot to tell you that you need to
update *both* from TeX Live *and* update LuaTeX and ConTeXt afterwards
manually with some more recent version.

Updating with tlmgr only serves the purpose to get the right version
of luatools, mtxrun and texmfstart. They used to be copies of scripts,
while they are symbolic links now. If you woud update ConTeXt without
updating the version from TL, it would not update the scripts.

Specifically on MacTeX you need some more patches that have been
written after ConTeXt and LuaTeX have been frozen on TeX Live. So you
could either set all the environmental variables or simply update
everything.

(See http://wiki.contextgarden.net/Running_Mark_IV where I forgot to
mention that one should update with tlmgr first.)

Mojca
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Hans Hagen

Lars Huttar wrote:


We have close to 7000 mpgraphics, and they add about 15 minutes to the
run time.


most of them are the same so reusing them made sense


But the run time was already quite long before we started using those.


- define fonts beforehand


OK, we will look into this. I'm sure Jelle knows about this but I'm a
noob. I'm pretty sure we are not *loading* fonts every time, but maybe
we're scaling fonts an unnecessary number of times.
For example, we have the following macro, which we use thousands of times:
\def\LN#1{{\switchtobodyfont[SansB,\LNfontsize]{#1}}}


indeed this will define the scaled ones again and again (whole sets of 
them since you use a complete switch); internall tex reuses them but it 
only know so when they're defined



Would it help much to instead use
\definefont[SansBLN][... at \LNfontsize]
and then
\def\LN#1{{\SansBLN{#1}}}
?


indeed:

\definefont[SansBLN][... at \LNfontsize]

but no extra { } needed:

\def\LN#1{{\SansBLN#1}}


- use unique mpgraphic when possible


I would be interested to know if this is possible in our situation. Most
of our mpgraphics are due to wanting thick-and-thin or single-and-double
borders on tables, which are not natively supported by the ConTeXt table
model.


i sent jelle the patched files


The advice I received said to define each mpgraphic using
\startuseMPgraphic (we have about 18 of these), associate them with
overlays using \defineoverlay (again, we have 18), and then use them in
table cells using statements like
\setupTABLE[c][first][background={LRtb}]
Empirically, this seems to end up using one mpgraphic per table cell,
hence our thousands of mpgraphics. I don't know why a new mpgraphic
would be created for each cell. Can someone suggest a way to avoid this?


metafun manual: unique mp graphics


i changes the definitions a bit and now get 5 pages per second on my
laptop in luatex; xetex processes the pages a bit faster but spends way
more time on the mp part


My last run gave about 0.25 pages per second on our fastest server, when
taking into account multiple passes; that comes out to about 2 pps for
--once.


the patched files do 5-10 pps on my laptop (was  1 sec pp) so an 
improvement factor of at least 5 is possible


there are probably other optimizations possible but i cannot spent too 
much time on it


Hans

-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] markiv, output written on...

2008-12-16 Thread Patrick Gundlach
Hi Hans,

 when I run a markiv file I don't get any message of the resulting pdf
 file. Is this done on purpose?

 probably (i overload some things)

May I then ask for a change in behaviour? I'd like to parse the messages
and search for the final pdf.

Patrick
-- 
ConTeXt wiki and more: http://contextgarden.net
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Lars Huttar
On 12/16/2008 2:08 AM, Taco Hoekwater wrote:
 
 Hi Lars,
 
 Lars Huttar wrote:
 Hello,

 We've been using TeX to typeset a 1200-page book, and at that size, the
 time it takes to run becomes a big issue (especially with multiple
 passes... about 8 on average). It takes us anywhere from 80 minutes on
 our fastest machine, to 9 hours on our slowest laptop.
 
 You should not need an average of 8 runs unless your document is
 ridiculously complex and I am curious what you are doing (but that
 is a different issue from what you are asking).
 
 So the question comes up, can TeX runs take advantage of parallelized or
 distributed processing? 
 
 No. For the most part, this is because of another requisite: for
 applications to make good use of threads, they have to deal with a
 problem that can be parallelized well. And generally speaking,
 typesetting  does not fall in this category. A seemingly small change
 on page 4 can easily affect each and every page right to the end
 of the document.

Thank you for your response.

Certainly this is true in general and in the worst case, as things stand
currently. But I don't think it has to be that way. The following could
greatly mitigate that problem:

- You could design your document *specifically* to make the parts
independent, so that the true and authoritative way to typeset them is
to typeset the parts independently. (You can do this part now without
modifying TeX at all... you just have the various sections' .tex files
input common headers / macro defs.) Then, by definition, a change in
one section cannot affect another section (except for page numbers, and
possibly left/right pages, q.v. below).

- Most large works are divisible into chunks separated by page breaks
and possibly page breaks that force a recto. This greatly limits the
effects that any section can have on another. The division (chunking)
of the whole document into fairly-separate parts could either be done
manually, or if there are clear page breaks, automatically.

- The remaining problem, as you noted, is how to fix page references
from one section to another. Currently, TeX resolves forward references
by doing a second (or third, ...) pass, which uses page information from
the previous pass. The same technique could be used for resolving
inter-chunk references and determining on what page each chunk should
start. After one pass on of the independent chunks (ideally performed
simultaneously by separate processing nodes), page information is sent
from each node to a coordinator process. E.g. the node that processed
section two tells the coordinator that chapter 11 starts 37 pages after
the beginning of section two. The coordinator knows in what sequence the
chunks are to be concatenated, thanks to a config file. It uses this
information together with info from each of the nodes to build a table
of what page each chunk should start on, and a table giving the absolute
page number of each page reference. If pagination has changed, or is
new, this info is sent back to the various nodes for another round of
processing.

If this distributed method of typesetting a document takes 1 additional
iteration compared to doing it in series, but you get to split the
document into say 5 roughly equal parts, you could presumably get the
job done a lot quicker in spite of the extra iteration.

This is a crude description but hopefully the idea is clear enough.

 parallel pieces so that you could guarantee that you would get the same
 result for section B whether or not you were typesetting the whole book
 at the same time?
 
 if you are willing to promiss yourself that all chapters will be exactly
 20 pages - no more, no less - they you can split the work off into
 separate job files yourself and take advantage of a whole server
 farm. If you can't ...

Yes, the splitting can be done manually now, and when the pain point
gets high enough, we do some manual separate TeX runs.

However, I'm thinking that for large works, there is enough gain to be
had that it would be worth systematizing the splitting process and
especially the recombining process, since the later is more error-prone.

I think people would do it a lot more if there were automation support
for it. I know we would.

But then, maybe our situation of having a large book with dual columns
and multipage tables is not common enough in the TeX world.
Maybe others who are typesetting similar books just use commercial
WYSIWYG typesetting tools, as we did in the previous edition of this book.

Lars
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net

Re: [NTG-context] PDF page numbers

2008-12-16 Thread Hans Hagen

Taco Hoekwater wrote:


Mohamed Bana wrote:

Derek CORDEIRO wrote:

In latex + book/memoir + hyperref, if \frontmatter \mainmatter are
used, the pdf files generated use the roman numbers in the pdf itself.
It appears as ii (2 of 500) in the pdfviewer. Is it possible to do the
same with ConTeXt?

It would be great if ConTeXt could.


sure, since it's just a page resource; i think i even played with it and 
dropped it when i got funny printing side effects (when selecting pages) 
in older versions of teh viewer



Doesn't sound like rocket science. Perhaps someone could dig up the
latex code that does this ?


something for the mkiv backend

Hans

-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] markiv, output written on...

2008-12-16 Thread Hans Hagen

Patrick Gundlach wrote:

Hi Hans,


when I run a markiv file I don't get any message of the resulting pdf
file. Is this done on purpose?

probably (i overload some things)


May I then ask for a change in behaviour? I'd like to parse the messages
and search for the final pdf.


keep in mind that context --result might overload the name so we cannot 
use jobname here.


to cont-new.mkiv add:

\ctxlua{
ctx.register_statistics(result saved in file, \letterpercent 
s.\letterpercent s, function()

return \outputfilename, (tex.pdfoutput0 and pdf) or dvi
end)
}

then you can parse for

mkiv lua stats : result saved in file  - x.pdf

(i'll add this)

Hans


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
 tel: 038 477 53 69 | fax: 038 477 53 74 | www.pragma-ade.com
 | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Lars Huttar
On 12/16/2008 11:37 AM, Hans Hagen wrote:
 Lars Huttar wrote:
 
 We have close to 7000 mpgraphics, and they add about 15 minutes to the
 run time.
 
 most of them are the same so reusing them made sense
 
 But the run time was already quite long before we started using those.

 - define fonts beforehand

 OK, we will look into this. I'm sure Jelle knows about this but I'm a
 noob. I'm pretty sure we are not *loading* fonts every time, but maybe
 we're scaling fonts an unnecessary number of times.
 For example, we have the following macro, which we use thousands of
 times:
 \def\LN#1{{\switchtobodyfont[SansB,\LNfontsize]{#1}}}
 
 indeed this will define the scaled ones again and again (whole sets of
 them since you use a complete switch); internall tex reuses them but it
 only know so when they're defined
 
 Would it help much to instead use
 \definefont[SansBLN][... at \LNfontsize]
 and then
 \def\LN#1{{\SansBLN{#1}}}
 ?
 
 indeed:
 
 \definefont[SansBLN][... at \LNfontsize]
 
 but no extra { } needed:
 
 \def\LN#1{{\SansBLN#1}}

Thanks, we will try this.
(Jelle, since you have worked with this a lot longer than I have, please
stop me if you have concerns about my making this sort of change.)

 - use unique mpgraphic when possible

 I would be interested to know if this is possible in our situation. Most
 of our mpgraphics are due to wanting thick-and-thin or single-and-double
 borders on tables, which are not natively supported by the ConTeXt table
 model.
 
 i sent jelle the patched files

OK, I'll look to hear from him. Are these patches to support these kinds
of borders on tables, thus no longer needing to use MPgraphics?

 The advice I received said to define each mpgraphic using
 \startuseMPgraphic (we have about 18 of these), associate them with
 overlays using \defineoverlay (again, we have 18), and then use them in
 table cells using statements like
 \setupTABLE[c][first][background={LRtb}]
 Empirically, this seems to end up using one mpgraphic per table cell,
 hence our thousands of mpgraphics. I don't know why a new mpgraphic
 would be created for each cell. Can someone suggest a way to avoid this?
 
 metafun manual: unique mp graphics

Great...
I converted our useMPgraphics to uniqueMPgraphics. This reduced our
number of mpgraphics from 7000 to 800!

Unfortunately the result doesn't look quite right... but since we may
not need to use mpgraphics anyway thanks to your patches, I'll hold off
on debugging the result.

 i changes the definitions a bit and now get 5 pages per second on my
 laptop in luatex; xetex processes the pages a bit faster but spends way
 more time on the mp part

 My last run gave about 0.25 pages per second on our fastest server, when
 taking into account multiple passes; that comes out to about 2 pps for
 --once.
 
 the patched files do 5-10 pps on my laptop (was  1 sec pp) so an
 improvement factor of at least 5 is possible
 
 there are probably other optimizations possible but i cannot spent too
 much time on it

Thanks for all your help thus far.

Lars

___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Mojca Miklavec
On Tue, Dec 16, 2008 at 6:25 PM, Lars Huttar wrote:
 Most
 of our mpgraphics are due to wanting thick-and-thin or single-and-double
 borders on tables, which are not natively supported by the ConTeXt table
 model.
 The advice I received said to define each mpgraphic using
 \startuseMPgraphic (we have about 18 of these), associate them with
 overlays using \defineoverlay (again, we have 18), and then use them in
 table cells using statements like
\setupTABLE[c][first][background={LRtb}]

In that case I would suggest you to use vrules/hrules to achieve the
same. As long as you don't need to use complicated graphics (like Hans
uses random frames) this should be doable and would speed up the
process enormously. What's the compile time if you omit the MP
graphics?

Mojca
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Martin Schröder
2008/12/16 Lars Huttar lars_hut...@sil.org:
 - You could design your document *specifically* to make the parts
 independent, so that the true and authoritative way to typeset them is
 to typeset the parts independently. (You can do this part now without
 modifying TeX at all... you just have the various sections' .tex files
 input common headers / macro defs.) Then, by definition, a change in
 one section cannot affect another section (except for page numbers, and
 possibly left/right pages, q.v. below).

True. Also with TeX if your paragraphs are independent of each other
(i.e. they don't include references to others), they could the typeset
in parallel and then handed over to the page builder.

 - Most large works are divisible into chunks separated by page breaks
 and possibly page breaks that force a recto. This greatly limits the
 effects that any section can have on another. The division (chunking)
 of the whole document into fairly-separate parts could either be done
 manually, or if there are clear page breaks, automatically.

pdfTeX 1.50 knows about the page diversions (analogue to m4's divert
and undivert). They have a lot of potential.

 page number of each page reference. If pagination has changed, or is
 new, this info is sent back to the various nodes for another round of
 processing.

Hopefully stopping at some point. If you use something like varioref,
you can end with infinite circles. :-)

Best
   Martin
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] setupdelimitedtext: linespacing in final quoted paragraph

2008-12-16 Thread Sanjoy Mahajan
Thanks for the responses.  I've wikified a short example, putting it at
end of this page:

http://wiki.contextgarden.net/Long_quotations_and_extracts

Feel free to move or improve.

Will LuaTeX's paragraph builder have the same restriction -- perhaps for
compatability -- that linespacing be constant throughout the paragraph?

-Sanjoy

`Until lions have their historians, tales of the hunt shall always
 glorify the hunters.'  --African Proverb
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Lars Huttar
On 12/16/2008 3:31 PM, Martin Schröder wrote:
 2008/12/16 Lars Huttar lars_hut...@sil.org:
 - You could design your document *specifically* to make the parts
 independent, so that the true and authoritative way to typeset them is
 to typeset the parts independently. (You can do this part now without
 modifying TeX at all... you just have the various sections' .tex files
 input common headers / macro defs.) Then, by definition, a change in
 one section cannot affect another section (except for page numbers, and
 possibly left/right pages, q.v. below).
 
 True. Also with TeX if your paragraphs are independent of each other
 (i.e. they don't include references to others), they could the typeset
 in parallel and then handed over to the page builder.

Good point... although doesn't the page optimization feed back into
paragraph layout?

 - Most large works are divisible into chunks separated by page breaks
 and possibly page breaks that force a recto. This greatly limits the
 effects that any section can have on another. The division (chunking)
 of the whole document into fairly-separate parts could either be done
 manually, or if there are clear page breaks, automatically.
 
 pdfTeX 1.50 knows about the page diversions (analogue to m4's divert
 and undivert). They have a lot of potential.

Sounds useful. It's impressive if you can get a correct table of
contents in the first run (says
http://www.gust.org.pl/BachoTeX/2008/presentations/ms/handout.pdf)

 page number of each page reference. If pagination has changed, or is
 new, this info is sent back to the various nodes for another round of
 processing.
 
 Hopefully stopping at some point. If you use something like varioref,
 you can end with infinite circles. :-)

But this is just a problem of typesetting with TeX in general, not
particular to parallel/distributed typesetting, right?
IIRC, Knuth even says in the TeXbook that a really pathological case
might never stabilize.

Lars
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Martin Schröder
2008/12/16 Lars Huttar lars_hut...@sil.org:
 Good point... although doesn't the page optimization feed back into
 paragraph layout?

No. :-(

Best
   Martin
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread luigi scarso
On Tue, Dec 16, 2008 at 9:08 AM, Taco Hoekwater t...@elvenkind.com wrote:


 Hi Lars,

 Lars Huttar wrote:

 Hello,

 We've been using TeX to typeset a 1200-page book, and at that size, the
 time it takes to run becomes a big issue (especially with multiple
 passes... about 8 on average). It takes us anywhere from 80 minutes on
 our fastest machine, to 9 hours on our slowest laptop.


 You should not need an average of 8 runs unless your document is
 ridiculously complex and I am curious what you are doing (but that
 is a different issue from what you are asking).

  So the question comes up, can TeX runs take advantage of parallelized or
 distributed processing?


 No. For the most part, this is because of another requisite: for
 applications to make good use of threads, they have to deal with a
 problem that can be parallelized well. And generally speaking,
 typesetting  does not fall in this category. A seemingly small change
 on page 4 can easily affect each and every page right to the end
 of the document.


Also
3.11 Theory of page breaking
www.cs.utk.edu/~eijkhout/594-LaTeX/handouts/TeX%20LaTeX%20*course*.pdf


-- 
luigi
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


[NTG-context] tiny reshuffle of ConTeXt versions on the Minimals

2008-12-16 Thread Mojca Miklavec
Hello,

Just a short notice before it causes confusion.

Since Hans is preparing some really big changes in mkiv (basically
rewriting almost the whole ConTeXt core if I understood correctly), he
is still waiting to update ConTeXt on his website until the changes
are finished or at least working again.

Since neither current nor beta are compatible with the latest LuaTeX,
I have applied some patches that he sent me to official beta and now
both current and beta are pointing to the same patched version of
beta. (Maybe I should finally modify the timestamp ...)

Else current on minimals would be broken, or we would be unable to
upload luatex for quite a while.

In theory everything should still work normally, beta has also been
around for quite a while now, but before anyone asks me why are the
versions on minimals different from the ones that Hans provides or
it works with context beta/context current on my machine, why doesn't
it work on yours, here you have some short explanation.

This deviation from the official version will be dropped as soon as
ConTeXt gets stable again.

Mojca
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] distributed / parallel TeX?

2008-12-16 Thread Lars Huttar
On 12/16/2008 3:15 PM, luigi scarso wrote:
 
 
 On Tue, Dec 16, 2008 at 9:08 AM, Taco Hoekwater t...@elvenkind.com
 mailto:t...@elvenkind.com wrote:
 
 
 Hi Lars,
 
 
 Lars Huttar wrote:
...
 So the question comes up, can TeX runs take advantage of
 parallelized or
 distributed processing?
 
 
 No. For the most part, this is because of another requisite: for
 applications to make good use of threads, they have to deal with a
 problem that can be parallelized well. And generally speaking,
 typesetting  does not fall in this category. A seemingly small change
 on page 4 can easily affect each and every page right to the end
 of the document.
 
 
 Also
 3.11 Theory of page breaking
 www.cs.utk.edu/~eijkhout/594-LaTeX/handouts/TeX%20LaTeX%20
 http://www.cs.utk.edu/~eijkhout/594-LaTeX/handouts/TeX%20LaTeX%20*course*.pdf

Certainly that is a tough problem (particularly in regard to laying out
figures near the references to them). But again, if you can break down
the document into chunks that are fairly independent of each other (and
you almost always can for large texts), this problem seems no worse for
distributed processing than for sequential processing. For example, the
difficult part of laying out figures in Section 1 is confined to Section
1; it does not particularly interact with Section 2. References in
Section 2 to Section 1 figures are going to be relatively distant from
those figures regardless of page breaking decisions. Thus the difficult
problem of page breaking is reduced to the sequential-processing case...
still a hard problem, but one that can be attacked in chunks. Indeed,
the greater amount of CPU time per page that is made available through
distributed processing may mean that the algorithms can do a better job
of page breaking than through sequential processing.

Lars

___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___


Re: [NTG-context] tiny reshuffle of ConTeXt versions on the Minimals

2008-12-16 Thread Mojca Miklavec
Hello,

I also forgot to mention ... TeX Gyre is now the default font when
using postscript/times/helvetica/palatino.

In case you experience any problems, in particular with math, please report.

Mojca
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://tex.aanhet.net
archive  : https://foundry.supelec.fr/projects/contextrev/
wiki : http://contextgarden.net
___