Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-23 Thread Karl Berry
This document was generated by a program I am writing which generates the latex file. In that case, it seems like it should be easy to change the program to generate several (many) files instead of one huge one, and only recompile more or less what's changed. Although it would take some

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-22 Thread Nasser M. Abbasi
On 8/22/2018 7:14 PM, Reinhard Kotucha wrote: On 2018-08-22 at 01:39:45 -0500, Nasser M. Abbasi wrote: > I think all the speed is comming from the make -jN method. If you are talking about speed, yes. But in practice you save even much more time if you follow Karl Berry's advice. |

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-22 Thread Reinhard Kotucha
On 2018-08-22 at 01:39:45 -0500, Nasser M. Abbasi wrote: > I think all the speed is comming from the make -jN method. If you are talking about speed, yes. But in practice you save even much more time if you follow Karl Berry's advice. | Another thought: can you break the document into parts?

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-21 Thread Reinhard Kotucha
On 2018-08-21 at 08:31:05 -0500, Nasser M. Abbasi wrote: > Or may be becuase on VBox, only one core can be used (as far as I > know) and make4ht checked for this, and that is why it used j1 > while on windows linux subsystem more cores can be used and it used > all available there? VBox can

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-21 Thread Michal Hoftich
Hi Nasser, > > Or may be becuase on VBox, only one core can be used (as far as I know) > and make4ht checked for this, and that is why it used j1 while on > windows linux subsystem more cores can be used and it used all available > there? > > My PC happens to have 6 cores actually. > > In this

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-21 Thread Nasser M. Abbasi
The speed is AMAZING !! minutes now instead of hours. It might be due to the use of make -j6 -f try-images.mk I have a question: How does make4ht decide on using make -j6 or other number for the parallel part? I am asking, becuase as I am testing it, I see it used make -j1 in here

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-21 Thread Michal Hoftich
> > The speed is AMAZING !! minutes now instead of hours. It might be due to > the use of > > make -j6 -f try-images.mk > > or something else. I do not know what magic Michal did, but now > svg is generated so fast and not only that, the math issue I saw before, are > gone now (may be the

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-21 Thread Michal Hoftich
> > I checked today at work. Cygwin inherits the environment from > Windows. Don't know whether it works with other releases than > Windows 7. > Thanks Reinhard, I've added this code to the make4ht extension. Best, Michal

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Nasser M. Abbasi
This is an update on using Michal new setup to speed make4ht. I've been trying it and added to it all the options I normally use in my main build. Everything is working wonderfully. The command I am using now, after some trial and error is make4ht -ulm default -e new.mk4 -c

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Reinhard Kotucha
On 2018-08-19 at 22:36:37 +0200, Reinhard Kotucha wrote: > local cpu_cnt = 4 -- set a reasonable default for non-Linux systems > > if os.name == 'linux' then > cpu_cnt = 0 > local cpuinfo=assert(io.open('/proc/cpuinfo', 'r')) > for line in cpuinfo:lines() do > if

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Martin Gieseking
Hi Michal, Just a first observation: If I understand the dvireader script correctly, it reads all bytes following a "bop" command until the "eop" value 140 is reached. Since many DVI commands require additional parameters, it's likely that one of these bytes is 140 as well so that the MD5 sum

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Michal Hoftich
Hi Nasser, > > Thank you again Michal; > > I do not understand why it works for you and it hangs for me. > > I made a small movie so you can see. This is animated GIF file, > so please click on this in your browser to see the movie > > https://www.12000.org/tmp/hangs/movie.gif > > > When I

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Nasser M. Abbasi
On 8/20/2018 7:47 AM, Michal Hoftich wrote: I got the chance now to try it. It still hangs for me. Does it not hang on your system? I've put a zip file with all you code, using your new dvisvgm-hashes.lua and a large latex file to try it on. The zip contains: I was able to compile this

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Michal Hoftich
Hi Martin > > Just a first observation: If I understand the dvireader script correctly, it > reads all bytes following a "bop" command until the "eop" value 140 is > reached. Since many DVI commands require additional parameters, it's likely > that one of these bytes is 140 as well so that the

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Michal Hoftich
> I got the chance now to try it. > > It still hangs for me. Does it not hang on your system? > > I've put a zip file with all you code, using your new dvisvgm-hashes.lua > and a large latex file to try it on. The zip contains: > I was able to compile this file and even the first one (index.tex)

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Martin Gieseking
Am 20.08.2018 um 11:52 schrieb Nasser M. Abbasi: On 8/20/2018 4:32 AM, Martin Gieseking wrote: Am 19.08.2018 um 13:53 schrieb Michal Hoftich: When I tried it on a really small file, it worked.  _BUT_ and this is a big _BUT_, some of the math generated looks really bad but some look like normal

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Michal Hoftich
Hi Reinhard, > > The advantage of make is not only that it can run processes in > parallel but that it only invokes a program when the source is newer > than the target. If pages can be processed separately, this saves an > enourmous amount of time. The processing is much slower when separate

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Martin Gieseking
Am 19.08.2018 um 13:53 schrieb Michal Hoftich: When I tried it on a really small file, it worked. _BUT_ and this is a big _BUT_, some of the math generated looks really bad but some look like normal SVG. Here is screen shot https://www.12000.org/tmp/08182018/image.png Looking at the HTML, it

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-20 Thread Martin Gieseking
Am 19.08.2018 um 01:10 schrieb Michal Hoftich: However, technically it shouldn't be necessary to convert all math fragments every time the document is processed. In the (proprietary) infrastructure at our university we only convert the portions that have actually changed. This is simply done by

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Nasser M. Abbasi
On 8/19/2018 6:53 AM, Michal Hoftich wrote: I've made a new version of dvisvgm-hashes.lua. It compresses consecutive page ranges, so it shouldn't crate such long page lists. Moreover, if GNU Make is available on the system, it makes a Makefile, which calls dvisvgm only on subset of pages (64

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Reinhard Kotucha
On 2018-08-19 at 13:53:49 +0200, Michal Hoftich wrote: > Moreover, if GNU Make is available on the system, it makes a > Makefile, which calls dvisvgm only on subset of pages (64 by > default) and it calls it in parallel (thanks to Reinhard for this > idea). It should speed up the compilation

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Michal Hoftich
Hi Reinhard, > Hi Michal, > if you use -j without an argument make will try to do everything at > the same time. This leads to zillions of processes and open > files. It's better to specify the number of available CPUs: > Ah, this explains why my computer freeze for 20 minutes when I tried

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Reinhard Kotucha
On 2018-08-19 at 13:53:49 +0200, Michal Hoftich wrote: > local command = "make -j -f " .. makefilename Hi Michal, if you use -j without an argument make will try to do everything at the same time. This leads to zillions of processes and open files. It's better to specify the number of

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Nasser M. Abbasi
On 8/19/2018 6:53 AM, Michal Hoftich wrote: I can see it. I am not sure what is the issue, maybe Martin know? The call to dvisvgm is in the form: dvisvgm -v 4 -n --exact -c 1.15,1.15 -p 1-5 filename.idv 2> filename.dlog When "-p 1-5"? instead of "-p 1-" only? Is that becuase you know only

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-19 Thread Michal Hoftich
> > It might be a linux shell piping issue, since your are second large > number of file using > destination file I noticed. > > I made sure the input latex file I gave it try_1.tex is not too > large, so not to run into the dvisvgm overflow problem. This file > has only 1,000 pages and may be

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-18 Thread Martin Gieseking
Am 18.08.2018 um 00:26 schrieb Karl Berry: Martin: please see https://tug.org/pipermail/tex4ht/2018q3/001984.html and thread following about slow operation handling a huge document (not surprising in my view). Is it perhaps possible to cache results so that dvisvgm does not have to recompute

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-17 Thread Nasser M. Abbasi
On 8/17/2018 5:26 PM, Karl Berry wrote: Martin: please see https://tug.org/pipermail/tex4ht/2018q3/001984.html and thread following about slow operation handling a huge document (not surprising in my view). Is it perhaps possible to cache results so that dvisvgm does not have to recompute every

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-17 Thread Karl Berry
Martin: please see https://tug.org/pipermail/tex4ht/2018q3/001984.html and thread following about slow operation handling a huge document (not surprising in my view). Is it perhaps possible to cache results so that dvisvgm does not have to recompute every math fragment every time? Any other

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-17 Thread Michal Hoftich
Hi Nasser, > --- foo.tex- > \documentclass[11pt]{article} > \usepackage{amsmath,mathtools,amssymb} > > %fix for Maple bad latex generated > %see > https://tex.stackexchange.com/questions/191479/how-to-automatically-convert-cases-to-begincases-endcases > > \let\amscases\cases >

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-16 Thread Nasser M. Abbasi
On 8/16/2018 7:29 AM, Michal Hoftich wrote: Hi Nasser, I am still struggling with very slow latex to HTML conversion for large document with lots of math. using svg for math, the bottleneck is always dvisvgm. For a document of 4,000 pages, with about 50,000 math expressions, it takes 8 hrs

Re: [tex4ht] what is the fastest way to convert large document to HTML?

2018-08-16 Thread Michal Hoftich
Hi Nasser, > I am still struggling with very slow latex to HTML conversion > for large document with lots of math. > > using svg for math, the bottleneck is always dvisvgm. > > For a document of 4,000 pages, with about 50,000 math expressions, it > takes 8 hrs on Linux in a Vbox and 2.5 hrs on PC

[tex4ht] what is the fastest way to convert large document to HTML?

2018-08-16 Thread Nasser M. Abbasi
I am still struggling with very slow latex to HTML conversion for large document with lots of math. using svg for math, the bottleneck is always dvisvgm. For a document of 4,000 pages, with about 50,000 math expressions, it takes 8 hrs on Linux in a Vbox and 2.5 hrs on PC running Linux. It