Re: [NTG-context] Running mtxrun in a continuous batch mode

[Ramkumar KB]

Hi,

Thank you for the tips. Yes, I will probably write something similar (but
probably in Go  or Python
, as they are easier to maintain
in the environments that I work).

Thank you very much!
Ramkumar

On Fri, Nov 6, 2020 at 10:52 PM Aditya Mahajan  wrote:

> On Fri, 6 Nov 2020, thierry horsin wrote:
>
> > Hi Ramkumar,
> >
> > Maybe you could have some scripts that check when the JSON file is
> modified
> > and if so launches the compilation. I did that sort of script in order
> to have
> > a continuous compilation process by checking every second if my current
> .tex
> > file is modified.
>
> Here is a script which does this for general tasks:
>
> http://users.fred.net/tds/lab/ftp/atchange
>
> It is also possible to build something on top of inotifywait on linux.
>
> Aditya
>
> ___
> If your question is of interest to others as well, please add an entry to
> the Wiki!
>
> maillist : ntg-context@ntg.nl /
> http://www.ntg.nl/mailman/listinfo/ntg-context
> webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
> archive  : https://bitbucket.org/phg/context-mirror/commits/
> wiki : http://contextgarden.net
>
> ___
>
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[Aditya Mahajan]

On Fri, 6 Nov 2020, thierry horsin wrote:

> Hi Ramkumar,
> 
> Maybe you could have some scripts that check when the JSON file is modified
> and if so launches the compilation. I did that sort of script in order to have
> a continuous compilation process by checking every second if my current .tex
> file is modified.

Here is a script which does this for general tasks:

http://users.fred.net/tds/lab/ftp/atchange

It is also possible to build something on top of inotifywait on linux.

Aditya
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[thierry horsin]

Hi Ramkumar,

Maybe you could have some scripts that check when the JSON file is 
modified and if so launches the compilation. I did that sort of script 
in order to have a continuous compilation process by checking every 
second if my current .tex file is modified.


Best

Thierry

On 06/11/2020 09:38, Ramkumar KB wrote:

Hans,

Thank you for the suggestions. Let me try them out and feedback back 
to you (as the batch set-up involves > 10,000 docs).


Best,
Ramkumar

On Wed, Nov 4, 2020 at 7:24 PM Hans Hagen > wrote:


On 11/4/2020 4:48 AM, Ramkumar KB wrote:
> Hans,
>
> Thank you so much for the kind response.
>
> Apologies that I was not very clear in my initial query. I have -
>
>  1. MyStatement.tex, with all the necessary fonts, static text,
images,
>     header, footer etc
>  2. MyStatement.tex reads from a JSON file for certain sections
of the
>     document for the dynamic content
>  3. context MyStatement.tex produces MyStatement.pdf
>
> How do I do the above steps 1 to 3 in a continuous fashion such
that -
>
>   * Output is MyStatement_1.pdf (reads from data_1.json),
>     MyStatement_2.pdf (reads from data_2.json) and so on
>   * The tex processing is fast as fonts etc are loaded once (as
the Tex
>     template is same for all the MyStatement_n.pdf)
>
>  From the sample code that you gave, I get a hint that this can be
> possibly done using ConTeXt Lua Document approach. Is this correct?
i'd just write a script that calls context like

context mystatement.tex --myjsonsection=1 --batch
context mystatement.tex --myjsonsection=2 --batch
context mystatement.tex --myjsonsection=3 --batch

etc .. maybe --once if no multipass is needed, or --runs=2 if you
know
how many runs are needed

to stay in a run and kind of restart is asking for troubles
because what
should be reset? of course i could context make do that buit it
doesn't
pay off

an alternative that you generate one document with all statements and
use mutool to split of the pages

but anyway, nowadays machines are fast enough to have separate
runs (and
one can run them in parallel)

Hans


-
                                           Hans Hagen | PRAGMA ADE
               Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
        tel: 038 477 53 69 | www.pragma-ade.nl
 | www.pragma-pod.nl

-


___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[Ramkumar KB]

Hans,

Thank you for the suggestions. Let me try them out and feedback back to you
(as the batch set-up involves > 10,000 docs).

Best,
Ramkumar

On Wed, Nov 4, 2020 at 7:24 PM Hans Hagen  wrote:

> On 11/4/2020 4:48 AM, Ramkumar KB wrote:
> > Hans,
> >
> > Thank you so much for the kind response.
> >
> > Apologies that I was not very clear in my initial query. I have -
> >
> >  1. MyStatement.tex, with all the necessary fonts, static text, images,
> > header, footer etc
> >  2. MyStatement.tex reads from a JSON file for certain sections of the
> > document for the dynamic content
> >  3. context MyStatement.tex produces MyStatement.pdf
> >
> > How do I do the above steps 1 to 3 in a continuous fashion such that -
> >
> >   * Output is MyStatement_1.pdf (reads from data_1.json),
> > MyStatement_2.pdf (reads from data_2.json) and so on
> >   * The tex processing is fast as fonts etc are loaded once (as the Tex
> > template is same for all the MyStatement_n.pdf)
> >
> >  From the sample code that you gave, I get a hint that this can be
> > possibly done using ConTeXt Lua Document approach. Is this correct?
> i'd just write a script that calls context like
>
> context mystatement.tex --myjsonsection=1 --batch
> context mystatement.tex --myjsonsection=2 --batch
> context mystatement.tex --myjsonsection=3 --batch
>
> etc .. maybe --once if no multipass is needed, or --runs=2 if you know
> how many runs are needed
>
> to stay in a run and kind of restart is asking for troubles because what
> should be reset? of course i could context make do that buit it doesn't
> pay off
>
> an alternative that you generate one document with all statements and
> use mutool to split of the pages
>
> but anyway, nowadays machines are fast enough to have separate runs (and
> one can run them in parallel)
>
> Hans
>
>
> -
>Hans Hagen | PRAGMA ADE
>Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
> tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -
>
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[Hans Hagen]

On 11/4/2020 4:48 AM, Ramkumar KB wrote:

Hans,

Thank you so much for the kind response.

Apologies that I was not very clear in my initial query. I have -

 1. MyStatement.tex, with all the necessary fonts, static text, images,
header, footer etc
 2. MyStatement.tex reads from a JSON file for certain sections of the
document for the dynamic content
 3. context MyStatement.tex produces MyStatement.pdf 


How do I do the above steps 1 to 3 in a continuous fashion such that -

  * Output is MyStatement_1.pdf (reads from data_1.json),
MyStatement_2.pdf (reads from data_2.json) and so on
  * The tex processing is fast as fonts etc are loaded once (as the Tex
template is same for all the MyStatement_n.pdf)

 From the sample code that you gave, I get a hint that this can be 
possibly done using ConTeXt Lua Document approach. Is this correct?

i'd just write a script that calls context like

context mystatement.tex --myjsonsection=1 --batch
context mystatement.tex --myjsonsection=2 --batch
context mystatement.tex --myjsonsection=3 --batch

etc .. maybe --once if no multipass is needed, or --runs=2 if you know 
how many runs are needed


to stay in a run and kind of restart is asking for troubles because what 
should be reset? of course i could context make do that buit it doesn't 
pay off


an alternative that you generate one document with all statements and 
use mutool to split of the pages


but anyway, nowadays machines are fast enough to have separate runs (and 
one can run them in parallel)


Hans


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
   tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[Ramkumar KB]

Hans,

Thank you so much for the kind response.

Apologies that I was not very clear in my initial query. I have -

   1. MyStatement.tex, with all the necessary fonts, static text, images,
   header, footer etc
   2. MyStatement.tex reads from a JSON file for certain sections of the
   document for the dynamic content
   3. context MyStatement.tex produces MyStatement.pdf

How do I do the above steps 1 to 3 in a continuous fashion such that -

   - Output is MyStatement_1.pdf (reads from data_1.json), MyStatement_2.pdf
   (reads from data_2.json) and so on
   - The tex processing is fast as fonts etc are loaded once (as the Tex
   template is same for all the MyStatement_n.pdf)

>From the sample code that you gave, I get a hint that this can be possibly
done using ConTeXt Lua Document approach. Is this correct?

Thank you once again,

Best,
Ramkumar

On Tue, Nov 3, 2020 at 7:02 PM Hans Hagen  wrote:

> On 11/3/2020 1:55 AM, Ramkumar KB wrote:
> > Hello,
> >
> > I am trying to run mtxrun in a batch process and read the available
> > documentation here -
> > https://wiki.contextgarden.net/Running_ConTeXt_without_a_shell
> >
> > My usecase is being able to create PDF docs based on the input data that
> > comes in a JSON format (in a continuous batch process). In ConTeXt,
> > thanks to the Lua engine, I am able to inject a JSON read from a file -
> > and use the JSON in the tex file (here variable `tab` holds the JSON).
> >
> > How do I make this as a continuous webservice - in comes JSON and out
> > comes the PDF ?
> >
> > \startluacode
> > require("test-json")
> > \stopluacode
> >
> > The name of the document is \ctxlua{tex.print(tab['documentName'])}
> >
> > Any tips or pointers would be much appreciated.
> foo.cld :
>
> require("util-jsn")
>
> -- local str = io.loaddata("somefile.json")
>
> local str = [[ {
>  "title": "Some JSON",
>  "text" : "Just an example.",
>  "data" : [
>  { "a" : "first 1", "b" : "last 1" },
>  { "b" : "last 2", "a" : "first 2" }
>  ]
> } ]]
>
> local tmp = utilities.json.tolua(str)
>
> context.starttext()
>
>  context.starttitle { title = tmp.title }
>
>  context(tmp.text) context.par()
>
>  context.starttabulate { "|c|c|" }
>  for i=1,#tmp.data do
>  local d = tmp.data[i]
>  context.NC()
>  context(d.a) context.NC()
>  context(d.b) context.NC()
>  context.NR()
>  end
>  context.stoptabulate()
>
>  context.stoptitle()
>
> context.stoptext()
>
> and then
>
> context foo.cld
>
>
> -
>Hans Hagen | PRAGMA ADE
>Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
> tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
> -
>
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___

Re: [NTG-context] Running mtxrun in a continuous batch mode

[Hans Hagen]

On 11/3/2020 1:55 AM, Ramkumar KB wrote:

Hello,

I am trying to run mtxrun in a batch process and read the available 
documentation here - 
https://wiki.contextgarden.net/Running_ConTeXt_without_a_shell


My usecase is being able to create PDF docs based on the input data that 
comes in a JSON format (in a continuous batch process). In ConTeXt, 
thanks to the Lua engine, I am able to inject a JSON read from a file - 
and use the JSON in the tex file (here variable `tab` holds the JSON).


How do I make this as a continuous webservice - in comes JSON and out 
comes the PDF ?


\startluacode
require("test-json")
\stopluacode

The name of the document is \ctxlua{tex.print(tab['documentName'])}

Any tips or pointers would be much appreciated.

foo.cld :

require("util-jsn")

-- local str = io.loaddata("somefile.json")

local str = [[ {
"title": "Some JSON",
"text" : "Just an example.",
"data" : [
{ "a" : "first 1", "b" : "last 1" },
{ "b" : "last 2", "a" : "first 2" }
]
} ]]

local tmp = utilities.json.tolua(str)

context.starttext()

context.starttitle { title = tmp.title }

context(tmp.text) context.par()

context.starttabulate { "|c|c|" }
for i=1,#tmp.data do
local d = tmp.data[i]
context.NC()
context(d.a) context.NC()
context(d.b) context.NC()
context.NR()
end
context.stoptabulate()

context.stoptitle()

context.stoptext()

and then

context foo.cld


-
  Hans Hagen | PRAGMA ADE
  Ridderstraat 27 | 8061 GH Hasselt | The Netherlands
   tel: 038 477 53 69 | www.pragma-ade.nl | www.pragma-pod.nl
-
___
If your question is of interest to others as well, please add an entry to the 
Wiki!

maillist : ntg-context@ntg.nl / http://www.ntg.nl/mailman/listinfo/ntg-context
webpage  : http://www.pragma-ade.nl / http://context.aanhet.net
archive  : https://bitbucket.org/phg/context-mirror/commits/
wiki : http://contextgarden.net
___