Re: [fpc-pascal] TProcess vs RunProcess()

[Sven Barth]

Am 30.01.2017 11:07 schrieb "Lars" :
>
> On Mon, January 30, 2017 1:06 am, Marco van de Voort wrote:
> >> Reason I want to avoid TProcess is simply because cgi programs being
> >> around 30-100K are easier to upload for people with limited internet
> >> connection speeds. As soon as I pull in large units that involve lots
> >> of classes the cgi programs become over 200-500K.
> >
> > Yes. And they might fit on one side of a 5.25" inch dd floppy too! Yeah,
> > let's party like it's 1999!
>
>
> You don't seem to understand the concept of RAD web development ;-)

RAD is for Rapid Application *Development*, not Deployment.
For developing/testing applications one should use a local server anyway as
that also allows to use a debugger if necessary.

Regards,
Sven
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Mon, January 30, 2017 1:06 am, Marco van de Voort wrote:
>> Reason I want to avoid TProcess is simply because cgi programs being
>> around 30-100K are easier to upload for people with limited internet
>> connection speeds. As soon as I pull in large units that involve lots
>> of classes the cgi programs become over 200-500K.
>
> Yes. And they might fit on one side of a 5.25" inch dd floppy too! Yeah,
> let's party like it's 1999!


You don't seem to understand the concept of RAD web development ;-)

If it takes me 2 minutes to upload a massive 2mb cgi program through FTP,
since I pay half price for my internet connection for half speed
DSL/Cable.. That's not rad.

Sometimes I develop a cgi program and upload it every few minutes, so
waiting 1-2 minutes for a cgi to upload with total commander is just
pathetic.

Uploading a 50kb or 30kb cgi is where it is at.

I live in a first world country but in third world countries with dial up
or whatever slow dsl they use, I can imagine a 2MB golang cgi (or lazarus
zeos/classes/other cgi) taking more than 2 minutes. That's not RAD

Just try compling some hello world or simple programs in lazarus and
imagine they take 2 minutes to compile. That's what C++ compilers do.  If
you are a c++ fan I guess 2MB/2minute upload cgi programs are
satisfactory...

Some of us do RAD cgi programming where everything is instant.

Good for you living in a country where dsl/cable is very fast and you
don't care about FTP upload times.  I think they call that being spoiled,
or being white or something.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Mon, January 30, 2017 12:57 am, Marco van de Voort wrote:
>
>> Or, get assignstream working on windows and not just unix, but that
>> will be a bit of a big undertaking, and again may not be possible.
>
> Again, duplicating TProcess.
>

Except assignstream doesn't require TProcess.Create and Free and uses old
school programming, which, for some reason, I'm addicted to. Maybe just
stuck in 1970's even though I was born in 1980s
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Lars said:
> > Executeprocess doesn't support piping. It orignally was mainly meant as a
> >  portable dos.exec with parameters passed separately and without string
> > length limits, implementable on RTL level.
> 
> But doesn't all processes report to stderr?

Afaik there are only conventions. And the convention to return a non zero
returnvalue on error is much stronger than parsing stderr. 

> Another idea is to modify AssignStream to work on windows as AFAIK it only
> works on unix from the unix units of rtl..

Yes, Unix only, and the abstracted OS independent pipes already are in unit
pipes used by TProcess.
 
> Reason I want to avoid TProcess is simply because cgi programs being
> around 30-100K are easier to upload for people with limited internet
> connection speeds. As soon as I pull in large units that involve lots of
> classes the cgi programs become over 200-500K.

Yes. And they might fit on one side of a 5.25" inch dd floppy too! Yeah,
let's party like it's 1999!

> However I will use tprocess if it's a time save, which indeed it is
> instead of rewriting assignstream for windows.

Good.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Lars said:
> > Indeed, that is one of the big differences between TProcess and
> > ExecuteProcess. I use TProcess often to execute something and then
> > capture the output results.  fpGUI's example IDE (Maximus) shows how it
> > can be done with TProcess.
> 
> Well I may modify executeprocess/fpsystem to deal with stderr if that is a
> missing feature (may not be possible, haven't looked into it much).

Not without effectively changing it into TProcess. 
 
> Or, get assignstream working on windows and not just unix, but that will
> be a bit of a big undertaking, and again may not be possible.

Again, duplicating TProcess.

> mseide/msegui may also have some tips in its source code as it captures a
> lot of stuff and puts it into the ide in its custom widgets that trap
> outputs, actually kind of a bit like a Blackbox ComponentPascal IDE from
> switzerland, but still different ;-)

So thus Lazarus. It uses TProcess.

Hint: use TProcess :_)
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Jonas Maebe]

Lars wrote:
> Calling the shell is useful for not requiring absolute paths to
> commands... which is why I like fpSystem...

Never ship a program that uses fpSystem for that purpose, because it's
one of the most basic security holes you can have:
https://www.securecoding.cert.org/confluence/pages/viewpage.action?pageId=2130132


Jonas
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Jonas Maebe]

Lars wrote:
> But doesn't all processes report to stderr?
> Ones that are created in a program do not report to stderr?
> 
> The OS makes it optional if you want to report to stderr?  Interesting..

Every process (on Windows: every non-GUI process) on maintstream OSes
(and on many non-mainstream OSes too) has by default two standard file
handles available to which it can write output: stdout and stderr.


Jonas
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Sun, January 29, 2017 3:18 am, Marco van de Voort wrote:
> In our previous episode, Lars said:
>
>> Assign(StdErr, 'somefile.txt')
>> Rewrite(StdErr)
>>
>>
>> And didn't seem to work
>>
>>
>> So if you run a process and nothing prints to stdout, and there is some
>>  data printed to stderr, I do not know how to capture it. Which makes
>> me want to try TProcess instead, but if I could do it with FpSystem and
>> ExecuteProcess that would be nice
>>
>
> Executeprocess doesn't support piping. It orignally was mainly meant as a
>  portable dos.exec with parameters passed separately and without string
> length limits, implementable on RTL level.

But doesn't all processes report to stderr?
Ones that are created in a program do not report to stderr?

The OS makes it optional if you want to report to stderr?  Interesting..

Another idea is to modify AssignStream to work on windows as AFAIK it only
works on unix from the unix units of rtl..

Quote:
function AssignStream(var StreamIn, StreamOut, StreamErr: Text; const
prog: String): LongInt;
{
  Starts the program in 'prog' and makes its input, output and error output
  the other end of three pipes, which are the stdin, stdout and stderr of a
  program specified in 'prog'.

^^^ very useful function!

Reason I want to avoid TProcess is simply because cgi programs being
around 30-100K are easier to upload for people with limited internet
connection speeds. As soon as I pull in large units that involve lots of
classes the cgi programs become over 200-500K.

However I will use tprocess if it's a time save, which indeed it is
instead of rewriting assignstream for windows.

Plus, golang cgi exe's are about 2MB in size, so golang ain't got anything
on fpc at this moment (fpc still creates small programs compared to GoLang
runtime programs!)
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Sun, January 29, 2017 10:46 am, Graeme Geldenhuys wrote:
> On 2017-01-29 07:59, Lars wrote:
>
>> So if you run a process and nothing prints to stdout, and there is some
>>  data printed to stderr, I do not know how to capture it. Which makes
>> me want to try TProcess instead
>
>
> Indeed, that is one of the big differences between TProcess and
> ExecuteProcess. I use TProcess often to execute something and then
> capture the output results.  fpGUI's example IDE (Maximus) shows how it
> can be done with TProcess.

Well I may modify executeprocess/fpsystem to deal with stderr if that is a
missing feature (may not be possible, haven't looked into it much).

Or, get assignstream working on windows and not just unix, but that will
be a bit of a big undertaking, and again may not be possible.

mseide/msegui may also have some tips in its source code as it captures a
lot of stuff and puts it into the ide in its custom widgets that trap
outputs, actually kind of a bit like a Blackbox ComponentPascal IDE from
switzerland, but still different ;-)

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Sun, January 29, 2017 2:41 pm, Marco van de Voort wrote:
> Some likely scenarios (e.g. run and capture stdout) have been prepared in
>  RunCommand, also in unit Process.
>
>
> The main motivation for adding these is that 90+% of the TProcess and
> CreateProcess usage in the forums was faulty (specially for larger outputs
>  or when the process also printed a lot on stderr (thanks Pierre!)).
>

So people were having issues with stderr going crazy (lots of prints to
stderr)?

And this is the problem I have with executeprocess: I want the results
from stderr, and need them, but can't figure out how to access them.. only
stdout is what I retrieve writes from.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Sun, January 29, 2017 1:36 pm, Santi wrote:

>  Maybe ExecuteProcess is calling some kind of shell,
> Freepascal developers will tell.
>


To call the shell you send in cmd /C

FpSystem does this, but it's on unix

Calling the shell is useful for not requiring absolute paths to
commands... which is why I like fpSystem...

And to call a dos command you need a shell like cmd /C because calling a
dos command won't do anything, it's not an exe (there is no DIR.exe file
for example AFAIK).. so to call DIR (same as LS on unix) you need to
invoke a shell.


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Santi said:
> and people nowadays love to name directories and files with strings that 
> instead of titles look like abstracts.
> 
> In delphi and other languages I've hit may times the 260 character limit 
> in windows running  a Shell or using ShellApi (I supposed the limit was 
> 255, not 260, there are always new things to learn) , I have also hit 
> that limit using the command line. Nevertheless using windows API 
> createprocess, there is not such limit (but the call is less handy and 
> more verbose)

There is a lot of confusion about this. (only -W calls can do larger
versions, and only if \\.\, but there are also reports that it works with -A
as long as the system is capable)

> or maybe I haven't hit it because parameters and call function are
> separated strings , so they are shorter strings.  On the other hand,
> CreateProcess must call an executable file, can't execute internal
> commands like 'dir' or 'del', not a big deal but there it is.  I think
> that the limit is in the command line interpreters command.exe and
> cmd.exe.

Internal commands are not internal to the OS, but internal to the shell, so
you need to execute a shell for that. On both Linux and Windows.
 
> As far as I see, ExecuteProcess finally calls to an external symbol 
> named 'FPC_SYSCALLx'. And TProcess finally calls an external symbol 
> 'FPC_SYSC_FORK'. Maybe ExecuteProcess is calling some kind of shell, 
> Freepascal developers will tell.

On *nix, both will call fork followed by execve. On Windows it will call
CreateProcess, though executeprocess will call -W in trunk, and afaik
TProcess still -A.

> In my delphi days we implemented a procedure ("RunCommand" was it's 
> name, If memory serves), that was a warp to a complex call to 
> createprocess to avoid these problems. Maybe executeProcess could be 
> implemented this way. Don't know what are the drawbacks (slower, more 
> memory..?)

ExecuteProcess is to have some portable execution functionality in the base
RTL. For more complex cases there is the TProcess class that also supports
piping etc.

Some likely scenarios (e.g. run and capture stdout) have been prepared in
RunCommand, also in unit Process.

The main motivation for adding these is that 90+% of the TProcess and
CreateProcess usage in the forums was faulty (specially for larger outputs
or when the process also printed a lot on stderr (thanks Pierre!)).

Abstracting them in RunCommand reduces the amount of faulty code in play.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Santi]

El 27/01/17 a las 18:24, Graeme Geldenhuys escribió:

If you have a huge list of parameters, RunProcess() - at least under
Windows - will most likely hit a 260 character limit, and then fail to
actually execute. I don't actually know where that 260 limit originates
from (Windows API, Command Prompt, FPC etc).
it is easy to use more that 260 if you pass file names with long paths, 
and people nowadays love to name directories and files with strings that 
instead of titles look like abstracts.


In delphi and other languages I've hit may times the 260 character limit 
in windows running  a Shell or using ShellApi (I supposed the limit was 
255, not 260, there are always new things to learn) , I have also hit 
that limit using the command line. Nevertheless using windows API 
createprocess, there is not such limit (but the call is less handy and 
more verbose), or maybe I haven't hit it because parameters and call 
function are separated strings , so they are shorter strings. On the 
other hand, CreateProcess must call an executable file, can't execute 
internal commands like 'dir' or 'del', not a big deal but there it is. I 
think that the limit is in the command line interpreters command.exe and 
cmd.exe.


As far as I see, ExecuteProcess finally calls to an external symbol 
named 'FPC_SYSCALLx'. And TProcess finally calls an external symbol 
'FPC_SYSC_FORK'. Maybe ExecuteProcess is calling some kind of shell, 
Freepascal developers will tell.


In my delphi days we implemented a procedure ("RunCommand" was it's 
name, If memory serves), that was a warp to a complex call to 
createprocess to avoid these problems. Maybe executeProcess could be 
implemented this way. Don't know what are the drawbacks (slower, more 
memory..?)


___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Graeme Geldenhuys]

On 2017-01-29 07:59, Lars wrote:
> So if you run a process and nothing prints to stdout, and there is some
> data printed to stderr, I do not know how to capture it. Which makes me
> want to try TProcess instead


Indeed, that is one of the big differences between TProcess and
ExecuteProcess. I use TProcess often to execute something and then
capture the output results.  fpGUI's example IDE (Maximus) shows how it
can be done with TProcess.

Regards,
  Graeme

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Lars said:
> Assign(StdErr, 'somefile.txt')
> Rewrite(StdErr)
> 
> And didn't seem to work
> 
> So if you run a process and nothing prints to stdout, and there is some
> data printed to stderr, I do not know how to capture it. Which makes me
> want to try TProcess instead, but if I could do it with FpSystem and
> ExecuteProcess that would be nice

Executeprocess doesn't support piping. It orignally was mainly meant as a
portable dos.exec with parameters passed separately and without string
length limits, implementable on RTL level.

If I were you I would simply look at tprocess sources how it pipes stderr.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Fri, January 27, 2017 10:24 am, Graeme Geldenhuys wrote:
> On 2017-01-26 14:28, Michael Van Canneyt wrote:
>
>> TProcess gives you full access to the process. You can kill it,
>> pause it, write to stdin, read from stdout.
>>
>> These things cannot be done with RunProcess.
>> That's one-shot and wait till it exits.
>>
>
> Another difference simply so others might learn from this discussion
> (like I did).
>
>
> If you have a huge list of parameters, RunProcess() - at least under
> Windows - will most likely hit a 260 character limit, and then fail to
> actually execute. I don't actually know where that 260 limit originates
> from (Windows API, Command Prompt, FPC etc).
>

One issue I discovered with ExecuteProcess, and fpSystem, is I cannot
figure out a way to capture StdErr... TProcess likely has more control
over this. Tried

Assign(StdErr, 'somefile.txt')
Rewrite(StdErr)

And didn't seem to work

So if you run a process and nothing prints to stdout, and there is some
data printed to stderr, I do not know how to capture it. Which makes me
want to try TProcess instead, but if I could do it with FpSystem and
ExecuteProcess that would be nice
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Lars]

On Thu, January 26, 2017 7:28 am, Michael Van Canneyt wrote:
>

>
> On Thu, 26 Jan 2017, Graeme Geldenhuys wrote:
>
>
>> Hi,
>>
>>
>> Whats the differences or pros and cons between using TProcess vs
>> RunProcess().
>>
>
> TProcess gives you full access to the process. You can kill it,
> pause it, write to stdin, read from stdout.
>
> These things cannot be done with RunProcess.
> That's one-shot and wait till it exits.
>

Similar to FpSystem() and ExecuteProcess, and then there is AssignStream
but assignstream/fpsystem unix only

http://www.freepascal.org/docs-html/rtl/sysutils/executeprocess.html

http://www.freepascal.org/docs-html/rtl/unix/assignstream.html

http://www.freepascal.org/docs-html/rtl/unix/fpsystem.html

I like fpsystem because it does not require absolute paths to commands,
too bad fpsystem not available on Windows.

All the above is used in webcmd (part of powtils) and also fpcRun ;-)

But I may try RunProcess/TProcess soon!
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Graeme Geldenhuys said:
> > 
> > if you are talking about executeprocess, test again with trunk.
> 
> At the moment we are using FPC 2.6.4 and might move to FPC 3.0.2 when
> that is final. Not using Trunk I'm afraid.
> 
> Was the 260 char limit removed/fixed in FPC trunk?

First, I'm not aware of a different between the two, so if you have tests
for that I'm interested. (at least if you are testing 3.0 or newer, but that
I can easily do myself if you have simple reproduction case).

The strange thing is that all variants use CreateProcess, but afaik only
executeprocess in trunk uses the -w variant. I started upgrading tprocess,
but it got out of hand, and I shelved the work.

Afaik by prefixing with \\.\ you can use larger paths with -w variants.

___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Marco van de Voort]

In our previous episode, Graeme Geldenhuys said:
> > pause it, write to stdin, read from stdout.
> > 
> > These things cannot be done with RunProcess. 
> > That's one-shot and wait till it exits.
> 
> Another difference simply so others might learn from this discussion
> (like I did).

> If you have a huge list of parameters, RunProcess() - at least under
> Windows - will most likely hit a 260 character limit, and then fail to
> actually execute. I don't actually know where that 260 limit originates
> from (Windows API, Command Prompt, FPC etc).

Which runprocess are you talking about? I know a process.runcommand and a
sysutils.executeprocess, but not a runprocess.

> Whereas if you use TProcess and the Parameters (TStrings) property, you
> don't have such a limitation.
> 
> This was what I experienced.

if you are talking about executeprocess, test again with trunk.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Graeme Geldenhuys]

On 2017-01-26 14:28, Michael Van Canneyt wrote:
> TProcess gives you full access to the process. You can kill it,
> pause it, write to stdin, read from stdout.
> 
> These things cannot be done with RunProcess. 
> That's one-shot and wait till it exits.

Another difference simply so others might learn from this discussion
(like I did).

If you have a huge list of parameters, RunProcess() - at least under
Windows - will most likely hit a 260 character limit, and then fail to
actually execute. I don't actually know where that 260 limit originates
from (Windows API, Command Prompt, FPC etc).

Whereas if you use TProcess and the Parameters (TStrings) property, you
don't have such a limitation.

This was what I experienced.

Regards,
  Graeme

-- 
fpGUI Toolkit - a cross-platform GUI toolkit using Free Pascal
http://fpgui.sourceforge.net/

My public PGP key:  http://tinyurl.com/graeme-pgp
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Graeme Geldenhuys]

On 2017-01-26 14:28, Michael Van Canneyt wrote:
> These things cannot be done with RunProcess. 
> That's one-shot and wait till it exits.


Perfect and very handy to know the difference. Thanks for the explanation.



Regards,
  Graeme
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal

Re: [fpc-pascal] TProcess vs RunProcess()

[Michael Van Canneyt]

On Thu, 26 Jan 2017, Graeme Geldenhuys wrote:


Hi,

Whats the differences or pros and cons between using TProcess vs
RunProcess().


TProcess gives you full access to the process. You can kill it,
pause it, write to stdin, read from stdout.

These things cannot be done with RunProcess. 
That's one-shot and wait till it exits.


Michael.
___
fpc-pascal maillist  -  fpc-pascal@lists.freepascal.org
http://lists.freepascal.org/cgi-bin/mailman/listinfo/fpc-pascal