Tom, Interesting thoughts; I'll consider putting a package together. For the time being, I've implemented these ideas in Gaston, and after some testing it looks like the implementation is solid.
Thanks, -- mb On Tue, Jan 12, 2016 at 2:36 AM, Tomas Lycken <[email protected]> wrote: > The other global variables actually hold data, which changes according to > the values read from the different streams. Inspired by the wrapper idea, I > wrapped them in a type so that it becomes clear that they are always > strings; hopefully this will help with performance. > > You won’t see any performance difference from just wrapping them. However, > since you can now rebind the fields of the wrappers (which have strictly > specified types) instead of rebinding the variables themselves, all those > variables can now be marked const. This should give you some additional > speed gain. > > Another thing to consider, is that you probably want to be able to do this > with several commands in parallel. (Consider, for example, running multiple > echo programs and passing stuff between them) For that, you would need to > wrap all of this in a function anyway. I would consider the following API: > > function popen3(f::Function, cmd::Cmd) > # setup all streams, like your current popen3 function does > > # also, create all state handling for the different streams > > # finally, pass suitable arguments to f to let the user provide the > actual action > f(pin.in, out.out, err.out) # for example, maybe something else makes > more sense > end > > The caller can then use a do block to consume this: > > popen3(`./inout`) do pin, pout, perr > # use the streams here > end > > Since it’s all now wrapped in a function, “globals” won’t be a performance > problem anymore. As a bonus, you can work with several, parallel, > invocations without conflict between them. You might still see some > slowness due to passing anonymous functions, but it’s a) probably > negligible compared to the IO cost, and b) it’s actively being worked on > <https://github.com/JuliaLang/julia/pull/13412> and will eventually be > fast. > > // T > > On Tuesday, January 12, 2016 at 3:50:04 AM UTC+1, Miguel Bazdresch wrote: > > Tomas, >> >> Thanks for your feedback! I couldn't find a package that handles this >> already. If this technique proves to work reliably in Gaston, I'll be happy >> to put it in a package. >> >> Regarding `readnow`, I never rebind it. It doesn't hold any actual value; >> it is just a `Condition()`, which can be used to notify asynchronous tasks. >> I followed your advice and made it a `const`. >> >> The other global variables actually hold data, which changes according to >> the values read from the different streams. Inspired by the wrapper idea, I >> wrapped them in a type so that it becomes clear that they are always >> strings; hopefully this will help with performance. >> >> The latest version of the code is at >> https://gist.github.com/mbaz/bb7e2cbaaecc031b1d88 >> >> -- mb >> >> On Mon, Jan 11, 2016 at 9:05 AM, Tomas Lycken <[email protected]> >> wrote: >> >>> Interesting question! If you find a good approach to do this, wrapping >>> it in a package is certainly interesting (have you checked that there >>> aren’t any packages that handle this already?). >>> >>> I can’t help much with the threading stuff, but regarding your global >>> variable it should be possible to work around the slowness of that. I don’t >>> know how, or when, you re-bind readnow, but there are two ways to fix >>> it depending on the specifics: >>> >>> 1. >>> >>> You never rebind readnow, just mutate it. Mark it const, and it’s >>> fast. (const means *impossible to rebind*, not *impossible to mutate* >>> ) >>> 2. >>> >>> You rebind readnow, and Condition() is mutable: Mark it const and >>> mutate it instead of rebinding. >>> 3. >>> >>> You rebind readnow and Condition() is immutable: Wrap the Condition >>> in a mutable type, which you assign to const readnow instead, and >>> then rebind the field in this mutable type. Something like this: >>> >>> type ConditionWrapper >>> c::Condition >>> end >>> >>> const readnow = ConditionWrapper(Condition()) >>> >>> # where you update: >>> readnow.c = Condition(args...) >>> >>> This all assumes that Condition is an immutable concrete type, and you >>> just want to switch it out for an instance with other field values. If you >>> actually need to change the *type* of readnow, all bets are off and >>> this trick won’t work. >>> >>> // T >>> >>> On Friday, January 8, 2016 at 9:28:02 PM UTC+1, Miguel Bazdresch wrote: >>> >>> Hello, >>>> >>>> I'd be grateful if you could take a look at some code and suggest >>>> improvements. >>>> >>>> I'm trying to interact with a long-lived process (gnuplot). This >>>> process reads commands from its STDIN; after each command is executed, it >>>> produces output on either its STDOUT or STDERR. It's impossible to predict >>>> ahead of time which of the two streams will be used. >>>> >>>> To simplify my tests, I wrote an "echo server" in C, which reads a >>>> character from its STDIN and outputs it again over either STDOUT or STDERR. >>>> The code is here: https://gist.github.com/mbaz/1e242694a9c4f1eca576 >>>> >>>> Then I wrote a julia program that reads a character from its own STDIN, >>>> sends it to the C echo server, and then tries to read the server's STDOUT >>>> and STDERR until it finds a character. The code is here: >>>> https://gist.github.com/mbaz/bb7e2cbaaecc031b1d88 >>>> >>>> This code works, but I don't know if it is the best approach. Two >>>> specific questions I have: >>>> >>>> * Is my `popen3()` function necessary? This closed issue seems to >>>> suggest it's not, but I can't figure out how to accomplish what I need in a >>>> more simple manner: https://github.com/JuliaLang/julia/issues/11824 >>>> >>>> * Are global variables required to share data with asynchronous tasks? >>>> Since global variables are slow, this approach may produce undesired code >>>> slowdowns. >>>> >>>> Thanks, >>>> >>>> -- mb >>>> >>> >>> >> >> >
