[julia-users] Re: Control system library for Julia?
Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting, etc. Compile-run-tweak cycles are not a convenient way to do control design (or much of anything, let's be honest). Generating embedded controllers in LLVM IR rather than C is currently very atypical, but eventually I think it could be made to work. A Julia replacement for realtime workshop is a good long-term goal. On Sunday, September 14, 2014 12:58:36 PM UTC-7, Andrew Wagner wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object files and link against them from C? Then you'd still have to write glue code in a systems language, but at least the implementation of the controller wouldn't have to cross a language boundary... Cheers, Andrew On Thursday, February 20, 2014 10:56:20 PM UTC+1, Uwe Fechner wrote: Hello, I could not find any control system library for Julia yet. Would that make sense? There is a control system library available for Python: http://www.cds.caltech.edu/~murray/wiki/index.php/Python-control Perhaps this could be used as starting point? I think that implementing this in Julia should be easier and faster than in Python. Any comments? Should I open a feature request? Uwe Fechner, TU Delft, The Netherlands
Re: [julia-users] Re: Control system library for Julia?
Hi Spencer! My job in airborne wind energy is ending soon so I don't have a specific application (aside from control), but I would want to stay sub-ms for anything in-process. I have been using Orocos extensively for the last few years. It's the best control middleware in the open source world, but I think a lot of things could be improved if it was re-implemented in a language with a better typesystem and introspection... one example would be that adding a new type to the system requires quite a bit of boilerplate code, creating an incentive for users to just pass data in flat arrays, subverting type safety. Cheers, Andrew On Mon, Sep 15, 2014 at 7:03 AM, Spencer Russell spencer.f.russ...@gmail.com wrote: Hi Andrew, What are your realtime deadlines? I'm working on live audio processing stuff with Julia, where I'd like to get the audio latency down into a few ms. Julia definitely isn't there yet (and might never get true hard-realtime), but there's some promising work being done on the GC to reduce pause time for lower-latency applications. It's also helpful to profile the code to reduce allocations (and the need for GC) down to a minimum. I haven't yet gotten down to zero-allocation code in my render loop, but once I got it down below 100 bytes I moved on to other more pressing features. At some point I'll dig deeper to see if I can get rid of the last few allocations. I'd definitely be happy if there are some more folks out there driving demand for lower-latency Julia. :) peace, s On Sun, Sep 14, 2014 at 3:58 PM, Andrew Wagner drewm1...@gmail.com wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object files and link against them from C? Then you'd still have to write glue code in a systems language, but at least the implementation of the controller wouldn't have to cross a language boundary... Cheers, Andrew On Thursday, February 20, 2014 10:56:20 PM UTC+1, Uwe Fechner wrote: Hello, I could not find any control system library for Julia yet. Would that make sense? There is a control system library available for Python: http://www.cds.caltech.edu/~murray/wiki/index.php/Python-control Perhaps this could be used as starting point? I think that implementing this in Julia should be easier and faster than in Python. Any comments? Should I open a feature request? Uwe Fechner, TU Delft, The Netherlands
Re: [julia-users] Does Julia have something similar to Python's documentation string?
On Saturday, September 13, 2014 2:26:18 AM UTC+2, Michael Hatherly wrote:
Hi Francesco,
Docile.jl partially covers what I think you're wanting out of your
docstrings, namely testing examples for correctness. I've been thinking
about exporting the docstrings to ijulia notebooks which might provide a
more interactive experience in some cases.
Docile.jl looks great, but I think that the API should be made into
comments. One of Julia's goals is to have a simple syntax that even people
who are not acquainted with programming can easily understand.
I still think that the best solution would be to create a new AST object to
handle blocks of comments tagged as documentation. Maybe blocks of comments
starting with # followed by a special character. I also think that
documentation examples/tests should also be parsed by Julia's parser as
valid AST objects, that is blocks containing input code and expected
answers.
I believe that a tagged comment is much more readable than a block
introduced by @doc or doc.
Possible examples:
##
# docstring
#
# Metadata {
# key1 = value1
# }
#
# Examples
#
#
# julia f(3)
# 9
#
function f(x)
x^2
end
or an other idea:
#=
docstring for square number [references.wikip]
.metadata {
key1 = value1,
key2 = value2,
...
}
Examples
julia f(3)
9
.references (
wikip : https://en.wikipedia.org/wiki/Square_number
)
=#
function f(x)
x^2
end
Re: [julia-users] Does Julia have something similar to Python's documentation string?
Docile.jl looks great, but I think that the API should be made into comments. One of Julia's goals is to have a simple syntax that even people who are not acquainted with programming can easily understand. Python, despite using docstrings, is a great example of a language having a simple syntax that ... understand I believe that a tagged comment is much more readable than a block introduced by @doc or doc. Much more readable is maybe a bit exaggerated, can you explain why you believe so?
Re: [julia-users] Re: Control system library for Julia?
Hi, I am working an airborne wind energy as well. I wrote a kite-power system simulator in Python, where also one of the controllers (the winch controller) is implemented in Python. ( https://bitbucket.org/ufechner/freekitesim ) With Python you can reach a jitter of less than 2 ms in the 20Hz control loop quite easily (on low-latency Linux). In my case this is sufficient for prototyping, but the real flight control system should run at a higher update frequency (perhaps 200 Hz). In contrast to Julia Python is using reference counting, and in my Python applications I just turn off the garbage collection. For Julia (and I would love to rewrite the simulator in Julia) this is probably not an option. A better garbage collector (which is in the pipeline, see: https://github.com/JuliaLang/julia/pull/5227 ) would definitely help. Generating embedded controllers in LLVM IR would be great! Best regards: Uwe On Monday, September 15, 2014 8:32:02 AM UTC+2, Andrew Wagner wrote: Hi Spencer! My job in airborne wind energy is ending soon so I don't have a specific application (aside from control), but I would want to stay sub-ms for anything in-process. I have been using Orocos extensively for the last few years. It's the best control middleware in the open source world, but I think a lot of things could be improved if it was re-implemented in a language with a better typesystem and introspection... one example would be that adding a new type to the system requires quite a bit of boilerplate code, creating an incentive for users to just pass data in flat arrays, subverting type safety. Cheers, Andrew On Mon, Sep 15, 2014 at 7:03 AM, Spencer Russell spencer@gmail.com javascript: wrote: Hi Andrew, What are your realtime deadlines? I'm working on live audio processing stuff with Julia, where I'd like to get the audio latency down into a few ms. Julia definitely isn't there yet (and might never get true hard-realtime), but there's some promising work being done on the GC to reduce pause time for lower-latency applications. It's also helpful to profile the code to reduce allocations (and the need for GC) down to a minimum. I haven't yet gotten down to zero-allocation code in my render loop, but once I got it down below 100 bytes I moved on to other more pressing features. At some point I'll dig deeper to see if I can get rid of the last few allocations. I'd definitely be happy if there are some more folks out there driving demand for lower-latency Julia. :) peace, s On Sun, Sep 14, 2014 at 3:58 PM, Andrew Wagner drew...@gmail.com javascript: wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object files and link against them from C? Then you'd still have to write glue code in a systems language, but at least the implementation of the controller wouldn't have to cross a language boundary... Cheers, Andrew On Thursday, February 20, 2014 10:56:20 PM UTC+1, Uwe Fechner wrote: Hello, I could not find any control system library for Julia yet. Would that make sense? There is a control system library available for Python: http://www.cds.caltech.edu/~murray/wiki/index.php/Python-control Perhaps this could be used as starting point? I think that implementing this in Julia should be easier and faster than in Python. Any comments? Should I open a feature request? Uwe Fechner, TU Delft, The Netherlands
Re: [julia-users] Re: Control system library for Julia?
Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman t...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting, etc. Compile-run-tweak cycles are not a convenient way to do control design (or much of anything, let's be honest). Generating embedded controllers in LLVM IR rather than C is currently very atypical, but eventually I think it could be made to work. A Julia replacement for realtime workshop is a good long-term goal. On Sunday, September 14, 2014 12:58:36 PM UTC-7, Andrew Wagner wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object files and link against them from C? Then you'd still have to write glue code in a systems language, but at least the implementation of the controller wouldn't have to cross a language boundary... Cheers, Andrew On Thursday, February 20, 2014 10:56:20 PM UTC+1, Uwe Fechner wrote: Hello, I could not find any control system library for Julia yet. Would that make sense? There is a control system library available for Python: http://www.cds.caltech.edu/~murray/wiki/index.php/Python-control Perhaps this could be used as starting point? I think that implementing this in Julia should be easier and faster than in Python. Any comments? Should I open a feature request? Uwe Fechner, TU Delft, The Netherlands
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr javascript:: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
Re: [julia-users] Re: Control system library for Julia?
You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net javascript: wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting, etc. Compile-run-tweak cycles are not a convenient way to do control design (or much of anything, let's be honest). Generating embedded controllers in LLVM IR rather than C is currently very atypical, but eventually I think it could be made to work. A Julia replacement for realtime workshop is a good long-term goal. On Sunday, September 14, 2014 12:58:36 PM UTC-7, Andrew Wagner wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object files and link against them from C? Then you'd still have to write glue code in a systems language, but at least the implementation of the controller wouldn't have to cross a language boundary... Cheers, Andrew On Thursday, February 20, 2014 10:56:20 PM UTC+1, Uwe Fechner wrote: Hello, I could not find any control system library for Julia yet. Would that make sense? There is a control system library available for Python: http://www.cds.caltech.edu/~murray/wiki/index.php/Python-control Perhaps this could be used as starting point? I think that implementing this in Julia should be easier and faster than in Python. Any comments? Should I open a feature request? Uwe Fechner, TU Delft, The Netherlands
Re: [julia-users] slow julia version of c code
https://gist.github.com/Zac12345/519bd7a503a1fd1b8d98 has the updated function and code_typed output Pkg.clone(https://github.com/Zac12345/Sparse;) should be fine for the test as the gist does'nt require functions using the shared lib. Are there any specific things i should be looking for in the code_typed results?
Re: [julia-users] Re: Control system library for Julia?
Hi Tony and Uwe! Sorry, I'm more aware of these issues than I indicated in my post. Of course one or two steps of code generation / specialization is necessary to get from human readable to machine executable code. I was actually sitting here reading the thread about the improved garbage collector that is in development. Compared to the stack, RAII, and reference counting, this still sounds like a mess if you want deterministic timings... Thanks for the info about the standalone executables, Tony; I'll go read that too later. If it's just packing up everything into one executable that still has very non-deterministic timings when you run it, it's not a win for control, of course, but maybe the plan is to also do more aggressive specialization of parts of the code (no gc? static binding? stack only?) at the same time; I will go read! Thanks for the anecdote about your experience using Python in the loop Uwe; that's interesting! Cheers, Andrew On Mon, Sep 15, 2014 at 10:45 AM, Tony Kelman t...@kelman.net wrote: You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting, etc. Compile-run-tweak cycles are not a convenient way to do control design (or much of anything, let's be honest). Generating embedded controllers in LLVM IR rather than C is currently very atypical, but eventually I think it could be made to work. A Julia replacement for realtime workshop is a good long-term goal. On Sunday, September 14, 2014 12:58:36 PM UTC-7, Andrew Wagner wrote: Hello again Uwe! It's fun running into someone I know on a language geek forum :) I'm helping one of our bachelor's students implement an LQR controller on our carousel in Freiburg. It's an ugly hack, but I'm calling an octave script to recompute the feedback gains online. Octave wraps slicot, so if the licenses are compatible, perhaps wrapping slicot is the way to go for some functions, if the licenses are compatible. Personally, I have a burning desire for a better language we can actually do control in (rust?). I doubt Julia qualifies due to the garbage collection, but does anyone know if Julia has some sort of way to JIT Julia expressions to code that does ~not have any garbage collection? If so, is there a way to export them as object
[julia-users] Send variable to parallel workers for pmap
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't think
it is smart since A is expand in memery unnessarily. Is there any simple
way to just send a copy of A, or the reference of A, to all parallel
mashines?
Re: [julia-users] Re: Control system library for Julia?
The initial goal is packaging up conventional Julia code with the conventional runtime and GC, so no you don't get determinism from that. Even with explicit memory management you're unlikely to get true hard real-time determinism unless your code is running on a specialized RTOS anyway, and those are very restricted environments where nothing aside from C or a limited subset of C++ is likely to be allowed anyway. Unless you want to write at that low level, you'll need a DSL to design the allocation-free purely computational parts of a controller, and the communication and synchronization algorithms. Galois has done a lot of work on this kind of thing hosted in Haskell, see their Ivory and Tower DSL's. I don't know of anyone doing this kind of work in Julia yet, but it would be cool to see, and in my non-expert opinion I think it should be feasible. On Monday, September 15, 2014 2:26:41 AM UTC-7, Andrew Wagner wrote: Hi Tony and Uwe! Sorry, I'm more aware of these issues than I indicated in my post. Of course one or two steps of code generation / specialization is necessary to get from human readable to machine executable code. I was actually sitting here reading the thread about the improved garbage collector that is in development. Compared to the stack, RAII, and reference counting, this still sounds like a mess if you want deterministic timings... Thanks for the info about the standalone executables, Tony; I'll go read that too later. If it's just packing up everything into one executable that still has very non-deterministic timings when you run it, it's not a win for control, of course, but maybe the plan is to also do more aggressive specialization of parts of the code (no gc? static binding? stack only?) at the same time; I will go read! Thanks for the anecdote about your experience using Python in the loop Uwe; that's interesting! Cheers, Andrew On Mon, Sep 15, 2014 at 10:45 AM, Tony Kelman to...@kelman.net javascript: wrote: You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting, etc. Compile-run-tweak cycles are not a convenient way to do control design (or much of anything, let's be honest). Generating
Re: [julia-users] Re: ANN: ApproxFun v0.0.3 with general linear PDE solving on rectangles
Here’s a partial list of features in Chebfun not in ApproxFun: 1)Automatic edge detection and domain splitting The automatic splitting capability of chebfun is definitely really cool, but it always seemed to me to be a bit more then one would need for most use cases. That is, if I am defining some function like f = Fun(g::Function,[-1,1]) where g is composed of things like absolute values and step functions I might need to do something sophisticated to figure out how to break up the domain, but if I instead pass something like f = Fun(g::PiecewiseFunction,[-1,1]) which has some g that has been annotated by the user in some obvious way (or semiautomatically, given some basic rules for composing PiecewiseFunction types under standard operations) I might have a much easier time. In practice, when setting up problems in the first place one is often paying attention to where discontinuities are anyway, so providing such a mechanism might even be a natural way to help someone set up their problem. Maybe this kind of thing is incompatible with ApproxFun (sorry, I didn't look in detail yet). But at any rate, super cool work! If there are any plans to start a gallery of examples ala chebfun I would be happy to contribute some from population dynamics. On Friday, September 12, 2014 1:43:27 AM UTC+2, Sheehan Olver wrote: Chebfun is a lot more full featured, and ApproxFun is _very_ rough around the edges. ApproxFun will probably end up a very different animal than chebfun: right now the goal is to tackle PDEs on a broader class of domains, something I think is beyond the scope of Chebfun due to issues with Matlab's speed, memory management, etc. Here’s a partial list of features in Chebfun not in ApproxFun: 1)Automatic edge detection and domain splitting 2)Support for delta functions 3)Built-in time stepping (pde15s) 4)Eigenvalue problems 5)Automatic nonlinear ODE solver 6)Operator exponential 7)Smarter constructor for determining convergence 8)Automatic differentiation I have no concrete plans at the moment of adding these features, though eigenvalue problems and operator exponentials will likely find their way in at some point. Sheehan On 12 Sep 2014, at 12:14 am, Steven G. Johnson steve...@gmail.com javascript: wrote: This is great! At this point, what are the major differences in functionality between ApproxFun and Chebfun?
Re: [julia-users] Re: Control system library for Julia?
Ivory looks interesting. But what does DSL mean (in your post)? On Monday, September 15, 2014 11:55:03 AM UTC+2, Tony Kelman wrote: The initial goal is packaging up conventional Julia code with the conventional runtime and GC, so no you don't get determinism from that. Even with explicit memory management you're unlikely to get true hard real-time determinism unless your code is running on a specialized RTOS anyway, and those are very restricted environments where nothing aside from C or a limited subset of C++ is likely to be allowed anyway. Unless you want to write at that low level, you'll need a DSL to design the allocation-free purely computational parts of a controller, and the communication and synchronization algorithms. Galois has done a lot of work on this kind of thing hosted in Haskell, see their Ivory and Tower DSL's. I don't know of anyone doing this kind of work in Julia yet, but it would be cool to see, and in my non-expert opinion I think it should be feasible. On Monday, September 15, 2014 2:26:41 AM UTC-7, Andrew Wagner wrote: Hi Tony and Uwe! Sorry, I'm more aware of these issues than I indicated in my post. Of course one or two steps of code generation / specialization is necessary to get from human readable to machine executable code. I was actually sitting here reading the thread about the improved garbage collector that is in development. Compared to the stack, RAII, and reference counting, this still sounds like a mess if you want deterministic timings... Thanks for the info about the standalone executables, Tony; I'll go read that too later. If it's just packing up everything into one executable that still has very non-deterministic timings when you run it, it's not a win for control, of course, but maybe the plan is to also do more aggressive specialization of parts of the code (no gc? static binding? stack only?) at the same time; I will go read! Thanks for the anecdote about your experience using Python in the loop Uwe; that's interesting! Cheers, Andrew On Mon, Sep 15, 2014 at 10:45 AM, Tony Kelman to...@kelman.net wrote: You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded controllers, hard or soft real time. Array views and the like will make this easier, but I don't see any reason you couldn't refactor Julia code to do the vast majority of allocation up-front and end up generating more or less the same LLVM IR that Rust's compiler would. And I doubt Rust will ever be as good of an interactive technical computing environment as Julia in terms of REPL, plotting, simulation, experimenting,
Re: [julia-users] Re: Control system library for Julia?
Another quick note, Tony: There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. At least the auditing for restrictions part of that problem ~seems like it should be easy in a well designed language. I want to just declare an environment and have the compiler throw errors if anything inside it violates the restrictions I care about for whatever my personal definition of what a realtime or typesafe or pure functional language subset is. Something along the lines of: #[deny(managed_heap_memory)]: https://github.com/rust-lang/rust/issues/9984 Of course as you point out there is the other part of the problem achieving code reuse and complexity, which is much harder.
Re: [julia-users] Re: Control system library for Julia?
RE Ivory and Tower, and similar efforts in Haskell, I am aware of them and I do find them really interesting. Since Rust is based on the same type theory as Haskell, I'm hoping similar things will be possible in Rust, without having an additional layer of code generation. I'm still trying to work out how much of the special sauce in Haskell is in the type system, and how much is in the functional purity. On Mon, Sep 15, 2014 at 12:13 PM, Uwe Fechner uwe.fechner@gmail.com wrote: Ivory looks interesting. But what does DSL mean (in your post)? On Monday, September 15, 2014 11:55:03 AM UTC+2, Tony Kelman wrote: The initial goal is packaging up conventional Julia code with the conventional runtime and GC, so no you don't get determinism from that. Even with explicit memory management you're unlikely to get true hard real-time determinism unless your code is running on a specialized RTOS anyway, and those are very restricted environments where nothing aside from C or a limited subset of C++ is likely to be allowed anyway. Unless you want to write at that low level, you'll need a DSL to design the allocation-free purely computational parts of a controller, and the communication and synchronization algorithms. Galois has done a lot of work on this kind of thing hosted in Haskell, see their Ivory and Tower DSL's. I don't know of anyone doing this kind of work in Julia yet, but it would be cool to see, and in my non-expert opinion I think it should be feasible. On Monday, September 15, 2014 2:26:41 AM UTC-7, Andrew Wagner wrote: Hi Tony and Uwe! Sorry, I'm more aware of these issues than I indicated in my post. Of course one or two steps of code generation / specialization is necessary to get from human readable to machine executable code. I was actually sitting here reading the thread about the improved garbage collector that is in development. Compared to the stack, RAII, and reference counting, this still sounds like a mess if you want deterministic timings... Thanks for the info about the standalone executables, Tony; I'll go read that too later. If it's just packing up everything into one executable that still has very non-deterministic timings when you run it, it's not a win for control, of course, but maybe the plan is to also do more aggressive specialization of parts of the code (no gc? static binding? stack only?) at the same time; I will go read! Thanks for the anecdote about your experience using Python in the loop Uwe; that's interesting! Cheers, Andrew On Mon, Sep 15, 2014 at 10:45 AM, Tony Kelman to...@kelman.net wrote: You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine platform for code generation of embedded
Re: [julia-users] Re: Control system library for Julia?
(E)DSL is a buzzword functional programming people use that really just means API. On Mon, Sep 15, 2014 at 12:16 PM, Tony Kelman t...@kelman.net wrote: Domain specific language. Sometimes with an E in front for embedded domain specific language, meaning a DSL embedded within a different general-purpose language as opposed to a standalone DSL that only does one thing. (not the type of embedded that the rest of this conversation has been about) On Monday, September 15, 2014 3:13:42 AM UTC-7, Uwe Fechner wrote: Ivory looks interesting. But what does DSL mean (in your post)? On Monday, September 15, 2014 11:55:03 AM UTC+2, Tony Kelman wrote: The initial goal is packaging up conventional Julia code with the conventional runtime and GC, so no you don't get determinism from that. Even with explicit memory management you're unlikely to get true hard real-time determinism unless your code is running on a specialized RTOS anyway, and those are very restricted environments where nothing aside from C or a limited subset of C++ is likely to be allowed anyway. Unless you want to write at that low level, you'll need a DSL to design the allocation-free purely computational parts of a controller, and the communication and synchronization algorithms. Galois has done a lot of work on this kind of thing hosted in Haskell, see their Ivory and Tower DSL's. I don't know of anyone doing this kind of work in Julia yet, but it would be cool to see, and in my non-expert opinion I think it should be feasible. On Monday, September 15, 2014 2:26:41 AM UTC-7, Andrew Wagner wrote: Hi Tony and Uwe! Sorry, I'm more aware of these issues than I indicated in my post. Of course one or two steps of code generation / specialization is necessary to get from human readable to machine executable code. I was actually sitting here reading the thread about the improved garbage collector that is in development. Compared to the stack, RAII, and reference counting, this still sounds like a mess if you want deterministic timings... Thanks for the info about the standalone executables, Tony; I'll go read that too later. If it's just packing up everything into one executable that still has very non-deterministic timings when you run it, it's not a win for control, of course, but maybe the plan is to also do more aggressive specialization of parts of the code (no gc? static binding? stack only?) at the same time; I will go read! Thanks for the anecdote about your experience using Python in the loop Uwe; that's interesting! Cheers, Andrew On Mon, Sep 15, 2014 at 10:45 AM, Tony Kelman to...@kelman.net wrote: You don't write in assembly, do you? Every stack is implicitly doing multiple languages and some form of code generation in the process of compilation, you just don't have to know whether it's happening. Systems programming as intended by Rust and as a software engineer would mean the term is a rather different thing than what you want for the purposes of embedded real-time control. There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. I'd say reducing the number of mandatory dependencies and making it possible to statically compile Julia code into minimal standalone executables and libraries is absolutely an explicit goal of the main Julia developers, just check the 0.4-projects milestone on Github. It's not there yet, but progress is being made. On Monday, September 15, 2014 1:27:15 AM UTC-7, Andrew Wagner wrote: Hi Tony! I'm a bit burned out on software systems that are pieced together out of multiple languages, doing code generation, etc... If there's not a single language that can do everything I need for the project, I would not start it yet. It's really hard for me to tell at this point whether it is more likely in the future for Julia to become suitable for systems programming, or for rust to become suitable for interactive use... Those are explicitly non-goals for the main developers of the two languages, so it's hard to predict. Cheers, Andrew On Mon, Sep 15, 2014 at 8:27 AM, Tony Kelman to...@kelman.net wrote: Slicot's not ideal from a licensing standpoint, see the earlier discussion from February in this thread. There is a GPL-licensed version of Slicot available from Debian, which works for the sake of a package. Long term it makes more sense to reimplement the important parts of Slicot that wrap Lapack to provide Sylvester and Lyapunov and Riccati solvers for discrete and continuous time, in base Julia. I think Jim Crist spent the summer doing a Python GSoC, so progress on control packages for Julia has been mostly stalled. Eventually I think Julia could make a fine
Re: [julia-users] Re: ANN: ApproxFun v0.0.3 with general linear PDE solving on rectangles
I’m in the process of retooling ApproxFun to support general “FunctionSpace”s, so that one needs to only override a few routines (points,transform,itransform,diff,…) to get most the features of ApproxFun for free for othert function spaces (e.g., JacobiSpace for Jacobi polynomials, CosSpace for cosine expansion, etc.). A PiecewiseSpace would be really easy to implement in this framework. There are some examples in ApproxFun/examples/ but no where near as extensive as chebfun. A gallery sounds like a good idea, though maybe once the basic framework settles down a bit. On 15 Sep 2014, at 8:11 pm, Gabriel Mitchell gajomi271...@gmail.com wrote: Here’s a partial list of features in Chebfun not in ApproxFun: 1)Automatic edge detection and domain splitting The automatic splitting capability of chebfun is definitely really cool, but it always seemed to me to be a bit more then one would need for most use cases. That is, if I am defining some function like f = Fun(g::Function,[-1,1]) where g is composed of things like absolute values and step functions I might need to do something sophisticated to figure out how to break up the domain, but if I instead pass something like f = Fun(g::PiecewiseFunction,[-1,1]) which has some g that has been annotated by the user in some obvious way (or semiautomatically, given some basic rules for composing PiecewiseFunction types under standard operations) I might have a much easier time. In practice, when setting up problems in the first place one is often paying attention to where discontinuities are anyway, so providing such a mechanism might even be a natural way to help someone set up their problem. Maybe this kind of thing is incompatible with ApproxFun (sorry, I didn't look in detail yet). But at any rate, super cool work! If there are any plans to start a gallery of examples ala chebfun I would be happy to contribute some from population dynamics. On Friday, September 12, 2014 1:43:27 AM UTC+2, Sheehan Olver wrote: Chebfun is a lot more full featured, and ApproxFun is _very_ rough around the edges. ApproxFun will probably end up a very different animal than chebfun: right now the goal is to tackle PDEs on a broader class of domains, something I think is beyond the scope of Chebfun due to issues with Matlab's speed, memory management, etc. Here’s a partial list of features in Chebfun not in ApproxFun: 1)Automatic edge detection and domain splitting 2)Support for delta functions 3)Built-in time stepping (pde15s) 4)Eigenvalue problems 5)Automatic nonlinear ODE solver 6)Operator exponential 7)Smarter constructor for determining convergence 8)Automatic differentiation I have no concrete plans at the moment of adding these features, though eigenvalue problems and operator exponentials will likely find their way in at some point. Sheehan On 12 Sep 2014, at 12:14 am, Steven G. Johnson steve...@gmail.com wrote: This is great! At this point, what are the major differences in functionality between ApproxFun and Chebfun?
Re: [julia-users] self dot product of all columns in a matrix: Julia vs. Octave
Honestly this sounds like a horrible workaround for a DSFMT library design problem. If this is going to be scoped inside a DSFMT module then fine, it's a low-level API that is tied to a particular library and we have plenty of those. But as part of our high-level RNG API, yuck. On Mon, Sep 15, 2014 at 6:50 AM, Viral Shah vi...@mayin.org wrote: MersenneTwisterArray sounds like a good idea, separating it clearly from the scalar version. Even the method names could be different to avoid any confusion. -viral On 15-Sep-2014, at 6:25 am, Andreas Noack andreasnoackjen...@gmail.com wrote: The problem is that we cannot mix the calls to the scalar and array generators. Maybe a solution could be to define a new MersenneTwisterArray type that only has methods defined for arrays. Med venlig hilsen Andreas Noack 2014-09-14 10:21 GMT-04:00 gael.mc...@gmail.com: I wonder if we should provide access to DSFMT's random array generation, so that one can use an array generator. The requirements are that one has to generate at least 384 random numbers at a time or more, and the size of the array must necessarily be even. We should not allow this with the global seed, and it can be through a randarray!() function. We can even avoid exporting this function by default, since there are lots of conditions it needs, but it gives really high performance. Are the conditions needed limited to n384 and n even? Why not providing it by default then with a single if statement to check for the n384 condition? The n even condition is not really a problem as Julia does not allocate the exact amount of data needed. Even for fixed-size array, adding 1 extra element (not user accessible) does not seem to be much of a drawback. -viral
Re: [julia-users] self dot product of all columns in a matrix: Julia vs. Octave
Yes it is going to not be exported of we do it. -viral On 15 Sep 2014 16:29, Stefan Karpinski ste...@karpinski.org wrote: Honestly this sounds like a horrible workaround for a DSFMT library design problem. If this is going to be scoped inside a DSFMT module then fine, it's a low-level API that is tied to a particular library and we have plenty of those. But as part of our high-level RNG API, yuck. On Mon, Sep 15, 2014 at 6:50 AM, Viral Shah vi...@mayin.org wrote: MersenneTwisterArray sounds like a good idea, separating it clearly from the scalar version. Even the method names could be different to avoid any confusion. -viral On 15-Sep-2014, at 6:25 am, Andreas Noack andreasnoackjen...@gmail.com wrote: The problem is that we cannot mix the calls to the scalar and array generators. Maybe a solution could be to define a new MersenneTwisterArray type that only has methods defined for arrays. Med venlig hilsen Andreas Noack 2014-09-14 10:21 GMT-04:00 gael.mc...@gmail.com: I wonder if we should provide access to DSFMT's random array generation, so that one can use an array generator. The requirements are that one has to generate at least 384 random numbers at a time or more, and the size of the array must necessarily be even. We should not allow this with the global seed, and it can be through a randarray!() function. We can even avoid exporting this function by default, since there are lots of conditions it needs, but it gives really high performance. Are the conditions needed limited to n384 and n even? Why not providing it by default then with a single if statement to check for the n384 condition? The n even condition is not really a problem as Julia does not allocate the exact amount of data needed. Even for fixed-size array, adding 1 extra element (not user accessible) does not seem to be much of a drawback. -viral
Re: [julia-users] Help needed with creating Julia package
On 15 September 2014 07:38, Tony Kelman t...@kelman.net wrote: I've compiled many many dozens of libraries using MinGW-w64 on MSYS, and cross-compiling from Cygwin and various Linux distributions, and not encountered nearly as many problems as you seem to be. Pretending to be autotools but hacking it up yourself is the worst possible thing you could do - you're completely nonstandard and nonfunctional in cross-compilation for example, and you have as many or more dependencies - requiring bash for example is often not acceptable on FreeBSD and similar systems. I take it you are referring to flint rather than MPIR here. I was referring to hacking the MPIR autotools, which has been necessary long before MPIR even existed. It was inherited from GMP and certainly necessary for a multitude of reasons: making fat binaries, distinguishing microprocessor microarchitectures, linking C code with assembly code cross platform, compiling on Windows and OSX, excluding compilers which compile everything else just fine but miscompile bignum libraries, tuning, etc, etc, etc. Autotools has always been unsuitable for such things at some level. Part of the problem is the way MSYS2 identifies itself changes depending on what environment variables you have set (or which bat file you start it with). The MSYS uname is unrecognized by autotools because MSYS2 is new, and you almost never want to actually be compiling with the MSYS host, since that causes your application to link to the msys-2.0.dll posix compatibility layer. I will have to look into that. It doesn't sound like something we want. Though I don't really understand why changing the host should change the way gcc compiles code, unless it does so for cygwin itself. It's like using the standard gcc in Cygwin, any users would need Cygwin and the associated GPL posix runtime library to use the compiled code. The MINGW64 uname is also new and unrecognized, and totally nonstandard. This is why you should be querying the COMPILER for information about what system the generated code is supposed to run on, NOT the build environment. It was my understanding that the Mingw compiler could be used from Cygwin. Perhaps I am confused about it, but it seems like interrogating the compiler may not give the information we need. The accepted GNU triples for MinGW are: i686-pc-mingw32: MinGW.org 32 bit compiler, outdated and obsolete, don't use this i686-w64-mingw32: 32 bit compiler from MinGW-w64 project, use this for 32 bit host x86_64-w64-mingw32: 64 bit compiler from MinGW-w64 project, use this for 64 bit host Yes the names are confusing and make no sense, but this is the GNU standard that every other conforming project in the world is using. LLVM decided to go out on a limb and rearrange some of these names recently, adding to the confusion, but GCC, autotools, and MinGW are unlikely to follow their lead. We can't use the accepted gnu triples in MPIR and that has always been the case for GMP too. They don't give nearly enough information for our needs. As far as MPIR is concerned the first part of the triple has to be the microarchitecture of the CPU for assembly language purposes. Then we have information about the platform/OS. If we reserve that for microarchitecture, clearly we can't use the triples given above. Canonicalising triples in MPIR is a two step process. Firstly we call the standard FSF config.guess, plus any hacks necessary for systems that are known to be incorrectly identified by the standard config.guess. We update the standard config.guess every now and again. We then have our own config.guess, which uses a completely different convention which gives us the information we require. This has also always been the case in GMP. We also use a cpuid utility in conjunction with config.guess to give us information about the microarchitecture. Personally I'd like to get rid of autotools in MPIR altogether. It is much more effort to maintain than it is worth, with thousands of lines of hacks. And our main autotools maintainer died in 2012. I know much less about it. On Sunday, September 14, 2014 1:21:42 PM UTC-7, Bill Hart wrote: That was the problem. I've added hack number 40001 to the MPIR autotools to fix this issue. Nemo now passes its tests on Window 64 for the first time ever. Bill. On Sunday, 14 September 2014 21:40:45 UTC+2, Bill Hart wrote: I traced through the precise calls to libflint and libgmp that make it segfault and wrote a few test programs from the msys2 command line, taking Julia right out of the loop. At first I thought that it was only segfaulting when gmp was called from flint. But in fact I can make it segfault calling gmp directly from the program, though only in slightly strange circumstances. If I take the integer a = 123456789, I can compute the square and cube of this without problems. But if I try to compute the fourth power it segfaults. If I call gmp_printf it
Re: [julia-users] Help needed with creating Julia package
I will have to look into that. It doesn't sound like something we want. Though I don't really understand why changing the host should change the way gcc compiles code, unless it does so for cygwin itself. Changing the host changes which compiler gets used to create the binaries. In Cygwin or Linux, you can call the MinGW compilers by providing --host=x86_64-w64-mingw32 or similar, if you're using conventional autotools. It's also possible in Cmake but it's about 5 lines worth of flags (or a toolchain file with the same content) - it works, eventually, but it's really picky. It was my understanding that the Mingw compiler could be used from Cygwin. Perhaps I am confused about it, but it seems like interrogating the compiler may not give the information we need. Right, but when you configure with --host=x86_64-w64-mingw32 in Cygwin or Linux, you're using a cross-compiler that is prefixed by the host triple: x86_64-w64-mingw32-gcc and similar for binutils. So you can't just ask gcc for information - in a lot of ways MSYS(2) is a hack to avoid worrying about MinGW cross-compilation, and it leads to some bad ways of doing things. I think it's better to do the work to make a library possible to cross-compile, than to give up and say MSYS(2) is the only method of compiling it for Windows. I've been cross-compiling Julia itself and all of its dependencies from both Linux and Cygwin for months, I very rarely touch MSYS(2) any more. We can't use the accepted gnu triples in MPIR and that has always been the case for GMP too. They don't give nearly enough information for our needs. Of course they don't provide a complete story when you're worrying about assembly and various SIMD instruction sets. But that's not the GNU triple's job. That information should be determined and represented separately, you're not going to include this kind of information in the host prefix of the name of a cross-compiler executable for example. GMP is also a pretty bad actor in terms of its autotools usage - making a symlink to libgmp.dll called libgmp.lib in lib/ is completely the wrong thing to do for MinGW. Not your fault I know, but sounds like you may have inherited some autotools abuses from GMP.
Re: [julia-users] Help needed with creating Julia package
On 15 September 2014 13:39, Tony Kelman t...@kelman.net wrote: I will have to look into that. It doesn't sound like something we want. Though I don't really understand why changing the host should change the way gcc compiles code, unless it does so for cygwin itself. Changing the host changes which compiler gets used to create the binaries. In Cygwin or Linux, you can call the MinGW compilers by providing --host=x86_64-w64-mingw32 or similar, if you're using conventional autotools. It's also possible in Cmake but it's about 5 lines worth of flags (or a toolchain file with the same content) - it works, eventually, but it's really picky. Sorry, I think I misunderstood what you were referring to when you said host. So my response doesn't make any sense in the way that you meant it. I understand what you are getting at now. It was my understanding that the Mingw compiler could be used from Cygwin. Perhaps I am confused about it, but it seems like interrogating the compiler may not give the information we need. Right, but when you configure with --host=x86_64-w64-mingw32 in Cygwin or Linux, you're using a cross-compiler that is prefixed by the host triple: x86_64-w64-mingw32-gcc and similar for binutils. So you can't just ask gcc for information - in a lot of ways MSYS(2) is a hack to avoid worrying about MinGW cross-compilation, and it leads to some bad ways of doing things. I think it's better to do the work to make a library possible to cross-compile, than to give up and say MSYS(2) is the only method of compiling it for Windows. I've been cross-compiling Julia itself and all of its dependencies from both Linux and Cygwin for months, I very rarely touch MSYS(2) any more. I think this is very common. But it means that people who really want their code to actually compile on Windows frequently have an inferior system with which to do it, because all the real hackers are cross compiling from Linux. There's nothing wrong in that, it just leads to additional frustration for some people. We can't use the accepted gnu triples in MPIR and that has always been the case for GMP too. They don't give nearly enough information for our needs. Of course they don't provide a complete story when you're worrying about assembly and various SIMD instruction sets. But that's not the GNU triple's job. That information should be determined and represented separately, you're not going to include this kind of information in the host prefix of the name of a cross-compiler executable for example. Sure, but just to emphasise, we use the standard gnu triples in the first level of config.guess (except when it's broken). The hack I applied was to the second level, which uses the MPIR triples convention. GMP is also a pretty bad actor in terms of its autotools usage - making a symlink to libgmp.dll called libgmp.lib in lib/ is completely the wrong thing to do for MinGW. As I understand it, few of their core developers even have access to Windows boxes. They also reportedly don't support stack unwinding on Windows (I've never actually checked this) or provide Visual Studio project files. Their assembly code is just modified with lead-in and lead-out code so they can use the same assembly code for Windows and Linux, despite the differences in ABI. (It works relatively well, but it's a hack -- one that we use quite a bit too, I confess). One enormous frustration for us is that MSVC has no inline assembler on Windows 64, and gas format assembly has not traditionally been available in the assemblers available for Windows. There are many, many problems that we work around that require many hacks, even on the linux side. We could fix them all and make it less hackish, but you are talking about something like a year or two of full time work, just to fix all the build system issues alone! Not your fault I know, but sounds like you may have inherited some autotools abuses from GMP. Many of them are inherited from there. But remember that GMP has been in development since at least 1994! They did things back then that many other projects have not even thought about doing, to this day. MPIR and GMP are *incredibly* complex projects and few people recognise the subtleties which simply don't arise for other types of software. Bill.
Re: [julia-users] Re: Control system library for Julia?
I didn't really mean by way of compiler flags or language pragmas. I can try -fno-rtti or -fno-exceptions with g++ today, it doesn't do me much good to spend a week getting half a dozen dependency libraries built with the obscure proprietary embedded board compiler only to find the very last thing I need won't work. The type of control I do is pretty sophisticated - optimization-based, nonlinear, large-scale MIMO systems, incorporating a prediction model of the system and satisfying state and input constraints - takes a bunch of libraries to do properly, not the kind of thing that's easy to throw together from scratch in C. It would be nice to get guarantees by construction that my controller and all the code it uses would be deployable to an embedded environment from the very beginning. Luckily microcontrollers are getting much more powerful these days and the line is starting to blur between embedded RTOS and lightly tweaked Linux. You can fit lots more code and do more interesting things with the same power/thermal footprint, getting lots more flops and IO in soft-but-pretty-damn-good realtime at the same or better speeds. Though what this thread was originally about was tools for doing offline control design, where Matlab/Simulink absolutely dominate. You don't want to do simulation, analysis, testing, and tuning in the exact same low-level restricted language that the realtime controller gets deployed to, the requirements are completely different. On Monday, September 15, 2014 3:13:32 AM UTC-7, Andrew Wagner wrote: Another quick note, Tony: There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. At least the auditing for restrictions part of that problem ~seems like it should be easy in a well designed language. I want to just declare an environment and have the compiler throw errors if anything inside it violates the restrictions I care about for whatever my personal definition of what a realtime or typesafe or pure functional language subset is. Something along the lines of: #[deny(managed_heap_memory)]: https://github.com/rust-lang/rust/issues/9984 Of course as you point out there is the other part of the problem achieving code reuse and complexity, which is much harder.
[julia-users] Module Includes Problem
I created a module to load and parse a set collection of data files. It
worked in Julia 0.2. Here is the module with hopefully only the
non-relevant code removed.
module Aerodyne
using DataFrames
using Dates
require(dirlist.jl)
export Status, aerodyne, aerodyne_load
type Status
# Constructor for Status type
function Status{T:Int64}(vals::DataFrames.DataArray{T,1})
Status(vector(vals))
end # End Status{T:Int64}(vals::DataFrames.DataArray{T,1}) constructor
# Constructor for Status type
function Status(vals::Array{Int64})
end # End of constructor
end # End of type
function aerodyne_load(Dr::String,mindate::DateTime)
end # End of aerodyne_load(Dr::String,mindate::DateTime)
function aerodyne_load(Dr::String,mindate::DateTime,maxdate::DateTime)
## Load STR and STC files in the given directory (Dr) between the given
dates ##
# Check for Directory
if isdir(Dr) == false
error(First input should be a directory)
end
# List Files
(Fstr,folders) = dirlist(Dr,regex=r\.str$) # List STR files # Line 140
(Fstc,folders) = dirlist(Dr,regex=r\.stc$) # List STC files
end # End of aerodyne_load(Dr::String,mindate::DateTime,maxdate::DateTime)
function aerodyne_load{T:String}(F::Array{T,1})
end # End aerodyne_load(F::Array{String,1})
function parse_time{T:String}(F::Array{T,1})
end # End of parse_time(F::Array{String,1})
function aerodyne_load(F::String)
end # End of aerodyne_load(F::String)
function aerodyne()
end # End of aerodyne()
end # End of module
The error that is returned is as follows.
ERROR: dirlist not defined
while loading O:\Code\Julia\qcl_plot_basic.jl, in expression starting on
line 34
while loading In[4], in expression starting on line 1
in aerodyne_load at O:\Code\Julia\Aerodyne.jl:140
Line 140 simply calls dirlist which is suppose to recursively go through a
given directory and return files matching the input regular expression.
dirlist.jl and the directory where all of my code resides are loaded into
Julia every time from the .juliarc.jl file.
(Fstr,folders) = dirlist(Dr,regex=r\.str$) # List STR files, line 140
Any ideas?
Re: [julia-users] Send variable to parallel workers for pmap
You need to store A in such a way that it is guaranteed to continue to
exist. If there is no other reference to it, it may e.g. be garbage
collected. One way would be to store it in a global variable, or in a
dictionary. Try e.g.:
@everywhere (global A=rand())
pmap(a-(global A;A)+x,1:3)
Obviously, in real code you would wrap accessing A in a function
instead of writing (global A;A) all the time.
-erik
On Mon, Sep 15, 2014 at 5:52 AM, xiongji...@gmail.com wrote:
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't think
it is smart since A is expand in memery unnessarily. Is there any simple way
to just send a copy of A, or the reference of A, to all parallel mashines?
--
Erik Schnetter schnet...@cct.lsu.edu
http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] Help needed with creating Julia package
I think this is very common. But it means that people who really want their code to actually compile on Windows frequently have an inferior system with which to do it, because all the real hackers are cross compiling from Linux. Or Cygwin. Part of the point is the exact same process should result in the same built binaries for cross-compilation from Linux or cross-compilation from Cygwin. Any complaint about Cygwin in terms of size or performance applies equally to MSYS2, and MSYS2's newness and multitude of build/run modes causes more trouble than it's worth in many cases. or provide Visual Studio project files Can you blame them? Speaking of awful, impenetrable build systems that don't translate well to other platforms... MPIR and GMP are *incredibly* complex projects and few people recognise the subtleties which simply don't arise for other types of software. I'm not familiar with writing this type of software, but plenty familiar with building it, and I really don't see a whole lot that goes into a bignum library that doesn't also happen in, say, an optimized BLAS/LAPACK implementation. Of course those are also notorious for having messy and fragile build systems.
[julia-users] Re: Send variable to parallel workers for pmap
Chris Strickland
https://groups.google.com/forum/#!msg/julia-users/jlKoEtErRL4/0ZcB_hxyJlYJ
lists one approach for the general problem
you could precreate A and then define it everywhere, or send a copy over as
a parameter to whatever function you use in pmap, similar but not the same
as above
Another approach which was useful to me was the @parallel for loops
Any variables used inside the parallel loop will be copied and broadcast
to each process.
http://julia.readthedocs.org/en/latest/manual/parallel-computing/
and shared arrays
http://julia.readthedocs.org/en/latest/manual/parallel-computing/#shared-arrays-experimental-unix-only-feature
if using linux could be useful (I've not tried them).
On Monday, September 15, 2014 10:52:33 AM UTC+1, xiong...@gmail.com wrote:
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't think
it is smart since A is expand in memery unnessarily. Is there any simple
way to just send a copy of A, or the reference of A, to all parallel
mashines?
Re: [julia-users] Re: Control system library for Julia?
On Monday, September 15, 2014 3:25:07 AM UTC-5, Uwe Fechner wrote: With Python you can reach a jitter of less than 2 ms in the 20Hz control loop quite easily (on low-latency Linux). In my case this is sufficient for prototyping, but the real flight control system should run at a higher update frequency (perhaps 200 Hz). Semi-off-topic: 200 Hz seems really fast to me. What kind of high-frequency dynamics do these things have? I'm not familiar with this particular design space. Any good (open-access) papers? Patrick
[julia-users] Path separator on Windows
Hello all, I recently encountered an issue https://github.com/JuliaLang/Tk.jl/issues/62#issuecomment-55392934 with path separators on Windows and was wondering why backslashes were chosen over slashes in Julia Base. Windows usually interprets / just as well as \, especially if the paths are quoted. Something like shell pushd C:/users works fine, although shell pushd C:/users does not, which might prove problematic (but then again, paths for cmd.exe should be quoted anyways because of the bad escaping of ). One might consider changing the separator to a slash which might reduce compatibility issues as well as (hopefully) make most of the special @windows_only cases in base/path.jl obsolete. As a bonus, the regexps wouldn't look all that crazy anymore :) So, my question is: Are there problems with that approach? And if I wanted to try to implement that, how would I go about it? As far as I can see, Julia needs to be recompiled for changes in base to take effect, correct? Oh, btw, thanks for this great language. It's really nice to see Julia come along and get better and better by every release! :)
[julia-users] Random numbers and the Poisson distribution
Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those event rates. Is it possible to do the same in Julia? (I can write rand(6) and get a vector of uniformly distributed random numbers, but I need the for loop to do the same for other distributions). In Octave you can pretty much write something like X += dX * poissrnd(rates * dt); all in one line (where X is a vector of populations, and dX is the event rate / state change matrix), and it would be nice to be as elegant as that in Julia. Thank you very much in advance. Regards, Jamie
[julia-users] Re: Send variable to parallel workers for pmap
you could precreate A and then define it everywhere
Could you please show me some code how to do it?
On Monday, September 15, 2014 2:51:08 PM UTC+2, John Drummond wrote:
Chris Strickland
https://groups.google.com/forum/#!msg/julia-users/jlKoEtErRL4/0ZcB_hxyJlYJ
lists one approach for the general problem
you could precreate A and then define it everywhere, or send a copy over
as a parameter to whatever function you use in pmap, similar but not the
same as above
Another approach which was useful to me was the @parallel for loops
Any variables used inside the parallel loop will be copied and broadcast
to each process.
http://julia.readthedocs.org/en/latest/manual/parallel-computing/
and shared arrays
http://julia.readthedocs.org/en/latest/manual/parallel-computing/#shared-arrays-experimental-unix-only-feature
if using linux could be useful (I've not tried them).
On Monday, September 15, 2014 10:52:33 AM UTC+1, xiong...@gmail.com wrote:
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't
think it is smart since A is expand in memery unnessarily. Is there any
simple way to just send a copy of A, or the reference of A, to all parallel
mashines?
Re: [julia-users] Random numbers and the Poisson distribution
I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem space...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those event rates. Is it possible to do the same in Julia? (I can write rand(6) and get a vector of uniformly distributed random numbers, but I need the for loop to do the same for other distributions). In Octave you can pretty much write something like X += dX * poissrnd(rates * dt); all in one line (where X is a vector of populations, and dX is the event rate / state change matrix), and it would be nice to be as elegant as that in Julia. Thank you very much in advance. Regards, Jamie
[julia-users] ANN: MeCab.jl - Julia binding of most popular Japanese morphological analyzer
Hi, all. I've released first version of MeCab.jl, Julia binding of MeCab. MeCab is the most popular Japanese morphological analyzer. https://github.com/chezou/MeCab.jl Enjoy! -- Michiaki
[julia-users] How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I have
the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second argument,
since the second argument is just allocating space for an array. I don't
want to return the Cint for success/fail, but rather the array that splits
the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here to
make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] Re: Send variable to parallel workers for pmap
with parallel
addprocs(2)
println(Number of processors, nprocs())
@everywhere function foo(x,y)
println(myid(), , x, , y)
return x + y
end
let a = rand()
@sync @parallel for x in 1:3
foo(a,x)
end
end
On Monday, September 15, 2014 3:14:50 PM UTC+1, xiong...@gmail.com wrote:
you could precreate A and then define it everywhere
Could you please show me some code how to do it?
On Monday, September 15, 2014 2:51:08 PM UTC+2, John Drummond wrote:
Chris Strickland
https://groups.google.com/forum/#!msg/julia-users/jlKoEtErRL4/0ZcB_hxyJlYJ
lists one approach for the general problem
you could precreate A and then define it everywhere, or send a copy over
as a parameter to whatever function you use in pmap, similar but not the
same as above
Another approach which was useful to me was the @parallel for loops
Any variables used inside the parallel loop will be copied and
broadcast to each process.
http://julia.readthedocs.org/en/latest/manual/parallel-computing/
and shared arrays
http://julia.readthedocs.org/en/latest/manual/parallel-computing/#shared-arrays-experimental-unix-only-feature
if using linux could be useful (I've not tried them).
On Monday, September 15, 2014 10:52:33 AM UTC+1, xiong...@gmail.com
wrote:
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't
think it is smart since A is expand in memery unnessarily. Is there any
simple way to just send a copy of A, or the reference of A, to all parallel
mashines?
Re: [julia-users] Re: Control system library for Julia?
more informations on kite control you can find in the following papers: Fechner, U., Schmehl, R. (2012). Design of a Distributed Kite Power Control System. In Proceedings of the IEEE Multi - Conference on Systems and Control (p. 800-806). October 3-5, 2012. Dubrovnik, Croatia. http://www.kitepower.eu/images/stories/publications/fechner12b.pdf U. Fechner and R. Schmehl, “Feed-Forward Control of Kite Power Systems,” Journal of Physics, 2014. http://iopscience.iop.org/1742-6596/524/1/012081 Uwe Fechner On Monday, September 15, 2014 3:44:21 PM UTC+2, Patrick O'Leary wrote: On Monday, September 15, 2014 3:25:07 AM UTC-5, Uwe Fechner wrote: With Python you can reach a jitter of less than 2 ms in the 20Hz control loop quite easily (on low-latency Linux). In my case this is sufficient for prototyping, but the real flight control system should run at a higher update frequency (perhaps 200 Hz). Semi-off-topic: 200 Hz seems really fast to me. What kind of high-frequency dynamics do these things have? I'm not familiar with this particular design space. Any good (open-access) papers? Patrick
Re: [julia-users] Does Julia have something similar to Python's documentation string?
Thanks for having a look at Docile.jl. I’ll try to explain some of
my decisions regarding it’s design.
*Introducing a new AST object:*
Having something built into the language would be great and I’d
definitely support that. There’s been some discussion, I think, about
doing that - probably in this thread or another from earlier, perhaps on
GitHub.
I needed something that works now and wouldn’t need changes to any
internals, hence the use of macros for this.
*Using comments rather than strings:*
A much earlier version of Docile.jl did do this, in part, but what I
found was that since comments get discarded during evaluation of the
code this required a separate run to capture documentation. I may have
missed a few tricks to get that to work though.
Another gripe I have about using comments: unless you use some special
syntax/tag (such as the extra # character) to denote what is actually
documentation and what isn’t then commenting out code temporarily
*could* cause problems if docstrings are parsed by Julia “as valid AST
objects”. Using a docstring “tag” to avoid this would work I think, so
long as it *visually distinguishes* plain comments from docstring
comments.
*Metadata in the docstring/comment:*
##
# docstring
#
# Metadata {
# key1 = value1
# }
#
# Examples
#
An earlier version I had did embed the metadata directly into the
docstring using the YAML.jl package. Switching to an actual Dict
simplified the code and also makes it easier to generate metadata
programmatically if necessary.
*Readability of @doc:*
I think that this probably just comes down to personal preference for me -
I’ve not done an extensive comparison between different syntax.
@doc introduces a docstring and seems pretty straightforward to me. It
explicitly states that what follows is documentation. That example from
Docile.jl could probably do with some simplifications since that metadata
section looks terrible if I’m honest. Something like the following might be
better as an initial example:
module PackageName
using Docile
@docstrings # must appear before any `@doc` calls
@doc
Markdown formatted text goes here...
-
function myfunc(x, y)
x + y
end
end
And then leave introducing metadata until after this since I’ve found
metadata to not be needed for every docstring I write.
I’m not sure about the “clearly visible bounded block” though, what in
particular could be clearer? I’m asking since I’ve been staring at these
for a while now and have become quite accustomed to them.
— Mike
Re: [julia-users] Random numbers and the Poisson distribution
I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasnoackjen...@gmail.com wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem space...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those event rates. Is it possible to do the same in Julia? (I can write rand(6) and get a vector of uniformly distributed random numbers, but I need the for loop to do the same for other distributions). In Octave you can pretty much write something like X += dX * poissrnd(rates * dt); all in one line (where X is a vector of populations, and dX is the event rate / state change matrix), and it would be nice to be as elegant as that in Julia. Thank you very much in advance. Regards, Jamie
[julia-users] Re: Module Includes Problem
How does your dirlist.jl file look?
I can't find any reference to dirlist.jl or the function dirlist anywhere.
Ivar
kl. 14:30:25 UTC+2 mandag 15. september 2014 skrev RecentConvert følgende:
I created a module to load and parse a set collection of data files. It
worked in Julia 0.2. Here is the module with hopefully only the
non-relevant code removed.
module Aerodyne
using DataFrames
using Dates
require(dirlist.jl)
export Status, aerodyne, aerodyne_load
type Status
# Constructor for Status type
function Status{T:Int64}(vals::DataFrames.DataArray{T,1})
Status(vector(vals))
end # End Status{T:Int64}(vals::DataFrames.DataArray{T,1})
constructor
# Constructor for Status type
function Status(vals::Array{Int64})
end # End of constructor
end # End of type
function aerodyne_load(Dr::String,mindate::DateTime)
end # End of aerodyne_load(Dr::String,mindate::DateTime)
function aerodyne_load(Dr::String,mindate::DateTime,maxdate::DateTime)
## Load STR and STC files in the given directory (Dr) between the
given dates ##
# Check for Directory
if isdir(Dr) == false
error(First input should be a directory)
end
# List Files
(Fstr,folders) = dirlist(Dr,regex=r\.str$) # List STR files # Line
140
(Fstc,folders) = dirlist(Dr,regex=r\.stc$) # List STC files
end # End of aerodyne_load(Dr::String,mindate::DateTime,maxdate::DateTime)
function aerodyne_load{T:String}(F::Array{T,1})
end # End aerodyne_load(F::Array{String,1})
function parse_time{T:String}(F::Array{T,1})
end # End of parse_time(F::Array{String,1})
function aerodyne_load(F::String)
end # End of aerodyne_load(F::String)
function aerodyne()
end # End of aerodyne()
end # End of module
The error that is returned is as follows.
ERROR: dirlist not defined
while loading O:\Code\Julia\qcl_plot_basic.jl, in expression starting on
line 34
while loading In[4], in expression starting on line 1
in aerodyne_load at O:\Code\Julia\Aerodyne.jl:140
Line 140 simply calls dirlist which is suppose to recursively go through a
given directory and return files matching the input regular expression.
dirlist.jl and the directory where all of my code resides are loaded into
Julia every time from the .juliarc.jl file.
(Fstr,folders) = dirlist(Dr,regex=r\.str$) # List STR files, line 140
Any ideas?
Re: [julia-users] Random numbers and the Poisson distribution
Thanks for your response! I can do that (once I figure out how). Regards, Jamie On Monday, 15 September 2014 15:33:25 UTC+1, Andreas Noack wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com javascript:: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those event rates. Is it possible to do the same in Julia? (I can write rand(6) and get a vector of uniformly distributed random numbers, but I need the for loop to do the same for other distributions). In Octave you can pretty much write something like X += dX * poissrnd(rates * dt); all in one line (where X is a vector of populations, and dX is the event rate / state change matrix), and it would be nice to be as elegant as that in Julia. Thank you very much in advance. Regards, Jamie
Re: [julia-users] Random numbers and the Poisson distribution
Hi John, thanks for your response. If you don't think silently accepting lambda = 0 is good practice, how best should I approach this? The only other thing I can think of is writing my own function wrapper specifically to catch the case where lambda = 0. I don't know how other people work with random numbers, but my suspicions are that being able to get a random number when lambda = 0 might be more common than needing skew and kurtosis for that case (although I do accept these are all problematic cases, ballooning to infinity). The Poisson distribution is very important to modellers, and zero rates do need to be handled somehow (currently R, Octave, and the GSL in C++ all accept lambda = 0, so the textbook definition isn't necessarily the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasno...@gmail.com javascript: wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com javascript:: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those event rates. Is it possible to do the same in Julia? (I can write rand(6) and get a vector of uniformly distributed random numbers, but I need the for loop to do the same for other distributions). In Octave you can pretty much write something like X += dX * poissrnd(rates * dt); all in one line (where X is a vector of populations, and dX is the event rate / state change matrix), and it would be nice to be as elegant as that in Julia. Thank you very much in advance. Regards,
Re: [julia-users] Lint.jl status update
Jacob,
I recently got my eyes on integrating this with in ST somehow too, so let
me know if I can help (I'm @tlycken at github). I was thinking that
integration through SublimeLinter (http://www.sublimelinter.com/en/latest/)
might be a good way to do it, but I haven't had time to look further into
it.
// Tomas
On Monday, September 15, 2014 8:04:07 AM UTC+2, Tony Fong wrote:
Jacob, Mike,
I just made a 2-line change to help you with that. You can do (block of
code as text) + (file/line info) - Array( LintMessage,1 )
using Lint
ctx = LintContext()
ctx.file = file
ctx.path = path # optional, to direct include() correctly
msgs = lintstr( code, ctx, lineoffset ) # lineoffset is how you shift
from the relative line of the code to actual
On module, however, this is tricky. Some of the Lint code needs to have
the whole module code in place to work, such as duplicated export, import
using relative path (import .Show). I'd recommend that when you care about
those Lint warnings you'd do a lint on the entire module file. I hope it
isn't too bad of a restriction that meanwhile you still get most of the
local-level lint functionality.
Tony
On Monday, September 15, 2014 6:33:59 AM UTC+7, Mike Innes wrote:
We can get really nice integration for Lint.jl and Light Table – I've
actually already got some of the GUI parts worked out, so it won't be crazy
difficult to do.
Quick question: How's Lint.jl's support for modules? For LT it's pretty
essential that the API looks like (block of code as text) + (file/line
info) + (module) = (warnings/errors). Of course, it's fine if I can
wrap Lint.jl's existing functionality to have that, but current AFAICT it
currently only works in terms of whole files.
On Sunday, 14 September 2014 01:12:49 UTC-4, Viral Shah wrote:
I wonder if these can be integrated into LightTable and IJulia, so that
they always automatically are running in the background on all code one
writes.
-viral
On Sunday, September 14, 2014 8:38:09 AM UTC+5:30, Spencer Russell wrote:
Any comments on how Lint.jl and @astrieanna's also-awesome TypeCheck.jl
relate? Are you two working together, or are there different use cases for
the two libraries?
peace,
s
On Sat, Sep 13, 2014 at 3:34 PM, Tony Fong tony.h...@gmail.com wrote:
Fellow Julians,
I think it is time to post an update on Lint.jl
https://github.com/tonyhffong/Lint.jl, as it has improved quite a
bit from the initial version I started about 3 months ago.
Notable new features
- Local variable type tracking, which enables a range of features,
such as
- Variable type stability warning within a function scope.
- Incompatibility between type assertion and assignment
- Omission of returning the constructed object in a type
constructor
- Check the call signature of a selected set of methods with
collection (push!, append!, etc.)
- More function checks, such as
- repeated arguments
- wrong signatures, e.g. f( x::Array{Number,1} )
- Mispelled constructor (calls new but the function name
doesn't match the enclosing type)
- Ability to silence lint warning via lintpragma() function, e.g.
- lintpragma( Ignore unstable type variable [variable name] )
- lintpragma( Ignore Unused [variable name] )
Also, there is now quite a range of test scripts showing sample codes
with lint problems, so it's easy to grep your own lint warnings in that
folder and see a distilled version of the issue.
Again, please let me know about gaps and false positives.
Tony
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here to
make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
Re: [julia-users] Random numbers and the Poisson distribution
This is not a big deal to me, but what exactly would break by allowing λ=0? Returning inf for skewness and kurtosis and an error for entropy doesn't seem problematic to me. I don't do estimation of KL divergences, but it appears a bit strange to that extending the parameter space with a point could make a difference as the degenerate case can be approximated arbitrarily well with the non degenerate distribution (except for floating point rounding). Med venlig hilsen Andreas Noack 2014-09-15 11:25 GMT-04:00 spaceLem space...@minioak.com: Hi John, thanks for your response. If you don't think silently accepting lambda = 0 is good practice, how best should I approach this? The only other thing I can think of is writing my own function wrapper specifically to catch the case where lambda = 0. I don't know how other people work with random numbers, but my suspicions are that being able to get a random number when lambda = 0 might be more common than needing skew and kurtosis for that case (although I do accept these are all problematic cases, ballooning to infinity). The Poisson distribution is very important to modellers, and zero rates do need to be handled somehow (currently R, Octave, and the GSL in C++ all accept lambda = 0, so the textbook definition isn't necessarily the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasno...@gmail.com wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/ poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only defined for lambda 0, but it really does make sense to handle the case lambda = 0 as returning 0. Somehow I feel that editing that file was probably not the correct thing to do (although it was great that I was able to), but I'd like to follow good practice, and I'm going to run into problems if I ever need to share my code. And my second question: in Octave I can specify an array of lambdas and get back an array of random numbers distributed according to those
Re: [julia-users] Random numbers and the Poisson distribution
Why not just write out an explicit check in your code for the special case? Here's an example of how one might do that: n_samples = 10 n_lambdas = 3 lambdas = [0.0, 0.1, 0.2] samples = Array(Int, n_samples, n_lambdas) for (i, lambda) in enumerate(lambdas) for sample in 1:n_samples if rate == 0.0 samples[sample, i] = 0 else samples[sample, i] = rand(Poisson(lambda)) end end end In my mind, this is the best possible approach to this kind of problem: you want slighly unusual behavior, so you should make the place where your goals are slightly unusual as explicit as possible so that anyone who reads your code will understand exactly what you're doing and exactly where you're breaking from textbook definitions. As for precedent, I believe the list of systems you've provided breaks down as follows: RNG's take in numbers, not objects: * R * Octave * GSL RNG's take in objects, not numbers: * SciPy * Julia So it seems like the split in behavior is perfectly explicable in terms of the types of arguments you provide to RNG's. This seems like a very good argument for not changing the behavior of Distributions. -- John On Sep 15, 2014, at 8:25 AM, spaceLem space...@minioak.com wrote: Hi John, thanks for your response. If you don't think silently accepting lambda = 0 is good practice, how best should I approach this? The only other thing I can think of is writing my own function wrapper specifically to catch the case where lambda = 0. I don't know how other people work with random numbers, but my suspicions are that being able to get a random number when lambda = 0 might be more common than needing skew and kurtosis for that case (although I do accept these are all problematic cases, ballooning to infinity). The Poisson distribution is very important to modellers, and zero rates do need to be handled somehow (currently R, Octave, and the GSL in C++ all accept lambda = 0, so the textbook definition isn't necessarily the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasno...@gmail.com wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon,
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
Thanks Ivar. When I get further along, I'd love for you to take a look at
what I did. Right now, I'm blindly wrapping the functions as practice, but
if you look in my code/tests, I can tell there are functions there that are
either duplicative or don't make sense. And if my naive wrapping is leaking
memory because the original function is leaking memory, I'd love to get
that taken care of before adding to METADATA :)
On Monday, September 15, 2014 11:37:15 AM UTC-4, Ivar Nesje wrote:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here to
make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
Re: [julia-users] Random numbers and the Poisson distribution
The KL divergence (and other comparisons) between this edge case and any standard instance of the Poisson distribution are problematic because the inclusion of this point in parameter space changes the support of the distribution from the entire set of non-negative integers to the set containing only zero. -- John On Sep 15, 2014, at 8:42 AM, Andreas Noack andreasnoackjen...@gmail.com wrote: This is not a big deal to me, but what exactly would break by allowing λ=0? Returning inf for skewness and kurtosis and an error for entropy doesn't seem problematic to me. I don't do estimation of KL divergences, but it appears a bit strange to that extending the parameter space with a point could make a difference as the degenerate case can be approximated arbitrarily well with the non degenerate distribution (except for floating point rounding). Med venlig hilsen Andreas Noack 2014-09-15 11:25 GMT-04:00 spaceLem space...@minioak.com: Hi John, thanks for your response. If you don't think silently accepting lambda = 0 is good practice, how best should I approach this? The only other thing I can think of is writing my own function wrapper specifically to catch the case where lambda = 0. I don't know how other people work with random numbers, but my suspicions are that being able to get a random number when lambda = 0 might be more common than needing skew and kurtosis for that case (although I do accept these are all problematic cases, ballooning to infinity). The Poisson distribution is very important to modellers, and zero rates do need to be handled somehow (currently R, Octave, and the GSL in C++ all accept lambda = 0, so the textbook definition isn't necessarily the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasno...@gmail.com wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate occurs zero times. I found that editing poisson.jl (mentioned in the error), and changing line 4 from if l 0 to if l = 0, then the error goes away and when events have zero rates, they correctly occur zero times. I know the Poisson distribution is technically only
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
[julia-users] Re: constants and containers
But this is not the same thing. AFAIK, this is not possible yet as a Base class. But I'm far from knowing everything. I've seen discussions about providing a way to inherit default methods from a field of a composite type but I can't find them. One problem of the method is type preservation. But whatever, you could still do that and set the methods appropriately. This would be tedious but should work: all you have to do is to not set a setindex! method for your new type. Or you may want instead to inherit from AbstractArray and define everything missing from there and again, set setindex! to something else for your new type. I've never done it, I don't know what it would require to make that working. While I would love to see immutability by default on variables which would avoid a whole class of bugs and inefficiencies for a modest cost (except maybe in the global scope for the REPL? or maybe that would solve the global scope efficiency problem?), I'm more sceptical about immutable arrays/dict. Not because it is not sensible, but because if someone is tempted to change one of your array values, nothing can prevent him to do so (in Julia and in many other languages). My take on this would be to go for the composite type and prefix the private things with private or _ or whatever and add a comment stating that this should never be accessed directly.
[julia-users] Re: Send variable to parallel workers for pmap
@spawnat(2,whos()) is useful to know what's defined in other processes
(e.g. for process 2 in this case)
if you want to gather the return values, use a reducer function
e.g.
let a = rand()
@parallel hcat for x in 1:3
foo(a,x)
end
end
trying to remember the other approach which I thought I got working rather
than copying it over each run.
One other thing is that requiring a file after adding the processes should
load it on all processes which is another approach to setting up the
environment for each process
On Monday, September 15, 2014 3:51:43 PM UTC+1, John Drummond wrote:
with parallel
addprocs(2)
println(Number of processors, nprocs())
@everywhere function foo(x,y)
println(myid(), , x, , y)
return x + y
end
let a = rand()
@sync @parallel for x in 1:3
foo(a,x)
end
end
On Monday, September 15, 2014 3:14:50 PM UTC+1, xiong...@gmail.com wrote:
you could precreate A and then define it everywhere
Could you please show me some code how to do it?
On Monday, September 15, 2014 2:51:08 PM UTC+2, John Drummond wrote:
Chris Strickland
https://groups.google.com/forum/#!msg/julia-users/jlKoEtErRL4/0ZcB_hxyJlYJ
lists one approach for the general problem
you could precreate A and then define it everywhere, or send a copy over
as a parameter to whatever function you use in pmap, similar but not the
same as above
Another approach which was useful to me was the @parallel for loops
Any variables used inside the parallel loop will be copied and
broadcast to each process.
http://julia.readthedocs.org/en/latest/manual/parallel-computing/
and shared arrays
http://julia.readthedocs.org/en/latest/manual/parallel-computing/#shared-arrays-experimental-unix-only-feature
if using linux could be useful (I've not tried them).
On Monday, September 15, 2014 10:52:33 AM UTC+1, xiong...@gmail.com
wrote:
I want to transfer a variable to all parallel workers. However, if I do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't
think it is smart since A is expand in memery unnessarily. Is there any
simple way to just send a copy of A, or the reference of A, to all
parallel
mashines?
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
Re: [julia-users] Does Julia have something similar to Python's documentation string?
On Saturday, September 13, 2014 5:18:54 AM UTC-4, Stefan Karpinski wrote: There will always, however, be more documentation that doesn't affect the behavior of the code. I'm not even sure what it means for that kind of documentation to be semantically meaningful – you mean that it has a specified format means something? If you can access the documentation from the code that creates it, without running any separate documentation-processing step, then it is semantically meaningful (and can affect the behavior of the code if the code so chooses). Also, our documentation will necessarily be tied to the semantics of the language -- for example, it can be tied to a Function in general or to a Method (as opposed to semantics-unaware documentation systems where functions can only be indexed by name), and I still have yet to see a clean way to do this with comments. Furthermore, I like having a clean separation between code comments (meant for programmers reading the source) and documentation (meant for users not looking at the source). But my biggest problem with using documentation in comments remains the lack of flexibility in formatting, metadata, etcetera. You're going to end up with either an inflexible system that is not easily extensible later to include richer information, or you're going to end up inventing your own mini-language for the docstrings (see the multiple PEPs on docstring formatting and metadata). We already have a pretty good language; why not use it? Would you want your choices of what can be represented in the documentation/metadata system right now to be dictated by formatting tools and environments circa 2004? circa 1994? How do you think Julia users will feel in 2024? In 2034? 20 years is not that long of a time for a living programming language; 10 years is about the time it takes to become mainstream.
[julia-users] C Global Structs
I'd like to see if Julia is running in code-coverage mode or not.
In the REPL I can do the following:
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x7fa3e01bbc90
The struct is defined like so:
https://github.com/JuliaLang/julia/blob/aab2c6e67b5aaee7f23bc5a52897f7219473c153/src/julia.h#L1330-L1338
How can I access the code_coverage member?
Thanks,
Steve
[julia-users] Re: constants and containers
On Sunday, September 14, 2014 6:04:20 PM UTC-4, Yakir Gagnon wrote: My original intention was to ask if there was any way we could declare a const array who's elements are also constants. Right now, the only way to do this is if you declare a new AbstractArray subtype, call it ReadOnlyArray, that has getindex but not setindex! methods.
Re: [julia-users] C Global Structs
julia immutable JCopt
build_path::Ptr{Uint8}
cc::Int8
ml::Int8
cb::Int8
dbc::Int8
il::Cint
ce::Int8
end
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x6d7daf90
julia unsafe_load(reinterpret(Ptr{JCopt}, a))
JCopt(Ptr{Uint8} @0x,1,0,0,0,0,16)
On Mon, Sep 15, 2014 at 12:30 PM, Steve Kelly kd2...@gmail.com wrote:
I'd like to see if Julia is running in code-coverage mode or not.
In the REPL I can do the following:
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x7fa3e01bbc90
The struct is defined like so:
https://github.com/JuliaLang/julia/blob/aab2c6e67b5aaee7f23bc5a52897f7219473c153/src/julia.h#L1330-L1338
How can I access the code_coverage member?
Thanks,
Steve
[julia-users] Re: C Global Structs
julia immutable CompilerOpts
build_path::Ptr{Cchar}
code_coverage::Int8
malloc_log::Int8
check_bounds::Int8
dumpbitcode::Int8
int_literals::Cint
compile_enabled::Int8
end
julia a = cglobal(:jl_compileropts, CompilerOpts)
Ptr{CompilerOpts} @0x00010f840a88
julia unsafe_load(a)
CompilerOpts(Ptr{Int8} @0x,0,0,0,0,0,1)
On Monday, September 15, 2014 12:30:48 PM UTC-4, Steve Kelly wrote:
I'd like to see if Julia is running in code-coverage mode or not.
In the REPL I can do the following:
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x7fa3e01bbc90
The struct is defined like so:
https://github.com/JuliaLang/julia/blob/aab2c6e67b5aaee7f23bc5a52897f7219473c153/src/julia.h#L1330-L1338
How can I access the code_coverage member?
Thanks,
Steve
[julia-users] Re: Send variable to parallel workers for pmap
r = remote_call(2, rand)
fetch(r)
fetch(@spawnat 2 1+fetch(r)+myid())
shows transferring data to another process and then running a query on it
(iterate through the processes as per Strickland to do this everywhere)
I didn't find a way to do this with pmap - you can transfer the data each
time though
pmap(x-myid()+x[1]+x[2], map(x-(x,a),1:3))
On Monday, September 15, 2014 5:09:18 PM UTC+1, John Drummond wrote:
@spawnat(2,whos()) is useful to know what's defined in other processes
(e.g. for process 2 in this case)
if you want to gather the return values, use a reducer function
e.g.
let a = rand()
@parallel hcat for x in 1:3
foo(a,x)
end
end
trying to remember the other approach which I thought I got working rather
than copying it over each run.
One other thing is that requiring a file after adding the processes should
load it on all processes which is another approach to setting up the
environment for each process
On Monday, September 15, 2014 3:51:43 PM UTC+1, John Drummond wrote:
with parallel
addprocs(2)
println(Number of processors, nprocs())
@everywhere function foo(x,y)
println(myid(), , x, , y)
return x + y
end
let a = rand()
@sync @parallel for x in 1:3
foo(a,x)
end
end
On Monday, September 15, 2014 3:14:50 PM UTC+1, xiong...@gmail.com wrote:
you could precreate A and then define it everywhere
Could you please show me some code how to do it?
On Monday, September 15, 2014 2:51:08 PM UTC+2, John Drummond wrote:
Chris Strickland
https://groups.google.com/forum/#!msg/julia-users/jlKoEtErRL4/0ZcB_hxyJlYJ
lists one approach for the general problem
you could precreate A and then define it everywhere, or send a copy
over as a parameter to whatever function you use in pmap, similar but not
the same as above
Another approach which was useful to me was the @parallel for loops
Any variables used inside the parallel loop will be copied and
broadcast to each process.
http://julia.readthedocs.org/en/latest/manual/parallel-computing/
and shared arrays
http://julia.readthedocs.org/en/latest/manual/parallel-computing/#shared-arrays-experimental-unix-only-feature
if using linux could be useful (I've not tried them).
On Monday, September 15, 2014 10:52:33 AM UTC+1, xiong...@gmail.com
wrote:
I want to transfer a variable to all parallel workers. However, if I
do:
A=rand()
pmap(x-A+x,1:3)
Return error:
exception on 2: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
exception on 3: ERROR: A not defined
in anonymous at none:1
in anonymous at multi.jl:855
in run_work_thunk at multi.jl:621
in anonymous at task.jl:855
2-element Array{Any,1}:
UndefVarError(:A)
UndefVarError(:A)
The result of
@everywhere A=rand()
pmap(x-A+x,1:3)
is not what I want, since I hope A in all mashines are the same.
I know that pmap((x,y)-x+y,1:3,fill(A,3)) will be work, but I don't
think it is smart since A is expand in memery unnessarily. Is there any
simple way to just send a copy of A, or the reference of A, to all
parallel
mashines?
[julia-users] Re: Module Includes Problem
dirlist.jl is a custom function that I made. The fourth line is the require and the line I singled out actually calls it. I've already debugged dirlist.jl for Julia 0.3 and have used it in other functions but this is my only module. At the moment I don't have access to my code but I can post anything needed on Wednesday at the earliest. Windows 7 Julia Version 0.3.0 Commit 7681878 (2014-08-20 20:43 UTC) Platform Info: System: Windows (x86_64-w64-mingw32) CPU: Intel(R) Core(TM) i5 CPU 670 @ 3.47GHz WORD_SIZE: 64 Microsoft Windows [Version 6.1.7601]
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Okay. I can see the problem now. The reason is that you have defined b as a matrix and I had it defined as a vector. It works if you define b = rand(1000), but the stack overflow is a bug. I'll look into it. Med venlig hilsen Andreas Noack 2014-09-15 12:12 GMT-04:00 Stéphane Mottelet stephane.motte...@gmail.com: Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
It is probably easier to just to use c_malloc in this case and free the
memory yourself.
On Monday, September 15, 2014 11:45:41 AM UTC-4, Randy Zwitch wrote:
Thanks Ivar. When I get further along, I'd love for you to take a look at
what I did. Right now, I'm blindly wrapping the functions as practice, but
if you look in my code/tests, I can tell there are functions there that are
either duplicative or don't make sense. And if my naive wrapping is leaking
memory because the original function is leaking memory, I'd love to get
that taken care of before adding to METADATA :)
On Monday, September 15, 2014 11:37:15 AM UTC-4, Ivar Nesje wrote:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
managed to get something working on my machine, so it might help you. this
is the code:
a = Array(Ptr{Ptr{Uint8}},1)
str =
oauth_token=2a71d1c73d2771b00f13ca0acb9836a10477d3c56oauth_token_secret=a1b5c00c1f3e23fb314a0aa22e990266
result =
ccall((:oauth_split_url_parameters,liboauth),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),convert(Ptr{Uint8},str),a)
aa = unsafe_load(a[1])
bb = unsafe_load(a[1],2)
bytestring(aa)
bytestring(bb)
cheers
On Monday, September 15, 2014 6:45:41 PM UTC+3, Randy Zwitch wrote:
Thanks Ivar. When I get further along, I'd love for you to take a look at
what I did. Right now, I'm blindly wrapping the functions as practice, but
if you look in my code/tests, I can tell there are functions there that are
either duplicative or don't make sense. And if my naive wrapping is leaking
memory because the original function is leaking memory, I'd love to get
that taken care of before adding to METADATA :)
On Monday, September 15, 2014 11:37:15 AM UTC-4, Ivar Nesje wrote:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Well, should it also work when b is a one column matrix ? Im am a complete newbie (coming from the Scilab world...) so I forgot this subtle difference in Julia... S. Le lundi 15 septembre 2014 18:49:02 UTC+2, Andreas Noack a écrit : Okay. I can see the problem now. The reason is that you have defined b as a matrix and I had it defined as a vector. It works if you define b = rand(1000), but the stack overflow is a bug. I'll look into it. Med venlig hilsen Andreas Noack 2014-09-15 12:12 GMT-04:00 Stéphane Mottelet stephane...@gmail.com javascript:: Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
It ought to work for matrix right hand sides as well so I'll fix that now. Med venlig hilsen Andreas Noack 2014-09-15 12:52 GMT-04:00 Stéphane Mottelet stephane.motte...@gmail.com: Well, should it also work when b is a one column matrix ? Im am a complete newbie (coming from the Scilab world...) so I forgot this subtle difference in Julia... S. Le lundi 15 septembre 2014 18:49:02 UTC+2, Andreas Noack a écrit : Okay. I can see the problem now. The reason is that you have defined b as a matrix and I had it defined as a vector. It works if you define b = rand(1000), but the stack overflow is a bug. I'll look into it. Med venlig hilsen Andreas Noack 2014-09-15 12:12 GMT-04:00 Stéphane Mottelet stephane...@gmail.com: Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
Re: [julia-users] constants and containers
FunctionalCollections.jl actually implements an immutable array type. I haven't looked at it recently, so I don't know it's current status. Cheers, Kevin On Monday, September 15, 2014, Steven G. Johnson stevenj@gmail.com wrote: On Sunday, September 14, 2014 6:04:20 PM UTC-4, Yakir Gagnon wrote: My original intention was to ask if there was any way we could declare a const array who's elements are also constants. Right now, the only way to do this is if you declare a new AbstractArray subtype, call it ReadOnlyArray, that has getindex but not setindex! methods.
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
Thanks a lot, It is really cool to find UMFpack out-of-the-box in Julia. S. Le lundi 15 septembre 2014 18:56:12 UTC+2, Andreas Noack a écrit : It ought to work for matrix right hand sides as well so I'll fix that now. Med venlig hilsen Andreas Noack 2014-09-15 12:52 GMT-04:00 Stéphane Mottelet stephane...@gmail.com javascript:: Well, should it also work when b is a one column matrix ? Im am a complete newbie (coming from the Scilab world...) so I forgot this subtle difference in Julia... S. Le lundi 15 septembre 2014 18:49:02 UTC+2, Andreas Noack a écrit : Okay. I can see the problem now. The reason is that you have defined b as a matrix and I had it defined as a vector. It works if you define b = rand(1000), but the stack overflow is a bug. I'll look into it. Med venlig hilsen Andreas Noack 2014-09-15 12:12 GMT-04:00 Stéphane Mottelet stephane...@gmail.com: Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
Re: [julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
managed to get something working on my machine, so it might help you. this
is the code:
You need to also add:
ccall((:oauth_free_array,liboauth), Void, (Ptr{Cint},
Ptr{Ptr{Ptr{Cchar_t}}}), [result], a)
(roughly - untested)
On Mon, Sep 15, 2014 at 12:52 PM, David Gonzales dvdgonzale...@gmail.com
wrote:
managed to get something working on my machine, so it might help you. this
is the code:
a = Array(Ptr{Ptr{Uint8}},1)
str =
oauth_token=2a71d1c73d2771b00f13ca0acb9836a10477d3c56oauth_token_secret=a1b5c00c1f3e23fb314a0aa22e990266
result =
ccall((:oauth_split_url_parameters,liboauth),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),convert(Ptr{Uint8},str),a)
aa = unsafe_load(a[1])
bb = unsafe_load(a[1],2)
bytestring(aa)
bytestring(bb)
cheers
On Monday, September 15, 2014 6:45:41 PM UTC+3, Randy Zwitch wrote:
Thanks Ivar. When I get further along, I'd love for you to take a look at
what I did. Right now, I'm blindly wrapping the functions as practice, but
if you look in my code/tests, I can tell there are functions there that are
either duplicative or don't make sense. And if my naive wrapping is leaking
memory because the original function is leaking memory, I'd love to get
that taken care of before adding to METADATA :)
On Monday, September 15, 2014 11:37:15 AM UTC-4, Ivar Nesje wrote:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/
master/src/OAuth.jl#L248-254
function oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{
Uint8}}})
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
Re: [julia-users] Random numbers and the Poisson distribution
or with comprehension and unicode-sugar: using Distributions Λ = 50rand(6) ; Λ[3] = 0.0 ; Δt = 0.1 [λ 0.0 ? rand(Poisson(λ*Δt)) : 0 for λ in Λ] @assert Julia max(Matlab,Octave,c++) On Monday, September 15, 2014 6:51:39 PM UTC+3, John Myles White wrote: The KL divergence (and other comparisons) between this edge case and any standard instance of the Poisson distribution are problematic because the inclusion of this point in parameter space changes the support of the distribution from the entire set of non-negative integers to the set containing only zero. -- John On Sep 15, 2014, at 8:42 AM, Andreas Noack andreasno...@gmail.com javascript: wrote: This is not a big deal to me, but what exactly would break by allowing λ=0? Returning inf for skewness and kurtosis and an error for entropy doesn't seem problematic to me. I don't do estimation of KL divergences, but it appears a bit strange to that extending the parameter space with a point could make a difference as the degenerate case can be approximated arbitrarily well with the non degenerate distribution (except for floating point rounding). Med venlig hilsen Andreas Noack 2014-09-15 11:25 GMT-04:00 spaceLem spac...@minioak.com javascript:: Hi John, thanks for your response. If you don't think silently accepting lambda = 0 is good practice, how best should I approach this? The only other thing I can think of is writing my own function wrapper specifically to catch the case where lambda = 0. I don't know how other people work with random numbers, but my suspicions are that being able to get a random number when lambda = 0 might be more common than needing skew and kurtosis for that case (although I do accept these are all problematic cases, ballooning to infinity). The Poisson distribution is very important to modellers, and zero rates do need to be handled somehow (currently R, Octave, and the GSL in C++ all accept lambda = 0, so the textbook definition isn't necessarily the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in Distributions that assumes that every Poisson object defines a non-delta distribution. At a minimum, all of the following functions would need to have branches inserted into them to work around the lambda = 0 case: * entropy * kurtosis * skewness In addition, every person who ever wrote code in the future that worked with Poisson objects would need to know that our definition of the Poisson distribution contradicted the definition found in textbooks. This would affect people writing code to estimate KL divergences, for example. — John On Sep 15, 2014, at 7:33 AM, Andreas Noack andreasno...@gmail.com wrote: I can see that R accepts zero as rate parameter so maybe we should do the same. Could you open a pull request to Distributions.jl with that change? Regarding the vectorized version, the answer is that you can do almost what you want with a comprehension, i.e. something like X += dX*[rand(Poisson(r*dt)) for r in rates]. Med venlig hilsen Andreas Noack 2014-09-15 10:05 GMT-04:00 spaceLem spac...@minioak.com: Hi all, I work in disease modelling, using use a mix of C++ and Octave. I'm fairly new to Julia, although I've managed to get a basic model up and running, and at 1/5.5 times the speed of C++ I'm pretty impressed (although hoping to close in on C++). I'd love to be able to work in one language all the time, and I'm feeling that I'm not far off. I have two questions regarding random numbers and the Poisson distribution. One algorithm I use has a number of possible events, each with an associated event rate. From these events, you choose a time step dt, then the number of times each event happens is Poisson distributed with lambda = rate of the event * dt. In Octave I could write code along these lines (simplified to get the gist of things): rates = 50*rand(1,6); rates(3) = 0; dt = 0.1; K = poissrnd(rates*dt); % = [1 6 0 4 3 4] where K is an array giving the number of times each event occurs. In Julia, I would write using Distributions rates = 50rand(6) rates[3] = 0 dt = 0.1 K = zeros(Float64,6) for i = 1:6; K[i] = rand(Poisson(rates[i] * dt)); end This gives: ERROR: lambda must be positive in Poisson at /Users/spacelem/.julia/Distributions/src/univariate/ poisson.jl:4 in anonymous at no file:1 Which brings me to my first question: how best to handle when the events have zero rates (which is not uncommon, and needs to be handled)? The correct behaviour is that an event with zero rate
Re: [julia-users] How to manage local, proprietary packages
Keno's suggestion of using a custom METADATA is working well. The users only have to invoke Pkg.init(our local METADATA repo) once when first setting up their environment. Works well on Windows and Linux. One little issue: I thought it might be best to create my own local branch of METADATA and then merge changes to `metadata-v2` at origin into it. `Pkg.init` supports using this new branch via its second argument. However, `Pkg.update()` seems to always reset the working branch to `metadata-v2`, so I have been simply adding local commits to that branch, which still allows me to pull from origin. Not sure if this is something worth flagging as an issue, or if it is intentional. Also, created PR #8365 https://github.com/JuliaLang/julia/pull/8365, which attempts to improve the `Pkg.init` documentation. Thanks, --Peter On Friday, September 12, 2014 6:39:38 AM UTC-7, Peter Simon wrote: I will have a chance to play with this over the weekend, and will be happy to share any positive results. I will also see if I can improve the documentation. --Peter On Friday, September 12, 2014 3:27:13 AM UTC-7, Ivar Nesje wrote: I would not say that the documentation is wrong. It is just confusing and fails to mention the optional arguments. As we have standardized on ~/.julia, we should probably mention where we will end up creating files. If you think you got a grip of how the function works, you can suggest improvements to the docs here https://github.com/JuliaLang/julia/edit/master/doc/stdlib/pkg.rst. If not you should open an issue on github about the confusion so that we can add an appropriate milestone. Ivar kl. 23:07:40 UTC+2 torsdag 11. september 2014 skrev Peter Simon følgende: Ah! I had a look at the source code for `Pkg.init` and see that its default first argument is DEFAULT_META, which is initialized to git:// github.com/JuliaLang/METADATA.jl. So `Pkg.init` is used to point the package manager to the global METAFILE package directory. This is definitely not what I took away from the documentation, which reads: help? Pkg.init Base.Pkg.init() Initialize Pkg.dir() as a package directory. This will be done automatically when the JULIA_PKGDIR is not set and Pkg.dir() uses its default value. This is the same as the online docs. Since the documentation talks about `Pkg.dir()` I thought it was for establishing the local package repository location. Looks like the documentation wrong/out of date? Thanks, --Peter
Re: [julia-users] Does Julia have something similar to Python's documentation string?
Just to engage in some bikeshedding is @doc better than defining doc_str or d_str? The triple quote notation seems like an unnecessary pythonism. doc_str gives: doc Markdown formatted text goes here... - function myfunc(x, y) x + y end On Monday, September 15, 2014 10:02:49 AM UTC-5, Michael Hatherly wrote: *Readability of @doc:* I think that this probably just comes down to personal preference for me - I’ve not done an extensive comparison between different syntax. @doc introduces a docstring and seems pretty straightforward to me. It explicitly states that what follows is documentation. That example from Docile.jl could probably do with some simplifications since that metadata section looks terrible if I’m honest. Something like the following might be better as an initial example: module PackageName using Docile @docstrings # must appear before any `@doc` calls @doc Markdown formatted text goes here... - function myfunc(x, y) x + y end end And then leave introducing metadata until after this since I’ve found metadata to not be needed for every docstring I write. I’m not sure about the “clearly visible bounded block” though, what in particular could be clearer? I’m asking since I’ve been staring at these for a while now and have become quite accustomed to them.
Re: [julia-users] Does Julia have something similar to Python's documentation string?
I should add that I'm excited to try out the package as is and successfully document my functions. On Monday, September 15, 2014 12:36:03 PM UTC-5, Gray Calhoun wrote: Just to engage in some bikeshedding is @doc better than defining doc_str or d_str? The triple quote notation seems like an unnecessary pythonism. doc_str gives: doc Markdown formatted text goes here... - function myfunc(x, y) x + y end On Monday, September 15, 2014 10:02:49 AM UTC-5, Michael Hatherly wrote: *Readability of @doc:* I think that this probably just comes down to personal preference for me - I’ve not done an extensive comparison between different syntax. @doc introduces a docstring and seems pretty straightforward to me. It explicitly states that what follows is documentation. That example from Docile.jl could probably do with some simplifications since that metadata section looks terrible if I’m honest. Something like the following might be better as an initial example: module PackageName using Docile @docstrings # must appear before any `@doc` calls @doc Markdown formatted text goes here... - function myfunc(x, y) x + y end end And then leave introducing metadata until after this since I’ve found metadata to not be needed for every docstring I write. I’m not sure about the “clearly visible bounded block” though, what in particular could be clearer? I’m asking since I’ve been staring at these for a while now and have become quite accustomed to them.
Re: [julia-users] Does Julia have something similar to Python's documentation string?
*Well Leah’s already answered this while I was fighting with my formatting, but here’s mine anyway :)* Welcome to the shed :) I really like that syntax and if it’s possible to get it to work that would be really nice.The problem is that a @doc_str macro wouldn’t capture the Expr that is being documented. See the dumps below: julia dump(quote doc Markdown formatted text goes here... - function myfunc(x, y) x + y end end) Expr head: Symbol block args: Array(Any,(2,)) 1: Expr head: Symbol line args: Array(Any,(2,)) 1: Int64 3 2: Symbol none typ: Any 2: Expr head: Symbol - args: Array(Any,(2,)) 1: Expr head: Symbol macrocall args: Array(Any,(2,)) typ: Any 2: Expr head: Symbol block args: Array(Any,(2,)) typ: Any typ: Any typ: Any julia dump(quote @doc Markdown formatted text goes here... - function myfunc(x, y) x + y end end) Expr head: Symbol block args: Array(Any,(2,)) 1: Expr head: Symbol line args: Array(Any,(2,)) 1: Int64 3 2: Symbol none typ: Any 2: Expr head: Symbol macrocall args: Array(Any,(2,)) 1: Symbol @doc 2: Expr head: Symbol - args: Array(Any,(2,)) typ: Any typ: Any typ: Any @doc_str takes the contents of the string in as an argument (you can pass some flags in as well, see the Regex syntax for examples) and not the Expr appearing after. The @doc macro takes a varargs macro doc(args...) and so can capture everything after. The trick is the - which does “line continuation” (or something like that). The - also allows Docile to capture line number information of things other than method definitions. If you’ve looked at the generated docs for Docile you’ll see that everything has file and line number information provided. Glad you’re excited. Give me a shout if you run into any issues.
Re: [julia-users] Does Julia have something similar to Python's documentation string?
Yes, this covers it quite well. -- Mike
Re: [julia-users] Solve transpose(A)*x=b with sparse LU factors of A
It is fixed now and I have backported the fix to 0.3 so when we release 0.3.1 it should also work there. You can also try the nightlies tomorrow or build the latest version. Med venlig hilsen Andreas Noack 2014-09-15 13:01 GMT-04:00 Stéphane Mottelet stephane.motte...@gmail.com: Thanks a lot, It is really cool to find UMFpack out-of-the-box in Julia. S. Le lundi 15 septembre 2014 18:56:12 UTC+2, Andreas Noack a écrit : It ought to work for matrix right hand sides as well so I'll fix that now. Med venlig hilsen Andreas Noack 2014-09-15 12:52 GMT-04:00 Stéphane Mottelet stephane...@gmail.com: Well, should it also work when b is a one column matrix ? Im am a complete newbie (coming from the Scilab world...) so I forgot this subtle difference in Julia... S. Le lundi 15 septembre 2014 18:49:02 UTC+2, Andreas Noack a écrit : Okay. I can see the problem now. The reason is that you have defined b as a matrix and I had it defined as a vector. It works if you define b = rand(1000), but the stack overflow is a bug. I'll look into it. Med venlig hilsen Andreas Noack 2014-09-15 12:12 GMT-04:00 Stéphane Mottelet stephane...@gmail.com: Hello, I use Julia-0.3.0 (OSX version). Here is a testcase : a=sprand(1000,1000,0.1); af=factorize(a); b=rand(1000,1); x=(af')\b; ERROR: stack overflow in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times) S. Le lundi 15 septembre 2014 18:01:46 UTC+2, Viral Shah a écrit : Which version of Julia are you using? Please do provide a test case to generate Af and b, so that I can try reproduce it. -viral On Monday, September 15, 2014 2:00:11 PM UTC+5:30, Stéphane Mottelet wrote: Hello, It seems that it does not work as you said : *julia (A**f)'\b* *ERROR: stack overflow* * in Ac_ldiv_B! at linalg/umfpack.jl:354 (repeats 8 times)* In order to motivate my needs, I have to solve both systems, one with A and the other one with transpose(A). S. Le vendredi 12 septembre 2014 17:43:50 UTC+2, Andreas Noack a écrit : I believe that if Af = lufact(A) for sparse A then Af\b will give you what you want. The expression is parsed such that the transpose is not actually computed. Instead it calls the methods Ac_ldiv_B which calls the right solver in UMFPack. Med venlig hilsen Andreas Noack 2014-09-12 9:55 GMT-04:00 Stéphane Mottelet stephane...@free.fr: Hello, How do you solve transpose(A)*x=b (without refactoring) when A is sparse and has been factored ? UMFPack allows this in other implementations (e.g. Scilab). Thanks for help S.
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
I should probably not post this untested code, but it is way too tempting
when I need to install a library to actually test it.
function oauth_split_url_parameters(url::String)
argv = Ptr{Array{Uint8,1}}[C_NULL]
argc =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
result = Array(UTF8String, argc)
for i = 1:argc
result[i] = bytestring(argv[1][i])
end
ccall((:oath_free_array,LIBOAUTH), Nothing, (Ptr{Cint},
Ptr{Ptr{Ptr{Uint8}}}), argc, argv)
return result
end
kl. 17:37:15 UTC+2 mandag 15. september 2014 skrev Ivar Nesje følgende:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here to
make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
No problem, I'll test it out...that's what VM's are for :)
On Monday, September 15, 2014 2:22:31 PM UTC-4, Ivar Nesje wrote:
I should probably not post this untested code, but it is way too tempting
when I need to install a library to actually test it.
function oauth_split_url_parameters(url::String)
argv = Ptr{Array{Uint8,1}}[C_NULL]
argc =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
result = Array(UTF8String, argc)
for i = 1:argc
result[i] = bytestring(argv[1][i])
end
ccall((:oath_free_array,LIBOAUTH), Nothing, (Ptr{Cint},
Ptr{Ptr{Ptr{Uint8}}}), argc, argv)
return result
end
kl. 17:37:15 UTC+2 mandag 15. september 2014 skrev Ivar Nesje følgende:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
Ivar, your code returned an error:
ERROR: type: ccall: in argument 2, expected Ptr{Ptr{Uint8}}, got
Ptr{Array{Uint8,1}}
in oauth_split_url_parameters at none:3
David, your code worked for me, but it seems that you need to know how many
parameters are in the string. Is there a way to just return as an array?
On Monday, September 15, 2014 2:27:57 PM UTC-4, Randy Zwitch wrote:
No problem, I'll test it out...that's what VM's are for :)
On Monday, September 15, 2014 2:22:31 PM UTC-4, Ivar Nesje wrote:
I should probably not post this untested code, but it is way too tempting
when I need to install a library to actually test it.
function oauth_split_url_parameters(url::String)
argv = Ptr{Array{Uint8,1}}[C_NULL]
argc =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
result = Array(UTF8String, argc)
for i = 1:argc
result[i] = bytestring(argv[1][i])
end
ccall((:oath_free_array,LIBOAUTH), Nothing, (Ptr{Cint},
Ptr{Ptr{Ptr{Uint8}}}), argc, argv)
return result
end
kl. 17:37:15 UTC+2 mandag 15. september 2014 skrev Ivar Nesje følgende:
That seems like a really tricky function to wrap. You need to do C style
memory handling in order for this to work, and C style memory handling
requires you to really look in the documentation (for the function) to see
how it works. Example code for how this function is used correctly from C
is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/master/src/OAuth.jl#L248-254
function
oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{Uint8}}})
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array
that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result =
ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} , but
that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
Re: [julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
oauth_split_url_parameters returns the number of parameters it finds. This
is a fairly common C idiom: caller passes a double or triple pointer, and
the function returns number of things it allocated.
On Mon, Sep 15, 2014 at 2:55 PM, Randy Zwitch randy.zwi...@fuqua.duke.edu
wrote:
Ivar, your code returned an error:
ERROR: type: ccall: in argument 2, expected Ptr{Ptr{Uint8}}, got
Ptr{Array{Uint8,1}}
in oauth_split_url_parameters at none:3
David, your code worked for me, but it seems that you need to know how
many parameters are in the string. Is there a way to just return as an
array?
On Monday, September 15, 2014 2:27:57 PM UTC-4, Randy Zwitch wrote:
No problem, I'll test it out...that's what VM's are for :)
On Monday, September 15, 2014 2:22:31 PM UTC-4, Ivar Nesje wrote:
I should probably not post this untested code, but it is way too
tempting when I need to install a library to actually test it.
function oauth_split_url_parameters(url::String)
argv = Ptr{Array{Uint8,1}}[C_NULL]
argc = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
result = Array(UTF8String, argc)
for i = 1:argc
result[i] = bytestring(argv[1][i])
end
ccall((:oath_free_array,LIBOAUTH), Nothing, (Ptr{Cint},
Ptr{Ptr{Ptr{Uint8}}}), argc, argv)
return result
end
kl. 17:37:15 UTC+2 mandag 15. september 2014 skrev Ivar Nesje følgende:
That seems like a really tricky function to wrap. You need to do C
style memory handling in order for this to work, and C style memory
handling requires you to really look in the documentation (for the
function) to see how it works. Example code for how this function is used
correctly from C is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch
følgende:
Continuing on my attempt to learn Julia ccall by wrapping liboauth...I
have the following code that was mostly generated by Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/
master/src/OAuth.jl#L248-254
function oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{
Uint8}}})
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an array.
I don't want to return the Cint for success/fail, but rather the array
that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes here
to make this work?
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} ,
but that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] More with embedding with dynamic libraries (Linux)
Hi all, Remember this problem with dlopen and openblas https://groups.google.com/forum/#!topic/julia-users/A8HwlmldVTM? That problem was resolved by modifying how embedded Julia loads (perhaps only on OS/X).Well now there is a similar problem on Linux. Same software. The exact problem is different because the dynamic linkers are different. It appears to me from LD_DEBUG that symbols in libjulia.so are not being resolved correctly when sys.so is initialized. Much of the functionality is missing, but for example tuple() still works... The relevant debug information is: 18788: find library=libjulia.so [0]; searching 18788: search path=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64: /usr/local/julia-aa5ffc6ac6/lib/julia/tls:/usr/local/julia-aa5ffc6ac6/lib/ julia/x86_64:/usr/local/julia-aa5ffc6ac6/lib/julia (RPATH from file /home/jeffw/src/nj-test2/node_modules/node-julia/build/Release/nj.node) 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/tls/libjulia. so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/x86_64/ libjulia.so 18788: trying file=/usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Followed by: 18788: calling init: /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so Which returns with no errors Then when calling init on my .so, the following happens 18788: /usr/local/julia-aa5ffc6ac6/lib/../lib/julia/sys.so: error: symbol lookup error: undefined symbol: jl_diverror_exception (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_ast_rettype ( fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_prepare_ast (fatal) 18788: node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) 18788:node: error: symbol lookup error: undefined symbol: jl_alloc_array_1d (fatal) followed by about 50 more unresolved symbols... But those symbols are in libjulia.so, The effect of this is later on is the following exception Error: Julia ccall: could not find function jl_prepare_ast Here's the output of ldd of the .so library . In linux @rpath is $ORIGIN, and I don't see that here, so maybe that's the culprit, or maybe a red herring linux-vdso.so.1 = (0x7fff765d9000) libjulia.so = /usr/local/julia-aa5ffc6ac6/lib/julia/libjulia.so ( 0x7f89c4cd1000) libstdc++.so.6 = /usr/lib/x86_64-linux-gnu/libstdc++.so.6 ( 0x7f89c49c3000) libgcc_s.so.1 = /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x7f89c47ac000) libpthread.so.0 = /lib/x86_64-linux-gnu/libpthread.so.0 (0x7f89c458f000 ) libc.so.6 = /lib/x86_64-linux-gnu/libc.so.6 (0x7f89c41cf000) librt.so.1 = /lib/x86_64-linux-gnu/librt.so.1 (0x7f89c3fc6000) libdl.so.2 = /lib/x86_64-linux-gnu/libdl.so.2 (0x7f89c3dc2000) libz.so.1 = /lib/x86_64-linux-gnu/libz.so.1 (0x7f89c3bab000) libm.so.6 = /lib/x86_64-linux-gnu/libm.so.6 (0x7f89c38ae000) /lib64/ld-linux-x86-64.so.2 (0x7f89c5fb1000)
Re: [julia-users] Re: How to allocate Ptr{Ptr{Ptr{Uint8}}} with ccall?
Perfect, thank you, I see it now (David's code 'result' is 2). So I can
loop over unsafe_load and add to an array if I chose.
Though, likely I won't. This library is starting to make me believe I could
more quickly re-write these functions in Julia once I know what it is they
do!
On Monday, September 15, 2014 3:14:52 PM UTC-4, Isaiah wrote:
oauth_split_url_parameters returns the number of parameters it finds. This
is a fairly common C idiom: caller passes a double or triple pointer, and
the function returns number of things it allocated.
On Mon, Sep 15, 2014 at 2:55 PM, Randy Zwitch randy@fuqua.duke.edu
javascript: wrote:
Ivar, your code returned an error:
ERROR: type: ccall: in argument 2, expected Ptr{Ptr{Uint8}}, got
Ptr{Array{Uint8,1}}
in oauth_split_url_parameters at none:3
David, your code worked for me, but it seems that you need to know how
many parameters are in the string. Is there a way to just return as an
array?
On Monday, September 15, 2014 2:27:57 PM UTC-4, Randy Zwitch wrote:
No problem, I'll test it out...that's what VM's are for :)
On Monday, September 15, 2014 2:22:31 PM UTC-4, Ivar Nesje wrote:
I should probably not post this untested code, but it is way too
tempting when I need to install a library to actually test it.
function oauth_split_url_parameters(url::String)
argv = Ptr{Array{Uint8,1}}[C_NULL]
argc = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
result = Array(UTF8String, argc)
for i = 1:argc
result[i] = bytestring(argv[1][i])
end
ccall((:oath_free_array,LIBOAUTH), Nothing, (Ptr{Cint},
Ptr{Ptr{Ptr{Uint8}}}), argc, argv)
return result
end
kl. 17:37:15 UTC+2 mandag 15. september 2014 skrev Ivar Nesje følgende:
That seems like a really tricky function to wrap. You need to do C
style memory handling in order for this to work, and C style memory
handling requires you to really look in the documentation (for the
function) to see how it works. Example code for how this function is used
correctly from C is almost mandatory.
I found this example
http://liboauth.sourceforge.net/tests_2oauthexample_8c-example.html,
apparently from the OAuth site, where they demonstrate how to do a poor
job
and leak memory. DON'T USE IT!!!
I'll try to write a correct implemementation that uses
oauth_free_array
http://liboauth.sourceforge.net/oauth_8h.html#ada81dd39be59cadbc1bd77df78ffef52(int*
c, char**arg) to clean up.
Ivar
kl. 16:47:44 UTC+2 mandag 15. september 2014 skrev Randy Zwitch
følgende:
Continuing on my attempt to learn Julia ccall by wrapping
liboauth...I have the following code that was mostly generated by
Clang.jl:
https://github.com/randyzwitch/OAuth.jl/blob/
master/src/OAuth.jl#L248-254
function oauth_split_url_parameters(url::String,argv::Ptr{Ptr{Ptr{
Uint8}}})
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return result
end
I'm pretty sure I want to re-write this code to remove the second
argument, since the second argument is just allocating space for an
array.
I don't want to return the Cint for success/fail, but rather the array
that
splits the 'url' argument.
function oauth_split_url_parameters(url::String)
argv = *Ptr{Ptr{Ptr{Uint8}}}* #What Julia code actually goes
here to make this work?
result = ccall((:oauth_split_url_parameters,LIBOAUTH),Cint,(
Ptr{Uint8},Ptr{Ptr{Ptr{Uint8}}}),url,argv)
if result == C_NULL
error(oauth_split_url_parameters failed)
end
return argv
end
I've tried allocating an array() in place of Ptr{Ptr{Ptr{Uint8}}} ,
but that doesn't work. I think this is an array of pointers to string
variables? How can I make this work?
Thanks.
[julia-users] Re: neither sub syntax works
I found it at the bottom of this page
http://juliastats.github.io/DataFrames.jl/subsets.html
This seems like a big feature not to support. What I am looking for is the
equivalent of where() in pandas
On Sunday, September 14, 2014 4:12:25 PM UTC-7, Arshak Navruzyan wrote:
I can't get either subset syntax working for some reason. I tried to
Pkg.installed(DataFrames)
v0.5.7
df = DataFrame(A = 1:4, B = [M, F, F, M])
sub(df,:(A .== M))
`getindex` has no method matching getindex(::SimpleIndex, ::Expr)
while loading In[72], in expression starting on line 13
in sub at
/Users/arshakn/.julia/v0.3/DataFrames/src/subdataframe/subdataframe.jl:97
df[:(A % 2 .== 0),:]
`getindex` has no method matching getindex(::DataFrame, ::Expr,
::UnitRange{Int64})
while loading In[73], in expression starting on line 15
Re: [julia-users] Re: neither sub syntax works
Le lundi 15 septembre 2014 à 12:49 -0700, Arshak Navruzyan a écrit : I found it at the bottom of this page http://juliastats.github.io/DataFrames.jl/subsets.html This seems like a big feature not to support. What I am looking for is the equivalent of where() in pandas For now, you can try using the DataFrameMeta package: https://github.com/JuliaStats/DataFramesMeta.jl Regards
Re: [julia-users] Re: Control system library for Julia?
Hi Tony! I just skimmed your web page; looks like you're doing some interesting stuff with NMPC in matlab/Simulink with your group's BLOM library: http://www.mpc.berkeley.edu/software/blom so it looks like you're the right guy to be working on something similar for Julia :) Off topic: Since you're also at the intersection of programming language and optimal control nerd-dom, you may also be interested in work Greg Horn is doing here in Moritz Diehl's group; he has written a Haskell wraper around casADi (developed mainly by Joel and Joris, two other guys in our group), which in turn wraps a handful of solvers, i.e. ipopt. https://github.com/ghorn/dynobud Cheers, Andrew On Mon, Sep 15, 2014 at 2:26 PM, Tony Kelman t...@kelman.net wrote: I didn't really mean by way of compiler flags or language pragmas. I can try -fno-rtti or -fno-exceptions with g++ today, it doesn't do me much good to spend a week getting half a dozen dependency libraries built with the obscure proprietary embedded board compiler only to find the very last thing I need won't work. The type of control I do is pretty sophisticated - optimization-based, nonlinear, large-scale MIMO systems, incorporating a prediction model of the system and satisfying state and input constraints - takes a bunch of libraries to do properly, not the kind of thing that's easy to throw together from scratch in C. It would be nice to get guarantees by construction that my controller and all the code it uses would be deployable to an embedded environment from the very beginning. Luckily microcontrollers are getting much more powerful these days and the line is starting to blur between embedded RTOS and lightly tweaked Linux. You can fit lots more code and do more interesting things with the same power/thermal footprint, getting lots more flops and IO in soft-but-pretty-damn-good realtime at the same or better speeds. Though what this thread was originally about was tools for doing offline control design, where Matlab/Simulink absolutely dominate. You don't want to do simulation, analysis, testing, and tuning in the exact same low-level restricted language that the realtime controller gets deployed to, the requirements are completely different. On Monday, September 15, 2014 3:13:32 AM UTC-7, Andrew Wagner wrote: Another quick note, Tony: There are plenty of embedded environments where if you're allowed to use C++ at all, it's only a limited subset, anything involving exceptions or RTTI is often not supported. It's tough to audit 3rd-party libraries for these kinds of restrictions, and achieve any kind of code reuse or non-trivial complexity. At least the auditing for restrictions part of that problem ~seems like it should be easy in a well designed language. I want to just declare an environment and have the compiler throw errors if anything inside it violates the restrictions I care about for whatever my personal definition of what a realtime or typesafe or pure functional language subset is. Something along the lines of: #[deny(managed_heap_memory)]: https://github.com/rust-lang/rust/issues/9984 Of course as you point out there is the other part of the problem achieving code reuse and complexity, which is much harder.
[julia-users] Re: Module Includes Problem
If dirlist.jl declares a module, you might have to use the full name `YourModule.dirlist`. Another potential gotcha is that require() only will load the file once, and when you're inside a module definition you might want to include() the file so that it will be loaded multiple times in the same session, or define a package you can be `using` instead.
Re: [julia-users] How to manage local, proprietary packages
Thanks for the report. We need someone to actually run Pkg in this mode to ensure it works. Resetting the branch is an issue worth reporting on github. I guess it was done when the METADATA format changed for 0.2, and users had to be moved to the new branch with the new format.
[julia-users] Article on `@simd`
I've posted an article on the @simd feature to https://software.intel.com/en-us/articles/vectorization-in-julia . @simd is an experimental feature http://julia.readthedocs.org/en/release-0.3/manual/performance-tips/#performance-annotations in Julia 0.3 that gives the compiler more latitude to .vectorize loops. Corrections/suggestions appreciated. - Arch D. Robison Intel Corporation
Re: [julia-users] Re: dispatch based on expression head
Thanks, that's an extremely helpful explanation. The VideoIO.jl example is great. On Sunday, September 14, 2014 10:54:23 AM UTC-5, Kevin Squire wrote: Sorry, I got sidetracked with the rest of the discussion and forgot to come back to this until now. On Fri, Sep 12, 2014 at 5:48 PM, Gray Calhoun gcal...@iastate.edu javascript: wrote: Thanks, that's pretty cool. For simple cases like I'm using, do you know if there are advantages (or disadvantages) to using Match.jl, or should I just view it as a nicer syntax? (Obviously, when things get more complicated Match.jl looks very appealing). [...] I used Match.jl quite extensively to match and rewrite Exprs when wrapping VideoIO. A clean Expr match example is https://github.com/kmsquire/VideoIO.jl/blob/master/util/wrap_libav_split.jl#L215-L226. You can search in the rest of that file for examples of Expr rewriting.
Re: [julia-users] Does Julia have something similar to Python's documentation string?
Hi Leah, thanks for the explanation. That makes a lot of sense. On Monday, September 15, 2014 12:48:37 PM UTC-5, Leah Hanson wrote: The @doc macro lets you do things that the doc_str can't: 1) Attach to the following method/function/etc. The string just sits there; the macro can do the work to put the string into the documentation. (the doc_str wouldn't be able to see the context around the string it's parsing) 2) Add a metadata dictionary after the doc string. 3) Allow other formats of documentation string (rst, asciidoc, whatever) as long as the implement some interface of functions (likely some writemime methods). Something like `@doc rst ...rst formatted text`, where using `doc text` would remove the possibility of format tagging via rst_str. -- Leah On Mon, Sep 15, 2014 at 12:37 PM, Gray Calhoun gcal...@iastate.edu javascript: wrote: I should add that I'm excited to try out the package as is and successfully document my functions. On Monday, September 15, 2014 12:36:03 PM UTC-5, Gray Calhoun wrote: Just to engage in some bikeshedding is @doc better than defining doc_str or d_str? The triple quote notation seems like an unnecessary pythonism. doc_str gives: doc Markdown formatted text goes here... - function myfunc(x, y) x + y end On Monday, September 15, 2014 10:02:49 AM UTC-5, Michael Hatherly wrote: *Readability of @doc:* I think that this probably just comes down to personal preference for me - I’ve not done an extensive comparison between different syntax. @doc introduces a docstring and seems pretty straightforward to me. It explicitly states that what follows is documentation. That example from Docile.jl could probably do with some simplifications since that metadata section looks terrible if I’m honest. Something like the following might be better as an initial example: module PackageName using Docile @docstrings # must appear before any `@doc` calls @doc Markdown formatted text goes here... - function myfunc(x, y) x + y end end And then leave introducing metadata until after this since I’ve found metadata to not be needed for every docstring I write. I’m not sure about the “clearly visible bounded block” though, what in particular could be clearer? I’m asking since I’ve been staring at these for a while now and have become quite accustomed to them.
Re: [julia-users] How to manage local, proprietary packages
OK, I opened the issue regarding `Pkg.update()` always resetting to the default branch: #8370 https://github.com/JuliaLang/julia/issues/8370 On Monday, September 15, 2014 1:50:33 PM UTC-7, Ivar Nesje wrote: Thanks for the report. We need someone to actually run Pkg in this mode to ensure it works. Resetting the branch is an issue worth reporting on github. I guess it was done when the METADATA format changed for 0.2, and users had to be moved to the new branch with the new format.
Re: [julia-users] Article on `@simd`
Hey Arch, Looking at the second code blob, on line 08 you have t4 = b+c, but I think you mean t4 = t2+t3. -E On Mon, Sep 15, 2014 at 2:39 PM, Arch Robison arch.d.robi...@gmail.com wrote: I've posted an article on the @simd feature to https://software.intel.com/en-us/articles/vectorization-in-julia . @simd is an experimental feature http://julia.readthedocs.org/en/release-0.3/manual/performance-tips/#performance-annotations in Julia 0.3 that gives the compiler more latitude to .vectorize loops. Corrections/suggestions appreciated. - Arch D. Robison Intel Corporation
Re: [julia-users] Article on `@simd`
It also looks as if there is some HTML that sneaked into your code. On line 11 of the fourth code blob:strongend/strong -E On Mon, Sep 15, 2014 at 3:06 PM, Elliot Saba staticfl...@gmail.com wrote: Hey Arch, Looking at the second code blob, on line 08 you have t4 = b+c, but I think you mean t4 = t2+t3. -E On Mon, Sep 15, 2014 at 2:39 PM, Arch Robison arch.d.robi...@gmail.com wrote: I've posted an article on the @simd feature to https://software.intel.com/en-us/articles/vectorization-in-julia . @simd is an experimental feature http://julia.readthedocs.org/en/release-0.3/manual/performance-tips/#performance-annotations in Julia 0.3 that gives the compiler more latitude to .vectorize loops. Corrections/suggestions appreciated. - Arch D. Robison Intel Corporation
Re: [julia-users] slow julia version of c code
It sounds like you've looked for a type problem and not found one. One last thing to check is that there's no more memory allocation than you'd expect given the size of your temporaries (which seem to be Aold, dA, and w). You can use `julia --track-allocation=user testfile.jl` to see if there are any other lines allocating memory. Assuming the answer is no, you can be pretty confident that you don't have a type problem. Then I'd ask, have you also checked that the two implementations are actually doing the same amount of work? It wouldn't be the first time that one implementation was doing more iterations (or something) than another. For most uses, Julia really does give performance within a small factor of C, so you're right to suspect that there must be something wrong. --Tim On Monday, September 15, 2014 02:16:36 AM Zac wrote: https://gist.github.com/Zac12345/519bd7a503a1fd1b8d98 has the updated function and code_typed output Pkg.clone(https://github.com/Zac12345/Sparse;) should be fine for the test as the gist does'nt require functions using the shared lib. Are there any specific things i should be looking for in the code_typed results?
Re: [julia-users] Article on `@simd`
- Under the first bullet point of What is Vectorization?, Writing you code should be Writing your code - In the Speedup Surprise section, the last sentence says The compiler than..., should be The compiler then - In the section Inspecting Whether Code Vectorizes, you can actually use the pattern @code_llvm axpy(1.414f0, x, y) (using the original example/function call). This gives the LLVM output of the outermost function call and is quite handy vs. dealing with the tuple of argument types (as is noted in your article). - In the section Trip Count must be Obvious, down to the bullet points, I think your second bullet point should be - *first*(*r*) returns the first index value. - In the section The Loop Body Should be Straight Line Code, there is an extra period (.) after *type-stable code* - In the section Subscripts should be Unit Stride, it says When working with nested loops on two-dimensional arrays, use *@simd* on the inner loop and make that loop index the column index of the arrays. But then the example has `i` as the inner loop variable and the matrices are indexed like A[i, j], which seems like `j` is column-index. Am I misinterpreting this? Loved the article! For those of us unable to make it to JuliaCon, it's been awesome to be able to watch the videos and read up on articles like this. Thanks for all the work! -Jacob On Mon, Sep 15, 2014 at 6:26 PM, Patrick O'Leary patrick.ole...@gmail.com wrote: Under Inspecting Whether Code Vectorizes code_llvm(axpy,(T1,T2,T2}) The next-to-last character should be a paren. This is a very informative article; thanks for putting it and the feature together!
Re: [julia-users] Article on `@simd`
Thanks to all for taking time to report my errors. I missed @code_llvm's introduction into Julia. It let me shorten that section slightly. Jacob is right about the row/column index. I tend to think of the leftmost subscript as the runs down the column index. When I'm working with Fortran/Julia, I'm constantly thinking runs down the column; runs down the column to override my C++ habit. *B*ut I mistakenly phrased it as column index. I changed the text to say leftmost subscript. I think that I've fixed all the bugs reported up to the time this note is posted. On Mon, Sep 15, 2014 at 5:55 PM, Jacob Quinn quinn.jac...@gmail.com wrote: - Under the first bullet point of What is Vectorization?, Writing you code should be Writing your code - In the Speedup Surprise section, the last sentence says The compiler than..., should be The compiler then - In the section Inspecting Whether Code Vectorizes, you can actually use the pattern @code_llvm axpy(1.414f0, x, y) (using the original example/function call). This gives the LLVM output of the outermost function call and is quite handy vs. dealing with the tuple of argument types (as is noted in your article). - In the section Trip Count must be Obvious, down to the bullet points, I think your second bullet point should be - *first*(*r*) returns the first index value. - In the section The Loop Body Should be Straight Line Code, there is an extra period (.) after *type-stable code* - In the section Subscripts should be Unit Stride, it says When working with nested loops on two-dimensional arrays, use *@simd* on the inner loop and make that loop index the column index of the arrays. But then the example has `i` as the inner loop variable and the matrices are indexed like A[i, j], which seems like `j` is column-index. Am I misinterpreting this? Loved the article! For those of us unable to make it to JuliaCon, it's been awesome to be able to watch the videos and read up on articles like this. Thanks for all the work! -Jacob On Mon, Sep 15, 2014 at 6:26 PM, Patrick O'Leary patrick.ole...@gmail.com wrote: Under Inspecting Whether Code Vectorizes code_llvm(axpy,(T1,T2,T2}) The next-to-last character should be a paren. This is a very informative article; thanks for putting it and the feature together!
Re: [julia-users] Re: C Global Structs
This works excellent, thank you both!
Do you think this would be useful to have these user-facing in Base?
On Mon, Sep 15, 2014 at 12:46 PM, Jake Bolewski jakebolew...@gmail.com
wrote:
julia immutable CompilerOpts
build_path::Ptr{Cchar}
code_coverage::Int8
malloc_log::Int8
check_bounds::Int8
dumpbitcode::Int8
int_literals::Cint
compile_enabled::Int8
end
julia a = cglobal(:jl_compileropts, CompilerOpts)
Ptr{CompilerOpts} @0x00010f840a88
julia unsafe_load(a)
CompilerOpts(Ptr{Int8} @0x,0,0,0,0,0,1)
On Monday, September 15, 2014 12:30:48 PM UTC-4, Steve Kelly wrote:
I'd like to see if Julia is running in code-coverage mode or not.
In the REPL I can do the following:
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x7fa3e01bbc90
The struct is defined like so:
https://github.com/JuliaLang/julia/blob/aab2c6e67b5aaee7f23bc5a52897f7
219473c153/src/julia.h#L1330-L1338
How can I access the code_coverage member?
Thanks,
Steve
Re: [julia-users] Re: C Global Structs
I'm not sure; what do you want to do with this? It seems kind of devious to
me to change the runtime behavior based on whether coverage is enabled :)
On Mon, Sep 15, 2014 at 9:30 PM, Steve Kelly kd2...@gmail.com wrote:
This works excellent, thank you both!
Do you think this would be useful to have these user-facing in Base?
On Mon, Sep 15, 2014 at 12:46 PM, Jake Bolewski jakebolew...@gmail.com
wrote:
julia immutable CompilerOpts
build_path::Ptr{Cchar}
code_coverage::Int8
malloc_log::Int8
check_bounds::Int8
dumpbitcode::Int8
int_literals::Cint
compile_enabled::Int8
end
julia a = cglobal(:jl_compileropts, CompilerOpts)
Ptr{CompilerOpts} @0x00010f840a88
julia unsafe_load(a)
CompilerOpts(Ptr{Int8} @0x,0,0,0,0,0,1)
On Monday, September 15, 2014 12:30:48 PM UTC-4, Steve Kelly wrote:
I'd like to see if Julia is running in code-coverage mode or not.
In the REPL I can do the following:
julia a = cglobal(:(jl_compileropts))
Ptr{Void} @0x7fa3e01bbc90
The struct is defined like so:
https://github.com/JuliaLang/julia/blob/aab2c6e67b5aaee7f23bc5a52897f7
219473c153/src/julia.h#L1330-L1338
How can I access the code_coverage member?
Thanks,
Steve
Re: [julia-users] Re: Control system library for Julia?
On Monday, September 15, 2014 9:56:23 AM UTC-5, Uwe Fechner wrote: more informations on kite control you can find in the following papers: Fascinating! Thanks for sharing.
Re: [julia-users] slow julia version of c code
There is most certainly a type problem. You're not getting type information
for sparse_grid.lvl_l which deoptimizes a lot of things. In your code, you
have:
type sparse_gridd::Int64q::Int64n::Int64grid
::Array{Float64}ind::Array{Int64}lvl::Array{Int64}
lvl_l::Array{Int64}lvl_s::Array{Float64}bounds::Array{
Float64}gtypeBBfubend
Try specifying the number of dimensions for the arrays, e.g.
grid::Array{Float64,2}. Also using any of the untyped fields will slow code
down, but I don't see them in that function.
A hackish way to find type issues is:
filter!(x-!isleaftype(x[2]), @code_typed(f(args))[1].args[2][2])
which returns a list of variables where type inference couldn't infer a
concrete/leaf type. In some cases these will be hidden variables so you
may still need to inspect the output of code_typed. TypeCheck.jl and
Lint.jl may also provide more sophisticated tools for this purpose.
Simon
On Monday, September 15, 2014 5:16:36 AM UTC-4, Zac wrote:
https://gist.github.com/Zac12345/519bd7a503a1fd1b8d98 has the updated
function and code_typed output
Pkg.clone(https://github.com/Zac12345/Sparse;) should be fine for the
test as the gist does'nt require functions using the shared lib.
Are there any specific things i should be looking for in the code_typed
results?
Re: [julia-users] Article on `@simd`
Great article! I used some of your tips on a complicated inner loop just now and immediately got a 3.6x speedup. On Monday, September 15, 2014 7:57:17 PM UTC-5, Arch Robison wrote: Thanks to all for taking time to report my errors. I missed @code_llvm's introduction into Julia. It let me shorten that section slightly. Jacob is right about the row/column index. I tend to think of the leftmost subscript as the runs down the column index. When I'm working with Fortran/Julia, I'm constantly thinking runs down the column; runs down the column to override my C++ habit. *B*ut I mistakenly phrased it as column index. I changed the text to say leftmost subscript. I think that I've fixed all the bugs reported up to the time this note is posted. On Mon, Sep 15, 2014 at 5:55 PM, Jacob Quinn quinn@gmail.com javascript: wrote: - Under the first bullet point of What is Vectorization?, Writing you code should be Writing your code - In the Speedup Surprise section, the last sentence says The compiler than..., should be The compiler then - In the section Inspecting Whether Code Vectorizes, you can actually use the pattern @code_llvm axpy(1.414f0, x, y) (using the original example/function call). This gives the LLVM output of the outermost function call and is quite handy vs. dealing with the tuple of argument types (as is noted in your article). - In the section Trip Count must be Obvious, down to the bullet points, I think your second bullet point should be - *first*(*r*) returns the first index value. - In the section The Loop Body Should be Straight Line Code, there is an extra period (.) after *type-stable code* - In the section Subscripts should be Unit Stride, it says When working with nested loops on two-dimensional arrays, use *@simd* on the inner loop and make that loop index the column index of the arrays. But then the example has `i` as the inner loop variable and the matrices are indexed like A[i, j], which seems like `j` is column-index. Am I misinterpreting this? Loved the article! For those of us unable to make it to JuliaCon, it's been awesome to be able to watch the videos and read up on articles like this. Thanks for all the work! -Jacob On Mon, Sep 15, 2014 at 6:26 PM, Patrick O'Leary patrick...@gmail.com javascript: wrote: Under Inspecting Whether Code Vectorizes code_llvm(axpy,(T1,T2,T2}) The next-to-last character should be a paren. This is a very informative article; thanks for putting it and the feature together!
Re: [julia-users] Article on `@simd`
Excellent article. When you describe how sets of vector instructions occur, simultaneously would seem be more correct than instantaneously. Stylistically, the concerns paragraph seems a bit overly concerned – perhaps one could be changed to a different word. On Sep 15, 2014, at 11:39 PM, Arch Robison arch.d.robi...@gmail.com wrote: I've posted an article on the @simd feature to https://software.intel.com/en-us/articles/vectorization-in-julia . @simd is an experimental feature in Julia 0.3 that gives the compiler more latitude to .vectorize loops. Corrections/suggestions appreciated. - Arch D. Robison Intel Corporation
