Re: [julia-users] Base.Threads user feedback: it's great. please keep it. anyone tried hybrid (addprocs + Threads)?

[Florian Oswald]

ok, thanks for that. do you think this is going to change already in v0.6
or will that have to wait until a future release?


On 14 November 2016 at 23:59, Yichao Yu <yyc1...@gmail.com> wrote:

> On Mon, Nov 14, 2016 at 5:08 PM, Florian Oswald
> <florian.osw...@gmail.com> wrote:
> > I'm not sure how many people are using Base.Threads out there, I came
> across
> > it by accident and think it works great. It's under the heading
> > "experimental" in the manual, so I just wanted to encourage the
> developers
> > that this is a great feature, please don't drop it. I just wrote
> @threads in
> > front of a loop in my code and time was cut by pretty much exactly
> > JULIA_NUM_THREADS.
>
> It's experimental in that apart from some really simple cases, code
> that uses thread can easily crash.
> It is not recommended to use threading at this stage.
>
> >
> > Along those lines, has anyone tried to run a hybrid job, i.e. connecting
> > several machines via `addprocs` and running several threads on each of
> those
> > machines? Is something like that even possible and/or do you recommend
> > something like that? There is not too much in the manual, so I would just
> > like to get some more info.
>
> Doing so is certainly possible but is not recommended since threading
> itself isn't.
>
> >
> > thanks.
> >
> >
>

[julia-users] Base.Threads user feedback: it's great. please keep it. anyone tried hybrid (addprocs + Threads)?

[Florian Oswald]

I'm not sure how many people are using Base.Threads out there, I came 
across it by accident and think it works great. It's under the heading 
"experimental" in the manual, so I just wanted to encourage the developers 
that this is a great feature, please don't drop it. I just wrote @threads 
in front of a loop in my code and time was cut by pretty much exactly 
JULIA_NUM_THREADS. 

Along those lines, has anyone tried to run a hybrid job, i.e. connecting 
several machines via `addprocs` and running several threads on each of 
those machines? Is something like that even possible and/or do you 
recommend something like that? There is not too much in the manual, so I 
would just like to get some more info.

thanks.

Re: [julia-users] Unit Testing: should I use Base.Test instead of FactCheck?

[Florian Oswald]

Awesome. Thanks!


On Monday, 31 October 2016, Spencer Russell <s...@media.mit.edu> wrote:

> I generally have pretty granular `@testset`s, which can have messages
> attached that are printed on failure. That way I still get a description of
> what failed.
>
> I think the lack of indentation for nested testsets is a regression that
> hit 0.5 (BaseTestNext on 0.4 indents nested testsets), and there’s
> currently an issue filed: https://github.com/JuliaLang/julia/issues/18611
>
> so they should get re-prettified soon enough.
>
> -s
>
>
> On Fri, Oct 28, 2016, at 10:30 AM, Florian Oswald wrote:
>
> I know there are many testing packages out there. I have been using
> FactCheck, but have seen it's been relegated to the JuliaArchive. I take
> that as a sign it's a good moment to use something else.
>
> There is 1 thing I sorely miss from Base.Test coming from FactCheck, and
> that is the ability to say
>
> @fact false --> true "your test failed: false is not true"
>
> i.e. i can print a custom error message to be shown upon failure only.
>
> I also think the display of test results (indentation of lower level
> "contexts" (@testsets in Base.Test)) is nicer in FactCheck, but that's not
> that important.
>
> Thanks for any advice.
>
>
>
>

Re: [julia-users] Unit Testing: should I use Base.Test instead of FactCheck?

[Florian Oswald]

D'oh.

Thanks!

On Saturday, 29 October 2016, Steven G. Johnson <stevenj@gmail.com>
wrote:

>
>
> On Friday, October 28, 2016 at 10:30:28 AM UTC-4, Florian Oswald wrote:
>>
>> There is 1 thing I sorely miss from Base.Test coming from FactCheck, and
>> that is the ability to say
>>
>> @fact false --> true "your test failed: false is not true"
>>
>
> @test false || error("your test failed: false is not true.")
>

[julia-users] Gallium: how to remove a breakpoint?

[Florian Oswald]

hi all,

i have got a rather simple question but can't figure it out:

https://github.com/Keno/Gallium.jl/issues/166

thanks

[julia-users] Re: [ANNOUNCE] TestSetExtensions.jl

[Florian Oswald]

Good stuff! I like the idea of extending Base.Test

anyway you could easily integrate printing messages after failed tests?

https://groups.google.com/d/msg/julia-users/_cZ8y_-JAVA/8EcTWdxcAQAJ

cheers

On Thursday, 27 October 2016 17:27:38 UTC+2, Spencer Russell wrote:
>
> Hey All,
>
> I just registered the TestSetExtensions package, which collects some 
> extensions and convenience utilities to maximize your testing enjoyment. It 
> builds on the new Base.Test infrastructure in Julia v0.5 (also available in 
> v0.4 with the BaseTestNext package). It's designed so that you shouldn't 
> need to modify your tests at all if you're already 
> using @testset and @test - all the interactions with this package happen at 
> the top-level of your tests.
>
> https://github.com/ssfrr/TestSetExtensions.jl
>
> Right now is supports two features:
>
>
>1. Printing out green dots as your tests run so you can see them 
>progressing
>2. Collecting your test files and running them automatically so you 
>don’t have to manually add them to `runtests.jl`, and also so you can 
>easily specify a sub-set of your tests to run from the command line
>
>
> I mostly wrote it to encapsulate the things I want for my tests, but I’m 
> happy to take PRs for new features.
>
> Please kick the tires and let me know if it works (or doesn’t work) for 
> you!
>
> ssfrr
>

[julia-users] Unit Testing: should I use Base.Test instead of FactCheck?

[Florian Oswald]

I know there are many testing packages out there. I have been using 
FactCheck, but have seen it's been relegated to the JuliaArchive. I take 
that as a sign it's a good moment to use something else.

There is 1 thing I sorely miss from Base.Test coming from FactCheck, and 
that is the ability to say

@fact false --> true "your test failed: false is not true"

i.e. i can print a custom error message to be shown upon failure only.

I also think the display of test results (indentation of lower level 
"contexts" (@testsets in Base.Test)) is nicer in FactCheck, but that's not 
that important.

Thanks for any advice.

Re: [julia-users] use macros like a preprocessor directive?

[Florian Oswald]

Ok. any suggestions how to get close to the C preprocessor behaviour though?

On Monday, 24 October 2016 11:13:18 UTC+2, Mauro wrote:
>
> On Mon, 2016-10-24 at 10:37, Florian Oswald <florian...@gmail.com 
> > wrote: 
> > I have something that this in C++ I would write like 
> > 
> > double f(double x){ 
> >  // do something with x 
> > #ifdef MACROVAR 
> >  // do something else with x 
> > #endif 
> >  return(x) 
> > } 
> > 
> > I was trying to understand how i could use julia macro's to achieve 
> > something similar. I have seen how the Logging.jl uses macros to this 
> end, 
> > at least that's how I understand 
> > it: 
> https://github.com/kmsquire/Logging.jl/blob/master/src/logging_macros.jl 
> > . However I'm a bit unclear as to how to use this in my case. in my 
> case, 
> > the condition of whether or not to insert a piece of code is stored in a 
> > custom type. Here is what I tried: 
> > 
> > # simplified type: 
> > type Param 
> > nD_rent :: Int 
> > end 
> > 
> > # macro: 
> > macro add_buy(p::Param) 
> > quote 
> > if $(esc(p.nD_rent)) > 4 
> > # insert some additional code 
> > println("here comes additional code") 
> > println(typeof(p)) 
> > 
> > else 
> > println(typeof(p)) 
> > :nothing 
> > end 
> > end 
> > end 
>
> No, a macro receives the AST.  Check this 
>
> macro tmp(p) 
> println(p) 
> println(typeof(p)) 
> end 
>
> So your signature should probably just read: 
>
> macro add_buy(p) 
> ... 
> end 
>
> > *julia> **bk.@add_buy(p)* 
> > 
> > *ERROR: MethodError: no method matching @add_buy(::Symbol)* 
> > 
> > Closest candidates are: 
> > 
> >   @add_buy(*::bk.Param*) at 
> > /Users/florian.oswald/git/bk/bk.jl/src/solver.jl:641 
> > 
> > 
> > so, I have a type `Param`, and depending on the value in field nD_rent, 
> I 
> > would like to insert that code or not. thanks for any suggestions. 
>
> That is not what your above macro does.  And that is not what macros 
> should do (ideally), they should be syntax transformations independent 
> of the values.  You can eval in a macro but that is *strongly* 
> discouraged. 
>

[julia-users] use macros like a preprocessor directive?

[Florian Oswald]

I have something that this in C++ I would write like

double f(double x){
 // do something with x
#ifdef MACROVAR
 // do something else with x
#endif
 return(x)
}   

I was trying to understand how i could use julia macro's to achieve 
something similar. I have seen how the Logging.jl uses macros to this end, 
at least that's how I understand 
it: https://github.com/kmsquire/Logging.jl/blob/master/src/logging_macros.jl 
. However I'm a bit unclear as to how to use this in my case. in my case, 
the condition of whether or not to insert a piece of code is stored in a 
custom type. Here is what I tried:

# simplified type:
type Param
nD_rent :: Int
end

# macro:
macro add_buy(p::Param)
quote
if $(esc(p.nD_rent)) > 4 
# insert some additional code
println("here comes additional code")
println(typeof(p))

else
println(typeof(p))
:nothing
end
end
end

*julia> **bk.@add_buy(p)*

*ERROR: MethodError: no method matching @add_buy(::Symbol)*

Closest candidates are:

  @add_buy(*::bk.Param*) at 
/Users/florian.oswald/git/bk/bk.jl/src/solver.jl:641


so, I have a type `Param`, and depending on the value in field nD_rent, I 
would like to insert that code or not. thanks for any suggestions.

[julia-users] Interpolations.jl: how to use the scale function?

[Florian Oswald]

Hi folks,

I have question on SO that needs some attention. thanks!

http://stackoverflow.com/questions/40045208/how-to-use-scale-in-interpolations-jl

Re: [julia-users] help with @generated function call please?

[Florian Oswald]

Yes I think you are right about that. Using a comprehension is just as good
for my case. But part of me would like to just finally understand that part
of Julia! Ok will have to wait for the next opportunity. :-)

Thanks anyway.
Florian

On Saturday, 15 October 2016, Erik Schnetter <schnet...@gmail.com> wrote:

> A generated function is only useful if you perform a non-trivial
> calculation based on the argument types. You don't do that here, so I
> wonder whether simply using the Cartesian indexing macros by themselves
> would be sufficient.
>
> Note also that you don't need to write `$N` in your code; using `N`
> directly has the same effect here.
>
> I'm not saying that generated functions should be avoided at all costs,
> but if it isn't necessary here you might as well skip the associated
> complications.
>
> -erik
>
> On Fri, Oct 14, 2016 at 11:51 AM, Florian Oswald <florian.osw...@gmail.com
> <javascript:_e(%7B%7D,'cvml','florian.osw...@gmail.com');>> wrote:
>
>> hi all,
>>
>> I want to evaluate a function at each index of an array. There is a N
>> dimensional function, and I want to map it onto an N-dimensional array:
>>
>> fpoly(x::Array{Real,5}) = x[1] + x[2]^2 + x[3] + x[4]^2 + x[5]
>>
>> want to do
>>
>> a = rand(2,2,2,2,2);
>> b = similar(a)
>>
>> for i1 in indices(a,1)
>> for i2 in indices(a,2)
>> ...
>> b[i1,i2,i3,i4,i5] = fpoly(a[i1,i2,i3,i4,i5])
>> end
>> end...
>>
>> I tried:
>> # actually want to do it inplace
>> @generated function set_poly!{T,N}(a::Array{T,N})
>> quote
>> @nloops $N i a begin
>> @nref $N a i = @ncall $N fpoly i->a[i]
>> end
>> end
>> end
>>
>> but that fails. I dont get further than:
>>
>> macroexpand(:(@nloops 3 j a begin
>> x = @ncall 3 fpoly i->a[j]
>> end))
>>
>> *quote  # cartesian.jl, line 62:*
>>
>> *for j_3 = indices(a,3) # cartesian.jl, line 63:*
>>
>> *nothing # cartesian.jl, line 64:*
>>
>> *begin  # cartesian.jl, line 62:*
>>
>> *for j_2 = indices(a,2) # cartesian.jl, line 63:*
>>
>> *nothing # cartesian.jl, line 64:*
>>
>> *begin  # cartesian.jl, line 62:*
>>
>> *for j_1 = indices(a,1) # cartesian.jl, line 63:*
>>
>> *nothing # cartesian.jl, line 64:*
>>
>> *begin  # REPL[145], line 2:*
>>
>> *x = fpoly(a[j],a[j],a[j])*
>>
>> *end # cartesian.jl, line 65:*
>>
>> *nothing*
>>
>> *end*
>>
>> *end # cartesian.jl, line 65:*
>>
>> *nothing*
>>
>> *end*
>>
>> *end # cartesian.jl, line 65:*
>>
>> *nothing*
>>
>> *end*
>>
>> *end*
>>
>>
>>
>> *which is a start but how can I get the LHS right the indices of a right?*
>>
>>
>>
>
>
> --
> Erik Schnetter <schnet...@gmail.com
> <javascript:_e(%7B%7D,'cvml','schnet...@gmail.com');>>
> http://www.perimeterinstitute.ca/personal/eschnetter/
>

[julia-users] help with @generated function call please?

[Florian Oswald]

hi all, 

I want to evaluate a function at each index of an array. There is a N 
dimensional function, and I want to map it onto an N-dimensional array:

fpoly(x::Array{Real,5}) = x[1] + x[2]^2 + x[3] + x[4]^2 + x[5] 

want to do 

a = rand(2,2,2,2,2);
b = similar(a)

for i1 in indices(a,1)
for i2 in indices(a,2)
...
b[i1,i2,i3,i4,i5] = fpoly(a[i1,i2,i3,i4,i5])
end
end...

I tried:
# actually want to do it inplace
@generated function set_poly!{T,N}(a::Array{T,N})
quote
@nloops $N i a begin
@nref $N a i = @ncall $N fpoly i->a[i]
end
end
end

but that fails. I dont get further than:

macroexpand(:(@nloops 3 j a begin
x = @ncall 3 fpoly i->a[j]
end))

*quote  # cartesian.jl, line 62:*

*for j_3 = indices(a,3) # cartesian.jl, line 63:*

*nothing # cartesian.jl, line 64:*

*begin  # cartesian.jl, line 62:*

*for j_2 = indices(a,2) # cartesian.jl, line 63:*

*nothing # cartesian.jl, line 64:*

*begin  # cartesian.jl, line 62:*

*for j_1 = indices(a,1) # cartesian.jl, line 63:*

*nothing # cartesian.jl, line 64:*

*begin  # REPL[145], line 2:*

*x = fpoly(a[j],a[j],a[j])*

*end # cartesian.jl, line 65:*

*nothing*

*end*

*end # cartesian.jl, line 65:*

*nothing*

*end*

*end # cartesian.jl, line 65:*

*nothing*

*end*

*end*



*which is a start but how can I get the LHS right the indices of a right?*

Re: [julia-users] How to debug julia v0.5 code? Debug.jl is down, how to use Gallium.jl

[Florian Oswald]

Oh! Sorry I didn't get that. Thanks!

On Thursday, 13 October 2016, Michele Zaffalon <michele.zaffa...@gmail.com>
wrote:

> From ASTInterpreter's README:
>
>- `stuff runs stuff in the current frame's context
>
>
> On Thu, Oct 13, 2016 at 5:47 PM, Florian Oswald <florian.osw...@gmail.com
> <javascript:_e(%7B%7D,'cvml','florian.osw...@gmail.com');>> wrote:
>
>> how can I print the value of a variable while in debug mode?
>>
>>
>>
>> On Thursday, 13 October 2016 17:19:12 UTC+2, Michele Zaffalon wrote:
>>>
>>> Aside from Gallium.jl, you should also check ASTInterpreter.jl
>>> <https://github.com/Keno/ASTInterpreter.jl>'s README which gives the
>>> list of commands. The documentation seems to be spread across the two
>>> packages.
>>>
>>>
>>> On Thu, Oct 13, 2016 at 5:15 PM, Florian Oswald <florian...@gmail.com>
>>> wrote:
>>>
>>>> hi,
>>>>
>>>> i usually used Debug.jl to debug code, which is great, but errors on
>>>> julia 0.5: https://github.com/toivoh/Debug.jl/issues/80
>>>>
>>>> I have no idea how to use Gallium.jl just from looking at the examples.
>>>> Anyone? thanks.
>>>>
>>>
>>>
>

Re: [julia-users] How to debug julia v0.5 code? Debug.jl is down, how to use Gallium.jl

[Florian Oswald]

how can I print the value of a variable while in debug mode?



On Thursday, 13 October 2016 17:19:12 UTC+2, Michele Zaffalon wrote:
>
> Aside from Gallium.jl, you should also check ASTInterpreter.jl 
> <https://github.com/Keno/ASTInterpreter.jl>'s README which gives the list 
> of commands. The documentation seems to be spread across the two packages.
>
>
> On Thu, Oct 13, 2016 at 5:15 PM, Florian Oswald <florian...@gmail.com 
> > wrote:
>
>> hi,
>>
>> i usually used Debug.jl to debug code, which is great, but errors on 
>> julia 0.5: https://github.com/toivoh/Debug.jl/issues/80
>>
>> I have no idea how to use Gallium.jl just from looking at the examples. 
>> Anyone? thanks.
>>
>
>

Re: [julia-users] How to debug julia v0.5 code? Debug.jl is down, how to use Gallium.jl

[Florian Oswald]

works pretty good actually!!

soem more examples on teh readme woulnd't hurt though i guess.

On 13 October 2016 at 17:18, Michele Zaffalon <michele.zaffa...@gmail.com>
wrote:

> Aside from Gallium.jl, you should also check ASTInterpreter.jl
> <https://github.com/Keno/ASTInterpreter.jl>'s README which gives the list
> of commands. The documentation seems to be spread across the two packages.
>
>
> On Thu, Oct 13, 2016 at 5:15 PM, Florian Oswald <florian.osw...@gmail.com>
> wrote:
>
>> hi,
>>
>> i usually used Debug.jl to debug code, which is great, but errors on
>> julia 0.5: https://github.com/toivoh/Debug.jl/issues/80
>>
>> I have no idea how to use Gallium.jl just from looking at the examples.
>> Anyone? thanks.
>>
>
>

[julia-users] How to debug julia v0.5 code? Debug.jl is down, how to use Gallium.jl

[Florian Oswald]

hi,

i usually used Debug.jl to debug code, which is great, but errors on julia 
0.5: https://github.com/toivoh/Debug.jl/issues/80

I have no idea how to use Gallium.jl just from looking at the examples. 
Anyone? thanks.

[julia-users] Re: why do we have Base.isless(a, ::NAtype) but not Base.isless(a, ::Nullable)?

[Florian Oswald]

ok i found the conversation here:

https://groups.google.com/d/topic/julia-users/W6yyV4i_W_k/discussion



On Thursday, 13 October 2016 15:45:23 UTC+2, Florian Oswald wrote:
>
> I mean, do I have to cycle through the array and basically clean it of 
> #NULL before findign the maximium or is there another way?
>
> On Thursday, 13 October 2016 15:42:02 UTC+2, Florian Oswald wrote:
>>
>> i'm trying to understand why we don't have something similar in terms of 
>> comparison for Nullable as we have for DataArrays NAtype (below). point me 
>> to the relevant github conversation, if any, is fine. 
>>
>> How would I implement methods to find the maximium of an 
>> Array{Nullable{Float64}}? like so?
>>
>> Base.isless(a::Any, x::Nullable{Float64}) = isnull(x) ? true : 
>> Base.isless(a,get(x))
>>
>>
>> ~/.julia/v0.5/DataArrays/src/operators.jl:502
>>
>> #
>> # Comparison operators
>> #
>>
>> Base.isequal(::NAtype, ::NAtype) = true
>> Base.isequal(::NAtype, b) = false
>> Base.isequal(a, ::NAtype) = false
>> Base.isless(::NAtype, ::NAtype) = false
>> Base.isless(::NAtype, b) = false
>> Base.isless(a, ::NAtype) = true
>>
>>

Re: [julia-users] Re: What's the status of SIMD instructions from a user's perspective in v0.5?

[Florian Oswald]

Hi Erik,

that's great thanks. I may have a hot inner loop where this could be very 
helpful. I'll have a closer look and come back with any questions later on 
if that's ok. 

cheers
florian

On Thursday, 13 October 2016 16:24:03 UTC+2, Erik Schnetter wrote:
>
> If you want to use the SIMD package, then you need to manually vectorized 
> the code. That is, all (most of) the local variables you're using will have 
> a SIMD `Vec` type. For convenience, your input and output arrays will 
> likely still hold scalar values, and the `vload` and vstore` functions 
> access scalar arrays, reading/writing SIMD vectors. The function you quote 
> above (from the SIMD examples) does just this.
>
> What vector length `N` is best depends on the particular machine. Usually, 
> you would look at the CPU instruction set and choose the largest SIMD 
> vector size that the CPU supports, but sometimes twice that size or half 
> that size might also work well. Note that using a larger SIMD vector size 
> roughly corresponds to loop unrolling, which might be beneficial if the 
> compiler isn't clever enough to do this automatically.
>
> There's additional complication if the array size is not a multiple of the 
> vector size. In this case, extending the array via dummy elements if often 
> the easiest way to go.
>
> Note that SIMD vectorization is purely a performance improvement. It does 
> not make sense to make such changes without measuring performance before 
> and after. Given the low-level nature if the changes, looking at the 
> generated assembler code via `@code_native` is usually also insightful.
>
> I'll be happy to help if you have a specific problem on which you're 
> working.
>
> -erik
>
>
> On Thu, Oct 13, 2016 at 9:51 AM, Florian Oswald <florian...@gmail.com 
> > wrote:
>
>> ok thanks! and so I should define my SIMD-able function like
>>
>> function vadd!{N,T}(xs::Vector{T}, ys::Vector{T}, ::Type{Vec{N,T}})
>> @assert length(ys) == length(xs)
>> @assert length(xs) % N == 0
>> @inbounds for i in 1:N:length(xs)
>> xv = vload(Vec{N,T}, xs, i)
>> yv = vload(Vec{N,T}, ys, i)
>> xv += yv
>> vstore(xv, xs, i)
>> endend
>>
>> i.e. using vload() and vstore() methods?
>>
>>
>> On Thursday, 13 October 2016 15:29:50 UTC+2, Valentin Churavy wrote:
>>>
>>> If you want explicit simd the best way right now is the great SIMD.jl 
>>> package https://github.com/eschnett/SIMD.jl  it is builds on top of 
>>> VecElement.
>>>
>>> In many cases we can perform automatic vectorisation, but you have to 
>>> start Julia with -O3
>>>
>>> On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
>>>>
>>>> i see on the docs 
>>>> http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD 
>>>> that there is a vecElement that is build for SIMD support. I don't 
>>>> understand if as a user I should construct vecElement arrays and hope for 
>>>> some SIMD optimization? thanks.
>>>>
>>>>
>
>
> -- 
> Erik Schnetter <schn...@gmail.com > 
> http://www.perimeterinstitute.ca/personal/eschnetter/
>

[julia-users] Re: What's the status of SIMD instructions from a user's perspective in v0.5?

[Florian Oswald]

ok thanks! and so I should define my SIMD-able function like

function vadd!{N,T}(xs::Vector{T}, ys::Vector{T}, ::Type{Vec{N,T}})
@assert length(ys) == length(xs)
@assert length(xs) % N == 0
@inbounds for i in 1:N:length(xs)
xv = vload(Vec{N,T}, xs, i)
yv = vload(Vec{N,T}, ys, i)
xv += yv
vstore(xv, xs, i)
endend

i.e. using vload() and vstore() methods?


On Thursday, 13 October 2016 15:29:50 UTC+2, Valentin Churavy wrote:
>
> If you want explicit simd the best way right now is the great SIMD.jl 
> package https://github.com/eschnett/SIMD.jl  it is builds on top of 
> VecElement.
>
> In many cases we can perform automatic vectorisation, but you have to 
> start Julia with -O3
>
> On Thursday, 13 October 2016 22:15:00 UTC+9, Florian Oswald wrote:
>>
>> i see on the docs 
>> http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD 
>> that there is a vecElement that is build for SIMD support. I don't 
>> understand if as a user I should construct vecElement arrays and hope for 
>> some SIMD optimization? thanks.
>>
>>

[julia-users] Re: why do we have Base.isless(a, ::NAtype) but not Base.isless(a, ::Nullable)?

[Florian Oswald]

I mean, do I have to cycle through the array and basically clean it of 
#NULL before findign the maximium or is there another way?

On Thursday, 13 October 2016 15:42:02 UTC+2, Florian Oswald wrote:
>
> i'm trying to understand why we don't have something similar in terms of 
> comparison for Nullable as we have for DataArrays NAtype (below). point me 
> to the relevant github conversation, if any, is fine. 
>
> How would I implement methods to find the maximium of an 
> Array{Nullable{Float64}}? like so?
>
> Base.isless(a::Any, x::Nullable{Float64}) = isnull(x) ? true : 
> Base.isless(a,get(x))
>
>
> ~/.julia/v0.5/DataArrays/src/operators.jl:502
>
> #
> # Comparison operators
> #
>
> Base.isequal(::NAtype, ::NAtype) = true
> Base.isequal(::NAtype, b) = false
> Base.isequal(a, ::NAtype) = false
> Base.isless(::NAtype, ::NAtype) = false
> Base.isless(::NAtype, b) = false
> Base.isless(a, ::NAtype) = true
>
>

[julia-users] why do we have Base.isless(a, ::NAtype) but not Base.isless(a, ::Nullable)?

[Florian Oswald]

i'm trying to understand why we don't have something similar in terms of
comparison for Nullable as we have for DataArrays NAtype (below). point me
to the relevant github conversation, if any, is fine.

How would I implement methods to find the maximium of an
Array{Nullable{Float64}}? like so?

Base.isless(a::Any, x::Nullable{Float64}) = isnull(x) ? true :
Base.isless(a,get(x))


~/.julia/v0.5/DataArrays/src/operators.jl:502

#
# Comparison operators
#

Base.isequal(::NAtype, ::NAtype) = true
Base.isequal(::NAtype, b) = false
Base.isequal(a, ::NAtype) = false
Base.isless(::NAtype, ::NAtype) = false
Base.isless(::NAtype, b) = false
Base.isless(a, ::NAtype) = true

[julia-users] What's the status of SIMD instructions from a user's perspective in v0.5?

[Florian Oswald]

i see on the 
docs http://docs.julialang.org/en/release-0.5/stdlib/simd-types/?highlight=SIMD 
that there is a vecElement that is build for SIMD support. I don't 
understand if as a user I should construct vecElement arrays and hope for 
some SIMD optimization? thanks.

[julia-users] Re: problem when building julia 0.5 on plattform without indirect linking

[Florian Oswald]

see 

https://github.com/JuliaLang/julia/issues/18666

On Sunday, 25 September 2016 15:54:30 UTC+2, Florian Oswald wrote:
>
> hi,
>
> i am trying to build the julia release on a remote unix system and get 
> until this step:
>
> [uctpfos@jake julia]$ make
>
> *LINK* *usr/bin/julia*
>
> /opt/rh/devtoolset-3/root/usr/libexec/gcc/x86_64-redhat-linux/4.9.1/ld: 
> /opt/rh/devtoolset-3/root/usr/lib/gcc/x86_64-redhat-linux/4.9.1/libstdc++_nonshared.a(system_error.o):
>  
> undefined reference to symbol '_ZTVSt14error_category@@GLIBCXX_3.4.11'
>
> /usr/lib64/libstdc++.so.6: error adding symbols: DSO missing from command 
> line
>
> collect2: error: ld returned 1 exit status
>
>
> [uctpfos@jake julia]$ echo $LD_LIBRARY_PATH
>
>
> /usr/lib64/libstdc++.so.6:/opt/rh/devtoolset-3/root/usr/lib/gcc/x86_64-redhat-linux/4.9.1
>
>
> I read here (
> http://stackoverflow.com/questions/24989432/linking-error-dso-missing-from-command-line)
>  
> that i should explicitly link to the missing library. how do i do this with 
> the julia makefile? what else needs to be on my LD_LBIRARY_PATH?
>
>
> thanks
>

[julia-users] problem when building julia 0.5 on plattform without indirect linking

[Florian Oswald]

hi,

i am trying to build the julia release on a remote unix system and get 
until this step:

[uctpfos@jake julia]$ make

*LINK* *usr/bin/julia*

/opt/rh/devtoolset-3/root/usr/libexec/gcc/x86_64-redhat-linux/4.9.1/ld: 
/opt/rh/devtoolset-3/root/usr/lib/gcc/x86_64-redhat-linux/4.9.1/libstdc++_nonshared.a(system_error.o):
 
undefined reference to symbol '_ZTVSt14error_category@@GLIBCXX_3.4.11'

/usr/lib64/libstdc++.so.6: error adding symbols: DSO missing from command 
line

collect2: error: ld returned 1 exit status


[uctpfos@jake julia]$ echo $LD_LIBRARY_PATH

/usr/lib64/libstdc++.so.6:/opt/rh/devtoolset-3/root/usr/lib/gcc/x86_64-redhat-linux/4.9.1


I read here 
(http://stackoverflow.com/questions/24989432/linking-error-dso-missing-from-command-line)
 
that i should explicitly link to the missing library. how do i do this with 
the julia makefile? what else needs to be on my LD_LBIRARY_PATH?


thanks

[julia-users] Re: Julia v0.5.0 always killed on a memory limited cluster

[Florian Oswald]

This is affecting my work in exactly the same way. my jobs have been down 
on an SGE managed cluster since I upgraded to v0.5-xxx for other reasons. 
Some support on this would be very good.

On Thursday, 22 September 2016 11:26:16 UTC+2, Alan Crawford wrote:
>
> I am using Julia v0.5.0 on a memory limited SGE cluster. In particular, 
>  to submit jobs on the cluster, both h_vmem and tmem resource flags need to 
> be passed to in qsub command. However, all of Julia jobs keep on being 
> killed because workers seem to be very hungry for virtual memory and ask 
> for it outside of the SGE scheduler. 
>
> A more detailed description of the problem can be found in the last post 
> in this thread 
> . 
> It seems that the best workaround (listed on this thread 
> ) is to change  #define 
> REGION_PG_COUNT 16*8*4096 to #define REGION_PG_COUNT 8*4096 in 
> src/gc-pages.c when compiling Julia. 
>
> However, I am just a user of the cluster and don't have permission to 
> compile Julia on it. Nor do i have access to machinery to compile my own 
> binary . As such I am wondering if there could be a more user friendly 
> option?
>
> Two ideas come to mind:
>
> 1) Offer a low virtual memory for Julia binaries; or
> 2) Add a flag when opening julia flag that lets you set K where K is 
> defined as REGION_PG_COUNT K*8*4096 ... or some other equivalent...
>
> In particular, option 2 would really be addition to the language and 
> greatly aid users - especially those without capacity to compile julia. If, 
> for some reason, this is undesirable, perhaps option 1 would present the 
> best short term fix?
>
>

Re: [julia-users] v0.5 deprecates :call. so how do you :call now?

[Florian Oswald]

erm... yes. sorry that was a stupid question. 
thanks anyway!

On Monday, 19 September 2016 10:48:44 UTC+2, Bart Janssens wrote:
>
>
>
> On Mon, Sep 19, 2016 at 10:11 AM Florian Oswald <florian...@gmail.com 
> > wrote:
>
>>
>> So I read somewhere that my construction involving `eval` is bad 
>> practice. can anyone give me some guidance as to what is the proper julian 
>> way of expressing this:
>>
>> eval(Expr(:call,f,3))?
>>
>>
>
> Just calling the function:
> f(3)
>
> ?
>
>  
>

Re: [julia-users] v0.5 deprecates :call. so how do you :call now?

[Florian Oswald]

Oh, I see! 

So I read somewhere that my construction involving `eval` is bad practice. 
can anyone give me some guidance as to what is the proper julian way of 
expressing this:

eval(Expr(:call,f,3))?

thanks!

On Monday, 19 September 2016 09:46:55 UTC+2, Bart Janssens wrote:
>
> In 0.5-rc4 your code runs without any warnings. I think only the call 
> function was deprecated, not the :call symbol in an expression tree: 
> dump(:(f(3))) yields:
> Expr
>   head: Symbol call
>   args: Array{Any}((2,))
> 1: Symbol f
> 2: Int64 3
>   typ: Any
>
> Cheers,
>
> Bart
>
> On Mon, Sep 19, 2016 at 9:32 AM Florian Oswald <florian...@gmail.com 
> > wrote:
>
>> i was trying to make sense of the final outcome of the discussion here 
>> https://github.com/JuliaLang/julia/pull/13412, but no way. Also I'm not 
>> sure what to search for in the documentation (?Base.call tells me it's 
>> deprecated.)
>>
>> So: 
>>
>> if up to now I programmatically evaluate a function f like that:
>>
>> f(x) = x^2
>>
>> eval(Expr(:call,f,3))
>> 9
>>
>> what's the new way of doing that?
>> cheers
>> florian
>>
>

[julia-users] v0.5 deprecates :call. so how do you :call now?

[Florian Oswald]

i was trying to make sense of the final outcome of the discussion 
here https://github.com/JuliaLang/julia/pull/13412, but no way. Also I'm 
not sure what to search for in the documentation (?Base.call tells me it's 
deprecated.)

So: 

if up to now I programmatically evaluate a function f like that:

f(x) = x^2

eval(Expr(:call,f,3))
9

what's the new way of doing that?
cheers
florian

[julia-users] can someone help me read julia's memory footprint on this cluster? [SGE]

[Florian Oswald]

hi all,

i get the following output from the SGE command `qstat -j jobnumber` of a 
julia job that uses 30 workers. I am confused by the mem column. am I using 
more memory than what I asked for? I asked for max 4G on each processor.


job-array tasks:1-30:1

usage1: cpu=00:00:08, mem=20.61903 GBs, io=0.08026, 
vmem=2.684G, maxvmem=2.684G

usage2: cpu=00:00:13, mem=35.36547 GBs, io=0.14832, 
vmem=2.754G, maxvmem=2.754G

usage3: cpu=00:00:16, mem=47.97179 GBs, io=0.10563, 
vmem=3.084G, maxvmem=3.084G

usage4: cpu=00:00:17, mem=52.39960 GBs, io=0.14685, 
vmem=3.146G, maxvmem=3.146G

usage5: cpu=00:00:13, mem=38.00948 GBs, io=0.06336, 
vmem=3.208G, maxvmem=3.208G

usage6: cpu=00:00:14, mem=41.84277 GBs, io=0.08085, 
vmem=3.208G, maxvmem=3.208G

usage7: cpu=00:00:16, mem=49.34722 GBs, io=0.10563, 
vmem=3.208G, maxvmem=3.208G

usage8: cpu=00:00:18, mem=56.29933 GBs, io=0.14685, 
vmem=3.208G, maxvmem=3.208G

usage9: cpu=00:00:21, mem=61.30837 GBs, io=0.13234, 
vmem=3.145G, maxvmem=3.145G

usage   10: cpu=00:00:18, mem=53.78262 GBs, io=0.10650, 
vmem=3.209G, maxvmem=3.209G

usage   11: cpu=00:00:19, mem=58.20804 GBs, io=0.16047, 
vmem=3.208G, maxvmem=3.208G

usage   12: cpu=00:00:19, mem=58.90526 GBs, io=0.15296, 
vmem=3.209G, maxvmem=3.209G

usage   13: cpu=00:00:13, mem=37.73257 GBs, io=0.06336, 
vmem=3.208G, maxvmem=3.208G

usage   14: cpu=00:00:15, mem=43.44044 GBs, io=0.08085, 
vmem=3.208G, maxvmem=3.208G

usage   15: cpu=00:00:19, mem=58.27114 GBs, io=0.13234, 
vmem=3.208G, maxvmem=3.208G

usage   16: cpu=00:00:17, mem=51.33971 GBs, io=0.10650, 
vmem=3.208G, maxvmem=3.208G

usage   17: cpu=00:00:19, mem=56.00911 GBs, io=0.16047, 
vmem=3.208G, maxvmem=3.208G

usage   18: cpu=00:00:19, mem=57.45101 GBs, io=0.15301, 
vmem=3.209G, maxvmem=3.209G

usage   19: cpu=00:00:19, mem=56.42524 GBs, io=0.13240, 
vmem=3.208G, maxvmem=3.208G

usage   20: cpu=00:00:18, mem=52.25189 GBs, io=0.10650, 
vmem=3.209G, maxvmem=3.209G

usage   21: cpu=00:00:20, mem=60.76601 GBs, io=0.12465, 
vmem=3.208G, maxvmem=3.208G

usage   22: cpu=00:00:22, mem=65.11690 GBs, io=0.14843, 
vmem=3.207G, maxvmem=3.207G

usage   23: cpu=00:00:18, mem=52.75353 GBs, io=0.11566, 
vmem=3.146G, maxvmem=3.146G

usage   24: cpu=00:00:15, mem=44.21442 GBs, io=0.04204, 
vmem=3.207G, maxvmem=3.207G

usage   25: cpu=00:00:20, mem=58.85802 GBs, io=0.14714, 
vmem=3.209G, maxvmem=3.209G

usage   26: cpu=00:00:18, mem=53.52543 GBs, io=0.12236, 
vmem=3.208G, maxvmem=3.208G

usage   27: cpu=00:00:20, mem=59.24938 GBs, io=0.13234, 
vmem=3.208G, maxvmem=3.208G

usage   28: cpu=00:00:20, mem=59.86234 GBs, io=0.12465, 
vmem=3.208G, maxvmem=3.208G

usage   29: cpu=00:00:18, mem=53.94314 GBs, io=0.11566, 
vmem=3.209G, maxvmem=3.209G

usage   30: cpu=00:00:16, mem=48.74432 GBs, io=0.10222, 
vmem=3.208G, maxvmem=3.208G

[julia-users] Re: @profile does not allow variable assignment on v0.5

[Florian Oswald]

I'm pretty sure this behaved differently some time ago, because that was a 
piece of working code. It's not worth a bug report though.

On Friday, 9 September 2016 13:19:12 UTC+2, Lutfullah Tomak wrote:
>
> I think it is on the way @profile defined makes it not processed in outer 
> scope
>
> julia> macroexpand(:(@profile y = f(5)))
> quote  # profile.jl, line 11:
> try  # profile.jl, line 12:
> #10#status = (Base.Profile.start_timer)() # profile.jl, line 13:
> if #10#status < 0 # profile.jl, line 14:
> (Base.Profile.error)((Base.Profile.error_codes)[#10#status])
> end # profile.jl, line 16:
> y = f(5)
> finally  # profile.jl, line 18:
> (Base.Profile.stop_timer)()
> end
> end
>
> try finally end block makes y a local variable if not defined beforehand.
>
> julia> try
>y=125
>finally
>nothing
>end
> 125
>
>
> julia> y
> ERROR: UndefVarError: y not defined
>
>
> julia> y =5
> 5
>
>
> julia> try
>y=125
>finally
>nothing
>    end
> 125
>
>
> If you think it is a regression file a bug report for profile.
>
>
> On Friday, September 9, 2016 at 1:04:21 PM UTC+3, Florian Oswald wrote:
>>
>> hi all, 
>>
>> on v0.5-rc3, I see this
>>
>> *julia> **f(x)=x^3*
>>
>> *f (generic function with 1 method)*
>>
>>
>> *julia> **@profile y = f(5)*
>>
>> *125*
>>
>>
>> *julia> **y*
>>
>> *ERROR: UndefVarError: y not defined*
>>
>>
>>
>> which used to work in previous versions. Is that intended behaviour?
>>
>>
>>

[julia-users] @profile does not allow variable assignment on v0.5

[Florian Oswald]

hi all, 

on v0.5-rc3, I see this

*julia> **f(x)=x^3*

*f (generic function with 1 method)*


*julia> **@profile y = f(5)*

*125*


*julia> **y*

*ERROR: UndefVarError: y not defined*



which used to work in previous versions. Is that intended behaviour?

Re: [julia-users] how to set the maximal waiting time for ClusterManagers.addprocs_sge()?

[Florian Oswald]

Yes this seems to be ten same issue. When my cluster is heavily loaded it
sometimes is so slow that I hit the 60sec limit anyway.

Why don't we just grab the list of nodes from the sge env variable
PE_HOSTFILE as soon as this becomes available and put that list into a
simple addprocs( machines ) call?

On Thursday, 2 June 2016, David van Leeuwen <david.vanleeu...@gmail.com>
wrote:

> Could this be related to this
> <https://github.com/JuliaParallel/ClusterManagers.jl/issues/4#event-677794220>
> ?
>
> On Wednesday, June 1, 2016 at 7:26:50 PM UTC+2, Florian Oswald wrote:
>>
>> i'm having a problem on a cluster where setting up the connections via
>> ClusterManagers.addprocs_sge() takes longer than the 60 second limit. how
>> can I extend that limit? thanks!
>>
>

[julia-users] how to set the maximal waiting time for ClusterManagers.addprocs_sge()?

[Florian Oswald]

i'm having a problem on a cluster where setting up the connections via 
ClusterManagers.addprocs_sge() takes longer than the 60 second limit. how 
can I extend that limit? thanks!

[julia-users] Re: installing nlopt.jl on unix with manually installed NLopt?

[Florian Oswald]

figured it out:

https://github.com/JuliaOpt/NLopt.jl/issues/36

On Monday, 18 May 2015 15:23:57 UTC+1, Florian Oswald wrote:

 Hi all,

 I'm working on a scienfitic unix cluster where the NLopt.jl provided 
 build.jl fails. I paste the error output in the end. I installed NLopt 
 manually into ~/local. How can I build NLopt.jl such that it takes my 
 ~/local/lib/libnlopt.a? I have done

 push!(Sys.DL_LOAD_PATH, ${HOME}/local/lib)

 thanks!






 *[ ERROR: NLopt 
 ]*


 *could not spawn 
 setenv(`/home/eisuc151/.julia/v0.3/NLopt/deps/src/nlopt-2.4/configure 
 --enable-shared --without-guile --without-python --without-octave 
 --without-matlab --with-cxx 
 --prefix=/home/eisuc151/.julia/v0.3/NLopt/deps/usr`,Union(UTF8String,ASCIIString)[PATH=/home/eisuc151/.julia/v0.3/NLopt/deps/usr/bin:/home/eisuc151/local/bin:/home/eisuc151/.autojump/bin:/home/eisuc151/local/bin:/local/software/rh53/hdf5/1.8.4/gcc/bin:/local/software/rh53/python/2.6.5/gcc/bin:/local/software/rh53/protobuf/2.4.1/gcc/bin:/local/software/rh53/samtools/0.1.16/bin:/local/software/rh53/gcc/4.8.1/bin:/usr/lib64/qt-3.3/bin:/local/software/rh53/acroread/8.1.6/Adobe/Reader8/bin:/local/software/maui/3.3.1/bin:/usr/lpp/mmfs//bin:/local/software/torque/default/sbin:/local/software/torque/default/bin:/usr/local/bin:/bin:/usr/bin,HDF5_INC_DIR=/local/software/rh53/hdf5/1.8.4/gcc/include,QTDIR=/usr/lib64/qt-3.3,LD_LIBRARY_PATH=/local/software/rh53/hdf5/1.8.4/gcc/lib:/local/software/rh53/mpc/0.9/gcc/lib:/local/software/rh53/gmp/5.0.2/gcc/lib:/local/software/rh53/mpfr/2.4.2/gcc/lib:/local/software/rh53/gcc/4.8.1/lib:/local/software/rh53/gcc/4.8.1/lib64:/local/software/torque/default/lib,MANPATH=/local/software/rh53/python/2.6.5/gcc/share/man:/local/software/rh53/protobuf/2.4.1/gcc/share/man:/local/software/rh53/samtools/0.1.16/man:/local/software/rh53/gcc/4.8.1/share/man:/local/software/torque/default/man:/usr/local/share/man:/usr/share/man/en:/usr/share/man:/local/Modules/3.2.6/man,MODULEPATH=/local/Modules/versions:/local/Modules/\$MODULE_VERSION/modulefiles:/local/Modules/modulefiles:,LIBPATH=/local/software/rh53/gcc/4.8.1/libexec/gcc/x86_64-unknown-linux-gnu/4.8.1,USER=eisuc151,KDEDIR=/usr,PBS_SERVER=blue30,blue31,SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass,LESSOPEN=|/usr/bin/lesspipe.sh
  
 %s,MAUIHOMEDIR=/local/maui/3.3.1,SHELL=/bin/bash,QTLIB=/usr/lib64/qt-3.3/lib,SSH_CLIENT=152.78.130.89
  
 44959 
 22,SSH_TTY=/dev/pts/7,HOME=/home/eisuc151,TERM=xterm-256color,HOSTNAME=blue34,QTINC=/usr/lib64/qt-3.3/include,HISTSIZE=1000,LC_ALL=C,LC_CTYPE=en_GB.UTF-8,LANG=en_US.UTF-8,SHLVL=1,LOADEDMODULES=torque/current:gpfs:moab/6.1.9:acroread/8.1.6:gcc/4.8.1:samtools/0.1.16:protobuf/2.4.1:python/2.6.5/gcc:hdf5/1.8.4,LOGNAME=eisuc151,OLDPWD=/home/eisuc151/local,_LMFILES_=/local/Modules/modulefiles/torque/current:/local/Modules/modulefiles/gpfs:/local/Modules/modulefiles/moab/6.1.9:/local/Modules/3.2.6/modulefiles/acroread/8.1.6:/local/Modules/3.2.6/modulefiles/gcc/4.8.1:/local/Modules/3.2.6/modulefiles/samtools/0.1.16:/local/Modules/3.2.6/modulefiles/protobuf/2.4.1:/local/Modules/3.2.6/modulefiles/python/2.6.5/gcc:/local/Modules/3.2.6/modulefiles/hdf5/1.8.4,HDF5_LIB_DIR=/local/software/rh53/hdf5/1.8.4/gcc/lib,SHARKCLONEROOT=/usr/lpp/mmfs//src,_=/home/eisuc151/local/bin/julia,CVS_RSH=ssh,G_BROKEN_FILENAMES=1,MODULE_VERSION_STACK=3.2.6,GIT_SSL_NO_VERIFY=true,PWD=/home/eisuc151/local/lib,MODULESHOME=/local/Modules/3.2.6,PYTHONPATH=/home/eisuc151/local/lib/python2.6/site-packages/,BASH_FUNC_module()=()
  
 {  eval `/local/Modules/\$MODULE_VERSION/bin/modulecmd bash 
 \$*`\n},SSH_CONNECTION=152.78.130.89 44959 152.78.31.149 
 22,AUTOJUMP_ERROR_PATH=/home/eisuc151/.local/share/autojump/errors.log,OPENBLAS_NUM_THREADS=8,CPPFLAGS=
  
 -I/home/eisuc151/.julia/v0.3/NLopt/deps/usr/include,MAIL=/var/spool/mail/eisuc151,LDFLAGS=
  
 -L/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib -Wl,-rpath 
 -Wl,/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib,MODULE_VERSION=3.2.6,PKG_CONFIG_PATH=/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib/pkgconfig,INPUTRC=/etc/inputrc,KDE_NO_IPV6=1,LS_COLORS=di=1;34:fi=0:ln=31:pi=5:so=5:bd=5:cd=5:or=31:mi=0:ex=35:*.sh=31:*.r=32:*.R=32:*.out=35]):
  
 no such file or directory (ENOENT)*

 *while loading /home/eisuc151/.julia/v0.3/NLopt/deps/build.jl, in 
 expression starting on line 54*

[julia-users] installing nlopt.jl on unix with manually installed NLopt?

[Florian Oswald]

Hi all,

I'm working on a scienfitic unix cluster where the NLopt.jl provided 
build.jl fails. I paste the error output in the end. I installed NLopt 
manually into ~/local. How can I build NLopt.jl such that it takes my 
~/local/lib/libnlopt.a? I have done

push!(Sys.DL_LOAD_PATH, ${HOME}/local/lib)

thanks!






*[ ERROR: NLopt 
]*


*could not spawn 
setenv(`/home/eisuc151/.julia/v0.3/NLopt/deps/src/nlopt-2.4/configure 
--enable-shared --without-guile --without-python --without-octave 
--without-matlab --with-cxx 
--prefix=/home/eisuc151/.julia/v0.3/NLopt/deps/usr`,Union(UTF8String,ASCIIString)[PATH=/home/eisuc151/.julia/v0.3/NLopt/deps/usr/bin:/home/eisuc151/local/bin:/home/eisuc151/.autojump/bin:/home/eisuc151/local/bin:/local/software/rh53/hdf5/1.8.4/gcc/bin:/local/software/rh53/python/2.6.5/gcc/bin:/local/software/rh53/protobuf/2.4.1/gcc/bin:/local/software/rh53/samtools/0.1.16/bin:/local/software/rh53/gcc/4.8.1/bin:/usr/lib64/qt-3.3/bin:/local/software/rh53/acroread/8.1.6/Adobe/Reader8/bin:/local/software/maui/3.3.1/bin:/usr/lpp/mmfs//bin:/local/software/torque/default/sbin:/local/software/torque/default/bin:/usr/local/bin:/bin:/usr/bin,HDF5_INC_DIR=/local/software/rh53/hdf5/1.8.4/gcc/include,QTDIR=/usr/lib64/qt-3.3,LD_LIBRARY_PATH=/local/software/rh53/hdf5/1.8.4/gcc/lib:/local/software/rh53/mpc/0.9/gcc/lib:/local/software/rh53/gmp/5.0.2/gcc/lib:/local/software/rh53/mpfr/2.4.2/gcc/lib:/local/software/rh53/gcc/4.8.1/lib:/local/software/rh53/gcc/4.8.1/lib64:/local/software/torque/default/lib,MANPATH=/local/software/rh53/python/2.6.5/gcc/share/man:/local/software/rh53/protobuf/2.4.1/gcc/share/man:/local/software/rh53/samtools/0.1.16/man:/local/software/rh53/gcc/4.8.1/share/man:/local/software/torque/default/man:/usr/local/share/man:/usr/share/man/en:/usr/share/man:/local/Modules/3.2.6/man,MODULEPATH=/local/Modules/versions:/local/Modules/\$MODULE_VERSION/modulefiles:/local/Modules/modulefiles:,LIBPATH=/local/software/rh53/gcc/4.8.1/libexec/gcc/x86_64-unknown-linux-gnu/4.8.1,USER=eisuc151,KDEDIR=/usr,PBS_SERVER=blue30,blue31,SSH_ASKPASS=/usr/libexec/openssh/gnome-ssh-askpass,LESSOPEN=|/usr/bin/lesspipe.sh
 
%s,MAUIHOMEDIR=/local/maui/3.3.1,SHELL=/bin/bash,QTLIB=/usr/lib64/qt-3.3/lib,SSH_CLIENT=152.78.130.89
 
44959 
22,SSH_TTY=/dev/pts/7,HOME=/home/eisuc151,TERM=xterm-256color,HOSTNAME=blue34,QTINC=/usr/lib64/qt-3.3/include,HISTSIZE=1000,LC_ALL=C,LC_CTYPE=en_GB.UTF-8,LANG=en_US.UTF-8,SHLVL=1,LOADEDMODULES=torque/current:gpfs:moab/6.1.9:acroread/8.1.6:gcc/4.8.1:samtools/0.1.16:protobuf/2.4.1:python/2.6.5/gcc:hdf5/1.8.4,LOGNAME=eisuc151,OLDPWD=/home/eisuc151/local,_LMFILES_=/local/Modules/modulefiles/torque/current:/local/Modules/modulefiles/gpfs:/local/Modules/modulefiles/moab/6.1.9:/local/Modules/3.2.6/modulefiles/acroread/8.1.6:/local/Modules/3.2.6/modulefiles/gcc/4.8.1:/local/Modules/3.2.6/modulefiles/samtools/0.1.16:/local/Modules/3.2.6/modulefiles/protobuf/2.4.1:/local/Modules/3.2.6/modulefiles/python/2.6.5/gcc:/local/Modules/3.2.6/modulefiles/hdf5/1.8.4,HDF5_LIB_DIR=/local/software/rh53/hdf5/1.8.4/gcc/lib,SHARKCLONEROOT=/usr/lpp/mmfs//src,_=/home/eisuc151/local/bin/julia,CVS_RSH=ssh,G_BROKEN_FILENAMES=1,MODULE_VERSION_STACK=3.2.6,GIT_SSL_NO_VERIFY=true,PWD=/home/eisuc151/local/lib,MODULESHOME=/local/Modules/3.2.6,PYTHONPATH=/home/eisuc151/local/lib/python2.6/site-packages/,BASH_FUNC_module()=()
 
{  eval `/local/Modules/\$MODULE_VERSION/bin/modulecmd bash 
\$*`\n},SSH_CONNECTION=152.78.130.89 44959 152.78.31.149 
22,AUTOJUMP_ERROR_PATH=/home/eisuc151/.local/share/autojump/errors.log,OPENBLAS_NUM_THREADS=8,CPPFLAGS=
 
-I/home/eisuc151/.julia/v0.3/NLopt/deps/usr/include,MAIL=/var/spool/mail/eisuc151,LDFLAGS=
 
-L/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib -Wl,-rpath 
-Wl,/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib,MODULE_VERSION=3.2.6,PKG_CONFIG_PATH=/home/eisuc151/.julia/v0.3/NLopt/deps/usr/lib/pkgconfig,INPUTRC=/etc/inputrc,KDE_NO_IPV6=1,LS_COLORS=di=1;34:fi=0:ln=31:pi=5:so=5:bd=5:cd=5:or=31:mi=0:ex=35:*.sh=31:*.r=32:*.R=32:*.out=35]):
 
no such file or directory (ENOENT)*

*while loading /home/eisuc151/.julia/v0.3/NLopt/deps/build.jl, in 
expression starting on line 54*

[julia-users] Re: 4D interpolation from scattered points

[Florian Oswald]

Hi there,
xdata (or any other grid) does not need to be on a grid, that's just the 
way the example is written. but no extrapolation, that's right.

On Thursday, 14 May 2015 01:05:55 UTC+1, Yakir Gagnon wrote:

 Thanks a ton people!!!

 ExtremelyRandomizedTrees.jl: Might be really good, but errored a lot on 
 version 4.
 ApproXD.jl: Very cool, I'll come back to that later, but for now: What 
 Lininterp cannot do: Multidimensional extrapolation which I need (and 
 accept is prone to silliness). Plus, according to the examples, the known 
 xdata needs to be on a grid. My x,y,z are not equally spaced that way, they 
 are scattered.
 compute the distances: I started with that, and in light of these results 
 I might fall back to doing just that. 
 Polyharmonic Splines: does exactly what I want, but not faster than just 
 distances (but probably a lot more accurate). 
 CHull.jl: might be awesome for this, but I didn't quickly see how I can 
 use it to interpolate and extrapolate.

 I feel like I owe you guys some background: I have some color-map 
 functions that take a value, let's call it X (for some 0X1 and for some 
 -pi/2Xpi/2) and return an RGB. I need to use these to convert images that 
 have colors that are similar (but not always identical) to the RGB values 
 from these color-maps back to the value (X) that would have resulted in 
 said RGB (so it's like the inverse of those color-map functions). After 
 much trial and error, I think that since the images I want to convert this 
 way are all 8-bit, the fastest way would be to construct a lookup table for 
 all 2^(8*3) possible RGB combinations. This table (Dict) will take any RGB 
 value from those images I need to convert and return X. This makes sense 
 because I have only 4 such color-map functions, and potentially endless 
 amounts of images to convert (currently a couple of tera). 
 So I think I'll use simple distances, run that once, save those lookup 
 tables and use them again and again on new images. 




 On Thursday, May 14, 2015 at 4:19:18 AM UTC+10, Luke Stagner wrote:

 You can use Polyharmonic Splines 
 http://nbviewer.ipython.org/gist/lstagner/04a05b120e0be7de9915


 On Wednesday, May 13, 2015 at 5:33:08 AM UTC-7, Yakir Gagnon wrote:

 I have a bunch (~1000) of x,y,z and a corresponding value, V. One unique 
 V for each x,y,z. I want to interpolate and extrapolate wildly (so I 
 really don't care about how accurate or correct it is). The x,y,z I have 
 are not regularly spaced or anything. They're scattered across some range 
 (they all share similar ranges), and I want to know what value (i.e. new V) 
 I should be getting at new and regularly spaced x,y,z. I think 
 Matlab's griddata would do what I want.
 I tired following the lower-level functionality from Grid.jl, 
 Interpolations.jl, and Dierckx.jl, but couldn't figure out how to get this 
 to work. 
 I think I need to fit some curve to my scattered points and then use 
 that to find the values at the new xyz (at least that's how I think 
 griddata works)... Any good simple ideas out there?

[julia-users] Lexicon.jl Index() function?

[Florian Oswald]

Hi all,

I'm trying to follow the steps outline in the Lexicon 
documentation http://lexiconjl.readthedocs.org/en/latest/api/Lexicon/

and I am stuck with this:

using Lexicon, Docile, Docile.Interface
index  = Index()
update!(index, save(docs/api/Lexicon.md, Lexicon));
update!(index, save(docs/api/Docile.md, Docile));
update!(index, save(docs/api/Docile.Interface.md, Docile.Interface));
# save a joined Reference-Index
save(docs/api/api-index.md, index);

it says `*ERROR: Index not defined`. *

Where does the Index() function come from?

thanks
florian

[julia-users] Re: Lexicon.jl Index() function?

[Florian Oswald]

hi,

I'm on julia 0.3.6. A Pkg.update() did

*INFO: Upgrading Docile: v0.4.8 = v0.4.13*

*INFO: Upgrading Lexicon: v0.1.4 = v0.1.9*

and everything's fine. you guys work too fast! :-)

sorry about that, my bad. I may have some more reasonable questions later 
on.

florian

On Tuesday, 12 May 2015 11:39:25 UTC+1, Michael Hatherly wrote:

 Hi Florian,

  it says `

 *ERROR: Index not defined`. *Is that the full output from the error 
 message? When I run the example it doesn't produce an error. What versions 
 of both Lexicon and Docile are you using? Also, is this on Julia 0.3 or 0.4?

 -- Mike

 On Tuesday, 12 May 2015 12:29:17 UTC+2, Florian Oswald wrote:

 Hi all,

 I'm trying to follow the steps outline in the Lexicon documentation 
 http://lexiconjl.readthedocs.org/en/latest/api/Lexicon/

 and I am stuck with this:

 using Lexicon, Docile, Docile.Interface
 index  = Index()
 update!(index, save(docs/api/Lexicon.md, Lexicon));
 update!(index, save(docs/api/Docile.md, Docile));
 update!(index, save(docs/api/Docile.Interface.md, Docile.Interface));
 # save a joined Reference-Index
 save(docs/api/api-index.md, index);

 it says `*ERROR: Index not defined`. *

 Where does the Index() function come from?

 thanks
 florian

[julia-users] problem with dataframes.Index when loading JLD dataset

[Florian Oswald]

Hi all,

I'm currently trying to work out why I get an error when trying to load a 
dict with several dataframes in it with HDF5, JLD, see 
https://github.com/timholy/HDF5.jl/issues/158

which results in this error:
*ERROR: stored type DataFrames.Index does not match currently loaded type*

I just wanted to reach out to the dataframes crowd: I don't actually know 
what DataFrames.Index is or does. In the issue I filed a small example with 
2 dataframes works fine, so I'm trying to narrow my case down somehow. I 
know that the DataFrame constructor is

type DataFrame : AbstractDataFrame
columns::Vector{Any}
colindex::Index

Is Index a julia type or is it defined somewhere in dataframes? how could 
I check where a certain type is defined?

Re: [julia-users] ANN: QuantEcon.jl

[Florian Oswald]

If I may, I've got an implementation of linear interpolation that I'm quite 
happy with. It uses a GSL kind of accelerator that remembers the index 
position of the last evaluation. the binary search for the correct bracket 
of x in the grid is the main bottleneck in my experience. So taking 
advantage of this (particularly with monotonic functions) speeds up things 
considerably. 

the documentation is sparse and it's not quite ready to be advertised, but 
the tests are quite complete, so check it out at 

https://github.com/floswald/ApproXD.jl

On Saturday, 20 September 2014 02:42:01 UTC+1, Tim Holy wrote:

 On Friday, September 19, 2014 05:06:41 PM Spencer Lyon wrote: 
 2. Linear interpolation. The function numpy.interp does simple 
 piecewise 
  linear interpolation. We ended up using the CoordInterpGrid type from 
  Grid.jl to accomplish this. As of right now the interpolation steps are 
 the 
  biggest bottleneck in most of the functions we have. 

 Performance of Grid is not terrific. There's an effort to change that 
 (current 
 efforts are focused at https://github.com/tlycken/Interpolations.jl). But 
 I 
 think it's held up now by the need to clarify exactly how to handle 
 https://github.com/timholy/Grid.jl/issues/37. 

 Of course, there are quite a few algorithms where, even if optimized, 
 interpolation is the main bottleneck. But it would be less of a bottleneck 
 if 
 it were faster. 

 --Tim 

[julia-users] How to print a StatsBase.CoefTable / other table object to latex?

[Florian Oswald]

just wondering if anyone has a way to print for example a GLM regression 
output table to tex? I think I'm looking for something like 
[http://cran.r-project.org/web/packages/texreg/index.html], although maybe 
that's not the best way to achieve this in julia.

right now I'm converting the coeftable object to a dataframe and print 
that, but of course it would be nice to print some default statistics 
(Rsquared, pval stars, etc) with it.

thanks for any suggestions!

[julia-users] unexpected domain error for ^(float,float)

[Florian Oswald]

# define a variable gamma:

gamma = 1.4
mgamma = 1.0-gamma

julia mgamma 
-0.3999

# this works:

julia -0.3999^2.5 
-0.10119288512475567

# this doesn't:

julia mgamma^2.5 
ERROR: DomainError 
in ^ at math.jl:252

Re: [julia-users] unexpected domain error for ^(float,float)

[Florian Oswald]

i see!

*julia **:(-0.4^-2.5)*

*:(-(0.4^-2.5))*

is good to know! didnt' think of this at all so far.

On 18 September 2014 14:01, Jutho juthohaege...@gmail.com wrote:

 because it is not recognized/parsed as literal but as the application of a
 unary minus, which has lower precedence than ^

 I guess it is not possible to give binary minus a lower precedence than ^
 and unary minus of higher precedence, since these are just different
 methods of the same function/operator.

 Op donderdag 18 september 2014 14:54:26 UTC+2 schreef Florian Oswald:

 yes - not sure why -0.4 and (-0.4) are any different.

 On 18 September 2014 13:52, Patrick O'Leary patrick...@gmail.com wrote:

 Seems like the literal -0.4^2.5 should throw the same error, though?


 On Thursday, September 18, 2014 6:42:56 AM UTC-5, Tim Holy wrote:

 http://docs.julialang.org/en/latest/manual/faq/#why-does-jul
 ia-give-a-domainerror-for-certain-seemingly-sensible-operations

 On Thursday, September 18, 2014 03:24:00 AM Florian Oswald wrote:
  # define a variable gamma:
 
  gamma = 1.4
  mgamma = 1.0-gamma
 
  julia mgamma
  -0.3999
 
  # this works:
 
  julia -0.3999^2.5
  -0.10119288512475567
 
  # this doesn't:
 
  julia mgamma^2.5
  ERROR: DomainError
  in ^ at math.jl:252

Re: [julia-users] unexpected domain error for ^(float,float)

[Florian Oswald]

well, I guess most computer scientists would be surprised. writing on a
piece of paper

-10^2

and

-(10^2)

I think most people are going to say the first expression is 100 and the
second is -100. I take the point that what I did was a bit stupid and Julia
is not making any mistake here.

On 18 September 2014 16:50, Gunnar Farnebäck gun...@lysator.liu.se wrote:

 It's not like Julia is doing anything strange or uncommon here. Most
 people would be really surprised if -10² meant positive 100.

 Den torsdagen den 18:e september 2014 kl. 15:01:44 UTC+2 skrev Jutho:

 because it is not recognized/parsed as literal but as the application of
 a unary minus, which has lower precedence than ^

 I guess it is not possible to give binary minus a lower precedence than ^
 and unary minus of higher precedence, since these are just different
 methods of the same function/operator.

 Op donderdag 18 september 2014 14:54:26 UTC+2 schreef Florian Oswald:

 yes - not sure why -0.4 and (-0.4) are any different.

 On 18 September 2014 13:52, Patrick O'Leary patrick...@gmail.com
 wrote:

 Seems like the literal -0.4^2.5 should throw the same error, though?


 On Thursday, September 18, 2014 6:42:56 AM UTC-5, Tim Holy wrote:

 http://docs.julialang.org/en/latest/manual/faq/#why-does-jul
 ia-give-a-domainerror-for-certain-seemingly-sensible-operations

 On Thursday, September 18, 2014 03:24:00 AM Florian Oswald wrote:
  # define a variable gamma:
 
  gamma = 1.4
  mgamma = 1.0-gamma
 
  julia mgamma
  -0.3999
 
  # this works:
 
  julia -0.3999^2.5
  -0.10119288512475567
 
  # this doesn't:
 
  julia mgamma^2.5
  ERROR: DomainError
  in ^ at math.jl:252

Re: [julia-users] unexpected domain error for ^(float,float)

[Florian Oswald]

ok guys i won't dig myself a deeper hole here - you win.

(savored my 3 seconds of fame before steven corrected that typo tough!)

On 18 September 2014 18:21, Jameson Nash vtjn...@gmail.com wrote:

 I'm not sure about most people, but given the first expression, I would
 have handed the paper back and told the author to clarify the ambiguity.


 On Thursday, September 18, 2014, Florian Oswald florian.osw...@gmail.com
 wrote:

 well, I guess most computer scientists would be surprised. writing on a
 piece of paper

 -10^2

 and

 -(10^2)

 I think most people are going to say the first expression is 100 and the
 second is -100. I take the point that what I did was a bit stupid and Julia
 is not making any mistake here.

 On 18 September 2014 16:50, Gunnar Farnebäck gun...@lysator.liu.se
 wrote:

 It's not like Julia is doing anything strange or uncommon here. Most
 people would be really surprised if -10² meant positive 100.

 Den torsdagen den 18:e september 2014 kl. 15:01:44 UTC+2 skrev Jutho:

 because it is not recognized/parsed as literal but as the application
 of a unary minus, which has lower precedence than ^

 I guess it is not possible to give binary minus a lower precedence than
 ^ and unary minus of higher precedence, since these are just different
 methods of the same function/operator.

 Op donderdag 18 september 2014 14:54:26 UTC+2 schreef Florian Oswald:

 yes - not sure why -0.4 and (-0.4) are any different.

 On 18 September 2014 13:52, Patrick O'Leary patrick...@gmail.com
 wrote:

 Seems like the literal -0.4^2.5 should throw the same error, though?


 On Thursday, September 18, 2014 6:42:56 AM UTC-5, Tim Holy wrote:

 http://docs.julialang.org/en/latest/manual/faq/#why-does-jul
 ia-give-a-domainerror-for-certain-seemingly-sensible-operations

 On Thursday, September 18, 2014 03:24:00 AM Florian Oswald wrote:
  # define a variable gamma:
 
  gamma = 1.4
  mgamma = 1.0-gamma
 
  julia mgamma
  -0.3999
 
  # this works:
 
  julia -0.3999^2.5
  -0.10119288512475567
 
  # this doesn't:
 
  julia mgamma^2.5
  ERROR: DomainError
  in ^ at math.jl:252

[julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

[Florian Oswald]

i'm trying to do this:

using DataFrames
df = DataFrame(a=[hi,there],x = rand(2))
df2 = DataFrame(a=[oh,yeah],x = rand(2))

for e in eachrow(df)
append!(df2,e)
end

ERROR: `append!` has no method matching append!(::DataFrame, 
::DataFrameRow{DataFrame}) 
in anonymous at no file:2

or 

julia for i in 1:nrow(df) 
push!(df2,df[i,:]) 
end

but that errors as well. 

this works:

julia for i in 1:nrow(df) 
push!(df2,array(df[i,:])) 
end

but wondering whether that's the best way of achieving this efficiently. 

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

[Florian Oswald]

yeah I wasn't very clear in that example. i really need to append one row
at a time.

On 12 September 2014 14:50, Leah Hanson astriea...@gmail.com wrote:

 Have you tried append!(df2,df)?

 ~~~
 julia using DataFrames



 julia df = DataFrame(a=[hi,there],x = rand(2))


 2x2 DataFrame
 |---|-|--|
 | Row # | a   | x|
 | 1 | hi| 0.862957 |
 | 2 | there | 0.101378 |



 julia df2 = DataFrame(a=[oh,yeah],x = rand(2))


 2x2 DataFrame
 |---|||
 | Row # | a  | x  |
 | 1 | oh   | 0.00803615 |
 | 2 | yeah | 0.0222873  |



 julia append!(df2,df)


 4x2 DataFrame
 |---|-||
 | Row # | a   | x  |
 | 1 | oh| 0.00803615 |
 | 2 | yeah  | 0.0222873  |
 | 3 | hi| 0.862957   |
 | 4 | there | 0.101378   |
 ~~~



 On Fri, Sep 12, 2014 at 7:57 AM, Florian Oswald florian.osw...@gmail.com
 wrote:

 i'm trying to do this:

 using DataFrames
 df = DataFrame(a=[hi,there],x = rand(2))
 df2 = DataFrame(a=[oh,yeah],x = rand(2))

 for e in eachrow(df)
 append!(df2,e)
 end

 ERROR: `append!` has no method matching append!(::DataFrame,
 ::DataFrameRow{DataFrame})
 in anonymous at no file:2

 or

 julia for i in 1:nrow(df)
 push!(df2,df[i,:])
 end

 but that errors as well.

 this works:

 julia for i in 1:nrow(df)
 push!(df2,array(df[i,:]))
 end

 but wondering whether that's the best way of achieving this efficiently.

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

[Florian Oswald]

Leah: yeah that works. but i think i almost prefer my previous solution,
instead of this
push!(df2,[v for (_,v) in e])
that:
push!(df2,array(e))

not sure about the performance implications though.




On 12 September 2014 22:18, Gray Calhoun gcalh...@iastate.edu wrote:

 Oh, I wasn't thinking of that. Good point. A mutating OrderedDict
 constructor would allow reuse, but isn't as generic.

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

[Florian Oswald]

oh, i didnt' know it's slow. yes in my case it's a way of transferring a
row from one df to another. what's a better way of doing this?

On 12 September 2014 22:39, John Myles White johnmyleswh...@gmail.com
wrote:

 What does that mean? A DataFrameRow can't be easily created without
 reference to an existing DataFrame, so this seems like it's either a
 mechanism for transferring rows from one DataFrame to another very slowly
 or a mechanism for inserting duplicate rows.

  -- John

 On Sep 12, 2014, at 3:37 PM, Florian Oswald florian.osw...@gmail.com
 wrote:

 I'll submit a PR for Base.append!(adf::AbstracDataFrame,dfr::DataFrameRow)
 unless you tell me that's useless.

 On 12 September 2014 22:31, Florian Oswald florian.osw...@gmail.com
 wrote:

 Leah: yeah that works. but i think i almost prefer my previous solution,
 instead of thispush!(df2,[v for (_,v) in e])
 that:
 push!(df2,array(e))

 not sure about the performance implications though.




 On 12 September 2014 22:18, Gray Calhoun gcalh...@iastate.edu wrote:

 Oh, I wasn't thinking of that. Good point. A mutating OrderedDict
 constructor would allow reuse, but isn't as generic.

[julia-users] how to turn off redirect_stdout() ?

[Florian Oswald]

hi!

this works fine:

julia rdstdout, wrstdout = redirect_stdout()
(Pipe(open, 0 bytes waiting),Pipe(open, 0 bytes waiting))


*run(`date`)**julia *
*s = readavailable(rdstdout)**Tue  2 Sep 2014 17:11:33 BST\n*


*but now I want to turn the redirection off again. how do I do this?*

thanks!

Re: [julia-users] Re: how to turn off redirect_stdout() ?

[Florian Oswald]

brilliant! thanks!


On 2 September 2014 19:09, David Gonzales dvdgonzale...@gmail.com wrote:

 i tried saving the old STDOUT before redirecting and then using the saved
 stream to redirect output back to normal. in code:

 oldout = STDOUT
 (rd,wr) = redirect_stdout()
 println(hello world)
 readline(rd)
 redirect_stdout(oldout)
 println(hello world)

 (with the last line outputting to screen normally)

 On Tuesday, September 2, 2014 7:25:05 PM UTC+3, Florian Oswald wrote:

 hi!

 this works fine:

 julia rdstdout, wrstdout = redirect_stdout()
 (Pipe(open, 0 bytes waiting),Pipe(open, 0 bytes waiting))


 *run(`date`)**julia *
 *s = readavailable(rdstdout) **Tue  2 Sep 2014 17:11:33 BST\n*


 *but now I want to turn the redirection off again. how do I do this?*

 thanks!

Re: [julia-users] what is Julia for R cut()

[Florian Oswald]

thanks john.
say, I would find a comprehensive Julia dictionary for R users extremely
helpful. your https://github.com/johnmyleswhite/JuliaVsR seems just the
perfect place for this. would you accept PRs that add for example

R   julia
cutDataArrays.cut

to the readme.md?
I realize you set this up for Base Julia and probably with performance
comparisons in mind. Also there may be situations where several
packages/functions do the same thing. But it's hard for the user to find
find simple things like that I think. let me know what you think.




On 29 August 2014 22:29, John Myles White johnmyleswh...@gmail.com wrote:

 DataArrays has a cut function.

  -- John

 On Aug 29, 2014, at 11:17 AM, Florian Oswald florian.osw...@gmail.com
 wrote:

 hi

 what's the julia equivalent of this R call? i don't need the levels and
 labels, just some kind of grouping index.

  cut(sample(1:10,10,TRUE),c(0,3,6,10))

 [1] (3,6]  (6,10] (0,3]  (6,10] (3,6]  (6,10] (0,3]  (0,3]  (3,6]  (3,6]

 Levels: (0,3] (3,6] (6,10]

[julia-users] Re: best way to run parallel julia in batch mode?

[Florian Oswald]

sorry correct my call to

julia -p 32 exper.jl  myout.out

(without the first  , not sure that makes a difference)

On Saturday, 30 August 2014 12:58:57 UTC+1, Florian Oswald wrote:

 @require should work for what you want. i usually run batch jobs like this

 julia -p 32  exper.jl  myout.out

 maybe give it a try?
 also, do you have 32 CPUs? not sure how stable this is if you use plenty 
 more processes than cores.

 here is a working example for a large cluster:
 https://github.com/floswald/parallelTest/tree/master/julia/iridis

 the setup is different, but you should be able to figure out from sge.jl 
 how I load the functions. make sure you are in the right directory?

 On Saturday, 30 August 2014 04:01:00 UTC+1, Travis Porco wrote:

 julia versioninfo()
 Julia Version 0.3.1-pre+405
 Commit 444fafe* (2014-08-27 20:11 UTC)
 Platform Info:
   System: Linux (x86_64-linux-gnu)
   CPU: Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
   WORD_SIZE: 64
   BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge)
   LAPACK: libopenblas
   LIBM: libopenlibm
   LLVM: libLLVM-3.3


 On Friday, August 29, 2014 10:15:54 AM UTC-7, Travis Porco wrote:

 Hello--I'd like to be able to run something like this:
 nohup ../julia/julia -p 32  mscript.jl
 where inside mscript.jl, I would like each worker to read in and have 
 access to a large script (something like require(analysis.jl) )
 and then call a function defined in my own file, nside which various 
 pieces of a computation are done in parallel.
 Does anyone have a working example? Nothing I have tried has worked (I 
 must have just misunderstood the manual). 
 Thanks.

[julia-users] Re: Logging output in batch mode

[Florian Oswald]

try standard unix cat 

julia script.jl  output.txt

On Friday, 29 August 2014 17:12:37 UTC+1, Thomas Covert wrote:

 Suppose I wanted to run a julia script in batch mode, like this

 bash$ julia script.jl

 How would I tell julia to save the visual output of this to a text file? 
  As far as I can tell, julia script.jl only sends println() statements 
 and the final statement in script.jl to STDOUT.  Is there a way to get 
 behavior more similar to the output of interactive mode?

 That is, if I had run julia interactively, and typed 

 julia include(script.jl)

 I'd see all the commands that script.jl executes, anything that is printed 
 using println, and the value of the last expression.  How can I replicate 
 this behavior in batch mode?

[julia-users] Re: Installing Julia on Mac

[Florian Oswald]

any reason you are using homebrew to install julia? both the build and 
precompiled binary are straightforward to install, I would go with that.

On Saturday, 30 August 2014 10:26:34 UTC+1, idontgetoutmuch wrote:



 I am trying to install Julia using brew

 brew install --HEAD --64bit julia

 and getting

 == Using DSFMT: /Library/Caches/Homebrew/julia--dsfmt-2.2.tar.gz
 == make -C contrib -f repackage_system_suitesparse4.make 
 prefix=/usr/local/Cellar/julia/HEAD USE
 clang++ -stdlib=libc++ -mmacosx-version-min=10.7 -shared -Xlinker 
 -all_load /usr/local/opt/suite-sparse-julia/lib/libsuitesparseconfig.a 
 /usr/local/opt/suite-sparse-julia/lib/libspqr.a   -o 
 /private/tmp/julia-cXk8/usr/lib/libspqr.dylib 
 -L/private/tmp/julia-cXk8/usr/lib -L/usr/local/opt/suite-sparse-julia/lib 
 -L/usr/local/opt/arpack-julia/lib -L/usr/local/opt/openblas-julia/lib 
 -L/usr/local/opt/llvm33-julia/lib -L/usr/local/opt/libffi/lib 
 -L/usr/local/opt/cloog018-julia/lib -L/usr/local/opt/isl011-julia/lib 
 -L/usr/local/opt/gmp4-julia/lib -L/usr/local/lib -F/usr/local/Frameworks 
 -Wl,-headerpad_max_install_names -headerpad_max_install_names -lcholmod 
 -lcolamd -lamd -lopenblas -Wl,-rpath,'@loader_path/'  \
 install_name_tool -id @rpath/libspqr.dylib 
 /private/tmp/julia-cXk8/usr/lib/libspqr.dylib
 ld: file not found: 
 /usr/local/lib/gcc/x86_64-apple-darwin13.3.0/4.9.1/libgfortran.3.dylib for 
 architecture x86_64
 clang: error: linker command failed with exit code 1 (use -v to see 
 invocation)
 make: *** [default] Error 1

 READ THIS: https://github.com/Homebrew/homebrew/wiki/troubleshooting
 /usr/local/Library/Homebrew/exceptions.rb:182:in `dump': undefined local 
 variable or method `f' for #BuildError:0x007fc004d17da8 (NameError)
 from /usr/local/Library/brew.rb:158:in `rescue in main'
 from /usr/local/Library/brew.rb:66:in `main'

 I am not really sure what this is telling me.

Re: [julia-users] what is Julia for R cut()

[Florian Oswald]

ok sounds good! that gh-page would be awesome.


On 30 August 2014 21:38, John Myles White johnmyleswh...@gmail.com wrote:

 I think we should find a better home for that material. There’s also a
 more up-to-date list in this Google Doc:
 https://docs.google.com/spreadsheet/ccc?key=0Ai9cmDERDCGgdDJ6VDQtQjBGWm5LTzh6R0lRNHY1RVEusp=sharing

 I’d prefer setting up a special website (possibly using Github Pages) that
 houses information for R users coming to Julia. That way we’d be able to
 house more than just spreadsheets.

  — John

 On Aug 30, 2014, at 4:35 AM, Florian Oswald florian.osw...@gmail.com
 wrote:

 thanks john.
 say, I would find a comprehensive Julia dictionary for R users extremely
 helpful. your https://github.com/johnmyleswhite/JuliaVsR seems just the
 perfect place for this. would you accept PRs that add for example

 R   julia
 cutDataArrays.cut

 to the readme.md?
 I realize you set this up for Base Julia and probably with performance
 comparisons in mind. Also there may be situations where several
 packages/functions do the same thing. But it's hard for the user to find
 find simple things like that I think. let me know what you think.




 On 29 August 2014 22:29, John Myles White johnmyleswh...@gmail.com
 wrote:

 DataArrays has a cut function.

  -- John

 On Aug 29, 2014, at 11:17 AM, Florian Oswald florian.osw...@gmail.com
 wrote:

 hi

 what's the julia equivalent of this R call? i don't need the levels and
 labels, just some kind of grouping index.

  cut(sample(1:10,10,TRUE),c(0,3,6,10))

 [1] (3,6]  (6,10] (0,3]  (6,10] (3,6]  (6,10] (0,3]  (0,3]  (3,6]  (3,6]

 Levels: (0,3] (3,6] (6,10]

[julia-users] what is Julia for R cut()

[Florian Oswald]

hi

what's the julia equivalent of this R call? i don't need the levels and 
labels, just some kind of grouping index.

 cut(sample(1:10,10,TRUE),c(0,3,6,10))

[1] (3,6]  (6,10] (0,3]  (6,10] (3,6]  (6,10] (0,3]  (0,3]  (3,6]  (3,6] 

Levels: (0,3] (3,6] (6,10]

Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes

[Florian Oswald]

no problem about adding this to clustermanagers.jl. just one question: is
that repo still maintained? it seemed there was very little activity there
recently. i can submit a PR there if that's the preferred solution.


On 28 August 2014 00:38, Stefan Karpinski ste...@karpinski.org wrote:

 That's cool. It would be great if we can extract the parts of this that
 are not specific to the particular system you're running on and generic to
 PBS and added it to the ClusterManagers package. Thanks also for reporting
 your success story – it's always nice to hear them, regardless of magnitude
 :-)


 On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com
 wrote:

 Dear All,

 after bugging this list long enough with questions about how to get Julia
 running in parallel on a Torque/PBS managed cluster I thought I'd share my
 experience with the list. I realise that by julia standards this is a
 rather modest achievement, but I'd been happy to come across something like
 this post a while ago. :-)

 So, there's nothing special about the cluster being PBS managed (rather
 than SGE or whatever), I just found that each system is as idiosyncratic as
 the sysadmin person who set it up (starting from the format of nodenames to
 scheduler options to how environment variables get forwarded into a node,
 etc etc), so it always takes a fair amount of hacking to get something
 running. It's pretty low quality hacking I would say, but it's painful on a
 cluster. You basically need to adapt the functions in iridis_launcher.jl in
 the below repo to your system. Very few of the issues actually had anything
 to do with Julia itself, so I tried to explain as much about the
 environment as possible. Again, just sharing this in the hope someone out
 there is trying to achieve something similar may find this useful:

 https://github.com/floswald/parallelTest/tree/master/julia/iridis

Re: [julia-users] Sharing success: running Julia on PBS cluster across compute nodes

[Florian Oswald]

I'm mentioning this because we there's a dangling issue on the topic since
24 of May:

https://github.com/nlhepler/ClusterManagers.jl/issues/13


On 28 August 2014 10:53, Florian Oswald florian.osw...@gmail.com wrote:

 no problem about adding this to clustermanagers.jl. just one question: is
 that repo still maintained? it seemed there was very little activity there
 recently. i can submit a PR there if that's the preferred solution.


 On 28 August 2014 00:38, Stefan Karpinski ste...@karpinski.org wrote:

 That's cool. It would be great if we can extract the parts of this that
 are not specific to the particular system you're running on and generic to
 PBS and added it to the ClusterManagers package. Thanks also for reporting
 your success story – it's always nice to hear them, regardless of magnitude
 :-)


 On Wed, Aug 27, 2014 at 6:38 PM, Florian Oswald florian.osw...@gmail.com
  wrote:

 Dear All,

 after bugging this list long enough with questions about how to get
 Julia running in parallel on a Torque/PBS managed cluster I thought I'd
 share my experience with the list. I realise that by julia standards this
 is a rather modest achievement, but I'd been happy to come across something
 like this post a while ago. :-)

 So, there's nothing special about the cluster being PBS managed (rather
 than SGE or whatever), I just found that each system is as idiosyncratic as
 the sysadmin person who set it up (starting from the format of nodenames to
 scheduler options to how environment variables get forwarded into a node,
 etc etc), so it always takes a fair amount of hacking to get something
 running. It's pretty low quality hacking I would say, but it's painful on a
 cluster. You basically need to adapt the functions in iridis_launcher.jl in
 the below repo to your system. Very few of the issues actually had anything
 to do with Julia itself, so I tried to explain as much about the
 environment as possible. Again, just sharing this in the hope someone out
 there is trying to achieve something similar may find this useful:

 https://github.com/floswald/parallelTest/tree/master/julia/iridis

[julia-users] Sharing success: running Julia on PBS cluster across compute nodes

[Florian Oswald]

Dear All,

after bugging this list long enough with questions about how to get Julia 
running in parallel on a Torque/PBS managed cluster I thought I'd share my 
experience with the list. I realise that by julia standards this is a 
rather modest achievement, but I'd been happy to come across something like 
this post a while ago. :-)

So, there's nothing special about the cluster being PBS managed (rather 
than SGE or whatever), I just found that each system is as idiosyncratic as 
the sysadmin person who set it up (starting from the format of nodenames to 
scheduler options to how environment variables get forwarded into a node, 
etc etc), so it always takes a fair amount of hacking to get something 
running. It's pretty low quality hacking I would say, but it's painful on a 
cluster. You basically need to adapt the functions in iridis_launcher.jl in 
the below repo to your system. Very few of the issues actually had anything 
to do with Julia itself, so I tried to explain as much about the 
environment as possible. Again, just sharing this in the hope someone out 
there is trying to achieve something similar may find this useful:

https://github.com/floswald/parallelTest/tree/master/julia/iridis

[julia-users] question about performance hit because of build options on unix HPC

[Florian Oswald]

Hi!

I'm running julia release 0.3 on an hpc system that features two 6-core 
2.4GHz Intel Westmere processors (a total of 12 processor cores) per node. 
I noticed my program is significantly slower on that machine than on my 
mac, where I run this on 1 core (2.4GHz Intel Core i5). (runtime is 60 secs 
vs 90 secs)

I would expect the hpc cores to be slightly slower but was surprised it was 
that much. I was wondering if any of the options I set to build this are 
culprit for this? I do run the binary of release 0.3 on my mac, whereas I 
build from source with those options on the hpc:

OPENBLAS_TARGET_ARCH=NEHALEM 
OPENBLAS_DYNAMIC_ARCH = 0 
OPENBLAS_USE_THREAD=0 
JULIA_CPU_TARGET=core2

I can build julia without the final option. I cannot build it without the 
first three options.

I cannot run multiple nodes without the final option, since I get an error 

Target architecture mismatch. Please delete or regenerate 
sys.{so,dll,dylib}.

Re: [julia-users] Suggestion: Start using Julia 0.3-RC4

[Florian Oswald]

hey!

yeah no worries! when I overcame my impatience (which led me to abort the
download after 2 seconds because of the wrong name), I found out that the
package had the correct content.

cheers


On 19 August 2014 16:54, Elliot Saba staticfl...@gmail.com wrote:

 Whoops, sorry about that Florian!  There must be a small error in the .dmg
 filename, but if you open it, you see there is a Julia-0.3.0-rc4.app inside.
 -E


 On Tue, Aug 19, 2014 at 9:08 AM, Ivar Nesje iva...@gmail.com wrote:

 #7846 https://github.com/JuliaLang/julia/pull/7846

 and

 Base.runtests
 http://docs.julialang.org/en/latest/stdlib/test/?highlight=runtests#Base.runtests

 kl. 14:52:00 UTC+2 tirsdag 19. august 2014 skrev Johan Sigfrids følgende:

 Is there a way for a user that downloads the RC4 binary to run the test
 suit? That would be a good way to help uncover those pesky bugs.

 On Monday, August 18, 2014 6:36:47 PM UTC+3, Stefan Karpinski wrote:

 +1,000,000


 On Mon, Aug 18, 2014 at 11:25 AM, John Myles White 
 johnmyl...@gmail.com wrote:

 One of the best things you can do for the Julia community is to start
 using the release candidates of 0.3. Everyone will be better off if bugs
 are found now rather than when 0.3 is released. You can help make 0.3 a
 better release by installing one of the latest binaries and testing it 
 out.

 This is especially important for projects like DataFrames, which have
 fewer developer hours to spare on issues related to 0.2.

  — John

[julia-users] help with improving performance of nested loops

[Florian Oswald]

Hi all,

i'm trying to improve the performance of this function:

https://gist.github.com/floswald/9e79f6f51c276becbd74

In a nutshell, I have got a high-dimensional array vbar (in this instance 
it is 9D), and I want to obtain another array EV (also 9D), by 
matrix-multiplying several dimensions of vbar with transition matrices - 
they are prefixed with G... in the function. The G's are square matrices, 
where row i has the probability of moving from state i to state j. (some of 
those matrices are actually 3D because they depend on an additional state, 
but that doesn't matter.)

I devectorized it already but wanted to see whether anyone out there has a 
suggestion.

thanks!
florian

Re: [julia-users] help with improving performance of nested loops

[Florian Oswald]

Hi!

yes. I find that

37 % of time spent at line 26
51 % of time spent at line 29

in the gist.

line 26  is

idx1 = idx9(is1,iz1,iy1,ip1,itau1,ia,ih,ij,age,p)

line 29 is

@inbounds tmp += m.vbar[idx1] * Gz[iz + p.nz * (iz1 + p.nz *
(ij-1)-1)] * Gyp[iy + p.ny * ((ip-1) + p.np * ((iy1-1) + p.ny *
((ip1-1) + p.np * (ij-1 ] * Gs[is + p.ns * (is1-1)] * Gtau[itau1]






On 18 August 2014 19:13, Kevin Squire kevin.squ...@gmail.com wrote:

 Have you run it through the profiler already?


 On Monday, August 18, 2014, Florian Oswald florian.osw...@gmail.com
 wrote:

 Hi all,

 i'm trying to improve the performance of this function:

 https://gist.github.com/floswald/9e79f6f51c276becbd74

 In a nutshell, I have got a high-dimensional array vbar (in this instance
 it is 9D), and I want to obtain another array EV (also 9D), by
 matrix-multiplying several dimensions of vbar with transition matrices -
 they are prefixed with G... in the function. The G's are square matrices,
 where row i has the probability of moving from state i to state j. (some of
 those matrices are actually 3D because they depend on an additional state,
 but that doesn't matter.)

 I devectorized it already but wanted to see whether anyone out there has
 a suggestion.

 thanks!
 florian

Re: [julia-users] help with improving performance of nested loops

[Florian Oswald]

sounds intriguing! will have a look at that TensorOperations.jl.
the size of the array is

*julia **size(m.EV)*

*(2,4,3,3,2,17,2,9,29)*


On 18 August 2014 22:27, Jutho juthohaege...@gmail.com wrote:

 What are the dimensions (i.e. sizes) of these 9 dimensions? You might be
 interested in trying the tensorcontract routine from TensorOperations.jl
 and compare the method=:BLAS vs method=:native approach. Although I do
 assume that for a specific case like this (where basically every dimension
 is individually contracted with a matrix) there might be better approaches.


 Op maandag 18 augustus 2014 22:35:25 UTC+2 schreef Florian Oswald:

 Hi!

 yes. I find that

 37 % of time spent at line 26
 51 % of time spent at line 29

 in the gist.

 line 26  is

 idx1 = idx9(is1,iz1,iy1,ip1,itau1,ia,ih,ij,age,p)



 line 29 is

 @inbounds tmp += m.vbar[idx1] * Gz[iz + p.nz * (iz1 + p.nz * (ij-1)-1)] * 
 Gyp[iy + p.ny * ((ip-1) + p.np * ((iy1-1) + p.ny * ((ip1-1) + p.np * 
 (ij-1 ] * Gs[is + p.ns * (is1-1)] * Gtau[itau1]








 On 18 August 2014 19:13, Kevin Squire kevin@gmail.com wrote:

 Have you run it through the profiler already?


 On Monday, August 18, 2014, Florian Oswald florian...@gmail.com wrote:

 Hi all,

 i'm trying to improve the performance of this function:

 https://gist.github.com/floswald/9e79f6f51c276becbd74

 In a nutshell, I have got a high-dimensional array vbar (in this
 instance it is 9D), and I want to obtain another array EV (also 9D), by
 matrix-multiplying several dimensions of vbar with transition matrices -
 they are prefixed with G... in the function. The G's are square matrices,
 where row i has the probability of moving from state i to state j. (some of
 those matrices are actually 3D because they depend on an additional state,
 but that doesn't matter.)

 I devectorized it already but wanted to see whether anyone out there
 has a suggestion.

 thanks!
 florian

[julia-users] Re: what's the best way to do R table() in julia? (why does StatsBase.count(x,k) need k?)

[Florian Oswald]

I see! thanks.

On Sunday, 17 August 2014 14:56:29 UTC+1, Johan Sigfrids wrote:

 I think countmap comes closest to giving you what you want:

 using StatsBase
 data = sample([a, b, c], 20)
 countmap(data)


 Dict{ASCIIString,Int64} with 3 entries:
   c = 3
   b = 10
   a = 7


 On Sunday, August 17, 2014 4:45:21 PM UTC+3, Florian Oswald wrote:

 Hi

 I'm looking for the best way to count how many times a certain value x_i 
 appears in vector x, where x could be integers, floats, strings. In R I 
 would do table(x). I found StatsBase.counts(x,k) but I'm a bit confused by 
 k (where k goes into 1:k, i.e. the vector is scanned to find how many 
 elements locate at each point of 1:k). most of the times I don't know k, 
 and in fact I would do table(x) just to find out what k was. Apart from 
 that, I don't think I could use this with strings, as I can't construct a 
 range object from strings.

 I'm wondering whether a method StatsBase.counts(x::Vector) just returning 
 the frequency of each element appearing would be useful.

 The same applies to Base.hist if I understand correctly. I just don't 
 want to have to specify the edges of bins.

Re: [julia-users] Re: help with improving performance of a matrix multiplication

[Florian Oswald]

Hi Tim,

that *did* make a massive difference. in my test I went from 62 secs to 12
secs. just for completeness, your snippet was missing that stemp is a
matrix, so now looks like this:

for i in 1:m, j=1:n
vtmp = 0.0
for ji=1:n
 vtmp += stemp[j,ji] * v0[n*(i-1) + ji]
end
v1[m*(j-1) + i] = vtmp
 end

thanks!



On 8 August 2014 11:58, Tim Holy tim.h...@gmail.com wrote:

 Did you already try devectorizing the computation?
 stemp * v0[(n*(i-1)) + (1:n)]
 allocates memory, which is bad. (Does ProfileView show a red bar at this
 line?)
 You want to write that loop as
 for i in 1:m, j = 1:n
 v1[m*(j-1) + i] = stemp * v0[n*(i-1) + j]
end

 --Tim

 On Friday, August 08, 2014 03:43:26 AM Florian Oswald wrote:
  thanks for all your tips guys - sad to report that none of them worked
 out
  so far.
 
  Andrei: I checked I'm using Blas. called gemv directly as well.
  Stefan: removing the v0 = copy(v1) changes the algorithm. in the
 innermost
  loop I need the value of the previous v1, not the one currently being
  updated. Notice that the indices on the left hand side of the innermost
  loop are not full strides, but increase in integer steps.
  That's also where I don't understand Dahua's suggestion: I don't know how
  to achieve this operation with a matrix multiplication. I also tried out
  the ArrayViews function which seemed to be faster but it didn't deliver
 the
  correct results, not sure if and where I made a mistake. (see comments in
  below code) I think the biggest problem is that I have to do many
  non-contiguous index jumps.
 
  here is the repo with that function, also has tests indicating that the
  function computes the correct result.
 
  https://github.com/floswald/BSplines.jl/blob/master/src/approx.jl
 
  in case you are wondering what on earth I'm doing here, I'm trying to
  implement a version of the algorithm described in
 
  [De Boor(1979)] C. De Boor. Efficient computer manipulation of tensor
  products. ACM Transactions on Mathematical Software (TOMS), 5(2):173–182,
  1979.
 
  this is almost impossible to find online (and for free), so I attach a
 pdf
  in case anyone wants to have a look.
 
  cheers
 
  florian
 
  On Wednesday, 30 July 2014 01:17:16 UTC+1, Dahua Lin wrote:
   Look more carefully into the pattern of your codes.
  
   It seems that you may be able to reshape v1 into a matrix of size (m,
 n)
   and v0 into a matrix of size (n, m), and you may do a matrix-matrix
   multiplication once without looping over multiple strided vectors.
  
   Also, if you are using ArrayViews, you can write view(v0,
 n*(i-1)+1:n*i)
   instead of v0[(n * (i-1)) + (1:n)].
  
   Dahua
  
   On Tuesday, July 29, 2014 4:22:32 PM UTC-5, Florian Oswald wrote:
   Hi all,
  
   I've got an algorithm that hinges critically on fast matrix
   multiplication. I put up the function on this gist
  
  
 https://gist.github.com/floswald/6dea493417912536688d#file-tensor-jl-L45
  
   indicating the line (45) that takes most of the time, as you can see
 in
   the profile output that is there as well. I am trying to figure out if
   I'm
   doing something wrong here or if that line just takes as long as it
   takes.
   I have to do this many times, so if this takes too long I have to
 change
   my
   strategy.
  
   The core of the problem looks like that
  
   for imat in 2:nbm
  
   v0= copy(v1)
   stemp = ibm[ks[imat]]
   n = size(stemp,1)
   m = nall / n
   for i in 1:m
  
   v1[m*(0:(n-1)) + i] = stemp * v0[(n*(i-1)) + (1:n)]
  
  end
  
   end
  
  
   where v are vectors and stemp is a matrix. I spend a lot of time in
 the
   matrix multiplication line on the innermost loop. Any suggestions
 would
   be
   much appreciated. Thanks!

[julia-users] Re: Run Julia job on several workers on a cluster

[Florian Oswald]

Hi Ken,

sorry can i just ask you a question on this? Thibaut (post below) was 
actually able to use your function to set Julia up on our departmental SGE 
cluster and it works fine. That facility is down with a disk failure for 
the time being though, so I have been trying to get going on a different 
PBS cluster. I can do interactive on one node, but I am stuck going across 
nodes. 
I'm not sure I understand what you mean when you say convert to IB 
interface names. The names on my PBS_NODEFILE are the same for each node, 
i.e. all CPUs on node 1 are called node 1. 

I wrote this up in more detail in this SO 
post. 
http://stackoverflow.com/questions/25089733/julia-on-pbs-cluster-what-to-give-to-addprocs

It ends up throwing an SSH error. Do you have any clue what might be going 
wrong here? Thanks!

Florian

On Wednesday, 7 May 2014 20:52:17 UTC+1, ken...@sdsc.edu wrote:


 I've run parallel julia on a Torque cluster with Infiniband.  I start an 
 interactive session with qsub -I,
 look for allocated nodes in $PBS_NODEFILE, convert to IB interface names, 
 and addprocs.

 filestream = open(ENV[PBS_NODEFILE])
 seekstart(filestream)
 linearray = readlines(filestream)
 strippedarray = similar(linearray)
 for i in 1:length(linearray)
 strippedarray[i] = strip(linearray[i]) * -ipoib.ipoib
 end
 for i in 1:length(strippedarray)
  singlearray = [strip(strippedarray[i])]
  addprocs(singlearray)
 end
 print(workers())

 To start an interactive job, depending on your node configuration and 
 queue names:
 qsub -I -l nodes=2:ppn=32,walltime=00:30:00 -q normal

 When you get your nodes, start julia with the above setup file with:
 julia --load setupfilename

 This should addprocs then give you the julia prompt.

 But it looks like something is wrong with your modules?

 On Friday, April 25, 2014 5:09:57 AM UTC-7, Isaac wrote:

 Hi All,
  
  I also tried to submit the julia jobs on the cluster but failed. I wrote 
 the job script as follows:
 f










 *or((i = 1; i  10; i++))doecho # cd /data#PBS -l 
 walltime=00:10:00module add gcc/4.7.2module add julia/0.2.0module load 
 juliainclude(test.jl)test($i)test1job$i;qsub test1job$i;done*
 I got the errors: 
 julia/0.2.0(16):ERROR:151: Module 'julia/0.2.0' depends on one of the 
 module(s) 'gcc/4.7.2'
 julia/0.2.0(16):ERROR:102: Tcl command execution failed: prereq gcc/4.7.2

 /cm/local/apps/torque/current/mom_priv/jobs/1053.cluster.SC: line 7: 
 syntax error near unexpected token `a0d0.jl'
 /cm/local/apps/torque/current/mom_priv/jobs/1053.cluster.SC: line 7: 
 `include(a0d0.jl)'

 Does anybody know how to write the job script to submit julia job on a 
 cluster? Could you give an example?
 Thanks in advance.

 Isaac

  

Re: [julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

that does sound worrying. I doubt the admin wants to know what I'm
downloading but rather get (temporarily) rid of a problem. Does that
compromise the security of the hpc system or does it mean someone could
hack my github account?


On 1 August 2014 08:02, gael.mc...@gmail.com wrote:

 Sounds like a bad idea. If the SSL cert is not correct in your
 configuration (whereas it is in the outside world) it becomes clear that
 your admin just want to know what you are downloading.

 Basically, he told you Please let us perform MITM attack on your
 connexion. To make our job easier, please desactivate all the SSL checks so
 that our cert (and maybe others) are accepted.

 The extra downside is that now anyone can alter the data you are
 downloading and you won't have the slightest idea this is happening (if
 that happens).

Re: [julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

ok, thanks Ivar. I am not worried about the sysadmin at all. I would have
been worried if the system is shut down by an attack that enters through my
door, but I was advised to turn SSL checking off anyway. I agree that there
seems little to gain from attacking a facility like that (pure research
a.k.a. random guys software doing fun things :-) ).
thanks for chipping in gael as well - it's beyond me judging those things,
so good to hear from you guys.

florian


On 1 August 2014 12:34, Ivar Nesje iva...@gmail.com wrote:

 You are at a security level where it is acceptable to use random guys
 software downloaded from the internet, that explicitly says in the licence
 that you are offered NO guarantees for anything. The software is so complex
 that it is impossible for you to review even a fraction of it for security
 (or else you would have reported numerous bug reports when you discovered
 unrelated things along the way). Stefan and Jeff seems to be following code
 updates in the Julia repository closely, but I don't know how many others
 review every commit. They have also given away Push access to lots of
 random people, some whom he has never even met in person.

 I would not worry too much about the possibility that your sysadmin might
 change something that might compromise your system. He has access anyway.
 Large scale attackers (at the ISP layer) will probably attack protocols
 that are more used than git over https with a fake certificate. They will
 not know who verifies signatures and who does not.

 Ivar

 kl. 12:45:48 UTC+2 fredag 1. august 2014 skrev Florian Oswald følgende:

 that does sound worrying. I doubt the admin wants to know what I'm
 downloading but rather get (temporarily) rid of a problem. Does that
 compromise the security of the hpc system or does it mean someone could
 hack my github account?


 On 1 August 2014 08:02, gael@gmail.com wrote:

 Sounds like a bad idea. If the SSL cert is not correct in your
 configuration (whereas it is in the outside world) it becomes clear that
 your admin just want to know what you are downloading.

 Basically, he told you Please let us perform MITM attack on your
 connexion. To make our job easier, please desactivate all the SSL checks so
 that our cert (and maybe others) are accepted.

 The extra downside is that now anyone can alter the data you are
 downloading and you won't have the slightest idea this is happening (if
 that happens).

[julia-users] addprocs(machinefile) on a cluster with N nodes and ppn processors per node

[Florian Oswald]

Hi,

I'm trying to setup a cluster across machines on a PBS managed cluster. I 
parse the PBS_NODEFILE, which for the case of a submit script with option

#PBS -l nodes=2:ppn=12

looks like this: 

node001
node001
node001
...
node002
...
node002

I am unsure about what exactly I have to give to addprocs(). I don't want 
to add the master process, so I exclude ENV[HOST] = node001. but what 
about all processors with the same name node002? do i list all of those 

machines = [ node002 for i=1:12]

or just ones

machines = [node002]

in addprocs(machines)

?

[julia-users] Re: addprocs(machinefile) on a cluster with N nodes and ppn processors per node

[Florian Oswald]

here is the parsing function in case that helps:

https://github.com/floswald/mpitest/blob/removeMaster/julia/iridis/sge.jl

On Friday, 1 August 2014 15:03:49 UTC+1, Florian Oswald wrote:

 Hi,

 I'm trying to setup a cluster across machines on a PBS managed cluster. I 
 parse the PBS_NODEFILE, which for the case of a submit script with option

 #PBS -l nodes=2:ppn=12

 looks like this: 

 node001
 node001
 node001
 ...
 node002
 ...
 node002

 I am unsure about what exactly I have to give to addprocs(). I don't want 
 to add the master process, so I exclude ENV[HOST] = node001. but what 
 about all processors with the same name node002? do i list all of those 

 machines = [ node002 for i=1:12]

 or just ones

 machines = [node002]

 in addprocs(machines)

 ?

[julia-users] Re: addprocs(machinefile) on a cluster with N nodes and ppn processors per node

[Florian Oswald]

and here is the error message I get. there is something wrong with the 
ssh_exchange_identification? 

https://gist.github.com/floswald/236da440fb717c683e37




On Friday, 1 August 2014 15:18:07 UTC+1, Florian Oswald wrote:

 here is the parsing function in case that helps:

 https://github.com/floswald/mpitest/blob/removeMaster/julia/iridis/sge.jl

 On Friday, 1 August 2014 15:03:49 UTC+1, Florian Oswald wrote:

 Hi,

 I'm trying to setup a cluster across machines on a PBS managed cluster. I 
 parse the PBS_NODEFILE, which for the case of a submit script with option

 #PBS -l nodes=2:ppn=12

 looks like this: 

 node001
 node001
 node001
 ...
 node002
 ...
 node002

 I am unsure about what exactly I have to give to addprocs(). I don't want 
 to add the master process, so I exclude ENV[HOST] = node001. but what 
 about all processors with the same name node002? do i list all of those 

 machines = [ node002 for i=1:12]

 or just ones

 machines = [node002]

 in addprocs(machines)

 ?

Re: [julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

Hi again,
the sys admin got back to me saying it's ok to just turn SSL checking off.
I knew that hack but thought it's completely off limits (after reading the
SO thread). Anyway, setting this in .bashrc does the job:

export GIT_SSL_NO_VERIFY=true

best

florian


On 30 July 2014 23:27, Florian Oswald florian.osw...@gmail.com wrote:

 Hi Isaiah,

 just tried that but didnt work. it's again:

 error: SSL certificate problem, verify that the CA cert is OK.

 so this may be a problem with my system. thanks for checking back! I'll
 post any followups here.




 On 30 July 2014 22:47, Isaiah Norton isaiah.nor...@gmail.com wrote:

 I believe the following should work, provided you have github ssh keys
 configured (which seems to be the case):

 git config --global url.ssh://.insteadOf git://

 If you have a chance to test the above and it works, then we could add it
 as another suggestion to the README (especially for hpc and similar
 environments with tricky network configs). Temporarily renaming your
 ~/.julia folder is a safe way to do such a test.



 On Wed, Jul 30, 2014 at 5:01 PM, Florian Oswald florian.osw...@gmail.com
  wrote:

 Very helpful, thanks!
 I resorted to manually cloning all my packages, completely bypassing the
 package manager. That takes a while but it works.
 I tried hard to get this ssl thing going but it's difficult on a large
 hpc system. I'll go on begging the sys admins.


 On Wednesday, 30 July 2014, Isaiah Norton isaiah.nor...@gmail.com
 wrote:

 Ok, I see. The difference is that the first and third URLs (with the
 colon after the domain name) are scp-style addresses, so git uses SSH to
 connect [1]. At least for METADATA initialization, you can do this:

 Pkg.init(github.com:JuliaLang/METADATA.jl)


 However, I'm not sure what will happen when cloning packages. Perhaps
 some `insteadOf` rules can help.

 Regarding the original error, google suggests it may be due to a
 poorly-configured proxy, or missing CA certs. Stackoverflow has a large
 number of suggestions for possible solutions [2]

 [1] https://www.kernel.org/pub/software/scm/git/docs/git-push.html#URLS
 [2]
 http://stackoverflow.com/questions/3777075/ssl-certificate-rejected-trying-to-access-github-over-https-behind-firewall


 On Wed, Jul 30, 2014 at 1:47 PM, Florian Oswald 
 florian.osw...@gmail.com wrote:

 that didnt' work:

 git config --global url.https://.insteadOf git://
 notice that i don't just want to have https:// instead of git://
 (both don't work). i need a different call altogether, see below.


 On 30 July 2014 18:36, Isaiah Norton isaiah.nor...@gmail.com wrote:

 yes - search for insteadOf or https in the readme.


 On Wed, Jul 30, 2014 at 1:11 PM, Florian Oswald 
 florian.osw...@gmail.com wrote:

 hmm, I don't know. but - this works:

 git clone g...@github.com:tlamadon/Utils.git

 and this does not

 git clone git://github.com/JuliaLang/METADATA.jl

 but that does

 git clone g...@github.com:JuliaLang/METADATA.jl

 I see. is there a way to tell julia to use the latter version?



 On Wednesday, 30 July 2014 18:06:03 UTC+1, Ivar Nesje wrote:

 Do you clone the other repositories over https or the ssh based git
 protocol?

 kl. 19:03:06 UTC+2 onsdag 30. juli 2014 skrev Florian Oswald
 følgende:

 i'm working on a remote server that has

 git version 1.7.4.1

 I could use this git to build julia and clone a host of other
 repos, but when I want to do

 Pkg.status()

 it fails with

 error: SSL certificate problem, verify that the CA cert is OK.
 Details:

 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
 verify failed while accessing https://github.com/JuliaLang/
 METADATA.jl/info/refs

 is there anything special about the METADATA repo? is my git out
 of date?

 thanks.

[julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

i'm working on a remote server that has 

git version 1.7.4.1

I could use this git to build julia and clone a host of other repos, but 
when I want to do

Pkg.status() 

it fails with 

error: SSL certificate problem, verify that the CA cert is OK. Details:

error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify 
failed while accessing https://github.com/JuliaLang/METADATA.jl/info/refs

is there anything special about the METADATA repo? is my git out of date?

thanks.

[julia-users] Re: git SSL problems with METADATA.jl

[Florian Oswald]

hmm, I don't know. but - this works:

git clone g...@github.com:tlamadon/Utils.git

and this does not

git clone git://github.com/JuliaLang/METADATA.jl

but that does

git clone g...@github.com:JuliaLang/METADATA.jl

I see. is there a way to tell julia to use the latter version?



On Wednesday, 30 July 2014 18:06:03 UTC+1, Ivar Nesje wrote:

 Do you clone the other repositories over https or the ssh based git 
 protocol?

 kl. 19:03:06 UTC+2 onsdag 30. juli 2014 skrev Florian Oswald følgende:

 i'm working on a remote server that has 

 git version 1.7.4.1

 I could use this git to build julia and clone a host of other repos, but 
 when I want to do

 Pkg.status() 

 it fails with 

 error: SSL certificate problem, verify that the CA cert is OK. Details:

 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate 
 verify failed while accessing 
 https://github.com/JuliaLang/METADATA.jl/info/refs

 is there anything special about the METADATA repo? is my git out of date?

 thanks.

Re: [julia-users] Re: git SSL problems with METADATA.jl

[Florian Oswald]

that didnt' work:

git config --global url.https://.insteadOf git://
notice that i don't just want to have https:// instead of git:// (both
don't work). i need a different call altogether, see below.

On 30 July 2014 18:36, Isaiah Norton isaiah.nor...@gmail.com wrote:

 yes - search for insteadOf or https in the readme.


 On Wed, Jul 30, 2014 at 1:11 PM, Florian Oswald florian.osw...@gmail.com
 wrote:

 hmm, I don't know. but - this works:

 git clone g...@github.com:tlamadon/Utils.git

 and this does not

 git clone git://github.com/JuliaLang/METADATA.jl

 but that does

 git clone g...@github.com:JuliaLang/METADATA.jl

 I see. is there a way to tell julia to use the latter version?



 On Wednesday, 30 July 2014 18:06:03 UTC+1, Ivar Nesje wrote:

 Do you clone the other repositories over https or the ssh based git
 protocol?

 kl. 19:03:06 UTC+2 onsdag 30. juli 2014 skrev Florian Oswald følgende:

 i'm working on a remote server that has

 git version 1.7.4.1

 I could use this git to build julia and clone a host of other repos,
 but when I want to do

 Pkg.status()

 it fails with

 error: SSL certificate problem, verify that the CA cert is OK. Details:

 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
 verify failed while accessing https://github.com/JuliaLang/
 METADATA.jl/info/refs

 is there anything special about the METADATA repo? is my git out of
 date?

 thanks.

Re: [julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

Very helpful, thanks!
I resorted to manually cloning all my packages, completely bypassing the
package manager. That takes a while but it works.
I tried hard to get this ssl thing going but it's difficult on a large hpc
system. I'll go on begging the sys admins.

On Wednesday, 30 July 2014, Isaiah Norton isaiah.nor...@gmail.com wrote:

 Ok, I see. The difference is that the first and third URLs (with the colon
 after the domain name) are scp-style addresses, so git uses SSH to connect
 [1]. At least for METADATA initialization, you can do this:

 Pkg.init(github.com:JuliaLang/METADATA.jl)


 However, I'm not sure what will happen when cloning packages. Perhaps some
 `insteadOf` rules can help.

 Regarding the original error, google suggests it may be due to a
 poorly-configured proxy, or missing CA certs. Stackoverflow has a large
 number of suggestions for possible solutions [2]

 [1] https://www.kernel.org/pub/software/scm/git/docs/git-push.html#URLS
 [2]
 http://stackoverflow.com/questions/3777075/ssl-certificate-rejected-trying-to-access-github-over-https-behind-firewall


 On Wed, Jul 30, 2014 at 1:47 PM, Florian Oswald florian.osw...@gmail.com
 javascript:_e(%7B%7D,'cvml','florian.osw...@gmail.com'); wrote:

 that didnt' work:

 git config --global url.https://.insteadOf git://
 notice that i don't just want to have https:// instead of git:// (both
 don't work). i need a different call altogether, see below.


 On 30 July 2014 18:36, Isaiah Norton isaiah.nor...@gmail.com
 javascript:_e(%7B%7D,'cvml','isaiah.nor...@gmail.com'); wrote:

 yes - search for insteadOf or https in the readme.


 On Wed, Jul 30, 2014 at 1:11 PM, Florian Oswald 
 florian.osw...@gmail.com
 javascript:_e(%7B%7D,'cvml','florian.osw...@gmail.com'); wrote:

 hmm, I don't know. but - this works:

 git clone g...@github.com:tlamadon/Utils.git

 and this does not

 git clone git://github.com/JuliaLang/METADATA.jl

 but that does

 git clone g...@github.com:JuliaLang/METADATA.jl

 I see. is there a way to tell julia to use the latter version?



 On Wednesday, 30 July 2014 18:06:03 UTC+1, Ivar Nesje wrote:

 Do you clone the other repositories over https or the ssh based git
 protocol?

 kl. 19:03:06 UTC+2 onsdag 30. juli 2014 skrev Florian Oswald følgende:

 i'm working on a remote server that has

 git version 1.7.4.1

 I could use this git to build julia and clone a host of other repos,
 but when I want to do

 Pkg.status()

 it fails with

 error: SSL certificate problem, verify that the CA cert is OK.
 Details:

 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
 verify failed while accessing https://github.com/JuliaLang/
 METADATA.jl/info/refs

 is there anything special about the METADATA repo? is my git out of
 date?

 thanks.

Re: [julia-users] git SSL problems with METADATA.jl

[Florian Oswald]

Hi Isaiah,

just tried that but didnt work. it's again:

error: SSL certificate problem, verify that the CA cert is OK.

so this may be a problem with my system. thanks for checking back! I'll
post any followups here.




On 30 July 2014 22:47, Isaiah Norton isaiah.nor...@gmail.com wrote:

 I believe the following should work, provided you have github ssh keys
 configured (which seems to be the case):

 git config --global url.ssh://.insteadOf git://

 If you have a chance to test the above and it works, then we could add it
 as another suggestion to the README (especially for hpc and similar
 environments with tricky network configs). Temporarily renaming your
 ~/.julia folder is a safe way to do such a test.



 On Wed, Jul 30, 2014 at 5:01 PM, Florian Oswald florian.osw...@gmail.com
 wrote:

 Very helpful, thanks!
 I resorted to manually cloning all my packages, completely bypassing the
 package manager. That takes a while but it works.
 I tried hard to get this ssl thing going but it's difficult on a large
 hpc system. I'll go on begging the sys admins.


 On Wednesday, 30 July 2014, Isaiah Norton isaiah.nor...@gmail.com
 wrote:

 Ok, I see. The difference is that the first and third URLs (with the
 colon after the domain name) are scp-style addresses, so git uses SSH to
 connect [1]. At least for METADATA initialization, you can do this:

 Pkg.init(github.com:JuliaLang/METADATA.jl)


 However, I'm not sure what will happen when cloning packages. Perhaps
 some `insteadOf` rules can help.

 Regarding the original error, google suggests it may be due to a
 poorly-configured proxy, or missing CA certs. Stackoverflow has a large
 number of suggestions for possible solutions [2]

 [1] https://www.kernel.org/pub/software/scm/git/docs/git-push.html#URLS
 [2]
 http://stackoverflow.com/questions/3777075/ssl-certificate-rejected-trying-to-access-github-over-https-behind-firewall


 On Wed, Jul 30, 2014 at 1:47 PM, Florian Oswald 
 florian.osw...@gmail.com wrote:

 that didnt' work:

 git config --global url.https://.insteadOf git://
 notice that i don't just want to have https:// instead of git:// (both
 don't work). i need a different call altogether, see below.


 On 30 July 2014 18:36, Isaiah Norton isaiah.nor...@gmail.com wrote:

 yes - search for insteadOf or https in the readme.


 On Wed, Jul 30, 2014 at 1:11 PM, Florian Oswald 
 florian.osw...@gmail.com wrote:

 hmm, I don't know. but - this works:

 git clone g...@github.com:tlamadon/Utils.git

 and this does not

 git clone git://github.com/JuliaLang/METADATA.jl

 but that does

 git clone g...@github.com:JuliaLang/METADATA.jl

 I see. is there a way to tell julia to use the latter version?



 On Wednesday, 30 July 2014 18:06:03 UTC+1, Ivar Nesje wrote:

 Do you clone the other repositories over https or the ssh based git
 protocol?

 kl. 19:03:06 UTC+2 onsdag 30. juli 2014 skrev Florian Oswald
 følgende:

 i'm working on a remote server that has

 git version 1.7.4.1

 I could use this git to build julia and clone a host of other
 repos, but when I want to do

 Pkg.status()

 it fails with

 error: SSL certificate problem, verify that the CA cert is OK.
 Details:

 error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate
 verify failed while accessing https://github.com/JuliaLang/
 METADATA.jl/info/refs

 is there anything special about the METADATA repo? is my git out of
 date?

 thanks.

[julia-users] help with improving performance of a matrix multiplication

[Florian Oswald]

Hi all,

I've got an algorithm that hinges critically on fast matrix multiplication. 
I put up the function on this gist

https://gist.github.com/floswald/6dea493417912536688d#file-tensor-jl-L45

indicating the line (45) that takes most of the time, as you can see in the 
profile output that is there as well. I am trying to figure out if I'm 
doing something wrong here or if that line just takes as long as it takes. 
I have to do this many times, so if this takes too long I have to change my 
strategy.

The core of the problem looks like that

for imat in 2:nbm
v0= copy(v1)
stemp = ibm[ks[imat]]
n = size(stemp,1)
m = nall / n
for i in 1:m
v1[m*(0:(n-1)) + i] = stemp * v0[(n*(i-1)) + (1:n)]
   end
end


where v are vectors and stemp is a matrix. I spend a lot of time in the 
matrix multiplication line on the innermost loop. Any suggestions would be 
much appreciated. Thanks!

[julia-users] help with improving performance of a matrix multiplication

[Florian Oswald]

From the profile output it looks like a lot of time is spent in getindex. I
suppose that is bad news? Not sure how I could avoid any of the index
lookups.


On Tuesday, 29 July 2014, Dahua Lin linda...@gmail.com
javascript:_e(%7B%7D,'cvml','linda...@gmail.com'); wrote:

 You may have to check which is the bottleneck: getindex or matrix
 multiplication.

 Dahua

 On Tuesday, July 29, 2014 4:22:32 PM UTC-5, Florian Oswald wrote:

 Hi all,

 I've got an algorithm that hinges critically on fast matrix
 multiplication. I put up the function on this gist

 https://gist.github.com/floswald/6dea493417912536688d#file-tensor-jl-L45

 indicating the line (45) that takes most of the time, as you can see in
 the profile output that is there as well. I am trying to figure out if I'm
 doing something wrong here or if that line just takes as long as it takes.
 I have to do this many times, so if this takes too long I have to change my
 strategy.

 The core of the problem looks like that

 for imat in 2:nbm
 v0= copy(v1)
 stemp = ibm[ks[imat]]
 n = size(stemp,1)
 m = nall / n
 for i in 1:m
 v1[m*(0:(n-1)) + i] = stemp * v0[(n*(i-1)) + (1:n)]
end
 end


 where v are vectors and stemp is a matrix. I spend a lot of time in the
 matrix multiplication line on the innermost loop. Any suggestions would be
 much appreciated. Thanks!

Re: [julia-users] iterate over rows of sparse matrix and get index of nonzeros?

[Florian Oswald]

beautiful! can't wait.
cheers
florian


On 22 July 2014 10:25, Viral Shah vi...@mayin.org wrote:

 Julia 0.4 will have both compressed sparse column as well as compressed
 sparse row sparse matrices. We should probably write some iterators to work
 with cases such as these too. They will be convenient to use for sure, but
 may not give the best performance.

 -viral


 On Monday, July 21, 2014 12:15:46 PM UTC+5:30, Florian Oswald wrote:

 Transposing is fine! Thanks for that!

 On Sunday, 20 July 2014, Odd Andersen odd.ander...@gmail.com wrote:

 Sparse matrices in Julia are to my understanding stored as compressed
 sparse columns.  So it is very easy to get the nonzero elements for a given
 column, but not so easy for rows.

 To get the indices of nonzeros rows for column 'c' in sparse matrix M,
 one can use (at least in the current implementation):

  M.rowval[a.colptr[col] : M.colptr[col+1]-1]

 To do the same by rows would be more complicated (a quick-and-dirty
 solution would of course be to transpose your matrix first).
 I am however not a Julia expert, so perhaps there's a solution I am not
 aware of.

Re: [julia-users] iterate over rows of sparse matrix and get index of nonzeros?

[Florian Oswald]

Transposing is fine! Thanks for that!

On Sunday, 20 July 2014, Odd Andersen odd.ander...@gmail.com wrote:

 Sparse matrices in Julia are to my understanding stored as compressed
 sparse columns.  So it is very easy to get the nonzero elements for a given
 column, but not so easy for rows.

 To get the indices of nonzeros rows for column 'c' in sparse matrix M, one
 can use (at least in the current implementation):

  M.rowval[a.colptr[col] : M.colptr[col+1]-1]

 To do the same by rows would be more complicated (a quick-and-dirty
 solution would of course be to transpose your matrix first).
 I am however not a Julia expert, so perhaps there's a solution I am not
 aware of.

[julia-users] Pkg.update() fails: Key not found: Sampling

[Florian Oswald]

i'm getting this error here. any idea? I can't do Pkg.rm(GLM) - assuming 
that's where the problem is.



*julia **Pkg.update()*

*INFO: Updating METADATA...*

*INFO: Updating MOpt...*

*INFO: Updating StatsBase...*

*INFO: Updating Copulas...*

*INFO: Updating DataFrames...*

*INFO: Updating GaussQuadrature...*

*INFO: Updating GLM...*

*INFO: Computing changes...*

*ERROR: key not found: Sampling*

* in getindex at dict.jl:615*

* in dependencies_subset at pkg/query.jl:312*

* in prune_dependencies at pkg/query.jl:334*

* in resolve at pkg/entry.jl:379*

* in update at pkg/entry.jl:276*

* in anonymous at pkg/dir.jl:28*

* in cd at file.jl:20*

* in cd at pkg/dir.jl:28*

* in update at pkg.jl:41*

Re: [julia-users] Re: Pkg.update() fails: Key not found: Sampling

[Florian Oswald]

yeah it blocks everything. if you can work without distributions.jl, i
proceeded by wiping ~/.julia and starting from scratch. Dahua is currently
looking into this:
https://github.com/JuliaStats/Distributions.jl/issues/262


On 19 July 2014 15:03, Ryan B. Harvey ryan.b.har...@gmail.com wrote:

 I'm getting the same error.  This is making it really tough to get any
 work done, since I can't seem to add, remove or update packages now.


 On Saturday, July 19, 2014 7:47:05 AM UTC-4, Florian Oswald wrote:

 i'm getting this error here. any idea? I can't do Pkg.rm(GLM) -
 assuming that's where the problem is.



 *julia **Pkg.update()*

 *INFO: Updating METADATA...*

 *INFO: Updating MOpt...*

 *INFO: Updating StatsBase...*

 *INFO: Updating Copulas...*

 *INFO: Updating DataFrames...*

 *INFO: Updating GaussQuadrature...*

 *INFO: Updating GLM...*

 *INFO: Computing changes...*

 *ERROR: key not found: Sampling*

 * in getindex at dict.jl:615*

 * in dependencies_subset at pkg/query.jl:312*

 * in prune_dependencies at pkg/query.jl:334*

 * in resolve at pkg/entry.jl:379*

 * in update at pkg/entry.jl:276*

 * in anonymous at pkg/dir.jl:28*

 * in cd at file.jl:20*

 * in cd at pkg/dir.jl:28*

 * in update at pkg.jl:41*

[julia-users] iterate over rows of sparse matrix and get index of nonzeros?

[Florian Oswald]

i am looking for a way to do something like 


*s=sprand(3,5,0.6)*

*for i in 1:size(s,1)*

*  mySparseRowFunction(s[i, ])*

*end*

I guess I'm looking for an iterator over the nonzero elements, ideally by 
row. Does something like this exist? I know julia uses a different format, 
but I'm looking for something similar to Eigen's InnerIterator 
(http://eigen.tuxfamily.org/dox/group__TutorialSparse.html). 

Context: I have a very large tensor product of univariate B-Spline Basis 
functions - this forms a sparse matrix. I need to take each row of this 
product and multiply it with a vector.

Re: [julia-users] Re: ways to improve performance of a non-integer power?

[Florian Oswald]

hi simon,
very interesting to know indeed! I'll keep that in mind. thanks!


On 17 July 2014 13:10, Simon Byrne simonby...@gmail.com wrote:


 On Wednesday, 16 July 2014 20:39:39 UTC+1, Florian Oswald wrote:

 myexp(parameter * mylog(x) )

 and it does make a sizeable difference. I'll try your version right now.


 Keep in mind that this is going to be less accurate than using an x^y
 function, as you can be approximately |y*log(x)| ulps out. I'm guessing if
 you're this concerned with performance then you probably won't be too
 concerned about losing a few significant digits, but it is worth keeping in
 mind.

 If you look at the openlibm source, you can see that this is basically the
 approach it uses, albeit using some strategic double-double arithmetic to
 keep enough extra significant digits around.

 Interestingly, since |y*log(x)| can be at most 710 (otherwise the
 resulting exp would overflow or underflow), if you could work in 80-bit
 extended precision, the extra 11 bits in the significand should be
 sufficient so that the final Float64 result should be accurate to within an
 ulp.

 -Simon

[julia-users] question about GSL.jl package usage

[Florian Oswald]

Hi!

I want to use the function interpolation suite from the GSL library. I came 
across the GSL.jl package which wraps the entire GSL library (which is 
awesome).

I am stuck with setting up the interpolation object and wanted to reach out 
for help. my issue is described here in greater 
detail: https://github.com/jiahao/GSL.jl/issues/19

basically I don't know what to put in place of *T::Ptr{gsl_interp_type} in*

*spline_alloc(T::Ptr{gsl_interp_type},size::Integer)*
which is the function used to allocate a spline object. If I just do pS = 
interp_alloc(interp_linear,n) or pS = interp_alloc(interp_cspline,n) it 
rightly says interp_linear not defined. How can I get a pointer to a GSL 
interpolation type? I cannot find any function that returns such a pointer. 

thanks!

[julia-users] ways to improve performance of a non-integer power?

[Florian Oswald]

i'm looking for ways to improve the performance of ^ as in f(x) = x^1.4

I found this thread on integer powers, and it seems there are some benefits 
of calling system pow() functions 
(sometimes). https://github.com/JuliaLang/julia/issues/2741


is there anything related for non-integer powers or am I just stuck here?

thanks!

Re: [julia-users] Re: ways to improve performance of a non-integer power?

[Florian Oswald]

Hi Stefan,
I've got no idea about the technical implications, i.e. why julia's log is 
slower (on some systems?) than libm's and what's involved in bringing 
performance up to par. I mentioned the manual only because this is the 
second time that someone suggests the system-log trick to me, and it paid 
off handsomely in both occasions. Now, I am perfectly happy to keep it at 
that. I suspect however that there are a) plenty of dirty tricks I don't 
know of yet and b) plenty of people who don't either. the thorny issue of 
deciding when a trick becomes dirty is of course up to you. To be honest 
I'm not sure if the manual is the right place for things like that - but it 
would be good to have them in *some* place.

cheers
florian

On Wednesday, 16 July 2014 21:47:58 UTC+1, Stefan Karpinski wrote:

 On Wed, Jul 16, 2014 at 12:39 PM, Florian Oswald florian...@gmail.com 
 javascript: wrote:

 do you think this log issue may be worth a mention in the performance 
 tips section of the manual? I would have never guessed that there could be 
 a fast/slow issue with such basic functions. Little do I know!


 The trouble with mentioning this is that it's an arbitrarily deep rathole. 
 Whole books could (and have) been written on this kind of optimization. But 
 perhaps a passing mention might be good.

Re: [julia-users] how to use searchsortedlast(v::AbstractArray{T,1},x,lo::Int64,hi::Int64,o::Ordering)

[Florian Oswald]

thanks kevin,
that just cut my run time into half! brilliant.

I'm sure other users would benefit from adding this to the documentation.
then there's so many places where to add documentation in julia that it's
hard to even think about where to start. :-)

cheers
florian


On 14 July 2014 22:01, Kevin Squire kevin.squ...@gmail.com wrote:

 For the Ordering parameter, you can use Base.Forward or Base.Reverse.

 The plan is for this parameter to go away eventually, to be replaced by,
 e.g., a function (as you tried).  However, there is no specialization on
 functions right now, so the generated code is pretty slow.

 Generally, the assumption is that these functions are mostly for internal
 use, and that most people won't be using these versions of the functions,
 so they're not documented.  But your use case seems reasonable.  In fact,
 we could probably add a default value of Base.Forward for the ordering.

 You may not need this information, but there are other orderings that are
 specialized for different circumstances (Lexicographic, By, LT, Perm, Left,
 Right), which are used for sorting (and searching) under specific
 circumstances, and each of which has a version of `Base.lt` (less than)
 defined.

 It's also possible to define your own orderings for special circumstances.
  For example, DataFrames has it's own ordering for sorting the columns of a
 DataFrame.

 Cheers,
Kevin


 On Mon, Jul 14, 2014 at 12:50 PM, Florian Oswald florian.osw...@gmail.com
  wrote:

 hi all,

 i'm looking for ways to speed up an operation where i have an ordered
 array v, and to each index i of v I want to apply searchsortedlast(a,v[i]),
 where a is another array. obviously i know from v being increasingly
 ordererd that

 if searchsortedlast(a,v[i]) returns k
 then searchsortedlast(a,v[i+1]) = k

 i was wondering how to use the arguments lo::Int64,hi::Int64,o::Ordering.
 Can I tell the function that we know the last index for v[i] already? I'd
 like to do

  searchsortedlast(a,v[i+1],k,n,o::Ordering)

 in particular, i don't know what to supply as an ordering. that didnt
 work:

 *searchsortedlast(linspace(0.0,1.0,8),0.7,5,8,)*

 thanks!

[julia-users] how to use searchsortedlast(v::AbstractArray{T,1},x,lo::Int64,hi::Int64,o::Ordering)

[Florian Oswald]

hi all,

i'm looking for ways to speed up an operation where i have an ordered array 
v, and to each index i of v I want to apply searchsortedlast(a,v[i]), where 
a is another array. obviously i know from v being increasingly ordererd 
that 

if searchsortedlast(a,v[i]) returns k
then searchsortedlast(a,v[i+1]) = k

i was wondering how to use the arguments lo::Int64,hi::Int64,o::Ordering. 
Can I tell the function that we know the last index for v[i] already? I'd 
like to do

 searchsortedlast(a,v[i+1],k,n,o::Ordering)

in particular, i don't know what to supply as an ordering. that didnt work:

*searchsortedlast(linspace(0.0,1.0,8),0.7,5,8,)*

thanks!

Re: [julia-users] GLM DataFrameRegressionModel gives InexactError()

[Florian Oswald]

hi guys,
sorry for not closing this thread earlier: @simonster pushed exactly the
patch suggested by Milan yesterday evening. haven't tried yet, but should
be all good.
cheers
florian


On 5 July 2014 23:26, Milan Bouchet-Valat nalimi...@club.fr wrote:

 Le samedi 05 juillet 2014 à 15:06 -0700, John Myles White a écrit :
  Not really sure. Can you try printing out each of the columns to see
  where the error is? Maybe the problem is with our new code for
  printing shorthand rather than eps() when p is very small. (That’s
  just total speculation, though.)
 Line 46 of statmodels.jl is indeed this:
 return @sprintf(1e%2.2d, max(iceil(nextfloat(log10(pv))), -99))

 And an InexactError can happen in some cases, for example:
 julia iceil(nextfloat(log10(1e-400)))
 ERROR: InexactError()
  in iceil at float.jl:62

 This is because log(1e-400) returns -Inf. A very simple fix would be to
 call iceil() after max() so that we call iceil(-99) instead of
 iceil(-Inf). I'll make a PR.


 Regards

Re: [julia-users] GLM DataFrameRegressionModel gives InexactError()

[Florian Oswald]

here's the closed issue.
https://github.com/JuliaStats/GLM.jl/issues/80


On 6 July 2014 10:55, Florian Oswald florian.osw...@gmail.com wrote:

 hi guys,
 sorry for not closing this thread earlier: @simonster pushed exactly the
 patch suggested by Milan yesterday evening. haven't tried yet, but should
 be all good.
 cheers
 florian


 On 5 July 2014 23:26, Milan Bouchet-Valat nalimi...@club.fr wrote:

 Le samedi 05 juillet 2014 à 15:06 -0700, John Myles White a écrit :
  Not really sure. Can you try printing out each of the columns to see
  where the error is? Maybe the problem is with our new code for
  printing shorthand rather than eps() when p is very small. (That’s
  just total speculation, though.)
 Line 46 of statmodels.jl is indeed this:
 return @sprintf(1e%2.2d, max(iceil(nextfloat(log10(pv))), -99))

 And an InexactError can happen in some cases, for example:
 julia iceil(nextfloat(log10(1e-400)))
 ERROR: InexactError()
  in iceil at float.jl:62

 This is because log(1e-400) returns -Inf. A very simple fix would be to
 call iceil() after max() so that we call iceil(-99) instead of
 iceil(-Inf). I'll make a PR.


 Regards

[julia-users] how to convert several DataFrame columns to type PooledDataArray for GLM regression

[Florian Oswald]

Hi,

say I want to use the GLM package to run a regression that contains several 
categorical variables. My understanding is that I need the equivalent of an 
R factor, which is a PooledDataArray. Suppose I take this dataset

using RDatasets

*data = dataset(Ecdat,Benefits)*

and I want columns 6-14 as PooledDataArray

I tried

data[,:Sex] = convert(PooledDataArray,data[,:Sex])
and
pool!(data[,:Sex])  # I can see that this needs a second column with the 
numeric code

thanks!

[julia-users] GLM DataFrameRegressionModel gives InexactError()

[Florian Oswald]

HI,

in this example, why do I get an InexactError() when showing the fit 
summary? (I even converted :age to a Float)



*data = dataset(Ecdat,Benefits)**d2 = DataFrame(age = 
convert(DataArray{Float64},data[:Age]), StateUR = data[:StateUR], sex = 
convert(PooledDataArray,data[:Sex]))*

*julia **typeof(d2[:sex])*

*PooledDataArray{ASCIIString,Uint8,1} (constructor with 1 method)*

*julia **typeof(d2[:StateUR])*

*DataArray{Float64,1} (constructor with 1 method)*

*julia **typeof(d2[:age])*


*DataArray{Float64,1} (constructor with 1 method)*

gives this error

*julia **fm = fit(LinearModel, StateUR ~ age + sex,d2)*

*DataFrameRegressionModel{LinearModel{DensePredQR{Float64}},Float64}:*


*Coefficients:*

*Error showing value of type 
DataFrameRegressionModel{LinearModel{DensePredQR{Float64}},Float64}:*

*ERROR: InexactError()*

* in format_pvc at 
/Users/florianoswald/.julia/v0.3/StatsBase/src/statmodels.jl:46*

* in show at 
/Users/florianoswald/.julia/v0.3/StatsBase/src/statmodels.jl:61*

* in show at 
/Users/florianoswald/.julia/v0.3/DataFrames/src/statsmodels/statsmodel.jl:83*

* in anonymous at show.jl:1068*

* in showlimited at show.jl:1067*

* in writemime at replutil.jl:2*

* in display at REPL.jl:117*

* in display at REPL.jl:120*

* in display at multimedia.jl:149*

* in print_response at REPL.jl:139*

* in print_response at REPL.jl:124*

* in anonymous at REPL.jl:551*

* in anonymous at REPL.jl:719*

* in anonymous at LineEdit.jl:1185*

[julia-users] Re: how to convert several DataFrame columns to type PooledDataArray for GLM regression

[Florian Oswald]

ok - got it. the pool!() function worked after a restart. not sure what 
happened.

On Friday, 4 July 2014 20:46:17 UTC+1, Florian Oswald wrote:

 Hi,

 say I want to use the GLM package to run a regression that contains 
 several categorical variables. My understanding is that I need the 
 equivalent of an R factor, which is a PooledDataArray. Suppose I take this 
 dataset

 using RDatasets

 *data = dataset(Ecdat,Benefits)*

 and I want columns 6-14 as PooledDataArray

 I tried

 data[,:Sex] = convert(PooledDataArray,data[,:Sex])
 and
 pool!(data[,:Sex])  # I can see that this needs a second column with the 
 numeric code

 thanks!

Re: [julia-users] Re: build from source modifying Make.user on hpc cluster

[Florian Oswald]

it seems you are close. all tests should pass, so there's something wrong.
try to follow those steps:

https://github.com/JuliaLang/julia/issues/7240

or those

https://github.com/JuliaLang/julia/issues/7482




On 2 July 2014 15:30, Ollie Laslett olasl...@gmail.com wrote:

 I have been having a very similar issue, whilst trying to build julia in
 my home directory on a cluster. I also had problems building openblas,
 which was solved by using OPENBLAS_DYNAMIC_ARCH=0.

 I also have the same errors when running code:

 /lib64/libz.so.1: no version information available

 On top of that, when I run $make testthe file.jl tests fail on @test
 isreadable(file)  which I find peculiar.

 Building from source is such a nightmare sometimes, I didn't quite
 understand your discussion below. Is there anything I can do to rectify
 these issues?

 Thanks

 On Monday, 30 June 2014 22:50:00 UTC+1, Florian Oswald wrote:

 Hi all,

 I've got some very basic questions about building Julia from source on an
 hpc cluster. I got it to build fine a month ago, but now the makefile
 changed. I was partially  succesful this time as well (my test runs), but
 some strange things happen and there's a lot of stuff in that makefile I
 dont' understand.

- I can't get Make.user to work. My understanding was that i `cp
Make.inc Make.user`, then change the options I want changed, then `make`?
Nothing happens. Do I have to delete Make.inc after having made my copy?
- I needed to set `JULIA_CPU_TARGET=core2` when I last got it to
build. The cluster is built out of several different CPU types, so not
setting this resulted in target mismatch errors. I can't find that line
anymore in the current Make.inc, so I changed https://github.com/
JuliaLang/julia/blob/master/Make.inc#L345
https://github.com/JuliaLang/julia/blob/master/Make.inc#L345 from

  JULIA_CPU_TARGET ?= native
 to
  JULIA_CPU_TARGET = core2

- Is this still recommended practice or should I not touch this at
all?
- the openblas default build failed. I set this:
- OPENBLAS_DYNAMIC_ARCH=0
- OPENBLAS_USE_THREAD=0
- OPENBLAS_TARGET_ARCH=BARCELONA
- It says in the trouble-shooting section on
https://github.com/JuliaLang/julia/readme.md
https://github.com/JuliaLang/julia/readme.md that a possibel
solution to problems is to set the last variable to BARCELONA for AMD and
NEHALEM for Intel. What if I have both CPUs? Does it matter?
- With those settings it compiles, and my little test script runs.
However, I get this error/warning from all workers:
- ./julia: /lib64/libz.so.1: no version information available
(required by /data/uctpfos/git/julia/usr/bin/../lib/libjulia.so)
- In case this is helpful, my `cat /proc/version` is:
- Linux version 2.6.32-279.19.1.el6.x86_64 (mock...@sl6.fnal.gov)
(gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Dec 18
17:22:54 CST 2012

Re: [julia-users] build from source modifying Make.user on hpc cluster

[Florian Oswald]

great! thanks a lot.


On 1 July 2014 00:43, Tony Kelman t...@kelman.net wrote:

 Pull request opened here https://github.com/JuliaLang/julia/pull/7476

 If you get into trouble compiling the Haswell kernels due to no such
 instruction: vpermpd, give NO_AVX a try. Or upgrade binutils, up to you.



 On Monday, June 30, 2014 3:50:58 PM UTC-7, Florian Oswald wrote:

 thanks tony, just giving this a try from scratch. to be honest, I dont'
 know the exact age of those processors. Let me see how far I get without
 those newer binutils.

 On Monday, 30 June 2014 23:17:00 UTC+1, Tony Kelman wrote:

 That arrayops.jl test failure is because you're missing an LLVM patch,
 see https://github.com/JuliaLang/julia/issues/7197#issuecomment-46688404

 A lot of these build customization options should be better-documented,
 especially for the heterogenous-processor case. Don't copy Make.inc to
 Make.user, rather create a new Make.user from scratch where you set only
 the options you need to change from their defaults.

 If you have both Intel and AMD processors, you will definitely need to
 have OPENBLAS_DYNAMIC_ARCH enabled. Are any of your processors new enough
 to support AVX instructions? If so, you'll need a newer version of binutils
 than provided by Red Hat 6. See https://github.com/
 JuliaLang/julia/issues/7363#issuecomment-46927356

 I'll open a pull request shortly to add another OpenBLAS build option to
 disable AVX instructions for when you need DYNAMIC_ARCH but installing
 newer binutils is not possible.


 On Monday, June 30, 2014 2:56:48 PM UTC-7, Florian Oswald wrote:

 additional to that, `make testall` errors with this:

 while loading arrayops.jl, in expression starting on line 895

 ERROR: test failed: i7197() == (2,2)


 On 30 June 2014 22:50, Florian Oswald florian...@gmail.com wrote:

 Hi all,

 I've got some very basic questions about building Julia from source on
 an hpc cluster. I got it to build fine a month ago, but now the makefile
 changed. I was partially  succesful this time as well (my test runs), but
 some strange things happen and there's a lot of stuff in that makefile I
 dont' understand.

- I can't get Make.user to work. My understanding was that i `cp
Make.inc Make.user`, then change the options I want changed, then 
 `make`?
Nothing happens. Do I have to delete Make.inc after having made my 
 copy?
- I needed to set `JULIA_CPU_TARGET=core2` when I last got it to
build. The cluster is built out of several different CPU types, so not
setting this resulted in target mismatch errors. I can't find that line
anymore in the current Make.inc, so I changed https://github.com/
JuliaLang/julia/blob/master/Make.inc#L345
https://github.com/JuliaLang/julia/blob/master/Make.inc#L345 from

  JULIA_CPU_TARGET ?= native
 to
  JULIA_CPU_TARGET = core2

- Is this still recommended practice or should I not touch this at
all?
- the openblas default build failed. I set this:
- OPENBLAS_DYNAMIC_ARCH=0
- OPENBLAS_USE_THREAD=0
- OPENBLAS_TARGET_ARCH=BARCELONA
- It says in the trouble-shooting section on
https://github.com/JuliaLang/julia/readme.md
https://github.com/JuliaLang/julia/readme.md that a possibel
solution to problems is to set the last variable to BARCELONA for AMD 
 and
NEHALEM for Intel. What if I have both CPUs? Does it matter?
- With those settings it compiles, and my little test script runs.
However, I get this error/warning from all workers:
- ./julia: /lib64/libz.so.1: no version information available
(required by /data/uctpfos/git/julia/usr/bin/../lib/libjulia.so)
- In case this is helpful, my `cat /proc/version` is:
- Linux version 2.6.32-279.19.1.el6.x86_64 (mock...@sl6.fnal.gov)
(gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Dec 18
17:22:54 CST 2012

[julia-users] Building 35 day old master and get DataFrames and Distributions to run

[Florian Oswald]

I'm having trouble building the current master on a unix cluster, 
see https://github.com/JuliaLang/julia/issues/7482

I was able to build it some time ago, so this commit builds:

https://github.com/JuliaLang/julia/commit/ac46d58c5a23a8020c666789ca80614f5cda4d8f

that version:

Julia Version 0.3.0-prerelease+3219 
Commit ac46d58* (2014-05-26 22:51 UTC) 
Platform Info: 
System: Linux (x86_64-redhat-linux) 
CPU: Intel(R) Xeon(R) CPU 5150 @ 2.66GHz 
WORD_SIZE: 64 
BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY) 
LAPACK: libopenblas 
LIBM: libopenlibm

I am using the DataFrames and Distributions packages. they are not 
compatible with the 35 day old master. What's the best way to get those 
packages working with this version? grab a checkout from around that date?
thanks