Re: [julia-users] small bugging(probably very noob) error please help
and is not an operator in Julia, try .
On Jul 21, 2015, at 11:25 AM, Elsha Robin elisha@gmail.com wrote:
well i thought i should give julia a whirl since it got so much chatter
around it , so i thought i could start of slow and simple and i run into a
tree sort of ... so the problem is that i compared the results of two
functions in an if statement with an and operator but it spiting some very
basic error which i cant figure out what to do with ... so here is the code
in question ..
a(i)=20*i - i^2
b(j)=50 *j - j^2
c(k)=40 *k - k^2
for i in 20:50
for j in i:50
if a(20)==b(50) and c(j)==a(k)
print (got it)
break
end
end
end
__
the error spit out is this ...
syntax: extra token c after end of expression
while loading In[11], in expression starting on line 6
i can figure that the roblem has to do something with comparing the outputs
of functions using looping variables variable { c(j)==a(k) } ... but i dont
understand why that should be a problem ... so either ways can some one give
me the correct syntax or direct me to a specific DOC with
the similar example ... thanks in advance .
[julia-users] Re: small bugging(probably very noob) error please help
The and operator is spelled .
On Tuesday, July 21, 2015 at 10:53:50 AM UTC-5, Elsha Robin wrote:
well i thought i should give julia a whirl since it got so much chatter
around it , so i thought i could start of slow and simple and i run into a
tree sort of ... so the problem is that i compared the results of two
functions in an if statement with an and operator but it spiting some very
basic error which i cant figure out what to do with ... so here is the code
in question ..
a(i)=20*i - i^2
b(j)=50 *j - j^2
c(k)=40 *k - k^2
for i in 20:50
for j in i:50
if a(20)==b(50) and c(j)==a(k)
print (got it)
break
end
end
end
__
the error spit out is this ...
syntax: extra token c after end of expression
while loading In[11], in expression starting on line 6
i can figure that the roblem has to do something with comparing the outputs
of functions using looping variables variable { c(j)==a(k) } ... but i dont
understand why that should be a problem ... so either ways can some one give
me the correct syntax or direct me to a specific DOC with
the similar example ... thanks in advance .
Re: [julia-users] Passing keyword arguments through a function
f(floor(x); args...) should work. On Tue, Jul 21, 2015 at 10:14 AM, Linus Härenstam-Nielsen linush...@gmail.com wrote: I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this?
Re: [julia-users] Passing keyword arguments through a function
This worked! Didn't realize you could use ; when calling functions. On Tuesday, July 21, 2015 at 4:49:36 PM UTC+2, Stefan Karpinski wrote: f(floor(x); args...) should work. On Tue, Jul 21, 2015 at 10:14 AM, Linus Härenstam-Nielsen linu...@gmail.com javascript: wrote: I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this?
Re: [julia-users] small bugging(probably very noob) error please help
a(20)==b(50) and c(j)==a(k)
This is not a valid expression: you need to use instead of and. see
http://docs.julialang.org/en/release-0.3/manual/control-flow/?highlight=boolean#man-short-circuit-evaluation
(do also read the Performance Tips before you get too far along:
http://docs.julialang.org/en/release-0.3/manual/performance-tips/)
On Tue, Jul 21, 2015 at 11:25 AM, Elsha Robin elisha@gmail.com wrote:
well i thought i should give julia a whirl since it got so much chatter
around it , so i thought i could start of slow and simple and i run into a
tree sort of ... so the problem is that i compared the results of two
functions in an if statement with an and operator but it spiting some very
basic error which i cant figure out what to do with ... so here is the code
in question ..
a(i)=20*i - i^2
b(j)=50 *j - j^2
c(k)=40 *k - k^2
for i in 20:50
for j in i:50
if a(20)==b(50) and c(j)==a(k)
print (got it)
break
end
end
end
__
the error spit out is this ...
syntax: extra token c after end of expression
while loading In[11], in expression starting on line 6
i can figure that the roblem has to do something with comparing the outputs
of functions using looping variables variable { c(j)==a(k) } ... but i dont
understand why that should be a problem ... so either ways can some one give
me the correct syntax or direct me to a specific DOC with
the similar example ... thanks in advance .
[julia-users] Re: PyPlot; how to get rid of that annoying flashed-tk-window artifact?
On Tuesday, July 21, 2015 at 1:44:27 AM UTC-4, Jeff Waller wrote: Whenever PyPlot renders something, I'm getting a brief flashed window; I think it's a result of the backend trying to render in a window directly, but can't find a way to disable it. What OS are you on, and what Python distro? I haven't noticed this...
[julia-users] mouse event
Guys, I need to know if we can create mouse events using pyplot in Julia? And how? Thanks
[julia-users] Re: Passing keyword arguments through a function
That is, julia f(x::Int, y::Int=5, z::Int=3) = x+y+z f (generic function with 3 methods) julia f(x::Float64, args...) = f(floor(Int,x), args...) f (generic function with 4 methods) julia f(5.0,1,2) 8 julia f(5.0) 13 On Tuesday, July 21, 2015 at 7:53:01 AM UTC-7, Seth wrote: On Tuesday, July 21, 2015 at 7:14:02 AM UTC-7, Linus Härenstam-Nielsen wrote: I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this? I think you're getting the stack overflow because your call to floor doesn't convert to Int, so f(x::Float64; args...) keeps getting called recursively. What happens if you change your second method to f(x::Float64, y::Int=5, z::Int=3) = f(floor(Int, x), y=y, z=z) instead?
[julia-users] small bugging(probably very noob) error please help
well i thought i should give julia a whirl since it got so much chatter
around it , so i thought i could start of slow and simple and i run into a
tree sort of ... so the problem is that i compared the results of two
functions in an if statement with an and operator but it spiting some very
basic error which i cant figure out what to do with ... so here is the code
in question ..
a(i)=20*i - i^2
b(j)=50 *j - j^2
c(k)=40 *k - k^2
for i in 20:50
for j in i:50
if a(20)==b(50) and c(j)==a(k)
print (got it)
break
end
end
end
__
the error spit out is this ...
syntax: extra token c after end of expression
while loading In[11], in expression starting on line 6
i can figure that the roblem has to do something with comparing the outputs of
functions using looping variables variable { c(j)==a(k) } ... but i dont
understand why that should be a problem ... so either ways can some one give me
the correct syntax or direct me to a specific DOC with
the similar example ... thanks in advance .
[julia-users] Re: small bugging(probably very noob) error please help
The Google Groups view of this thread is very slow to update. All together
now!
On Tuesday, July 21, 2015 at 11:02:30 AM UTC-5, Patrick O'Leary wrote:
The and operator is spelled .
On Tuesday, July 21, 2015 at 10:53:50 AM UTC-5, Elsha Robin wrote:
well i thought i should give julia a whirl since it got so much chatter
around it , so i thought i could start of slow and simple and i run into a
tree sort of ... so the problem is that i compared the results of two
functions in an if statement with an and operator but it spiting some very
basic error which i cant figure out what to do with ... so here is the code
in question ..
a(i)=20*i - i^2
b(j)=50 *j - j^2
c(k)=40 *k - k^2
for i in 20:50
for j in i:50
if a(20)==b(50) and c(j)==a(k)
print (got it)
break
end
end
end
__
the error spit out is this ...
syntax: extra token c after end of expression
while loading In[11], in expression starting on line 6
i can figure that the roblem has to do something with comparing the outputs
of functions using looping variables variable { c(j)==a(k) } ... but i dont
understand why that should be a problem ... so either ways can some one give
me the correct syntax or direct me to a specific DOC with
the similar example ... thanks in advance .
Re: [julia-users] Deducing probability density functions from model equations
There's tons of WIP-code that implements arithmetic on Distributions. No one has the time to finish that code or volunteer to maintain it, so it's just sitting in limbo. OP: What you're asking for sounds like it's largely equivalent to probabilistic programming. There are a ton of ways you could implement it, but it's almost surely a mistake to implement this sort of thing on your own. I would encourage you to learn Stan (which you can call from Julia), which is a formal language for specifying joint probability distributions. A Stan encoding of your probability model will allow you to draw samples from arbitrary conditional distributions by conditioning on observed data. On Tuesday, July 21, 2015 at 6:43:05 AM UTC-7, Tamas Papp wrote: In general, f(x,theta) does not necessarily belong to any frequently used distribution family, even if theta does -- it is easy to come up with examples. Some distribution families are closed under certain operations (eg addition, and multiplication by scalars for the normal), and some distributions arise as transformations of other distributions (eg chi square from normal, etc). AFAIK Distributions.jl does not support arithmetic on distributions, but you can always try programming the relevant generic functions, eg + for (Real, Distributions.Normal) if the results of your transformations are distributions in Distribution.jl. Best, Tamas On Tue, Jul 21 2015, amik...@gmail.com wrote: Dear all, I don't know if that's the best place to ask such a question but I'll give it a try: I need to code stochastic models: xnplus1 = f(xn, theta) where xn is the state of my system at time n and theta is a set of parameters for this model, constant through time, and possibly containing noise. As a matter of example, let's consider the following model: function transition(n, xn, theta) e = rand(Normal(theta.mu, theta.sigma)) xnplus1 = xn + e return xnplus1 end My goal is to write such equations and to deduce automatically the transition probability density function: p(xnplus1 | xn, theta). I intended to parse the code of the model and look for the rand keyword, the name of the law used to generate this random variable and the values given to it. In the previous case, I could deduce that: p(xnplus1 | xn, theta) = normal_pdf(xnplus1, xn + theta.mu, theta.sigma) where normal_pdf(x, mu, sigma) = 1 / (sigma * sqrt(2 * pi)) * exp( - 1 / 2 * (x - mu) ^ 2 / sigma ^ 2) (rather) easily I think by specifying that whenever a constant is added to a normal variable, the mean of the new variable (left value in the equation) is normal and of mean increased by this constant and of the same standard deviation. But that's the simplest case of all and it requires me to specify some rules about the normal distribution. I therefore have the following questions: - is what I'm trying to do understandable? - is it doable at all? and in Julia? - is specifying rules for each distribution (normal, uniform, Poisson; correlated, uncorrelated) the way to go or should I think of something even more generic? - do you have any other suggestions to solve this problem? Thank you very much,
Re: [julia-users] Passing keyword arguments through a function
On Tue, Jul 21, 2015 at 10:48 AM, Stefan Karpinski ste...@karpinski.org wrote: f(floor(x); args...) should work. Relevant documentation is here[1] in case you have other confusions about keyword arguments. [1] http://julia.readthedocs.org/en/latest/manual/functions/#keyword-arguments On Tue, Jul 21, 2015 at 10:14 AM, Linus Härenstam-Nielsen linush...@gmail.com wrote: I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this?
Re: [julia-users] Re: How to un-substring a string?!
maybe it should check to see if the end matches?
@glen - i think that will construct an ascii string, which isn't what you
want if the underlying data are unicode. i've always assumed string() does
something smart and returns the right thing, but haven't checked...
andrew
On Tuesday, 21 July 2015 20:49:12 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:38 PM, andrew cooke and...@acooke.org
javascript: wrote:
ah. for some reason i was thinking they were invisible (somewhere below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that* to
match still gives the memory issue.
Hmmm,
```
match(re::Regex, str::Union{ByteString,SubString}, idx::Integer,
add_opts::UInt32=UInt32(0)) =
match(re, utf8(str), idx, add_opts)
```
So match on a substring does a copy. I'm guessing this is because pcre
expect a c string (i.e. NULL terminated?)
so there's still something odd that i don't understand. maybe it's just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org
wrote:
ok, so match(regex, string, index) solves the problem. presumably it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing
string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that
because of
this you should use bytestring(...) before passing a string to c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to
start
at
as an argument. so i can avoid s[i:end] and just pass i into match
(i
just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a good
method
for
this also. it seems wrong to me also that `match` is making a
copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any
more
efficient solutions (and confirmation the analysis is correct -
is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so
the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so
there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be
passed
to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to
copy
m.match.
as far as i can tell string() doesn't help. so what works?
or
am i
wrong?
thanks,
andrew
[julia-users] Re: Passing keyword arguments through a function
On Tuesday, July 21, 2015 at 7:14:02 AM UTC-7, Linus Härenstam-Nielsen wrote: I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this? I think you're getting the stack overflow because your call to floor doesn't convert to Int, so f(x::Float64; args...) keeps getting called recursively. What happens if you change your second method to f(x::Float64, y::Int=5, z::Int=3) = f(floor(Int, x), y=y, z=z) instead?
Re: [julia-users] mouse event
http://matplotlib.org/users/event_handling.html On Tue, Jul 21, 2015 at 12:34 PM, Emerson Vitor Castelani emersonvi...@gmail.com wrote: Guys, I need to know if we can create mouse events using pyplot in Julia? And how? Thanks
[julia-users] HDF5 file id biger then txt. What wrong?
I have data in txt file, some milons like this: 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,1 0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1 Coding win1250. size of dane.txt is 1.3 GB D=readcsv(dane.txt) k,l=size(D) using HDF5, JLD hfi=h5open(D.h5,w) close(hfi) fid = h5open(D.h5,r+) g = fid[/] dset1 = d_create(g, /D, datatype(Int64), dataspace(k,l)) dset1[:,:]=D close(fid) After save to h5 file the file has 6.3 GB ? Why new file is 4 times biger? Paul
Re: [julia-users] mouse event
Hi I got the following example related with this thread https://gist.github.com/elsuizo/5338605a6a5c7ab54730 but fail in https://gist.github.com/elsuizo/5338605a6a5c7ab54730#file-phase_portrait_mouse_event-jl-L114 and I do not know why This is the python code which comes: https://gist.github.com/elsuizo/6957889d8e63e9029377
Re: [julia-users] mouse event
Well, I already tried this , but I don't get to do work!Does canvas and mpl_connect work in Julia? Em terça-feira, 21 de julho de 2015 13:38:06 UTC-3, Isaiah escreveu: http://matplotlib.org/users/event_handling.html On Tue, Jul 21, 2015 at 12:34 PM, Emerson Vitor Castelani emerso...@gmail.com javascript: wrote: Guys, I need to know if we can create mouse events using pyplot in Julia? And how? Thanks
[julia-users] [Ann]: AutoTypeParemeters
for every time that you wanted to stick some weird value in a type parameter and couldn't. https://github.com/andrewcooke/AutoTypeParameters.jl andrew
Re: [julia-users] memory use profiling
ah https://julia.readthedocs.org/en/latest/manual/profile/#man-track-allocation thanks. although i am not seeing any mem files for some reason. i will try updating julia to latest. oh. i do now! doesn't seem to work with Pkg.test(), but does if you just execute commands at the prompt. excellent! andrew On Tuesday, 21 July 2015 10:28:01 UTC-3, Isaiah wrote: i can't see how to get a dump of objects on the heap, or tools to analyse that dump. In 0.4 there is a new --track-allocation command-line option that might help. If you don't mind reading LLVM IR, you can also look for calls to jl_gc_allocobj (or just allocobj in 0.3). On Tue, Jul 21, 2015 at 9:08 AM, andrew cooke and...@acooke.org javascript: wrote: hi, i feel i must be missing something obvious, sorry, but i can work out what tools are available for understanding memory leaks inside my own julia code. this isn't an issue with julia itself - it's my own dumb fault. but i can't see how to get a dump of objects on the heap, or tools to analyse that dump. can someone point me in the right direction, please? thanks, andrew (i know about - and have used profile and profileview - but they don't have memory info, do they? they do have gc information, but what i want is heap use - gc isn't being called!)
[julia-users] Re: PyPlot; how to get rid of that annoying flashed-tk-window artifact?
What OS are you on, and what Python distro? I haven't noticed this... OS/X Mavericks, Python 2.7.5 (Apple default) matplotlib 1.1.1 Julia Version 0.4.0-dev+6075 Commit 4b757af* (2015-07-19 00:53 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.4.0) CPU: Intel(R) Core(TM) i7-4850HQ CPU @ 2.30GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Haswell) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3
Re: [julia-users] HDF5 file id biger then txt. What wrong?
HDF5 file support compression. This is enabled via a flag when writing the file; when reading, it is automatically decompressed. I assume that compression would greatly reduce the file size. -erik On Tue, Jul 21, 2015 at 1:21 PM, Stefan Karpinski ste...@karpinski.org wrote: In your example data, each value is represented with two bytes: one for the value, one for a comma or newline. Each Int64 value is 8 bytes. If all your values are between 0 and 255, you could use UInt8 to represent them and cut the size in half. On Tue, Jul 21, 2015 at 1:16 PM, paul analyst paul.anal...@mail.com wrote: I have data in txt file, some milons like this: 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,1 0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1 Coding win1250. size of dane.txt is 1.3 GB D=readcsv(dane.txt) k,l=size(D) using HDF5, JLD hfi=h5open(D.h5,w) close(hfi) fid = h5open(D.h5,r+) g = fid[/] dset1 = d_create(g, /D, datatype(Int64), dataspace(k,l)) dset1[:,:]=D close(fid) After save to h5 file the file has 6.3 GB ? Why new file is 4 times biger? Paul -- Erik Schnetter schnet...@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
Re: [julia-users] HDF5 file id biger then txt. What wrong?
In your example data, each value is represented with two bytes: one for the value, one for a comma or newline. Each Int64 value is 8 bytes. If all your values are between 0 and 255, you could use UInt8 to represent them and cut the size in half. On Tue, Jul 21, 2015 at 1:16 PM, paul analyst paul.anal...@mail.com wrote: I have data in txt file, some milons like this: 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,1 0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1 Coding win1250. size of dane.txt is 1.3 GB D=readcsv(dane.txt) k,l=size(D) using HDF5, JLD hfi=h5open(D.h5,w) close(hfi) fid = h5open(D.h5,r+) g = fid[/] dset1 = d_create(g, /D, datatype(Int64), dataspace(k,l)) dset1[:,:]=D close(fid) After save to h5 file the file has 6.3 GB ? Why new file is 4 times biger? Paul
Re: [julia-users] Re: How to un-substring a string?!
unfortunately, the semantics for the match don't seem to be the same.
if you use a substring then ^ binds to the start of the substring.
if you use match(...) with an offset then ^ binds to the start of the
underlying string.
at least, that's how i understand the following:
julia match(r^\s, a b)
julia match(r^\s, a b[2:end])
RegexMatch( )
julia match(r^\s, a b, 2)
julia
which ruins everything for me again. :o(
well, i guess not totally - i can still use the copy and copy again
approach, but am making loads of huge copies.
andrew
On Tuesday, 21 July 2015 21:25:09 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 8:19 PM, andrew cooke and...@acooke.org
javascript: wrote:
maybe it should check to see if the end matches?
My guess, is that this is an optimization that won't have too much
benefit because in most cases you can just call `match` with a start
idx.
@glen - i think that will construct an ascii string, which isn't what
you
want if the underlying data are unicode. i've always assumed string()
does
something smart and returns the right thing, but haven't checked...
And from the code I pasted, `utf` should probably do the copy you want.
andrew
On Tuesday, 21 July 2015 20:49:12 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:38 PM, andrew cooke and...@acooke.org
wrote:
ah. for some reason i was thinking they were invisible (somewhere
below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that*
to
match still gives the memory issue.
Hmmm,
```
match(re::Regex, str::Union{ByteString,SubString}, idx::Integer,
add_opts::UInt32=UInt32(0)) =
match(re, utf8(str), idx, add_opts)
```
So match on a substring does a copy. I'm guessing this is because pcre
expect a c string (i.e. NULL terminated?)
so there's still something odd that i don't understand. maybe it's
just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org
wrote:
ok, so match(regex, string, index) solves the problem. presumably
it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking
/
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing
string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying
that
because of
this you should use bytestring(...) before passing a string to
c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to
start
at
as an argument. so i can avoid s[i:end] and just pass i into
match
(i
just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a
good
method
for
this also. it seems wrong to me also that `match` is making a
copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke
and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate
any
more
efficient solutions (and confirmation the analysis is
correct -
is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers
the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character,
so
the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is
that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
Re: [julia-users] Translating void* *bufs for a Julia type
immutable my_t
a::Cint
b::Cint
c::Cint
bufs::Ptr{Ptr{Void}}
flags::Cint
procFunc::Ptr{Void}
resetFunc::Ptr{Void}
end
On Tue, Jul 21, 2015 at 7:21 PM, Dave C dave.cr...@gmail.com wrote:
I'm trying to wrap some C code that has a struct in a header. I'm trying
to create a Julia type that emulates the following struct:
typedef struct
{
int a;
int b;
int c;
void* *bufs;
int flags;
void* procFunc;
void* resetFunc;
} my_t;
I'm fine with everything except how to handle:
void* *bufs;
Any advice or general thoughts? Thanks in advance - Dave.
Re: [julia-users] Iterating over Dicts
(SortedDict is already in DataStructures.jl. To be added are SortedMultiDict and SortedSets.) Cheers! On Tue, Jul 21, 2015 at 4:43 PM, Yichao Yu yyc1...@gmail.com wrote: On Tue, Jul 21, 2015 at 7:37 PM, Benjamin Deonovic bdeono...@gmail.com wrote: A few clarifying questions. I believe that when you iterate over julia Dicts, the iteration is not necessarily in any particular order (i.e. if keys are alpha-numeric it is not necessarily in sorted order, or it is not necessarily in order the keys were inserted). Is that correct? Yes. If I set the seed though am I guaranteed the same order each time and between Julia sessions? Julia don't randomize it intentionally either and my guess is that the order would be repeatable if the hash function is (which might requires turning off address space randomization). Even then, this is not a defined behavior. If you want defined iteration order, see OrderedDict (and to be merged SortedDict) in DataStructures.jl Thanks Benjamin
[julia-users] Re: Nullable parametric types
Thanks for the responses! I wasn't not sure how I could use typealiases to
deal with this, so I went with the macro route and ended up with
https://gist.github.com/darwindarak/0eb5e53c21c896392938. It converts the
functions/type declarations into the parametric form suggested by Mauro.
On Tuesday, July 21, 2015 at 2:50:16 PM UTC-7, David Gold wrote:
Or use typealiases?
On Tuesday, July 21, 2015 at 4:48:31 PM UTC-4, David Gold wrote:
My instinct is to write a macro.
On Tuesday, July 21, 2015 at 4:37:03 PM UTC-4, Darwin Darakananda wrote:
Hi all,
I'm in the process of replacing my Union{Nothing, T} annotations to
Nullable{T}, but I'm getting stuck when T is a parametric type. For
example, replacing the function
function myfunc(v::Union{Nothing, Vector})
...
end
with
function myfunc(v::Nullable{Vector})
...
end
makes it completely uncallable since any vector I pass in comes with an
eltype parameter and Julia's type parameters are invariant. For functions
or types that only have a couple Nullable types, I can do something like
function myfunc{T}(v::Nullable{Vector{T}})
...
end
type MyType{T}
v::Nullable{Vector{T}
end
but that gets very ugly when you have a large number of nullable types.
Does anyone know of a cleaner way to do this?
Thanks,
Darwin
Re: [julia-users] Re: Environment reification and lazy evaluation
More to do:
Expressions would also have to be escaped from quoting.
If we can't scope types within dicts, it might be necessary to have special
markers for types so they can avoid being scoped.
I don't think that macros will be necessary anymore
On Wednesday, July 22, 2015 at 11:14:38 AM UTC+8, Brandon Taylor wrote:
Ok so I've got a good start. I bet John Myles White didn't think I could
get this far. Anyway, I'm getting caught up in defining my own escape
function. I'm getting lost in multiple layers of meta.
using DataFrames
import Base.convert
# allow inheritance from modules
function convert(::Type{Dict}, m::Module)
dict = Dict()
for name in names(m)
dict[name] = eval( :(Base.$name) )
end
dict
end
base_dict = convert(Dict, Base)
# allow inheritance from DataFrames
function convert(::Type{Dict}, d::DataFrame)
dict = Dict()
for name in names(d)
dict[name] = d[name]
end
dict
end
# modify dicts such that if a key is not found, search the parent
function Base.getindex{K,V}(h::Dict{K,V}, key)
index = Base.ht_keyindex(h, key)
if index 0
if :_parent in keys(h)
Base.getindex(h[:_parent], key)
else
throw(KeyError(key))
end
else
h.vals[index]::V
end
end
# test expression
e =
quote
a = 1
# anonymous functions required for proper scoping
test = function()
b = a
end
end
# set up the global environment
_env = gensym()
eval(:($_env = [:_parent = base_dict] ) )
#_new_env = _env
# this is the code that needs to be escaped
#eval(:($_new_env = [:_parent = eval(_env)] ) )
# reformat code to use dict scoping
function env_replace!(e::Expr,
_env::Symbol = _env)
# expressions wrapped in _esc will be left alone
if length(e.args) 0
if (e.head == :call) (e.args[1] == :_esc)
return e.args[2]
end
end
# set a new scope for a new function. This will also have to be done
with for loops, modules, etc.
if (e.head == :function)
# insert a new scope definition into the function definition
_new_env = gensym()
e.args[2].args = [
e.args[2].args[1],
:_esc(), need help here ###
e.args[2].args[2:end] ]
_env = _new_env
end
# ignore line numbers
if e.head != :line
for i in 1:length(e.args)
# replace symbols with their dict scoped version
if typeof(e.args[i]) == Symbol
e.args[i] = :($_env[$(string(e.args[i]))])
# recur into new expressions
elseif typeof(e.args[i]) == Expr
e.args[i] = env_replace!(e.args[i], _env)
end
end
end
e
end
# here is an eval that allows evaluation within a certain dict scope
function lazy_eval(e::Expr,
_env::Symbol = _env)
eval(env_replace!(e), _env)
end
## TO DO
# fix _esc problem
# prevent environment symbols from being scoped (perhaps with a special
marker)
# rescope for, while, try, catch, finally, let, and type
# perhaps use fast anonymous to avoid performance slowdowns?
On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
And there would need to be a special marker for them, such that if I'm in
function f, f[:a] won't get preprocessed as f[:f][:a]
On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
Although that would probably require nested dicts. Each would have a
parent dict, and if a lookup isn't found in the current dict, the parent
dict would be searched.
On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
I should be possible to preprocess code such that everything is put
into a dict based on the name of enclosing function (and global variables
will just go into a dict called global).
On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
Dicts seem to work pretty well for this kind of thing.
On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
I'm getting a cannot assign variables in other modules error.
On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
brandon@gmail.com wrote:
Ok, a thought, Julia has an inbuilt idea of a module. Would it be
possible
to hijack this functionality to provide pseudo-environments? That
is, never
referring to anything that is not already in an explicit module?
And also,
have a data-frame simply be a module?
I think this would in principle works. A module is basically what
global scope means so all the performance concern applies.
On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor
wrote:
I don't know if you came across the vignette?
http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html
?
dplyr uses lazyeval extensively, see
Re: [julia-users] Re: How to un-substring a string?!
yeah, i guess in this case that's what the lib wants, so you need to force conversion if you don't have utf8. On Tuesday, 21 July 2015 21:25:09 UTC-3, Yichao Yu wrote: And from the code I pasted, `utf` should probably do the copy you want.
Re: [julia-users] Re: Environment reification and lazy evaluation
Ok so I've got a good start. I bet John Myles White didn't think I could
get this far. Anyway, I'm getting caught up in defining my own escape
function. I'm getting lost in multiple layers of meta.
using DataFrames
import Base.convert
# allow inheritance from modules
function convert(::Type{Dict}, m::Module)
dict = Dict()
for name in names(m)
dict[name] = eval( :(Base.$name) )
end
dict
end
base_dict = convert(Dict, Base)
# allow inheritance from DataFrames
function convert(::Type{Dict}, d::DataFrame)
dict = Dict()
for name in names(d)
dict[name] = d[name]
end
dict
end
# modify dicts such that if a key is not found, search the parent
function Base.getindex{K,V}(h::Dict{K,V}, key)
index = Base.ht_keyindex(h, key)
if index 0
if :_parent in keys(h)
Base.getindex(h[:_parent], key)
else
throw(KeyError(key))
end
else
h.vals[index]::V
end
end
# test expression
e =
quote
a = 1
# anonymous functions required for proper scoping
test = function()
b = a
end
end
# set up the global environment
_env = gensym()
eval(:($_env = [:_parent = base_dict] ) )
#_new_env = _env
# this is the code that needs to be escaped
#eval(:($_new_env = [:_parent = eval(_env)] ) )
# reformat code to use dict scoping
function env_replace!(e::Expr,
_env::Symbol = _env)
# expressions wrapped in _esc will be left alone
if length(e.args) 0
if (e.head == :call) (e.args[1] == :_esc)
return e.args[2]
end
end
# set a new scope for a new function. This will also have to be done with
for loops, modules, etc.
if (e.head == :function)
# insert a new scope definition into the function definition
_new_env = gensym()
e.args[2].args = [
e.args[2].args[1],
:_esc(), need help here ###
e.args[2].args[2:end] ]
_env = _new_env
end
# ignore line numbers
if e.head != :line
for i in 1:length(e.args)
# replace symbols with their dict scoped version
if typeof(e.args[i]) == Symbol
e.args[i] = :($_env[$(string(e.args[i]))])
# recur into new expressions
elseif typeof(e.args[i]) == Expr
e.args[i] = env_replace!(e.args[i], _env)
end
end
end
e
end
# here is an eval that allows evaluation within a certain dict scope
function lazy_eval(e::Expr,
_env::Symbol = _env)
eval(env_replace!(e), _env)
end
## TO DO
# fix _esc problem
# prevent environment symbols from being scoped (perhaps with a special
marker)
# rescope for, while, try, catch, finally, let, and type
# perhaps use fast anonymous to avoid performance slowdowns?
On Tuesday, July 21, 2015 at 10:10:58 AM UTC+8, Brandon Taylor wrote:
And there would need to be a special marker for them, such that if I'm in
function f, f[:a] won't get preprocessed as f[:f][:a]
On Tuesday, July 21, 2015 at 10:03:08 AM UTC+8, Brandon Taylor wrote:
Although that would probably require nested dicts. Each would have a
parent dict, and if a lookup isn't found in the current dict, the parent
dict would be searched.
On Tuesday, July 21, 2015 at 9:53:50 AM UTC+8, Brandon Taylor wrote:
I should be possible to preprocess code such that everything is put into
a dict based on the name of enclosing function (and global variables will
just go into a dict called global).
On Tuesday, July 21, 2015 at 9:42:00 AM UTC+8, Brandon Taylor wrote:
Dicts seem to work pretty well for this kind of thing.
On Tuesday, July 21, 2015 at 9:38:36 AM UTC+8, Brandon Taylor wrote:
I'm getting a cannot assign variables in other modules error.
On Tuesday, July 21, 2015 at 6:39:44 AM UTC+8, Yichao Yu wrote:
On Mon, Jul 20, 2015 at 6:35 PM, Brandon Taylor
brandon@gmail.com wrote:
Ok, a thought, Julia has an inbuilt idea of a module. Would it be
possible
to hijack this functionality to provide pseudo-environments? That
is, never
referring to anything that is not already in an explicit module?
And also,
have a data-frame simply be a module?
I think this would in principle works. A module is basically what
global scope means so all the performance concern applies.
On Friday, July 10, 2015 at 11:31:36 PM UTC+8, Brandon Taylor
wrote:
I don't know if you came across the vignette?
http://cran.r-project.org/web/packages/lazyeval/vignettes/lazyeval.html
?
dplyr uses lazyeval extensively, see
http://cran.r-project.org/web/packages/dplyr/vignettes/nse.html .
The cool
thing about being able to incorporate this kind of thing in Julia
would be
being able to use the self-reflection capabilities.
On Friday, July 10, 2015 at 10:57:16 AM UTC-4, Cedric St-Jean
wrote:
On Thursday, July 9, 2015 at 10:35:30 PM UTC-4, Brandon Taylor
wrote:
To walk back in time, you could say something like: compile this
like
this was is in line 8.
Re: [julia-users] HDF5 file id biger then txt. What wrong?
Yes, that could be even more effective. On Tue, Jul 21, 2015 at 2:09 PM, Erik Schnetter schnet...@gmail.com wrote: HDF5 file support compression. This is enabled via a flag when writing the file; when reading, it is automatically decompressed. I assume that compression would greatly reduce the file size. -erik On Tue, Jul 21, 2015 at 1:21 PM, Stefan Karpinski ste...@karpinski.org wrote: In your example data, each value is represented with two bytes: one for the value, one for a comma or newline. Each Int64 value is 8 bytes. If all your values are between 0 and 255, you could use UInt8 to represent them and cut the size in half. On Tue, Jul 21, 2015 at 1:16 PM, paul analyst paul.anal...@mail.com wrote: I have data in txt file, some milons like this: 0,0,2,0,0,0,0,0,0,0,0,0,0,0,0,0,0 0,0,0,0,0,0,0,2,0,0,0,2,0,0,0,0,1 0,0,0,0,1,0,0,0,1,0,0,0,0,0,0,0,1 Coding win1250. size of dane.txt is 1.3 GB D=readcsv(dane.txt) k,l=size(D) using HDF5, JLD hfi=h5open(D.h5,w) close(hfi) fid = h5open(D.h5,r+) g = fid[/] dset1 = d_create(g, /D, datatype(Int64), dataspace(k,l)) dset1[:,:]=D close(fid) After save to h5 file the file has 6.3 GB ? Why new file is 4 times biger? Paul -- Erik Schnetter schnet...@gmail.com http://www.perimeterinstitute.ca/personal/eschnetter/
[julia-users] Merge partitions
Is there any way to merge partitions together? For instance, could I do something like merge(partitions(1:n, 2), partitions(1:n, 3))? I essentially want to create one iterable object out of both of them.
[julia-users] Re: @manipulate button not updating Markdown display
Thank you so much for your help. I fixed the problem of writemime calling
display and am using the latest IJulia master. I can get (some) markdown to
display correctly when the button is clicked (though line breaks don't work
as expected). However, I have two problems:
1. I can't get LaTeX to display in Markdown using writemime. The display
function works perfectly, but writemime does not. Are LaTeX equations
currently unsupported, or did I mess something up?
2. The text doesn't clear when I click the button off. I've tried
replacing output by printing and clearing output with IJulia.
clear_output(false), but no luck.
Here's what I'm doing:
import Base: writemime, show, write
function write(io::IO, x::Revealable)
write(io, x.content)
end
function writemime(io::IO, ::MIMEtext/markdown, x::Revealable)
if x.show
write(io, x)
else
#write(io,) -- I tried this too without success
IJulia.clear_output(false)
end
end
function show(io::IO, x::Revealable)
Base.print_quoted_literal(io, x.content)
end
Here's the whole thing if you want to run it:
using Reactive
using Interact
using IJulia
type Revealable
content::ASCIIString
label::ASCIIString
show::Bool
end
function revealable(x::Revealable)
@manipulate for n in togglebutton(; label=string(Show/Hide, x.label ==
? : string( , uppercase(x.label[1]),x.label[2:end])), value=x.show,
signal=Input(x.show))
x.show = n
x
end
end
import Base: writemime, show, write
function write(io::IO, x::Revealable)
write(io, x.content)
end
function writemime(io::IO, ::MIMEtext/markdown, x::Revealable)
if x.show
write(io, x)
else
#write(io,) -- I tried this too without success
IJulia.clear_output(false)
end
end
function show(io::IO, x::Revealable)
Base.print_quoted_literal(io, x.content)
end
### To run it:
r = Revealable(#This is an *awesome* equation: \$e^u \\frac{du}{dx} e^x
dx\$.,example,false)
revealable(r)
On Friday, July 17, 2015 at 7:06:37 AM UTC-7, Steven G. Johnson wrote:
function writemime(stream, ::MIMEtext/latex, x::Revealable) #was
::MIMEtext/markdown in my actual code
if x.show
display(x.content)
else
display()
end
end
This is wrong. writemime should not call display. It should just write
the LaTeX form of x to the (text) stream using print or write. (You have
it backwards: display calls writemime.)
With regards to your other equation; for mixing equations and formatted
text, the latest IJulia master [Pkg.checkout(IJulia)] can display
text/markdown directly, including equations, e.g.:
display(text/markdown, This is an *awesome* equation: \$e^u
\\frac{du}{dx} e^x dx\$.)
So, in an @manipulate loop, you just need to return an object that knows
how to display itself as text/markdown (i.e. that has a writemime method to
output text/markdown text to an I/O stream).
Re: [julia-users] mouse event
I had a similar problem...I don't know why... Em terça-feira, 21 de julho de 2015 14:46:41 UTC-3, El suisse escreveu: Hi I got the following example related with this thread https://gist.github.com/elsuizo/5338605a6a5c7ab54730 but fail in https://gist.github.com/elsuizo/5338605a6a5c7ab54730#file-phase_portrait_mouse_event-jl-L114 and I do not know why This is the python code which comes: https://gist.github.com/elsuizo/6957889d8e63e9029377
Re: [julia-users] mouse event
Please describe what you tried and what was the error. You may need to either 1) yield the event loop 2) make sure the program stays open long enough to actually display your plot (ref: https://github.com/stevengj/PyPlot.jl/issues/41) On Tue, Jul 21, 2015 at 2:53 PM, Emerson Vitor Castelani emersonvi...@gmail.com wrote: I had a similar problem...I don't know why... Em terça-feira, 21 de julho de 2015 14:46:41 UTC-3, El suisse escreveu: Hi I got the following example related with this thread https://gist.github.com/elsuizo/5338605a6a5c7ab54730 but fail in https://gist.github.com/elsuizo/5338605a6a5c7ab54730#file-phase_portrait_mouse_event-jl-L114 and I do not know why This is the python code which comes: https://gist.github.com/elsuizo/6957889d8e63e9029377
[julia-users] Re: SOLVED: ERROR: `start` has no method matching start(::Nothing)
Not related, but how do you edit the thread name in order to mark it as SOLVED? I haven't found a way to do this, thanks! El lunes, 20 de julio de 2015, 14:09:00 (UTC-5), Kaj Wiik escribió: I started to get a strange error while debugging my code, here's a simplified example: julia function foo(a) println(foo..) end foo (generic function with 1 method) julia a = foo(2) foo.. julia a,b = foo(2) foo.. ERROR: `start` has no method matching start(::Nothing) So, the problem was a missing return value, it is strange that missing one value did not give error but two values It took a quite long time to track this down. Perhaps a bit more informative error message would be possible...? Cheers, Kaj
[julia-users] Re: Nullable parametric types
Or use typealiases?
On Tuesday, July 21, 2015 at 4:48:31 PM UTC-4, David Gold wrote:
My instinct is to write a macro.
On Tuesday, July 21, 2015 at 4:37:03 PM UTC-4, Darwin Darakananda wrote:
Hi all,
I'm in the process of replacing my Union{Nothing, T} annotations to
Nullable{T}, but I'm getting stuck when T is a parametric type. For
example, replacing the function
function myfunc(v::Union{Nothing, Vector})
...
end
with
function myfunc(v::Nullable{Vector})
...
end
makes it completely uncallable since any vector I pass in comes with an
eltype parameter and Julia's type parameters are invariant. For functions
or types that only have a couple Nullable types, I can do something like
function myfunc{T}(v::Nullable{Vector{T}})
...
end
type MyType{T}
v::Nullable{Vector{T}
end
but that gets very ugly when you have a large number of nullable types.
Does anyone know of a cleaner way to do this?
Thanks,
Darwin
[julia-users] Re: MongoDB and Julia
would a MongoDB JDBC driver paired with JDBC.jl be feasible for your purposes ... ? https://groups.google.com/forum/#!topic/julia-users/qqOTu4XL1HI good luck, cdm
[julia-users] Re: Why is the type of a variable different from inside and outside a let block?
Thank you both! I get it now: julia let x::Float64 = 5 x end::Float64 5.0 El lunes, 20 de julio de 2015, 23:03:03 (UTC-5), Ismael VC escribió: This was surprising: julia versioninfo() Julia Version 0.4.0-dev+5491 Commit cb77503 (2015-06-21 09:45 UTC) Platform Info: System: Linux (x86_64-unknown-linux-gnu) CPU: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 julia let x::Float64 = 5 end::Float64 ERROR: TypeError: typeassert: expected Float64, got Int64 julia let x::UTF8String = test end::UTF8String ERROR: TypeError: typeassert: expected UTF8String, got ASCIIString
[julia-users] Nullable parametric types
Hi all,
I'm in the process of replacing my Union{Nothing, T} annotations to
Nullable{T}, but I'm getting stuck when T is a parametric type. For
example, replacing the function
function myfunc(v::Union{Nothing, Vector})
...
end
with
function myfunc(v::Nullable{Vector})
...
end
makes it completely uncallable since any vector I pass in comes with an
eltype parameter and Julia's type parameters are invariant. For functions
or types that only have a couple Nullable types, I can do something like
function myfunc{T}(v::Nullable{Vector{T}})
...
end
type MyType{T}
v::Nullable{Vector{T}
end
but that gets very ugly when you have a large number of nullable types.
Does anyone know of a cleaner way to do this?
Thanks,
Darwin
Re: [julia-users] Nullable parametric types
If I understand correctly, this should do:
function myfunc{T:Vector}(v::Nullable{T})
...
end
On Tue, 2015-07-21 at 22:37, Darwin Darakananda darwinda...@gmail.com wrote:
Hi all,
I'm in the process of replacing my Union{Nothing, T} annotations to
Nullable{T}, but I'm getting stuck when T is a parametric type. For
example, replacing the function
function myfunc(v::Union{Nothing, Vector})
...
end
with
function myfunc(v::Nullable{Vector})
...
end
makes it completely uncallable since any vector I pass in comes with an
eltype parameter and Julia's type parameters are invariant. For functions
or types that only have a couple Nullable types, I can do something like
function myfunc{T}(v::Nullable{Vector{T}})
...
end
type MyType{T}
v::Nullable{Vector{T}
end
but that gets very ugly when you have a large number of nullable types.
Does anyone know of a cleaner way to do this?
Thanks,
Darwin
[julia-users] Re: Nullable parametric types
My instinct is to write a macro.
On Tuesday, July 21, 2015 at 4:37:03 PM UTC-4, Darwin Darakananda wrote:
Hi all,
I'm in the process of replacing my Union{Nothing, T} annotations to
Nullable{T}, but I'm getting stuck when T is a parametric type. For
example, replacing the function
function myfunc(v::Union{Nothing, Vector})
...
end
with
function myfunc(v::Nullable{Vector})
...
end
makes it completely uncallable since any vector I pass in comes with an
eltype parameter and Julia's type parameters are invariant. For functions
or types that only have a couple Nullable types, I can do something like
function myfunc{T}(v::Nullable{Vector{T}})
...
end
type MyType{T}
v::Nullable{Vector{T}
end
but that gets very ugly when you have a large number of nullable types.
Does anyone know of a cleaner way to do this?
Thanks,
Darwin
Re: [julia-users] Re: How to un-substring a string?!
On Tue, Jul 21, 2015 at 8:19 PM, andrew cooke and...@acooke.org wrote:
maybe it should check to see if the end matches?
My guess, is that this is an optimization that won't have too much
benefit because in most cases you can just call `match` with a start
idx.
@glen - i think that will construct an ascii string, which isn't what you
want if the underlying data are unicode. i've always assumed string() does
something smart and returns the right thing, but haven't checked...
And from the code I pasted, `utf` should probably do the copy you want.
andrew
On Tuesday, 21 July 2015 20:49:12 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:38 PM, andrew cooke and...@acooke.org wrote:
ah. for some reason i was thinking they were invisible (somewhere below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that* to
match still gives the memory issue.
Hmmm,
```
match(re::Regex, str::Union{ByteString,SubString}, idx::Integer,
add_opts::UInt32=UInt32(0)) =
match(re, utf8(str), idx, add_opts)
```
So match on a substring does a copy. I'm guessing this is because pcre
expect a c string (i.e. NULL terminated?)
so there's still something odd that i don't understand. maybe it's just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org
wrote:
ok, so match(regex, string, index) solves the problem. presumably it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing
string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that
because of
this you should use bytestring(...) before passing a string to c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to
start
at
as an argument. so i can avoid s[i:end] and just pass i into match
(i
just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a good
method
for
this also. it seems wrong to me also that `match` is making a
copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any
more
efficient solutions (and confirmation the analysis is correct -
is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so
the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so
there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be
passed
to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to
copy
m.match.
as far as i can tell string() doesn't help. so what works?
or
am i
wrong?
thanks,
andrew
[julia-users] Re: SOLVED: ERROR: `start` has no method matching start(::Nothing)
Hehe, I think you can't. I marked it as solved from the start because, well, the problem itself was solved already :-). About error messages, I think Simons suggestion $T is not iterable... would have lead me to the correct conclusion bit faster... Anyway, thanks for a good discussion and for your work on this lovely language! Cheers, Kaj On Tuesday, July 21, 2015 at 11:46:11 PM UTC+3, Ismael VC wrote: Not related, but how do you edit the thread name in order to mark it as SOLVED? I haven't found a way to do this, thanks! El lunes, 20 de julio de 2015, 14:09:00 (UTC-5), Kaj Wiik escribió: I started to get a strange error while debugging my code, here's a simplified example: julia function foo(a) println(foo..) end foo (generic function with 1 method) julia a = foo(2) foo.. julia a,b = foo(2) foo.. ERROR: `start` has no method matching start(::Nothing) So, the problem was a missing return value, it is strange that missing one value did not give error but two values It took a quite long time to track this down. Perhaps a bit more informative error message would be possible...? Cheers, Kaj
[julia-users] Re: How to un-substring a string?!
string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
(i was quite impressed that reverse(reverse(...)) didn't help either). On Tuesday, 21 July 2015 20:11:35 UTC-3, andrew cooke wrote: deepcopy didn't. i haven't actually tried copy. hang on... [computer hangs; oom killer steps in]. nope! On Tuesday, 21 July 2015 20:08:33 UTC-3, Jameson wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
deepcopy didn't. i haven't actually tried copy. hang on... [computer hangs; oom killer steps in]. nope! On Tuesday, 21 July 2015 20:08:33 UTC-3, Jameson wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org javascript: wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtjn...@gmail.com wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) Isn't it `s[i:end]` that is doing the copy? On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
[julia-users] How to un-substring a string?!
well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: MongoDB and Julia
Probably not since it's oriented towards RDBMS. Thanks On Tue, Jul 21, 2015 at 7:11 PM, cdm cdmclean@gmail.com wrote: would a MongoDB JDBC driver paired with JDBC.jl be feasible for your purposes ... ? https://groups.google.com/forum/#!topic/julia-users/qqOTu4XL1HI good luck, cdm
[julia-users] Puzzling over @parallel
Running 0.4.0-dev+5933. I expected the print outs from this code to be the same and to equal zero. What am I doing wrong? Thanks in advance. addprocs(4); @everywhere function test(state) return state.^2.0 end const n = 1; const m = 1; states = randn(n); store = zeros(n,m); for i = 1:n j = 1:m state = states[i] store[i,j] = i-j+test(state) end stored = SharedArray(typeof(1.0),n,m); @parallel for i = 1:n j = 1:m state = states[i] stored[i,j] = i-j+test(state) end println(maximum(store-stored)) println(maximum(store-stored)) rmprocs(2:5);
Re: [julia-users] Re: How to un-substring a string?!
i think that returns a substring (ir a view onto the backing string). but i am not sure. i did read a discussion somewhere saying that because of this you should use bytestring(...) before passing a string to c. which is all the evidence i have for my guess. incidentally, match(...) has a method that takes the offset to start at as an argument. so i can avoid s[i:end] and just pass i into match (i just found this). however, somewhat surprisingly, it also has the same problem. andrew On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote: On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com javascript: wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) Isn't it `s[i:end]` that is doing the copy? On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org javascript: wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
hmm. ignore that last statement (same problem). still checking / confused. sorry. On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote: i think that returns a substring (ir a view onto the backing string). but i am not sure. i did read a discussion somewhere saying that because of this you should use bytestring(...) before passing a string to c. which is all the evidence i have for my guess. incidentally, match(...) has a method that takes the offset to start at as an argument. so i can avoid s[i:end] and just pass i into match (i just found this). however, somewhat surprisingly, it also has the same problem. andrew On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote: On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) Isn't it `s[i:end]` that is doing the copy? On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
ok, so match(regex, string, index) solves the problem. presumably it exists exactly for this reason? andrew On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote: hmm. ignore that last statement (same problem). still checking / confused. sorry. On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote: i think that returns a substring (ir a view onto the backing string). but i am not sure. i did read a discussion somewhere saying that because of this you should use bytestring(...) before passing a string to c. which is all the evidence i have for my guess. incidentally, match(...) has a method that takes the offset to start at as an argument. so i can avoid s[i:end] and just pass i into match (i just found this). however, somewhat surprisingly, it also has the same problem. andrew On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote: On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com wrote: does `copy` work? although `bytestring` also seems like a good method for this also. it seems wrong to me also that `match` is making a copy of the original string (if that is indeed what it is doing) Isn't it `s[i:end]` that is doing the copy? On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org wrote: string(bytestring(...)) seems to do it. would appreciate any more efficient solutions (and confirmation the analysis is correct - is this worth filing as an issue?) On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote: well, this was fun... the following code rapidly triggers the OOM killer on my machine (julia 0.4 trunk): s = repeat(a, 100) l = Any[] r = r^\w for i in 1:length(s) m = match(r, s[i:end]) push!(l, m.match) end note that: (1) the regexp is only matching one character, so the array l is at most a million characters long. what i think is happening (but this is only a guess) is that s[i:end] is being passed though to the c level regexp library as a new string. the result (m.match) is then a substring into that. because the substring is kept around, the backing string cannot be collected. and so there's an n^2 memory use. ideally, i don't think a new copy of the string should be passed to the regexp engine. maybe i am wrong? anyway, for now, if the above is right, i need some way to copy m.match. as far as i can tell string() doesn't help. so what works? or am i wrong? thanks, andrew
Re: [julia-users] Re: How to un-substring a string?!
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org wrote:
ok, so match(regex, string, index) solves the problem. presumably it exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that because of
this you should use bytestring(...) before passing a string to c. which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to start at
as an argument. so i can avoid s[i:end] and just pass i into match (i just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com wrote:
does `copy` work? although `bytestring` also seems like a good method
for
this also. it seems wrong to me also that `match` is making a copy of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any more
efficient solutions (and confirmation the analysis is correct - is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be passed to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to copy
m.match.
as far as i can tell string() doesn't help. so what works? or am i
wrong?
thanks,
andrew
Re: [julia-users] Re: How to un-substring a string?!
ah. for some reason i was thinking they were invisible (somewhere below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that* to
match still gives the memory issue.
so there's still something odd that i don't understand. maybe it's just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org
javascript: wrote:
ok, so match(regex, string, index) solves the problem. presumably it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that
because of
this you should use bytestring(...) before passing a string to c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to start
at
as an argument. so i can avoid s[i:end] and just pass i into match (i
just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a good
method
for
this also. it seems wrong to me also that `match` is making a copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any more
efficient solutions (and confirmation the analysis is correct - is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so
there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be passed
to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to copy
m.match.
as far as i can tell string() doesn't help. so what works? or
am i
wrong?
thanks,
andrew
[julia-users] Iterating over Dicts
A few clarifying questions. I believe that when you iterate over julia Dicts, the iteration is not necessarily in any particular order (i.e. if keys are alpha-numeric it is not necessarily in sorted order, or it is not necessarily in order the keys were inserted). Is that correct? If I set the seed though am I guaranteed the same order each time and between Julia sessions? Thanks Benjamin
[julia-users] Julia's Scheduler and Event Loop
Hi there, I'd like to try some event driven programming, where a segment of code is set to run, but only when an event occurs - think like EventListeners in Node.js. I think this should be pretty easy with Julia's Scheduler, and I'm currently going over the standard library reference documentation to see how this would be done. I see that notify() and conditions are rather like events. I suppose if you schedule a task that waits for a condition as the first statement then it functions like an event listener. I guess if you then put the code in an infinite loop with the statement to wait on the conditions, this would be like the event listeners that are fireweed repeatedly whenerve the event is fired, whereas otherwise the task would just be completed and removed from the scheduler when it was finished, and so is more like the 'once' event listeners from Node.js. I wonder however if there is more detailed information out there on how the Scheduler works - for example, in the flow of a 'normal script', when exactly does control pass to the scheduler: I guess for @schedule, flow goes to the task right away, but say you had several tasks scheduled to be run in a script with several @schedule or @task statements - not waiting for a condition. Does the control go from the script to the task which finishes, back to the script to add the second task to the scheduler, which gets finished. Or once control is switched to the scheduler, does it work through all the tasks it has sued until they are all waiting on something, and then at that point control passes back to the main script? In addition, what variables from my script are accessible from within a task? Thanks, Ben.
Re: [julia-users] Re: How to un-substring a string?!
I've been using ascii().
On Tuesday, July 21, 2015 at 7:38:28 PM UTC-4, andrew cooke wrote:
ah. for some reason i was thinking they were invisible (somewhere below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that* to
match still gives the memory issue.
so there's still something odd that i don't understand. maybe it's just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org wrote:
ok, so match(regex, string, index) solves the problem. presumably it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that
because of
this you should use bytestring(...) before passing a string to c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to start
at
as an argument. so i can avoid s[i:end] and just pass i into match
(i just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a good
method
for
this also. it seems wrong to me also that `match` is making a copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any
more
efficient solutions (and confirmation the analysis is correct -
is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so
there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be passed
to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to copy
m.match.
as far as i can tell string() doesn't help. so what works? or
am i
wrong?
thanks,
andrew
Re: [julia-users] Iterating over Dicts
On Tue, Jul 21, 2015 at 7:37 PM, Benjamin Deonovic bdeono...@gmail.com wrote: A few clarifying questions. I believe that when you iterate over julia Dicts, the iteration is not necessarily in any particular order (i.e. if keys are alpha-numeric it is not necessarily in sorted order, or it is not necessarily in order the keys were inserted). Is that correct? Yes. If I set the seed though am I guaranteed the same order each time and between Julia sessions? Julia don't randomize it intentionally either and my guess is that the order would be repeatable if the hash function is (which might requires turning off address space randomization). Even then, this is not a defined behavior. If you want defined iteration order, see OrderedDict (and to be merged SortedDict) in DataStructures.jl Thanks Benjamin
Re: [julia-users] Re: How to un-substring a string?!
On Tue, Jul 21, 2015 at 7:38 PM, andrew cooke and...@acooke.org wrote:
ah. for some reason i was thinking they were invisible (somewhere below
julia).
ok, thanks. so that explains things more clearly
...except that(!) using SubString(s, i, endof(s)) and passing *that* to
match still gives the memory issue.
Hmmm,
```
match(re::Regex, str::Union{ByteString,SubString}, idx::Integer,
add_opts::UInt32=UInt32(0)) =
match(re, utf8(str), idx, add_opts)
```
So match on a substring does a copy. I'm guessing this is because pcre
expect a c string (i.e. NULL terminated?)
so there's still something odd that i don't understand. maybe it's just
that the regexp lib doesn't know about SubString.
andrew
On Tuesday, 21 July 2015 20:32:53 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:26 PM, andrew cooke and...@acooke.org wrote:
ok, so match(regex, string, index) solves the problem. presumably it
exists
exactly for this reason?
At least I think this is a valid usecase.
andrew
On Tuesday, 21 July 2015 20:23:57 UTC-3, andrew cooke wrote:
hmm. ignore that last statement (same problem). still checking /
confused. sorry.
On Tuesday, 21 July 2015 20:20:46 UTC-3, andrew cooke wrote:
i think that returns a substring (ir a view onto the backing string).
```
julia typeof(aaa[2:end])
ASCIIString
julia SubString(aaa, 2, 3)
aa
julia typeof(SubString(aaa, 2, 3))
SubString{ASCIIString}
```
but i am not sure. i did read a discussion somewhere saying that
because of
this you should use bytestring(...) before passing a string to c.
which is
all the evidence i have for my guess.
incidentally, match(...) has a method that takes the offset to start
at
as an argument. so i can avoid s[i:end] and just pass i into match (i
just
found this).
however, somewhat surprisingly, it also has the same problem.
andrew
On Tuesday, 21 July 2015 20:15:58 UTC-3, Yichao Yu wrote:
On Tue, Jul 21, 2015 at 7:08 PM, Jameson Nash vtj...@gmail.com
wrote:
does `copy` work? although `bytestring` also seems like a good
method
for
this also. it seems wrong to me also that `match` is making a copy
of
the
original string (if that is indeed what it is doing)
Isn't it `s[i:end]` that is doing the copy?
On Tue, Jul 21, 2015 at 6:57 PM andrew cooke and...@acooke.org
wrote:
string(bytestring(...)) seems to do it. would appreciate any more
efficient solutions (and confirmation the analysis is correct - is
this
worth filing as an issue?)
On Tuesday, 21 July 2015 19:33:05 UTC-3, andrew cooke wrote:
well, this was fun... the following code rapidly triggers the
OOM
killer
on my machine (julia 0.4 trunk):
s = repeat(a, 100)
l = Any[]
r = r^\w
for i in 1:length(s)
m = match(r, s[i:end])
push!(l, m.match)
end
note that: (1) the regexp is only matching one character, so the
array l
is at most a million characters long.
what i think is happening (but this is only a guess) is that
s[i:end] is
being passed though to the c level regexp library as a new
string.
the
result (m.match) is then a substring into that. because the
substring is
kept around, the backing string cannot be collected. and so
there's
an n^2
memory use.
ideally, i don't think a new copy of the string should be passed
to
the
regexp engine. maybe i am wrong?
anyway, for now, if the above is right, i need some way to copy
m.match.
as far as i can tell string() doesn't help. so what works? or
am i
wrong?
thanks,
andrew
[julia-users] Translating void* *bufs for a Julia type
I'm trying to wrap some C code that has a struct in a header. I'm trying
to create a Julia type that emulates the following struct:
typedef struct
{
int a;
int b;
int c;
void* *bufs;
int flags;
void* procFunc;
void* resetFunc;
} my_t;
I'm fine with everything except how to handle:
void* *bufs;
Any advice or general thoughts? Thanks in advance - Dave.
[julia-users] Re: Why is the type of a variable different from inside and outside a let block?
Surprising it may be, but also sensible. An assignment always returns the assigned value, not the variable it is assigned to: let x::Float64 = 5 x end::Float64 I think there's something in the docs that describes that behaviour, but I can't seem to find it right now... Am Dienstag, 21. Juli 2015 06:03:03 UTC+2 schrieb Ismael VC: This was surprising: julia versioninfo() Julia Version 0.4.0-dev+5491 Commit cb77503 (2015-06-21 09:45 UTC) Platform Info: System: Linux (x86_64-unknown-linux-gnu) CPU: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 julia let x::Float64 = 5 end::Float64 ERROR: TypeError: typeassert: expected Float64, got Int64 julia let x::UTF8String = test end::UTF8String ERROR: TypeError: typeassert: expected UTF8String, got ASCIIString
Re: [julia-users] Re: Solving nonlinear equations quickly using FastAnonymous @anon and Julia 0.4
@anon builds a type for you, creates a field called w, and defines call() for
that type. IF you're running julia 0.4, that is.
My point was simply, as a busy person I haven't read this long email chain
through and might forget about it. If there's something you want fixed, best to
file an issue with the appropriate package, with a test case that illustrates
the problem you're having.
Best,
--Tim
On Monday, July 20, 2015 09:57:24 PM Andrew wrote:
I'm not quite sure what you mean. It looks like I could sort of replicate
my approach using @anon by just defining an @anon function right below my
initial function definition. Then I can use the @anon version elsewhere.
For example,
function FOC(x,w)
return x*w
end
p_FOC = @anon x - FOC(x,w) # maybe I should say const p_FOC for
performance of global variables?
But this doesn't work because I haven't defined w anywhere. It would work
if I set a dummy value for w, like w = 5., but that looks sort of odd. It
would basically come down to what I'm doing above anyway.
Is that what you meant?
On Monday, July 20, 2015 at 10:01:12 PM UTC-4, Tim Holy wrote:
@anon is supposed to work this way for you under the hood; it's a bit of a
puzzle if it doesn't. Can you file an issue, with test case, with
FastAnonymous?
--Tim
On Monday, July 20, 2015 06:14:24 PM Andrew wrote:
Update: I've found a significantly cleaner way to do this which avoids
FastAnonymous entirely and runs faster for me. Now in 0.4 arbitrary
objects
can be overloaded to use the f() syntax using Base.call. I create an
accessory type for each function that I want to pass to a non-linear
solver
or optimization routine. This accessory type encapsulates the parameters
of
the function, Then I define a new method for Base.call which acts like a
single-variable function, and simply redirects the parameters stored in
my
type into the function. It looks like this
function
hoursFOC(UF::UtilityFunction,hours,state::State{IdioState1,AggState1},
aprime)
w = state.as.w
consump = budget_consump_givenhours(state, aprime, hours)
w*uc(UF,consump,hours) - ul(UF,consump,hours) # Is w*uc = ul
end
immutable pars_hoursFOC{TUF : UtilityFunction, TIS:IdioState,
TAS:AggState}
UF::TUF
state::State{TIS,TAS}
aprime::Float64
end
Base.call(f::pars_hoursFOC, h) = hoursFOC(f.UF, h, f.state, f.aprime)
When I need the nonlinear equation solver, I do
f = pars_hoursFOC(UF,state,aprime)
j = pars_JACOBhoursFOC(UF, state, aprime) # jacobian, same idea
hours = myNewton(f, j, hoursguess)
f and j are treated like functions, so this works as expected.
This is much cleaner than my previous method where I was using @anon to
store an anonymous function, then passing that function around
everywhere.
Also it's faster for some reason. My new code runs in 14s, the version
using @anon was about 20s, and a version with nested functions(or
regular
anonymous functions) would be 30s.
I see mentioned in #8712 https://github.com/JuliaLang/julia/pull/8712
that
there may be a Callable type. That would be useful, since currently
these
callable types don't work with the Optim functions or anything else that
asks for a ::Function argument.
On Thursday, July 16, 2015 at 9:20:51 PM UTC-4, Andrew wrote:
fzero(f, j, guess) works for me when f and j are functions. f(Af,
guess)
works for me now when Af is an @anon function.
On Tuesday, July 7, 2015 at 7:34:39 PM UTC-4, j verzani wrote:
Okay, this just got fixed as much as I could with v0.1.15 (there is
no
fzero(f,j,guess) signature).
On Tuesday, July 7, 2015 at 4:38:41 PM UTC-4, Andrew wrote:
Just checked. So, Roots.fzero(f, guess) does work. However,
Roots.fzero(f, j, guess) doesn't work, and neither does
Roots.newton(f,
j,
guess).
I looked at the Roots.jl source and I see ::Function annotations on
the
methods with the jacobian, but not the regular one.
On Tuesday, July 7, 2015 at 4:22:17 PM UTC-4, j verzani wrote:
It isn't your first choice, but `Roots.fzero` can have `@anon`
functions passed to it, unless I forgot to tag a new version after
making
that change on master not so long ago.
On Tuesday, July 7, 2015 at 2:29:51 PM UTC-4, Andrew wrote:
I'm writing this in case other people are trying to do the same
thing
I've done, and also to see if anyone has any suggestions.
Recently I have been writing some code that requires solving
lots(tens
of thousands) of simple non-linear equations. The application is
economics,
I am solving an intratemporal first order condition for optimal
labor
supply given the state and a savings decision. This requires
solving
the
Re: [julia-users] Mutual exclusion when writing to SharedArrays?
On Monday, July 20, 2015 08:19:43 PM John Brock wrote: Isn't that what DArrays are for, though? With a DArray, each chunk is local to only one process. That makes it really expensive to read data from other chunks. With SharedArrays, you have fast read access to the whole thing. But you have to be careful with updates. Does Julia provide a mechanism for mutual exclusion/marking critical sections? The easiest is to restrict your changes to localindexes(S) I'm imagining something like: shared_result = SharedArray(Int64, (2,), init = S - S[localindexes(S)] = 0 ) @parallel for i=1:3 lock shared_result shared_result[:] += [i, i] end end I'm not aware of any implementation of locking, but you should be able to implement it yourself: https://en.wikipedia.org/wiki/Peterson%27s_algorithm and the alternatives linked in See also. --Tim On Monday, July 20, 2015 at 6:59:03 PM UTC-7, Tim Holy wrote: Usually the whole point of a SharedArray is that workers only update the piece they own. You can make it work different if you implement locking, but lock contention can be a bottleneck. --Tim On Monday, July 20, 2015 04:29:04 PM John Brock wrote: I'm seeing inconsistent results when multiple workers write values to a SharedArray at the same time, presumably because += isn't atomic. Is this intended behavior, and is there a workaround? Behavior is reproducible in 0.3.8-pre+22 and 0.3.9. Sample code: function doStuff() result_shared = SharedArray(Int64, (2,), init = S - S[localindexes(S)] = 0) @sync for i=1:3 @spawn begin result_shared[:] += [i, i] end end return sdata(result_shared) end julia -p 3 julia dump(doStuff()) Array(Int64,(2,)) [3,3] julia dump(doStuff()) Array(Int64,(2,)) [6,6]
Re: [julia-users] Unable to add HDF5
Strange. That commit is present in the repository: see https://github.com/timholy/HDF5.jl/commits/master Anyone else having the same problem? I just did this and it went off without a hitch (julia-old is my installation of the 0.3 release): $ mkdir /tmp/packages $ JULIA_PKGDIR=/tmp/packages julia-old _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 0.3.7-pre+1 (2015-02-17 22:12 UTC) _/ |\__'_|_|_|\__'_| | Commit d15f183* (153 days old release-0.3) |__/ | x86_64-linux-gnu julia Pkg.init() INFO: Initializing package repository /tmp/packages/v0.3 INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl julia Pkg.add(HDF5) INFO: Cloning cache of BinDeps from git://github.com/JuliaLang/BinDeps.jl.git INFO: Cloning cache of Blosc from git://github.com/stevengj/Blosc.jl.git INFO: Cloning cache of Compat from git://github.com/JuliaLang/Compat.jl.git INFO: Cloning cache of HDF5 from git://github.com/timholy/HDF5.jl.git INFO: Cloning cache of SHA from git://github.com/staticfloat/SHA.jl.git INFO: Cloning cache of URIParser from git://github.com/JuliaWeb/URIParser.jl.git INFO: Installing BinDeps v0.3.14 INFO: Installing Blosc v0.1.2 INFO: Installing Compat v0.4.9 INFO: Installing HDF5 v0.5.0 INFO: Installing SHA v0.0.4 INFO: Installing URIParser v0.0.5 INFO: Building Blosc % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1210 1210 0553 0 --:--:-- --:--:-- --:--:-- 555 0 00 233k0 0 221k 0 --:--:-- 0:00:01 --:--:-- 427k Compiling libblosc... CC blosc.c CC blosclz.c CC shuffle.c LINK libblosc INFO: Building HDF5 INFO: Package database updated julia which went like a charm. Maybe try Pkg.update()? --Tim On Monday, July 20, 2015 10:36:25 PM Roger Herikstad wrote: Hi, For some reason I am unable to add HDF5. Not sure if this is related to the recent change splitting JLD off into its own package. This is the error I got ulia Pkg.add(HDF5) INFO: Updating cache of HDF5... ERROR: Missing package versions (possible metadata misconfiguration): HDF5 v(nothing,v0.5.0) [883165af1cef09980a3d9b1718477579d629754c[1:10]] in wait at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib (repeats 2 times) in wait at task.jl:48 in sync_end at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in add at pkg/entry.jl:319 in add at pkg/entry.jl:71 in anonymous at pkg/dir.jl:28 in cd at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in __cd#229__ at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in add at pkg.jl:20 This is my versioninfo: julia versioninfo() Julia Version 0.3.10 Commit c8ceeef* (2015-06-24 13:54 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.4.0) CPU: Intel(R) Xeon(R) CPU X5472 @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 I appreciate any advice on what the problem might be. Thanks!
Re: [julia-users] Unable to add HDF5
Hm, just tried Pkg.update() on 0.4.0-dev+5955 and I got the new version of HDF5 without any problems, so there's probably something wonky going on with Pkg on my 0.3.10. Is there anything I can do to try and reset Pkg, short of nuking my whole ~/.julia/v0.3 directory and starting over? On Tue, Jul 21, 2015 at 6:38 PM, Tim Holy tim.h...@gmail.com wrote: Strange. That commit is present in the repository: see https://github.com/timholy/HDF5.jl/commits/master Anyone else having the same problem? I just did this and it went off without a hitch (julia-old is my installation of the 0.3 release): $ mkdir /tmp/packages $ JULIA_PKGDIR=/tmp/packages julia-old _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 0.3.7-pre+1 (2015-02-17 22:12 UTC) _/ |\__'_|_|_|\__'_| | Commit d15f183* (153 days old release-0.3) |__/ | x86_64-linux-gnu julia Pkg.init() INFO: Initializing package repository /tmp/packages/v0.3 INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl julia Pkg.add(HDF5) INFO: Cloning cache of BinDeps from git:// github.com/JuliaLang/BinDeps.jl.git INFO: Cloning cache of Blosc from git://github.com/stevengj/Blosc.jl.git INFO: Cloning cache of Compat from git:// github.com/JuliaLang/Compat.jl.git INFO: Cloning cache of HDF5 from git://github.com/timholy/HDF5.jl.git INFO: Cloning cache of SHA from git://github.com/staticfloat/SHA.jl.git INFO: Cloning cache of URIParser from git://github.com/JuliaWeb/URIParser.jl.git INFO: Installing BinDeps v0.3.14 INFO: Installing Blosc v0.1.2 INFO: Installing Compat v0.4.9 INFO: Installing HDF5 v0.5.0 INFO: Installing SHA v0.0.4 INFO: Installing URIParser v0.0.5 INFO: Building Blosc % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1210 1210 0553 0 --:--:-- --:--:-- --:--:-- 555 0 00 233k0 0 221k 0 --:--:-- 0:00:01 --:--:-- 427k Compiling libblosc... CC blosc.c CC blosclz.c CC shuffle.c LINK libblosc INFO: Building HDF5 INFO: Package database updated julia which went like a charm. Maybe try Pkg.update()? --Tim On Monday, July 20, 2015 10:36:25 PM Roger Herikstad wrote: Hi, For some reason I am unable to add HDF5. Not sure if this is related to the recent change splitting JLD off into its own package. This is the error I got ulia Pkg.add(HDF5) INFO: Updating cache of HDF5... ERROR: Missing package versions (possible metadata misconfiguration): HDF5 v(nothing,v0.5.0) [883165af1cef09980a3d9b1718477579d629754c[1:10]] in wait at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib (repeats 2 times) in wait at task.jl:48 in sync_end at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in add at pkg/entry.jl:319 in add at pkg/entry.jl:71 in anonymous at pkg/dir.jl:28 in cd at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in __cd#229__ at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/julia /sys.dylib in add at pkg.jl:20 This is my versioninfo: julia versioninfo() Julia Version 0.3.10 Commit c8ceeef* (2015-06-24 13:54 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.4.0) CPU: Intel(R) Xeon(R) CPU X5472 @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 I appreciate any advice on what the problem might be. Thanks! -- ~ Roger
Re: [julia-users] Unable to add HDF5
You could try Pkg.rm(HDF5) and then delete the HDF5 directory in ~/.julia/.cache. --Tim On Tuesday, July 21, 2015 06:45:55 PM Roger Herikstad wrote: Hm, just tried Pkg.update() on 0.4.0-dev+5955 and I got the new version of HDF5 without any problems, so there's probably something wonky going on with Pkg on my 0.3.10. Is there anything I can do to try and reset Pkg, short of nuking my whole ~/.julia/v0.3 directory and starting over? On Tue, Jul 21, 2015 at 6:38 PM, Tim Holy tim.h...@gmail.com wrote: Strange. That commit is present in the repository: see https://github.com/timholy/HDF5.jl/commits/master Anyone else having the same problem? I just did this and it went off without a hitch (julia-old is my installation of the 0.3 release): $ mkdir /tmp/packages $ JULIA_PKGDIR=/tmp/packages julia-old _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | | | | |_| | | | (_| | | Version 0.3.7-pre+1 (2015-02-17 22:12 UTC) _/ |\__'_|_|_|\__'_| | Commit d15f183* (153 days old release-0.3) |__/ | x86_64-linux-gnu julia Pkg.init() INFO: Initializing package repository /tmp/packages/v0.3 INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl julia Pkg.add(HDF5) INFO: Cloning cache of BinDeps from git:// github.com/JuliaLang/BinDeps.jl.git INFO: Cloning cache of Blosc from git://github.com/stevengj/Blosc.jl.git INFO: Cloning cache of Compat from git:// github.com/JuliaLang/Compat.jl.git INFO: Cloning cache of HDF5 from git://github.com/timholy/HDF5.jl.git INFO: Cloning cache of SHA from git://github.com/staticfloat/SHA.jl.git INFO: Cloning cache of URIParser from git://github.com/JuliaWeb/URIParser.jl.git INFO: Installing BinDeps v0.3.14 INFO: Installing Blosc v0.1.2 INFO: Installing Compat v0.4.9 INFO: Installing HDF5 v0.5.0 INFO: Installing SHA v0.0.4 INFO: Installing URIParser v0.0.5 INFO: Building Blosc % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1210 1210 0553 0 --:--:-- --:--:-- --:--:-- 555 0 00 233k0 0 221k 0 --:--:-- 0:00:01 --:--:-- 427k Compiling libblosc... CC blosc.c CC blosclz.c CC shuffle.c LINK libblosc INFO: Building HDF5 INFO: Package database updated julia which went like a charm. Maybe try Pkg.update()? --Tim On Monday, July 20, 2015 10:36:25 PM Roger Herikstad wrote: Hi, For some reason I am unable to add HDF5. Not sure if this is related to the recent change splitting JLD off into its own package. This is the error I got ulia Pkg.add(HDF5) INFO: Updating cache of HDF5... ERROR: Missing package versions (possible metadata misconfiguration): HDF5 v(nothing,v0.5.0) [883165af1cef09980a3d9b1718477579d629754c[1:10]] in wait at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib (repeats 2 times) in wait at task.jl:48 in sync_end at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in add at pkg/entry.jl:319 in add at pkg/entry.jl:71 in anonymous at pkg/dir.jl:28 in cd at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in __cd#229__ at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in add at pkg.jl:20 This is my versioninfo: julia versioninfo() Julia Version 0.3.10 Commit c8ceeef* (2015-06-24 13:54 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.4.0) CPU: Intel(R) Xeon(R) CPU X5472 @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 I appreciate any advice on what the problem might be. Thanks!
[julia-users] Re: Why is the type of a variable different from inside and outside a let block?
It’s because assignment returns the right-hand operand. In other words, x::Float64 = 5 returns 5, not convert(Float64, 5). Thus, the type assertions on the last statements of your let blocks actually *don’t* make sure that the *return value* is of a certain type. Try julia let x = convert(Float64, 5) end::Float64 5.0 // T On Tuesday, July 21, 2015 at 6:03:03 AM UTC+2, Ismael VC wrote: This was surprising: julia versioninfo() Julia Version 0.4.0-dev+5491 Commit cb77503 (2015-06-21 09:45 UTC) Platform Info: System: Linux (x86_64-unknown-linux-gnu) CPU: Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Sandybridge) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 julia let x::Float64 = 5 end::Float64 ERROR: TypeError: typeassert: expected Float64, got Int64 julia let x::UTF8String = test end::UTF8String ERROR: TypeError: typeassert: expected UTF8String, got ASCIIString
[julia-users] Re: Indexing Array with empty vector in v.0.4.
But I think Julia started off as very intuitive and educational and I hope it will continue on this road =) + 1e1e1e...
Re: [julia-users] Unable to add HDF5
Ah, of course. Thanks! It works now. Sorry for the noise. On Tue, Jul 21, 2015 at 6:51 PM, Tim Holy tim.h...@gmail.com wrote: You could try Pkg.rm(HDF5) and then delete the HDF5 directory in ~/.julia/.cache. --Tim On Tuesday, July 21, 2015 06:45:55 PM Roger Herikstad wrote: Hm, just tried Pkg.update() on 0.4.0-dev+5955 and I got the new version of HDF5 without any problems, so there's probably something wonky going on with Pkg on my 0.3.10. Is there anything I can do to try and reset Pkg, short of nuking my whole ~/.julia/v0.3 directory and starting over? On Tue, Jul 21, 2015 at 6:38 PM, Tim Holy tim.h...@gmail.com wrote: Strange. That commit is present in the repository: see https://github.com/timholy/HDF5.jl/commits/master Anyone else having the same problem? I just did this and it went off without a hitch (julia-old is my installation of the 0.3 release): $ mkdir /tmp/packages $ JULIA_PKGDIR=/tmp/packages julia-old _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | | | | |_| | | | (_| | | Version 0.3.7-pre+1 (2015-02-17 22:12 UTC) _/ |\__'_|_|_|\__'_| | Commit d15f183* (153 days old release-0.3) |__/ | x86_64-linux-gnu julia Pkg.init() INFO: Initializing package repository /tmp/packages/v0.3 INFO: Cloning METADATA from git://github.com/JuliaLang/METADATA.jl julia Pkg.add(HDF5) INFO: Cloning cache of BinDeps from git:// github.com/JuliaLang/BinDeps.jl.git INFO: Cloning cache of Blosc from git:// github.com/stevengj/Blosc.jl.git INFO: Cloning cache of Compat from git:// github.com/JuliaLang/Compat.jl.git INFO: Cloning cache of HDF5 from git://github.com/timholy/HDF5.jl.git INFO: Cloning cache of SHA from git:// github.com/staticfloat/SHA.jl.git INFO: Cloning cache of URIParser from git://github.com/JuliaWeb/URIParser.jl.git INFO: Installing BinDeps v0.3.14 INFO: Installing Blosc v0.1.2 INFO: Installing Compat v0.4.9 INFO: Installing HDF5 v0.5.0 INFO: Installing SHA v0.0.4 INFO: Installing URIParser v0.0.5 INFO: Building Blosc % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 1210 1210 0553 0 --:--:-- --:--:-- --:--:-- 555 0 00 233k0 0 221k 0 --:--:-- 0:00:01 --:--:-- 427k Compiling libblosc... CC blosc.c CC blosclz.c CC shuffle.c LINK libblosc INFO: Building HDF5 INFO: Package database updated julia which went like a charm. Maybe try Pkg.update()? --Tim On Monday, July 20, 2015 10:36:25 PM Roger Herikstad wrote: Hi, For some reason I am unable to add HDF5. Not sure if this is related to the recent change splitting JLD off into its own package. This is the error I got ulia Pkg.add(HDF5) INFO: Updating cache of HDF5... ERROR: Missing package versions (possible metadata misconfiguration): HDF5 v(nothing,v0.5.0) [883165af1cef09980a3d9b1718477579d629754c[1:10]] in wait at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib (repeats 2 times) in wait at task.jl:48 in sync_end at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in add at pkg/entry.jl:319 in add at pkg/entry.jl:71 in anonymous at pkg/dir.jl:28 in cd at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in __cd#229__ at /home/roger/Applications/Julia-0.3.10.app/Contents/Resources/julia/lib/jul ia /sys.dylib in add at pkg.jl:20 This is my versioninfo: julia versioninfo() Julia Version 0.3.10 Commit c8ceeef* (2015-06-24 13:54 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.4.0) CPU: Intel(R) Xeon(R) CPU X5472 @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT DYNAMIC_ARCH NO_AFFINITY Penryn) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 I appreciate any advice on what the problem might be. Thanks! -- ~ Roger
[julia-users] Re: NetCDF Pkg fails when the time is of type 'long'
Ok, I think with the latest master branch this should work now, could you please try to run Pkg.checkout(NetCDF) and then try to read your file again?
[julia-users] Deducing probability density functions from model equations
Dear all, I don't know if that's the best place to ask such a question but I'll give it a try: I need to code stochastic models: xnplus1 = f(xn, theta) where xn is the state of my system at time n and theta is a set of parameters for this model, constant through time, and possibly containing noise. As a matter of example, let's consider the following model: function transition(n, xn, theta) e = rand(Normal(theta.mu, theta.sigma)) xnplus1 = xn + e return xnplus1 end My goal is to write such equations and to deduce automatically the transition probability density function: p(xnplus1 | xn, theta). I intended to parse the code of the model and look for the rand keyword, the name of the law used to generate this random variable and the values given to it. In the previous case, I could deduce that: p(xnplus1 | xn, theta) = normal_pdf(xnplus1, xn + theta.mu, theta.sigma) where normal_pdf(x, mu, sigma) = 1 / (sigma * sqrt(2 * pi)) * exp( - 1 / 2 * (x - mu) ^ 2 / sigma ^ 2) (rather) easily I think by specifying that whenever a constant is added to a normal variable, the mean of the new variable (left value in the equation) is normal and of mean increased by this constant and of the same standard deviation. But that's the simplest case of all and it requires me to specify some rules about the normal distribution. I therefore have the following questions: - is what I'm trying to do understandable? - is it doable at all? and in Julia? - is specifying rules for each distribution (normal, uniform, Poisson; correlated, uncorrelated) the way to go or should I think of something even more generic? - do you have any other suggestions to solve this problem? Thank you very much,
[julia-users] Re: NetCDF Pkg fails when the time is of type 'long'
You should probably raise an issue on the NetCDF.jl
repository: https://github.com/meggart/NetCDF.jl/issues
On Tuesday, 21 July 2015 04:56:10 UTC+1, RBhupi wrote:
I havent done much with Julia yet so cant provide more info at this time.
ncinfo and ncread fails when the time axis is of type 'long'.
--Terminal---
julia t=NetCDF.ncread(/home/bhupendra/temp/temp-test/test.nc, time)
ERROR: NetCDF type currently not supported, please file an issue on
https://github.com/meggart/NetCDF.jl
in readvar at /home/bhupendra/.julia/v0.3/NetCDF/src/NetCDF.jl:126
in readvar at /home/bhupendra/.julia/v0.3/NetCDF/src/NetCDF.jl:107
in ncread at /home/bhupendra/.julia/v0.3/NetCDF/src/NetCDF.jl:379
--Terminal
julia NetCDF.ncinfo(/home/bhupendra/temp/temp-test/test.nc)
# NetCDF File #
/home/bhupendra/temp/temp-test/test.nc
# Dimensions #
Name Length
---
latitude 256
time 4716
longitude 256
# Variables #
NameType Dimensions
---
latitude FLOAT latitude,
timeError showing value of type NcFile:
ERROR: key not found: 10
in getindex at dict.jl:617
in show at /home/bhupendra/.julia/v0.3/NetCDF/src/NetCDF.jl:505
in anonymous at show.jl:1159
in with_output_limit at ./show.jl:1136
in showlimited at show.jl:1158
in writemime at replutil.jl:2
in display at REPL.jl:117
in display at REPL.jl:120
in display at multimedia.jl:149
in print_response at REPL.jl:139
in print_response at REPL.jl:124
in anonymous at REPL.jl:586
in run_interface at ./LineEdit.jl:1379
in run_frontend at ./REPL.jl:818
:_Documentation =
http://www.unidata.ucar.edu/software/netcdf/docs/cxx4/index.html; ; in
run_repl at ./REPL.jl:169
in _start at ./client.jl:400
julia
---
*ncdump -h output looks like this *ncdump -h
/home/bhupendra/temp/temp-test/test.nc
netcdf test {
dimensions:
latitude = 256 ;
longitude = 256 ;
time = UNLIMITED ; // (4716 currently)
variables:
float latitude(latitude) ;
latitude:units = degrees_north ;
float longitude(longitude) ;
longitude:units = degrees_east ;
int64 time(time) ;
time:units = seconds since 1970-01-01 00:00:00 UTC ;
float rain(time, latitude, longitude) ;
rain:units = mm/hr ;
rain:_FillValue = -999.f ;
rain:_compression = Zlib with HDF-5 ;
// global attributes:
:_libNCIO = developed by Bhupendra Raut (www.baraut.info) at
Monash Weather and Climate Group. ;
:_Documentation =
http://www.unidata.ucar.edu/software/netcdf/docs/cxx4/index.html; ;
:_created = Wed May 13 16:26:12 2015\n ;
Cheers,
RB
[julia-users] Re: NetCDF Pkg fails when the time is of type 'long'
I am currently working on a fix, so far reading and writing NC_LONG data was not supported, I will push the changes later today. Fabian
[julia-users] Re: MongoDB and Julia
Hi, I just spoke to Jesse Davis, the author and maintainer of the C Mongo driver, and he gave me some precious advice. The C driver passes the YAML tests. Using this driver's public API will also get me the implementation of two specs right away and for free: the Server Selection Spec and the Server Discovery and Monitoring Spec, which is probably the most complicated part about MongoDB and allows an app using it to be highly available. He pointed out to the Lua driver, which is a wrapper around the C driver https://github.com/10gen-labs/mongorover He said the Julia driver won't need to test against YAML tests or the two specs mentioned. He told me to pass a connection string to the C driver from Julia, and the C driver will implement the specs. On Monday, July 13, 2015 at 6:30:33 PM UTC-3, Kevin Liu wrote: Thanks Jeff, I will look into it and see if that's the case. I will review it carefully because I want the driver to run smoothly. In my last post, there was a typo, so I'm just pasting the source http://www.slideshare.net/NorbertoLeite/how-mongodb-drv for slide 28/29. On Monday, July 13, 2015 at 6:19:08 PM UTC-3, Jeff Waller wrote: On Monday, July 13, 2015 at 3:27:49 AM UTC-4, Kevin Liu wrote: Any help would be greatly appreciated. I am even debating over the idea of contributing to the development of this package because I believe so much in the language and need to use MongoDB. I think this is why it's untestable. https://travis-ci.org/pzion/Mongo.jl/jobs/54034564 Lytol/Mongo.jl looks abandoned. It has a bunch of issues created over the past 2 years and the last update was in 2013 the pzion repo is a fork which was updated 4 months ago, maybe it's abandoned too and you'll have to fork. But it's at least work contacting him.
[julia-users] Re: Errors while trying to use cxx and embed Julia together
Hello again, Unfortunately and let me apologize in advance I still find myself confused on how to make things work properly. Here are the symbolic links. I believe with some simple changes to Cxx, the need for these will disappear, but for now there's where it searches. /usr/local/julia/lib bizarro% ls -l total 16 drwxr-xr-x 4 jeffw staff 136 Jul 18 18:26 clang lrwxr-xr-x 1 root staff10 Jul 19 03:16 include - ../include drwxr-xr-x 49 jeffw staff 1666 Jul 18 20:11 julia lrwxr-xr-x 1 root staff 1 Jul 19 03:16 lib - . If I get this right does it mean that for all the files that are included in the list * DirectoryLayout* https://gist.github.com/waTeim/ec622a0630f220e6b3c3#file-directorylayout I need to create a soft link to link them with the directory /usr/local/julia/lib (which doesn't exist in my system)? Aslo regarding the instructions of part *BuildJuliaLLVMforCXX* https://gist.github.com/waTeim/ec622a0630f220e6b3c3#file-buildjuliallvmforcxx where you present the files that need to be cp to the directory julia-f428392003 in my system I face the following problems: 1) The image files (sys.ji, sys.dylib,sys-debug.dylib) don't exist anywhere in my system 2) The folder llvm-svn (in the source julia directory) has completely different structure in my system, In my julia (source code) folder there are two folders with the name llvm-svn the one has a structure that you can look in the attach picture called name (llvmsn), while the second one is located in the path /home/kostav/julia/usr-staging/llvm-svn and has a single folder named build_Release+Asserts. All in all I am confused on how to follow your instructions (for which I do really appreciate the effort you put to provide me with). Is what I have to do: 1) Try to copy paste the folders as presented in *BuildJuliaLLVMforCXX* https://gist.github.com/waTeim/ec622a0630f220e6b3c3#file-buildjuliallvmforcxx ? 2) Create links for all the files listed * DirectoryLayout * https://gist.github.com/waTeim/ec622a0630f220e6b3c3#file-directorylayouttowards the folder /usr/local/julia/lib? On Sunday, July 19, 2015 at 10:44:07 PM UTC+2, Jeff Waller wrote: Symbolic links to link which files though? Sorry if the questions I raise are already answered by the git, I just failed to understand it properly so that's why I come back with these questions now. I've added a listing of the entire directory structure https://gist.github.com/waTeim/ec622a0630f220e6b3c3#file-directorylayout Here are the symbolic links. I believe with some simple changes to Cxx, the need for these will disappear, but for now there's where it searches. /usr/local/julia/lib bizarro% ls -l total 16 drwxr-xr-x 4 jeffw staff 136 Jul 18 18:26 clang lrwxr-xr-x 1 root staff10 Jul 19 03:16 include - ../include drwxr-xr-x 49 jeffw staff 1666 Jul 18 20:11 julia lrwxr-xr-x 1 root staff 1 Jul 19 03:16 lib - .
Re: [julia-users] memory use profiling
i can't see how to get a dump of objects on the heap, or tools to analyse that dump. In 0.4 there is a new --track-allocation command-line option that might help. If you don't mind reading LLVM IR, you can also look for calls to jl_gc_allocobj (or just allocobj in 0.3). On Tue, Jul 21, 2015 at 9:08 AM, andrew cooke and...@acooke.org wrote: hi, i feel i must be missing something obvious, sorry, but i can work out what tools are available for understanding memory leaks inside my own julia code. this isn't an issue with julia itself - it's my own dumb fault. but i can't see how to get a dump of objects on the heap, or tools to analyse that dump. can someone point me in the right direction, please? thanks, andrew (i know about - and have used profile and profileview - but they don't have memory info, do they? they do have gc information, but what i want is heap use - gc isn't being called!)
Re: [julia-users] Error when installing package DataFrames
If you don't do this, you get all kinds of spam on the list, mostly from recruiters. On Mon, Jul 20, 2015 at 9:53 PM, Kevin Squire kevin.squ...@gmail.com wrote: On Mon, Jul 20, 2015 at 4:39 PM, Yichao Yu yyc1...@gmail.com wrote: On Mon, Jul 20, 2015 at 7:31 PM, Kevin Squire kevin.squ...@gmail.com wrote: Since first posts are moderated, Serge probably posted once, didn't see his post arrive, and posted again a little later. I thought julia-user isn't moderated Only first (new member) posts are moderated.
Re: [julia-users] Deducing probability density functions from model equations
In general, f(x,theta) does not necessarily belong to any frequently used distribution family, even if theta does -- it is easy to come up with examples. Some distribution families are closed under certain operations (eg addition, and multiplication by scalars for the normal), and some distributions arise as transformations of other distributions (eg chi square from normal, etc). AFAIK Distributions.jl does not support arithmetic on distributions, but you can always try programming the relevant generic functions, eg + for (Real, Distributions.Normal) if the results of your transformations are distributions in Distribution.jl. Best, Tamas On Tue, Jul 21 2015, amik...@gmail.com wrote: Dear all, I don't know if that's the best place to ask such a question but I'll give it a try: I need to code stochastic models: xnplus1 = f(xn, theta) where xn is the state of my system at time n and theta is a set of parameters for this model, constant through time, and possibly containing noise. As a matter of example, let's consider the following model: function transition(n, xn, theta) e = rand(Normal(theta.mu, theta.sigma)) xnplus1 = xn + e return xnplus1 end My goal is to write such equations and to deduce automatically the transition probability density function: p(xnplus1 | xn, theta). I intended to parse the code of the model and look for the rand keyword, the name of the law used to generate this random variable and the values given to it. In the previous case, I could deduce that: p(xnplus1 | xn, theta) = normal_pdf(xnplus1, xn + theta.mu, theta.sigma) where normal_pdf(x, mu, sigma) = 1 / (sigma * sqrt(2 * pi)) * exp( - 1 / 2 * (x - mu) ^ 2 / sigma ^ 2) (rather) easily I think by specifying that whenever a constant is added to a normal variable, the mean of the new variable (left value in the equation) is normal and of mean increased by this constant and of the same standard deviation. But that's the simplest case of all and it requires me to specify some rules about the normal distribution. I therefore have the following questions: - is what I'm trying to do understandable? - is it doable at all? and in Julia? - is specifying rules for each distribution (normal, uniform, Poisson; correlated, uncorrelated) the way to go or should I think of something even more generic? - do you have any other suggestions to solve this problem? Thank you very much,
[julia-users] memory use profiling
hi, i feel i must be missing something obvious, sorry, but i can work out what tools are available for understanding memory leaks inside my own julia code. this isn't an issue with julia itself - it's my own dumb fault. but i can't see how to get a dump of objects on the heap, or tools to analyse that dump. can someone point me in the right direction, please? thanks, andrew (i know about - and have used profile and profileview - but they don't have memory info, do they? they do have gc information, but what i want is heap use - gc isn't being called!)
[julia-users] Passing keyword arguments through a function
I am looking for a way to automatically pass on all keyword arguments through a function. Naively I would like to be able to do something like this: f(x::Int; y::Int=5, z::Int=3) = x+y+z f(x::Float64; args...) = f(floor(x), args...) But that doesn't work currently (actually, it causes StackOverflowError in 0.4.0). So far I've been using the following instead: f(x::Float64, y::Int=5, z::Int=3) = f(floor(x), y=y, z=z) But that gets messy very quickly if there are several keyword arguments to handle. Does anyone know of a clean way to do this?
