[julia-users] Re: calculating limit of a function
thank you
On Monday, 11 August 2014 09:42:07 UTC-4, j verzani wrote:
The SymPy package adds / for PyObjects and other functions so that working
with SymPy is a bit easier that going through PyCall directly. The package
needed a besselj function though. One is added now, output below. The
problem is that the sympy functions are not generated programmatically,
rather added by hand one by one.
julia using SymPy
julia r= Sym(r)
julia limit(besselj(1, r)/r, r, 0)
1/2
On Monday, August 11, 2014 4:37:49 AM UTC-4, Hans W Borchers wrote:
As far as I see, Sympy *does* know the limit at 0.0:
from sympy import *
x = symbols('x')
limit(besselj(1, x)/x, x, 0)
1/2
If in Julia you call it like limit(besselj(1, x)/x, x, 0), then Julia
assumes it to be its own function, not SymPy's.
If on the other hand you call all SymPy functions fully expanded:
julia using PyCall
julia @pyimport sympy
julia x = sympy.symbols(x)
julia sympy.limit(sympy.besselj(1, x)/x, x, 0.0)
`/` has no method matching /(::PyObject, ::PyObject)
So the problem seems to be with how PyCall interprets this expression
before sending it to SymPy.
On Monday, August 11, 2014 5:55:56 AM UTC+2, Zahirul ALAM wrote:
Is there way of calculating limit of a function?
I have tried SymPy Package. But it unfortunately doesnot compute limit
for bessel function.
r= Sym(r)
limit(besselj(1, r)/r, r, 0)
returns the following error:
`besselj` has no method matching besselj(::Int64, ::Sym)
while loading In[27], in expression starting on line 1
the it works for cosine, sine,
r= Sym(r)
limit(sin( r)/r, r, 0)
Any idea? is there a built an undocumented Julia function which can
calculate limit?
[julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
Re: [julia-users] JuliaCon Opening Session Videos Posted!
Yes, very sorry Pontus. Luckily I used your trick for the youtube and blog post, and then must have gotten over-confident when doing this post! -Jacob On Tue, Aug 12, 2014 at 12:26 AM, Stefan Karpinski ste...@karpinski.org wrote: Sorry, Pontus – I spelled your name wrong in an entirely different (and far less American) manner. Although star dwelling vs stone dwelling is maybe ok ;-) On Tue, Aug 12, 2014 at 12:11 AM, Pontus Stenetorp pon...@stenetorp.se wrote: To Jacob and the team, thank you for all your hard work. These videos will surely help spread the word about Julia. On 11 August 2014 20:53, Jacob Quinn karbar...@gmail.com wrote: Gather round and here the tales of a wonderous new language, presented by Tim Holy, Pontus Stenetrop, and Arch Robison. Just to nip this one in the bud, it is Stenetorp, not Stenetrop or Sterntorp [1] (I like the last one, it has a woody quality to it). No harm done, I am used to it and do find it amusing to see possible permutations of my name. =) I have a bit of a trick that I use to avoid getting names wrong. If the name is not common enough not to be in the spelling dictionary, I copy paste the blasted thing, this is something that I have learned over the years after messing up the first line of far too many e-mails... Pontus [1]: https://news.ycombinator.com/item?id=8163607
[julia-users] Re: [PSA]: curve_fit from Optim.jl is moving to LsqFit.jl
Can LsqFit.jl be used to fit multivariate model? I tried this but must have missed something (maybe xv and Y need to have same length?): using LsqFit # We want to try to use curve_fit to fit the parameters of this multivariate model: mvmodel(x, p) = begin p[1] .* ((x[1,:] .^ p[2]) ./ (x[2,:] .^ p[3])) end N = 10 M = 2 X = randn(M, N) Params = [1.0, 2.0, 2.0] # Actual params that we are looking for # Dependent var with small error term Y = mvmodel(X, Params)[:] + 0.01*randn(N) # We need a vector to submit to curve_fit: xv = X[:] # Reshape to get back the original matrix. orig_matrix(xv, M) = reshape(xv, (M, int(length(xv)/M))) # The model function we supply to curve_fit needs to take a vector... model(xv, p) = mvmodel(orig_matrix(xv, M), p) fit = curve_fit(model, xv, Y, [0.5, 0.5, 0.5]) Den måndagen den 14:e juli 2014 kl. 20:10:02 UTC+2 skrev Blake Johnson: The curve fitting functionality in Optim.jl is being moved into its own package: LsqFit.jl: https://github.com/JuliaOpt/LsqFit.jl Installable via Pkg.add(LsqFit).
[julia-users] Re: [PSA]: curve_fit from Optim.jl is moving to LsqFit.jl
Actually this might be a bug. I really don't understand this behavior (latest 0.3.0-rc3): feldt:~/tmp$ julia03 _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 0.3.0-rc3 (2014-08-10 02:54 UTC) _/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release |__/ | x86_64-apple-darwin13.3.0 julia -0.11292333772901704 ^ 1.060554544523 -0.11292184633488864 julia x = -0.11292333772901704 -0.11292333772901704 julia p = 1.060554544523 1.060554544523 julia x ^ p ERROR: DomainError in ^ at math.jl:252 Den tisdagen den 12:e augusti 2014 kl. 14:04:58 UTC+2 skrev Robert Feldt: Can LsqFit.jl be used to fit multivariate model? I tried this but must have missed something (maybe xv and Y need to have same length?): using LsqFit # We want to try to use curve_fit to fit the parameters of this multivariate model: mvmodel(x, p) = begin p[1] .* ((x[1,:] .^ p[2]) ./ (x[2,:] .^ p[3])) end N = 10 M = 2 X = randn(M, N) Params = [1.0, 2.0, 2.0] # Actual params that we are looking for # Dependent var with small error term Y = mvmodel(X, Params)[:] + 0.01*randn(N) # We need a vector to submit to curve_fit: xv = X[:] # Reshape to get back the original matrix. orig_matrix(xv, M) = reshape(xv, (M, int(length(xv)/M))) # The model function we supply to curve_fit needs to take a vector... model(xv, p) = mvmodel(orig_matrix(xv, M), p) fit = curve_fit(model, xv, Y, [0.5, 0.5, 0.5]) Den måndagen den 14:e juli 2014 kl. 20:10:02 UTC+2 skrev Blake Johnson: The curve fitting functionality in Optim.jl is being moved into its own package: LsqFit.jl: https://github.com/JuliaOpt/LsqFit.jl Installable via Pkg.add(LsqFit).
[julia-users] Re: [PSA]: curve_fit from Optim.jl is moving to LsqFit.jl
Robert, your first expression is really julia -(0.11292333772901704 ^ 1.060554544523) -(0.11292333772901704 ^ 1.060554544523) while the second one changes it to julia (-0.11292333772901704) ^ 1.060554544523 ERROR: DomainError in ^ at math.jl:252 which is indeed mathematically undefined. On Tuesday, August 12, 2014 2:24:39 PM UTC+2, Robert Feldt wrote: Actually this might be a bug. I really don't understand this behavior (latest 0.3.0-rc3): feldt:~/tmp$ julia03 _ _ _ _(_)_ | A fresh approach to technical computing (_) | (_) (_)| Documentation: http://docs.julialang.org _ _ _| |_ __ _ | Type help() for help. | | | | | | |/ _` | | | | |_| | | | (_| | | Version 0.3.0-rc3 (2014-08-10 02:54 UTC) _/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release |__/ | x86_64-apple-darwin13.3.0 julia -0.11292333772901704 ^ 1.060554544523 -0.11292184633488864 julia x = -0.11292333772901704 -0.11292333772901704 julia p = 1.060554544523 1.060554544523 julia x ^ p ERROR: DomainError in ^ at math.jl:252 Den tisdagen den 12:e augusti 2014 kl. 14:04:58 UTC+2 skrev Robert Feldt: Can LsqFit.jl be used to fit multivariate model? I tried this but must have missed something (maybe xv and Y need to have same length?): using LsqFit # We want to try to use curve_fit to fit the parameters of this multivariate model: mvmodel(x, p) = begin p[1] .* ((x[1,:] .^ p[2]) ./ (x[2,:] .^ p[3])) end N = 10 M = 2 X = randn(M, N) Params = [1.0, 2.0, 2.0] # Actual params that we are looking for # Dependent var with small error term Y = mvmodel(X, Params)[:] + 0.01*randn(N) # We need a vector to submit to curve_fit: xv = X[:] # Reshape to get back the original matrix. orig_matrix(xv, M) = reshape(xv, (M, int(length(xv)/M))) # The model function we supply to curve_fit needs to take a vector... model(xv, p) = mvmodel(orig_matrix(xv, M), p) fit = curve_fit(model, xv, Y, [0.5, 0.5, 0.5]) Den måndagen den 14:e juli 2014 kl. 20:10:02 UTC+2 skrev Blake Johnson: The curve fitting functionality in Optim.jl is being moved into its own package: LsqFit.jl: https://github.com/JuliaOpt/LsqFit.jl Installable via Pkg.add(LsqFit).
[julia-users] character encoding
Hi, I'm using ODBC to query a DB. As part of the returned query, I get objects of type UTF8String. The problem is that characters like ã are printed as \u8f. How can I properly handle character encoding ? Thanks. Frederico.
Re: [julia-users] Re: erroneous cholfact behavior in conjunction with addprocs()
I don't see this with my own build of OpenBLAS. @staticfloat, how old are the bottles? Med venlig hilsen Andreas Noack 2014-08-12 0:15 GMT-04:00 Thomas Covert thom.cov...@gmail.com: I think this is broken again: I'm observing single-threaded failures (i.e., the last line of the sample code fails when the addprocs(8) line is not commented out) on both a somewhat recent OS X home-brew build and also the current --HEAD home-brew build right now. I am using the standard 64-bit openblas, arpack, and suite-sparse from @staticfloat's home-brew setup (i.e., compiled without the --HEAD option). If it helps, here is my versioninfo(): *julia **versioninfo()* Julia Version 0.4.0-prerelease+52 Commit daa3f82* (2014-08-12 03:33 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.3.0) CPU: Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 -thom On Friday, July 18, 2014 4:34:06 PM UTC-5, Viral Shah wrote: This is now fixed now on master by using a new openblas release that fixes the bug. You can get the fix in tomorrow's nightly. Thanks for reporting! -viral On Thursday, July 10, 2014 4:20:07 AM UTC+5:30, Thomas Covert wrote: I have found cholfact to behave differently (erroneously?) under parallel processing contexts than under standard settings. What I mean by parallel processing is simply having previously called addprocs(). Here is some example code that I am running on my mid-2009 MacBook Pro using a somewhat recent brew of @staticfloat's homebrew distribution: addprocs(8) N = 1000 x = 10 * randn(N) X = zeros(N,N) for i = 1:N for j = 1:N X[i,j] = exp(-.5 * (x[i]-x[j])^2) end end X = X + diagm(.5 * ones(N)) C = cholfact(X) iC = inv(C) CiC = cholfact(iC) I believe this code generates an X which is positive definite by construction. If I run this code as-is, I get the following error (or something similar, the PosDefException sometimes changes): *ERROR: PosDefException(12)* * in cholfact! at linalg/factorization.jl:36* * in cholfact at linalg/factorization.jl:39* *while loading /Users/tcovert/path_to_code.jl, in expression starting on line 16* However, if I comment out the addprocs(8) line, everything works fine. Also, for smaller values of N the problem goes away (N=100,200 is fine, N=400 is not). Here is my versioninfo() if that helps: *julia **versioninfo()* Julia Version 0.3.0-prerelease+3868 Commit e7a9a7d* (2014-06-24 19:39 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.2.0) CPU: Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm
Re: [julia-users] Re: [PSA]: curve_fit from Optim.jl is moving to LsqFit.jl
Yes, sorry, seems I'm still on vacation... ;) Ruby parses this another way
so I confused the two languages.
Actually, by changing my model in the LsqFit example I now also get the
multivariate curve fitting to work; code below. I guess in general
convergence might be a problem unless the initial guess is relatively close.
If there are faster/simpler ways to accomplish this (with LsqFit) please
share.
Regards, Robert
using LsqFit
# Try to use curve_fit for this multivariate model:
mvmodel(x, p) = begin
n = abs(x[1,:]) .^ p[2]
d = abs(x[2,:]) .^ p[3]
p[1] .* (n ./ d)
end
N = 10
M = 2
X = randn(M, N)
Params = [1.0, 2.0, 2.0] # Actual params that we are looking for
errors = 0.01*randn(N)
# Dependent var with small error term
Y = mvmodel(X, Params)[:] .+ errors
# We need a vector to submit to curve_fit:
xv = X[:] # reshape(X, (N*M, 1))[:]
# Reshape to get back the original matrix.
orig_matrix(xv, M) = reshape(xv, (M, int(length(xv)/M)))
# The model function we supply to curve_fit needs to take a vector...
model(xv, p) = mvmodel(orig_matrix(xv, M), p)[:]
fit = curve_fit(model, xv, Y, [0.5, 1.0, 1.0])
julia fit.param
3-element Array{Float64,1}:
0.999694
1.99672
1.99926
On Tue, Aug 12, 2014 at 3:15 PM, Hans W Borchers hwborch...@gmail.com
wrote:
Robert,
your first expression is really
julia -(0.11292333772901704 ^ 1.060554544523)
-(0.11292333772901704 ^ 1.060554544523)
while the second one changes it to
julia (-0.11292333772901704) ^ 1.060554544523
ERROR: DomainError
in ^ at math.jl:252
which is indeed mathematically undefined.
On Tuesday, August 12, 2014 2:24:39 PM UTC+2, Robert Feldt wrote:
Actually this might be a bug. I really don't understand this behavior
(latest 0.3.0-rc3):
feldt:~/tmp$ julia03
_
_ _ _(_)_ | A fresh approach to technical computing
(_) | (_) (_)| Documentation: http://docs.julialang.org
_ _ _| |_ __ _ | Type help() for help.
| | | | | | |/ _` | |
| | |_| | | | (_| | | Version 0.3.0-rc3 (2014-08-10 02:54 UTC)
_/ |\__'_|_|_|\__'_| | Official http://julialang.org/ release
|__/ | x86_64-apple-darwin13.3.0
julia -0.11292333772901704 ^ 1.060554544523
-0.11292184633488864
julia x = -0.11292333772901704
-0.11292333772901704
julia p = 1.060554544523
1.060554544523
julia x ^ p
ERROR: DomainError
in ^ at math.jl:252
Den tisdagen den 12:e augusti 2014 kl. 14:04:58 UTC+2 skrev Robert Feldt:
Can LsqFit.jl be used to fit multivariate model? I tried this but must
have missed something (maybe xv and Y need to have same length?):
using LsqFit
# We want to try to use curve_fit to fit the parameters of this
multivariate model:
mvmodel(x, p) = begin
p[1] .* ((x[1,:] .^ p[2]) ./ (x[2,:] .^ p[3]))
end
N = 10
M = 2
X = randn(M, N)
Params = [1.0, 2.0, 2.0] # Actual params that we are looking for
# Dependent var with small error term
Y = mvmodel(X, Params)[:] + 0.01*randn(N)
# We need a vector to submit to curve_fit:
xv = X[:]
# Reshape to get back the original matrix.
orig_matrix(xv, M) = reshape(xv, (M, int(length(xv)/M)))
# The model function we supply to curve_fit needs to take a vector...
model(xv, p) = mvmodel(orig_matrix(xv, M), p)
fit = curve_fit(model, xv, Y, [0.5, 0.5, 0.5])
Den måndagen den 14:e juli 2014 kl. 20:10:02 UTC+2 skrev Blake Johnson:
The curve fitting functionality in Optim.jl is being moved into its own
package: LsqFit.jl:
https://github.com/JuliaOpt/LsqFit.jl
Installable via Pkg.add(LsqFit).
--
Best regards,
/Robert Feldt
--
Tech. Dr. (PhD), Professor of Software Engineering
Blekinge Institute of Technology, Software Engineering Research Lab, and
Chalmers, Software Engineering Dept
Explanea.com - Igniting your Software innovation
robert.feldt (a) bth.seorrobert.feldt (a) chalmers.seor
robert.feldt (a) gmail.com
Mobile phone: +46 (0) 733 580 580
http://www.robertfeldt.net http://www.cse.chalmers.se/~feldt
[julia-users] find nearest non-zero element in matrix
Hi julia-users, I am looking for a function that takes a two-dimensional matrix and a set of matrix indices as inputs and spits out the nearest non-zero element in taxicab distance over the matrix index. I am asking for your advice because this function needs to be very fast as I have to do this search many times. Any ideas or advice would be greatly appreciated. Thanks!
[julia-users] Read a certain number of lines in readdlm ?
I’m writing a Julia parser to read in a large output from a Fortran program which is essentially a load of concatenated matrices of differing dimensions. It would be really useful to be able to do something along the lines of readdlm(file,nlines=3) to pull in i.e. the 3x3 matrix you know that follows. Currently I’m resorting to things like: celltext=string(readline(f),readline(f),readline(f)) cell=readdlm(IOBuffer(celltext)) And this really doesn’t feel like a very elegant method (not helped as neither readline nor readlines appear to accept ‘number of lines’ as an argument). Am I missing the Julia way to do things here? Or should I start writing @macros to expand to this level of nitty gritty?
[julia-users] Re: find nearest non-zero element in matrix
In other words, is there anything more optimized for this problem than concatenating findn and the distance measure of which I pick the minimum? On Tuesday, August 12, 2014 10:24:47 AM UTC-4, tcs wrote: Hi julia-users, I am looking for a function that takes a two-dimensional matrix and a set of matrix indices as inputs and spits out the nearest non-zero element in taxicab distance over the matrix index. I am asking for your advice because this function needs to be very fast as I have to do this search many times. Any ideas or advice would be greatly appreciated. Thanks!
[julia-users] rt_sandybridge.o error during make
Dear All, I am trying to install Julia into our cluster. I ran git to obtain data, and then ran 'make'. I got the following error: make[4]: *** [strmm_kernel_RT_SANDYBRIDGE.o] Error 1 make[3]: *** [libs] Error 1 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.10/libopenblas.so] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 To fix, I ran 'make OPENBLAS_USE_THREAD=0 OPENBLAS_TARGET_ARCH=NEHALEM', but I obtained the same error again. How can I fix this? Cheers.
[julia-users] [FIRST POST] JULIA_HOME and julia in terminal
Hey everyone, I am new to julia. Will be using it for my machine learning projects. I have downloaded 'julia-0.2.1-osx10.7+.dmg' for mac. I can use commands on terminal. For running julia files on terminal I will need 'julia filename.jl' command (read in previous posts). However, I get an error saying julia command not found. I need to put $JULIA_HOME in $PATH but I am not not able to figure out the value that has to used for $JULIA_HOME as $JULIA_HOME is not yet defined by me. Cheers, Anuj
Re: [julia-users] [FIRST POST] JULIA_HOME and julia in terminal
Hi Anuj, In general, there’s no need to worry about environment variables like JULIA_HOME, since that’s not essential to how setting PATH works. The important thing to do is to find the path of the Julia executable and to add that path to your value of the environment variable called PATH. On a Mac, that path should look something like /Applications/Julia-*.app/Contents/Resources/julia/bin/julia Hope that helps. — John On Aug 12, 2014, at 8:02 AM, Anuj Prakash anuj1...@gmail.com wrote: Hey everyone, I am new to julia. Will be using it for my machine learning projects. I have downloaded 'julia-0.2.1-osx10.7+.dmg' for mac. I can use commands on terminal. For running julia files on terminal I will need 'julia filename.jl' command (read in previous posts). However, I get an error saying julia command not found. I need to put $JULIA_HOME in $PATH but I am not not able to figure out the value that has to used for $JULIA_HOME as $JULIA_HOME is not yet defined by me. Cheers, Anuj
Re: [julia-users] Re: find nearest non-zero element in matrix
If it needs to be fast, I would just write the logic out with for loops. On Tue, Aug 12, 2014 at 10:59 AM, tcs tobiass...@gmail.com wrote: In other words, is there anything more optimized for this problem than concatenating findn and the distance measure of which I pick the minimum? On Tuesday, August 12, 2014 10:24:47 AM UTC-4, tcs wrote: Hi julia-users, I am looking for a function that takes a two-dimensional matrix and a set of matrix indices as inputs and spits out the nearest non-zero element in taxicab distance over the matrix index. I am asking for your advice because this function needs to be very fast as I have to do this search many times. Any ideas or advice would be greatly appreciated. Thanks!
Re: [julia-users] Re: erroneous cholfact behavior in conjunction with addprocs()
I just did a clean reinstall (brew rm julia openblas64-julia arpack64-julia suite-sparse64-julia; brew install --HEAD --64bit julia) and it seems to be working now. Sorry for the false alarm. -thom On Tuesday, August 12, 2014 9:02:02 AM UTC-5, Andreas Noack wrote: I don't see this with my own build of OpenBLAS. @staticfloat, how old are the bottles? Med venlig hilsen Andreas Noack 2014-08-12 0:15 GMT-04:00 Thomas Covert thom@gmail.com javascript: : I think this is broken again: I'm observing single-threaded failures (i.e., the last line of the sample code fails when the addprocs(8) line is not commented out) on both a somewhat recent OS X home-brew build and also the current --HEAD home-brew build right now. I am using the standard 64-bit openblas, arpack, and suite-sparse from @staticfloat's home-brew setup (i.e., compiled without the --HEAD option). If it helps, here is my versioninfo(): *julia **versioninfo()* Julia Version 0.4.0-prerelease+52 Commit daa3f82* (2014-08-12 03:33 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.3.0) CPU: Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 -thom On Friday, July 18, 2014 4:34:06 PM UTC-5, Viral Shah wrote: This is now fixed now on master by using a new openblas release that fixes the bug. You can get the fix in tomorrow's nightly. Thanks for reporting! -viral On Thursday, July 10, 2014 4:20:07 AM UTC+5:30, Thomas Covert wrote: I have found cholfact to behave differently (erroneously?) under parallel processing contexts than under standard settings. What I mean by parallel processing is simply having previously called addprocs(). Here is some example code that I am running on my mid-2009 MacBook Pro using a somewhat recent brew of @staticfloat's homebrew distribution: addprocs(8) N = 1000 x = 10 * randn(N) X = zeros(N,N) for i = 1:N for j = 1:N X[i,j] = exp(-.5 * (x[i]-x[j])^2) end end X = X + diagm(.5 * ones(N)) C = cholfact(X) iC = inv(C) CiC = cholfact(iC) I believe this code generates an X which is positive definite by construction. If I run this code as-is, I get the following error (or something similar, the PosDefException sometimes changes): *ERROR: PosDefException(12)* * in cholfact! at linalg/factorization.jl:36* * in cholfact at linalg/factorization.jl:39* *while loading /Users/tcovert/path_to_code.jl, in expression starting on line 16* However, if I comment out the addprocs(8) line, everything works fine. Also, for smaller values of N the problem goes away (N=100,200 is fine, N=400 is not). Here is my versioninfo() if that helps: *julia **versioninfo()* Julia Version 0.3.0-prerelease+3868 Commit e7a9a7d* (2014-06-24 19:39 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.2.0) CPU: Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm
Re: [julia-users] Re: erroneous cholfact behavior in conjunction with addprocs()
No problem. Good to hear that it works. Med venlig hilsen Andreas Noack 2014-08-12 11:27 GMT-04:00 Thomas Covert thom.cov...@gmail.com: I just did a clean reinstall (brew rm julia openblas64-julia arpack64-julia suite-sparse64-julia; brew install --HEAD --64bit julia) and it seems to be working now. Sorry for the false alarm. -thom On Tuesday, August 12, 2014 9:02:02 AM UTC-5, Andreas Noack wrote: I don't see this with my own build of OpenBLAS. @staticfloat, how old are the bottles? Med venlig hilsen Andreas Noack 2014-08-12 0:15 GMT-04:00 Thomas Covert thom@gmail.com: I think this is broken again: I'm observing single-threaded failures (i.e., the last line of the sample code fails when the addprocs(8) line is not commented out) on both a somewhat recent OS X home-brew build and also the current --HEAD home-brew build right now. I am using the standard 64-bit openblas, arpack, and suite-sparse from @staticfloat's home-brew setup (i.e., compiled without the --HEAD option). If it helps, here is my versioninfo(): *julia **versioninfo()* Julia Version 0.4.0-prerelease+52 Commit daa3f82* (2014-08-12 03:33 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.3.0) CPU: Intel(R) Core(TM) i7-4578U CPU @ 3.00GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm LLVM: libLLVM-3.3 -thom On Friday, July 18, 2014 4:34:06 PM UTC-5, Viral Shah wrote: This is now fixed now on master by using a new openblas release that fixes the bug. You can get the fix in tomorrow's nightly. Thanks for reporting! -viral On Thursday, July 10, 2014 4:20:07 AM UTC+5:30, Thomas Covert wrote: I have found cholfact to behave differently (erroneously?) under parallel processing contexts than under standard settings. What I mean by parallel processing is simply having previously called addprocs(). Here is some example code that I am running on my mid-2009 MacBook Pro using a somewhat recent brew of @staticfloat's homebrew distribution: addprocs(8) N = 1000 x = 10 * randn(N) X = zeros(N,N) for i = 1:N for j = 1:N X[i,j] = exp(-.5 * (x[i]-x[j])^2) end end X = X + diagm(.5 * ones(N)) C = cholfact(X) iC = inv(C) CiC = cholfact(iC) I believe this code generates an X which is positive definite by construction. If I run this code as-is, I get the following error (or something similar, the PosDefException sometimes changes): *ERROR: PosDefException(12)* * in cholfact! at linalg/factorization.jl:36* * in cholfact at linalg/factorization.jl:39* *while loading /Users/tcovert/path_to_code.jl, in expression starting on line 16* However, if I comment out the addprocs(8) line, everything works fine. Also, for smaller values of N the problem goes away (N=100,200 is fine, N=400 is not). Here is my versioninfo() if that helps: *julia **versioninfo()* Julia Version 0.3.0-prerelease+3868 Commit e7a9a7d* (2014-06-24 19:39 UTC) Platform Info: System: Darwin (x86_64-apple-darwin13.2.0) CPU: Intel(R) Core(TM)2 Duo CPU P8700 @ 2.53GHz WORD_SIZE: 64 BLAS: libopenblas (USE64BITINT NO_AFFINITY) LAPACK: libopenblas LIBM: libopenlibm
[julia-users] Re: export within a macro
Thank you Jameson. Dear all ...but I am still stuck. It's like that: the methods I generate by macro in my module all overload Base functions (cos). If I write a new method without using a macro, I don't need to export it explicitely (I imagine, because I am not creating a new _function_). But if my method is macro generate, it does not export: module moo importall Base # need to import Base.cos, Base.sin to add methods to it export Typ # export all that is to be public - note position of export statement at top of module :) type Typ# public, because exported x end cos(a::Typ) = cos(a.x) # add method to base function - this does NOT require any export out of this module or import by the user macro makefoo(OP) return quote $OP(a::Typ)= $OP(a.x) # add method to base function - fails to export. Explicit export statement does not help end end @makefoo(sin) end importall moo println(methods(cos)) # 9 methods :) ... I know how to export functions from a module println(methods(sin)) # 8 methods :( ... But not if I generated them by macro Is that difference intentional? Sorry Stephane, but the code of @deprecated is, after scrutiny, beyond me. I'll have to take your word for now on the potential evils of exporting macro generated function. But, then can you suggest a workaround? Can I export the macro (I failed), and have the macro calls outside the module? I'd like to keep the module - because it contains a bunch of private functions... Philippe
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
Since Julia's libraries are in a non-standard directory, (usually prefix/lib/julia, where prefix is the root of your julia installation) we rely on a dynamic linker feature known as the RPATH to find these libraries at runtime. You can inspect the rpath of the julia executable with otool: $ otool -l julia | grep -A 2 LC_RPATH cmd LC_RPATH cmdsize 48 path @executable_path/../lib/julia (offset 12) -- cmd LC_RPATH cmdsize 40 path @executable_path/../lib (offset 12) This is encoded into the binary calling julia by putting in extra linker commands when compiling julia. Things like -Wl,-rpath,@executable_path/../lib I haven't played around with whether encoding RPATHS into libraries will work, so you either have to try one of the following: * Change the RPATH of the executable you're embedding julia into * Change the RPATH of libjulia itself * Tack things onto DYLD_FALLBACK_LIBRARY_PATH (Try to stay away from DYLD_LIBRARY_PATH, it's better to mess with the FALLBACK variant http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s when you can) * Move the libraries into an easier-to-find place. Either a place that is already on your executable's RPATH, or a location that is searched by default by the linker. The default locations are the default list of DYLD_FALLBACK_LIBRARY_PATH entries, listed in this man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, and are $(HOME)/lib:/usr/local/lib:/lib:/usr/lib. -E On Tue, Aug 12, 2014 at 3:12 AM, Jeff Waller truth...@gmail.com wrote: This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
Re: [julia-users] rt_sandybridge.o error during make
Can you post the full log somewhere? Unfortunately the actual error is a little higher up in the log, and we can't see it here. If the log is too long for email, you can create a gist https://gist.github.com/ or a pastebin http://pastebin.com/ of it and link it here. Also, information about your cluster hardware and operating system would be helpful. Is it virtualized at all? -E On Tue, Aug 12, 2014 at 4:54 AM, Batuhan Kav batuhan...@gmail.com wrote: Dear All, I am trying to install Julia into our cluster. I ran git to obtain data, and then ran 'make'. I got the following error: make[4]: *** [strmm_kernel_RT_SANDYBRIDGE.o] Error 1 make[3]: *** [libs] Error 1 *** Clean the OpenBLAS build with 'make -C deps clean-openblas'. Rebuild with 'make OPENBLAS_USE_THREAD=0 if OpenBLAS had trouble linking libpthread.so, and with 'make OPENBLAS_TARGET_ARCH=NEHALEM' if there were errors building SandyBridge support. Both these options can also be used simultaneously. *** make[2]: *** [openblas-v0.2.10/libopenblas.so] Error 1 make[1]: *** [julia-release] Error 2 make: *** [release] Error 2 To fix, I ran 'make OPENBLAS_USE_THREAD=0 OPENBLAS_TARGET_ARCH=NEHALEM', but I obtained the same error again. How can I fix this? Cheers.
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
Either of those variables an be modified, but in both cases you should be attentive to the side effects. For example, IIRC, if the fallback path is set, the link won't search the usual fallback locations, like /usr/lib (unless you explicitly add them to your fallback path) The primary failure case for usage of either environment variable is the same: having multiple possible libraries that could satisfy a dependency, and ending up with multiple copies of the dylib loaded, with independent global state. On Tuesday, August 12, 2014, Elliot Saba staticfl...@gmail.com wrote: Since Julia's libraries are in a non-standard directory, (usually prefix/lib/julia, where prefix is the root of your julia installation) we rely on a dynamic linker feature known as the RPATH to find these libraries at runtime. You can inspect the rpath of the julia executable with otool: $ otool -l julia | grep -A 2 LC_RPATH cmd LC_RPATH cmdsize 48 path @executable_path/../lib/julia (offset 12) -- cmd LC_RPATH cmdsize 40 path @executable_path/../lib (offset 12) This is encoded into the binary calling julia by putting in extra linker commands when compiling julia. Things like -Wl,-rpath,@executable_path/../lib I haven't played around with whether encoding RPATHS into libraries will work, so you either have to try one of the following: * Change the RPATH of the executable you're embedding julia into * Change the RPATH of libjulia itself * Tack things onto DYLD_FALLBACK_LIBRARY_PATH (Try to stay away from DYLD_LIBRARY_PATH, it's better to mess with the FALLBACK variant http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s when you can) * Move the libraries into an easier-to-find place. Either a place that is already on your executable's RPATH, or a location that is searched by default by the linker. The default locations are the default list of DYLD_FALLBACK_LIBRARY_PATH entries, listed in this man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, and are $(HOME)/lib:/usr/local/lib:/lib:/usr/lib. -E On Tue, Aug 12, 2014 at 3:12 AM, Jeff Waller truth...@gmail.com javascript:_e(%7B%7D,'cvml','truth...@gmail.com'); wrote: This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
Yes, Jameson brings up a really important point; if you do end up mucking around with any of the LIBRARY_PATH variables, they have silent defaults that should be added in. You can read about it in the man page linked above, but modifying the RPATH of the binary is a better choice in virtually every case. -E On Tue, Aug 12, 2014 at 1:14 PM, Jameson Nash vtjn...@gmail.com wrote: Either of those variables an be modified, but in both cases you should be attentive to the side effects. For example, IIRC, if the fallback path is set, the link won't search the usual fallback locations, like /usr/lib (unless you explicitly add them to your fallback path) The primary failure case for usage of either environment variable is the same: having multiple possible libraries that could satisfy a dependency, and ending up with multiple copies of the dylib loaded, with independent global state. On Tuesday, August 12, 2014, Elliot Saba staticfl...@gmail.com wrote: Since Julia's libraries are in a non-standard directory, (usually prefix/lib/julia, where prefix is the root of your julia installation) we rely on a dynamic linker feature known as the RPATH to find these libraries at runtime. You can inspect the rpath of the julia executable with otool: $ otool -l julia | grep -A 2 LC_RPATH cmd LC_RPATH cmdsize 48 path @executable_path/../lib/julia (offset 12) -- cmd LC_RPATH cmdsize 40 path @executable_path/../lib (offset 12) This is encoded into the binary calling julia by putting in extra linker commands when compiling julia. Things like -Wl,-rpath,@executable_path/../lib I haven't played around with whether encoding RPATHS into libraries will work, so you either have to try one of the following: * Change the RPATH of the executable you're embedding julia into * Change the RPATH of libjulia itself * Tack things onto DYLD_FALLBACK_LIBRARY_PATH (Try to stay away from DYLD_LIBRARY_PATH, it's better to mess with the FALLBACK variant http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s when you can) * Move the libraries into an easier-to-find place. Either a place that is already on your executable's RPATH, or a location that is searched by default by the linker. The default locations are the default list of DYLD_FALLBACK_LIBRARY_PATH entries, listed in this man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, and are $(HOME)/lib:/usr/local/lib:/lib:/usr/lib. -E On Tue, Aug 12, 2014 at 3:12 AM, Jeff Waller truth...@gmail.com wrote: This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
If you don't have control over the compilation process of the binary, you can use `install_name_tool` to change the rpath of the binary after it's been compiled, but that is dependent on how long the new path is and how much slack space the compiler added to the dynamic library header beforehand.
[julia-users] Re: character encoding
On Tuesday, August 12, 2014 2:14:00 PM UTC-3, Steven G. Johnson wrote: On Tuesday, August 12, 2014 9:47:31 AM UTC-4, Frederico Novaes wrote: I'm using ODBC to query a DB. As part of the returned query, I get objects of type UTF8String. The problem is that characters like ã are printed as \u8f. How can I properly handle character encoding ? Probably that means that your query is not actually returning UTF-8 encoded data. You need to figure out what encoding your data is actually using. I'm not sure yet, it may be Cp1252. If this is the case, how can I set that this is the encoding to be used ? Thanks.
[julia-users] Re: export within a macro
Thank you again, Jacob! I pondered that under the shower, and it did the trick. For record, if somebody wants to generate a function from a macro in a function, here is a little example. Case closed. Philippe module moo importall Base # need to import Base.cos, Base.sin to add methods to it export Typ # export all that is to be public type Typ# public, because exported x end cos(a::Typ) = cos(a.x) # add method to base function - this does NOT require any export out of this module or import by the user macro makefoo(OP) return quote $(esc(OP))(a::Typ)= $OP(a.x) # add method to base function. Note the $(esc(OP)) to prevent macro hygiene from changing the name of generated function end end println(macroexpand(:(@makefoo(sin @makefoo(sin) end importall moo println(methods(cos)) println(methods(sin))
Re: [julia-users] Re: character encoding
You'd have to implement that string encoding, unfortunately. If it's Latin-1, it would be much easier and we could probably just resurrect an old implementation that we used to have. On Tue, Aug 12, 2014 at 1:50 PM, Frederico Novaes frederico.nov...@gmail.com wrote: On Tuesday, August 12, 2014 2:14:00 PM UTC-3, Steven G. Johnson wrote: On Tuesday, August 12, 2014 9:47:31 AM UTC-4, Frederico Novaes wrote: I'm using ODBC to query a DB. As part of the returned query, I get objects of type UTF8String. The problem is that characters like ã are printed as \u8f. How can I properly handle character encoding ? Probably that means that your query is not actually returning UTF-8 encoded data. You need to figure out what encoding your data is actually using. I'm not sure yet, it may be Cp1252. If this is the case, how can I set that this is the encoding to be used ? Thanks.
Re: [julia-users] Re: export within a macro
I think you probably should escape both instances of $OP not just the first one. On Tue, Aug 12, 2014 at 1:55 PM, Philippe Maincon philippe.main...@gmail.com wrote: Thank you again, Jacob! I pondered that under the shower, and it did the trick. For record, if somebody wants to generate a function from a macro in a function, here is a little example. Case closed. Philippe module moo importall Base # need to import Base.cos, Base.sin to add methods to it export Typ # export all that is to be public type Typ# public, because exported x end cos(a::Typ) = cos(a.x) # add method to base function - this does NOT require any export out of this module or import by the user macro makefoo(OP) return quote $(esc(OP))(a::Typ)= $OP(a.x) # add method to base function. Note the $(esc(OP)) to prevent macro hygiene from changing the name of generated function end end println(macroexpand(:(@makefoo(sin @makefoo(sin) end importall moo println(methods(cos)) println(methods(sin))
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
On Tuesday, August 12, 2014 12:40:02 PM UTC-4, Elliot Saba wrote: Since Julia's libraries are in a non-standard directory, (usually prefix/lib/julia, where prefix is the root of your julia installation) we rely on a dynamic linker feature known as the RPATH to find these libraries at runtime. You can inspect the rpath of the julia executable with otool: $ otool -l julia | grep -A 2 LC_RPATH cmd LC_RPATH cmdsize 48 path @executable_path/../lib/julia (offset 12) -- cmd LC_RPATH cmdsize 40 path @executable_path/../lib (offset 12) Nice, this is a useful. I knew about rpath and am using it, but not the option and the grep to pull it out of compiled binary for verification. I've verified that rpath is part of the object code created and is the right value (or at least corresponds to a value of DYLD_LIBRARY_PATH) that makes things work. This is encoded into the binary calling julia by putting in extra linker commands when compiling julia. Things like -Wl,-rpath,@executable_path/../lib I haven't played around with whether encoding RPATHS into libraries will work, so you either have to try one of the following: * Change the RPATH of the executable you're embedding julia into I think this is the critical problem. I did try that, BUT. I'm not creating an executable but another dynamically loaded library -- a module -- which is loaded by another system at runtime. I have no control of that other system, only the module. * Change the RPATH of libjulia itself I have no control over that; this is for others. This has to work with any deployment, but I am able to control how my thing is compiled. I can insert any flags, constants, or run any sort configuration gathering script I want as a pre-compilation step (within reason of course). * Tack things onto DYLD_FALLBACK_LIBRARY_PATH (Try to stay away from DYLD_LIBRARY_PATH, it's better to mess with the FALLBACK variant http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s when you can) Well this looks like what I will have to do for the time being at least. I agree this is the least desirable. * Move the libraries into an easier-to-find place. Either a place that is already on your executable's RPATH, or a location that is searched by default by the linker. The default locations are the default list of DYLD_FALLBACK_LIBRARY_PATH entries, listed in this man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, and are $(HOME)/lib:/usr/local/lib:/lib:/usr/lib. -E Likewise as above, I have no control over that. On Tue, Aug 12, 2014 at 3:12 AM, Jeff Waller trut...@gmail.com javascript: wrote: This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
[julia-users] Re: Read a certain number of lines in readdlm ?
No need for macros!
Its an interesting feature request, maybe open a Github issue so people can
discuss it.
I think your solution is not terrible, you could generalize it to
readcell(f, nlines) = readdlm(IOBuffer(string([readline(f) for i in
1:nlines])))
Then do something like
f = open(mydata,r)
cells = {}
while !eof(f)
push!(cells, readcell(f, 3))
end
close(f)
On Tuesday, August 12, 2014 10:40:24 AM UTC-4, Jarvist Moore Frost wrote:
I’m writing a Julia parser to read in a large output from a Fortran
program which is essentially a load of concatenated matrices of differing
dimensions. It would be really useful to be able to do something along the
lines of readdlm(file,nlines=3) to pull in i.e. the 3x3 matrix you know
that follows.
Currently I’m resorting to things like:
celltext=string(readline(f),readline(f),readline(f))
cell=readdlm(IOBuffer(celltext))
And this really doesn’t feel like a very elegant method (not helped as
neither readline nor readlines appear to accept ‘number of lines’ as an
argument).
Am I missing the Julia way to do things here? Or should I start writing
@macros to expand to this level of nitty gritty?
Re: [julia-users] export within a macro
That's one of the really fiddly parts of macro hygiene -- the two uses of esc have a different meaning and apply to different contexts. In the first, it allows the function definition to escape into the surrounding environment. In the second, it changes the lookup scope of the global variable from the callee's module to the callers module or function. Is the 0.4 tag already applied to the PR that changes this esc stuff to tagged symbols? On Tuesday, August 12, 2014, Stefan Karpinski ste...@karpinski.org wrote: I think you probably should escape both instances of $OP not just the first one. On Tue, Aug 12, 2014 at 1:55 PM, Philippe Maincon philippe.main...@gmail.com javascript:_e(%7B%7D,'cvml','philippe.main...@gmail.com'); wrote: Thank you again, Jacob! I pondered that under the shower, and it did the trick. For record, if somebody wants to generate a function from a macro in a function, here is a little example. Case closed. Philippe module moo importall Base # need to import Base.cos, Base.sin to add methods to it export Typ # export all that is to be public type Typ# public, because exported x end cos(a::Typ) = cos(a.x) # add method to base function - this does NOT require any export out of this module or import by the user macro makefoo(OP) return quote $(esc(OP))(a::Typ)= $OP(a.x) # add method to base function. Note the $(esc(OP)) to prevent macro hygiene from changing the name of generated function end end println(macroexpand(:(@makefoo(sin @makefoo(sin) end importall moo println(methods(cos)) println(methods(sin))
Re: [julia-users] export within a macro
Oh, no, that's a good one to include. On Tue, Aug 12, 2014 at 2:26 PM, Jameson Nash vtjn...@gmail.com wrote: That's one of the really fiddly parts of macro hygiene -- the two uses of esc have a different meaning and apply to different contexts. In the first, it allows the function definition to escape into the surrounding environment. In the second, it changes the lookup scope of the global variable from the callee's module to the callers module or function. Is the 0.4 tag already applied to the PR that changes this esc stuff to tagged symbols? On Tuesday, August 12, 2014, Stefan Karpinski ste...@karpinski.org wrote: I think you probably should escape both instances of $OP not just the first one. On Tue, Aug 12, 2014 at 1:55 PM, Philippe Maincon philippe.main...@gmail.com wrote: Thank you again, Jacob! I pondered that under the shower, and it did the trick. For record, if somebody wants to generate a function from a macro in a function, here is a little example. Case closed. Philippe module moo importall Base # need to import Base.cos, Base.sin to add methods to it export Typ # export all that is to be public type Typ# public, because exported x end cos(a::Typ) = cos(a.x) # add method to base function - this does NOT require any export out of this module or import by the user macro makefoo(OP) return quote $(esc(OP))(a::Typ)= $OP(a.x) # add method to base function. Note the $(esc(OP)) to prevent macro hygiene from changing the name of generated function end end println(macroexpand(:(@makefoo(sin @makefoo(sin) end importall moo println(methods(cos)) println(methods(sin))
Re: [julia-users] Read a certain number of lines in readdlm ?
As a slight optimization, you could note that string works by creating an
IOBuffer and printing the arguments into it, and then converting the result
to a string. Thus, you could skip the extra conversion to a string and back
by making the IOBuffer directly.
On Tuesday, August 12, 2014, Iain Dunning iaindunn...@gmail.com wrote:
No need for macros!
Its an interesting feature request, maybe open a Github issue so people
can discuss it.
I think your solution is not terrible, you could generalize it to
readcell(f, nlines) = readdlm(IOBuffer(string([readline(f) for i in
1:nlines])))
Then do something like
f = open(mydata,r)
cells = {}
while !eof(f)
push!(cells, readcell(f, 3))
end
close(f)
On Tuesday, August 12, 2014 10:40:24 AM UTC-4, Jarvist Moore Frost wrote:
I’m writing a Julia parser to read in a large output from a Fortran
program which is essentially a load of concatenated matrices of differing
dimensions. It would be really useful to be able to do something along the
lines of readdlm(file,nlines=3) to pull in i.e. the 3x3 matrix you know
that follows.
Currently I’m resorting to things like:
celltext=string(readline(f),readline(f),readline(f))
cell=readdlm(IOBuffer(celltext))
And this really doesn’t feel like a very elegant method (not helped as
neither readline nor readlines appear to accept ‘number of lines’ as an
argument).
Am I missing the Julia way to do things here? Or should I start writing
@macros to expand to this level of nitty gritty?
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
Alright, I worked up a little proof-of-concept demo here https://github.com/staticfloat/dltest/tree/master, but the long and the short of it is that you should be able to alter the rpath of YOUR library to include whatever paths you need, (probably relative to @loader_path so that your library just needs to know the relative path to openblas and friends) and you'll then be able to dlopen these guys to your heart's content. If you have any questions about this, please feel free to ask, but it should be as simple as adding a -Wl,-rpath,@loader_path/../lib or whatever to the linker options for your library. -E On Tue, Aug 12, 2014 at 2:02 PM, Jeff Waller truth...@gmail.com wrote: On Tuesday, August 12, 2014 12:40:02 PM UTC-4, Elliot Saba wrote: Since Julia's libraries are in a non-standard directory, (usually prefix/lib/julia, where prefix is the root of your julia installation) we rely on a dynamic linker feature known as the RPATH to find these libraries at runtime. You can inspect the rpath of the julia executable with otool: $ otool -l julia | grep -A 2 LC_RPATH cmd LC_RPATH cmdsize 48 path @executable_path/../lib/julia (offset 12) -- cmd LC_RPATH cmdsize 40 path @executable_path/../lib (offset 12) Nice, this is a useful. I knew about rpath and am using it, but not the option and the grep to pull it out of compiled binary for verification. I've verified that rpath is part of the object code created and is the right value (or at least corresponds to a value of DYLD_LIBRARY_PATH) that makes things work. This is encoded into the binary calling julia by putting in extra linker commands when compiling julia. Things like -Wl,-rpath,@executable_path/../lib I haven't played around with whether encoding RPATHS into libraries will work, so you either have to try one of the following: * Change the RPATH of the executable you're embedding julia into I think this is the critical problem. I did try that, BUT. I'm not creating an executable but another dynamically loaded library -- a module -- which is loaded by another system at runtime. I have no control of that other system, only the module. * Change the RPATH of libjulia itself I have no control over that; this is for others. This has to work with any deployment, but I am able to control how my thing is compiled. I can insert any flags, constants, or run any sort configuration gathering script I want as a pre-compilation step (within reason of course). * Tack things onto DYLD_FALLBACK_LIBRARY_PATH (Try to stay away from DYLD_LIBRARY_PATH, it's better to mess with the FALLBACK variant http://stackoverflow.com/questions/3146274/is-it-ok-to-use-dyld-library-path-on-mac-os-x-and-whats-the-dynamic-library-s when you can) Well this looks like what I will have to do for the time being at least. I agree this is the least desirable. * Move the libraries into an easier-to-find place. Either a place that is already on your executable's RPATH, or a location that is searched by default by the linker. The default locations are the default list of DYLD_FALLBACK_LIBRARY_PATH entries, listed in this man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, and are $(HOME)/lib:/usr/local/lib:/lib:/usr/lib. -E Likewise as above, I have no control over that. On Tue, Aug 12, 2014 at 3:12 AM, Jeff Waller trut...@gmail.com wrote: This is on OS/X I do have a workaround, but I'd like to get to the bottom of it. I don't see anywhere in the code where either of these 2 libraries are loaded explicitly --- maybe it's in loading sys.ji? My workaround can be to set an env variable, but it would be better to understand and address the problem directly. If DYLD_LIBRARY_PATH is set to point to the Julia library directory, then all works, but if this environment variable is not set, then the following occurs Warning: error initializing module LinAlg: ErrorException(error compiling __init__: error compiling check_blas: error compiling blas_vendor: could not load module libopenblas: dlopen(libopenblas.dylib, 1): image not found) Entropy pool not available to seed RNG; using ad-hoc entropy sources. Warning: error initializing module Random: ErrorException(could not load module libdSFMT: dlopen(libdSFMT.dylib, 1): image not found) === Those additional errors I'm sure are just side effects cascading form failing to load those libraries. The engine still does function, but the linear algebra stuff is unavailable. I've tried various combinations of jl_init and jl_init_with_image to no effect. -Jeff
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
On Tuesday, August 12, 2014 2:36:03 PM UTC-4, Elliot Saba wrote: Alright, I worked up a little proof-of-concept demo here https://github.com/staticfloat/dltest/tree/master, but the long and the short of it is that you should be able to alter the rpath of YOUR library to include whatever paths you need, (probably relative to @loader_path so that your library just needs to know the relative path to openblas and friends) and you'll then be able to dlopen these guys to your heart's content. If you have any questions about this, please feel free to ask, but it should be as simple as adding a -Wl,-rpath,@loader_path/../lib or whatever to the linker options for your library. -E Hmm ok. What needs to dlopen openblas, etc? dlopen returns a value and needs to be saved does it not? If dlopen occurs on a library that is already loaded, it doesn't error, but does it succeed in the way that Julia expects it to? Or are you saying that openblas, etc just needs to be loaded somehow and then it all works out, my library can invoke dlopen brute-force like with maybe an arbitrary location. on these things? To be honest, I don't quite understand why this is even happening except for possibly side effects of resolving symbols at link time. After all libopenblas and libdSFMT are in the same exact directory as libjulia, and libjulia does get loaded no problem. Hmm @loader_path is a value eh? I don't see that documented. -Jeff
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
I wrote up a huge long response talking about dynamic loading, why you need to alter libjulia, etc... and then about a page and a half in I realized that there was a very simple change on our end that could fix this. So I pushed this commit https://github.com/JuliaLang/julia/commit/1a18e8782f39dabfb68e75892cd29f27a565baae. Any julia version newer than today should automatically have `@loader_path/` added to libjulia's RPATH, which means that you should be able to have everything just work when embedding. If you clone and compile your own julia to test this out, pulling the latest and rebuilding should fix this for you. If you live off of binaries, a new nightly should be up tonight with this fix included. @loader_path is documented in the dyld man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, which is hardly light reading, I know. -E On Tue, Aug 12, 2014 at 5:01 PM, Jeff Waller truth...@gmail.com wrote: On Tuesday, August 12, 2014 2:36:03 PM UTC-4, Elliot Saba wrote: Alright, I worked up a little proof-of-concept demo here https://github.com/staticfloat/dltest/tree/master, but the long and the short of it is that you should be able to alter the rpath of YOUR library to include whatever paths you need, (probably relative to @loader_path so that your library just needs to know the relative path to openblas and friends) and you'll then be able to dlopen these guys to your heart's content. If you have any questions about this, please feel free to ask, but it should be as simple as adding a -Wl,-rpath,@loader_path/../lib or whatever to the linker options for your library. -E Hmm ok. What needs to dlopen openblas, etc? dlopen returns a value and needs to be saved does it not? If dlopen occurs on a library that is already loaded, it doesn't error, but does it succeed in the way that Julia expects it to? Or are you saying that openblas, etc just needs to be loaded somehow and then it all works out, my library can invoke dlopen brute-force like with maybe an arbitrary location. on these things? To be honest, I don't quite understand why this is even happening except for possibly side effects of resolving symbols at link time. After all libopenblas and libdSFMT are in the same exact directory as libjulia, and libjulia does get loaded no problem. Hmm @loader_path is a value eh? I don't see that documented. -Jeff
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
Be sure to let me know if it doesn't work! On Tue, Aug 12, 2014 at 6:12 PM, Jeff Waller truth...@gmail.com wrote: On Tuesday, August 12, 2014 5:54:47 PM UTC-4, Elliot Saba wrote: I wrote up a huge long response talking about dynamic loading, why you need to alter libjulia, etc... and then about a page and a half in I realized that there was a very simple change on our end that could fix this. So I pushed this commit https://github.com/JuliaLang/julia/commit/1a18e8782f39dabfb68e75892cd29f27a565baae. Any julia version newer than today should automatically have `@loader_path/` added to libjulia's RPATH, which means that you should be able to have everything just work when embedding. ... ! That's awesome! If you clone and compile your own julia to test this out, pulling the latest and rebuilding should fix this for you. If you live off of binaries, a new nightly should be up tonight with this fix included. No problem, I have that already. @loader_path is documented in the dyld man page https://developer.apple.com/library/mac/documentation/Darwin/Reference/ManPages/man1/dyld.1.html, which is hardly light reading, I know. -E Well that explains that. I was man-ing ld, not dyld
[julia-users] ANN: YT.jl, a wrapper for the Python-based yt astrophysical analysis package
Hello Julia-users, I wanted to take the opportunity to inform the community of a new package I've written, YT.jl. yt (http://yt-project.org) is a Python package for analyzing and visualizing volumetric, multi-resolution data from astrophysical simulations, radio telescopes, and a burgeoning interdisciplinary community. yt is more than a visualization package: it is a tool to seamlessly handle simulation output files to make analysis simple. yt aims to provide a simple uniform way of handling volumetric data, regardless of where it is generated. yt currently supports such astrophysical data formats as FLASH, Enzo, Boxlib, Athena, arbitrary volumes, and support is in progress for Gadget, Tipsy, ART, RAMSES and MOAB. The YT.jl package seeks to expose a number of yt's most important features from within the Julia environment. These include: * Volumetric data containers: spheres, rectangular regions, slices, projections, profiles, etc. * YTArrays and YTQuantities, which are arrays and real numbers associated with symbolic units, and associated mathematical operations on them * Simple visualization tools, such as generating fixed resolution images from non-fixed resolution data, and Matplotlib-based slice and projection plots To find out more about the YT.jl package, visit the documentation: http://www.jzuhone.com/yt_julia or the GitHub project page: http://github.com/jzuhone/YT.jl Thanks for reading, and sorry for the spam. Best, John ZuHone
[julia-users] GUI in Julia
Are there any GUI packages for julia available yet or is it too early to ask?
Re: [julia-users] GUI in Julia
On 13 August 2014 09:24, ccsv.1056...@gmail.com wrote: Are there any GUI packages for julia available yet or is it too early to ask? If you head over to the available packages list [1], there are several. While I have not experimented with them myself, Gtk.jl [2] and Tk.jl [3] seems to be the most mature ones. Pontus [1]: http://julia.readthedocs.org/en/latest/packages/packagelist/ [2]: https://github.com/JuliaLang/Gtk.jl [3]: https://github.com/JuliaLang/Tk.jl
Re: [julia-users] Unable to dlopen openblas and dSFMT by default when using embedded Julia
In fact this does indeed just work. I did need to specify rpath, but didn't need to do anything else, this one value of rpath during the link step was enough to resolve both libjulia and libopenblas, etc. e.g. -Wl,-rpath,/usr/local/julia/lib/julia, Also. *Version 0.4.0-dev+92 (2014-08-12 22:52 UTC) --- congratz!*
[julia-users] Re: character encoding
This might be easier to do on the server side.
Many DBMS, including MySQL and PostGreSQL, support encoding conversions.
eg convert('my_string', 'UTF8', 'ISO_8859_2')
SELECT CAST(_latin1'test' AS CHAR CHARACTER SET utf8)
