Re: [julia-users] string processing after readdlm
El lunes, 13 de abril de 2015, 5:56:05 (UTC-5), JKPie escribió: ok, here is the example: a = Array{Float32}[] push!(a,[1 2 3]) push!(a,[4 5 4]) push!(a,[7 8 9]) Try `a = [vcat(a...)]` (I always forget this syntax... Maybe we need a convenience function for this, e.g. `make_matrix`?) This converts `a` into a more useful form: julia a 3x3 Array{Float32,2}: 1.0 2.0 3.0 4.0 5.0 4.0 7.0 8.0 9.0 which can now be `writedlm`ed and `readdlm`ed easily. David. println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) the result is: Array{Array{Float32,N},1} Array{Any,2} Any[Float32[1.0 2.0 3.0]] and the same behawior here: a = {} push!(a,rand(5)) push!(a,rand(3)) push!(a,rand(7)) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) println(typeof(b[1,1])) and the result: - - - - - - - cljs Array{Any,1} Array{Any,2} Any[[0.054008985630280115,0.8947273976690304,0.14961853234717193,0.72895523733,0.8907801902141823]] SubString{ASCIIString} in both cases I would like to have a method to write the data and next to read it and have it with the same structure and types (or just a function to process this substrings). J On Sunday, April 12, 2015 at 7:55:30 PM UTC+2, Mauro wrote: An example which can be copy-pasted would be helpful. M On Sun, 2015-04-12 at 16:47, JKpie fable...@gmail.com wrote: Hi, I am new Julia coder (I started to learn Julia two weeks ago). I have a problem with writedlm and readdlm functions. I have an array of vectors: Vector Array{Float32,N},4 When I try to save it using writedlm to txt file and next read that file using readdlm I obtain a matrix: Matrix Any, 4 which contains four substrings like: Float32[1,2,3... so my question is: is there any simple way to convert that strings into arrays of floats? I also need to write and read files of Any type arrays which contains float vectors of different lengths. Thank you very much for your help, J
[julia-users] Puzzling behavior with multiple processes and STDOUT usage
Hi, My goal is to have 10 parallel processes read the same file and each process consume 1/10th of that file. They of course all read all lines of the file, but skips over lines not belonging to it. So process #1 would process lines 1, 11, 21 etc. Second one would process lines 2, 12, 22 etc. The issue that I have has nothing to do with efficiency or performance of this, so let us forget the efficiency part of it, for now. The code is checked in to a repository here *https://github.com/harikb/scratchpad1 *(including some sample data), but also quoted in the email at the end. $ julia --version julia version 0.3.7 *# Input - some sample from nyc public database (see repo link above, but any file might be enough)* $ wc -l nyc311calls.csv 25 nyc311calls.csv *# Ignore why I am not using a csv reader. this is just test data. there are no multi-line quoted csv data here.* $ julia -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 250001 *# Non-parallel run, everything is fine. One extra line is the initial print statement from _driver.jl* *# Now, let us run with 10 parallel processes* $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 26420 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 40915 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -c 1919321 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -c 2172839 *Output seems all over the place. I think the processes stop after reaching certain input.* $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | tail From worker 8: Process 8 is processing line 46617 From worker 5: Process 5 is processing line 46614 From worker 2: Process 2 is processing line 50751 From worker 4: Process 4 is processing line 45593 From worker 11: Process 11 is processing line 45380 From worker 6: Process 6 is processing line 46685 From worker 7: Process 7 is processing line 50756 From worker 9: Process 9 is processing line 46688 From worker 10: Process 10 is processing line 46699 From worker 3: Process 3 is processing line 46692 Now, I could buy that the STDOUT is getting clobbered by multiple parallel writes to it. I am used to STDOUT getting garbled/mixed data from other environments/languages, but I haven't seen missing data. The characters eventually make it in some form to the output. But if I redirect the output to a file, it is perfectly fine every single time. Why is that STDOUT does not get clobbered in that case? $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx $ wc -l xx 250001 xx $ wc -c xx 12988916 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx *== Code below same as the quoted github link ==* $ cat ./julia_test_parallel.jl #!/usr/local/julia-cb9bcae93a/bin/julia function processOneFile(filename) np = nprocs() jump = np - 1 jump = jump == 0 ? 1 : jump selfid = myid() # in a single-process setup, this function will be called for parent (id=1) assert(jump == 1 || selfid != 1) f = open(filename); offset = np == 1 ? selfid : selfid - 1 lnum = 0 for l in eachline(f) lnum += 1 if lnum == offset println(Process $(selfid) is processing line $(lnum)) offset += jump end end end $ cat ./julia_test_parallel_driver.jl #!/usr/local/julia-cb9bcae93a/bin/julia filename = nyc311calls.csv np = nprocs() println(Started $(np) processes) if (np 1) if (myid() == 1) # Mulitprocess and I am the parent @sync begin for i = 2:nprocs() @async remotecall_wait(i, processOneFile, filename) end end end else processOneFile(filename) end *Any help is appreciated.* Thanks -- Harry
[julia-users] Speeding up indexing into a Dict{Vector,Vector}
I'm hoping I can get some advice on optimizing some code. I have a data structure which is very conveniently represented as a Dict{Vector{Int},Vector{Float64}}. That is, I need to look up floating-point vectors stored at certain multidimensional (usually 2-10) positions represented as a Vector{Int}. Before I construct this Dict, I don't have a priori knowledge of what position vectors will be represented. I'm aware of the problems of using mutable objects as Dict keys, but let's assume that's not really an issue. Profiling reveals that the real time sink is testing arrays for equality when indexing into this data structure, and indexing happens very frequently. What would be a good way to speed this process up? Is there a way to represent the keys which will be cheaper to compare? I could convert the keys to tuples, but it's helpful to have them as mutable arrays to do some calculations with them at certain points. Is there a different data structure which might work better? Should I roll my own hash table implementation which is particularly optimized for vector keys?
[julia-users] Re: smallest eigenvalue of sparse matrix
When the matrix is real and symmetric, ARPACK does resort to Lanczos, or at least the implicitly restarted version thereof. Straight from the homepage: When the matrix A is symmetric it reduces to a variant of the Lanczos process called the Implicitly Restarted Lanczos Method (IRLM). I always wondered why the ARPACK creators didn't bother to use the same for complex Hermitian problems. Even for that case, the Lanczos / Arnoldi orthogonalization will automatically yield a reduced matrix which is real and tridiagonal (instead of Heisenberg in the generic case) so I would think there is some benefit. On a different note, typically you cannot easily find smallest eigenvalues of a matrix with a Krylov method if you just multiply with A, if smallest is supposed to mean closest to zero in absolute value (smallest magnitude). You can only find extremal eigenvalues, which are on the outer regions of the spectrum. For the smallest magnitude eigenvalues, in the generic case, one has to use a Krylov subspace built using the inverse of A. Since your matrix is Hermitian, you know the spectrum will be real, but still it might have many positive and negative eigenvalues and so the same comments still apply if you are looking for the eigenvalues with smallest magnitude. If by smallest you mean, most negative, then there is no problem (you should use :SR in eigs). If you know more, e.g. that your matrix is not only Hermitian but also positive definite, then all eigenvalues are positive and smallest magnitude also means smallest real.
[julia-users] Re: Puzzling behavior with multiple processes and STDOUT usage
I have confirmed the same behavior with the latest nightly as well $ julia --version julia version 0.4.0-dev+4159 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 38968 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 53050 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 250001 Thanks -- Harry On Monday, April 13, 2015 at 12:28:17 PM UTC-7, Harry B wrote: Hi, My goal is to have 10 parallel processes read the same file and each process consume 1/10th of that file. They of course all read all lines of the file, but skips over lines not belonging to it. So process #1 would process lines 1, 11, 21 etc. Second one would process lines 2, 12, 22 etc. The issue that I have has nothing to do with efficiency or performance of this, so let us forget the efficiency part of it, for now. The code is checked in to a repository here *https://github.com/harikb/scratchpad1 https://github.com/harikb/scratchpad1 *(including some sample data), but also quoted in the email at the end. $ julia --version julia version 0.3.7 *# Input - some sample from nyc public database (see repo link above, but any file might be enough)* $ wc -l nyc311calls.csv 25 nyc311calls.csv *# Ignore why I am not using a csv reader. this is just test data. there are no multi-line quoted csv data here.* $ julia -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 250001 *# Non-parallel run, everything is fine. One extra line is the initial print statement from _driver.jl* *# Now, let us run with 10 parallel processes* $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 26420 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -l 40915 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -c 1919321 $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | wc -c 2172839 *Output seems all over the place. I think the processes stop after reaching certain input.* $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl | tail From worker 8: Process 8 is processing line 46617 From worker 5: Process 5 is processing line 46614 From worker 2: Process 2 is processing line 50751 From worker 4: Process 4 is processing line 45593 From worker 11: Process 11 is processing line 45380 From worker 6: Process 6 is processing line 46685 From worker 7: Process 7 is processing line 50756 From worker 9: Process 9 is processing line 46688 From worker 10: Process 10 is processing line 46699 From worker 3: Process 3 is processing line 46692 Now, I could buy that the STDOUT is getting clobbered by multiple parallel writes to it. I am used to STDOUT getting garbled/mixed data from other environments/languages, but I haven't seen missing data. The characters eventually make it in some form to the output. But if I redirect the output to a file, it is perfectly fine every single time. Why is that STDOUT does not get clobbered in that case? $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx $ wc -l xx 250001 xx $ wc -c xx 12988916 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx $ julia -p 10 -L ./julia_test_parallel.jl ./julia_test_parallel_driver.jl xx; wc -l xx 250001 xx *== Code below same as the quoted github link ==* $ cat ./julia_test_parallel.jl #!/usr/local/julia-cb9bcae93a/bin/julia function processOneFile(filename) np = nprocs() jump = np - 1 jump = jump == 0 ? 1 : jump selfid = myid() # in a single-process setup, this function will be called for parent (id=1) assert(jump == 1 || selfid != 1) f = open(filename); offset = np == 1 ? selfid : selfid - 1 lnum = 0 for l in eachline(f) lnum += 1 if lnum == offset println(Process $(selfid) is processing line $(lnum)) offset += jump end end end $ cat ./julia_test_parallel_driver.jl #!/usr/local/julia-cb9bcae93a/bin/julia filename = nyc311calls.csv np = nprocs() println(Started $(np) processes) if (np 1) if (myid() == 1) # Mulitprocess and I am the parent @sync begin
[julia-users] Increase size of a created sparse matrix?
Is it possible to increase size of a created sparse matrix in a following manner: julia m = sparse([2,3], [1,1], [1,3]) 3x1 sparse matrix with 2 Int64 entries: [2, 1] = 1 [3, 1] = 3 julia m[4,1] = 1 ERROR: BoundsError in setindex! at sparse/sparsematrix.jl:1493 julia m.m = 4 4 julia m[4,1] = 1 1 julia m 4x1 sparse matrix with 3 Int64 entries: [2, 1] = 1 [3, 1] = 3 [4, 1] = 1 Or it will be a dirty hack? What about sparse vector?
Re: [julia-users] Re: Spark and Julia
Hi julia-users, I have the beginnings of a Spark interface for Julia up at https://github.com/jey/Spock.jl. This implementation follows the design used in Spark's Python and R bindings. It so far has the core of the Spark RDD interface implemented, but does not yet implement the additional features of spark-core such as broadcast variables and accumulators. Adding interfaces to other Spark components such as spark-mllib would be an additional significant undertaking that should be revisited once the remainder of spark-core has been implemented. If anyone is interested in working on this, I would be more than happy to provide assistance and guidance. -Jey On Sun, Apr 5, 2015 at 1:59 AM, Viral Shah vi...@mayin.org wrote: It would be nice to co-ordinate these efforts under the JuliaParallel organization. -viral On Sunday, April 5, 2015 at 9:39:51 AM UTC+5:30, wil...@gmail.com wrote: Spark integration is a tricky thing. Python and R bindings go in a great length to map language specific functions into Spark JVM library calls. I guess same could be done with JavaCall.jl package in a manner similar to SparkR. Look at slide 20 from here: http://spark-summit.org/wp- content/uploads/2014/07/SparkR-SparkSummit.pdf. Spark is a clever distributed data access paradigm which grew from Hadoop slowness and limitations. I believe that Julia could provide competitive model for a distributed data storage given Julia's parallel computing approach. Right now, I am writing Julia bindings for Mesos. The idea is to provide, though ClusterManager, access to any Mesos-supervised distributed system and run Julia code that environment. In conjuction with DistributedArrays and DataFrames, it will create powerful toolbox for building distributed systems. After all, machine learning on JVM, really?!. On Saturday, April 4, 2015 at 11:21:35 AM UTC-4, Jeff Waller wrote: On Saturday, April 4, 2015 at 2:22:38 AM UTC-4, Viral Shah wrote: I am changing the subject of this thread from GSOC to Spark. I was just looking around and found this: https://github.com/d9w/Spark.jl https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Fd9w%2FSpark.jlsa=Dsntz=1usg=AFQjCNGQwuYJvzEZVDmzbAm0SsqeV2l46Q Hey, wow, that's interesting, is this an attempt to reimplement Spark or create a binding? The real question is with all the various systems out there, what is the level of abstraction that julia should work with. Julia's DataFrames is one level of abstraction, which could also transparently map to csv files (rather than doing readtable), or a database table, or an HBase table. Why would Spark users want Julia, and why would Julia users want Spark? I guess if we can nail this down - the rest of the integration is probably easy to figure out. As a potential user, I will try to answer in a few parts There are currently 3 official language bindings (Java, Scala, Python) and some unofficial ones as well and R in the works; one thing that users would want is whatever the others get but in the language they desire with an abstraction similar to the other language bindings so that examples in other languages could be readily translated to theirs. Whatever the abstraction turns out the be, there are at least 3 big things that Spark offers; simplification, speed, and lazy evaluation. The abstraction should not make that cumbersome. For me, the advantage of Julia is the syntax, the speed, and the connection to all of the Julia packages and because of that the community of Julia package authors. The advantage of Spark is the machinery of Spark, access to mlib and likewise the community of Spark users. How about an example? This is simply from Spark examples -- good old K-means. This is assuming the Python binding because probably Julia and Python are most alike, how would we expect this to look using Julia? from pyspark.mllib.clustering import KMeans from numpy import array from math import sqrt # Load and parse the data data = sc.textFile(data/mllib/kmeans_data.txt) parsedData = data.map(lambda line: array([float(x) for x in line.split(' ')])) # Build the model (cluster the data) clusters = KMeans.train(parsedData, 2, maxIterations=10, runs=10, initializationMode=random) # Evaluate clustering by computing Within Set Sum of Squared Errors def error(point): center = clusters.centers[clusters.predict(point)] return sqrt(sum([x**2 for x in (point - center)])) WSSSE = parsedData.map(lambda point: error(point)).reduce(lambda x, y: x + y) print(Within Set Sum of Squared Error = + str(WSSSE))
[julia-users] Re: Speeding up indexing into a Dict{Vector,Vector}
Two ideas that *might* work: 1) is hashing the key - that is, if you can represent the Vector{Int} as an immutable type (say, by somehow quickly changing it into an Int64 or Int128, perhaps using hash()) then you store the computed result as the key. The thing to keep in mind here is that if you mutate the vector, you will mutate the hash as well. That is likely what you want, but it's good to be explicit. You will also not be able to go back unless you create a separate dict with the hash values as a key and the Vector{Int} as the value. 2) is creating two data structures: one is a Vector{Vector{Int}}, the other a Dict{Int, Vector{Float64}}. The index of the first data structure is the key of the second, and it should be* faster to check the former against a given vector then going through a dict. Unlike #1, the index will NOT change if you mutate the inner vector. On Monday, April 13, 2015 at 1:56:55 PM UTC-7, Will Kearney wrote: I'm hoping I can get some advice on optimizing some code. I have a data structure which is very conveniently represented as a Dict{Vector{Int},Vector{Float64}}. That is, I need to look up floating-point vectors stored at certain multidimensional (usually 2-10) positions represented as a Vector{Int}. Before I construct this Dict, I don't have a priori knowledge of what position vectors will be represented. I'm aware of the problems of using mutable objects as Dict keys, but let's assume that's not really an issue. Profiling reveals that the real time sink is testing arrays for equality when indexing into this data structure, and indexing happens very frequently. What would be a good way to speed this process up? Is there a way to represent the keys which will be cheaper to compare? I could convert the keys to tuples, but it's helpful to have them as mutable arrays to do some calculations with them at certain points. Is there a different data structure which might work better? Should I roll my own hash table implementation which is particularly optimized for vector keys?
[julia-users] Re: :(::) rather than Symbol in args[1] in :lambda Expr for cartesianarray
Here's a small example: function nef(weights, input_B) delta_B = cartesianarray(Float64, (input_B),) do j sum(weights[spikes_A, j]) end end ct = code_typed(nef, (Array{Float64,2}, Int64)) println(ct) Here's the output: {:($(Expr(:lambda, {:weights,:input_B}, {{:#s2,:delta_B},{{:weights,Array{Float64,2},1},{:input_B,Int64,0},{:#s2,Any,18},{:delta_B,Any,18}},{}}, :(begin #s2 = cartesianarray(AST(:($(Expr(:lambda, {:(j::Any)}, {{},{{:j,Any,0}},{{:weights,Array{Float64,2},1}}}, :(begin # /mnt/home/taanders/pse-hpc/benchmarks2/nengo/ex2.jl, line 3: return sum(getindex(weights,spikes_A,j)) end),Float64,input_B::Int64) delta_B = #s2 return #s2 end} This shows the first lambda arg again as :(j::Any) of type :(::). In my real code, it was at least figuring out in the second lambda arg to type j as {:j,Int64,0} but in this example it doesn't even figure out that j has to be of some Unsigned type and punts back to Any ({:j, Any, 0}).
[julia-users] Using Cartesian to define multidimensional convolution
Hi all, I'm trying to implement a multidimensional convolution defined like this in 2d: We initialize to zero a matrix c and add to it, starting at the (i,j)th position, submatrices b[i, j] * a[:, :] padded with zeros. I have this function: stagedfunction convn{T,N}(A::Array{T,N}, B::Array{T,N}) quote retsize = [size(A)...] + [size(B)...] retsize = tuple(retsize...) ret = zeros(T, retsize) @nloops $N i A begin @nloops $N j B begin (@nref $N ret k-(i_d + j_d)) = (@nref A i) * (@nref B j) end end return ret end end but I'm getting a cryptic ERROR: wrong number of arguments message to this input A = randn(5, 5) convn(A, A) Any help is appreciated. Thanks, Derek
[julia-users] Infinite Dimensional Vectors
Hello All, I am looking for an efficient way to represent vectors that exist in an infinite dimensional space. Specifically I am working with large amounts of text data and will be receiving a lot of data that contains previously unseen words. Each text represents a vector that exists in the space of all possible strings and each word in the text represents a dimension. As such these vectors are extremely sparse. Currently we handle this by using a dictionary to represent each text as a bag of words http://en.wikipedia.org/wiki/Bag-of-words_model vector. If a word does not exist in the vector we return zero. This allows use to perform computations as so: [the=3,and=2,is=4] + [this=5,was=1,where=6] = [where=6,the=3,is=4,this=5,was=1,and=2] euclidean([the=3,and=2,is=4], [this=5,was=1,where=6]) = 9.539392014169456 Is a dictionary the proper associative structure, or should we use a different data structure like a JudyArray or Trie? -MT
[julia-users] Julia Installation Conflict with R
Hi. I am pretty new to Julia, and I did manage to install Julia on Ubuntu precise 64. Everything works except that the installation of Julia updates some libraries and these updates makes the R shared lib /usr/lib/libR.so complain that the symbol _pcre_valid_utf8 is undefined. The libraries updated by Julia that affect R are libpcre3, libpcrecpp0, libpcre3-dev. I am wondering have any of julia users have encounted this issue, and how should I resolve this issue? Best Regards
[julia-users] Julia installation issue
Hi I am very new to Julia, and I have encountered an installation issue of Julia. I did successfully install Julia in my Ubuntu precise. However it seems Julia also updated libpcre3, libpcrecpp2, libpcre3-dev etc..., and these updates broke my original R installation (R installed as a shared library) After the installation of Julia, /usr/lib/libR.so is complaining that the symbol _pcre_valid_utf8 is undefined. I am wondering does anyone else have had this problems, and how can I resolve this? Best Regards
Re: [julia-users] Using Cartesian to define multidimensional convolution
You forgot the $N on the two @nref on the right hand side: On Monday, April 13, 2015 08:05:15 AM Derek Thomas wrote: (@nref $N ret k-(i_d + j_d)) = (@nref A i) * (@nref B j) --Tim
[julia-users] Re: Advice on vectorizing functions
For code outside of my inner loops, I still prefer vectorized functions for compactness and readability. This works: function ldexp!(x::Array{Float64}, e::Int) for i=1:length(x) x[i] = ldexp(x[i], e) end x end Why does this version result in complaints about no matching method for (::Array(Float64,1), ::Int64)? super(Float64) is FloatingPoint, and ldexp() has methods for all subtypes of FloatingPoint paired with Int. function ldexp!(x::Array{FloatingPoint}, e::Int) for i=1:length(x) x[i] = ldexp(x[i], e) end x end
Re: [julia-users] Julia stuck at large floating point number array from source code
It's kind of unclear what you're trying to do. Are you printing an array and the repl hangs? On Mon, Apr 13, 2015 at 10:49 PM, Siyi Deng mr.siyi.d...@gmail.com wrote: Hi, I have a coefficient array which looks like b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537, ], with about 320 elements. The entire array in ascii is about 7000 chars. I cannot paste the array directly in REPL, julia simply stuck. I cannot put it in a script and include it, also stuck the session. Is this a known issue? What shall I do to get my coefficients?
Re: [julia-users] Re: Speeding up indexing into a Dict{Vector,Vector}
If you want to have the original mutable vector around, you may want to keep it as a value instead of a key. You can do something like map the Vector{Int} to a single Int value, use that as a key and then keep the Vector{Int} as part of the Dict value. On Mon, Apr 13, 2015 at 6:06 PM, Seth catch...@bromberger.com wrote: Two ideas that *might* work: 1) is hashing the key - that is, if you can represent the Vector{Int} as an immutable type (say, by somehow quickly changing it into an Int64 or Int128, perhaps using hash()) then you store the computed result as the key. The thing to keep in mind here is that if you mutate the vector, you will mutate the hash as well. That is likely what you want, but it's good to be explicit. You will also not be able to go back unless you create a separate dict with the hash values as a key and the Vector{Int} as the value. 2) is creating two data structures: one is a Vector{Vector{Int}}, the other a Dict{Int, Vector{Float64}}. The index of the first data structure is the key of the second, and it should be* faster to check the former against a given vector then going through a dict. Unlike #1, the index will NOT change if you mutate the inner vector. On Monday, April 13, 2015 at 1:56:55 PM UTC-7, Will Kearney wrote: I'm hoping I can get some advice on optimizing some code. I have a data structure which is very conveniently represented as a Dict{Vector{Int},Vector{Float64}}. That is, I need to look up floating-point vectors stored at certain multidimensional (usually 2-10) positions represented as a Vector{Int}. Before I construct this Dict, I don't have a priori knowledge of what position vectors will be represented. I'm aware of the problems of using mutable objects as Dict keys, but let's assume that's not really an issue. Profiling reveals that the real time sink is testing arrays for equality when indexing into this data structure, and indexing happens very frequently. What would be a good way to speed this process up? Is there a way to represent the keys which will be cheaper to compare? I could convert the keys to tuples, but it's helpful to have them as mutable arrays to do some calculations with them at certain points. Is there a different data structure which might work better? Should I roll my own hash table implementation which is particularly optimized for vector keys?
Re: [julia-users] Re: :(::) rather than Symbol in args[1] in :lambda Expr for cartesianarray
This shows the first lambda arg again as :(j::Any) of type :(::). In my real code, it was at least figuring out in the second lambda arg to type j as {:j,Int64,0} but in this example it doesn't even figure out that j has to be of some Unsigned type and punts back to Any ({:j, Any, 0}). I'm not sure I follow -- I don't see any type annotation of `:(::)`. However, more generally, there are currently some (known) limitations of Julia's type inference for anonymous functions. A do-block creates an anonymous function, so I would guess that is the underlying issue with the inferred types here. (By the way, have you seen the Cartesian macros in base? http://julia.readthedocs.org/en/release-0.3/devdocs/cartesian/) On Mon, Apr 13, 2015 at 7:46 PM, Todd Anderson drtod...@comcast.net wrote: Here's a small example: function nef(weights, input_B) delta_B = cartesianarray(Float64, (input_B),) do j sum(weights[spikes_A, j]) end end ct = code_typed(nef, (Array{Float64,2}, Int64)) println(ct) Here's the output: {:($(Expr(:lambda, {:weights,:input_B}, {{:#s2,:delta_B},{{:weights,Array{Float64,2},1},{:input_B,Int64,0},{:#s2,Any,18},{:delta_B,Any,18}},{}}, :(begin #s2 = cartesianarray(AST(:($(Expr(:lambda, {:(j::Any)}, {{},{{:j,Any,0}},{{:weights,Array{Float64,2},1}}}, :(begin # /mnt/home/taanders/pse-hpc/benchmarks2/nengo/ex2.jl, line 3: return sum(getindex(weights,spikes_A,j)) end),Float64,input_B::Int64) delta_B = #s2 return #s2 end} This shows the first lambda arg again as :(j::Any) of type :(::). In my real code, it was at least figuring out in the second lambda arg to type j as {:j,Int64,0} but in this example it doesn't even figure out that j has to be of some Unsigned type and punts back to Any ({:j, Any, 0}).
[julia-users] Julia stuck at large floating point number array from source code
Hi, I have a coefficient array which looks like b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537, ], with about 320 elements. The entire array in ascii is about 7000 chars. I cannot paste the array directly in REPL, julia simply stuck. I cannot put it in a script and include it, also stuck the session. Is this a known issue? What shall I do to get my coefficients?
Re: [julia-users] Using Cartesian to define multidimensional convolution
By the way, you can do this more easily now, without macros, like this: for IA in CartesianRange(size(A)) for IB in CartesianRange(size(B)) ret[IA+IB] = A[IA] * B[IB] end end --Tim On Monday, April 13, 2015 08:05:15 AM Derek Thomas wrote: Hi all, I'm trying to implement a multidimensional convolution defined like this in 2d: We initialize to zero a matrix c and add to it, starting at the (i,j)th position, submatrices b[i, j] * a[:, :] padded with zeros. I have this function: stagedfunction convn{T,N}(A::Array{T,N}, B::Array{T,N}) quote retsize = [size(A)...] + [size(B)...] retsize = tuple(retsize...) ret = zeros(T, retsize) @nloops $N i A begin @nloops $N j B begin (@nref $N ret k-(i_d + j_d)) = (@nref A i) * (@nref B j) end end return ret end end but I'm getting a cryptic ERROR: wrong number of arguments message to this input A = randn(5, 5) convn(A, A) Any help is appreciated. Thanks, Derek
Re: [julia-users] Using Cartesian to define multidimensional convolution
Thanks. I noticed that right after I posted but couldn't find my post to fix it. Here's a working version using Base.Cartesian stagedfunction convn{T,N}(A::Array{T,N}, B::Array{T,N}) quote retsize = [size(A)...] + [size(B)...] - 1 retsize = tuple(retsize...) ret = zeros(T, retsize) @nloops $N i A begin @nloops $N j B begin (@nref $N ret d-(i_d + j_d - 1)) += (@nref $N A i) * (@nref $N B j) end end return ret end end On Monday, April 13, 2015 at 8:06:08 PM UTC-5, Tim Holy wrote: You forgot the $N on the two @nref on the right hand side: On Monday, April 13, 2015 08:05:15 AM Derek Thomas wrote: (@nref $N ret k-(i_d + j_d)) = (@nref A i) * (@nref B j) --Tim
Re: [julia-users] Infinite Dimensional Vectors
Hard to say what underlying data structure you want here, but it may well be helpful for the keys to be lexicographically ordered – you may want to try a SortedDict as provided by the DataStructures https://github.com/JuliaLang/DataStructures.jl package. You may also want to define a WordVector type that can wrap arbitrary Associative{UTF8String,Int} structures and provides a default value of zero, has methods for various vector operations and norms and such. You may also want to look at the TextAnalysis https://github.com/johnmyleswhite/TextAnalysis.jl package. On Mon, Apr 13, 2015 at 4:41 PM, Mark Tabor mtab...@slu.edu wrote: Hello All, I am looking for an efficient way to represent vectors that exist in an infinite dimensional space. Specifically I am working with large amounts of text data and will be receiving a lot of data that contains previously unseen words. Each text represents a vector that exists in the space of all possible strings and each word in the text represents a dimension. As such these vectors are extremely sparse. Currently we handle this by using a dictionary to represent each text as a bag of words http://en.wikipedia.org/wiki/Bag-of-words_model vector. If a word does not exist in the vector we return zero. This allows use to perform computations as so: [the=3,and=2,is=4] + [this=5,was=1,where=6] = [where=6,the=3,is=4,this=5,was=1,and=2] euclidean([the=3,and=2,is=4], [this=5,was=1,where=6]) = 9.539392014169456 Is a dictionary the proper associative structure, or should we use a different data structure like a JudyArray or Trie? -MT
Re: [julia-users] Julia stuck at large floating point number array from source code
Can you share the script? It's hard to troubleshoot this kind of problem without seeing exactly what you're running. On Mon, Apr 13, 2015 at 9:37 PM, Siyi Deng mr.siyi.d...@gmail.com wrote: No, I'm copy and pasting the array, from a text editor to the REPL, and it hangs there. On Monday, April 13, 2015 at 7:53:29 PM UTC-7, Stefan Karpinski wrote: It's kind of unclear what you're trying to do. Are you printing an array and the repl hangs? On Mon, Apr 13, 2015 at 10:49 PM, Siyi Deng mr.siy...@gmail.com wrote: Hi, I have a coefficient array which looks like b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537, ], with about 320 elements. The entire array in ascii is about 7000 chars. I cannot paste the array directly in REPL, julia simply stuck. I cannot put it in a script and include it, also stuck the session. Is this a known issue? What shall I do to get my coefficients?
Re: [julia-users] Julia stuck at large floating point number array from source code
Hi, my array as follows: b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537,-0.000332845978767382,-0.000588690359282295, -0.000924685049847179,-0.00131849652730193,-0.00172663844738962,-0.00208786977252048,-0.00233221212288082, -0.00239463249655303,-0.00223084413415438,-0.00183153503523871,-0.00123096220524368,-0.000506656901088092, 0.000231273247512499,0.000860169184825533,0.00127049209181452,0.00139133627405064,0.00120931235016613, 0.000775243960106711,0.000195783532681523,-0.000389333441868186,-0.000839715315002484,-0.00104874482507810, -0.000971082976220408,-0.000635588269858116,-0.000139316676086390,0.000376823211516953,0.000765144786939901, 0.000911381735418337,0.000767069729306576,0.000364716481529794,-0.000189255166775865,-0.000742853249617161, -0.00114090399749477,-0.00126989294326956,-0.00109238473975333,-0.000660599971040823,-0.000104137167563394, 0.000405603863401683,0.000704810151823340,0.000686449793195227,0.000334652310940532,-0.000266860012267053, -0.000955811497614575,-0.00153703335145101,-0.00184017465821164,-0.00177248602611196,-0.00135043224798997, -0.000699772991238628,-2.20952367649259e-05,0.000463080417609062,0.000584671530928735,0.000274639234341784, -0.000406253385432046,-0.00128088558535365,-0.00210457446485342,-0.00263694553018369,-0.00271607763844648, -0.00231211432643654,-0.00154295258453597,-0.000645162310074997,9.30623752723795e-05,0.000418248208810247, 0.000191535169468778,-0.000562773863174475,-0.00165664584627324,-0.00278826438187876,-0.00362966080543623, -0.00392678199601546,-0.00358172954744133,-0.00269079897606094,-0.00152435244151553,-0.000451787547517668, 0.000167885370887781,9.77532529617552e-05,-0.000697522948378624,-0.00203068838350111,-0.00353881580196248, -0.00478700223086237,-0.00539960290566932,-0.00517967374898320,-0.00417899466767010,-0.00269445536778436, -0.00118825003033828,-0.000152730366029521,4.08989266963567e-05,-0.000736069064388297,-0.00231761657719439, -0.00427961363828440,-0.00605844774593019,-0.00711896455145624,-0.00712217721790469,-0.00604060957165538, -0.00418280179774257,-0.00211471511846733,-0.000497104824650714,0.000115009784236061,-0.000549582574749379, -0.00237902012751033,-0.00489297896606703,-0.00737250869105017,-0.00907094519283787,-0.00944469951647819, -0.0082955737742,-0.00603030886114787,-0.00321654433831640,-0.000769870161381384,0.000497241843209471, 9.42673474216162e-05,-0.0019763469977,-0.0051916587014,-0.00864083031861910,-0.0112859793931032, -0.012268458734,-0.0112312476909986,-0.00837390013709095,-0.00451267862397702,-0.000818957743044659, 0.00151305882838224,0.00163675539214858,-0.000660398517845471,-0.00484819866174476,-0.00976807860243673, -0.0139500017269034,-0.0160478371765713,-0.0152671906488912,-0.0116534872041850,-0.00614172383099131, -0.000335212041994754,0.00393904297918806,0.00517924921134713,0.00270245065682847,-0.00307207700391787, -0.0106530390555229,-0.0178381403614641,-0.0223253306966040,-0.0224124046400330,-0.0175833758777929, -0.00879634423103870,0.00164096711938262,0.0106118431964252,0.0150317893165007,0.0127790454275655, 0.00345979321865954,-0.0112219144884763,-0.0276295433851027,-0.0409003922756233,-0.0460536030548546, -0.0392442385172429,-0.0188523667724536,0.0138805014852943,0.0548558827557978,0.0977996803069081, 0.135488956158212,0.161253124368556,0.170403259362864,0.161253124368556,0.135488956158212,0.0977996803069081, 0.0548558827557978,0.0138805014852943,-0.0188523667724536,-0.0392442385172429,-0.0460536030548546, -0.0409003922756233,-0.0276295433851027,-0.0112219144884763,0.00345979321865954,0.0127790454275655, 0.0150317893165007,0.0106118431964252,0.00164096711938262,-0.00879634423103870,-0.0175833758777929, -0.0224124046400330,-0.0223253306966040,-0.0178381403614641,-0.0106530390555229,-0.00307207700391787, 0.00270245065682847,0.00517924921134713,0.00393904297918806,-0.000335212041994754,-0.00614172383099131, -0.0116534872041850,-0.0152671906488912,-0.0160478371765713,-0.0139500017269034,-0.00976807860243673, -0.00484819866174476,-0.000660398517845471,0.00163675539214858,0.00151305882838224,-0.000818957743044659, -0.00451267862397702,-0.00837390013709095,-0.0112312476909986,-0.012268458734,-0.0112859793931032, -0.00864083031861910,-0.0051916587014,-0.0019763469977,9.42673474216162e-05,0.000497241843209471, -0.000769870161381384,-0.00321654433831640,-0.00603030886114787,-0.0082955737742,-0.00944469951647819, -0.00907094519283787,-0.00737250869105017,-0.00489297896606703,-0.00237902012751033,-0.000549582574749379, 0.000115009784236061,-0.000497104824650714,-0.00211471511846733,-0.00418280179774257,-0.00604060957165538,
Re: [julia-users] Re: Julia installation issue
Thanks for following up with the answer! I'm sure others will find this helpful. On Tue, Apr 14, 2015 at 12:30 AM, Yudong Ma mayud...@gmail.com wrote: It turns out I have to update R to the latest version. Older version R is not compatible with the updated version of libpcre3 (v1:8.30) shipped with Julia. I have included the link as follow. https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1006573.html Hopefully this is useful for any user who has this issue too. Best. On Monday, April 13, 2015 at 4:37:54 PM UTC-7, Yudong Ma wrote: Hi I am very new to Julia, and I have encountered an installation issue of Julia. I did successfully install Julia in my Ubuntu precise. However it seems Julia also updated libpcre3, libpcrecpp2, libpcre3-dev etc..., and these updates broke my original R installation (R installed as a shared library) After the installation of Julia, /usr/lib/libR.so is complaining that the symbol _pcre_valid_utf8 is undefined. I am wondering does anyone else have had this problems, and how can I resolve this? Best Regards
[julia-users] Re: Julia installation issue
It turns out I have to update R to the latest version. Older version R is not compatible with the updated version of libpcre3 (v1:8.30) shipped with Julia. I have included the link as follow. https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1006573.html Hopefully this is useful for any user who has this issue too. Best. On Monday, April 13, 2015 at 4:37:54 PM UTC-7, Yudong Ma wrote: Hi I am very new to Julia, and I have encountered an installation issue of Julia. I did successfully install Julia in my Ubuntu precise. However it seems Julia also updated libpcre3, libpcrecpp2, libpcre3-dev etc..., and these updates broke my original R installation (R installed as a shared library) After the installation of Julia, /usr/lib/libR.so is complaining that the symbol _pcre_valid_utf8 is undefined. I am wondering does anyone else have had this problems, and how can I resolve this? Best Regards
Re: [julia-users] Julia stuck at large floating point number array from source code
Are you using Windows? Pasting into the REPL there is especially bad, but `include` should work fine (and I just tested it). I also pasted this array into a putty session to a linux box and it only took a few seconds (due to putty not Julia, pretty sure it would be instant in a local login). I cannot put it in a script and include it, also stuck the session. Can you explain why not? On Tue, Apr 14, 2015 at 12:36 AM, Siyi Deng mr.siyi.d...@gmail.com wrote: Hi, my array as follows: b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537,-0.000332845978767382,-0.000588690359282295, -0.000924685049847179,-0.00131849652730193,-0.00172663844738962,-0.00208786977252048,-0.00233221212288082, -0.00239463249655303,-0.00223084413415438,-0.00183153503523871,-0.00123096220524368,-0.000506656901088092, 0.000231273247512499,0.000860169184825533,0.00127049209181452,0.00139133627405064,0.00120931235016613, 0.000775243960106711,0.000195783532681523,-0.000389333441868186,-0.000839715315002484,-0.00104874482507810, -0.000971082976220408,-0.000635588269858116,-0.000139316676086390,0.000376823211516953,0.000765144786939901, 0.000911381735418337,0.000767069729306576,0.000364716481529794,-0.000189255166775865,-0.000742853249617161, -0.00114090399749477,-0.00126989294326956,-0.00109238473975333,-0.000660599971040823,-0.000104137167563394, 0.000405603863401683,0.000704810151823340,0.000686449793195227,0.000334652310940532,-0.000266860012267053, -0.000955811497614575,-0.00153703335145101,-0.00184017465821164,-0.00177248602611196,-0.00135043224798997, -0.000699772991238628,-2.20952367649259e-05,0.000463080417609062,0.000584671530928735,0.000274639234341784, -0.000406253385432046,-0.00128088558535365,-0.00210457446485342,-0.00263694553018369,-0.00271607763844648, -0.00231211432643654,-0.00154295258453597,-0.000645162310074997,9.30623752723795e-05,0.000418248208810247, 0.000191535169468778,-0.000562773863174475,-0.00165664584627324,-0.00278826438187876,-0.00362966080543623, -0.00392678199601546,-0.00358172954744133,-0.00269079897606094,-0.00152435244151553,-0.000451787547517668, 0.000167885370887781,9.77532529617552e-05,-0.000697522948378624,-0.00203068838350111,-0.00353881580196248, -0.00478700223086237,-0.00539960290566932,-0.00517967374898320,-0.00417899466767010,-0.00269445536778436, -0.00118825003033828,-0.000152730366029521,4.08989266963567e-05,-0.000736069064388297,-0.00231761657719439, -0.00427961363828440,-0.00605844774593019,-0.00711896455145624,-0.00712217721790469,-0.00604060957165538, -0.00418280179774257,-0.00211471511846733,-0.000497104824650714,0.000115009784236061,-0.000549582574749379, -0.00237902012751033,-0.00489297896606703,-0.00737250869105017,-0.00907094519283787,-0.00944469951647819, -0.0082955737742,-0.00603030886114787,-0.00321654433831640,-0.000769870161381384,0.000497241843209471, 9.42673474216162e-05,-0.0019763469977,-0.0051916587014,-0.00864083031861910,-0.0112859793931032, -0.012268458734,-0.0112312476909986,-0.00837390013709095,-0.00451267862397702,-0.000818957743044659, 0.00151305882838224,0.00163675539214858,-0.000660398517845471,-0.00484819866174476,-0.00976807860243673, -0.0139500017269034,-0.0160478371765713,-0.0152671906488912,-0.0116534872041850,-0.00614172383099131, -0.000335212041994754,0.00393904297918806,0.00517924921134713,0.00270245065682847,-0.00307207700391787, -0.0106530390555229,-0.0178381403614641,-0.0223253306966040,-0.0224124046400330,-0.0175833758777929, -0.00879634423103870,0.00164096711938262,0.0106118431964252,0.0150317893165007,0.0127790454275655, 0.00345979321865954,-0.0112219144884763,-0.0276295433851027,-0.0409003922756233,-0.0460536030548546, -0.0392442385172429,-0.0188523667724536,0.0138805014852943,0.0548558827557978,0.0977996803069081, 0.135488956158212,0.161253124368556,0.170403259362864,0.161253124368556,0.135488956158212,0.0977996803069081, 0.0548558827557978,0.0138805014852943,-0.0188523667724536,-0.0392442385172429,-0.0460536030548546, -0.0409003922756233,-0.0276295433851027,-0.0112219144884763,0.00345979321865954,0.0127790454275655, 0.0150317893165007,0.0106118431964252,0.00164096711938262,-0.00879634423103870,-0.0175833758777929, -0.0224124046400330,-0.0223253306966040,-0.0178381403614641,-0.0106530390555229,-0.00307207700391787, 0.00270245065682847,0.00517924921134713,0.00393904297918806,-0.000335212041994754,-0.00614172383099131, -0.0116534872041850,-0.0152671906488912,-0.0160478371765713,-0.0139500017269034,-0.00976807860243673, -0.00484819866174476,-0.000660398517845471,0.00163675539214858,0.00151305882838224,-0.000818957743044659, -0.00451267862397702,-0.00837390013709095,-0.0112312476909986,-0.012268458734,-0.0112859793931032, -0.00864083031861910,-0.0051916587014,-0.0019763469977,9.42673474216162e-05,0.000497241843209471, -0.000769870161381384,-0.00321654433831640,-0.00603030886114787,-0.0082955737742,-0.00944469951647819,
Re: [julia-users] Julia stuck at large floating point number array from source code
Yes I am using windows. Is there a way to help the pasting issue? because it severely limits the usefulness of REPL if you cannot paste directly into it.Including this array in a script hanged for me the first time I tried (was like 2 minutes). But seems to be working fine when I tested it again separately, so I guess it was because of other parts of my script, not the array. On Monday, April 13, 2015 at 9:51:54 PM UTC-7, Isaiah wrote: Are you using Windows? Pasting into the REPL there is especially bad, but `include` should work fine (and I just tested it). I also pasted this array into a putty session to a linux box and it only took a few seconds (due to putty not Julia, pretty sure it would be instant in a local login). I cannot put it in a script and include it, also stuck the session. Can you explain why not? On Tue, Apr 14, 2015 at 12:36 AM, Siyi Deng mr.siy...@gmail.com javascript: wrote: Hi, my array as follows: b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537,-0.000332845978767382,-0.000588690359282295, -0.000924685049847179,-0.00131849652730193,-0.00172663844738962,-0.00208786977252048,-0.00233221212288082, -0.00239463249655303,-0.00223084413415438,-0.00183153503523871,-0.00123096220524368,-0.000506656901088092, 0.000231273247512499,0.000860169184825533,0.00127049209181452,0.00139133627405064,0.00120931235016613, 0.000775243960106711,0.000195783532681523,-0.000389333441868186,-0.000839715315002484,-0.00104874482507810, -0.000971082976220408,-0.000635588269858116,-0.000139316676086390,0.000376823211516953,0.000765144786939901, 0.000911381735418337,0.000767069729306576,0.000364716481529794,-0.000189255166775865,-0.000742853249617161, -0.00114090399749477,-0.00126989294326956,-0.00109238473975333,-0.000660599971040823,-0.000104137167563394, 0.000405603863401683,0.000704810151823340,0.000686449793195227,0.000334652310940532,-0.000266860012267053, -0.000955811497614575,-0.00153703335145101,-0.00184017465821164,-0.00177248602611196,-0.00135043224798997, -0.000699772991238628,-2.20952367649259e-05,0.000463080417609062,0.000584671530928735,0.000274639234341784, -0.000406253385432046,-0.00128088558535365,-0.00210457446485342,-0.00263694553018369,-0.00271607763844648, -0.00231211432643654,-0.00154295258453597,-0.000645162310074997,9.30623752723795e-05,0.000418248208810247, 0.000191535169468778,-0.000562773863174475,-0.00165664584627324,-0.00278826438187876,-0.00362966080543623, -0.00392678199601546,-0.00358172954744133,-0.00269079897606094,-0.00152435244151553,-0.000451787547517668, 0.000167885370887781,9.77532529617552e-05,-0.000697522948378624,-0.00203068838350111,-0.00353881580196248, -0.00478700223086237,-0.00539960290566932,-0.00517967374898320,-0.00417899466767010,-0.00269445536778436, -0.00118825003033828,-0.000152730366029521,4.08989266963567e-05,-0.000736069064388297,-0.00231761657719439, -0.00427961363828440,-0.00605844774593019,-0.00711896455145624,-0.00712217721790469,-0.00604060957165538, -0.00418280179774257,-0.00211471511846733,-0.000497104824650714,0.000115009784236061,-0.000549582574749379, -0.00237902012751033,-0.00489297896606703,-0.00737250869105017,-0.00907094519283787,-0.00944469951647819, -0.0082955737742,-0.00603030886114787,-0.00321654433831640,-0.000769870161381384,0.000497241843209471, 9.42673474216162e-05,-0.0019763469977,-0.0051916587014,-0.00864083031861910,-0.0112859793931032, -0.012268458734,-0.0112312476909986,-0.00837390013709095,-0.00451267862397702,-0.000818957743044659, 0.00151305882838224,0.00163675539214858,-0.000660398517845471,-0.00484819866174476,-0.00976807860243673, -0.0139500017269034,-0.0160478371765713,-0.0152671906488912,-0.0116534872041850,-0.00614172383099131, -0.000335212041994754,0.00393904297918806,0.00517924921134713,0.00270245065682847,-0.00307207700391787, -0.0106530390555229,-0.0178381403614641,-0.0223253306966040,-0.0224124046400330,-0.0175833758777929, -0.00879634423103870,0.00164096711938262,0.0106118431964252,0.0150317893165007,0.0127790454275655, 0.00345979321865954,-0.0112219144884763,-0.0276295433851027,-0.0409003922756233,-0.0460536030548546, -0.0392442385172429,-0.0188523667724536,0.0138805014852943,0.0548558827557978,0.0977996803069081, 0.135488956158212,0.161253124368556,0.170403259362864,0.161253124368556,0.135488956158212,0.0977996803069081, 0.0548558827557978,0.0138805014852943,-0.0188523667724536,-0.0392442385172429,-0.0460536030548546, -0.0409003922756233,-0.0276295433851027,-0.0112219144884763,0.00345979321865954,0.0127790454275655,
Re: [julia-users] Julia stuck at large floating point number array from source code
No, I'm copy and pasting the array, from a text editor to the REPL, and it hangs there. On Monday, April 13, 2015 at 7:53:29 PM UTC-7, Stefan Karpinski wrote: It's kind of unclear what you're trying to do. Are you printing an array and the repl hangs? On Mon, Apr 13, 2015 at 10:49 PM, Siyi Deng mr.siy...@gmail.com javascript: wrote: Hi, I have a coefficient array which looks like b = [4.67933552111843e-07,-6.32591924726271e-05,-0.000160070579209537, ], with about 320 elements. The entire array in ascii is about 7000 chars. I cannot paste the array directly in REPL, julia simply stuck. I cannot put it in a script and include it, also stuck the session. Is this a known issue? What shall I do to get my coefficients?
[julia-users] Re: Advice on vectorizing functions
On Tuesday, April 14, 2015 at 9:40:49 AM UTC+10, SixString wrote: For code outside of my inner loops, I still prefer vectorized functions for compactness and readability. This works: function ldexp!(x::Array{Float64}, e::Int) for i=1:length(x) x[i] = ldexp(x[i], e) end x end Why does this version result in complaints about no matching method for (::Array(Float64,1), ::Int64)? super(Float64) is FloatingPoint, and ldexp() has methods for all subtypes of FloatingPoint paired with Int. But Float64 : FloatingPoint does not imply Array{Float64} : Array{FloatingPoint}, see http://en.wikipedia.org/wiki/Covariance_and_contravariance_%28computer_science%29#Arrays for why. Cheers Lex function ldexp!(x::Array{FloatingPoint}, e::Int) for i=1:length(x) x[i] = ldexp(x[i], e) end x end
[julia-users] Re: pre-allocation for sparse matrices
If you are overwriting all your previous values you can just send in the matrix into the function and update with A[i,j] = v On Monday, April 13, 2015 at 6:47:21 PM UTC+2, Andrei Berceanu wrote: Yes but so how do I update it inplace? On Monday, April 13, 2015 at 5:37:50 PM UTC+2, Kristoffer Carlsson wrote: If you have the same sparse structures in your matrices then it should be sufficient to to the IJV - CSC sparse matrix conversion only once and then update A in place instead of generating a new IJV and converting to CSC again and again.
Re: [julia-users] Re: ApproxFun First Order PDE Question
Thanks very much for this. I had assumed by specifying `\[f]` that I *had* given just one boundary condition. I tried your solution but ran into a problem which I have raised as a ticket as you suggest: https://github.com/ApproxFun/ApproxFun.jl/issues/159 https://github.com/ApproxFun/ApproxFun.jl/issues/159 Dominic Steinitz domi...@steinitz.org http://idontgetoutmuch.wordpress.com On 13 Apr 2015, at 03:00, Sheehan Olver dlfivefi...@gmail.com wrote: You have too many boundary conditions: dirichlet conditions on a rectangle defaults to conditions on all 4 sides. Imposing on just the left side gives an answer: d = domain(Fun(identity,[0.0,1.0])) f = Fun(x-exp(-x^2),d) D = Derivative(d) L = 2.0D⊗I+I⊗D u = [ldirichlet(d)⊗I,L]\[f] though you may need another boundary condition. to get the answer you are looking for. PS I'm more likely to see this if you post it as a git issue on ApproxFun's github page. On Sunday, April 12, 2015 at 12:10:24 AM UTC+10, idontgetoutmuch wrote: I am trying to solve a first order PDE to which I know the solution $2u_x + u_y = 0$ with boundary condition $u(x,0) = exp{-x^2}$. The solution is $exp{-(x-2t)^2}$. I have tried the code below Pkg.checkout(ApproxFun) using ApproxFun d = domain(Fun(identity,[0.0,1.0])) f = Fun(x-exp(-x^2),d) D = Derivative(d) L = 2.0D⊗I+I⊗D u = [dirichlet(d^2),L]\[f] but I get endless WARNING: Maximum number of iterations 10 reached WARNING: Maximum number of iterations 10 reached WARNING: Maximum number of iterations 10 reached Any help would be gratefully received.
Re: [julia-users] string processing after readdlm
ok, here is the example: a = Array{Float32}[] push!(a,[1 2 3]) push!(a,[4 5 4]) push!(a,[7 8 9]) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) the result is: Array{Array{Float32,N},1} Array{Any,2} Any[Float32[1.0 2.0 3.0]] and the same behawior here: a = {} push!(a,rand(5)) push!(a,rand(3)) push!(a,rand(7)) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) println(typeof(b[1,1])) and the result: - - - - - - - cljs Array{Any,1} Array{Any,2} Any[[0.054008985630280115,0.8947273976690304,0.14961853234717193,0.72895523733,0.8907801902141823]] SubString{ASCIIString} in both cases I would like to have a method to write the data and next to read it and have it with the same structure and types (or just a function to process this substrings). J On Sunday, April 12, 2015 at 7:55:30 PM UTC+2, Mauro wrote: An example which can be copy-pasted would be helpful. M On Sun, 2015-04-12 at 16:47, JKpie fable...@gmail.com javascript: wrote: Hi, I am new Julia coder (I started to learn Julia two weeks ago). I have a problem with writedlm and readdlm functions. I have an array of vectors: Vector Array{Float32,N},4 When I try to save it using writedlm to txt file and next read that file using readdlm I obtain a matrix: Matrix Any, 4 which contains four substrings like: Float32[1,2,3... so my question is: is there any simple way to convert that strings into arrays of floats? I also need to write and read files of Any type arrays which contains float vectors of different lengths. Thank you very much for your help, J
[julia-users] Problem with addprocs
I'm unable to add a remote instance using addprocs (or a machine file). This works (without prompt): ssh xxx.xxx.xxx.xxx This command hangs with no output for the first minute: addprocs([usern...@xxx.xxx.xxx.xxx]) Then after one minute, the following message appears: Master process (id 1) could not connect within 60.0 seconds This command works: addprocs([username@mycurrenthostname]) Thanks in advance for your help! John
[julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
Hi, I had some code like if isa(spec, Symbol) A[i,spec] = 1 elseif isa(spec, Vector{(Symbol,Float64)}) @assert(isapprox(sum(map(x - x[2],spec)),1)) for (obs,prob) = spec A[i,obs] = prob end else error(don't know how to parse $spec for aggregation) end but got bittenby a spec = [(:I,0),(:U,1)] which is valid for my problem, so I realized I need to test for Real, not Float64. If this was a method of a generic fuction, I could simply use T : Real and then use Vector{(Symbol,T)} in the type signature --- but it isn't, so I would be interested in how to test for the condition in the subject line in an if branch. isa(Vector{(Symbol,Real)}) won't work, my understanding is that this is because of invariance. Best, Tamas
[julia-users] Re: Symbolic relations package
How about Equations? It captures the main theme so that people scrolling through the list of packages will instantly gather its purpose. Any other suggestions or objections? On Sunday, 12 April 2015 14:02:02 UTC+3, Marcus Appelros wrote: Am working on code to derive mathematical relations, its initial purpose was in the field of non-commutative geometry however it appears to have become generally useful, feels like a good idea if we make the base self contained and release it as a separate package upon which more specialized ones can be built. The code is here: https://github.com/jhlq/fys/tree/master/ncg quin@Nicol:~/fys/ncg$ julia -L equations -q Let's start from the top, the most recent file is matchers.jl which currently contains one matcher function that checks if an equation matches the pattern ax^2+bx+c=0. By writing such matchers (and introducing types for new operators) the base can readily be extended to solve custom equations. The quadratic matcher in action: julia @time solve(:a*(:x*:y*:z)^2+:b*:y*:z*:x+:c,Function)[end] #the type specifies which kind of operator to apply elapsed time: 1.779767594 seconds (426648608 bytes allocated, 25.89% gc time) Equation(Expression({:x,:z,:y}),Expression({-1,:b,÷(Expression({2,:a})),:+,-1,Sqrt(Expression({:b,:b,:+,-4,:a,:c})),÷(Expression({2,:a}))})) The Equation type is specified in equations.jl, it has two fields, lhs and rhs. The solve function when given an expression assumes it is the lhs of an equation with rhs=0 and finds solutions by calling the matches function which so far also accepts the Sqrt and Div types as ops, these types implement their own simplify methods. The original matches passes terms and factors from left to right: julia matches(:x+:y) 6-element Array{Equation,1}: Equation(:y,Expression({-1,:x})) Equation(1,Expression({-1,:x,÷(:y)})) Equation(0,Expression({-1,:x,:+,-1,:y})) Equation(:x,Expression({-1,:y})) Equation(1,Expression({-1,:y,÷(:x)})) Equation(0,Expression({-1,:y,:+,-1,:x})) Now we descend into the base at common.jl. The function simplify is essential and called everywhere (probably too manywhere), it resolves nested expressions and sums it all up: julia ex=(:x+3+9*:x)^3 Expression({Expression({:x,:+,3,:+,Expression({9,:x})}),Expression({Expression({:x,:+,3,:+,Expression({9,:x})}),Expression({:x,:+,3,:+,Expression({9,:x})})})}) julia simex=simplify(ex) Expression({1000,:x,:x,:x,:+,900,:x,:x,:+,270,:x,:+,27}) julia evaluate(ex,[:x=1])==evaluate(simex,[:x=1]) true A crucial component of simplify is componify which is the trustable cookie unwrapper (technical term), it extracts components without eating anyting: julia ex=:x-(:x-3)+(:y+3)^2 Expression({:x,:+,-1,:x,:+,-1,-1,3,:+,Expression({Expression({:y,:+,3}),Expression({:y,:+,3})})}) julia ex=componify(ex) Expression({:x,:+,-1,:x,:+,-1,-1,3,:+,:y,:y,:+,:y,3,:+,3,:y,:+,3,3}) One final frequently used internal function is addparse, it returns an array of arrays with factors. For many cases the iterative shorthand will be sufficient: julia for term in ex;print(term);end {:x}{-1,:x}{-1,3}{:y,:y}{:y,3}{3,:y}{3,3} There are a whole host of additional details that optimization pros presumably can have many field days, camping trips and holidays with. Will continue nurturing these wines to climb toward quantum gravity and desire the community to be involved so everyone is very welcome to hack away at the jungle, add branches and tread the paths created. We have a choice to make, representing a structural birth for the code: What shall this package be named?
Re: [julia-users] string processing after readdlm
Is that supposed to be a lot of blank lines or is there something missing from the message? On Mon, Apr 13, 2015 at 6:56 AM, JKpie fablekil...@gmail.com wrote: ok, here is the example: a = Array{Float32}[] push!(a,[1 2 3]) push!(a,[4 5 4]) push!(a,[7 8 9]) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) the result is: Array{Array{Float32,N},1} Array{Any,2} Any[Float32[1.0 2.0 3.0]] and the same behawior here: a = {} push!(a,rand(5)) push!(a,rand(3)) push!(a,rand(7)) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) println(typeof(b[1,1])) and the result: - - - - - - - cljs Array{Any,1} Array{Any,2} Any[[0.054008985630280115,0.8947273976690304,0.14961853234717193,0.72895523733,0.8907801902141823]] SubString{ASCIIString} in both cases I would like to have a method to write the data and next to read it and have it with the same structure and types (or just a function to process this substrings). J On Sunday, April 12, 2015 at 7:55:30 PM UTC+2, Mauro wrote: An example which can be copy-pasted would be helpful. M On Sun, 2015-04-12 at 16:47, JKpie fable...@gmail.com wrote: Hi, I am new Julia coder (I started to learn Julia two weeks ago). I have a problem with writedlm and readdlm functions. I have an array of vectors: Vector Array{Float32,N},4 When I try to save it using writedlm to txt file and next read that file using readdlm I obtain a matrix: Matrix Any, 4 which contains four substrings like: Float32[1,2,3... so my question is: is there any simple way to convert that strings into arrays of floats? I also need to write and read files of Any type arrays which contains float vectors of different lengths. Thank you very much for your help, J
[julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
You could make a typealias with a type parameter T for your type and check against that with isa.
[julia-users] Re: pre-allocation for sparse matrices
OK so I'm not sure how to directly access the CSC structure. Anyway, for clarity, here is the method which generates my sparse matrix. In a typical usage scenario, I need to generate and diagonalize a lot of these matrices, which are obtained varying the *s* Function. I guess this qualifies as a low-order method? *function genspmat(l::Function,r::Function,u::Function,d::Function,s::Function, N::Int,nz::Int,α::Float64)# PreallocateI = Array(Int64,nz)J = Array(Int64,nz)V = Array(Complex{Float64},nz) function setnzelem(i::Int,n::Int,m::Int; pos::ASCIIString = self)if pos==leftk += 1J[k] = i-N; I[k] = i; V[k] = l(n,m,α)elseif pos==rightk += 1J[k] = i+N; I[k] = i; V[k] = r(n,m,α)elseif pos==upk += 1J[k] = i-1; I[k] = i; V[k] = u(n,m,α)elseif pos==downk += 1J[k] = i+1; I[k] = i; V[k] = d(n,m,α)elseif pos==selfk += 1J[k] = i; I[k] = i; V[k] = s(n,m,α)endend# maximum value of m or n indicesmaxm = div(N-1,2) k = 0for i in 1:N^2m = getm(i,N)n = getn(i,N)setnzelem(i,n,m; pos=self) #corners#top leftif n==maxm m==-maxm setnzelem(i,n,m; pos=right)setnzelem(i,n,m; pos=down)#top rightelseif n==maxm m==maxm setnzelem(i,n,m; pos=left)setnzelem(i,n,m; pos=down) #bottom rightelseif n==-maxm m==maxm setnzelem(i,n,m; pos=left)setnzelem(i,n,m; pos=up) #bottom leftelseif n==-maxm m==-maxm setnzelem(i,n,m; pos=right)setnzelem(i,n,m; pos=up) #edges#topelseif n == maxmsetnzelem(i,n,m; pos=right)setnzelem(i,n,m; pos=left) setnzelem(i,n,m; pos=down)#rightelseif m == maxmsetnzelem(i,n,m; pos=left)setnzelem(i,n,m; pos=up)setnzelem(i,n,m; pos=down)#bottom elseif n == -maxmsetnzelem(i,n,m; pos=left) setnzelem(i,n,m; pos=up)setnzelem(i,n,m; pos=right) #leftelseif m == -maxmsetnzelem(i,n,m; pos=down)setnzelem(i,n,m; pos=up) setnzelem(i,n,m; pos=right)else #bulksetnzelem(i,n,m; pos=down)setnzelem(i,n,m; pos=up) setnzelem(i,n,m; pos=right)setnzelem(i,n,m; pos=left)endendreturn sparse(I,J,V)end* On Friday, April 10, 2015 at 10:15:22 PM UTC+2, Christoph Ortner wrote: For example, for finite elements or finite differences, the connectivity information gives you the sparsity pattern, which you could assemble once and for all and then change the entries as needed. If it is a low-order method, then you have few elements in each column and it would be very efficient (and easy to implement) to make these changes by directly accessing the CSC structure. Christoph
Re: [julia-users] string processing after readdlm
I think you're expecting too much from writedlm/readdlm. I don't think it is all that useful beyond 2D arrays of simple datatype. An array of an array is not simple. What it seems to be doing write a string representation of `a` to the file: repr(a) You can recover its meaning with eval(parse(repr(a))) b = readdlm(joinpath(path,text.txt), '\n') b = map(x-eval(parse(x)), b) or b= Array{Float32}[eval(parse(bb)) for bb in b] However, it would be better, faster and safer to just a more powerful file fromat like HDF5.jl. On Mon, 2015-04-13 at 12:56, JKpie fablekil...@gmail.com wrote: ok, here is the example: a = Array{Float32}[] push!(a,[1 2 3]) push!(a,[4 5 4]) push!(a,[7 8 9]) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) the result is: Array{Array{Float32,N},1} Array{Any,2} Any[Float32[1.0 2.0 3.0]] and the same behawior here: a = {} push!(a,rand(5)) push!(a,rand(3)) push!(a,rand(7)) println(typeof(a)) writedlm(joinpath(path,text.txt),a) b = readdlm(joinpath(path,text.txt)) println(typeof(b)) println(b[1,:]) println(typeof(b[1,1])) and the result: - - - - - - - cljs Array{Any,1} Array{Any,2} Any[[0.054008985630280115,0.8947273976690304,0.14961853234717193,0.72895523733,0.8907801902141823]] SubString{ASCIIString} in both cases I would like to have a method to write the data and next to read it and have it with the same structure and types (or just a function to process this substrings). J On Sunday, April 12, 2015 at 7:55:30 PM UTC+2, Mauro wrote: An example which can be copy-pasted would be helpful. M On Sun, 2015-04-12 at 16:47, JKpie fable...@gmail.com javascript: wrote: Hi, I am new Julia coder (I started to learn Julia two weeks ago). I have a problem with writedlm and readdlm functions. I have an array of vectors: Vector Array{Float32,N},4 When I try to save it using writedlm to txt file and next read that file using readdlm I obtain a matrix: Matrix Any, 4 which contains four substrings like: Float32[1,2,3... so my question is: is there any simple way to convert that strings into arrays of floats? I also need to write and read files of Any type arrays which contains float vectors of different lengths. Thank you very much for your help, J
[julia-users] Re: Holt-Winters D-E Smoothing in Julia
On Saturday, April 11, 2015 at 1:19:39 AM UTC-4, colint...@gmail.com wrote: Great! If you decide to open source it when done then please feel free to post a link into this thread or start a new thread on julia-stats as I for one would definitely be interested. Will do.
Re: [julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
On Mon, 2015-04-13 at 16:22, Tamas Papp tkp...@gmail.com wrote: On Mon, Apr 13 2015, Mauro mauro...@runbox.com wrote: Why can you not make it into a function? function foo!(spec::Symbol, A, i) A[i,spec] = 1 end function foo!{T:Real}(spec::Vector{(Symbol,T), A, i) @assert(isapprox(sum(map(x - x[2],spec)),1)) for (obs,prob) = spec A[i,obs] = prob end A[i,spec] = 1 end foo!(spec, A, i) = error( ...) that way your code will be cleaner too: foo!(spec, A, i) Sure, I can do this, but since foo! would be used in one place only, I would consider this suboptimal design. In Julia, when you use an isa, then ask yourself whether it might not be cleaner using methods instead. I am not sure I understand the reasoning behind this. I find polluting the namespace with wrapping up trivial code that is used in exactly one place icky. Then you have to use Toivo's trick or wait for https://github.com/JuliaLang/julia/issues/8974 which should make this possible (as far as I understand it). I've used above approach and I like it. If foo! is well named then reading of the main function becomes much easier.
Re: [julia-users] Re: non-concatenating vector creation
This is perfect --- I think I will end up with something like typealias NCA Any and then I can use NCA[1,2,[3,4]] and later on remove NCA from my code easily. Thanks, Tamas On Mon, Apr 13 2015, Matt Bauman mbau...@gmail.com wrote: Is this what you're after? I think this should still work after the concatenation change is applied. julia Any[1,2,[3,4]] 3-element Array{Any,1}: 1 2 [3,4] On Monday, April 13, 2015 at 8:55:19 AM UTC-4, Tamas Papp wrote: Hi, I have been following issue 7128 [1] and the related other issues, and noticed that constructs like [1,2,[3,4]] give a warning in the dev version I am using at the moment (v0.4.0-dev+4219), but I am wondering if there is a construct that I can use for non-concatenating vector creation _at the moment_, that is reasonably future-proof (eg it works for a couple of months and when it does not, it fails with an error). Best, Tamas [1] https://github.com/JuliaLang/julia/issues/7128
Re: [julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
On Mon, Apr 13 2015, Mauro mauro...@runbox.com wrote: Why can you not make it into a function? function foo!(spec::Symbol, A, i) A[i,spec] = 1 end function foo!{T:Real}(spec::Vector{(Symbol,T), A, i) @assert(isapprox(sum(map(x - x[2],spec)),1)) for (obs,prob) = spec A[i,obs] = prob end A[i,spec] = 1 end foo!(spec, A, i) = error( ...) that way your code will be cleaner too: foo!(spec, A, i) Sure, I can do this, but since foo! would be used in one place only, I would consider this suboptimal design. In Julia, when you use an isa, then ask yourself whether it might not be cleaner using methods instead. I am not sure I understand the reasoning behind this. I find polluting the namespace with wrapping up trivial code that is used in exactly one place icky. Best, Tamas
[julia-users] Re: Problem with addprocs
Oops, let me clarify: mycurrenthostname is not the same computer as xxx.xxx.xxx.xxx, it's my local computer's name on the network, whereas xxx.xxx.xxx.xxx is remote. On Monday, April 13, 2015 at 3:02:40 PM UTC+1, wil...@gmail.com wrote: This could be DNS issue. Try add 'mycurrenthostname xxx.xxx.xxx.xxx' to the 'hosts' file. On Monday, April 13, 2015 at 6:10:11 AM UTC-4, John wrote: I'm unable to add a remote instance using addprocs (or a machine file). This works (without prompt): ssh xxx.xxx.xxx.xxx This command hangs with no output for the first minute: addprocs([usern...@xxx.xxx.xxx.xxx]) Then after one minute, the following message appears: Master process (id 1) could not connect within 60.0 seconds This command works: addprocs([username@mycurrenthostname]) Thanks in advance for your help! John
Re: [julia-users] Is the official name of the language Julia or julia?
About that font: Twitter discussion with Muthu Nedumaran: https://twitter.com/typographica/status/572304422610452480 FontsInUse entry: http://fontsinuse.com/typefaces/38527/mn-latin
[julia-users] Re: non-concatenating vector creation
Is this what you're after? I think this should still work after the concatenation change is applied. julia Any[1,2,[3,4]] 3-element Array{Any,1}: 1 2 [3,4] On Monday, April 13, 2015 at 8:55:19 AM UTC-4, Tamas Papp wrote: Hi, I have been following issue 7128 [1] and the related other issues, and noticed that constructs like [1,2,[3,4]] give a warning in the dev version I am using at the moment (v0.4.0-dev+4219), but I am wondering if there is a construct that I can use for non-concatenating vector creation _at the moment_, that is reasonably future-proof (eg it works for a couple of months and when it does not, it fails with an error). Best, Tamas [1] https://github.com/JuliaLang/julia/issues/7128
Re: [julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
Why can you not make it into a function? function foo!(spec::Symbol, A, i) A[i,spec] = 1 end function foo!{T:Real}(spec::Vector{(Symbol,T), A, i) @assert(isapprox(sum(map(x - x[2],spec)),1)) for (obs,prob) = spec A[i,obs] = prob end A[i,spec] = 1 end foo!(spec, A, i) = error( ...) that way your code will be cleaner too: foo!(spec, A, i) In Julia, when you use an isa, then ask yourself whether it might not be cleaner using methods instead. If that is not an option you have to make a function for the check... On Mon, 2015-04-13 at 14:06, Tamas Papp tkp...@gmail.com wrote: Hi, I had some code like if isa(spec, Symbol) A[i,spec] = 1 elseif isa(spec, Vector{(Symbol,Float64)}) @assert(isapprox(sum(map(x - x[2],spec)),1)) for (obs,prob) = spec A[i,obs] = prob end else error(don't know how to parse $spec for aggregation) end but got bittenby a spec = [(:I,0),(:U,1)] which is valid for my problem, so I realized I need to test for Real, not Float64. If this was a method of a generic fuction, I could simply use T : Real and then use Vector{(Symbol,T)} in the type signature --- but it isn't, so I would be interested in how to test for the condition in the subject line in an if branch. isa(Vector{(Symbol,Real)}) won't work, my understanding is that this is because of invariance. Best, Tamas
Re: [julia-users] how to test if object is a Vector{(Symbol,T)} with T : Real
Thanks. Is there a local solution that does not interfere with the namespace? Best, Tamas On Mon, Apr 13 2015, Toivo Henningsson toivo@gmail.com wrote: You could make a typealias with a type parameter T for your type and check against that with isa.
Re: [julia-users] Re: in-place matrix division
The problem is, as I loop though my parameter space, I need to solve a lot of linear systems of the type \(A,B), where B stays the same but A changes depending on the parameters. Therefore a method that overwrites B doen't really help. On Friday, April 10, 2015 at 6:19:34 PM UTC+2, Andreas Noack wrote: Yes. This has to be made more clear. The problem is that the libraries we are using for sparse factorizations don't allow us to update b in place. 2015-04-10 4:26 GMT-07:00 Kristoffer Carlsson kcarl...@gmail.com javascript:: I think it should be A_ldiv_B! however this seems inconcistent to me: julia a = rand(3,3); b = rand(3) 3-element Array{Float64,1}: 0.95134 0.43887 0.719551 julia A_ldiv_B!(sparse(a), b) 3-element Array{Float64,1}: 0.455667 0.712479 0.326213 julia b # Note: b is not overwritten 3-element Array{Float64,1}: 0.95134 0.43887 0.719551 julia A_ldiv_B!(lufact(a), b) 3-element Array{Float64,1}: 0.455667 0.712479 0.326213 julia b # For this, b is overwritten 3-element Array{Float64,1}: 0.455667 0.712479 0.326213 On Friday, April 10, 2015 at 11:19:45 AM UTC+2, Andrei Berceanu wrote: I am solving a linear system of the type A*X == B, where A is a sparse matrix. I use the matrix division function \(*A*, *B*) Is there an in-place version of this function, where one can pre-allocate the output X and then pass it, i.e. \!(A,B,X)
[julia-users] non-concatenating vector creation
Hi, I have been following issue 7128 [1] and the related other issues, and noticed that constructs like [1,2,[3,4]] give a warning in the dev version I am using at the moment (v0.4.0-dev+4219), but I am wondering if there is a construct that I can use for non-concatenating vector creation _at the moment_, that is reasonably future-proof (eg it works for a couple of months and when it does not, it fails with an error). Best, Tamas [1] https://github.com/JuliaLang/julia/issues/7128
[julia-users] Re: Problem with addprocs
This could be DNS issue. Try add 'mycurrenthostname xxx.xxx.xxx.xxx' to the 'hosts' file. On Monday, April 13, 2015 at 6:10:11 AM UTC-4, John wrote: I'm unable to add a remote instance using addprocs (or a machine file). This works (without prompt): ssh xxx.xxx.xxx.xxx This command hangs with no output for the first minute: addprocs([usern...@xxx.xxx.xxx.xxx]) Then after one minute, the following message appears: Master process (id 1) could not connect within 60.0 seconds This command works: addprocs([username@mycurrenthostname]) Thanks in advance for your help! John
Re: [julia-users] code review: my first outer constructor :)
Hi Gabriel, thanks a lot for your reply! (1) I use the WaveFunction type inside the Spectrum type, which as you can see is a collection of wavefunctions. For a typical usage case, have a look at http://nbviewer.ipython.org/github/berceanu/topo-photon/blob/master/anc/exploratory.ipynb I am not quite sure what you mean with defining a methods for the int attribute. (2) As you can see, the gauge is an attribute of the Spectrum type, which is what I use in practice so I didn't see much point in storing it inside every wavefunction. Do you have a suggestion for a better implementation? In fact there are only these 2 gauge choices, landau and symmetric. (3) P should be a Vector of length N^2, and I thought that declaring N to be an int means implicit conversion as well - is that not so? (4) genspmat generates the (sparse) matrix of the Hamiltonian, so countnonzeros() simply counts beforehand how many nonzero elements there will be in this sparse matrix. If you think of an NxN 2D lattice, countnonzeros() simply counts the number of neighbours of each site (4 for a center site, 3 for an edge site and 2 for a corner site). What do you mean by irrelevant in this case? Are you refering to the gauge parameter? On Sunday, April 12, 2015 at 11:28:48 PM UTC+2, Gabriel Mitchell wrote: Hi Andrei. I am not really Julia expert, but I do have a couple of high level questions about your code, which might help anyone that feels inclined to contribute a review. (1) Why have you made the WaveFunction type in the first place? In particular, is there any reason that you just don't use alias Vector{Complex{Float64}} and define a methods for the int attribute? (2) the outer construction seems to allow for the option for different gauges (which, in my unsophisticated mindset, I think of as alternative parameterizations). Since different gauge choices in some sense imply different semantics for the values in Psi (although presumably not other function invariant to the gauge choice), it would seem that the user (or a function naive to the gauge choice) would want a means to inspect this change in the object. In other words why is the gauge not an attribute of the WaveFunction object, assuming you actually want this type? A related question would be how many different gauges choices does one expect the user to want to use. Just these two? These two plus a few more? All of them? (3) Line 17 asserts that N is an integer, but sqrt(length(P)) could be non-integral. (4) I don't really understand what is going on with countnonzeros, but maybe a pattern matching syntax ala Match.jl could help to make a more declarative version of this function? As a side note, I think the problem of how to describe and take advantage of irrelevant degrees of freedom in numerical computations is pretty interesting and certainly has applications in all kids of fields, so it would be cool if you had some ideas about how to systematically approach this problem. On Sunday, April 12, 2015 at 10:06:10 PM UTC+2, Andrei Berceanu wrote: Hi Mauro, I realised after posting this that I should have been much more specific. I apologise for that! Anyway, thanks for you reply. Not sure what you mean by OO programming, as I thought Julia uses multiple dispatch and not OO. PS: You're the first person I see that also uses runbox, that makes us email brothers I guess :p On Sunday, April 12, 2015 at 8:29:44 PM UTC+2, Mauro wrote: Hi Andrei, just a general note, unless someone is actually interested in using your code, a code review might be too much to ask of people. Thus the lack in responses. See: https://groups.google.com/forum/#!searchin/julia-users/karpinski$20reivew/julia-users/C5cVjAuGA8U/HLV5rAjIuLMJ The way to get most feedback, is to condense your problem into a small snippet which can just be run with copy-paste (you're gists fails there) and ask something specific. An exception seem to be questions of the kind I ported this from C/Fortran/... and it is 100x slower, where did I go wrong, which seem to regularly attract much feedback. I didn't look into the code in detail: It's the first time I try to use Julia constructors properly (or improperly?!) so I need your opinion on a couple of points. Use of constructors seem to be fine. 1. On Julia's IRC channel I was told that using AbstractArray instead of e.g. Matrix/Vector might yield a performance boost - is that the case? No, I don't think so. Did you read the Performance section of the manual? There it also tells you how to profile your code. 2. Can you spot any major performance killers in my code? 3. Coming from Python, I am used to things like enumerate etc, but perhaps that is not very Julian? :) So this last aspect concerns more the coding style, I guess. Using enumerate is totally fine. Just don't do object-oriented programming. Looks all good
Re: [julia-users] Julia gets a mention in phys.org
very interesting! 2015-04-13 13:30 GMT-03:00 René Donner li...@donner.at: You can find more information on Picture here: http://mrkulk.github.io/www_cvpr15/ Am 13.04.2015 um 18:24 schrieb Seth catch...@bromberger.com: http://phys.org/news/2015-04-probabilistic-lines-code-thousands.html Joining Kulkarni on the paper are his adviser, professor of brain and cognitive sciences Josh Tenenbaum; Vikash Mansinghka, a research scientist in MIT's Department of Brain and Cognitive Sciences; and Pushmeet Kohli of Microsoft Research Cambridge. For their experiments, they created a probabilistic programming language they call Picture, which is an extension of Julia, another language developed at MIT.
[julia-users] Re: pre-allocation for sparse matrices
Yes but so how do I update it inplace? On Monday, April 13, 2015 at 5:37:50 PM UTC+2, Kristoffer Carlsson wrote: If you have the same sparse structures in your matrices then it should be sufficient to to the IJV - CSC sparse matrix conversion only once and then update A in place instead of generating a new IJV and converting to CSC again and again.
Re: [julia-users] Is the official name of the language Julia or julia?
Thanks for the additional info! On Mon, Apr 13, 2015 at 10:45 AM, cormull...@mac.com wrote: About that font: Twitter discussion with Muthu Nedumaran: https://twitter.com/typographica/status/572304422610452480 FontsInUse entry: http://fontsinuse.com/typefaces/38527/mn-latin
[julia-users] Method fieldnames not defined
Hi, I would like to find the field of SolverResults{Float64}. So according to the manual http://julia.readthedocs.org/en/latest/manual/types/, I just need to use the method fieldnames but unfortunately it is not defined. What can I do? Thank you, Antoine
[julia-users] Re: Error using pcolor
The same code works fine on my machine. Maybe try running Pkg.update()?
[julia-users] Julia gets a mention in phys.org
http://phys.org/news/2015-04-probabilistic-lines-code-thousands.html Joining Kulkarni on the paper are his adviser, professor of brain and cognitive sciences Josh Tenenbaum; Vikash Mansinghka, a research scientist in MIT's Department of Brain and Cognitive Sciences; and Pushmeet Kohli of Microsoft Research Cambridge. For their experiments, they created a probabilistic programming language they call Picture, which is an extension of Julia, another language developed at MIT.
Re: [julia-users] Is the official name of the language Julia or julia?
The only remaining question is the significance of the four colored circles in the logo... :) Where's a symbologist when you need one?
Re: [julia-users] string processing after readdlm
No need to apologize – I was just trying to figure out if there was supposed to be an image there or something. On Mon, Apr 13, 2015 at 11:58 AM, JKpie fablekil...@gmail.com wrote: Thank you very much Mauro and Peter, hdf5 and JLD is a good solution. I just didn't know if it was a standard behavior of writedlm and readdlm functions. Stefan: I do apologize for the white space in the massage. It was copy-pasted output from Juno console and I have noticed that the space appeared when the message was posted. thank you once again for quick answer, Jakub On Monday, April 13, 2015 at 5:13:20 PM UTC+2, Peter Simon wrote: Also, if you don't need a .csv file, consider using the HDF5 and JLD packages which preserve the types of stored values: julia a = [1.,2.,3.,4] 4-element Array{Float64,1}: 1.0 2.0 3.0 4..0 julia b = Vector{Float64}[copy(a),2*copy(a),3*copy(a)] 3-element Array{Array{Float64,1},1}: [1.0,2.0,3.0,4.0] [2.0,4.0,6.0,8.0] [3.0,6.0,9.0,12.0] julia using HDF5, JLD julia save(myfile.jld,b,b) julia c = load(myfile.jld,b) 3-element Array{Array{Float64,1},1}: [1.0,2.0,3.0,4.0] [2.0,4.0,6.0,8.0] [3.0,6.0,9.0,12.0]
Re: [julia-users] string processing after readdlm
Also, if you don't need a .csv file, consider using the HDF5 and JLD packages which preserve the types of stored values: julia a = [1.,2.,3.,4] 4-element Array{Float64,1}: 1.0 2.0 3.0 4..0 julia b = Vector{Float64}[copy(a),2*copy(a),3*copy(a)] 3-element Array{Array{Float64,1},1}: [1.0,2.0,3.0,4.0] [2.0,4.0,6.0,8.0] [3.0,6.0,9.0,12.0] julia using HDF5, JLD julia save(myfile.jld,b,b) julia c = load(myfile.jld,b) 3-element Array{Array{Float64,1},1}: [1.0,2.0,3.0,4.0] [2.0,4.0,6.0,8.0] [3.0,6.0,9.0,12.0]
Re: [julia-users] Problem with addprocs
Can you try whether using tunnel = true helps? http://docs.julialang.org/en/release-0.3/stdlib/parallel/#Base.addprocs When tunnel==false (the default), the workers need to be able to see each other directly, which will not work through firewalls which allow only ssh etc. Am 13.04.2015 um 12:10 schrieb John john.d.schul...@gmail.com: I'm unable to add a remote instance using addprocs (or a machine file). This works (without prompt): ssh xxx.xxx.xxx.xxx This command hangs with no output for the first minute: addprocs([usern...@xxx.xxx.xxx.xxx]) Then after one minute, the following message appears: Master process (id 1) could not connect within 60.0 seconds This command works: addprocs([username@mycurrenthostname]) Thanks in advance for your help! John
[julia-users] Re: pre-allocation for sparse matrices
If you have the same sparse structures in your matrices then it should be sufficient to to the IJV - CSC sparse matrix conversion only once and then update A in place instead of generating a new IJV and converting to CSC again and again.
Re: [julia-users] Method fieldnames not defined
You are probably on Julia 0.3.x in which this function was called `names`. You can either use `names` or use the Compat package which lets you use names from the future. On Mon, Apr 13, 2015 at 11:37 AM, 'Antoine Messager' via julia-users julia-users@googlegroups.com wrote: Hi, I would like to find the field of SolverResults{Float64}. So according to the manual http://julia.readthedocs.org/en/latest/manual/types/, I just need to use the method fieldnames but unfortunately it is not defined. What can I do? Thank you, Antoine