Re: [julia-users] Initializing a SharedArray Memory Error
After your gc() it should be able to be unmapped, see https://github.com/JuliaLang/julia/blob/f3c355115ab02868ac644a5561b788fc16738443/base/mmap.jl#L113 My guess is something in the parallel architecture is holding a reference. Have you tried going at this systematically from the internal representation of the SharedArray? For example, I might consider trying to put! new stuff in zeroMatrix.refs: for i = 1:length(zeroMatrix.refs) put!(zeroMatrix.refs[i], 1) end before calling gc(). I don't know if this will work, but it's where I'd start experimenting. If you can fix this, please do submit a pull request. Best, --Tim On Tuesday, December 09, 2014 08:06:10 PM ele...@gmail.com wrote: On Wednesday, December 10, 2014 12:28:29 PM UTC+10, benFranklin wrote: I've made a small example of the memory problems I've been running into. I can't find a way to deallocate a SharedArray, Someone more expert might find it, but I can't see anywhere that the mmapped memory is unmapped. if the code below runs once, it means the computer has enough memory to run this. If I can properly deallocate the memory I should be able to do it again, however, I run out of memory. Am I misunderstanding something about garbage collection in Julia? Thanks for your attention Code: @everywhere nQ = 60 @everywhere function inF(x::SharedArray,nQ::Int64) number = myid()-1; targetLength = nQ*nQ*3 startN = floor((number-1)*targetLength/nworkers()) + 1 endN = floor(number*targetLength/nworkers()) myIndexes = int64(startN:endN) for j in myIndexes inds = ind2sub((nQ,nQ,nQ),j) x[inds[1],inds[2],inds[3],:,:,:] = rand(nQ,nQ,nQ) end end while true zeroMatrix = SharedArray(Float64,(nQ,nQ,3,nQ,nQ,nQ),pids=workers(), init = x-inF(x,nQ)) println(ran!) @everywhere zeroMatrix = 1 @everywhere gc() end On Monday, 8 December 2014 23:43:03 UTC-5, Isaiah wrote: Hopefully you will get an answer on pmap from someone more familiar with the parallel stuff, but: have you tried splitting the init step? (see the example in the manual for how to init an array in chunks done by different workers). Just guessing though: I'm not sure if/how those will be serialized if each worker is contending for the whole array. On Fri, Dec 5, 2014 at 4:23 PM, benFranklin gca...@gmail.com wrote: Hi all, I'm trying to figure out how to best initialize a SharedArray, using a C function to fill it up that computes a huge matrix in parts, and all comments are appreciated. To summarise: Is A, making an empty shared array, computing the matrix in parallel using pmap and then filling it up serially, better than using B, computing in parallel and storing in one step by using an init function in the SharedArray declaration? The difference tends to be that B uses a lot more memory, each process using the exact same amount of memory. However it is much faster than A, as the copy step takes longer than the computation, but in A most of the memory usage is in one process, using less memory overall. Any tips on how to do this better? Also, this pmap is how I'm handling more complex paralellizations in Julia. Any comments on that approach? Thanks a lot! Best, Ben CODE A: Is this, making an empty shared array, computing the matrix in parallel and then filling it up serially: function findZeroDividends(model::ModelPrivate) nW = length(model.vW) nZ = length(model.vZ) nK = length(model.vK) nQ = length(model.vQ) zeroMatrix = SharedArray(Float64,(nW,nZ,nK,nQ,nQ,nQ),pids=workers()) input = [stateFindZeroK(w,z,k,model) for w in 1:nW, z in 1:nZ, k in 1:nK]; results = pmap(findZeroInC,input); for w in 1:nW for z in 1:nZ for k in 1:nK zeroMatrix[w,z,k,:,:,:] = results[w + nW*((z-1) + nZ*(k-1))] end end end return zeroMatrix end ___ CODE B: Better than these two: function start(x::SharedArray,nW::Int64,nZ::Int64,nK::Int64,model::ModelPrivate) for j in myid()-1:nworkers():(nW*nZ*nK) inds = ind2sub((nW,nZ,nK),j) x[inds[1],inds[2],inds[3],:,:,:] =findZeroInC(stateFindZeroK(inds[1],inds[2],inds[3],model)) end x end function findZeroDividendsSmart(model::ModelPrivate) nW = length(model.vW) nZ = length(model.vZ) nK = length(model.vK) nQ = length(model.vQ) #input = [stateFindZeroK(w,z,k,model) for w in 1:nW, z in 1:nZ, k in 1:nK]; #results = pmap(findZeroInC,input); zeroMatrix = SharedArray(Float64,(nW,nZ,nK,nQ,nQ,nQ),pids=workers(), init = x-start(x,nW,nZ,nK,model) ) return zeroMatrix end The C function being called is inside this wrapper and returns the pointer to double *capitalChoices = (double *)malloc(sizeof(double)*nQ*nQ*nQ); function findZeroInC(state::stateFindZeroK) w = state.wealth z = state.z k =
Re: [julia-users] Re: home page content
I like the Haskell one better than the Rust one. --Tim On Tuesday, December 09, 2014 11:14:41 PM Valentin Churavy wrote: An other nice example might be the new haskell homepage http://new-www.haskell.org/ For the runnable part. Maybe we could use tmpnb/juliabox to host an example notebook. We should probably use a docker image with an userimages otherwise the attention span will be over before Gadfly is loaded. Does this work for more than 10 minutes? https://cfa4733.tmpnb.org/user-C6qXAatonjbQ/notebooks/Julia%20Test.ipynb#? On Wednesday, 10 December 2014 07:16:42 UTC+1, cdm wrote: re tight code ... S. Danisch's code length v. speed plot may well be deserving of some real esate: https://lh3.googleusercontent.com/-7IPcrjXuxFY/VICwQ3TrgRI/JV0/_Hm DWZiBrXQ/s1600/benchmarks.png awesome. cdm On Tuesday, December 9, 2014 9:09:03 PM UTC-8, Stefan Karpinski wrote: On Tue, Dec 9, 2014 at 6:43 PM, Leah Hanson astri...@gmail.com wrote: I don't know if you want to encourage different styles, but seeing examples of Python like, c like, and functional-ish ways of writing Julia would be a way to show off the variety of things you can do. I really this idea. Having a grid of four code examples with different styles – Pythonic/Matlabish, C-like, functional and Julian (i.e. with types and multiple dispatch). Now we just need to come up with good examples. Another thing I wonder if it would be good to highlight is how tight the code generated for simple, high-level Julia code is. Maybe not on the main page though but on the about page.
Re: [julia-users] Re: How fast reduce the Vector range of values? (Recode for smaler numbers)
And how to do it if the J is an array rather than a vector? So that
was also Jcodes array of the same size as J?
julia J
1557211x2 Array{Int64,2}:
930070 930070
1475172 1475172
....
21474836496 21474836496
4296445417 4296445417
Paul
W dniu 2014-12-04 o 19:03, Steven G. Johnson pisze:
It sounds like you have an array J and you want to map each element of
J to a unique integer in 1:N for N as small as possible? This will do it:
d = (Int=eltype(J))[j = i for (i,j) in enumerate(unique(J))]
Jcodes = [d[j] for j in J]
Here, d is a dictionary mapping integers in 1:N to the corresponding
values in J, and Jcodes is the re-coded array.
[julia-users] Re: home page content
Yes some plots examples would be great to improve the applicability of Julia. Issue has been discussed by Steven G. Johnson on: Dec 3 https://github.com/gizmaa/Julia_Examples/issues/1 Also a wonderful notebook in https://gist.github.com/gizmaa/7214002. Anyway THANKS for the effort. - G. On Tuesday, December 9, 2014 11:23:26 PM UTC+1, Stefan Karpinski wrote: We're looking to redesign the JuliaLang.org home page and try to give it a little more focus than it currently has. Which raises the question of what to focus on. We could certainly have better code examples and maybe highlight features of the language and its ecosystem better. What do people think we should include?
Re: [julia-users] Re: home page content
I agree that displaying runnable code widgets are useless, but showing the good integration with Jupyter/IPython via juliabox/tmpnb/SAGE is not. It would enable to demonstrated people features of Julia without having them actually installing yet another programming environment, thus reducing the barrier of entry. On Wednesday, 10 December 2014 08:20:48 UTC+1, Jake Bolewski wrote: Am I the only one who thinks these runnable code widgets are totally useless? I'm curious as to how users interact with them in the real world. I bet 99% of them either ignore it or just press the button and see the default output. The ones who probably interact with it the most are going to be the same users that are going to download and run the language anyway. They displace huge amounts of real estate for basically no practical value. To me the landing page to a programming language is really a nice backdrop for four things, A giant button where I can go download what I want because I'm lazy and just click on the top google hit. Another prominent link to the Julia package ecosystem because I'm lazy and typing julia AND packages is way too much work (haskell did a survey a while back, if I remember correctly a majority of users to the front page fell under this category). In addition, enough background information to get people to click on the manual and a nice community / development activity section so I can see that things are happening. Please, please don't make me scroll past a huge useless web 2.0 header to get to what I actually want (again, lazy). I like the Racket, Haskell, and OCaml websites as I think they are utilitarian but actually useful. I agree that the Rust site is a bit too minimalist. I absolutely hate the python website. The R website is just laughably bad. Altogether, I don't think PL's set a high bar in this regard. -Jake On Wednesday, December 10, 2014 1:16:42 AM UTC-5, cdm wrote: re tight code ... S. Danisch's code length v. speed plot may well be deserving of some real esate: https://lh3.googleusercontent.com/-7IPcrjXuxFY/VICwQ3TrgRI/JV0/_HmDWZiBrXQ/s1600/benchmarks.png awesome. cdm On Tuesday, December 9, 2014 9:09:03 PM UTC-8, Stefan Karpinski wrote: On Tue, Dec 9, 2014 at 6:43 PM, Leah Hanson astri...@gmail.com wrote: I don't know if you want to encourage different styles, but seeing examples of Python like, c like, and functional-ish ways of writing Julia would be a way to show off the variety of things you can do. I really this idea. Having a grid of four code examples with different styles – Pythonic/Matlabish, C-like, functional and Julian (i.e. with types and multiple dispatch). Now we just need to come up with good examples. Another thing I wonder if it would be good to highlight is how tight the code generated for simple, high-level Julia code is. Maybe not on the main page though but on the about page.
Re: [julia-users] Re: home page content
Yeah, I don't think we need runnable widgets on the main page. A better option would be to have a Run in JuliaBox link which could start a new session. As far as code samples go, the ideal ones should: * be around 10 lines or so * demonstrate the key features of Julia (i.e. all the things under A Summary of Features on the current homepage). Haskell's site does this well. * do something that is cool and/or useful (some of the Wolfram Alpha examples are quite neat, albeit gimmicky). We should definitely have a prominent link to the packages: a common critique of new languages is that they lack the package ecosystem of Python/R/etc. On Wednesday, 10 December 2014 11:28:24 UTC, Valentin Churavy wrote: I agree that displaying runnable code widgets are useless, but showing the good integration with Jupyter/IPython via juliabox/tmpnb/SAGE is not. It would enable to demonstrated people features of Julia without having them actually installing yet another programming environment, thus reducing the barrier of entry. On Wednesday, 10 December 2014 08:20:48 UTC+1, Jake Bolewski wrote: Am I the only one who thinks these runnable code widgets are totally useless? I'm curious as to how users interact with them in the real world. I bet 99% of them either ignore it or just press the button and see the default output. The ones who probably interact with it the most are going to be the same users that are going to download and run the language anyway. They displace huge amounts of real estate for basically no practical value. To me the landing page to a programming language is really a nice backdrop for four things, A giant button where I can go download what I want because I'm lazy and just click on the top google hit. Another prominent link to the Julia package ecosystem because I'm lazy and typing julia AND packages is way too much work (haskell did a survey a while back, if I remember correctly a majority of users to the front page fell under this category). In addition, enough background information to get people to click on the manual and a nice community / development activity section so I can see that things are happening. Please, please don't make me scroll past a huge useless web 2.0 header to get to what I actually want (again, lazy). I like the Racket, Haskell, and OCaml websites as I think they are utilitarian but actually useful. I agree that the Rust site is a bit too minimalist. I absolutely hate the python website. The R website is just laughably bad. Altogether, I don't think PL's set a high bar in this regard. -Jake On Wednesday, December 10, 2014 1:16:42 AM UTC-5, cdm wrote: re tight code ... S. Danisch's code length v. speed plot may well be deserving of some real esate: https://lh3.googleusercontent.com/-7IPcrjXuxFY/VICwQ3TrgRI/JV0/_HmDWZiBrXQ/s1600/benchmarks.png awesome. cdm On Tuesday, December 9, 2014 9:09:03 PM UTC-8, Stefan Karpinski wrote: On Tue, Dec 9, 2014 at 6:43 PM, Leah Hanson astri...@gmail.com wrote: I don't know if you want to encourage different styles, but seeing examples of Python like, c like, and functional-ish ways of writing Julia would be a way to show off the variety of things you can do. I really this idea. Having a grid of four code examples with different styles – Pythonic/Matlabish, C-like, functional and Julian (i.e. with types and multiple dispatch). Now we just need to come up with good examples. Another thing I wonder if it would be good to highlight is how tight the code generated for simple, high-level Julia code is. Maybe not on the main page though but on the about page.
Re: [julia-users] Re: home page content
My two cents: - Plots are great, but please make them readable for the colourblind - use triangles/squares/etc in addition to circles, use lightness and saturation on top of hue. I can't make sense of the linked examples so far. - Code widgets are probably not that interesting outside of tutorial pages, but an online sandbox is fantastic for discussing short snippets of code. For example, play.golang.org is basically a runnable gist, and it is constantly used in online discussions like *how to do x* or *why doesn't y compile*?
Re: [julia-users] Re: home page content
One thing, (probably not on the front page) would be online access to the latest Git version for those of us limited to packaged versions.
[julia-users] Re: DataFrames and NamedArrays: are they suitable for heavier computations ?
Hi,
Thanks for the explanation. Suppose I have a named array X with 3 columns
x1, x2 and x3 and I do prod(X, 2). Will the resulting array (a single
columns in this case) have a sensible name like x1x2x3 ? Or more
generally, how are these new names generated and for which operations ?
For some operators NamedArrays tries to generate sensible names,
including prod():
julia n = NamedArray(rand(2,3), ([:a, :b], [:x1, :x2, :x3]), (rows,
cols))
2x3 NamedArray{Float64,2,Array{Float64,2},(Dict{Symbol,Int64},Dict{Symbol,
Int64})}
rows \ cols x1 x2 x3
a 0.712609 0.607843 0.18794
b 0.208052 0.4409 0.282238
julia prod(n, 2)
2x1 NamedArray{Float64,2,Array{Float64,2},(Dict{Symbol,Int64},Dict{
ASCIIString,Int64})}
rows \ cols prod(cols)
a 0.0814106
b 0.0258897
So the column in this case is called prod(cols). Note that we started
with symbols as names for columns indices in this example but prod()
normalizes this to ASCIIStrings.
I've worked hard to make the index type a free choice, but some
automatically generated names default to Strings.
You can find the list of operations for which NamedArrays tries to give
sensible names here
https://github.com/davidavdav/NamedArrays/blob/master/src/changingnames.jl,
in the source code. I am open for suggestions and pull requests for more
functions or other names.
As a result of your original question I am now implementing the matrix
operation support more thoroughly, basically following the Julia manual for
possible matrix operations. Some take a bit of effort to implement, for
some I need to think about sensible names for the dimensions. Expect an
update within the next week or so.
Cheers,
---david
Hi David,
Thanks for the example and a link to the source file. I found it very
educational and useful. My original motivation was to check what is (or
cloud be) a best practice in Julia for creating e.g. all interactions of
say 3 variables and have sensible names filled in automatically.
NamedArrays seems to be a good start.
In addition, is there any documentation for Symbol and Dictionary types ? I
found very little about it in Julia manual.
Best regards,
Jan
[julia-users] Installing packages system-wide
Hello, I'd like to ask how to install Julia packages system-wide so that users do not have to install packages individually on their own but rather just once by an admin. Thanks, Jan
Re: [julia-users] Re: home page content
Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
[julia-users] serialize a dictionary gives an error
I have a dictionary that like:
Dict{ASCIIString,Array{Array{Float64,2},1}} with 83 entries:
It is generated by reading 350 mb of dlm files. I would like to save so it is
faster reloadable, currently it takes 60s to load from the dlm files. I have
tried to serialize to save the data.
open(test.bin,w) do file
serialize(file,data_bolt)
end
data_new = deserialize(open(test.bin))
`convert` has no method matching convert(::Type{Int64...}, ::Int64)
while loading In[22], in expression starting on line 2
in convert at base.jl:13
in convert at base.jl:21
in deserialize at serialize.jl:447
But when I try to deserialize I get an error. Do I do something wrong?
I have tried both MAT and HDF5 and they do not seem to have a convenient way of
saving my data structure. I hope some of you guys can help, thanks in advance.
Best regards Daniel Høegh
Re: [julia-users] Re: home page content
From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwborch...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
Am I the only one who thinks these runnable code widgets are totally useless? I'm curious as to how users interact with them in the real world. I bet 99% of them either ignore it or just press the button and see the default output. The ones who probably interact with it the most are going to be the same users that are going to download and run the language anyway. They displace huge amounts of real estate for basically no practical value. To me the landing page to a programming language is really a nice backdrop for four things, A giant button where I can go download what I want because I'm lazy and just click on the top google hit. Another prominent link to the Julia package ecosystem because I'm lazy and typing julia AND packages is way too much work (haskell did a survey a while back, if I remember correctly a majority of users to the front page fell under this category). In addition, enough background information to get people to click on the manual and a nice community / development activity section so I can see that things are happening. Please, please don't make me scroll past a huge useless web 2.0 header to get to what I actually want (again, lazy). I like the Racket, Haskell, and OCaml websites as I think they are utilitarian but actually useful. I agree that the Rust site is a bit too minimalist. I absolutely hate the python website. The R website is just laughably bad. Altogether, I don't think PL's set a high bar in this regard. -Jake No you're not the only one ;) Unfortunately http://forio.com/julia/repl/ doesn't work anymore, in my opinion it was excellent and a link to it on the front page would be enough. For everyone who doesn't already know, I've uploaded a preview of my homepage revamp here: http://julialang.herokuapp.com/ If you like I can use that page for a sandbox for ideas that come up during this discussion.
[julia-users] BinDeps fails to find a built dependency (but only on Travis)
I'm in the process of wrapping part of an important tool for Radio Astronomers: CasaCore (relevant branch: https://github.com/mweastwood/CasaCore.jl/tree/maketraviswork). Unfortunately CasaCore is a C++ library, so I have first written a C interface that needs to be compiled. I am using BinDeps to download and compile CasaCore (if it is not already present on the system), and then build the C wrapper. On my own machine, this works perfectly (although I am cheating somewhat because CasaCore is already built on my machine): julia Pkg.build(CasaCore) INFO: Building CasaCore INFO: Attempting to Create directory /home/mweastwood/.julia/CasaCore/deps/downloads INFO: Downloading file ftp://ftp.atnf.csiro.au/pub/software/casacore/casacore-1.7.0.tar.bz2 % Total% Received % Xferd Average Speed TimeTime Time Current Dload Upload Total SpentLeft Speed 100 6019k 100 6019k0 0 414k 0 0:00:14 0:00:14 --:--:-- 925k INFO: Done downloading file ftp://ftp.atnf.csiro.au/pub/software/casacore/casacore-1.7.0.tar.bz2 INFO: Attempting to Create directory /home/mweastwood/.julia/CasaCore/deps/src INFO: Directory /home/mweastwood/.julia/CasaCore/deps/src already created INFO: Attempting to Create directory /home/mweastwood/.julia/CasaCore/deps INFO: Directory /home/mweastwood/.julia/CasaCore/deps already created INFO: Attempting to Create directory /home/mweastwood/.julia/CasaCore/deps/src/casacore-1.7.0 INFO: Attempting to Create directory /home/mweastwood/.julia/CasaCore/deps/builds/casacorewrapper INFO: Directory /home/mweastwood/.julia/CasaCore/deps/builds/casacorewrapper already created INFO: Changing Directory to /home/mweastwood/.julia/CasaCore/deps/builds/casacorewrapper g++ -c -Wall -Werror -fpic ../../src/casacorewrapper/containers.cpp -I../../src/casacore-1.7.0 g++ -c -Wall -Werror -fpic ../../src/casacorewrapper/tableswrapper.cpp -I../../src/casacore-1.7.0 g++ -c -Wall -Werror -fpic ../../src/casacorewrapper/measureswrapper.cpp -I../../src/casacore-1.7.0 g++ -shared -o libcasacorewrapper.so containers.o tableswrapper.o measureswrapper.o -L../../usr/lib -lcasa_tables -lcasa_measures mkdir -p ../../usr/lib cp libcasacorewrapper.so ../../usr/lib julia Pkg.test(CasaCore) INFO: Testing CasaCore INFO: CasaCore tests passed INFO: No packages to install, update or remove However, on Travis, I'm at my wits end trying to figure out why this doesn't work. See the build log here (note, scroll right to the bottom, CasaCore is a huge dependency that takes a while to compile): https://travis-ci.org/mweastwood/CasaCore.jl/builds/43496822 Confusingly, both CasaCore and my C wrapper seem to build successfully. Indeed, checking the deps/usr/lib directory on Travis indicates that all the appropriate shared libraries exist, but BinDeps fails with Provider BinDeps.BuildProcess failed to satisfy dependency libcasacorewrapper while loading /home/travis/.julia/v0.4/CasaCore/deps/build.jl, in expression starting on line 61 I would really appreciate any guidance on how to solve this problem. -- Michael
[julia-users] Julia on android
Hi all Do anyone knows how I can run a julia program in android platform? Is there any support given? Any help is appreciated!! Thanks and Regards googler
Re: [julia-users] Installing packages system-wide
There's a Julia variable called LOAD_PATH that is arranged to point at two
system directories under your julia installation. E.g.:
julia LOAD_PATH
2-element Array{Union(ASCIIString,UTF8String),1}:
/opt/julia-0.3.3/usr/local/share/julia/site/v0.3
/opt/julia-0.3.3/usr/share/julia/site/v0.3
If you install packages under either of those directories, then everyone
using that Julia will see them. One way to do this is to run julia as a
user who can write to those directories after doing `export
JULIA_PKGDIR=/opt/julia-0.3.3/usr/share/julia/site` in the shell. That way
Julia will use that as it's package directory and normal package commands
will allow you to install packages for everyone. Or you can just copy your
installed packages and change the ownership and permissions so that
everyone can access the files.
On Wed, Dec 10, 2014 at 8:16 AM, Ján Dolinský jan.dolin...@2bridgz.com
wrote:
Hello,
I'd like to ask how to install Julia packages system-wide so that users do
not have to install packages individually on their own but rather just once
by an admin.
Thanks,
Jan
[julia-users] Re: serialize a dictionary gives an error
H seems to work fine for me. Does the program in this gist
https://gist.github.com/Sean1708/92d9fddb88c7d9a513ea work for you?
On Wednesday, 10 December 2014 13:30:38 UTC, Daniel Høegh wrote:
I have a dictionary that like:
Dict{ASCIIString,Array{Array{Float64,2},1}} with 83 entries:
It is generated by reading 350 mb of dlm files. I would like to save so it
is faster reloadable, currently it takes 60s to load from the dlm files. I
have tried to serialize to save the data.
open(test.bin,w) do file
serialize(file,data_bolt)
end
data_new = deserialize(open(test.bin))
`convert` has no method matching convert(::Type{Int64...}, ::Int64)
while loading In[22], in expression starting on line 2
in convert at base.jl:13
in convert at base.jl:21
in deserialize at serialize.jl:447
But when I try to deserialize I get an error. Do I do something wrong?
I have tried both MAT and HDF5 and they do not seem to have a convenient
way of saving my data structure. I hope some of you guys can help, thanks
in advance.
Best regards Daniel Høegh
[julia-users] Re: serialize a dictionary gives an error
Yes it works when I remove the 0.4 syntax:) But when you put using PyPlot in the top it does not work:( It's the convertalypse. at https://github.com/JuliaLang/julia/issues/8631
Re: [julia-users] Re: serialize a dictionary gives an error
If you do a Pkg.update(), there's a good chance Gunnar's bandaid in Color will fix this for you. --Tim On Wednesday, December 10, 2014 08:23:02 AM Daniel Høegh wrote: Yes it works when I remove the 0.4 syntax:) But when you put using PyPlot in the top it does not work:( It's the convertalypse. at https://github.com/JuliaLang/julia/issues/8631
[julia-users] Re: serialize a dictionary gives an error
Sorry, it's just too natural nowadays :P On Wednesday, 10 December 2014 16:23:02 UTC, Daniel Høegh wrote: Yes it works when I remove the 0.4 syntax:) But when you put using PyPlot in the top it does not work:( It's the convertalypse. at https://github.com/JuliaLang/julia/issues/8631
[julia-users] Re: serialize a dictionary gives an error
Yes it works when I remove the 0.4 syntax:) Sorry it's just too natural for me nowadays :P On Wednesday, 10 December 2014 16:23:02 UTC, Daniel Høegh wrote: Yes it works when I remove the 0.4 syntax:) But when you put using PyPlot in the top it does not work:( It's the convertalypse. at https://github.com/JuliaLang/julia/issues/8631
Re: [julia-users] Re: serialize a dictionary gives an error
Thanks Tim it works. I really hope someone catch the root of the convertalypse soon:)
Re: [julia-users] Re: How fast reduce the Vector range of values? (Recode for smaler numbers)
Vector{T} is just a typealias for Array{T, 1} so it's still an array but
limited to one dimension. Your problem can be solved with
convert(Array{Int, 2}, Jcodes)
or equivalently
convert(Matrix{Int}, Jcodes)
On Wednesday, 10 December 2014 11:09:55 UTC, paul analyst wrote:
And how to do it if the J is an array rather than a vector? So that was
also Jcodes array of the same size as J?
julia J
1557211x2 Array{Int64,2}:
930070 930070
1475172 1475172
....
21474836496 21474836496
4296445417 4296445417
Paul
W dniu 2014-12-04 o 19:03, Steven G. Johnson pisze:
It sounds like you have an array J and you want to map each element of J
to a unique integer in 1:N for N as small as possible? This will do it:
d = (Int=eltype(J))[j = i for (i,j) in enumerate(unique(J))]
Jcodes = [d[j] for j in J]
Here, d is a dictionary mapping integers in 1:N to the corresponding
values in J, and Jcodes is the re-coded array.
Re: [julia-users] Re: home page content
One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com javascript: wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
I like the point: Solving P=NP reminds me of rust's * In theory. Rust is a work-in-progress and may do anything it likes up to and including eating your laundry. On Wednesday, 10 December 2014 19:15:05 UTC+1, Christian Peel wrote: One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
We can add a bullet point about Julia not eating your laundry. My point of view is that Julia's pre-1.0 status does not mean that it can do whatever it wants and we have no responsibility. Rather it sends the much milder signal that between now and 1.0 we may change the language and standard library in ways that will require you to change your code. Bugs that cause destructive behavior are no less serious now than later. On Wed, Dec 10, 2014 at 1:25 PM, Valentin Churavy v.chur...@gmail.com wrote: I like the point: Solving P=NP reminds me of rust's * In theory. Rust is a work-in-progress and may do anything it likes up to and including eating your laundry. On Wednesday, 10 December 2014 19:15:05 UTC+1, Christian Peel wrote: One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
On Wed, Dec 10 2014, Christian Peel sanpi...@gmail.com wrote: provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. there was http://thisweekinjulia.github.io/ but it has not been updated since late October. best, Tamas
Re: [julia-users] Re: home page content
As always in Julia (and OSS in general), I think the problem is that there's no labor supply to do most nice things for the community. Everybody would love to see weekly updates. Not many people have both the time and desire to do the work. -- John On Dec 10, 2014, at 10:41 AM, Tamas Papp tkp...@gmail.com wrote: On Wed, Dec 10 2014, Christian Peel sanpi...@gmail.com wrote: provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. there was http://thisweekinjulia.github.io/ but it has not been updated since late October. best, Tamas
Re: [julia-users] Initializing a SharedArray Memory Error
I think you are right about some references not being released yet: If I change the while loop to include you way of replacing every reference, the put! actually never gets executed, it just waits: while true zeroMatrix = SharedArray(Float64,(nQ,nQ,3,nQ,nQ,nQ),pids=workers(), init = x-inF(x,nQ)) println(ran!) for i = 1:length(zeroMatrix.refs) put!(zeroMatrix.refs[i], 1) end @everywhere gc() end ran! Runs once and stalls, after C-c: ^CERROR: interrupt in process_events at /usr/bin/../lib64/julia/sys.so in wait at /usr/bin/../lib64/julia/sys.so (repeats 2 times) in wait_full at /usr/bin/../lib64/julia/sys.so After C-d julia WARNING: Forcibly interrupting busy workers error in running finalizer: InterruptException() error in running finalizer: InterruptException() WARNING: Unable to terminate all workers [...] It seems after the init function not all workers are done. I'll see if there's something weird with that part, but if the SharedArray is being returned, I don't see any reason for this to be so. On Wednesday, 10 December 2014 05:19:55 UTC-5, Tim Holy wrote: After your gc() it should be able to be unmapped, see https://github.com/JuliaLang/julia/blob/f3c355115ab02868ac644a5561b788fc16738443/base/mmap.jl#L113 My guess is something in the parallel architecture is holding a reference. Have you tried going at this systematically from the internal representation of the SharedArray? For example, I might consider trying to put! new stuff in zeroMatrix.refs: for i = 1:length(zeroMatrix.refs) put!(zeroMatrix.refs[i], 1) end before calling gc(). I don't know if this will work, but it's where I'd start experimenting. If you can fix this, please do submit a pull request. Best, --Tim On Tuesday, December 09, 2014 08:06:10 PM ele...@gmail.com javascript: wrote: On Wednesday, December 10, 2014 12:28:29 PM UTC+10, benFranklin wrote: I've made a small example of the memory problems I've been running into. I can't find a way to deallocate a SharedArray, Someone more expert might find it, but I can't see anywhere that the mmapped memory is unmapped. if the code below runs once, it means the computer has enough memory to run this. If I can properly deallocate the memory I should be able to do it again, however, I run out of memory. Am I misunderstanding something about garbage collection in Julia? Thanks for your attention Code: @everywhere nQ = 60 @everywhere function inF(x::SharedArray,nQ::Int64) number = myid()-1; targetLength = nQ*nQ*3 startN = floor((number-1)*targetLength/nworkers()) + 1 endN = floor(number*targetLength/nworkers()) myIndexes = int64(startN:endN) for j in myIndexes inds = ind2sub((nQ,nQ,nQ),j) x[inds[1],inds[2],inds[3],:,:,:] = rand(nQ,nQ,nQ) end end while true zeroMatrix = SharedArray(Float64,(nQ,nQ,3,nQ,nQ,nQ),pids=workers(), init = x-inF(x,nQ)) println(ran!) @everywhere zeroMatrix = 1 @everywhere gc() end On Monday, 8 December 2014 23:43:03 UTC-5, Isaiah wrote: Hopefully you will get an answer on pmap from someone more familiar with the parallel stuff, but: have you tried splitting the init step? (see the example in the manual for how to init an array in chunks done by different workers). Just guessing though: I'm not sure if/how those will be serialized if each worker is contending for the whole array. On Fri, Dec 5, 2014 at 4:23 PM, benFranklin gca...@gmail.com wrote: Hi all, I'm trying to figure out how to best initialize a SharedArray, using a C function to fill it up that computes a huge matrix in parts, and all comments are appreciated. To summarise: Is A, making an empty shared array, computing the matrix in parallel using pmap and then filling it up serially, better than using B, computing in parallel and storing in one step by using an init function in the SharedArray declaration? The difference tends to be that B uses a lot more memory, each process using the exact same amount of memory. However it is much faster than A, as the copy step takes longer than the computation, but in A most of the memory usage is in one process, using less memory overall. Any tips on how to do this better? Also, this pmap is how I'm handling more complex paralellizations in Julia. Any comments on that approach? Thanks a lot! Best, Ben CODE A: Is this, making an empty shared array, computing the matrix in parallel and then filling it up serially: function findZeroDividends(model::ModelPrivate) nW = length(model.vW) nZ = length(model.vZ) nK = length(model.vK) nQ =
[julia-users] Poor parallel performance
I am getting performance degradation after parallelizing the code that is calculating graph centrality. Graph is relatively large, 100K vertices. Single threaded application take approximately 7 minutes. As recommended on julialang site ( http://julia.readthedocs.org/en/latest/manual/parallel-computing/#man-parallel-computing) I adapted code and used pmap api in order to parallelize calculations. I started calculation with 8 processes (julia -p 8 test_parallel_pmap). To my surprise I got 10 fold slow down. Parallel process now take more than hour. I noticed that it take several minutes for parallel process to initialize and starts calculation. Even after all 8 cpus are %100 busy with julia app, calculation is super slow. Attached is julia code: 1) test_parallel_pmap.jl reads grapg from file and starts parallel calculation. 2) centrality_mean.jl calculatse centrality. Code is based on https://gist.github.com/SirVer/3353761 Any suggestion how to improve parallel performance is greatly appreciated. Thanks, Dejan test_parallel_pmap.jl Description: Binary data centrality_mean.jl Description: Binary data
Re: [julia-users] Re: home page content
OK, thanks for the replies; John's reply below makes the situation clear. Chris On Wednesday, December 10, 2014 10:46:03 AM UTC-8, John Myles White wrote: As always in Julia (and OSS in general), I think the problem is that there's no labor supply to do most nice things for the community. Everybody would love to see weekly updates. Not many people have both the time and desire to do the work. -- John On Dec 10, 2014, at 10:41 AM, Tamas Papp tkp...@gmail.com javascript: wrote: On Wed, Dec 10 2014, Christian Peel sanp...@gmail.com javascript: wrote: provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. there was http://thisweekinjulia.github.io/ but it has not been updated since late October. best, Tamas
Re: [julia-users] Re: home page content
It might be nice to have a few examples of workflows that people who use Julia in real life have set up for themselves. On Wed, Dec 10, 2014 at 2:03 PM, Christian Peel sanpi...@gmail.com wrote: OK, thanks for the replies; John's reply below makes the situation clear. Chris On Wednesday, December 10, 2014 10:46:03 AM UTC-8, John Myles White wrote: As always in Julia (and OSS in general), I think the problem is that there's no labor supply to do most nice things for the community. Everybody would love to see weekly updates. Not many people have both the time and desire to do the work. -- John On Dec 10, 2014, at 10:41 AM, Tamas Papp tkp...@gmail.com wrote: On Wed, Dec 10 2014, Christian Peel sanp...@gmail.com wrote: provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. there was http://thisweekinjulia.github.io/ but it has not been updated since late October. best, Tamas
[julia-users] Re: home page content
I would really like to see a page along the lines of http://www.mathworks.com/examples/ I've tried to start something like it at http://homepages.warwick.ac.uk/staff/C.Ortner/index.php?page=julia This was aimed mostly at my own research group and some friends and colleagues. Some ideas: * If I want a specific standard problem solved, then I can go look how a competent Julia programmer did it. * They should to be attractive * Could be useful for teaching * Could have examples of 4 different coding paradigms that some people are keen on * Maybe some core Julia group could review submissions rather than letting anybody post notebooks * What I am unsure about is whether notebooks, once posted, should become open source or not. * Finally - I am unsure whether it would be really necessary to make them interactive. But possibly an [export to Julia-box] button? I think such a page will also quickly show where the bottlenecks are in getting Julia to a wider community. Christoph On Tuesday, 9 December 2014 22:23:26 UTC, Stefan Karpinski wrote: We're looking to redesign the JuliaLang.org home page and try to give it a little more focus than it currently has. Which raises the question of what to focus on. We could certainly have better code examples and maybe highlight features of the language and its ecosystem better. What do people think we should include?
Re: [julia-users] Re: home page content
Note that the framework is in place via juliabloggers.com. If someone wanted to pick up this task, but didn't want to dedicate creating a blog, I'm willing to create an author account to post directly. On Wednesday, December 10, 2014 1:46:03 PM UTC-5, John Myles White wrote: As always in Julia (and OSS in general), I think the problem is that there's no labor supply to do most nice things for the community. Everybody would love to see weekly updates. Not many people have both the time and desire to do the work. -- John On Dec 10, 2014, at 10:41 AM, Tamas Papp tkp...@gmail.com javascript: wrote: On Wed, Dec 10 2014, Christian Peel sanp...@gmail.com javascript: wrote: provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. there was http://thisweekinjulia.github.io/ but it has not been updated since late October. best, Tamas
[julia-users] Re: Reviewing a Julia programming book for Packt
I've likewise been contacted to review the book and have agreed. Dan On Thursday, December 4, 2014 6:31:13 PM UTC-5, Wilfred Hughes wrote: I received an email today about being a technical reviewer for a book on Julia! We're currently developing a book titled *Mastering Julia* aiming at building statistical models with linear regressions and analysis of variance (ANOVA) and will be working on probability, probability distributions, and random variables covering data structures such as matrices, lists, factors, and data frames. This book is targeted at Intermediate level developer in statistical languages and one who will be having understanding of Core elements and applications. Would you be interested in acting as reviewer for this book? Now, I enjoy Julia, and I'm happy to help promote the language, but I don't think I'm particularly qualified to be a technical reviewer of a book on Julia programming. I found this thread on julia-dev: https://groups.google.com/forum/#!msg/julia-dev/HrdpknFgdfk/SAVMyyacT_sJ where Packt contacted a large number of folks seeking an author. Has anyone else received something like this? In principle, I'm all in favour of producing promotional or teaching materials, but I'm surprised it lead to me being contacted.
[julia-users] Re: Finite element code in Julia: Curious overhead in .* product
Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you allocate millions of very small short-lived objects and that taxes the GC quite a lot.
[julia-users] Multiple assignment in multidimensional array
Hello!
I have a 4x4 array of zeros.
julia X = zeros(4,4)
4x4 Array{Float64,2}:
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
I have an 2xN array containing indices of elements in X that I want to
assign a new value.
julia ind = [1 1; 2 2; 3 3]
3x2 Array{Int64,2}:
1 1
2 2
3 3
What is the simplest way to assign a value to all elements in X whose
indices are rows in ind? (something like X[ind] = 2.0).
julia X
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Re: [julia-users] Re: home page content
Yeah, that's really a good why Julia. On Wed, Dec 10, 2014 at 2:55 PM, Isaiah Norton isaiah.nor...@gmail.com wrote: I've tried to start something like it at http://homepages.warwick.ac.uk/staff/C.Ortner/index.php?page=julia This was aimed mostly at my own research group and some friends and colleagues. Your Why Julia section is really fantastic. On Wed, Dec 10, 2014 at 2:47 PM, Christoph Ortner christophortn...@gmail.com wrote: I would really like to see a page along the lines of http://www.mathworks.com/examples/ I've tried to start something like it at http://homepages.warwick.ac.uk/staff/C.Ortner/index.php?page=julia This was aimed mostly at my own research group and some friends and colleagues. Some ideas: * If I want a specific standard problem solved, then I can go look how a competent Julia programmer did it. * They should to be attractive * Could be useful for teaching * Could have examples of 4 different coding paradigms that some people are keen on * Maybe some core Julia group could review submissions rather than letting anybody post notebooks * What I am unsure about is whether notebooks, once posted, should become open source or not. * Finally - I am unsure whether it would be really necessary to make them interactive. But possibly an [export to Julia-box] button? I think such a page will also quickly show where the bottlenecks are in getting Julia to a wider community. Christoph On Tuesday, 9 December 2014 22:23:26 UTC, Stefan Karpinski wrote: We're looking to redesign the JuliaLang.org home page and try to give it a little more focus than it currently has. Which raises the question of what to focus on. We could certainly have better code examples and maybe highlight features of the language and its ecosystem better. What do people think we should include?
Re: [julia-users] Multiple assignment in multidimensional array
The classic vectorized Matlab approach is to use sub2ind:
julia X[sub2ind(size(X), ind[:,1], ind[:,2])] = 2
2
julia X
4x4 Array{Float64,2}:
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Of course, in Julia you also have the option of just using a for loop:
for i = 1:size(ind,1)
X[ind[i,1],ind[i,2]] = 2
end
This is more efficient and arguably clearer.
On Wed, Dec 10, 2014 at 3:45 PM, Tudor Berariu tudor.bera...@gmail.com
wrote:
Hello!
I have a 4x4 array of zeros.
julia X = zeros(4,4)
4x4 Array{Float64,2}:
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
I have an 2xN array containing indices of elements in X that I want to
assign a new value.
julia ind = [1 1; 2 2; 3 3]
3x2 Array{Int64,2}:
1 1
2 2
3 3
What is the simplest way to assign a value to all elements in X whose
indices are rows in ind? (something like X[ind] = 2.0).
julia X
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Re: [julia-users] Multiple assignment in multidimensional array
please don't cross-post questions in multiple places – it's the same people
reading and answering in both places:
http://stackoverflow.com/questions/27410076/multiple-assignment-in-multidimensional-array
On Wed, Dec 10, 2014 at 4:06 PM, Stefan Karpinski ste...@karpinski.org
wrote:
The classic vectorized Matlab approach is to use sub2ind:
julia X[sub2ind(size(X), ind[:,1], ind[:,2])] = 2
2
julia X
4x4 Array{Float64,2}:
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Of course, in Julia you also have the option of just using a for loop:
for i = 1:size(ind,1)
X[ind[i,1],ind[i,2]] = 2
end
This is more efficient and arguably clearer.
On Wed, Dec 10, 2014 at 3:45 PM, Tudor Berariu tudor.bera...@gmail.com
wrote:
Hello!
I have a 4x4 array of zeros.
julia X = zeros(4,4)
4x4 Array{Float64,2}:
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
I have an 2xN array containing indices of elements in X that I want to
assign a new value.
julia ind = [1 1; 2 2; 3 3]
3x2 Array{Int64,2}:
1 1
2 2
3 3
What is the simplest way to assign a value to all elements in X whose
indices are rows in ind? (something like X[ind] = 2.0).
julia X
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Re: [julia-users] Multiple assignment in multidimensional array
Thank you for the answer! I'm sorry about cross-posting. It won't happen
again!
On Wednesday, December 10, 2014 11:08:53 PM UTC+2, Stefan Karpinski wrote:
please don't cross-post questions in multiple places – it's the same
people reading and answering in both places:
http://stackoverflow.com/questions/27410076/multiple-assignment-in-multidimensional-array
On Wed, Dec 10, 2014 at 4:06 PM, Stefan Karpinski ste...@karpinski.org
javascript: wrote:
The classic vectorized Matlab approach is to use sub2ind:
julia X[sub2ind(size(X), ind[:,1], ind[:,2])] = 2
2
julia X
4x4 Array{Float64,2}:
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Of course, in Julia you also have the option of just using a for loop:
for i = 1:size(ind,1)
X[ind[i,1],ind[i,2]] = 2
end
This is more efficient and arguably clearer.
On Wed, Dec 10, 2014 at 3:45 PM, Tudor Berariu tudor@gmail.com
javascript: wrote:
Hello!
I have a 4x4 array of zeros.
julia X = zeros(4,4)
4x4 Array{Float64,2}:
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
I have an 2xN array containing indices of elements in X that I want to
assign a new value.
julia ind = [1 1; 2 2; 3 3]
3x2 Array{Int64,2}:
1 1
2 2
3 3
What is the simplest way to assign a value to all elements in X whose
indices are rows in ind? (something like X[ind] = 2.0).
julia X
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Re: [julia-users] Multiple assignment in multidimensional array
No worries, glad I could answer your question.
On Wed, Dec 10, 2014 at 4:20 PM, Tudor Berariu tudor.bera...@gmail.com
wrote:
Thank you for the answer! I'm sorry about cross-posting. It won't happen
again!
On Wednesday, December 10, 2014 11:08:53 PM UTC+2, Stefan Karpinski wrote:
please don't cross-post questions in multiple places – it's the same
people reading and answering in both places: http://stackoverflow.
com/questions/27410076/multiple-assignment-in-multidimensional-array
On Wed, Dec 10, 2014 at 4:06 PM, Stefan Karpinski ste...@karpinski.org
wrote:
The classic vectorized Matlab approach is to use sub2ind:
julia X[sub2ind(size(X), ind[:,1], ind[:,2])] = 2
2
julia X
4x4 Array{Float64,2}:
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
Of course, in Julia you also have the option of just using a for loop:
for i = 1:size(ind,1)
X[ind[i,1],ind[i,2]] = 2
end
This is more efficient and arguably clearer.
On Wed, Dec 10, 2014 at 3:45 PM, Tudor Berariu tudor@gmail.com
wrote:
Hello!
I have a 4x4 array of zeros.
julia X = zeros(4,4)
4x4 Array{Float64,2}:
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
0.0 0.0 0.0 0.0
I have an 2xN array containing indices of elements in X that I want to
assign a new value.
julia ind = [1 1; 2 2; 3 3]
3x2 Array{Int64,2}:
1 1
2 2
3 3
What is the simplest way to assign a value to all elements in X whose
indices are rows in ind? (something like X[ind] = 2.0).
julia X
2.0 0.0 0.0 0.0
0.0 2.0 0.0 0.0
0.0 0.0 2.0 0.0
0.0 0.0 0.0 0.0
RE: [julia-users] Re: home page content
+1 on that! Even vague plans that are subject to change would be great to have. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Christian Peel Sent: Wednesday, December 10, 2014 10:15 AM To: julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org http://clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com javascript: wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff anth...@berkeley.edu wrote: +1 on that! Even vague plans that are subject to change would be great to have. *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] *On Behalf Of *Christian Peel *Sent:* Wednesday, December 10, 2014 10:15 AM *To:* julia-users@googlegroups.com *Subject:* Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] Re: home page content
Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff anth...@berkeley.edu wrote: +1 on that! Even vague plans that are subject to change would be great to have. From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Christian Peel Sent: Wednesday, December 10, 2014 10:15 AM To: julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
It would be helpful if you could have julia execute using BinDeps; BinDeps.debug(CasaCore) after attempting to build. It also looks like you have another error that we should look into: Warning: error initializing module GMP: ErrorException(The dynamically loaded GMP library (version 5.0.2 with __gmp_bits_per_limb == 64) does not correspond to the compile time version (version 5.1.3 with __gmp_bits_per_limb == 64). Please rebuild Julia.) Has Travis changed Ubuntu releases recently or something?
Re: [julia-users] Re: home page content
I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org javascript: wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff ant...@berkeley.edu javascript: wrote: +1 on that! Even vague plans that are subject to change would be great to have. *From:* julia...@googlegroups.com javascript: [mailto: julia...@googlegroups.com javascript:] *On Behalf Of *Christian Peel *Sent:* Wednesday, December 10, 2014 10:15 AM *To:* julia...@googlegroups.com javascript: *Subject:* Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that. It may not even be decided which version will win. [Clojure])http://clojure.org/) has a nice, simple and informative home page, while [Scala](http://www.scala-lang.org/) has overdone it like the new Haskell. For other approaches see the [Nim](http://nimrod-lang.org/) - formerly 'Nimrod' - and [Nemerle](http://nemerle.org/) home pages. In the end I feel the condensed form of the Python home page will attract more interest, for example with 'latest news' and 'upcoming events' on the first page.This gives the impression of a lively and engaged community. On Wednesday, December 10, 2014 11:23:37 AM UTC+1, Tim Holy wrote: I like the Haskell one better than the Rust one. --Tim
[julia-users] Re: Finite element code in Julia: Curious overhead in .* product
I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you allocate millions of very small short-lived objects and that taxes the GC quite a lot.
[julia-users] customized hash function (or equality function)?
Hi, this should be a simple affair.
I have a polynomial object that keeps track of its coefficients, it is
defined as follows:
immutable Poly1
coef :: Array{Float64}
end
I would like to make a dictionary with keys that are Poly1 type, like
follows:
# creation
r_dict1 = [Poly1([1.0, 2.0]) = 1]
# access
r_dict1[Poly1([1.0, 2.0])]
However currently it gives the arrow key not found
By looking at the hash we can see why, the has is not identical even for
equal polynomials:
hash(Poly1([1.0, 2.0])) # gives an arbitrary hex string
hash(Poly1([1.0, 2.0])) # gives a different hex string
How would I implement my own hash so the same polynomials (by same I mean
the coefficients are same, i.e. the coef::Array{Float64} part of the two
polynomials are the same?
Much thanks!!
--evan
Re: [julia-users] Re: home page content
-1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff ant...@berkeley.edu wrote: +1 on that! Even vague plans that are subject to change would be great to have. *From:* julia...@googlegroups.com [mailto:julia...@googlegroups.com] *On Behalf Of *Christian Peel *Sent:* Wednesday, December 10, 2014 10:15 AM *To:* julia...@googlegroups.com *Subject:* Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories, documentation, etc) B. provide information for potential new users (showcasing features of the language, links to tutorials). Given that space on the very front page is constrained (in the soft sense: no one wants pages that go on and on any more), I think that deciding on a balance between A and B would be a good way to focus the discussion. Once we have decided that, we can shamelessly copy good practices. For example, 1. the R website emphasizes content for existing users (in a non-flashy way that I am OK with), with very little material for new users, 2. about 1/3 of the middle bar on https://www.haskell.org/haskellwiki/Haskell is for new users (explanations/tutorials/etc), the 1/3 is for existing users (specs, libraries), and the final 1/3 is for both (forums, wiki, etc), 3. http://new-www.haskell.org/ is mostly caters to potential new users (see how great this language is), 4. the content of clojure.org is similarly for potential new users, while the sidebar has links for existing users. Best, Tamas On Wed, Dec 10 2014, Hans W Borchers hwbor...@gmail.com wrote: Look at the R home page. R is one of the most popular languages, and esp. so for statistical and computational applications. A programming language does not need bloated home pages. I like the old Haskell home page much more than the new one. The new one has large, uninformative background pictures and not much information in a small and readable view. The HaskellWiki front page was much better in that.
[julia-users] Roadmap
I hear you, and I didn’t think much before sending my email. Couple of points: I totally agree this should certainly not be on the homepage. I also agree that there is no need for a detailed schedule, deadlines or anything like that. I think the only thing that would be immensely helpful at least for me is just a very high level idea of what the core team is thinking about a roadmap/timing. Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess is that the core team has an idea about those kinds of questions, and it would be great if you could share that kind of stuff from time to time. Maybe one idea here would be that the core team just sends out a brief email after a major release what the current thinking is about the next version and the road to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, but that in itself would also be valuable information for us users. I am following the issue tracker and am subscribed to the email lists, and I don’t get any sense/picture about those kind of high level questions from those sources. Cheers, David From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman Sent: Wednesday, December 10, 2014 4:31 PM To: julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content -1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org mailto:ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff ant...@berkeley.edu mailto:ant...@berkeley.edu wrote: +1 on that! Even vague plans that are subject to change would be great to have. From: mailto:julia...@googlegroups.com julia...@googlegroups.com [ mailto:julia...@googlegroups.com mailto:julia...@googlegroups.com] On Behalf Of Christian Peel Sent: Wednesday, December 10, 2014 10:15 AM To: mailto:julia...@googlegroups.com julia...@googlegroups.com Subject: Re: [julia-users] Re: home page content One thing that I would very much appreciate is some kind of development schedule. For example - Some kind of general roadmap - a plan for when 0.4 and future releases will come - Any plans to switch to a regular schedule? (yearly, six months, ...) - What features remain before a 1.0 release? - When will following arrive? faster compilation pre-compiled modules Interactive debugging; line numbers for all errors Automatic reload on file modification. Solving P=NP I know that it's tough to make such a schedule, but anything that you can provide would be helpful. Also, I'd be happy for something like a weekly update; or a weekly blog post to help those who don't peruse this group in depth each day. Thanks! Chris On Wednesday, December 10, 2014 5:41:35 AM UTC-8, Tamas Papp wrote: From the discussion, it looks like that homepages for programming languages (and realed projects) serve two purposes: A. provide resources for the existing users (links to mailing lists, package directories,
Re: [julia-users] Roadmap
FWIW, my sense is that no one really knows what's going to happen between 0.4 and 1.0. There are lots of projects that are seen as essential before 1.0, but many of those are tenatively on the 0.4 release targets (static compilation, array views, package documentation, etc.). At JuliaCon, I realized that I was one of the longest standing users of Julia -- many people at JuliaCon had never tried Julia 0.1 and therefore don't remember how much the 0.2 release improved the language and redefined the way Julia code was written. I feel like 0.4 is going to be a similar release: a lot of the most egregious problems with the current version of Julia are going to be fixed. But once those problems are solved, it seems hard to believe that we won't start realizing that there are lots of parts of the language that could be cleaned up before 1.0. My sense is that Julia, like ggplot2, will start to be mature enough for almost all users well before 1.0 is released, but that1.0 will still thankfully have the freedom to make any changes that are necessary before something gets declared as the finished product. -- John On Dec 10, 2014, at 4:45 PM, David Anthoff anth...@berkeley.edu wrote: I hear you, and I didn’t think much before sending my email. Couple of points: I totally agree this should certainly not be on the homepage. I also agree that there is no need for a detailed schedule, deadlines or anything like that. I think the only thing that would be immensely helpful at least for me is just a very high level idea of what the core team is thinking about a roadmap/timing. Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess is that the core team has an idea about those kinds of questions, and it would be great if you could share that kind of stuff from time to time. Maybe one idea here would be that the core team just sends out a brief email after a major release what the current thinking is about the next version and the road to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, but that in itself would also be valuable information for us users. I am following the issue tracker and am subscribed to the email lists, and I don’t get any sense/picture about those kind of high level questions from those sources. Cheers, David From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman Sent: Wednesday, December 10, 2014 4:31 PM To: julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content -1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while I'm planning things and as that plan changes and then do the work and then document it. It's hard enough to actually do the work. On Wed, Dec 10, 2014 at 4:44 PM, David Anthoff ant...@berkeley.edu wrote: +1 on that! Even vague plans that are subject to change would be great to have. From: julia...@googlegroups.com [mailto:julia...@googlegroups.com] On Behalf OfChristian Peel Sent: Wednesday, December 10, 2014 10:15 AM To: julia...@googlegroups.com Subject: Re:
[julia-users] Re: customized hash function (or equality function)?
Base.hash(p::Poly1, h::Uint) = hash(p.coef, h) Base.(==)(p1::Poly1, p2::Poly2) = p1.coef == p2.coef (If you override hash you should always override == to be consistent, and vice versa.)
RE: [julia-users] Roadmap
Thanks, this is really useful! I would very much cherish an email like this maybe after a release from some of the core team members. It just gives a nice insight into the current plans. Cheers, David From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of John Myles White Sent: Wednesday, December 10, 2014 4:56 PM To: julia-users@googlegroups.com Subject: Re: [julia-users] Roadmap FWIW, my sense is that no one really knows what's going to happen between 0.4 and 1.0. There are lots of projects that are seen as essential before 1.0, but many of those are tenatively on the 0.4 release targets (static compilation, array views, package documentation, etc.). At JuliaCon, I realized that I was one of the longest standing users of Julia -- many people at JuliaCon had never tried Julia 0.1 and therefore don't remember how much the 0.2 release improved the language and redefined the way Julia code was written. I feel like 0.4 is going to be a similar release: a lot of the most egregious problems with the current version of Julia are going to be fixed. But once those problems are solved, it seems hard to believe that we won't start realizing that there are lots of parts of the language that could be cleaned up before 1.0. My sense is that Julia, like ggplot2, will start to be mature enough for almost all users well before 1.0 is released, but that1.0 will still thankfully have the freedom to make any changes that are necessary before something gets declared as the finished product. -- John On Dec 10, 2014, at 4:45 PM, David Anthoff anth...@berkeley.edu mailto:anth...@berkeley.edu wrote: I hear you, and I didn't think much before sending my email. Couple of points: I totally agree this should certainly not be on the homepage. I also agree that there is no need for a detailed schedule, deadlines or anything like that. I think the only thing that would be immensely helpful at least for me is just a very high level idea of what the core team is thinking about a roadmap/timing. Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess is that the core team has an idea about those kinds of questions, and it would be great if you could share that kind of stuff from time to time. Maybe one idea here would be that the core team just sends out a brief email after a major release what the current thinking is about the next version and the road to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, but that in itself would also be valuable information for us users. I am following the issue tracker and am subscribed to the email lists, and I don't get any sense/picture about those kind of high level questions from those sources. Cheers, David From: mailto:julia-users@googlegroups.com julia-users@googlegroups.com [ mailto:julia-users@googlegroups.com mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman Sent: Wednesday, December 10, 2014 4:31 PM To: mailto:julia-users@googlegroups.com julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content -1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski mailto:ste...@karpinski.org ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That
[julia-users] Re: Roadmap
Thanks for making this a new thread, it was getting off-topic. Jeff's probably very busily working on https://github.com/JuliaLang/julia/issues/8839 right now and might not have plans that extend too far beyond that, but you'll have to ask him. I think you'll get different answers to these questions from everyone you ask - if there's a centralized grand plan that only the core team (wherever you want to draw that line) knows, they probably would have let everyone know by now. Most of us are playing it by ear and enjoying it as we go. Julia's quite usable now, otherwise we wouldn't be here. It will continue to get better, people recognize the current problems and are working on them. As John said though there will be future problems that we don't know about yet. Declaring 1.0 means not expecting any breaking changes for at least a few minor versions, I feel like that would require more decoupling of the core language from the standard library (or minimizing the number of features we include in the standard library) as library features will need to continue evolving in possibly breaking ways even after the core language can be called ready for 1.0. There is a general theme of wanting to shrink the core that many of us would like to work towards over time, and in a parallel effort trying to make the installation experience for commonly-used packages better. And we're actually being stricter than semver requires for 0.x.y version numbers, I don't see any reason to rush things or make unreliable long-term predictions. On Wednesday, December 10, 2014 4:45:18 PM UTC-8, David Anthoff wrote: I hear you, and I didn’t think much before sending my email. Couple of points: I totally agree this should certainly not be on the homepage. I also agree that there is no need for a detailed schedule, deadlines or anything like that. I think the only thing that would be immensely helpful at least for me is just a very high level idea of what the core team is thinking about a roadmap/timing. Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess is that the core team has an idea about those kinds of questions, and it would be great if you could share that kind of stuff from time to time. Maybe one idea here would be that the core team just sends out a brief email after a major release what the current thinking is about the next version and the road to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, but that in itself would also be valuable information for us users. I am following the issue tracker and am subscribed to the email lists, and I don’t get any sense/picture about those kind of high level questions from those sources. Cheers, David *From:* julia...@googlegroups.com javascript: [mailto: julia...@googlegroups.com javascript:] *On Behalf Of *Tony Kelman *Sent:* Wednesday, December 10, 2014 4:31 PM *To:* julia...@googlegroups.com javascript: *Subject:* Re: [julia-users] Re: home page content -1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014 4:53:50 PM UTC-5, John Myles White wrote: Stefan, I shared your moment of terror about the idea of posting plans (essentially all of which will be invalidated) to the home page. Although it's huge volume of e-mail, I do feel like people who want to keep up with new developments in Julia should try to subscribe to the issue tracker and watch decisions get made in real time. It's a large increase in workload to ask people to both do work on Julia and write up regular reports about the work. -- John On Dec 10, 2014, at 1:48 PM, Stefan Karpinski ste...@karpinski.org wrote: I have to say the concept of putting plans up on the home page fills me with dread. That means I have update the home page while
[julia-users] defining types in macros broken in 0.4
The below works fine in 0.3 but I get
ERROR: error compiling anonymous: type definition not allowed inside a
local scope
In 0.4. How am I suppose to define types inside a Macro? As far as I can
tell, the esc() should cause everything to run in the current scope which
isn't local.
macro calculus_operator(Op,AbstOp,WrappOp)
return esc(quote
abstract $AbstOp{T} : BandedOperator{T}
immutable $Op{S:FunctionSpace,T:Number} : $AbstOp{T}
space::S# the domain space
order::Int
end
immutable $WrappOp{S:BandedOperator} : $AbstOp{Float64}
op::S
order::Int
end
## More code
end)
end
@calculus_operator(Derivative,AbstractDerivative,DerivativeWrapper)
[julia-users] Re: customized hash function (or equality function)?
wonderful! thanks!! On Wednesday, December 10, 2014 8:24:17 PM UTC-5, Steven G. Johnson wrote: Base.hash(p::Poly1, h::Uint) = hash(p.coef, h) Base.(==)(p1::Poly1, p2::Poly2) = p1.coef == p2.coef (If you override hash you should always override == to be consistent, and vice versa.)
Re: [julia-users] BinDeps fails to find a built dependency (but only on Travis)
Not sure if this is the same, but Mike Nolta and I also had a problem building FITSIO.jl on Travis that we could not reproduce locally on Ubuntu 14.04. We eventually gave up and just used the apt package for cfitsio instead of building it from source. See https://github.com/JuliaAstro/FITSIO.jl/pull/17 . There’s some noise in that issue, but the root problem for us seemed to be that the generated deps.jl file contained @checked_lib libcfitsio libcfitsio.so instead of @checked_lib libcfitsio actual/path/to/libcfitsio.so The shared library was built and installed as expected… just the path was wrong. Kyle
RE: [julia-users] Roadmap
Somewhere, not necessarily on the front page, some tips for people wondering where the project is heading would be good. Not a list of plans, but orienting info like no one knows, and an explanation of how to get a sense of current (i.e., for 0.4 right now) and future issues from the bug tracker. I found the tip I got awhile ago to search on a particular tag (0.4?) and sort by bug/issue activity pretty helpful. BTW, is 0.4 still in a you don't want to go there state for users of julia? Ross From: julia-users@googlegroups.com [julia-users@googlegroups.com] on behalf of David Anthoff [anth...@berkeley.edu] Sent: Wednesday, December 10, 2014 5:38 PM To: julia-users@googlegroups.com Subject: RE: [julia-users] Roadmap Thanks, this is really useful! I would very much cherish an email like this maybe after a release from some of the core team members. It just gives a nice insight into the current plans. Cheers, David From: julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of John Myles White Sent: Wednesday, December 10, 2014 4:56 PM To: julia-users@googlegroups.com Subject: Re: [julia-users] Roadmap FWIW, my sense is that no one really knows what's going to happen between 0.4 and 1.0. There are lots of projects that are seen as essential before 1.0, but many of those are tenatively on the 0.4 release targets (static compilation, array views, package documentation, etc.). At JuliaCon, I realized that I was one of the longest standing users of Julia -- many people at JuliaCon had never tried Julia 0.1 and therefore don't remember how much the 0.2 release improved the language and redefined the way Julia code was written. I feel like 0.4 is going to be a similar release: a lot of the most egregious problems with the current version of Julia are going to be fixed. But once those problems are solved, it seems hard to believe that we won't start realizing that there are lots of parts of the language that could be cleaned up before 1.0. My sense is that Julia, like ggplot2, will start to be mature enough for almost all users well before 1.0 is released, but that1.0 will still thankfully have the freedom to make any changes that are necessary before something gets declared as the finished product. -- John On Dec 10, 2014, at 4:45 PM, David Anthoff anth...@berkeley.edumailto:anth...@berkeley.edu wrote: I hear you, and I didn’t think much before sending my email. Couple of points: I totally agree this should certainly not be on the homepage. I also agree that there is no need for a detailed schedule, deadlines or anything like that. I think the only thing that would be immensely helpful at least for me is just a very high level idea of what the core team is thinking about a roadmap/timing. Do you expect a 1.0 more in 10 years, or more in 1 year? Do you right now expect there to be a 0.5, 0.6, or many more releases before a 1.0? My gut guess is that the core team has an idea about those kinds of questions, and it would be great if you could share that kind of stuff from time to time. Maybe one idea here would be that the core team just sends out a brief email after a major release what the current thinking is about the next version and the road to 1.0? Such an email could be fuzzy and non-committal if the plans are fuzzy, but that in itself would also be valuable information for us users. I am following the issue tracker and am subscribed to the email lists, and I don’t get any sense/picture about those kind of high level questions from those sources. Cheers, David From: julia-users@googlegroups.commailto:julia-users@googlegroups.com [mailto:julia-users@googlegroups.com] On Behalf Of Tony Kelman Sent: Wednesday, December 10, 2014 4:31 PM To: julia-users@googlegroups.commailto:julia-users@googlegroups.com Subject: Re: [julia-users] Re: home page content -1 on trying to put plans, schedule, roadmap on the website. This week in Julia was a great contribution to the community but evidently took more effort than Matt had time to keep up with. New features get developed as the PR's for them get worked on and finished. You can subscribe to just the subset of issues/PR's for things you (along with everyone else) are eagerly awaiting. Better yet, help with testing and code review if you can. We have been doing a good job of monthly backport bugfix releases, we should be able to continue doing that. But 0.4 is still unstable and has several big-ticket items still open and being worked on (check the milestones on github). It's too early to try to make time estimates, if people are impatient and want a release sooner it's not going to be possible without punting on a number of targeted features and pushing them back to 0.5 or later. On Wednesday, December 10, 2014 1:58:52 PM UTC-8, Randy Zwitch wrote: I think it would please everyone if you moved daily televised scrums. On Wednesday, December 10, 2014
Re: [julia-users] Roadmap
Really nice summaries, John and Tony. On Thursday, December 11, 2014 02:08:54 AM Boylan, Ross wrote: BTW, is 0.4 still in a you don't want to go there state for users of julia? In short, yes---for most users I'd personally recommend sticking with 0.3. Unless you simply _must_ have some of its lovely new features. But be prepared to update your code basically every week or so to deal with changes. --Tim
Re: [julia-users] Re: Finite element code in Julia: Curious overhead in .* product
Can you post short but complete code examples in a gist? https://gist.github.com/ That would make it easier to follow than chasing code examples across long email chains. --Tim On Wednesday, December 10, 2014 02:35:38 PM Petr Krysl wrote: I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you allocate millions of very small short-lived objects and that taxes the GC quite a lot.
Re: [julia-users] defining types in macros broken in 0.4
Seems like it might be worth filing an issue. It would help if it's complete
enough that people could copy-paste the `macroexpand`ed code and get it to
work. (Things like `BandedOperator`, etc, are not available in this snippet.)
--Tim
On Wednesday, December 10, 2014 05:53:54 PM Sheehan Olver wrote:
The below works fine in 0.3 but I get
ERROR: error compiling anonymous: type definition not allowed inside a
local scope
In 0.4. How am I suppose to define types inside a Macro? As far as I can
tell, the esc() should cause everything to run in the current scope which
isn't local.
macro calculus_operator(Op,AbstOp,WrappOp)
return esc(quote
abstract $AbstOp{T} : BandedOperator{T}
immutable $Op{S:FunctionSpace,T:Number} : $AbstOp{T}
space::S# the domain space
order::Int
end
immutable $WrappOp{S:BandedOperator} : $AbstOp{Float64}
op::S
order::Int
end
## More code
end)
end
@calculus_operator(Derivative,AbstractDerivative,DerivativeWrapper)
[julia-users] scope using let and local/global variables
Hi folks, I have the following code fragment: x=1 let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end It contains two variables both named x, one inside the scope defined by let, and one at global scope. If I run this code the output is: 2 2 This indicates that (i) that line 1 is using the local version of x, and (ii) that line 2 is using the global version of x. If I remove this global x I now get an error because eval() is looking for the global x which no longer exists: let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end 2 ERROR: x not defined My question: when evaluating an expression using eval() such as line 2, how can I force Julia to use the local (not global) version of x and thus avoid this error? Thanks Mike
Re: [julia-users] Re: Finite element code in Julia: Curious overhead in .* product
The code is really short: N=200 function doit1(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns * (f * Jac); end Fe end function doit2(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns .* ([f] * Jac); end Fe end function doit3(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N fs= ([f] * Jac ); Fe += Ns .* fs; end Fe end function doit4(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N # fs= [f]; fs *= Jac; Fe += Ns .* fs; end Fe end # @time doit1(N) @time doit2(N) @time doit3(N) @time doit4(N) Here are the measurements on my machine: elapsed time: 0.461173619 seconds (384129944 bytes allocated, 44.45% gc time) elapsed time: 6.905249901 seconds (1904151036 bytes allocated, 12.84% gc time) elapsed time: 6.862259146 seconds (1904166072 bytes allocated, 13.23% gc time) elapsed time: 6.842678542 seconds (1904166256 bytes allocated, 12.81% gc time) I'll be grateful for any pointers as to how to structure the code so that the array operations not incur this horrendous penalty. Petr On Wednesday, December 10, 2014 7:06:43 PM UTC-8, Tim Holy wrote: Can you post short but complete code examples in a gist? https://gist.github.com/ That would make it easier to follow than chasing code examples across long email chains. --Tim On Wednesday, December 10, 2014 02:35:38 PM Petr Krysl wrote: I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you allocate millions of very small short-lived objects and that taxes the GC quite a lot.
[julia-users] Community Support for Julia on Travis CI
Everyone,
I am happy to announce that we now have Julia support on Travis [1].
This has been an effort on the part of the newly formed JuliaCI [2]
group and concretely this means that a complete `.travis.yml` for a
Julia project can now be as concise as:
language: julia
julia:
- release
- nightly
As opposed to:
language: cpp
compiler:
- clang
env:
matrix:
- JULIAVERSION=juliareleases
- JULIAVERSION=julianightlies
before_install:
- sudo add-apt-repository ppa:staticfloat/julia-deps -y
- sudo add-apt-repository ppa:staticfloat/${JULIAVERSION} -y
- sudo apt-get update -qq -y
- sudo apt-get install libpcre3-dev julia -y
- if [[ -a .git/shallow ]]; then git fetch --unshallow; fi
script:
- julia --check-bounds=yes -e 'versioninfo(); Pkg.init();
Pkg.clone(pwd()); Pkg.build(Foo.jl); Pkg.test(Foo.jl)'
A big thank you to all that made this possible with code, comments,
and feedback. Also a big shout out to all the people over at
Travis CI for all their help. These changes should soon be integrated
into `Pkg` so that new packages will make use of it and we will
hopefully soon also add support to test specific Julia release
branches and numbers.
Pontus Stenetorp
[1]:
http://blog.travis-ci.com/2014-12-10-community-driven-language-support-comes-to-travis-ci/
[2]: https://github.com/JuliaCI
Re: [julia-users] Re: Finite element code in Julia: Curious overhead in .* product
Multiplying two Float64s yields another Float64; most likely, this will be stored in the CPU's registers. In contrast, [f]*Jac creates an array, on each iteration, that has to be stored on the heap. A faster variant devectorizes: function doit1a(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N tmp = f*Jac for j = 1:length(Fe) Fe[j] += Ns[j]*tmp end end Fe end julia @time doit1(N); elapsed time: 0.810270399 seconds (384000320 bytes allocated, 61.23% gc time) julia @time doit1a(N); elapsed time: 0.022118726 seconds (320 bytes allocated) Note the tiny allocations in the second case. --Tim On Wednesday, December 10, 2014 07:54:00 PM Petr Krysl wrote: The code is really short: N=200 function doit1(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns * (f * Jac); end Fe end function doit2(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns .* ([f] * Jac); end Fe end function doit3(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N fs= ([f] * Jac ); Fe += Ns .* fs; end Fe end function doit4(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N # fs= [f]; fs *= Jac; Fe += Ns .* fs; end Fe end # @time doit1(N) @time doit2(N) @time doit3(N) @time doit4(N) Here are the measurements on my machine: elapsed time: 0.461173619 seconds (384129944 bytes allocated, 44.45% gc time) elapsed time: 6.905249901 seconds (1904151036 bytes allocated, 12.84% gc time) elapsed time: 6.862259146 seconds (1904166072 bytes allocated, 13.23% gc time) elapsed time: 6.842678542 seconds (1904166256 bytes allocated, 12.81% gc time) I'll be grateful for any pointers as to how to structure the code so that the array operations not incur this horrendous penalty. Petr On Wednesday, December 10, 2014 7:06:43 PM UTC-8, Tim Holy wrote: Can you post short but complete code examples in a gist? https://gist.github.com/ That would make it easier to follow than chasing code examples across long email chains. --Tim On Wednesday, December 10, 2014 02:35:38 PM Petr Krysl wrote: I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you
Re: [julia-users] Re: Finite element code in Julia: Curious overhead in .* product
https://gist.github.com/anonymous/054f6fd269f97a4442db On Wednesday, December 10, 2014 7:06:43 PM UTC-8, Tim Holy wrote: Can you post short but complete code examples in a gist? https://gist.github.com/ That would make it easier to follow than chasing code examples across long email chains. --Tim On Wednesday, December 10, 2014 02:35:38 PM Petr Krysl wrote: I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can see, the first two options are the fast one (multiplication with a scalar) and the slow one (multiplication with a one by one matrix). In the third option I tried to avoid the creation of an ad hoc temporary by allocating a variable outside of the loop. The effect unfortunately is nil. fs=[0.0]# Used only for option (3) # Now loop over all fes in the block for i=1:size(conns,1) ... for j=1:npts ... # Option (1): 7.193767019 seconds (1648850568 bytes # Fe += Ns[j] * (f * Jac * w[j]); # # Option (2): 17.301214583 seconds (3244458368 bytes #Fe += Ns[j] .* ([f] * Jac * w[j]); # # Option (3): 16.943314075 seconds (3232879120 fs= ([f] * Jac * w[j]); Fe += Ns[j] .* fs; end ... end What do you think? Why is the code still getting hit with a big performance/memory penalty? Thanks, Petr On Monday, December 8, 2014 2:03:02 PM UTC-8, Valentin Churavy wrote: I would think that when f is a 1x1 matrix Julia is allocating a new 1x1 matrix to store the result. If it is a scalar that allocation can be skipped. When this part of the code is now in a hot loop it might happen that you allocate millions of very small short-lived objects and that taxes the GC quite a lot.
Re: [julia-users] Re: Finite element code in Julia: Curious overhead in .* product
Thanks. Now my head is really spinning! See, before I posted the original question I tried expanding the loop in the actual FE code, and the code was SLOWER and was using MORE memory: With the expression Fe += Ns[j] * (f * Jac * w[j]); : 6.223416655 seconds (1648832052 bytes With the expanded loop for kx=1:length(Fe) # alternative (devectorization) Fe[kx] += Ns[j][kx] * (f * Jac * w[j]); end 7.340272676 seconds (1776971604 bytes allocated, In addition, your argument clearly demonstrates how to avoid the temporary array for doit1(), but doit2() adds to the 3 x 1 one additional 1x1 temporary (it seems to me), yet it is about 14 times slower. Why is that? Finally, if the only way I can get decent performance with lines like Fe += Ns[j] * (f * Jac * w[j]); # Fe and Ns[j] arrays is to manually write out all the loops, that would be terrible news indeed. Not only that is a lot of work when rewriting loads of Matlab code (with several matrices concatenated in many many expressions), but the legibility and maintainability tanks. I tought the point of having the array objects and the associated manipulation functions was to hide the loops while delivering decent performance... Petr On Wednesday, December 10, 2014 8:28:31 PM UTC-8, Tim Holy wrote: Multiplying two Float64s yields another Float64; most likely, this will be stored in the CPU's registers. In contrast, [f]*Jac creates an array, on each iteration, that has to be stored on the heap. A faster variant devectorizes: function doit1a(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N tmp = f*Jac for j = 1:length(Fe) Fe[j] += Ns[j]*tmp end end Fe end julia @time doit1(N); elapsed time: 0.810270399 seconds (384000320 bytes allocated, 61.23% gc time) julia @time doit1a(N); elapsed time: 0.022118726 seconds (320 bytes allocated) Note the tiny allocations in the second case. --Tim On Wednesday, December 10, 2014 07:54:00 PM Petr Krysl wrote: The code is really short: N=200 function doit1(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns * (f * Jac); end Fe end function doit2(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; for i=1:N Fe += Ns .* ([f] * Jac); end Fe end function doit3(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N fs= ([f] * Jac ); Fe += Ns .* fs; end Fe end function doit4(N) Fe=zeros(3,1);Ns=zeros(3,1)+1.0;f=-6.0;Jac=1.0; fs=[1.0] for i=1:N # fs= [f]; fs *= Jac; Fe += Ns .* fs; end Fe end # @time doit1(N) @time doit2(N) @time doit3(N) @time doit4(N) Here are the measurements on my machine: elapsed time: 0.461173619 seconds (384129944 bytes allocated, 44.45% gc time) elapsed time: 6.905249901 seconds (1904151036 bytes allocated, 12.84% gc time) elapsed time: 6.862259146 seconds (1904166072 bytes allocated, 13.23% gc time) elapsed time: 6.842678542 seconds (1904166256 bytes allocated, 12.81% gc time) I'll be grateful for any pointers as to how to structure the code so that the array operations not incur this horrendous penalty. Petr On Wednesday, December 10, 2014 7:06:43 PM UTC-8, Tim Holy wrote: Can you post short but complete code examples in a gist? https://gist.github.com/ That would make it easier to follow than chasing code examples across long email chains. --Tim On Wednesday, December 10, 2014 02:35:38 PM Petr Krysl wrote: I don't know if this is correct, but here is a guess: Option 3 still requires a temp array ( to calculate the result of the paren fs= ([f] * Jac * w[j]); ), and option 4 eliminates that temp. The cost of the temp over the 2 million loops is ~200MB and 0.6 sec CPU time. So WHY is the difference between 1 and 2 so HUUUGE? I think this calls for someone who wrote the compiler. Guys? Thanks a bunch, P On Wednesday, December 10, 2014 12:51:26 PM UTC-8, Petr Krysl wrote: Actually: option (4) was also tested: # 16.333766821 seconds (3008899660 bytes fs[1]= f; fs *= (Jac * w[j]); Fe += Ns[j] .* fs; So, allocation of memory was reduced somewhat, runtime not so much. On Wednesday, December 10, 2014 12:45:20 PM UTC-8, Petr Krysl wrote: Well, temporary array was also on my mind. However, things are I believe a little bit more complicated. Here is the code with three timed options. As you can
Re: [julia-users] scope using let and local/global variables
eval, by design, doesn't work that way. there are just too many better alternatives. typically, an anonymous function / lambda is the best and most direct replacement: let x=2 println(x)# Line 1 exp = () - x+1 println(exp())# Line 2 end On Wed Dec 10 2014 at 10:43:00 PM Michael Mayo mm...@waikato.ac.nz wrote: Hi folks, I have the following code fragment: x=1 let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end It contains two variables both named x, one inside the scope defined by let, and one at global scope. If I run this code the output is: 2 2 This indicates that (i) that line 1 is using the local version of x, and (ii) that line 2 is using the global version of x. If I remove this global x I now get an error because eval() is looking for the global x which no longer exists: let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end 2 ERROR: x not defined My question: when evaluating an expression using eval() such as line 2, how can I force Julia to use the local (not global) version of x and thus avoid this error? Thanks Mike
Re: [julia-users] scope using let and local/global variables
Thanks, but its not quite what I'm looking for. I want to be able to edit the Expr tree and then evaluate different expressions using variables defined in the local scope,not the global scope (e.g. for genetic programming, where random changes to an expression are repeatedly evaluated to find the best one). Using anonymous functions could work but modifying the .code property of an anonymous function looks much more complex than modifying the Expr types. Anyway thanks for your answer, maybe your suggestion is the only possible way to achieve this! Mike On Thursday, December 11, 2014 6:56:15 PM UTC+13, Jameson wrote: eval, by design, doesn't work that way. there are just too many better alternatives. typically, an anonymous function / lambda is the best and most direct replacement: let x=2 println(x)# Line 1 exp = () - x+1 println(exp())# Line 2 end On Wed Dec 10 2014 at 10:43:00 PM Michael Mayo mm...@waikato.ac.nz javascript: wrote: Hi folks, I have the following code fragment: x=1 let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end It contains two variables both named x, one inside the scope defined by let, and one at global scope. If I run this code the output is: 2 2 This indicates that (i) that line 1 is using the local version of x, and (ii) that line 2 is using the global version of x. If I remove this global x I now get an error because eval() is looking for the global x which no longer exists: let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end 2 ERROR: x not defined My question: when evaluating an expression using eval() such as line 2, how can I force Julia to use the local (not global) version of x and thus avoid this error? Thanks Mike
[julia-users] Unexpected append! behavior
Hi all, I'm an engineering student with interests in numerical simulation of, well, just about anything, and stumbled across Julia and have been experimenting a bit. I have no idea if this question has been addressed before, but I'm seeing a behavior that's not quite what I'd expect from the append! method. At the prompt, if I type X = [1,2]; Y = [1,2]; append!(X,Y) 4-element Array(Int64,1): 1 2 3 4 This is the output that I expect to see. But if I try to store this result back in X: X = append!(X,Y) 6-element Array(Int64,1): 1 2 3 4 3 4 This is counterintuitive to me. It looks as though the append is recursing and adding Y onto the end of the new X obtained from appending... but only once. If I'm causing a recursion by this call, why doesn't it continue? Thanks to anyone who can enlighten me. -- Sean
[julia-users] Changes to array are not visible from caller
Hello, n00b Julia user here. I have two functions that change the values of a passed-in array. One works (dxFromX), but for the other one (eulerStep) the caller does not see any changes to the array. Why is this? function dxFromX!(x,dxdt) fill!(dxdt,0.0) for itarg = 1:size(x,1) for isrc = 1:size(x,1) dx = x[isrc,1] - x[itarg,1] dy = x[isrc,2] - x[itarg,2] coeff = 1.0f0 / (dx^2 + dy^2 + 0.1f0) dxdt[itarg,1] -= coeff * dy dxdt[itarg,2] += coeff * dx end end end function eulerStep!(x) dxdt = zeros(x) print (\ndxdt before\n,dxdt[1:5,:],\n) dxFromX!(x,dxdt) print (\ndxdt after has changed\n,dxdt[1:5,:],\n) x += 1.0f-5*dxdt print (\nx inside\n,x[1:5,:],\n) end x = float32(rand(1024,2)) print (\nx before\n,x[1:5,:],\n) @time eulerStep!(x) print (\nx after is unchanged!\n,x[1:5,:],\n) I see the same behavior on 0.3.3 and 0.4.0, both release and debug binaries, on OSX and Linux.
Re: [julia-users] Unexpected append! behavior
Hi Sean, I'm really confused by the output you're showing. X = [1,2]; Y = [1,2]; append!(X,Y) 4-element Array(Int64,1): 1 2 3 4 Do you really get this output? That seems like a huge bug if so. But I don't see that at all, which is what I'd expect. -- John
Re: [julia-users] scope using let and local/global variables
I'm not quite sure what a genetic program of that sort would look like. I would be interested to hear if you get something out of it. Another alternative is to use a module as the environment: module MyEnv end eval(MyEnv, :(code block)) This is (roughly) how the REPL is implemented to work. On Thu Dec 11 2014 at 1:26:57 AM Michael Mayo mm...@waikato.ac.nz wrote: Thanks, but its not quite what I'm looking for. I want to be able to edit the Expr tree and then evaluate different expressions using variables defined in the local scope,not the global scope (e.g. for genetic programming, where random changes to an expression are repeatedly evaluated to find the best one). Using anonymous functions could work but modifying the .code property of an anonymous function looks much more complex than modifying the Expr types. Anyway thanks for your answer, maybe your suggestion is the only possible way to achieve this! Mike On Thursday, December 11, 2014 6:56:15 PM UTC+13, Jameson wrote: eval, by design, doesn't work that way. there are just too many better alternatives. typically, an anonymous function / lambda is the best and most direct replacement: let x=2 println(x)# Line 1 exp = () - x+1 println(exp())# Line 2 end On Wed Dec 10 2014 at 10:43:00 PM Michael Mayo mm...@waikato.ac.nz wrote: Hi folks, I have the following code fragment: x=1 let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end It contains two variables both named x, one inside the scope defined by let, and one at global scope. If I run this code the output is: 2 2 This indicates that (i) that line 1 is using the local version of x, and (ii) that line 2 is using the global version of x. If I remove this global x I now get an error because eval() is looking for the global x which no longer exists: let x=2 println(x)# Line 1 exp=:(x+1) println(eval(exp))# Line 2 end 2 ERROR: x not defined My question: when evaluating an expression using eval() such as line 2, how can I force Julia to use the local (not global) version of x and thus avoid this error? Thanks Mike
Re: [julia-users] Unexpected append! behavior
I assume the first line should be X = [1,2]; Y = [3,4]; Then the results you get makes sense. The thing is that julia has mutable arrays, and the ! at the end of append! indicates that it is a function that mutates it's argument.
Re: [julia-users] Changes to array are not visible from caller
`x += ...` is equivalent to writing `x = x + ...` which rebinds the variable within that function. Instead, do an explicit array assignment `x[:,:] = ...` This is discussed in the manual with a warning about type changes, but the implication for arrays should probably be made clear as well: http://julia.readthedocs.org/en/latest/manual/mathematical-operations/#updating-operators (there are some ongoing discussions about in-place array operators to improve the situation) On Wed, Dec 10, 2014 at 7:44 PM, Mark Stock mark.j.st...@gmail.com wrote: Hello, n00b Julia user here. I have two functions that change the values of a passed-in array. One works (dxFromX), but for the other one (eulerStep) the caller does not see any changes to the array. Why is this? function dxFromX!(x,dxdt) fill!(dxdt,0.0) for itarg = 1:size(x,1) for isrc = 1:size(x,1) dx = x[isrc,1] - x[itarg,1] dy = x[isrc,2] - x[itarg,2] coeff = 1.0f0 / (dx^2 + dy^2 + 0.1f0) dxdt[itarg,1] -= coeff * dy dxdt[itarg,2] += coeff * dx end end end function eulerStep!(x) dxdt = zeros(x) print (\ndxdt before\n,dxdt[1:5,:],\n) dxFromX!(x,dxdt) print (\ndxdt after has changed\n,dxdt[1:5,:],\n) x += 1.0f-5*dxdt print (\nx inside\n,x[1:5,:],\n) end x = float32(rand(1024,2)) print (\nx before\n,x[1:5,:],\n) @time eulerStep!(x) print (\nx after is unchanged!\n,x[1:5,:],\n) I see the same behavior on 0.3.3 and 0.4.0, both release and debug binaries, on OSX and Linux.
Re: [julia-users] Unexpected append! behavior
Ivar is correct; I was running in the Windows command prompt and couldn't copy and paste so I copied it by hand and made an error. Ok, so I understand that append!(X,Y) is modifying X in place. But I still do not get where the output for the second case, where the result of append!(X,Y) is assigned back into X is what it is. It would make sense to me if this resulted in a recursion with Y forever getting appended to X, but as it is I don't understand. Thanks. -- Sean On Thursday, December 11, 2014 12:42:45 AM UTC-6, Ivar Nesje wrote: I assume the first line should be X = [1,2]; Y = [3,4]; Then the results you get makes sense. The thing is that julia has mutable arrays, and the ! at the end of append! indicates that it is a function that mutates it's argument.
