[julia-users] Re: Newbie issue tag?
I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
[julia-users] ccall and ENV on Windows
I'm trying to understand how ENV works (at least on Windows): I'm trying to ccall a library that requires a particular addition to PATH. If I do this externally (through the Windows menus) it works okay, but not via ENV[PATH]. I assume that this means that ENV only changes the local process? If so, is there anyway I can modify the system variable from within Julia? -Simon
[julia-users] writing a DataFrame consisting of DataFrames to a table
I need to write and then read a DataFrame which consists of other DataFrames to a file. df=DataFrame(d=DataFrame(x=[1:5])) gives the desired DataFrame, but writetable(df.csv,df) readtable(df.csv) returns: 1x1 DataFrames.DataFrame | Row | d | |-|-| | 1 | 5x1 DataFrames.DataFrame\n| Row | x |\n|-|---|\n| 1 | 1 |\n| 2 | 2 |\n| 3 | 3 |\n| 4 | 4 |\n| 5 | 5 | | Thus the interiour DataFrame gets stored as string in the file. Is there anyway to properly read the inner DataFrame?
[julia-users] Re: Question about how to use uninitialized fields for Composite type
You can also use whichever order you want if you create the new instance with just new() in the constructor and then fill in all the fields that you want just after.
Re: [julia-users] Re: Newbie issue tag?
I am a Julia newbie, currently diving into the internals so that I can contribute later. Many design features of Julia are novel, and in flux, which makes it harder to contibute. Even when an issue seems simple, I am always concerned that there are ramifications I don't yet understand. Identifying issues which don't require such a deep understanding of Julia would be great. So I would find Tim's suggested interpretation of the newbie label practical and useful. Best, Tamas On Fri, May 08 2015, Tim Holy tim.h...@gmail.com wrote: While I agree that easy is not always easy to define, I also think that there is real merit in flagging issues that should not require a deep dive into internals. For many first-time contributors, just learning git and GitHub is quite a barrier in itself (it was for me). A one-line fix---like adding a missing method---is the perfect warmup exercise. To a potential contributor, s/he presumably has better access to what am I good at? than to what issues will not require three days of work even by someone with expertise in Julia's innards? --Tim On Friday, May 08, 2015 10:33:48 AM Mike Innes wrote: Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas.lyc...@gmail.com wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
Re: [julia-users] How to flush output of print?
On Friday, May 08, 2015 12:44:12 PM Ali wrote: Thanks, René. That package does answer my need. It would still be useful to know how to flush the output, just for the reference. To paraphrase Avik: Use the source, Luke! :-) Many of julia's packages are quite readable (or become so, if you invest the time in getting good at it). It's also one of the best ways to learn how to become good at writing julia code. I've been doing this for several years now, and I still learn new things that way on a near-weekly basis. --Tim On Fri, May 8, 2015 at 12:40 PM, René Donner li...@donner.at wrote: You can see such behaviour implemented in https://github.com/timholy/ProgressMeter.jl (which might already do what you want ;-) Am 08.05.2015 um 12:39 schrieb Ali Rezaee arv.ka...@gmail.com: Hi, I would like to show the progress of my Julia code while its running. But I do not want each different percent being printed in a different line in the command line. Is there a way to print something and then remove it before a new item is printed? Many thanks,
Re: [julia-users] Re: Newbie issue tag?
Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas.lyc...@gmail.com wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
Re: [julia-users] How to flush output of print?
Great. Thanks a lot. On Fri, May 8, 2015 at 12:52 PM, Avik Sengupta avik.sengu...@gmail.com wrote: If you look at the source of that package, it uses terminal escape characters to achieve this: print(io, \u1b[1G) # go to first column print_with_color(color, io, s) print(io, \u1b[K)# clear the rest of the line On Friday, 8 May 2015 11:44:34 UTC+1, Ali Rezaee wrote: Thanks, René. That package does answer my need. It would still be useful to know how to flush the output, just for the reference. On Fri, May 8, 2015 at 12:40 PM, René Donner li...@donner.at wrote: You can see such behaviour implemented in https://github.com/timholy/ProgressMeter.jl (which might already do what you want ;-) Am 08.05.2015 um 12:39 schrieb Ali Rezaee arv@gmail.com: Hi, I would like to show the progress of my Julia code while its running. But I do not want each different percent being printed in a different line in the command line. Is there a way to print something and then remove it before a new item is printed? Many thanks,
Re: [julia-users] How to flush output of print?
You can see such behaviour implemented in https://github.com/timholy/ProgressMeter.jl (which might already do what you want ;-) Am 08.05.2015 um 12:39 schrieb Ali Rezaee arv.ka...@gmail.com: Hi, I would like to show the progress of my Julia code while its running. But I do not want each different percent being printed in a different line in the command line. Is there a way to print something and then remove it before a new item is printed? Many thanks,
Re: [julia-users] Re: Newbie issue tag?
Le vendredi 08 mai 2015 à 10:33 +0100, Mike Innes a écrit : Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. Still, it will be much easier to go over a list of less than 50 issues that do not require deep knowledge of the Julia code base to fix, than to go over all open issues. One of the criteria of a newbie tag is that the fix is neither controversial nor requires a deep investigation -- and these are important for all kinds of newbies. New contributors will pick the issues they feel qualified to fix. So a big +1. Regards I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas.lyc...@gmail.com wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
Re: [julia-users] Re: Newbie issue tag?
While I agree that easy is not always easy to define, I also think that there is real merit in flagging issues that should not require a deep dive into internals. For many first-time contributors, just learning git and GitHub is quite a barrier in itself (it was for me). A one-line fix---like adding a missing method---is the perfect warmup exercise. To a potential contributor, s/he presumably has better access to what am I good at? than to what issues will not require three days of work even by someone with expertise in Julia's innards? --Tim On Friday, May 08, 2015 10:33:48 AM Mike Innes wrote: Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas.lyc...@gmail.com wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
Re: [julia-users] writing a DataFrame consisting of DataFrames to a table
A dataframe isn't really meant to be used as a column in another DataFrame. It may work for a few things, but it's likely to be broken for others. On May 8, 2015 4:56 AM, axsk smartmin...@gmail.com wrote: I need to write and then read a DataFrame which consists of other DataFrames to a file. df=DataFrame(d=DataFrame(x=[1:5])) gives the desired DataFrame, but writetable(df.csv,df) readtable(df.csv) returns: 1x1 DataFrames.DataFrame | Row | d | |-|-| | 1 | 5x1 DataFrames.DataFrame\n| Row | x |\n|-|---|\n| 1 | 1 |\n| 2 | 2 |\n| 3 | 3 |\n| 4 | 4 |\n| 5 | 5 | | Thus the interiour DataFrame gets stored as string in the file. Is there anyway to properly read the inner DataFrame?
[julia-users] Re: ccall and ENV on Windows
As a follow up, here's some code (which requires that R be installed):
ENV[PATH] = ENV[PATH]*;C:\\Program Files\\R\\R-3.2.0\\bin\\x64\\
const libR = C:\\Program Files\\R\\R-3.2.0\\bin\\x64\\R.dll
argv = [REmbed]
println(ccall((:Rf_initEmbeddedR,libR),Cint,(Cint,Ptr{Ptr{Uint8}}),length(argv),argv))
If I run this as a script, I get an error message (unable to load shared
library). However if I first set
PATH=PATH;C:\Program Files\R\R-3.2.0\bin\x64
then run the script (from within the same command prompt session), the
error goes away.
Any ideas?
Simon
On Friday, 8 May 2015 09:55:29 UTC+1, Simon Byrne wrote:
I'm trying to understand how ENV works (at least on Windows):
I'm trying to ccall a library that requires a particular addition to
PATH. If I do this externally (through the Windows menus) it works okay,
but not via ENV[PATH]. I assume that this means that ENV only changes the
local process? If so, is there anyway I can modify the system variable from
within Julia?
-Simon
[julia-users] Re: Question about how to use uninitialized fields for Composite type
Thanks Toivo, Best, Alex On Friday, May 8, 2015 at 4:55:38 AM UTC-4, Toivo Henningsson wrote: You can also use whichever order you want if you create the new instance with just new() in the constructor and then fill in all the fields that you want just after.
Re: [julia-users] Compile ReadStat under Windows7
You will most likely need to install mingw64 compilers, and compile ReadStat yourself. See the Julia README.Windows for install instructions, and https://github.com/WizardMac/ReadStat/issues/13 for someone who was able to successfully compile on Windows. On Fri, May 8, 2015 at 7:55 AM, bernhard kafis...@gmail.com wrote: Hi all I would like to be able to import SAS files to Julia (*.sas7bdat). I found https://github.com/WizardMac/DataRead.jl/ which apparently uses ReadStat https://github.com/WizardMac/ReadStat It seems to use Ragel /usr/local/bin/ragel What is the easiest way to make DataRead.jl work on Julia Windows? Bernhard
Re: [julia-users] Userability of Float16
Yes, Float16 http://en.wikipedia.org/wiki/Half-precision_floating-point_format is explicitly intended for storage only: It is intended for storage (of many floating-point values where higher precision need not be stored), not for performing arithmetic computations. On Fri, May 8, 2015 at 9:26 AM, Sisyphuss zhengwend...@gmail.com wrote: I think since Float16 type will be converted to Float32 during a calculation, there's no point to stock an array of Float16 (it will only slow down the computation), unless the main memory is highly demanding.
[julia-users] Userability of Float16
I think since Float16 type will be converted to Float32 during a calculation, there's no point to stock an array of Float16 (it will only slow down the computation), unless the main memory is highly demanding.
Re: [julia-users] `include()` vs `require()`
Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] Re: Newbie issue tag?
I think it might be worth making some more meta issues, like Tim Holy's test coverage one, that are new person friendly. Things like Code Cleanup, for the stuff Jake Bolewski was working on in LinAlg, or Better Error Messages so if you get a DimensionMismatch you know which dimensions were mismatched, or More Comments on Tests. You could make a checklist of files to look at in the issue, so that someone new can easily see where in base needs work and have other files to compare to. On Friday, 8 May 2015 09:25:23 UTC-7, Isaiah wrote: There are now 13 issues with the intro issue label. I would be interested in any comments on how people feel about the calibration of these. I have to say that after looking through a large fraction of the open issue list (titles), I basically agree with Stefan's assessment in the other thread that the truly low-hanging fruit gets dealt with pretty quickly. In part this is because we have developed a reflexive response of saying want to submit a pull request? when such an issue is submitted -- which usually does elicit a PR from the submitter or someone else in fairly short order. On Fri, May 8, 2015 at 6:41 AM, Tamas Papp tkp...@gmail.com javascript: wrote: I am a Julia newbie, currently diving into the internals so that I can contribute later. Many design features of Julia are novel, and in flux, which makes it harder to contibute. Even when an issue seems simple, I am always concerned that there are ramifications I don't yet understand. Identifying issues which don't require such a deep understanding of Julia would be great. So I would find Tim's suggested interpretation of the newbie label practical and useful. Best, Tamas On Fri, May 08 2015, Tim Holy tim@gmail.com javascript: wrote: While I agree that easy is not always easy to define, I also think that there is real merit in flagging issues that should not require a deep dive into internals. For many first-time contributors, just learning git and GitHub is quite a barrier in itself (it was for me). A one-line fix---like adding a missing method---is the perfect warmup exercise. To a potential contributor, s/he presumably has better access to what am I good at? than to what issues will not require three days of work even by someone with expertise in Julia's innards? --Tim On Friday, May 08, 2015 10:33:48 AM Mike Innes wrote: Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas@gmail.com javascript: wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
Re: [julia-users] [Vectorization] Comparison of codes
Yes, function get inlined within certain limits. You can check by using
@code_typed and looking for :call expressions (@code_typed includes the
inlining pass).
Best,
--Tim
On Friday, May 08, 2015 09:30:57 AM Sisyphuss wrote:
The following code is aimed to study the benefit of vectorization:
function f(a::Array{Float64,1})
b = Array(eltype(a),size(a))
n=length(a)
for i=1:n
b[i]=a[i]+1
end
return b
end
function f(a::Float64)
b = a + 1
end
function main1()
n = 100
a = zeros(n)
@timeit begin
b = Array(eltype(a),size(a))
for i=1:n
b[i] = f(a[i])
end
end
end
function main2()
n = 100
a = zeros(n)
@timeit b=f(a)
end
begin
println(==)
main1()
main2()
end
`f` is the overloaded function, which can be either vectorized or not.
The goal is to transform every element of the vector `a` to `f(a)`, and
store it in `b`.
`main1` is non-vecorized, while `main2` is vectorized.
I thought that `main2` could have been more efficient than `main1`, because
it has less function calls.
However, according to the experiment, there is actually no performance
difference between these two versions.
The reason, is it because that `f()` is inlined by the compiler secretly?
[julia-users] Re: merge functions from different modules
@David: Thank you for this. It does appear to resolve a class of problems. On Tuesday, May 5, 2015 at 11:43:34 AM UTC-4, David Gold wrote: @Scott: Glad to hear it may be useful to you! If you encounter any bugs, or if you think of a related feature you'd like to see, please do open an issue in the github repo. I'll try my best to accommodate. @Sisyphuss: As Stefan notes, this is an additional feature. I intend it to complement the fix in issue #4345, which actually provides the cleanest environment for using merge!(). As soon as I figure out git I will be able to turn this into a real package. I intend for that to happen asap. D On Tuesday, May 5, 2015 at 8:31:17 AM UTC-4, Scott Jones wrote: Great! +1 This will solve a lot of my problems about trying to maintain good software engineering practices with Julia, (but not all, it doesn't address being able to keep the implementation separate from the abstraction... I really don't want others to have the ability to play with my private parts [as somebody else here so aptly put it]) Scott On Monday, May 4, 2015 at 3:08:43 PM UTC-4, David Gold wrote: Based on recent discussions, it seems that at least some people would like the option to have unqualified use of a function name dispatch across modules when the argument on which the function is called unambiguously specifies an exported method. While Julia doesn't do this automatically, one option is for the user explicitly to merge functions defined in different modules. I've been working on an implementation of this functionality that doesn't require users to copy a bunch of method definitions: julia using MetaMerge julia f() = nothing f (generic function with 1 method) julia module A export f immutable Foo end f(::Foo) = print(This is Foo.) f(x::Int64) = x end julia module B export f immutable Bar end f(::Bar) = print(This is Bar.) f(x::Int64) = 2x end julia using A, B Warning: using A.f in module Main conflicts with an existing identifier. Warning: using B.f in module Main conflicts with an existing identifier. julia methods(f) # 1 method for generic function f: f() at none:1 julia merge!(f, (A,f), (B,f)) julia methods(f) # 3 methods for generic function f: f() at none:1 f(x1::Foo) f(x1::Bar) julia f(A.Foo()) This is Foo. julia f(B.Bar()) This is Bar. One can also use merge!() while writing modules to allow for unqualified use of a name within the module, such as in the definition of h below: julia module C export f using MetaMerge, A, B f(::Union()) = nothing merge!(f, (A,f), (B,f), conflicts_favor=A) h(x::Int64) = f(x) end julia using C Warning: using C.f in module Main conflicts with an existing identifier. julia C.h(2) 2 The repo is here: https://github.com/davidagold/MetaMerge.jl (I've never versioned anything before, so I apologize in advance if the numbers seem silly/arbitrary). I hope it can be useful to some. Folks should feel free to leave suggestions, bug reports or requests for other kinds of functionality that they'd like to see from something like this. I'll do my best to accommodate, though I'm rather new to programming and hence cannot make any promises. Cheers, David
Re: [julia-users] Re: Newbie issue tag?
There are now 13 issues with the intro issue label. I would be interested in any comments on how people feel about the calibration of these. I have to say that after looking through a large fraction of the open issue list (titles), I basically agree with Stefan's assessment in the other thread that the truly low-hanging fruit gets dealt with pretty quickly. In part this is because we have developed a reflexive response of saying want to submit a pull request? when such an issue is submitted -- which usually does elicit a PR from the submitter or someone else in fairly short order. On Fri, May 8, 2015 at 6:41 AM, Tamas Papp tkp...@gmail.com wrote: I am a Julia newbie, currently diving into the internals so that I can contribute later. Many design features of Julia are novel, and in flux, which makes it harder to contibute. Even when an issue seems simple, I am always concerned that there are ramifications I don't yet understand. Identifying issues which don't require such a deep understanding of Julia would be great. So I would find Tim's suggested interpretation of the newbie label practical and useful. Best, Tamas On Fri, May 08 2015, Tim Holy tim.h...@gmail.com wrote: While I agree that easy is not always easy to define, I also think that there is real merit in flagging issues that should not require a deep dive into internals. For many first-time contributors, just learning git and GitHub is quite a barrier in itself (it was for me). A one-line fix---like adding a missing method---is the perfect warmup exercise. To a potential contributor, s/he presumably has better access to what am I good at? than to what issues will not require three days of work even by someone with expertise in Julia's innards? --Tim On Friday, May 08, 2015 10:33:48 AM Mike Innes wrote: Part of the issue is figuring out what Newbie means. New to programming? Experienced in programming, but new to Julia? Experienced in Julia, but new to Base? New to open source? Arguably all of these are valid targets, but mixing them together ends up not being that helpful since people still have to sort through them. I agree with what Tomas has said about writing packages. I can definitely understand people wanting to contribute to Base, but if you just want to get some code out there and/or get a taste of the process contributing to packages will be much quicker and easier. The great thing about Julia's early stage is that (a) it's really easy to find holes in functionality and (b) if you fill those holes, chance are you'll have the package for that functionality, and people are actually going to use it. On top of that, you're much more likely to be interested in the work. That's a really great opportunity IMO. It's easy enough to pick something you're interested in and, depending on your level of confidence, start from scratch, port it from another language, experiment, whatever. As one option, the web stack is particularly ripe for development right now. (Which is a polite way of saying that there isn't much of one.) On 8 May 2015 at 07:03, Tomas Lycken tomas.lyc...@gmail.com wrote: I just want to put some emphasis on what Scott hinted at: if you want to contribute to Julia, start with figuring out what *you* know a little about. Sometimes there's code in base that does some of those things, but not all of them, and/or not as well as you know how to. Sometimes there's not a place in base for your problem domain, but I've found that contributing to a package (or building a new one) is just as good a way to get started writing some Julia code. And chances are pretty high that after a while you stumble upon something in base that needs improvement for your package development to be as easy as possible - voila! We've found someplace in base for you to contribute :) Bottom line is, it's usually pretty easy to write Julia code as long as you know what the code should do - the hard part is finding something that you know how to do (and where to put the code that does it). // T
[julia-users] [Vectorization] Comparison of codes
The following code is aimed to study the benefit of vectorization:
function f(a::Array{Float64,1})
b = Array(eltype(a),size(a))
n=length(a)
for i=1:n
b[i]=a[i]+1
end
return b
end
function f(a::Float64)
b = a + 1
end
function main1()
n = 100
a = zeros(n)
@timeit begin
b = Array(eltype(a),size(a))
for i=1:n
b[i] = f(a[i])
end
end
end
function main2()
n = 100
a = zeros(n)
@timeit b=f(a)
end
begin
println(==)
main1()
main2()
end
`f` is the overloaded function, which can be either vectorized or not.
The goal is to transform every element of the vector `a` to `f(a)`, and
store it in `b`.
`main1` is non-vecorized, while `main2` is vectorized.
I thought that `main2` could have been more efficient than `main1`, because
it has less function calls.
However, according to the experiment, there is actually no performance
difference between these two versions.
The reason, is it because that `f()` is inlined by the compiler secretly?
[julia-users] Re: [Vectorization] Comparison of codes
On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] `include()` vs `require()`
include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
[julia-users] Re: [ANN] NamedTuples.jl: Named Tuples for Julia
Awesome! Python's named tuple library is great. On Friday, May 8, 2015 at 2:07:48 PM UTC-4, Michael Francis wrote: NamedTuples.jl provides an implementation of type safe named tuples for Julia (cf named tuples in python). NamedTuples may be used anywhere that a tuple is currently being used, for example in the definition of a method or as the return value of a method. NamedTuples are implemented using Julia’s macro system ensuring that the run time cost is equivalent to constructing a regular immutable type. NamedTuples may also be used in cases where a small typed immutable dictionary is desired. https://github.com/blackrock/NamedTuples.jl Any bugs or issues please reach out or open an issue. module Test using NamedTuples function foo( y ) a = 1 x = 3 return @NT( a = 1, b = world, c = hello, d=a/x, y = a/y ) end function bar( nt::@NT( a::Int64, c::ASCIIString )) return repeat( nt.c, nt.a ) end end Test.foo( 1 ) # Returns a NamedTuple of 5 elements Test.bar( @NT( a= 2, c=hello)) # Returns `hellohello`
[julia-users] Correctly accessing DataFrame columns in loop
I've been trying to rename a bunch of columns in a DataFrame and as a former pandas user am a bit thrown off by the way columns are accessed using symbols. Let's say I have a list of column names (strings) which I want to all give the same name with ascending integers at the end. After a few unsuccesful attempts, I came up with the following, but I feel like there has to be a less ugly way of doing this: df = DataFrame(A = rand(5), B = rand(5), C = rand(5), D = rand(5)) varlist = [B, C] for i = 1:length(varlist) rename!(df, convert(Symbol, varlist[i])), convert(Symbol, column_*string(i)) end
[julia-users] Re: Correctly accessing DataFrame columns in loop
Since you’re in control of varlist, you could simplify this at least once
by using symbols from the beginning. Next, you can use the symbol function
and a different overload of string instead of convert to make the
concat-convert operation a little less ugly:
julia using DataFrames
julia df = DataFrame(A = rand(5), B = rand(5), C = rand(5), D = rand(5))
5x4 DataFrames.DataFrame
| Row | A| B| C| D|
|-|--|--|--|--|
| 1 | 0.550768 | 0.464531 | 0.141101 | 0.754492 |
| 2 | 0.629269 | 0.100223 | 0.981175 | 0.035041 |
| 3 | 0.26019 | 0.962588 | 0.948283 | 0.51513 |
| 4 | 0.755892 | 0.202503 | 0.727609 | 0.255172 |
| 5 | 0.28018 | 0.328776 | 0.684717 | 0.502154 |
julia varlist = [:B,:C]
2-element Array{Symbol,1}:
:B
:C
julia for i in eachindex(varlist)
rename!(df, varlist[i], symbol(string(varlist[i], i)))
end
julia df
5x4 DataFrames.DataFrame
| Row | A| B1 | C2 | D|
|-|--|--|--|--|
| 1 | 0.550768 | 0.464531 | 0.141101 | 0.754492 |
| 2 | 0.629269 | 0.100223 | 0.981175 | 0.035041 |
| 3 | 0.26019 | 0.962588 | 0.948283 | 0.51513 |
| 4 | 0.755892 | 0.202503 | 0.727609 | 0.255172 |
| 5 | 0.28018 | 0.328776 | 0.684717 | 0.502154 |
I usually think of symbols as strings with some constraints, and using
one instead of the other then becomes quite trivial, conceptually.
// T
On Friday, May 8, 2015 at 8:27:18 PM UTC+2, Nils Gudat wrote:
I've been trying to rename a bunch of columns in a DataFrame and as a
former pandas user am a bit thrown off by the way columns are accessed
using symbols.
Let's say I have a list of column names (strings) which I want to all give
the same name with ascending integers at the end. After a few unsuccesful
attempts, I came up with the following, but I feel like there has to be a
less ugly way of doing this:
df = DataFrame(A = rand(5), B = rand(5), C = rand(5), D = rand(5))
varlist = [B, C]
for i = 1:length(varlist)
rename!(df, convert(Symbol, varlist[i])), convert(Symbol,
column_*string(i))
end
Re: [julia-users] Re: [Vectorization] Comparison of codes
@Steven, we are not talking about the same thing. You are talking about vectorizing the code, while I am talking about vectorizing the function. On Fri, May 8, 2015 at 8:23 PM, Steven G. Johnson stevenj@gmail.com wrote: On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] `include()` vs `require()`
Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
[julia-users] [ANN] NamedTuples.jl: Named Tuples for Julia
NamedTuples.jl provides an implementation of type safe named tuples for Julia (cf named tuples in python). NamedTuples may be used anywhere that a tuple is currently being used, for example in the definition of a method or as the return value of a method. NamedTuples are implemented using Julia’s macro system ensuring that the run time cost is equivalent to constructing a regular immutable type. NamedTuples may also be used in cases where a small typed immutable dictionary is desired. https://github.com/blackrock/NamedTuples.jl Any bugs or issues please reach out or open an issue. module Test using NamedTuples function foo( y ) a = 1 x = 3 return @NT( a = 1, b = world, c = hello, d=a/x, y = a/y ) end function bar( nt::@NT( a::Int64, c::ASCIIString )) return repeat( nt.c, nt.a ) end end Test.foo( 1 ) # Returns a NamedTuple of 5 elements Test.bar( @NT( a= 2, c=hello)) # Returns `hellohello`
[julia-users] Detecting free variables in an anonymous function
Say I have a quoted anonymous function, like :(x-y=1; y+x+z). I'd like to determine that 'z' is not bound within the function (ie, is a reference to an enclosing outer/global scope). What's a good way to do it? I know I could recursively analyze the Expr, keeping track of which variables were assigned to at each scope level introduced within the Expr, but that seems like I'd be duplicating effort that must implicitly be in Julia already and might be brittle with regard to subtle scoping rules.
[julia-users] Re: scipy min_cg example
Here is the example from the SciPy manual, translated into working Julia code: @pyimport scipy.optimize as so using PyCall args = (2, 3, 7, 8, 9, 10) function fun(x, args...) u, v = x a, b, c, d, e, f = args return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end function gradf(x, args...) u, v = x a, b, c, d, e, f = args gu = 2*a*u + b*v + d # u-component of the gradient gv = b*u + 2*c*v + e # v-component of the gradient return [gu, gv] end x0 = [0, 0] so.fmin_cg(fun, x0, fprime=gradf, args=args) I copied their style, but it would have been cleaner to write e.g. function fun(x, a, b, c, d, e, f) u, v = x return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end rather than using varargs. Of course, it would be cleaner to omit the whole args nonsense to begin with, and just write: so.fmin_cg(x - fun2(x, a, b, c, d, e, f), x0, fprime= x - gradf(x, a, b, c, d, e, f)) since, as I mentioned above, their args is just a clumsy workaround for lexical scoping, which both Python and Julia already have.
[julia-users] JuliaCon registrations open
Hello all, JuliaCon registrations are now open. http://www.juliacon.org/ -viral
Re: [julia-users] Re: [Vectorization] Comparison of codes
It's unclear to me what you mean by vectorizing the function. Can you explain? On Fri, May 8, 2015 at 2:55 PM, Zheng Wendell zhengwend...@gmail.com wrote: @Steven, we are not talking about the same thing. You are talking about vectorizing the code, while I am talking about vectorizing the function. On Fri, May 8, 2015 at 8:23 PM, Steven G. Johnson stevenj@gmail.com wrote: On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] Detecting free variables in an anonymous function
You can call expand() on the AST, and let the frontend lower the code into
a form that is easier to analyze. In particular:
```
julia expand( :(x-y=1; y+x+z) ).args[1].ast.args[2][1]
1-element Array{Any,1}:
:y
```
On Fri, May 8, 2015 at 2:41 PM Jonathan Malmaud malm...@gmail.com wrote:
Say I have a quoted anonymous function, like :(x-y=1; y+x+z). I'd like to
determine that 'z' is not bound within the function (ie, is a reference to
an enclosing outer/global scope). What's a good way to do it?
I know I could recursively analyze the Expr, keeping track of which
variables were assigned to at each scope level introduced within the Expr,
but that seems like I'd be duplicating effort that must implicitly be in
Julia already and might be brittle with regard to subtle scoping rules.
[julia-users] Re: Detecting free variables in an anonymous function
Well, the scoping analysis is also duplicated within the Debug package :) in fact that is what most of the code does. I've thought about if this should be more widely exposed; anyway it's not very hard to access if you're interested.
Re: [julia-users] `include()` vs `require()`
Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
[julia-users] Re: Detecting free variables in an anonymous function
You may have already spotted this, but just in case not: the expression :(x - y=1; y+x+z) will eval to a declaration of the anonymous function x - y=1 and an attempt to add x, y and z. You should write :( x - (y=1; y+x+z) ) if you intend both of the latter expressions to be a part of the anonymous function's body. On Friday, May 8, 2015 at 2:41:00 PM UTC-4, Jonathan Malmaud wrote: Say I have a quoted anonymous function, like :(x-y=1; y+x+z). I'd like to determine that 'z' is not bound within the function (ie, is a reference to an enclosing outer/global scope). What's a good way to do it? I know I could recursively analyze the Expr, keeping track of which variables were assigned to at each scope level introduced within the Expr, but that seems like I'd be duplicating effort that must implicitly be in Julia already and might be brittle with regard to subtle scoping rules.
[julia-users] asymmetry between treatment of mean and cov
Hello, 1. I can apply mean to an array of vectors, but doing the same for cov produces an error. 2. I can apply cov to a matrix, which produces the covariance matrix treating each row as an observation. Applying mean to the same matrix produces a scalar average of all elements in the matrix. This asymmetric treatment is counter-intuitive. What is the rationale? Thanks! n=10 A=Array(Vector,n) for i=1:n A[i]=randn(3) end println(mean(A)) println(cov(A)) # produces an error B=Array(Float64,n,3) for i=1:n B[i,:]=A[i] end println(mean:,mean(B))# produces average of all elements in B println(covariance matrix:,cov(B))# produces covariance matrix of columns of B
Re: [julia-users] `include()` vs `require()`
Yes, I've seen that. But it doesn't help. On Fri, May 8, 2015 at 2:19 PM, Stefan Karpinski ste...@karpinski.org wrote: help? reload search: reload prevfloat parsefloat Base.reload(file::AbstractString) Like require, except forces loading of files regardless of whether they have been loaded before. Typically used when interactively developing libraries. On Fri, May 8, 2015 at 4:13 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: OK. But then what is the difference between include and reload? On Fri, May 8, 2015 at 1:54 PM, Stefan Karpinski ste...@karpinski.org wrote: Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] `include()` vs `require()`
reload is like require but forces the the loading to happen again even if it's already happened once. include and reload are not related. On Fri, May 8, 2015 at 4:34 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: Yes, I've seen that. But it doesn't help. On Fri, May 8, 2015 at 2:19 PM, Stefan Karpinski ste...@karpinski.org wrote: help? reload search: reload prevfloat parsefloat Base.reload(file::AbstractString) Like require, except forces loading of files regardless of whether they have been loaded before. Typically used when interactively developing libraries. On Fri, May 8, 2015 at 4:13 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: OK. But then what is the difference between include and reload? On Fri, May 8, 2015 at 1:54 PM, Stefan Karpinski ste...@karpinski.org wrote: Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] asymmetry between treatment of mean and cov
Ah. I misread your post. The explanation is that the computation of the mean generalizes to the case where each element is a vector, but this is not the case for the way we calculate the variance. It could be supported, but then I think it would make most sense to generalize the variance and not the covariance function. 2015-05-08 17:05 GMT-04:00 JPi pin...@gmail.com: Yes, the variance. But that doesn't explain why you can't get the covariance matrix of an array of vectors. On Friday, May 8, 2015 at 4:51:13 PM UTC-4, Andreas Noack wrote: Calculating the covariance requires two sequences of data points. Either from two vectors or between the columns of a matrix. The mean is different as it requires one sequence. What did you expect to get from the covariance function of a vector? The variance? 2015-05-08 16:01 GMT-04:00 JPi pin...@gmail.com: Hello, 1. I can apply mean to an array of vectors, but doing the same for cov produces an error. 2. I can apply cov to a matrix, which produces the covariance matrix treating each row as an observation. Applying mean to the same matrix produces a scalar average of all elements in the matrix. This asymmetric treatment is counter-intuitive. What is the rationale? Thanks! n=10 A=Array(Vector,n) for i=1:n A[i]=randn(3) end println(mean(A)) println(cov(A)) # produces an error B=Array(Float64,n,3) for i=1:n B[i,:]=A[i] end println(mean:,mean(B))# produces average of all elements in B println(covariance matrix:,cov(B))# produces covariance matrix of columns of B
Re: [julia-users] [Vectorization] Comparison of codes
@Tim, can I explicitly deactivate the inline functionality ?
On Fri, May 8, 2015 at 7:11 PM, Tim Holy tim.h...@gmail.com wrote:
Yes, function get inlined within certain limits. You can check by using
@code_typed and looking for :call expressions (@code_typed includes the
inlining pass).
Best,
--Tim
On Friday, May 08, 2015 09:30:57 AM Sisyphuss wrote:
The following code is aimed to study the benefit of vectorization:
function f(a::Array{Float64,1})
b = Array(eltype(a),size(a))
n=length(a)
for i=1:n
b[i]=a[i]+1
end
return b
end
function f(a::Float64)
b = a + 1
end
function main1()
n = 100
a = zeros(n)
@timeit begin
b = Array(eltype(a),size(a))
for i=1:n
b[i] = f(a[i])
end
end
end
function main2()
n = 100
a = zeros(n)
@timeit b=f(a)
end
begin
println(==)
main1()
main2()
end
`f` is the overloaded function, which can be either vectorized or not.
The goal is to transform every element of the vector `a` to `f(a)`, and
store it in `b`.
`main1` is non-vecorized, while `main2` is vectorized.
I thought that `main2` could have been more efficient than `main1`,
because
it has less function calls.
However, according to the experiment, there is actually no performance
difference between these two versions.
The reason, is it because that `f()` is inlined by the compiler secretly?
Re: [julia-users] `include()` vs `require()`
help? reload search: reload prevfloat parsefloat Base.reload(file::AbstractString) Like require, except forces loading of files regardless of whether they have been loaded before. Typically used when interactively developing libraries. On Fri, May 8, 2015 at 4:13 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: OK. But then what is the difference between include and reload? On Fri, May 8, 2015 at 1:54 PM, Stefan Karpinski ste...@karpinski.org wrote: Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] asymmetry between treatment of mean and cov
Calculating the covariance requires two sequences of data points. Either from two vectors or between the columns of a matrix. The mean is different as it requires one sequence. What did you expect to get from the covariance function of a vector? The variance? 2015-05-08 16:01 GMT-04:00 JPi pin...@gmail.com: Hello, 1. I can apply mean to an array of vectors, but doing the same for cov produces an error. 2. I can apply cov to a matrix, which produces the covariance matrix treating each row as an observation. Applying mean to the same matrix produces a scalar average of all elements in the matrix. This asymmetric treatment is counter-intuitive. What is the rationale? Thanks! n=10 A=Array(Vector,n) for i=1:n A[i]=randn(3) end println(mean(A)) println(cov(A)) # produces an error B=Array(Float64,n,3) for i=1:n B[i,:]=A[i] end println(mean:,mean(B))# produces average of all elements in B println(covariance matrix:,cov(B))# produces covariance matrix of columns of B
Re: [julia-users] Re: [Vectorization] Comparison of codes
I guess the real benefit comes from being able to define f!(b, a), which means you can save the allocation of memory for the result (by re-using memory you have from elsewhere). For example, it could be done like this: function f!(b, a::Array) n = length(a) @assert n == length(b) for i in 1:n b[i] = f(a[i]) end b end Now, calling b = f!(similar(a), a) will be equivalent to your b = f(a) in the array case, but e.g. f!(a,a) will be faster (by a factor of 2-3 on my machine, for this trivial function), especially for large arrays. In idiomatic Julia, you’d probably define your f(a::Array) as f(a::Array) = f!(similar(a), a), and expose both f and f!, to give the user the opportunity to avoid the allocation if it suits their application. // T In this case, I think On Friday, May 8, 2015 at 8:55:32 PM UTC+2, Sisyphuss wrote: @Steven, we are not talking about the same thing. You are talking about vectorizing the code, while I am talking about vectorizing the function. On Fri, May 8, 2015 at 8:23 PM, Steven G. Johnson steve...@gmail.com javascript: wrote: On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] `include()` vs `require()`
OK. But then what is the difference between include and reload? On Fri, May 8, 2015 at 1:54 PM, Stefan Karpinski ste...@karpinski.org wrote: Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] asymmetry between treatment of mean and cov
Yes, the variance On Friday, May 8, 2015 at 4:51:13 PM UTC-4, Andreas Noack wrote: Calculating the covariance requires two sequences of data points. Either from two vectors or between the columns of a matrix. The mean is different as it requires one sequence. What did you expect to get from the covariance function of a vector? The variance? 2015-05-08 16:01 GMT-04:00 JPi pin...@gmail.com javascript:: Hello, 1. I can apply mean to an array of vectors, but doing the same for cov produces an error. 2. I can apply cov to a matrix, which produces the covariance matrix treating each row as an observation. Applying mean to the same matrix produces a scalar average of all elements in the matrix. This asymmetric treatment is counter-intuitive. What is the rationale? Thanks! n=10 A=Array(Vector,n) for i=1:n A[i]=randn(3) end println(mean(A)) println(cov(A)) # produces an error B=Array(Float64,n,3) for i=1:n B[i,:]=A[i] end println(mean:,mean(B))# produces average of all elements in B println(covariance matrix:,cov(B))# produces covariance matrix of columns of B
Re: [julia-users] asymmetry between treatment of mean and cov
Yes, the variance. But that doesn't explain why you can't get the covariance matrix of an array of vectors. On Friday, May 8, 2015 at 4:51:13 PM UTC-4, Andreas Noack wrote: Calculating the covariance requires two sequences of data points. Either from two vectors or between the columns of a matrix. The mean is different as it requires one sequence. What did you expect to get from the covariance function of a vector? The variance? 2015-05-08 16:01 GMT-04:00 JPi pin...@gmail.com javascript:: Hello, 1. I can apply mean to an array of vectors, but doing the same for cov produces an error. 2. I can apply cov to a matrix, which produces the covariance matrix treating each row as an observation. Applying mean to the same matrix produces a scalar average of all elements in the matrix. This asymmetric treatment is counter-intuitive. What is the rationale? Thanks! n=10 A=Array(Vector,n) for i=1:n A[i]=randn(3) end println(mean(A)) println(cov(A)) # produces an error B=Array(Float64,n,3) for i=1:n B[i,:]=A[i] end println(mean:,mean(B))# produces average of all elements in B println(covariance matrix:,cov(B))# produces covariance matrix of columns of B
Re: [julia-users] multiple inner constructors
Awesome! Thanks guys!
On 23:13, Fri, 08/05/2015 Kevin Squire kevin.squ...@gmail.com wrote:
You could also just write it using outer constructors. It is slightly
more verbose, but cleans up the A type nicely.
type A{T}
x::T
end
function A{T:Number}(y::T)
y 0 ? A{T}(zero(y)) : A{T}(y)
end
function A(y::Char)
y == 'a' ? A{Char}('b') : A{Char}(y)
end
julia A('a')
A{Char}('b')
julia A('C')
A{Char}('C')
julia A(-100)
A{Int64}(0)
julia A(10.4)
A{Float64}(10.4)
Cheers,
Kevin
On Fri, May 8, 2015 at 1:51 AM, Toivo Henningsson toivo@gmail.com
wrote:
I think this is fine. You could also have a helper function that does the
checking and is called by the inner constructor. If you define it inside
the type it will only be accessible to the inner constructor (and other
functions within the type).
Re: [julia-users] Re: ccall and ENV on Windows
On Saturday, May 9, 2015 at 1:11:22 AM UTC+10, Simon Byrne wrote:
FWIW, the relevant R Docs are here:
http://cran.r-project.org/doc/manuals/r-release/R-exts.html#Calling-R_002edll-directly
Unfortunately, they're not very helpful, just saying:
You may need to ensure that R_HOME/bin is in your PATH so the R DLLs are
found.
But I can't figure out why the julia process PATH wouldn't carry over
to the R one.
Your problem may not be the dll itself, it could be one of its
dependencies. Also the windows dll search path is kind of complex
(see
https://msdn.microsoft.com/en-us/library/windows/desktop/ms682586(v=vs.85).aspx)
and PATH is only included in some of the cases.
On 8 May 2015 at 15:32, Simon Byrne simon...@gmail.com javascript:
wrote:
I tried it (adding both the extra PATH path, and the path of the
library it couldn't find), but that didn't seem to help either.
On 8 May 2015 at 15:17, Isaiah Norton isaiah...@gmail.com javascript:
wrote:
I'm also a bit surprised that it doesn't work. You could try calling
AddDllDirectory first:
https://msdn.microsoft.com/en-us/library/windows/desktop/hh310513(v=vs.85).aspx
On Fri, May 8, 2015 at 10:12 AM, Simon Byrne simon...@gmail.com
javascript: wrote:
As a follow up, here's some code (which requires that R be installed):
ENV[PATH] = ENV[PATH]*;C:\\Program Files\\R\\R-3.2.0\\bin\\x64\\
const libR = C:\\Program Files\\R\\R-3.2.0\\bin\\x64\\R.dll
argv = [REmbed]
println(ccall((:Rf_initEmbeddedR,libR),Cint,(Cint,Ptr{Ptr{Uint8}}),length(argv),argv))
If I run this as a script, I get an error message (unable to load
shared
library). However if I first set
PATH=PATH;C:\Program Files\R\R-3.2.0\bin\x64
then run the script (from within the same command prompt session), the
error goes away.
Any ideas?
Simon
On Friday, 8 May 2015 09:55:29 UTC+1, Simon Byrne wrote:
I'm trying to understand how ENV works (at least on Windows):
I'm trying to ccall a library that requires a particular addition to
PATH. If I do this externally (through the Windows menus) it works
okay,
but not via ENV[PATH]. I assume that this means that ENV only
changes the
local process? If so, is there anyway I can modify the system
variable from
within Julia?
-Simon
Re: [julia-users] Re: [Vectorization] Comparison of codes
We are thinking about the same thing ! On Fri, May 8, 2015 at 9:14 PM, Tomas Lycken tomas.lyc...@gmail.com wrote: I guess the real benefit comes from being able to define f!(b, a), which means you can save the allocation of memory for the result (by re-using memory you have from elsewhere). For example, it could be done like this: function f!(b, a::Array) n = length(a) @assert n == length(b) for i in 1:n b[i] = f(a[i]) end b end Now, calling b = f!(similar(a), a) will be equivalent to your b = f(a) in the array case, but e.g. f!(a,a) will be faster (by a factor of 2-3 on my machine, for this trivial function), especially for large arrays. In idiomatic Julia, you’d probably define your f(a::Array) as f(a::Array) = f!(similar(a), a), and expose both f and f!, to give the user the opportunity to avoid the allocation if it suits their application. // T In this case, I think On Friday, May 8, 2015 at 8:55:32 PM UTC+2, Sisyphuss wrote: @Steven, we are not talking about the same thing. You are talking about vectorizing the code, while I am talking about vectorizing the function. On Fri, May 8, 2015 at 8:23 PM, Steven G. Johnson steve...@gmail.com wrote: On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] Re: [Vectorization] Comparison of codes
It's a bit hard to explain. Typically, if we want to apply a filter/function to *n* objects one by one, we have two choices. Choice 1 is to literally loop over these objects and apply the function one by one. Choice 2 is that the function actually itself can accept multiple objects, and it has a loop *inside* the function to filter it one by one. The former is non-vectorized function, the latter is vectorized function. As I know, for Choice 1, we can use the *map* function to make the filter accept multiple objects. And it seems there is also a macro which do that. But if this filter takes more than one parameter, this trick is not likely to hold. I asked this question in order to know whether I should define two versions of function: one for scalar and the other for array. On Fri, May 8, 2015 at 9:12 PM, Stefan Karpinski ste...@karpinski.org wrote: It's unclear to me what you mean by vectorizing the function. Can you explain? On Fri, May 8, 2015 at 2:55 PM, Zheng Wendell zhengwend...@gmail.com wrote: @Steven, we are not talking about the same thing. You are talking about vectorizing the code, while I am talking about vectorizing the function. On Fri, May 8, 2015 at 8:23 PM, Steven G. Johnson stevenj@gmail.com wrote: On Friday, May 8, 2015 at 12:30:57 PM UTC-4, Sisyphuss wrote: The following code is aimed to study the benefit of vectorization: (Don't assume that vectorization is beneficial. In general, vectorizing your code in Julia leads to slower code, not faster code. e.g. compare x = y + z + 3y + 4z^2, where y and z are vectors, to writing out the loops. Vectorizing is much faster in languages like Python mainly because the loops are slow.)
Re: [julia-users] multiple inner constructors
You could also just write it using outer constructors. It is slightly more
verbose, but cleans up the A type nicely.
type A{T}
x::T
end
function A{T:Number}(y::T)
y 0 ? A{T}(zero(y)) : A{T}(y)
end
function A(y::Char)
y == 'a' ? A{Char}('b') : A{Char}(y)
end
julia A('a')
A{Char}('b')
julia A('C')
A{Char}('C')
julia A(-100)
A{Int64}(0)
julia A(10.4)
A{Float64}(10.4)
Cheers,
Kevin
On Fri, May 8, 2015 at 1:51 AM, Toivo Henningsson toivo@gmail.com
wrote:
I think this is fine. You could also have a helper function that does the
checking and is called by the inner constructor. If you define it inside
the type it will only be accessible to the inner constructor (and other
functions within the type).
[julia-users] Re: scipy min_cg example
This comes up pretty often with Optim as well, making a closure works fine when you're only solving a single problem with a single set of parameters, but it's a little cumbersome when you have a small set of parameter values that you're trying to loop over many times. David, can you give a sketch of the funky workaround you were using here? You wound up creating a one-element array that you were using to transmit the index into the parameter data vector into the inner-loop function, or something like that, right? On Friday, May 8, 2015 at 11:49:48 AM UTC-7, Steven G. Johnson wrote: Here is the example from the SciPy manual, translated into working Julia code: @pyimport scipy.optimize as so using PyCall args = (2, 3, 7, 8, 9, 10) function fun(x, args...) u, v = x a, b, c, d, e, f = args return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end function gradf(x, args...) u, v = x a, b, c, d, e, f = args gu = 2*a*u + b*v + d # u-component of the gradient gv = b*u + 2*c*v + e # v-component of the gradient return [gu, gv] end x0 = [0, 0] so.fmin_cg(fun, x0, fprime=gradf, args=args) I copied their style, but it would have been cleaner to write e.g. function fun(x, a, b, c, d, e, f) u, v = x return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end rather than using varargs. Of course, it would be cleaner to omit the whole args nonsense to begin with, and just write: so.fmin_cg(x - fun2(x, a, b, c, d, e, f), x0, fprime= x - gradf(x, a, b, c, d, e, f)) since, as I mentioned above, their args is just a clumsy workaround for lexical scoping, which both Python and Julia already have.
Re: [julia-users] Re: scipy min_cg example
In the context of Optim, I sketched an approach in https://github.com/JuliaOpt/Optim.jl/issues/102#issuecomment-74658825 that is now trivial with FastAnonymous if you're running 0.4: https://github.com/timholy/FastAnonymous.jl/blob/master/doc/README_0.4.md#changing-parameter-values But someone needs to modify Optim so it's not so picky about the input function. --Tim On Friday, May 08, 2015 06:57:28 PM Tony Kelman wrote: This comes up pretty often with Optim as well, making a closure works fine when you're only solving a single problem with a single set of parameters, but it's a little cumbersome when you have a small set of parameter values that you're trying to loop over many times. David, can you give a sketch of the funky workaround you were using here? You wound up creating a one-element array that you were using to transmit the index into the parameter data vector into the inner-loop function, or something like that, right? On Friday, May 8, 2015 at 11:49:48 AM UTC-7, Steven G. Johnson wrote: Here is the example from the SciPy manual, translated into working Julia code: @pyimport scipy.optimize as so using PyCall args = (2, 3, 7, 8, 9, 10) function fun(x, args...) u, v = x a, b, c, d, e, f = args return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end function gradf(x, args...) u, v = x a, b, c, d, e, f = args gu = 2*a*u + b*v + d # u-component of the gradient gv = b*u + 2*c*v + e # v-component of the gradient return [gu, gv] end x0 = [0, 0] so.fmin_cg(fun, x0, fprime=gradf, args=args) I copied their style, but it would have been cleaner to write e.g. function fun(x, a, b, c, d, e, f) u, v = x return a*u^2 + b*u*v + c*v^2 + d*u + e*v + f end rather than using varargs. Of course, it would be cleaner to omit the whole args nonsense to begin with, and just write: so.fmin_cg(x - fun2(x, a, b, c, d, e, f), x0, fprime= x - gradf(x, a, b, c, d, e, f)) since, as I mentioned above, their args is just a clumsy workaround for lexical scoping, which both Python and Julia already have.
[julia-users] Compile ReadStat under Windows7
Hi all I would like to be able to import SAS files to Julia (*.sas7bdat). I found https://github.com/WizardMac/DataRead.jl/ which apparently uses ReadStat https://github.com/WizardMac/ReadStat It seems to use Ragel /usr/local/bin/ragel What is the easiest way to make DataRead.jl work on Julia Windows? Bernhard
[julia-users] Re: writing a DataFrame consisting of DataFrames to a table
csv is really only meant for simple two dimensional data. A DataFrame inside a DataFrame is more complex than what the csv reader know how to deal with. If you want to store something like that you will have to use some other file format, like JLD. On Friday, May 8, 2015 at 11:56:05 AM UTC+3, axsk wrote: I need to write and then read a DataFrame which consists of other DataFrames to a file. df=DataFrame(d=DataFrame(x=[1:5])) gives the desired DataFrame, but writetable(df.csv,df) readtable(df.csv) returns: 1x1 DataFrames.DataFrame | Row | d | |-|-| | 1 | 5x1 DataFrames.DataFrame\n| Row | x |\n|-|---|\n| 1 | 1 |\n| 2 | 2 |\n| 3 | 3 |\n| 4 | 4 |\n| 5 | 5 | | Thus the interiour DataFrame gets stored as string in the file. Is there anyway to properly read the inner DataFrame?
Re: [julia-users] [Vectorization] Comparison of codes
Yes, if you're on 0.4. Preface the function with @noinline.
--Tim
On Friday, May 08, 2015 11:25:24 PM Zheng Wendell wrote:
@Tim, can I explicitly deactivate the inline functionality ?
On Fri, May 8, 2015 at 7:11 PM, Tim Holy tim.h...@gmail.com wrote:
Yes, function get inlined within certain limits. You can check by using
@code_typed and looking for :call expressions (@code_typed includes the
inlining pass).
Best,
--Tim
On Friday, May 08, 2015 09:30:57 AM Sisyphuss wrote:
The following code is aimed to study the benefit of vectorization:
function f(a::Array{Float64,1})
b = Array(eltype(a),size(a))
n=length(a)
for i=1:n
b[i]=a[i]+1
end
return b
end
function f(a::Float64)
b = a + 1
end
function main1()
n = 100
a = zeros(n)
@timeit begin
b = Array(eltype(a),size(a))
for i=1:n
b[i] = f(a[i])
end
end
end
function main2()
n = 100
a = zeros(n)
@timeit b=f(a)
end
begin
println(==)
main1()
main2()
end
`f` is the overloaded function, which can be either vectorized or not.
The goal is to transform every element of the vector `a` to `f(a)`, and
store it in `b`.
`main1` is non-vecorized, while `main2` is vectorized.
I thought that `main2` could have been more efficient than `main1`,
because
it has less function calls.
However, according to the experiment, there is actually no performance
difference between these two versions.
The reason, is it because that `f()` is inlined by the compiler
secretly?
Re: [julia-users] `include()` vs `require()`
Also see the issue logged here: https://github.com/JuliaLang/julia/issues/8000 On Fri, May 8, 2015 at 2:19 PM, Stefan Karpinski ste...@karpinski.org wrote: help? reload search: reload prevfloat parsefloat Base.reload(file::AbstractString) Like require, except forces loading of files regardless of whether they have been loaded before. Typically used when interactively developing libraries. On Fri, May 8, 2015 at 4:13 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: OK. But then what is the difference between include and reload? On Fri, May 8, 2015 at 1:54 PM, Stefan Karpinski ste...@karpinski.org wrote: Why wouldn't it work in the REPL? It means paste the contents of this file here. On Fri, May 8, 2015 at 3:46 PM, Bob Nnamtrop bob.nnamt...@gmail.com wrote: include is just about splitting a single file into multiple pieces Ok. But then why does include work at the REPL prompt? What is the difference between include and reload? I really think there are too many ways to do the same thing. Bob On Fri, May 8, 2015 at 11:58 AM, Stefan Karpinski ste...@karpinski.org wrote: Conceptually, they are quite different: - using/import/require are for acquiring shared resources – i.e. modules that different bits of code might all independently want to use. - include is just about splitting a single source file into multiple pieces. Not sure if that helps. On Fri, May 8, 2015 at 1:28 PM, Jack Minardi j...@minardi.org wrote: Thanks guys. I guess I thought only require didn't load twice as it is specifically mentioned in the doc string.
Re: [julia-users] `include()` vs `require()`
Eg include loads choose into the current module but reload/require/using/import loads it at the top level scope (actually in the Main module I guess, and when you create a module from within that it becomes a top level module).
