Re: [julia-users] Matlab minfunc - Optim.jl where the function evaluates the main cost function and its gradient simultaneously

2014-08-13 Thread John Myles White
That's not quite right: the highest performance case requires that you have three separate functions: (1) f, which only calculates the objective function as a pure function returning a Float64 value (2) g! which only calculates the gradient by mutating a Float64 array (3) fg!, which calculates

Re: [julia-users] Re: Package Popularity Ranking

2014-08-13 Thread John Myles White
This was a really nice read, even you can tell the author’s spent time with social scientists based on their assertion that a correlation of 0.3 is low. — John On Aug 13, 2014, at 6:41 PM, Samuel Jenkins svs14.41...@gmail.com wrote: Since this is on the topic of package popularity, this blog

Re: [julia-users] Gibbs sampling 10x slower than c++

2014-08-13 Thread John Myles White
Here are some things you might try: (1) Don’t use a global const. Pass N as an argument to all functions. (2) Use in-place multiplication via functions like A_mul_B! in lines like `@inbounds a = dot(W[:,n], state)`. In Julia 0.3, W[:,n] creates a copy of that column of W on every iteration.

Re: [julia-users] Fully Typed Function As Argument Type

2014-08-14 Thread John Myles White
Not possible in the current versions of Julia. Maybe one day. There are bunch of us who’d like to have this functionality, but it’s a non-trivial addition of complexity to the compiler. — John On Aug 14, 2014, at 4:59 AM, Chris Kellendonk chriskellend...@gmail.com wrote: I'm new to the

Re: [julia-users] pdf of a multivariate distribution

2014-08-14 Thread John Myles White
In Julia, anything with a description like 1x27 Array{Any,2} is a matrix, not a vector. Also, it's not the right kind of vector because it has no numeric type restriction. It needs to be a vector of Float64 values, like you'd get from doing something like: [1.0, 2.0, 3.0] -- John On Aug

Re: [julia-users] pdf of a multivariate distribution

2014-08-14 Thread John Myles White
, 1}? Thanks again! Pam. On Thursday, August 14, 2014 5:15:01 PM UTC+1, John Myles White wrote: In Julia, anything with a description like 1x27 Array{Any,2} is a matrix, not a vector. Also, it's not the right kind of vector because it has no numeric type restriction. It needs

[julia-users] Stabilizer LLVM Plugin

2014-08-14 Thread John Myles White
I just heard a talk about this (somewhat out-of-date) LLVM plugin for doing performance testing in a way that randomizes over possible memory layouts: https://github.com/ccurtsinger/stabilizer If the plugin gets updated for a newer version of LLVM it might be a fun project to try using this

Re: [julia-users] Typo in local variable name not detected

2014-08-14 Thread John Myles White
I think Lint.jl might handle this already: https://github.com/tonyhffong/Lint.jl -- John On Aug 14, 2014, at 2:39 PM, Ethan Anderes ethanande...@gmail.com wrote: It's funny, this issue is by far my biggest source of bugs when coding in Julia. I find myself often prototyping things with

Re: [julia-users] Vectorised boolean access to array elements

2014-08-15 Thread John Myles White
a . 5 On Aug 15, 2014, at 3:53 AM, David Higgins daithiohuig...@gmail.com wrote: Hi, Is there a mechanism for vectorised logical access to array elements in Julia? I basically mean, is there an equivalent to the Matlab notation a[a5] which should give all of the elements of a[] which

Re: [julia-users] moving from v0.3 to v0.4

2014-08-15 Thread John Myles White
I agree: I think most people should not use 0.4 beause it’s going to change so frequently. — John On Aug 15, 2014, at 12:28 AM, Tobias Knopp tobias.kn...@googlemail.com wrote: I think we should discourage using 0.4-dev version until feature freeze. There might be a lot of package breakage

Re: [julia-users] Best practice to type-check for array of floats?

2014-08-15 Thread John Myles White
Hi Brendan, It looks like you’re hitting Julia’s invariance for the first time: http://en.wikipedia.org/wiki/Covariance_and_contravariance_(computer_science) If you’re working with inputs to a function, you can do the typecheck using a type parameter, T: function foo{T :

Re: [julia-users] Best practice to type-check for array of floats?

2014-08-15 Thread John Myles White
, you get invariance. — John On Aug 15, 2014, at 7:58 PM, Brendan O'Connor breno...@cs.umass.edu wrote: On Fri, Aug 15, 2014 at 9:53 PM, John Myles White johnmyleswh...@gmail.com wrote: Hi Brendan, It looks like you’re hitting Julia’s invariance for the first time: http

Re: [julia-users] julia syntax

2014-08-16 Thread John Myles White
Try this section of the manual: http://julia.readthedocs.org/en/latest/manual/networking-and-streams/ — John On Aug 16, 2014, at 6:52 PM, frbertonce...@gmail.com wrote: Hi all, just start to look at this julia but it seems there is not syntax for I/O data.. Does anyone know if julia

Re: [julia-users] IJulia Notebook of Chapter 1 of Higham Higham's MATLAB Guide

2014-08-17 Thread John Myles White
This is great. Thanks for sharing this. — John On Aug 17, 2014, at 2:35 PM, Don MacMillen don.macmil...@gmail.com wrote: I've made a Julia version that is available here http://bit.ly/1pi53Ih I hope you find it useful.

Re: [julia-users] Re: Capitalization of package names

2014-08-18 Thread John Myles White
I can probably register KNN.jl and deprecate kNN.jl. — John On Aug 18, 2014, at 5:45 AM, Iain Dunning iaindunn...@gmail.com wrote: Yes, it is the Julia convention. There is only one (registered) exception, which is prehistoric in Julia terms (kNN.jl) On Monday, August 18, 2014 8:32:40

Re: [julia-users] Re: Capitalization of package names

2014-08-18 Thread John Myles White
systems like HFS+ on Macs. On Aug 18, 2014, at 10:08 AM, John Myles White johnmyleswh...@gmail.com wrote: I can probably register KNN.jl and deprecate kNN.jl. — John On Aug 18, 2014, at 5:45 AM, Iain Dunning iaindunn...@gmail.com wrote: Yes, it is the Julia convention. There is only

Re: [julia-users] Simple Integer DataFrame Automatic Conversion to Float (InexactError())

2014-08-18 Thread John Myles White
Hi Bradley, Would you consider using DataArrays for this? DataFrames no longer support these operations, so any upgrade in your setup would turn all of this code into errors. All of these operations work on DataArrays already. — John On Aug 18, 2014, at 7:28 AM, Bradley Setzler

Re: [julia-users] Simple Integer DataFrame Automatic Conversion to Float (InexactError())

2014-08-18 Thread John Myles White
, 2014 9:31:43 AM UTC-5, John Myles White wrote: Hi Bradley, Would you consider using DataArrays for this? DataFrames no longer support these operations, so any upgrade in your setup would turn all of this code into errors. All of these operations work on DataArrays already. — John

[julia-users] Suggestion: Start using Julia 0.3-RC4

2014-08-18 Thread John Myles White
One of the best things you can do for the Julia community is to start using the release candidates of 0.3. Everyone will be better off if bugs are found now rather than when 0.3 is released. You can help make 0.3 a better release by installing one of the latest binaries and testing it out.

Re: [julia-users] Simple Integer DataFrame Automatic Conversion to Float (InexactError())

2014-08-18 Thread John Myles White
not convert if divided by Integer. Best, Bradley On Monday, August 18, 2014 9:31:43 AM UTC-5, John Myles White wrote: Hi Bradley, Would you consider using DataArrays for this? DataFrames no longer support these operations, so any upgrade in your setup would turn all of this code

Re: [julia-users] How to require the current master of pacakge

2014-08-19 Thread John Myles White
I don't think this is possible. This sounds like a case where PDMats needs a new release. You can implement a release yourself, make a pull request to METADATA and ask Dahua for permission to merge it. -- John On Aug 19, 2014, at 10:13 AM, Spencer Lyon spencerly...@gmail.com wrote: Is there

Re: [julia-users] From array of Char to String

2014-08-19 Thread John Myles White
We should really have a FAQ for this. This is something that tripped me up for a while. I’ll write up sometime. — John On Aug 19, 2014, at 9:11 PM, Isaiah Norton isaiah.nor...@gmail.com wrote: That is the splice operator (also known as splat). It causes the contents of `a` to be passed as

Re: [julia-users] Re: needless loss of performance: immutable conflates two concepts

2014-08-20 Thread John Myles White
Copying things by default isn’t a very good strategy for a language used to do linear algebra on large arrays. Requiring that people use Ref to get reasonable performance for linear algebra operations would make Julia feel like a much more low-level language than it is currenty. — John On

Re: [julia-users] Optim.jl line search problems

2014-08-20 Thread John Myles White
? It looks like there is some debugging code in there but I'm not sure how to turn it on. -thom On Wednesday, July 30, 2014 6:24:26 PM UTC-5, John Myles White wrote: I’ve never seen our line search methods produce an error that wasn’t caused by errors in the gradient. The line search methods

Re: [julia-users] Optim.jl line search problems

2014-08-20 Thread John Myles White
search routines? It looks like there is some debugging code in there but I'm not sure how to turn it on. -thom On Wednesday, July 30, 2014 6:24:26 PM UTC-5, John Myles White wrote: I’ve never seen our line search methods produce an error that wasn’t caused by errors in the gradient

Re: [julia-users] Is there a REPL shortcut for a subscript numeral?

2014-08-20 Thread John Myles White
Try typing the sequence \_1TAB — John On Aug 20, 2014, at 8:08 AM, Douglas Bates dmba...@gmail.com wrote: I find it convenient to use identifiers like julia a₁ = 2 2 In the iESS emacs mode for a Julia session I can set the input-mode to TeX and type a_1 to create this identifier.

Re: [julia-users] keyword argument function masking non-kwarg function

2014-08-21 Thread John Myles White
Unfortunately, the rule is simple: dispatch never looks at keyword arguments. -- John On Aug 21, 2014, at 8:59 AM, Spencer Lyon spencerly...@gmail.com wrote: Consider the following code: function periodogram(x::Array) n = length(x) I_w = abs(fft(x)).^2 ./ n w = 2pi * [0:n-1]

Re: [julia-users] RFC: Colored output API

2014-08-22 Thread John Myles White
Abstractly, I’d love to be able to print using Color.jl. It seems like a much cleaner interface. But it’s not clear to me that Color.jl buys you a lot if your terminal doesn’t support a widerange of colors. — John On Aug 22, 2014, at 3:33 PM, Spencer Russell spencer.f.russ...@gmail.com

Re: [julia-users] possible timing regression in A\b

2014-08-23 Thread John Myles White
a = rand(5000,5000); b = rand(5000); x =a\b end; elapsed time: 30.509950844 seconds (400100776 bytes allocated, 0.11% gc time) It's mystery to me. If I learn anything more, will let you know. Thanks again. Don On Friday, August 22, 2014 9:54:22 PM UTC-7, John Myles White wrote: I don’t

Re: [julia-users] Multivariate Normal versus Multivariate Normal Canon in Distributions package

2014-08-24 Thread John Myles White
Can you clarify a bit what you mean by “same result”? It would also be great to see a simpler example that doesn’t involve the Bayesian updating steps, which hopefully aren’t essential to hitting an error. — John On Aug 23, 2014, at 4:33 PM, asim yahooans...@gmail.com wrote: I am trying

Re: [julia-users] Is it type-stable?

2014-08-24 Thread John Myles White
: That is no longer valid under in Julia 0.3. The code only runs 3x faster and the type stable code looks a bit more complex. On Monday, December 16, 2013 8:48:40 PM UTC+2, John Myles White wrote: I think Michael is suggesting that it would help to have an automated tool to do this. This stuff

Re: [julia-users] Is it type-stable?

2014-08-24 Thread John Myles White
looks a bit more complex. On Monday, December 16, 2013 8:48:40 PM UTC+2, John Myles White wrote: I think Michael is suggesting that it would help to have an automated tool to do this. This stuff comes up in subtle ways in complex code. While editing some of the t-SNE code someone posted

Re: [julia-users] Re: What's new in 0.3?

2014-08-25 Thread John Myles White
The NEWS.md file does cover this: • Ranges and arrays with the same elements are now unequal. This allows hashing and comparing ranges to be faster. (#5778) On Aug 25, 2014, at 8:45 AM, Ed Scheinerman edward.scheiner...@gmail.com wrote: Thanks again for the pointer to the release notes.

Re: [julia-users] Re: Does Julia have something similar to Python's documentation string?

2014-08-25 Thread John Myles White
The issue is that you want to have all code documentation show up in REPL. In the GoDoc approach, this might require an explicit build step -- which is a non-trivial cost in usability. -- John On Aug 25, 2014, at 3:01 PM, Job van der Zwan j.l.vanderz...@gmail.com wrote: On Monday, 25 August

Re: [julia-users] Multivariate Normal versus Multivariate Normal Canon in Distributions package

2014-08-25 Thread John Myles White
This looks like a failure to find functions from NumericFuns. What versions of Julia and stats packages are you using? -- John On Aug 25, 2014, at 9:03 AM, asim yahooans...@gmail.com wrote: mvCanonNotebook.pdf

Re: [julia-users] Package rename: Distance == Distances

2014-08-25 Thread John Myles White
Dahua, did you keep the original package around? — John On Aug 25, 2014, at 10:07 PM, Dahua Lin linda...@gmail.com wrote: Following Julia package naming convention, the package Distance was renamed to Distances. New package page: https://github.com/JuliaStats/Distances.jl All

Re: [julia-users] Package rename: Distance == Distances

2014-08-25 Thread John Myles White
Dahua On Tuesday, August 26, 2014 1:08:48 PM UTC+8, John Myles White wrote: Dahua, did you keep the original package around? — John On Aug 25, 2014, at 10:07 PM, Dahua Lin lind...@gmail.com wrote: Following Julia package naming convention, the package Distance was renamed

Re: [julia-users] Does Julia have something similar to Python's documentation string?

2014-08-26 Thread John Myles White
...@gmail.com wrote: On Tuesday, 26 August 2014 00:04:41 UTC+2, John Myles White wrote: The issue is that you want to have all code documentation show up in REPL. In the GoDoc approach, this might require an explicit build step -- which is a non-trivial cost in usability. -- John I assume you

Re: [julia-users] Simple Integer DataFrame Automatic Conversion to Float (InexactError())

2014-08-26 Thread John Myles White
1.0 1.5 2.0 julia A/2. 2x2 DataArray{Float64,2}: 0.5 1.0 1.5 2.0 julia A/2 InexactError() So it converts to Float if divided by Float, but does not convert if divided by Integer. Best, Bradley On Monday, August 18, 2014 9:31:43 AM UTC-5, John Myles White wrote: Hi Bradley

Re: [julia-users] Pushing Items to Dict

2014-08-27 Thread John Myles White
Hi Phillip, Could you provide a more complete example of what you're doing? In particular, I'm very confused by the idea of pushing items to a dict, which is an unordered data structure. Here is how dictionaries work in Julia: d = Dict() d[foo] = bar sizehint(d, 100) d[bar] = foo -- John

Re: [julia-users] sum( itr, dims )

2014-08-27 Thread John Myles White
sum(A, 1) works, as does sum(A, (1, )). More generally, sum(A, dims::Integer...) works, as does as sum(A, (dims::Integer...), ). -- John On Aug 27, 2014, at 2:44 PM, Andrew Dabrowski unhandya...@gmail.com wrote: In the doc for the Standard library I see: sum(A, dims) Sum elements of an

Re: [julia-users] Does Julia have something similar to Python's documentation string?

2014-08-27 Thread John Myles White
as a separate pass. That doesn't strike me as completely out of the question, but wouldn't be optimal. On Tue, Aug 26, 2014 at 11:32 AM, John Myles White johnmyl...@gmail.com wrote: No, I was talking about what I understood to be a design principle of GoDoc: doc generation and parsing occurs

Re: [julia-users] Does Julia have something similar to Python's documentation string?

2014-08-27 Thread John Myles White
formats. On Wednesday, August 27, 2014, John Myles White johnmyleswh...@gmail.com wrote: Ok, thanks for clarifying. I also like the idea of strategically placed comments as automatic documentation. -- John On Aug 27, 2014, at 2:54 PM, Job van der Zwan j.l.vanderz...@gmail.com wrote

Re: [julia-users] Multivariate Normal versus Multivariate Normal Canon in Distributions package

2014-08-27 Thread John Myles White
! function when generating multiple draws from the distribution, and this function is not found. I am not sure where this function is, either. Thanks Asim On Monday, August 25, 2014 6:14:40 PM UTC-4, John Myles White wrote: This looks like a failure to find functions from NumericFuns

Re: [julia-users] Does Julia have something similar to Python's documentation string?

2014-08-28 Thread John Myles White
As we're starting to get better ideas for a documentation system, two questions I have are how we do two things: (1) Handle documentation of generic functions and their specialized methods without requiring documentation of all specialized methods. (2) Handle documentation of functions that

Re: [julia-users] @inbounds is broken?

2014-08-29 Thread John Myles White
In theory, the compiler could potentially pull the bounds check out of the loop since the loop parameters are run-time constants, which means that the bounds check doesn't actually need to happen on each access. -- John On Aug 29, 2014, at 1:21 PM, Ed Scheinerman edward.scheiner...@gmail.com

Re: [julia-users] what is Julia for R cut()

2014-08-29 Thread John Myles White
DataArrays has a cut function. -- John On Aug 29, 2014, at 11:17 AM, Florian Oswald florian.osw...@gmail.com wrote: hi what's the julia equivalent of this R call? i don't need the levels and labels, just some kind of grouping index. cut(sample(1:10,10,TRUE),c(0,3,6,10)) [1] (3,6]

Re: [julia-users] what is Julia for R cut()

2014-08-30 Thread John Myles White
be situations where several packages/functions do the same thing. But it's hard for the user to find find simple things like that I think. let me know what you think. On 29 August 2014 22:29, John Myles White johnmyleswh...@gmail.com wrote: DataArrays has a cut function. -- John On Aug 29

Re: [julia-users] Blas trsv function: Argument mismatch between function and documentation (?)

2014-08-30 Thread John Myles White
Hi Asim, It’s a little hard to work with PDF’s. Would you consider using Gists? (https://gist.github.com) — John On Aug 30, 2014, at 1:47 PM, asim yahooans...@gmail.com wrote: Hi The Blas trsv function is described as needing 7 arguments in the documentation. However, it only appears

Re: [julia-users] Blas trsv function: Argument mismatch between function and documentation (?)

2014-08-30 Thread John Myles White
I can’t speak for others, but I’m very hesitant to download any kind of files from mailing lists. — John On Aug 30, 2014, at 2:00 PM, asim yahooans...@gmail.com wrote: Would this help? Asim On Saturday, August 30, 2014 4:49:23 PM UTC-4, John Myles White wrote: Hi Asim, It’s

Re: [julia-users] Re: Julia talk at EuroSciPy 2014

2014-08-30 Thread John Myles White
This might need to be part of the Zen of Julia. — John On Aug 30, 2014, at 2:11 PM, Jameson Nash vtjn...@gmail.com wrote: calling eval in a macro doesn't do what you think it does, so it doesn't do what you want On Sat, Aug 30, 2014 at 5:05 PM, Don MacMillen don.macmil...@gmail.com

Re: [julia-users] Issue Replacing NaN with 0

2014-08-31 Thread John Myles White
I don’t think this example had any views. Both bindings had an equal right to be considered the true binding. I think we’re better off doing more education to teach people to distinguish bindings and values. — John On Aug 31, 2014, at 11:27 AM, Ethan Anderes ethanande...@gmail.com wrote:

Re: [julia-users] Re: Has anyone successfully performed probit or logit regression in Julia?

2014-08-31 Thread John Myles White
Bradley, it’s especially easy to edit documentation because you can make a Pull Request right from the website. — John On Aug 31, 2014, at 11:30 AM, Bradley Setzler bradley.setz...@gmail.com wrote: Thank you Adam, this works. Let me suggest that this information be included in the GLM

Re: [julia-users] Re: Has anyone successfully performed probit or logit regression in Julia?

2014-08-31 Thread John Myles White
Merged. Thanks, Bradley. — John On Aug 31, 2014, at 12:29 PM, Bradley Setzler bradley.setz...@gmail.com wrote: Thank you for suggesting this, John. https://github.com/JuliaStats/GLM.jl/pull/90 Bradley On Sunday, August 31, 2014 1:33:04 PM UTC-5, John Myles White wrote: Bradley

Re: [julia-users] Has anyone successfully performed probit or logit regression in Julia?

2014-08-31 Thread John Myles White
something like a series option for non-parametric regression, glm(Y,X,data,family,link,seriesRank=2) where seriesRank=2 means all of the terms X1.^2, X1.*X2, X1.*X3,...,X5.^2 are included as regressors. Bradley On Sunday, August 31, 2014 2:32:30 PM UTC-5, John Myles White wrote: Merged

Re: [julia-users] Issue Replacing NaN with 0

2014-08-31 Thread John Myles White
I think there’s a broad issue that need resolution: how do you know when a function’s output takes control of the memory used by its arguments? — John On Aug 31, 2014, at 11:45 AM, Ethan Anderes ethanande...@gmail.com wrote: Yeah, I can see your point John. It's probably not reasonable to

Re: [julia-users] Has anyone successfully performed probit or logit regression in Julia?

2014-08-31 Thread John Myles White
.^2 X2.^2 X3.^2 X1.*X2 X1.*X3 X2.*X3 X1.*X2.*X3 Bradley On Sunday, August 31, 2014 2:55:22 PM UTC-5, John Myles White wrote: Bradley, you’re forgetting about interactions terms. — John On Aug 31, 2014, at 12:53 PM, Bradley Setzler bradley...@gmail.com wrote: No problem

Re: [julia-users] Has anyone successfully performed probit or logit regression in Julia?

2014-08-31 Thread John Myles White
be, (intercept) X1.^2 X2.^2 X3.^2 X1.*X2 X1.*X3 X2.*X3 X1.*X2.*X3 Bradley On Sunday, August 31, 2014 2:55:22 PM UTC-5, John Myles White wrote: Bradley, you’re forgetting about interactions terms. — John On Aug 31, 2014, at 12:53 PM, Bradley Setzler bradley...@gmail.com wrote

Re: [julia-users] Macros for mixing things like max and sum

2014-09-01 Thread John Myles White
Have you tried macroexpand? — John On Aug 31, 2014, at 10:28 PM, Mykel Kochenderfer m.kochender...@gmail.com wrote: I want to do a calculation like this $\max_{a \in A} \sum_{s \in S} g(s, a)$. Of course, I can do something like this: maximum([sum([g(s, a) for s in S]) for a in A])

Re: [julia-users] Issue Replacing NaN with 0

2014-09-01 Thread John Myles White
On Sunday, August 31, 2014 12:58:43 PM UTC-7, John Myles White wrote: I think there’s a broad issue that need resolution: how do you know when a function’s output takes control of the memory used by its arguments? — John ​

Re: [julia-users] Re: Strange Slicing Behaviour

2014-09-03 Thread John Myles White
Use vec On Sep 3, 2014, at 11:35 AM, David Smith david.sm...@gmail.com wrote: Ok, so you can continue using the old squeeze. Us reckless types can use the aggressive one. ;-) I don't see why it shouldn't be available. Is there a fear that new users will run into subtle errors and hate

Re: [julia-users] Re: Problem parsing file header

2014-09-04 Thread John Myles White
In general, type inference doesn’t always converge in the global scope. In this case, you’re seeing that type inference is missing information because of the reference to cols, which isn’t a compile-time constant. I personally recommend using typed comprehensions everywhere for explicitness

Re: [julia-users] exporting dynamically defined functions

2014-09-05 Thread John Myles White
It is definitely possible. You just need to use toplevel, which is a magic that Jeff doesn't want people to know about. -- John On Sep 5, 2014, at 12:52 PM, Ben Arthur bjarthu...@gmail.com wrote: sorry to resurrect this old post, but what is the definitive answer to whether it's possible

Re: [julia-users] Set of tuples?

2014-09-05 Thread John Myles White
How do you feel about using Set( [(1, 2)] )? — John On Sep 5, 2014, at 5:01 PM, Sam L sam.len...@gmail.com wrote: I can't figure out how to make a set of tuples in a very clean way. This works: julia push!(Set{(Int, Int)}(), (1,2)) Set{(Int64,Int64)}({(1,2)}) but this doesn't:

[julia-users] 0.4 Roadmap for DataFrames, DataArrays, etc...

2014-09-06 Thread John Myles White
I am hoping that the 0.4 release of Julia will coincide with a major cleanup of the Data* world. I wrote up a very high level overview of my goals here: https://gist.github.com/johnmyleswhite/ad5305ecaa9de01e317e There’s still more work to do to flesh out these ideas, but the basic principles

Re: [julia-users] Re: 0.4 Roadmap for DataFrames, DataArrays, etc...

2014-09-07 Thread John Myles White
, John Myles White wrote: I am hoping that the 0.4 release of Julia will coincide with a major cleanup of the Data* world. I wrote up a very high level overview of my goals here: https://gist.github.com/johnmyleswhite/ad5305ecaa9de01e317e There’s still more work to do to flesh out these ideas

Re: [julia-users] Are dataframes indexed?

2014-09-07 Thread John Myles White
No, DataFrames are not indexed. For now, you’d need to build a wrapper that indexes a DataFrame to get that kind of functionality. — John On Sep 7, 2014, at 9:53 AM, Steven Sagaert steven.saga...@gmail.com wrote: Hi, I was wondering if searching in a dataframe is indexed (in the DB sense,

Re: [julia-users] when you do push! on an array does it get copied each time or in chunks?

2014-09-07 Thread John Myles White
It gets expanded in chunks. — John On Sep 7, 2014, at 10:15 AM, Steven Sagaert steven.saga...@gmail.com wrote: When you start with an empty array and grow it one element at a time with push!, does the underlying array memory block get copied expanded by one or in larger chunks (like

Re: [julia-users] Are dataframes indexed?

2014-09-07 Thread John Myles White
: https://gist.github.com/johnmyleswhite/ad5305ecaa9de01e317e On Sun, Sep 7, 2014 at 12:54 PM, John Myles White johnmyleswh...@gmail.com wrote: No, DataFrames are not indexed. For now, you’d need to build a wrapper that indexes a DataFrame to get that kind of functionality. — John

Re: [julia-users] Re: Ignoring NaNs

2014-09-07 Thread John Myles White
I think we’re still not really interested in promoting the use of NaN as a surrogate for NULL, especially given that Nullable is going to be added to Base in 0.4. Your functions would perform substantially better if you iterated over the values of A. For example, function nanmean(A::Array)

Re: [julia-users] Are dataframes indexed?

2014-09-07 Thread John Myles White
. On Sunday, September 7, 2014 10:32:53 AM UTC-7, John Myles White wrote: FWIW, I think it’s much easier to index structures if every row has an atomic existence that is independent of the table it is currently part of. (This is a big part of my interest in moving away from matrix semantics

Re: [julia-users] when you do push! on an array does it get copied each time or in chunks?

2014-09-07 Thread John Myles White
whenever it grows beyond it's underlying size. (This might change when the array gets large, but I haven't looked at that code recently, so I forget the details.) Cheers, Kevin On Sunday, September 7, 2014, John Myles White johnmyleswh...@gmail.com wrote: It gets expanded in chunks

Re: [julia-users] For Loop over set of vector names

2014-09-07 Thread John Myles White
is apparently not the right way of thinking. n=5 x_foo=zeros(n,n) x_bar=zeros(n,n) for i in [foo,bar] x_$i = x_$i + n println(x_$i) end x_foo On Sunday, September 7, 2014 3:07:10 PM UTC-7, John Myles White wrote: Hi Alex, You can’t use things like x_$i

Re: [julia-users] Are dataframes indexed?

2014-09-07 Thread John Myles White
I kind of suspect my team (which is the team that invented Hive) isn't likely to stop using Hive anytime soon. -- John On Sep 7, 2014, at 4:50 PM, Steven Sagaert steven.saga...@gmail.com wrote: On Monday, September 8, 2014 1:37:50 AM UTC+2, John Myles White wrote: Well, you can write

Re: [julia-users] Parametric type fails

2014-09-07 Thread John Myles White
I think the types of your inputs might be wrong. In particular, I think the type of the inner containers isn't right: you're getting Array{Array{T,N},1} when you wanted Array{Array{E,1},1} for some specific E. -- John On Sep 7, 2014, at 5:50 PM, Zenna Tavares zennatava...@gmail.com wrote:

Re: [julia-users] Interact + PyPlot: only update when releasing slider

2014-09-08 Thread John Myles White
I suspect the only way to do this is to change Interact so that it exposes a minimum time threshold before it registers a state change. — John On Sep 8, 2014, at 4:16 AM, Andrei Berceanu andreiberce...@gmail.com wrote: I have some code along the lines of f = figure() @manipulate for

Re: [julia-users] intermediate results with reduce / FoldList

2014-09-09 Thread John Myles White
I don't believe this exists, but it should be easy to write by combining ideas from Julia's cumsum and foldl. You'd end up with cumfoldl and cumfoldr. -- John On Sep 9, 2014, at 7:29 AM, John Drummond john...@gmail.com wrote: What's the idiomatic way of writing Mathematica's Foldlist in

Re: [julia-users] How to append two arrays?

2014-09-09 Thread John Myles White
Try vcat/hcat. -- John On Sep 9, 2014, at 3:31 PM, Diego Tapias dandrove...@gmail.com wrote: Thanks for answering!, but what if I want to form a matrix of dimension 2 and not an array of dimension 1. 2014-09-09 17:27 GMT-05:00 Stefan Karpinski ste...@karpinski.org: append!(v,w) – it

Re: [julia-users] println a IOStream

2014-09-09 Thread John Myles White
You want show, not print. -- John On Sep 9, 2014, at 3:32 PM, muraveill murave...@gmail.com wrote: This is really confusing. In the REPL one can evaluate a stream to get useful info on what the object is: julia f IOStream(file test.txt) But in a script, just f would not work, so for

Re: [julia-users] type generation

2014-09-10 Thread John Myles White
This sure looks like you're not making any copies when you seem to want copies. In particular, this line: cellList[i] = Cell(i, oVector) probably needs to be cellList[i] = Cell(i, copy(oVector)) -- John On Sep 10, 2014, at 6:41 PM, Andre Bieler andre.biele...@gmail.com wrote: can

Re: [julia-users] 0.4 Roadmap for DataFrames, DataArrays, etc...

2014-09-11 Thread John Myles White
some easily parallelizable operations. -viral On Sunday, September 7, 2014 11:47:44 AM UTC+5:30, John Myles White wrote: Yeah, that’s a way more ambitious project. That’ll take at least a year to make any progress at all. Before I could even begin, I need to finish DBI and then build up

Re: [julia-users] Values vs. Bindings: making sense of things

2014-09-11 Thread John Myles White
For future reference, I'd really appreciate you not referring to my writing as quite erroneous when the claims made are in fact correct. -- John On Sep 11, 2014, at 2:38 PM, Mohammed El-Beltagy mohammed.elbelt...@gmail.com wrote: In a recent blog by John Myles White http

Re: [julia-users] Compiled R

2014-09-11 Thread John Myles White
This article is really good. Thanks for pointing it out. -- John On Sep 11, 2014, at 4:10 PM, asim Ansari asimansari...@gmail.com wrote: Hi This article by Duncan Temple Lang on Compiling R could be of interest. Mentions Juila a few times. Asim http://arxiv.org/pdf/1409.3144.pdf

Re: [julia-users] Compiled R

2014-09-12 Thread John Myles White
I'd love it if Duncan were to start working on Julia. His ideas would be invaluable. -- John On Sep 11, 2014, at 6:49 PM, Ethan Anderes ethanande...@gmail.com wrote: For the past year I've been trying to get Duncan to jump to our side of the tracks and join in on Julia development. I

Re: [julia-users] Re: Julia strange behaviour with GC

2014-09-12 Thread John Myles White
One point worth making: it's often much harder to estimate the tails of a distribution than the mean. The most natural example of this is estimating the mean of a normal distribution. From theory we know that the population median and the population mean are the same quantity, so we could use

Re: [julia-users] Suprises with for loops

2014-09-12 Thread John Myles White
As Lint keeps improving, I feel like we should probably highlight it more aggressively in the manual. -- John On Sep 12, 2014, at 10:56 AM, Tony Fong tony.hf.f...@gmail.com wrote: Lint v0.1.2 starts to track variables' type locally within a function declaration. So your case will correctly

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

2014-09-12 Thread John Myles White
We really need to standardize on a single type that reflects a single row of a tabular data structure that gets used both by DBI and by DataFrames. DataFrameRow is really nice because it's a zero-copy operation for DataFrames, but we can't provide zero-copy semantics when pulling rows out of a

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

2014-09-12 Thread John Myles White
convert methods than to re-represent rows as OrderedDicts (or whatever datatype). On Friday, September 12, 2014 12:19:47 PM UTC-5, John Myles White wrote: We really need to standardize on a single type that reflects a single row of a tabular data structure that gets used both by DBI

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

2014-09-12 Thread John Myles White
intermediate step). On Friday, September 12, 2014 3:26:47 PM UTC-5, John Myles White wrote: I'm not sure that losing zero copy semantics is actually a big performance hit in most pipelines. I think much more important is that you can't write generic code right now because the abstractions

Re: [julia-users] how to push!(::DataFrame,::DataFrameRow) or append!(::DataFrame,::DataFrameRow) ?

2014-09-12 Thread John Myles White
What does that mean? A DataFrameRow can't be easily created without reference to an existing DataFrame, so this seems like it's either a mechanism for transferring rows from one DataFrame to another very slowly or a mechanism for inserting duplicate rows. -- John On Sep 12, 2014, at 3:37 PM,

Re: [julia-users] Values vs. Bindings: making sense of things

2014-09-13 Thread John Myles White
No worries. I'm glad you enjoy my blog posts. -- John On Sep 13, 2014, at 11:32 AM, Mohammed El-Beltagy mohammed.elbelt...@gmail.com wrote: On Friday, September 12, 2014 8:33:03 AM UTC+3, John Myles White wrote: For future reference, I'd really appreciate you not referring to my writing

Re: [julia-users] Random numbers and the Poisson distribution

2014-09-15 Thread John Myles White
I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same distributions. In contrast, the proposed change here would break a lot of other code in

Re: [julia-users] Random numbers and the Poisson distribution

2014-09-15 Thread John Myles White
the popular one! On the other hand, Scipy returns an error). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes

Re: [julia-users] Random numbers and the Poisson distribution

2014-09-15 Thread John Myles White
). Regards, Jamie On Monday, 15 September 2014 16:03:32 UTC+1, John Myles White wrote: I’m not so sure that we should follow the lead of Octave and R here. Neither of those languages reify distributions as types, so changes to their RNG’s don’t affect other operations on those same

Re: [julia-users] How to assign values to variable inside a loop ?

2014-09-15 Thread John Myles White
Hi Josimar, Julia arrays are not indexed using parentheses: they are indexed using brackets. Assuming you have an array like x = zeros(10), you can do for i in 1:10 x[i] = i*i end You also don’t need semicolons in Julia. — John On Sep 15, 2014, at 5:15 PM, Josimar Silva

Re: [julia-users] slow julia version of c code

2014-09-16 Thread John Myles White
You are a genuine Julia community hero, Tony. -- John On Sep 16, 2014, at 9:17 AM, Tony Fong tony.hf.f...@gmail.com wrote: Thanks to this discussion, I have added some performance related lint messages related to type declaration. It would catch dimension-less array field type going

Re: [julia-users] Cell indexing

2014-09-18 Thread John Myles White
Hi Nils, Try something like: A = Array(Any, 10) for i in 1:10 A[i] = randn(1, 10) end On Sep 18, 2014, at 6:47 AM, nils.gu...@gmail.com wrote: I'm sure this is an extremely trivial question, but I can't seem to find an answer anywhere. I'm trying to store a couple of matrices of

Re: [julia-users] Incrementing integer slow?

2014-09-18 Thread John Myles White
1 has type Int. If you add it to something with a different type, you might be causing type instability. What happens if you replace the literal 1 with one(T) for the type you're working with? -- John On Sep 18, 2014, at 9:56 AM, G. Patrick Mauroy gpmau...@gmail.com wrote: Profiling shows

Re: [julia-users] unexpected domain error for ^(float,float)

2014-09-18 Thread John Myles White
I think that was a typo for not surprised. -- John On Sep 18, 2014, at 9:59 AM, Steven G. Johnson stevenj@gmail.com wrote: On Thursday, September 18, 2014 12:00:32 PM UTC-4, Florian Oswald wrote: well, I guess most computer scientists would be surprised. writing on a piece of paper

Re: [julia-users] Is there an inverse of `sparse`?

2014-09-18 Thread John Myles White
Try findnz. This seems to not be documented in the sparse section of the manual, but I would think it should be. — John On Sep 18, 2014, at 6:58 PM, DumpsterDoofus peter.richter@gmail.com wrote: Given column vectors I, J, and V, one can construct a sparse matrix using the following

<    1   2   3   4   5   6   7   8   >