Re: [julia-users] Re: Sys.CPU_CORES

[Júlio Hoffimann]

Thank you Isaiah, I'll follow your tip and use Hwloc.jl

-Júlio

Re: [julia-users] Re: Sys.CPU_CORES

[Júlio Hoffimann]

My question is, it returns the same concept in all OS? Is it physical
number of cores?

-Júlio

[julia-users] Sys.CPU_CORES

[Júlio Hoffimann]

Hi,

Sys.CPU_CORES returns the number of physical cores or processing units? Is 
it portable across different OS?

-Júlio

Re: [julia-users] Re: Sparse arrays in Julia

[Júlio Hoffimann]

Thank you Kristoffer, good to know about the package :)

-Júlio

[julia-users] Re: Sparse arrays in Julia

[Júlio Hoffimann]

Actually, I don't think it makes very much sense, one could use a 
dictionary of tuples instead. Sparse 2D exist for a reason: efficient 
matrix multiplication, etc.

Disregard this email.

-Júlio

Em quinta-feira, 27 de outubro de 2016 17:07:01 UTC-7, Júlio Hoffimann 
escreveu:
>
> Hello,
>
> Is it possible to create a multidimensional sparse array in Julia?
>
> sparse() works for matrices, I wonder if someone can extend it to 
> multidimensional arrays. Where should I open a feature request?
>
> -Júlio
>

[julia-users] Sparse arrays in Julia

[Júlio Hoffimann]

Hello,

Is it possible to create a multidimensional sparse array in Julia?

sparse() works for matrices, I wonder if someone can extend it to 
multidimensional arrays. Where should I open a feature request?

-Júlio

[julia-users] Dropping all explicit Any type annotation in Julia >= v0.5

[Júlio Hoffimann]

Hi,

In Julia v0.4 I had to write Any[eye(3),eye(3)] in order to get a list of 
arrays. In Julia v0.5, the syntax changed and now [eye(3),eye(3)] is what I 
need.

Could you please confirm I can drop all occurrences of the Any type 
annotation in these cases without side effects other than changing the type 
to be more specific Array{Any} versus Array{Array}?

-Júlio

Re: [julia-users] Re: Uniqueness check in PriorityQueue

[Júlio Hoffimann]

Thank you Steven, I fixed the issue by storing other objects in the
PriorityQueue.

-Júlio

2016-10-17 14:27 GMT-07:00 Steven G. Johnson :

> Make a copy before you mutate it.  In general, it is not safe to mutate
> the keys of a dictionary, which is what you are doing.
>

[julia-users] Uniqueness check in PriorityQueue

[Júlio Hoffimann]

Hi,

Consider the following snippet:

using LightGraphs
using Base.Collections

pq = PriorityQueue(DiGraph, Int)

G = DiGraph(3)
add_edge!(G, 1,2)

enqueue!(pq, G, 1)

# reverse edge
rem_edge!(G, 1,2)
add_edge!(G, 2,1)

enqueue!(pq, G, 2)

It produces this error:

ERROR: ArgumentError: PriorityQueue keys must be unique
 in 
enqueue!(::Base.Collections.PriorityQueue{LightGraphs.DiGraph,Int64,Base.Order.ForwardOrdering},
 
::LightGraphs.DiGraph, ::Int64) at ./collections.jl:366

The operator == is properly defined in LightGraphs and would have yield 
false, however; the PriorityQueue is handling the two items as if they were 
the same. How can this issue be solved or where should I report it?

-Júlio

Re: [julia-users] FactCheck.jl bundled with Julia?

[Júlio Hoffimann]

Ok, I am not switching to FactCheck then, didn't knew it is being
deprecated in a sense.

Thank you,
-Júlio

2016-10-14 18:05 GMT-07:00 Yichao Yu <yyc1...@gmail.com>:

> On Oct 14, 2016 8:52 PM, "Júlio Hoffimann" <julio.hoffim...@gmail.com>
> wrote:
> >
> > Oh really? I'm not following it closely. Please let me know why that is
> the case, I was planning to switch to FactCheck.
>
> Afaict the new test in base is a improved version of FactCheck.
>
> >
> > -Júlio
>

Re: [julia-users] FactCheck.jl bundled with Julia?

[Júlio Hoffimann]

Oh really? I'm not following it closely. Please let me know why that is the
case, I was planning to switch to FactCheck.

-Júlio

[julia-users] FactCheck.jl bundled with Julia?

[Júlio Hoffimann]

Hi,

It seems that FactCheck.jl has become the defacto standard for writing 
tests in Julia packages. Wouldn't it be a good idea to have it bundled with 
Julia? Any reason to keep the current test framework?

-Júlio

RE: [julia-users] Filtering DataFrame with a function

[Júlio Hoffimann]

That is really cool David, I fully agree with this modularization :)

-Júlio

Re: [julia-users] Filtering DataFrame with a function

[Júlio Hoffimann]

Hi Alex,

That is closer to what I had in mind originally, but I actually solved the
problem by reorganizing my algorithm to avoid filters.

Thank you,
-Júlio

2016-10-13 8:21 GMT-07:00 Alex Mellnik <a.r.mell...@gmail.com>:

> Hi Júlio,
>
> If you're just interested in using an arbitrary function to filter on rows
> you can do something like:
>
> df = DataFrame(Fish = ["Amir", "Betty", "Clyde"], Mass = [1.2, 3.3, 0.4])
> filter(row) = (row[:Fish][1] != "A")&(row[:Mass]>1)
> df = df[[filter(r) for r in eachrow(df)],:]
>
> Is that what you're looking for?  If not, can you give an example of what
> you want to do?
>
> Best,
>
> Alex
>
> On Wednesday, October 12, 2016 at 10:20:52 PM UTC-7, Júlio Hoffimann wrote:
>>
>> Thank you very Much David, these queries you showed are really nice. I
>> meant that ideally I wouldn't need to install another package for a simple
>> filter operation on the rows.
>>
>> -Júlio
>>
>> 2016-10-12 22:14 GMT-07:00 <ant...@berkeley.edu>:
>>
>>> Were you worried about Query being not lightweight enough in terms of
>>> overhead, or in terms of syntax?
>>>
>>> I just added a more lightweight syntax for this scenario to Query. You
>>> can now do the following two things:
>>>
>>> q = @where(df, i->i.price > 30.)
>>>
>>> that will return a filtered iterator. You can materialize that into a
>>> DataFrame with collect(q, DataFrame).
>>>
>>> I also added a counting option. Turns out that is actually a LINQ query
>>> operator, and the goal is to implement all of those in Query. The syntax is
>>> simple:
>>>
>>> @count(df, i->i.price > 30.)
>>>
>>> returns the number of rows for which the filter condition is true.
>>>
>>> Under the hood both of these new syntax options use the normal Query
>>> machinery, this just provides a simpler syntax relative to the more
>>> elaborate things I've posted earlier. In terms of LINQ, this corresponds to
>>> the method invocation API that LINQ has. I'm still figuring out how to
>>> surface something like @count in the query expression syntax, but for now
>>> one can use it via this macro.
>>>
>>> All of this is on master right now, so you would have to do
>>> Pkg.checkout("Query") to get these macros.
>>>
>>> Best,
>>> David
>>>
>>> On Wednesday, October 12, 2016 at 6:47:15 PM UTC-7, Júlio Hoffimann
>>> wrote:
>>>>
>>>> Hi David,
>>>>
>>>> Thank you for your elaborated answer and for writing a package for
>>>> general queries, that is great! I will keep the package in mind if I need
>>>> something more complex.
>>>>
>>>> I am currently looking for a lightweight solution within DataFrames,
>>>> filtering is a very common operation. Right now, I am considering
>>>> converting the DataFrame to an array and looping over the rows. I wonder if
>>>> there is a syntactic sugar for this loop.
>>>>
>>>> -Júlio
>>>>
>>>> 2016-10-12 17:48 GMT-07:00 David Anthoff <ant...@berkeley.edu>:
>>>>
>>>>> Hi Julio,
>>>>>
>>>>>
>>>>>
>>>>> you can use the Query package for the first part. To filter a
>>>>> DataFrame using some arbitrary julia expression, use something like this:
>>>>>
>>>>>
>>>>>
>>>>> using DataFrames, Query, NamedTuples
>>>>>
>>>>>
>>>>>
>>>>> q = @from i in df begin
>>>>>
>>>>> @where 
>>>>>
>>>>> @select i
>>>>>
>>>>> end
>>>>>
>>>>>
>>>>>
>>>>> You can use any julia code in . Say your DataFrame
>>>>> has a column called price, then you could filter like this:
>>>>>
>>>>>
>>>>>
>>>>> @where i.price > 30.
>>>>>
>>>>>
>>>>>
>>>>> The i will be a NamedTuple type, so you can access the columns either
>>>>> by their name, or also by their index, e.g.
>>>>>
>>>>>
>>>>>
>>>>> @where i[1] > 30.
>>>>>
>>>>>
>>>>>
>>>>> if you want to filter by t

Re: [julia-users] Filtering DataFrame with a function

[Júlio Hoffimann]

Thank you very Much David, these queries you showed are really nice. I
meant that ideally I wouldn't need to install another package for a simple
filter operation on the rows.

-Júlio

2016-10-12 22:14 GMT-07:00 <anth...@berkeley.edu>:

> Were you worried about Query being not lightweight enough in terms of
> overhead, or in terms of syntax?
>
> I just added a more lightweight syntax for this scenario to Query. You can
> now do the following two things:
>
> q = @where(df, i->i.price > 30.)
>
> that will return a filtered iterator. You can materialize that into a
> DataFrame with collect(q, DataFrame).
>
> I also added a counting option. Turns out that is actually a LINQ query
> operator, and the goal is to implement all of those in Query. The syntax is
> simple:
>
> @count(df, i->i.price > 30.)
>
> returns the number of rows for which the filter condition is true.
>
> Under the hood both of these new syntax options use the normal Query
> machinery, this just provides a simpler syntax relative to the more
> elaborate things I've posted earlier. In terms of LINQ, this corresponds to
> the method invocation API that LINQ has. I'm still figuring out how to
> surface something like @count in the query expression syntax, but for now
> one can use it via this macro.
>
> All of this is on master right now, so you would have to do
> Pkg.checkout("Query") to get these macros.
>
> Best,
> David
>
> On Wednesday, October 12, 2016 at 6:47:15 PM UTC-7, Júlio Hoffimann wrote:
>>
>> Hi David,
>>
>> Thank you for your elaborated answer and for writing a package for
>> general queries, that is great! I will keep the package in mind if I need
>> something more complex.
>>
>> I am currently looking for a lightweight solution within DataFrames,
>> filtering is a very common operation. Right now, I am considering
>> converting the DataFrame to an array and looping over the rows. I wonder if
>> there is a syntactic sugar for this loop.
>>
>> -Júlio
>>
>> 2016-10-12 17:48 GMT-07:00 David Anthoff <ant...@berkeley.edu>:
>>
>>> Hi Julio,
>>>
>>>
>>>
>>> you can use the Query package for the first part. To filter a DataFrame
>>> using some arbitrary julia expression, use something like this:
>>>
>>>
>>>
>>> using DataFrames, Query, NamedTuples
>>>
>>>
>>>
>>> q = @from i in df begin
>>>
>>> @where 
>>>
>>> @select i
>>>
>>> end
>>>
>>>
>>>
>>> You can use any julia code in . Say your DataFrame
>>> has a column called price, then you could filter like this:
>>>
>>>
>>>
>>> @where i.price > 30.
>>>
>>>
>>>
>>> The i will be a NamedTuple type, so you can access the columns either by
>>> their name, or also by their index, e.g.
>>>
>>>
>>>
>>> @where i[1] > 30.
>>>
>>>
>>>
>>> if you want to filter by the first column. You can also just call some
>>> function that you have defined somewhere else:
>>>
>>>
>>>
>>> @where foo(i)
>>>
>>>
>>>
>>> As long as the  returns a Bool, you should be good.
>>>
>>>
>>>
>>> If you run a query like this, q will be a standard julia iterator. Right
>>> now you can’t just say length(q), although that is something I should
>>> probably enable at some point (I’m also looking into the VB LINQ syntax
>>> that supports things like counting in the query expression itself).
>>>
>>>
>>>
>>> But you could materialize the query as an array and then look at the
>>> length of that:
>>>
>>>
>>>
>>> q = @from i in df begin
>>>
>>> @where 
>>>
>>> @select i
>>>
>>> @collect
>>>
>>> end
>>>
>>> count = length(q)
>>>
>>>
>>>
>>> The @collect statement means that the query will return an array of a
>>> NamedTuple type (you can also materialize it into a whole bunch of other
>>> data structures, take a look at the documentation).
>>>
>>>
>>>
>>> Let me know if this works, or if you have any other feedback on
>>> Query.jl, I’m much in need of some user feedback for the package at this
>>> point. Best way for that is to open issues here
>>> https://github.com/davidanthoff/Query.jl.
>>>
>>>
>>>
>>> Best,
>>>
>>> David
>>>
>>>
>>>
>>> *From:* julia...@googlegroups.com [mailto:julia...@googlegroups.com] *On
>>> Behalf Of *Júlio Hoffimann
>>> *Sent:* Wednesday, October 12, 2016 5:20 PM
>>> *To:* julia-users <julia...@googlegroups.com>
>>> *Subject:* [julia-users] Filtering DataFrame with a function
>>>
>>>
>>>
>>> Hi,
>>>
>>>
>>>
>>> I have a DataFrame for which I want to filter rows that match a given
>>> criteria. I don't have the number of columns beforehand, so I cannot
>>> explicitly list the criteria with the :symbol syntax or write down a fixed
>>> number of indices.
>>>
>>>
>>>
>>> Is there any way to filter with a lambda expression? Or even better, is
>>> there any efficient way to count the number of occurrences of a specific
>>> row of observations?
>>>
>>>
>>>
>>> -Júlio
>>>
>>
>>

Re: [julia-users] Filtering DataFrame with a function

[Júlio Hoffimann]

Hi David,

Thank you for your elaborated answer and for writing a package for general
queries, that is great! I will keep the package in mind if I need something
more complex.

I am currently looking for a lightweight solution within DataFrames,
filtering is a very common operation. Right now, I am considering
converting the DataFrame to an array and looping over the rows. I wonder if
there is a syntactic sugar for this loop.

-Júlio

2016-10-12 17:48 GMT-07:00 David Anthoff <anth...@berkeley.edu>:

> Hi Julio,
>
>
>
> you can use the Query package for the first part. To filter a DataFrame
> using some arbitrary julia expression, use something like this:
>
>
>
> using DataFrames, Query, NamedTuples
>
>
>
> q = @from i in df begin
>
> @where 
>
> @select i
>
> end
>
>
>
> You can use any julia code in . Say your DataFrame has
> a column called price, then you could filter like this:
>
>
>
> @where i.price > 30.
>
>
>
> The i will be a NamedTuple type, so you can access the columns either by
> their name, or also by their index, e.g.
>
>
>
> @where i[1] > 30.
>
>
>
> if you want to filter by the first column. You can also just call some
> function that you have defined somewhere else:
>
>
>
> @where foo(i)
>
>
>
> As long as the  returns a Bool, you should be good.
>
>
>
> If you run a query like this, q will be a standard julia iterator. Right
> now you can’t just say length(q), although that is something I should
> probably enable at some point (I’m also looking into the VB LINQ syntax
> that supports things like counting in the query expression itself).
>
>
>
> But you could materialize the query as an array and then look at the
> length of that:
>
>
>
> q = @from i in df begin
>
> @where 
>
> @select i
>
> @collect
>
> end
>
> count = length(q)
>
>
>
> The @collect statement means that the query will return an array of a
> NamedTuple type (you can also materialize it into a whole bunch of other
> data structures, take a look at the documentation).
>
>
>
> Let me know if this works, or if you have any other feedback on Query.jl,
> I’m much in need of some user feedback for the package at this point. Best
> way for that is to open issues here https://github.com/
> davidanthoff/Query.jl.
>
>
>
> Best,
>
> David
>
>
>
> *From:* julia-users@googlegroups.com [mailto:julia-users@googlegroups.com]
> *On Behalf Of *Júlio Hoffimann
> *Sent:* Wednesday, October 12, 2016 5:20 PM
> *To:* julia-users <julia-users@googlegroups.com>
> *Subject:* [julia-users] Filtering DataFrame with a function
>
>
>
> Hi,
>
>
>
> I have a DataFrame for which I want to filter rows that match a given
> criteria. I don't have the number of columns beforehand, so I cannot
> explicitly list the criteria with the :symbol syntax or write down a fixed
> number of indices.
>
>
>
> Is there any way to filter with a lambda expression? Or even better, is
> there any efficient way to count the number of occurrences of a specific
> row of observations?
>
>
>
> -Júlio
>

[julia-users] Filtering DataFrame with a function

[Júlio Hoffimann]

Hi,

I have a DataFrame for which I want to filter rows that match a given 
criteria. I don't have the number of columns beforehand, so I cannot 
explicitly list the criteria with the :symbol syntax or write down a fixed 
number of indices.

Is there any way to filter with a lambda expression? Or even better, is 
there any efficient way to count the number of occurrences of a specific 
row of observations?

-Júlio

Re: [julia-users] Re: Priority queue - peek versus dequeue!

[Júlio Hoffimann]

Thank you, I'll open an issue.

-Júlio

[julia-users] Priority queue - peek versus dequeue!

[Júlio Hoffimann]

Hi,

Could you explain why "peek" returns the pair (key,value) whereas 
"dequeue!" only returns the key?

using Base.Collections
pq = PriorityQueue()
enqueue!(pq, key, value)
key, value = Collections.peek(pq)
key = dequeue!(pq)

I wanted to have a single line in which I retrieve both (key,value) and at 
the same time remove the pair from the collection:

key, value = dequeue!(pq)

Should I open an issue on GitHub?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

I don't think there is anything like pushing the language to computer
scientists, it's the exact opposite, making it seamlessly fast without
forcing the user to manipulate types. Again, you write B = I*A and get B =
copy(A) performance. That is the original proposal.

Most of us follow the same development strategy while doing science, we
write something quick and profile later. The fact that Julia can be fast
from the beginning is the best thing about it, no need for rewriting code.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Sorry for the combative tone Christoph. I thought it was necessary in order
to not deviate too much from the core issue. Thank you for your
participation and for raising your personal opinions about the topic.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

So maybe add a dimension or create a type that makes more sense for the
application?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Why is it so important to have all this machinery around linspace and eye?
collect is more than enough in my opinion and all the proposals for keeping
both versions and pass a type as a parameter are diluting the main issue
here: the need for a smart mechanism that handles things efficiently and
seamlessly for the user. Teaching is a completely separate matter, let's
not enter in this territory, which is much more delicate and hard to be
done right.

Recaping the issue:

B = I*A should be as efficient as B = copy(A). Right now it is not.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Christoph,

Can you elaborate on why you want to have both? Also, why you wanted the
non-lazy linspace? If I understood correctly, you're talking about
returning a array with allocated memory versus a range object, isn't the
latter always preferred? I don't understand.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Tim,

Would it make sense to have "I" as an object that acts like UniformScaling
and doesn't require any memory allocation, but is only transformed into
sparse matrix via the [] operator? Maybe something similar to arrays ->
subarrays?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Why would one want dense identity matrix?

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

It is more than sparse, it acts as scalar at first, maybe the operator []
modifies the type to sparse?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Andreas, is there a way to get the best of both worlds? Let's say eye() is
deprecated, can we somehow set off-diagonal terms in a type that is smart
like UniformScaling and supports indexing with operator []?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

I'd like to understand the existence of eye() in Julia, it is still not
clear to me. Is it because one wants type stability when updating a matrix
iteratively? Is this possibly a limitation from the design of the language?

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

I still think that having a "global variable" named "I" is not robust. I've
read so many scripts in matlab that do I = eye(n). This approach is not
gonna work.

-Júlio

Re: [julia-users] Re: Return type of eye()

[Júlio Hoffimann]

Hi Andreas,

As a user I would like to write

B = eye(1) * A

and have the performance of

B = A

90% of the users won't be aware of this 1-character variable "I" defined in
Base nor use it. Also, I can guarantee that "I" is much easier to overwrite
than a well known function name.

-Júlio

[julia-users] CLFFT on 2D/3D arrays

[Júlio Hoffimann]

Hello,

I am trying to implement the classical tiled convolution with OpenCL in one 
of my Julia packages to get some 
speedup: https://github.com/JuliaGPU/CLFFT.jl/issues/5

If you have experience in CLFFT.jl, I appreciate your feedback.

Sincerely,
Júlio

Re: [julia-users] Fancy integer literals

[Júlio Hoffimann]

Interesting, thank you Yichao!

-Júlio

[julia-users] Fancy integer literals

[Júlio Hoffimann]

Today I saw for the first time a code written like:

a = 50_000

What is the name of this feature in the docs?

-Júlio

Re: [julia-users] Julia way of filling columns of a matrix

[Júlio Hoffimann]

That is a good catch Kevin, thanks!

2016-01-18 12:53 GMT-08:00 Kevin Squire <kevin.squ...@gmail.com>:

> As long as each row has a fixed, known size, you could do
>
> a = []
> for i=1:n
> append!(a, [1,2,3])
> end
> A = reshape(a, 3, n)
>
> The 1-D array grows as needed, and reshape still points to the original
> data, so no copying is done.
>
> Cheers,
>    Kevin
>
> On Mon, Jan 18, 2016 at 12:48 PM, Júlio Hoffimann <
> julio.hoffim...@gmail.com> wrote:
>
>> Yes, I will rely on the classical hcat() approach...
>>
>> A = []
>> for i=1:n
>>   push!(A, [1,2,3])
>> end
>> A = hcat(A...)
>>
>> Thank you.
>>
>> 2016-01-18 11:42 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:
>>
>>> Hi,
>>>
>>> Suppose I want to fill the columns of a matrix which size I don't know
>>> beforehand:
>>>
>>> A = zeros(3,0)
>>> for i=1:n
>>>   A = [A [1,2,3]]
>>> end
>>>
>>> Is there a memory efficient way of doing that in Julia?
>>>
>>> I understand that the above syntax is allocating 3*i entries at
>>> iteration i which gives 3*(1+2+...+n) = 3*(n+1)n/2 allocations as opposed
>>> to 3n.
>>>
>>> -Júlio
>>>
>>
>>
>

Re: [julia-users] Julia way of filling columns of a matrix

[Júlio Hoffimann]

Yes, I will rely on the classical hcat() approach...

A = []
for i=1:n
  push!(A, [1,2,3])
end
A = hcat(A...)

Thank you.

2016-01-18 11:42 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Hi,
>
> Suppose I want to fill the columns of a matrix which size I don't know
> beforehand:
>
> A = zeros(3,0)
> for i=1:n
>   A = [A [1,2,3]]
> end
>
> Is there a memory efficient way of doing that in Julia?
>
> I understand that the above syntax is allocating 3*i entries at iteration
> i which gives 3*(1+2+...+n) = 3*(n+1)n/2 allocations as opposed to 3n.
>
> -Júlio
>

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Hi Tim,

It makes sense now with --inline=no except for these two lines:

294311546   @assert ndims(training_image) == 3 "training image is not 3D
(add ghost dimension for 2D)"
-537275547   simgrid[iₛ:iₑ,jₛ:jₑ,kₛ:kₑ] = M.*simdev + !M.*TIdev

Why the first line is allocating that amount of memory?
What in the second line we have a negative sign?

I'll open an issue on GitHub.

Thanks,
-Júlio

2015-12-10 3:01 GMT-08:00 Tim Holy <tim.h...@gmail.com>:

> Definitely weird. Seems like either a path problem (but if you don't have
> multiple copies of iqsim.jl, that can't be it) or a line-numbering problem.
> Does this still happen with --inline=no? Are you using macros elsewhere?
>
> Assuming this is a julia bug, trimming it down to a minimal example would
> probably help isolate the problem. I'm afraid I don't have any other ideas.
>
> --Tim
>
> On Wednesday, December 09, 2015 07:54:47 PM Júlio Hoffimann wrote:
> > Hi Tim,
> >
> > I recompiled Julia, reinstalled all the packages and the *.mem file is
> > still showing those strange counts. Any idea?
> >
> > -Júlio
> >
> > 2015-12-09 8:44 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:
> > > Thanks Tim, I'll double check it.
> > >
> > > -Júlio
>
>

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Also, I appreciate if someone can explain this allocation pattern:

- function baz()
 61786572   a = ones(100,100,100)
  8000544   b = zeros(100,100,100)
-
 64631690   a[1:50,1:50,1:50] = b[1:50,1:50,1:50]
- end
-
- baz()
-

Why a and b allocate a different amount of memory even though they have the
same dimensions? Why the assignment is allocating memory?

-Júlio

2015-12-10 12:31 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Kristoffer,
>
> You mean the issue as a whole or the two questions I've asked?
>
> -Júlio
>
> 2015-12-10 12:28 GMT-08:00 Kristoffer Carlsson <kcarlsso...@gmail.com>:
>
>> Overflow of allocation counter
>
>
>

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

I just run the script with:

julia --track-allocation=user --inline=no baz.jl

Do I need to warm up something first?

-Júlio

2015-12-10 13:10 GMT-08:00 Kristoffer Carlsson :

> I meant regarding the negative allocation number. Regarding the different
> memories, did you compile, clear memory cache and then rerun the function.
> Else first call will include jit overhead.

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Maybe that is the reason Tim, I'm on latest master 0.5-dev-...

-Júlio

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Kristoffer,

You mean the issue as a whole or the two questions I've asked?

-Júlio

2015-12-10 12:28 GMT-08:00 Kristoffer Carlsson :

> Overflow of allocation counter

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

I tried both sub() and slice(), they make it worst:

- function baz()
  8000768   a = ones(100,100,100)
  8000544   b = zeros(100,100,100)
-
156003456   a[1:50,1:50,1:50] = sub(b,1:50,1:50,1:50)
- end
-
- baz()
- Profile.clear_malloc_data()
- baz()
-

-Júlio

2015-12-10 13:45 GMT-08:00 Tim Holy <tim.h...@gmail.com>:

> On Thursday, December 10, 2015 12:49:42 PM Júlio Hoffimann wrote:
> > Why the assignment is allocating memory?
>
> b[1:50,1:50,1:50] creates a new array. Try sub(b, 1:50, 1:50, 1:50).
>
> --Tim
>

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

It makes more sense now:

- function baz()
  8000768   a = ones(100,100,100)
  8000544   b = zeros(100,100,100)
-
 62981056   a[1:50,1:50,1:50] = b[1:50,1:50,1:50]
- end
-
- baz()
- Profile.clear_malloc_data()
- baz()
-

Some extra bytes here and there, but they are basically the same number.
What about the assignment? Any way to avoid it?

-Júlio

2015-12-10 13:41 GMT-08:00 Kristoffer Carlsson :

> Yes unless you want to track compilation allocations. Run your stuff once
> in the script, call Base.profile.clear_memory_cache() (taken from memory
> but something like that) and then run everything again (in the same julia
> session).

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Thank you everyone, I will delay this optimization for now. Will keep an
eye on the issue on GitHub.

-Júlio

2015-12-10 13:47 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> I tried both sub() and slice(), they make it worst:
>
> - function baz()
>   8000768   a = ones(100,100,100)
>   8000544   b = zeros(100,100,100)
> -
> 156003456   a[1:50,1:50,1:50] = sub(b,1:50,1:50,1:50)
> - end
> -
> - baz()
> - Profile.clear_malloc_data()
> - baz()
> -
>
> -Júlio
>
> 2015-12-10 13:45 GMT-08:00 Tim Holy <tim.h...@gmail.com>:
>
>> On Thursday, December 10, 2015 12:49:42 PM Júlio Hoffimann wrote:
>> > Why the assignment is allocating memory?
>>
>> b[1:50,1:50,1:50] creates a new array. Try sub(b, 1:50, 1:50, 1:50).
>>
>> --Tim
>>
>
>

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Thanks Tim, I'll double check it.

-Júlio

Re: [julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Hi Tim,

I recompiled Julia, reinstalled all the packages and the *.mem file is
still showing those strange counts. Any idea?

-Júlio

2015-12-09 8:44 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Thanks Tim, I'll double check it.
>
> -Júlio
>

[julia-users] Understanding memory allocation output

[Júlio Hoffimann]

Hi,

I don't understand why some lines like 6, 16, 29, 33, 186 are allocating 
memory in the attached example. Can someone explain?

-Júlio


iqsim.jl.mem
Description: Binary data

Re: [julia-users] Re: `findmin` with arbitrary comparison operator?

[Júlio Hoffimann]

Actually, I am starting to like the simple approach of filtering the tuple
suggested by Josh. It's cleaner and probably faster in general.

scores = map(t -> t[1], mytuplelist) # first tuple element as score
idx = indmin(scores)
mintuple = mytuplelist[idx]

I will close the issue.

Best,
-Júlio

2015-12-02 8:23 GMT-08:00 Josh Langsfeld <jdla...@gmail.com>:

> That's true; my_tuples[findmin(map(score_func, my_tuples))[2]] is a fair
> bit uglier than just interesting the map.
>
> On Tuesday, December 1, 2015 at 9:45:54 PM UTC-5, Erik Schnetter wrote:
>>
>> One of the tuple entries is the number by which I'd like to sort, so the
>> score_func is trivial. However, in the end I want not only the minimum
>> value, but also the additional tuple elements, which would be stripped off
>> by the score_func.
>>
>> -erik
>>
>> On Tue, Dec 1, 2015 at 12:38 PM, Josh Langsfeld <jdl...@gmail.com> wrote:
>>
>>> Doesn't 'min' imply/require the usage of '<' ? A whole lot of methods
>>> would need the extra argument added for consistency, including min,
>>> minimum, maximum, etc...
>>>
>>> Could a workable solution be to define a function that maps a tuple to a
>>> real number instead of a comparison function and do
>>> 'findmin(map(score_func, my_tuples))' ?
>>>
>>> On Tuesday, December 1, 2015 at 12:20:53 PM UTC-5, Júlio Hoffimann wrote:
>>>>
>>>> The issue on GitHub: https://github.com/JuliaLang/julia/issues/14216
>>>>
>>>> Thanks,
>>>> -Júlio
>>>>
>>>> 2015-12-01 8:57 GMT-08:00 Dan <get...@gmail.com>:
>>>>
>>>>> suggestion: call the named argument `lt` to match the argument of
>>>>> `sort`.
>>>>>
>>>>>
>>>>> On Tuesday, December 1, 2015 at 6:23:07 PM UTC+2, Júlio Hoffimann
>>>>> wrote:
>>>>>>
>>>>>> Hi Erik,
>>>>>>
>>>>>> What I meant was a derived type from Julia's Tuple type. Though I
>>>>>> agree with you that this may be too much of an overhead for such a simple
>>>>>> task.
>>>>>>
>>>>>> I'll open an issue on GitHub to ask if others are ok with an
>>>>>> additional predicate option a la C++:
>>>>>>
>>>>>> findmin(array, pred=<)
>>>>>>
>>>>>> If they are all positive, I'll add it for you.
>>>>>>
>>>>>> Best,
>>>>>> -Júlio
>>>>>>
>>>>>
>>>>
>>
>>
>> --
>> Erik Schnetter <schn...@gmail.com>
>> http://www.perimeterinstitute.ca/personal/eschnetter/
>>
>

Re: [julia-users] Re: `findmin` with arbitrary comparison operator?

[Júlio Hoffimann]

The issue on GitHub: https://github.com/JuliaLang/julia/issues/14216

Thanks,
-Júlio

2015-12-01 8:57 GMT-08:00 Dan <getz...@gmail.com>:

> suggestion: call the named argument `lt` to match the argument of `sort`.
>
>
> On Tuesday, December 1, 2015 at 6:23:07 PM UTC+2, Júlio Hoffimann wrote:
>>
>> Hi Erik,
>>
>> What I meant was a derived type from Julia's Tuple type. Though I agree
>> with you that this may be too much of an overhead for such a simple task.
>>
>> I'll open an issue on GitHub to ask if others are ok with an additional
>> predicate option a la C++:
>>
>> findmin(array, pred=<)
>>
>> If they are all positive, I'll add it for you.
>>
>> Best,
>> -Júlio
>>
>

Re: [julia-users] Re: `findmin` with arbitrary comparison operator?

[Júlio Hoffimann]

Hi Erik,

What I meant was a derived type from Julia's Tuple type. Though I agree
with you that this may be too much of an overhead for such a simple task.

I'll open an issue on GitHub to ask if others are ok with an additional
predicate option a la C++:

findmin(array, pred=<)

If they are all positive, I'll add it for you.

Best,
-Júlio

[julia-users] Re: `findmin` with arbitrary comparison operator?

[Júlio Hoffimann]

Hi Erik,

All you need to do is specialize the < operator for your tuple type.

Please check the definition of findmin by running:

@edit findmin([1,2,3])

You'll see the implementation is quite trivial. Let us know if you weren't 
able to make it work.

Best,
-Júlio

Em domingo, 29 de novembro de 2015 10:04:49 UTC-8, Erik Schnetter escreveu:
>
> Is there a version of `findmin` that allows passing in an arbitrary 
> comparison operator?
>
> In throw-away code, I often have an array containing tuples of both the 
> result and some additional operator. I want to find the minimum and its 
> location, but want to ignore the additional information. I would like to 
> use a user-defined comparison operator that looks only at one of the tuple 
> elements, yet `findmin` doesn't seem to offer this.
>
> -erik
>
> -- 
> Erik Schnetter  
> http://www.perimeterinstitute.ca/personal/eschnetter/
>

Re: [julia-users] float(X) vs. map(Float64, X)

[Júlio Hoffimann]

Hi David,

Yes, I got confused for a second. Yichao clarified on the GitHub issue.

I still don't know why these two functions are causing different behavior
in my package though.

-Júlio

2015-11-29 18:44 GMT-08:00 David P. Sanders :

> NaNs are not equal to each other.

Re: [julia-users] float(X) vs. map(Float64, X)

[Júlio Hoffimann]

It has something to do with NaN representation:

# 64bits
a = ones(10,10)
a[:,5] = NaN
b = float(a)
c = map(Float64, a)

b == c # this should be true, but it's not

I'll open an issue on GitHub.

-Júlio

2015-11-29 15:14 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Hi Yichao,
>
> What is the difference?
>
> -Júlio
>
> 2015-11-29 15:09 GMT-08:00 Yichao Yu <yyc1...@gmail.com>:
>
>> On Sun, Nov 29, 2015 at 6:02 PM, Júlio Hoffimann
>> <julio.hoffim...@gmail.com> wrote:
>> > Hi,
>> >
>> > Could you please confirm Xf = float(X) is a shortcut for Xf =
>> map(Float64,
>> > X) on a 64bits machine?
>>
>> No it isn't and it's independ of whether it's 32 or 64bits.
>>
>> Depending one what exactly you are doing, the error message should be
>> fairly clear about the difference.
>>
>> >
>> > For some reason the tests in my package fail when I replace one by the
>> > other.
>> >
>> > -Júlio
>>
>
>

Re: [julia-users] float(X) vs. map(Float64, X)

[Júlio Hoffimann]

Hi Eric,

I did all those steps, but thanks for spending time with it. :)

-Jùlio

2015-11-29 20:56 GMT-08:00 Eric Forgy <eric.fo...@gmail.com>:

> Hi Julio,
>
> I'm not sure if this helps (I'm still learning the ropes), but you can
> find the definition of "float(x)" in base/float.jl
> <https://github.com/JuliaLang/julia/blob/master/base/float.jl> on line
> 121:
>
> float(x) = convert(AbstractFloat, x)
>
> I found this from the following command:
>
> julia> which(float,(Number,))
> float(x) at float.jl:121
>
> Similarly,
>
> julia> which(map,(Float64,Number))
> map(f, x::Number) at number.jl:46
>
> where we find:
>
> map(f, x::Number, ys::Number...) = f(x, ys...)
>
> So if your "x" is a Number, then it looks like your asking if
>
> julia> float(10) == map(Float64,10)
> true
>
> But I think your "x" is probably an AbstractArray{Number}, in which case
> we find:
>
> julia> which(float,(AbstractArray{Number},))
> float{T}(A::AbstractArray{T,N}) at float.jl:431
>
> which is defined by:
>
> function float{T}(A::AbstractArray{T})
> if !isleaftype(T)
> error("`float` not defined on abstractly-typed arrays; please
> convert to a more specific type")
> end
> convert(AbstractArray{typeof(float(zero(T)))}, A)
> end
>
> Similarly:
>
> julia> which(map,(Float64,AbstractArray{Number}))
> map(f, A::AbstractArray{T,N}) at abstractarray.jl:1305
>
> and:
>
> function map(f, A::AbstractArray)
> if isempty(A)
> return isa(f,Type) ? similar(A,f) : similar(A)
> end
> first = f(A[1])
> dest = similar(A, typeof(first))
> dest[1] = first
> return map_to!(f, 2, dest, A)
> end
>
> AND... I run out of time. Gotta run. Hope it helps a little. Cheers!
>
> On Monday, November 30, 2015 at 11:43:55 AM UTC+8, Júlio Hoffimann wrote:
>>
>> Hi David,
>>
>> Yes, I got confused for a second. Yichao clarified on the GitHub issue.
>>
>> I still don't know why these two functions are causing different behavior
>> in my package though.
>>
>> -Júlio
>>
>> 2015-11-29 18:44 GMT-08:00 David P. Sanders <dpsa...@gmail.com>:
>>
>>> NaNs are not equal to each other.
>>
>>
>>

[julia-users] julia --track-allocation option is not generating *.mem file

[Júlio Hoffimann]

Hi,

The --track-allocation option on latest (master) Julia is not generating 
the *.mem file as described in the docs.

$ julia --track-allocation=user foo.jl

Anything changed recently? How to profile memory allocations nowadays?

-Júlio

Re: [julia-users] julia --track-allocation option is not generating *.mem file

[Júlio Hoffimann]

I opened an issue: https://github.com/JuliaLang/julia/issues/14172

Thanks,
-Júlio

2015-11-27 9:46 GMT-08:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Hi,
>
> The --track-allocation option on latest (master) Julia is not generating
> the *.mem file as described in the docs.
>
> $ julia --track-allocation=user foo.jl
>
> Anything changed recently? How to profile memory allocations nowadays?
>
> -Júlio
>

[julia-users] String argument for @sprintf

[Júlio Hoffimann]

Hi,

Could you please explain why this works:

@sprintf("foo%02i", 3)

but this doesn't:

s = string("foo","%02i")
@sprintf(s, 3)

How is a formatted string different than an usual string?

-Júlio

Re: [julia-users] Re: String argument for @sprintf

[Júlio Hoffimann]

Thank you Kristoffer, I had forgot about that thread on Stack Overflow.

Will use Formatting.jl

Cheers,
-Júlio

2015-11-18 10:40 GMT-08:00 Kristoffer Carlsson <kcarlsso...@gmail.com>:

> https://groups.google.com/forum/#!topic/julia-users/iG6qwZ_GWzA
>
>
> http://stackoverflow.com/questions/19783030/in-julia-why-is-printf-a-macro-instead-of-a-function
>
> https://groups.google.com/forum/#!topic/julia-users/7Sn5yys0UJE
>
> etc
>
>
> On Wednesday, November 18, 2015 at 6:38:09 PM UTC+1, Júlio Hoffimann wrote:
>>
>> Hi,
>>
>> Could you please explain why this works:
>>
>> @sprintf("foo%02i", 3)
>>
>> but this doesn't:
>>
>> s = string("foo","%02i")
>> @sprintf(s, 3)
>>
>> How is a formatted string different than an usual string?
>>
>> -Júlio
>>
>

Re: [julia-users] Re: Hausdorff distance

[Júlio Hoffimann]

Thanks, will take a look.

-Júlio

Re: [julia-users] Hausdorff distance

[Júlio Hoffimann]

I'll open an issue for it, thanks.

-Júlio

2015-10-23 15:28 GMT-07:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Hi,
>
> I want to make the Hausdorff distance (
> https://en.wikipedia.org/wiki/Hausdorff_distance) available in Julia, is
> the Distances.jl package a good fit or I should create a separate package
> just for this distance between point sets?
>
> I can think of a very simple (naive) implementation:
>
> using Distances
>
> A, B # matrices which columns represent the points in the pointset
> D = pairwise(Euclidean(), A, B)
> daB = maximum(minimum(D,2))
> dbA = maximum(minimum(D,1))
> result = max(daB, dbA)
>
> -Júlio
>

[julia-users] Hausdorff distance

[Júlio Hoffimann]

Hi,

I want to make the Hausdorff distance 
(https://en.wikipedia.org/wiki/Hausdorff_distance) available in Julia, is 
the Distances.jl package a good fit or I should create a separate package 
just for this distance between point sets?

I can think of a very simple (naive) implementation:

using Distances

A, B # matrices which columns represent the points in the pointset
D = pairwise(Euclidean(), A, B)
daB = maximum(minimum(D,2))
dbA = maximum(minimum(D,1))
result = max(daB, dbA)

-Júlio

Re: [julia-users] Re: Old Julia profiles in Jupyter session

[Júlio Hoffimann]

Hi David, what files exactly? I tried to delete the profiles, but I don't
see them under .ipython nor .jupyter

-Júlio

Re: [julia-users] Re: Old Julia profiles in Jupyter session

[Júlio Hoffimann]

Thank you Steven, problem solved. Found the kernels directory and deleted
the old versions.

-Júlio

2015-10-05 16:13 GMT-07:00 Steven G. Johnson <stevenj@gmail.com>:

>
>
> On Monday, October 5, 2015 at 5:19:51 PM UTC-4, Júlio Hoffimann wrote:
>>
>> Whenever I update Julia I see a new profile in my Jupyter session. How to
>> delete all these old versions in the attached screenshot?
>>
>
>  First, I have to say that your screenshot is odd.  New profiles should
> only be installed if the minor version changes (e.g. 0.4 to 0.5); any other
> change (e.g. 0.4-dev to 0.4-rc3) should overwrite the existing profile.
> I've updated Julia many times and only have three profiles (0.3, 0.4, and
> 0.5).   The next time you see a new profile appearing for the same minor
> version number, please file a bug report with IJulia and we'll try to find
> out what happened.
>
> Second, the kernels for Jupyter are in directories in one of the jupyter
> data directories -- run "jupyter --paths" to list all the directories where
> Jupyter stores its stuff.  I think in the first path listed under "data:"
> is where you'll find a directory called "kernels" that contains
> subdirectories for all the kernels that are installed.  Delete the ones you
> don't want.
>

[julia-users] Travis build failing

[Júlio Hoffimann]

Hi,

All my packages are failing the build on Travis/Linux even though I didn't 
touch the code recently. The build on Travis/Mac is working.

For instance: https://github.com/juliohm/GeoStatsImages.jl

Could you please confirm the error is in the Travis/Julia side?

-Júlio

Re: [julia-users] Package badge URLs for nightly build

[Júlio Hoffimann]

I'll proceed and update the URLs.

Thanks,
-Júlio

Re: [julia-users] Package badge URLs for nightly build

[Júlio Hoffimann]

Anyone has this info?

-Júlio

2015-09-29 10:56 GMT-07:00 Júlio Hoffimann <julio.hoffim...@gmail.com>:

> Hi,
>
> Could you please confirm the badge URLs for nightly builds are obsolete
> and that I have to use a specific Julia version now?
> https://github.com/juliohm/ImageQuilting.jl/blob/master/README.md
>
> -Júlio
>

[julia-users] Package badge URLs for nightly build

[Júlio Hoffimann]

Hi,

Could you please confirm the badge URLs for nightly builds are obsolete and 
that I have to use a specific Julia version 
now? https://github.com/juliohm/ImageQuilting.jl/blob/master/README.md

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Hi Kristoffer, sorry for the delay and thanks for the code.

What I want to do is very simple: I have an expensive loop for i=1:N such
that each iteration is independent and produces a large array of size M.
The result of this loop is a matrix of size MxN. I have many CPU cores at
my disposal and want to distribute this work.

In the past I accomplished that with MPI in Python:
https://github.com/juliohm/HUM/blob/master/pyhum/utils.py Whenever a
process in the pool is free it consumes an iteration of the loop. What
exactly the @parallel macro in Julia is doing? How can I modify the code I
previously posted in this thread to achieve such effect?

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Hi Ismael,

MPI is distributed memory, I'm trying to use all the cores in my single
workstation with shared memory instead. Thanks for the link anyways.

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Hi Sebastian, thanks for sharing your experience in parallelizing Julia
code. I used OpenMP in the past too, it was very convenient in my C++
codebase. I remember of an initiative OpenACC that was trying to bring
OpenMP and GPU accelerators together, I don't know the current status of
it. It may be of interest to Julia devs.

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

What am I doing wrong in the following code?

function foo(N; parallel=false)
  if parallel  nprocs()  CPU_CORES
addprocs(CPU_CORES - nprocs())
  end

  result = SharedArray(Float64, 9, N)
  @parallel for i=1:N
sleep(1)
result[:,i] = rand(3,3)[:]
  end

  result
end

If I call foo(60, parallel=true), result is all zeros. Expected behavior is
a random matrix.

-Júlio

[julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Hi,

Suppose I have a complicated but embarrassingly parallel loop, 
namely: 
https://github.com/juliohm/ImageQuilting.jl/blob/master/src/iqsim.jl#L167

How would you dispatch the iterations so that all cores in the client 
computer are busy working? There is any construct in the language for that 
usual scenario?

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Consider the following simplified example. There is an algorithm
implemented as a function foo(N). This algorithm repeats the same recipe N
times in a loop to fill in an array of arrays:

function foo(N)
  # bunch of auxiliary variables goes here
  # ...

  result = []
  for i=1:N
# complicated use of auxiliary variables to produce result[i]
# ...

push!(result, rand(3,3)) # push result[i] to the vector
  end

  return result
end

Each iteration is expensive and we would like to add an option foo(N;
parallel=false) to use all cores available. I would start with:

function foo(N; parallel=false)
  if parallel
addprocs(CPU_CORES - 1)
  end

# bunch of auxiliary variables goes here
# ...

  result = SharedVector(???)
  @parallel for i=1:N
# complicated use of auxiliary variables to produce result[i]
# ...

push!(result, rand(3,3))
  end

  return result
end

How to initialize the SharedVector? Is this the correct approach? What
about the auxiliary variables, will them be copied to all workers? If yes,
I can just replace Arrays by SharedArrays so that they are visible
everywhere without copying?

-Júlio

Re: [julia-users] Embarrassingly parallel workload

[Júlio Hoffimann]

Thank you Tim, will check it carefully.

-Júlio

Re: [julia-users] Re: Interact - basic usage

[Júlio Hoffimann]

Thank you Shashi.

-Júlio

[julia-users] Interact and PyPlot - @lift and withfig()

[Júlio Hoffimann]

Hi,

Suppose I have:

s = slider(1:10)
img = @lift eye(s)

How can I create the interactive plot in Jupyter using @lift?

@lift imshow(img)

I know @manipulate has the withfig() option where we can pass the PyPlot 
Figure object, what about @lift?

-Júlio

Re: [julia-users] Interact and PyPlot - @lift and withfig()

[Júlio Hoffimann]

Hi Shashi, thank you very much for your help.

What is slider_val in your code snippet? I actually have multiple sliders
and other widgets that I want to play with before plotting. Is there any
version of @manipulate where I can pass all sorts of widgets at once?

@manipulate slider, checkbox, togglebuttons, ... withfig() do
 ... plot commands ...
end

-Júlio

2015-08-02 11:51 GMT-07:00 Shashi Gowda shashigowd...@gmail.com:

 Hi,

 Easiest way to do this is with @manipulate and `withfig` as you have
 discovered.

 I'm inclined to remove @lift from reactive since it's hard to implement
 correctly and has caused frustrating problems. (also it's not eval-free)

 I recommend you use the lift function instead.

 s = slider(1:10)
 display(s)
 lift(x - greyim(eye(x)), s)

 Should do the trick for you.

 With PyPlot, this will be:

 f = figure()
 lift(s) do slider_val
withfig(f) # This basically says do the drawing on the same plot f.
 plot something with slider_val...
 end






 On Mon, Aug 3, 2015 at 12:06 AM, Júlio Hoffimann 
 julio.hoffim...@gmail.com wrote:

 Hi,

 Suppose I have:

 s = slider(1:10)
 img = @lift eye(s)

 How can I create the interactive plot in Jupyter using @lift?

 @lift imshow(img)

 I know @manipulate has the withfig() option where we can pass the PyPlot
 Figure object, what about @lift?

 -Júlio

Re: [julia-users] Interact and PyPlot - @lift and withfig()

[Júlio Hoffimann]

Hi Shashi,

When I type:

s = slider(1:10)
a = @lift eye(s)
display(s)

fig = figure()
@manipulate for a=a; withfig(fig) do
imshow(a)
end
end

I get the interactive plot correct, but an extra undesired print of the
array in Jupyter. What I can do to fix that?

-Júlio

Re: [julia-users] Interact and PyPlot - @lift and withfig()

[Júlio Hoffimann]

I got it, the widgets are shown automatically by @manipulate, I'm changing
the code already.

Thanks,
Júlio.

2015-08-02 16:42 GMT-07:00 Júlio Hoffimann julio.hoffim...@gmail.com:

 Hi Shashi,

 When I type:

 s = slider(1:10)
 a = @lift eye(s)
 display(s)

 fig = figure()
 @manipulate for a=a; withfig(fig) do
 imshow(a)
 end
 end

 I get the interactive plot correct, but an extra undesired print of the
 array in Jupyter. What I can do to fix that?

 -Júlio

[julia-users] Interact - basic usage

[Júlio Hoffimann]

Hi,

Suppose I have:

idx = slider(1:3)

How do I use the value stored in the widget to index an array, let's say 
A[idx]? I tried playing with signal() and @lift, but none worked. Could you 
please explain what is the correct approach?

-Júlio

[julia-users] NaN definition for Integer types?

[Júlio Hoffimann]

Hi,

Is there any definition of NaN for Integer types? Let's say I want to 
create a matrix with some unspecified entries, that is entries marked 
special. I am using NaN for Float64 but would like to make it work with 
other types too. Suggestions?

-Júlio

Re: [julia-users] NaN definition for Integer types?

[Júlio Hoffimann]

Thank you all, that is what I thought, I will stick with plain Float64 for
now.

-Júlio

2015-07-24 10:59 GMT-07:00 Erik Schnetter schnet...@gmail.com:

 On Fri, Jul 24, 2015 at 1:09 PM, Júlio Hoffimann 
 julio.hoffim...@gmail.com wrote:

 Hi,

 Is there any definition of NaN for Integer types? Let's say I want to
 create a matrix with some unspecified entries, that is entries marked
 special. I am using NaN for Float64 but would like to make it work with
 other types too. Suggestions?


 There is no pre-defined integer nan equivalent.

 Option 1 (fast but unsafe): Pick a special int value that is rarely used,
 e.g. typemin(Int).
 Option 2 (works with all types, including Float64, and is safer): Use a
 nullable type. This might complicate your code and may slightly increase
 your memory usage.

 -erik

 --
 Erik Schnetter schnet...@gmail.com
 http://www.perimeterinstitute.ca/personal/eschnetter/

Re: [julia-users] Re: Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

What I'm actually trying to do is create a type for spatial data:

typealias SpatialData Dict{Tuple{Integer,Integer,Integer},Real}

data = SpatialData([(i,j,k) = rand() for i=1:10, j=1:20, k=1:30])

However I need to do some checks on the size of the bounding box,
(10,20,30) in this case.

What I'm doing right now is loop over keys(data), but I would like to make
it a matrix and simply apply the builtin maximum() to the columns. Does
anyone suggest a better design? How to create such a type to support these
checks?

I also tried to play with sparse matrices, but I need 3D spatial data
representation.

-Júlio

Re: [julia-users] Re: Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

Hi Matt,

That is a very good suggestion! I'm using Julia 0.4, how would you retrieve
the locations from an AbstractArray?

immutable SpatialData : AbstractArray{Real,3} end

is all that is needed to define the type?

-Júlio

Re: [julia-users] Re: Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

Yes, unfortunately the unset values shouldn't be listed as 0 for instance.
I'll try move forward with my current implementation and see if the code is
ok.


Thanks,
-Júlio

[julia-users] Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

Hi,

How to convert:

1000-element Array{Tuple{Integer,Integer,Integer},1}:
 (10,2,1) 
 (5,7,10) 
 (5,7,4)  
 (1,1,6)  
 (2,3,6)  
 (8,6,4)  
 (10,2,4) 
 (1,3,9)  
 (9,3,7)  
 (5,2,4)  
 ⋮
 (1,6,8)  
 (4,6,6)  
 (3,9,5)  
 (10,4,10)
 (8,7,4)  
 (4,8,9)  
 (2,6,10) 
 (3,6,5)  
 (1,7,10) 

into the corresponding 1000x3 matrix in a clean fashion?

-Júlio

Re: [julia-users] Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

Hi Tim!

I'll try the loop or change some types to avoid the issue.

Thank you!

-Júlio

Re: [julia-users] Re: Convert Array{Tuple} to Matrix

[Júlio Hoffimann]

Hi Simon,

Thank you, will check it later.

-Júlio

Re: [julia-users] Tests failing on latest Julia

[Júlio Hoffimann]

Hi Matt,

The tests are still failing for the same reason, should I open an issue on
GitHub?

-Jùlio

Re: [julia-users] Tests failing on latest Julia

[Júlio Hoffimann]

Thank you very much Matt.

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Yes Kristoffer, very similar effect.

Do you think it is good idea to have that function in Base?

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Hi,

I would like to share the solution for my use case with you. I needed the
Vandermonde-ish matrix to implement another method in GeoStats.jl
https://github.com/juliohm/GeoStats.jl

The multinom_exp()
https://github.com/juliohm/GeoStats.jl/blob/master/src/utils.jl#L55 returns
the exponents in the multinomial expansion without expression generation. I
then call it inside unikrig()
https://github.com/juliohm/GeoStats.jl/blob/master/src/kriging.jl#L116 to
generate the matrix F.

Questions:

1. Any interest in adding multinom_exp() to Base?
2. Am I using the Julia doc system correctly? I can retrieve the doc
strings by typing '?' followed by the name of the function in the Julia
prompt, but not with help(funcname).

Thank you all,
-Júlio

Re: [julia-users] Equivalent to MATLAB/Octave accumarray()

[Júlio Hoffimann]

Totally agree Tim, thank you for sharing your thoughts on that.

-Júlio

Re: [julia-users] Equivalent to MATLAB/Octave accumarray()

[Júlio Hoffimann]

Thank you Isaiah, is there a reason to not add it to Base?

-Júlio

[julia-users] Equivalent to MATLAB/Octave accumarray()

[Júlio Hoffimann]

Hi,

Is there any equivalent to accumarray() in 
Julia? http://www.mathworks.com/help/matlab/ref/accumarray.html

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Hi Steven,

That is awesome, it looks like it's only generating the terms of order N,
but I'll workaround to make it generate all terms from order 0 to N.

Thank you all for this incredible help.

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

David, thank you very much for the detailed explanation, will read through
it.

Steven, sorry for not explaining the problem clearly. My issue is basically
trying to figure out a way to loop over all possible combinations of
exponents (e1, e2, e3) for which the sum is less or equal to N and generate
a vector with the terms Xj[1]^e1*Xj[2]^e2*Xj[3]^e3.

The j-th column of the resulting matrix contains the terms of this
polynomial evaluated at the j-th column of X.

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Thank you Matt, I used itertools for similar problems in the past, will
check it again.

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Thanks Daniel, I remember Base.Cartesian from previous posts, will
definitively check it.

-Júlio

Re: [julia-users] Re: Generate polynomial expression on the fly

[Júlio Hoffimann]

Hi Steven,

I'm actually trying to pass in a matrix X and get out the associated
Vandermonde-like matrix as numbers. I thought of expressions because the
loop itself is not trivial, we have to deal with all those combinatorial
indexing somehow.

I have a code that generates the exponents of all the monomials as a tuple,
but going from this tuple to the actual product is not clear to me. Let's
say I have (0,1,0) meaning x^0*y^1*z^0 = y^1. How to do this conversion?
Any trick?

-Júlio