Re: [julia-users] poor performance of threads

[pevnak]

Dear Sam,
the output of the benchmark is following

105.290122 seconds (31.43 k allocations: 1.442 MB, 0.00% gc time)
107.445101 seconds (1.37 M allocations: 251.368 MB, 0.12% gc time)

Tomas

[julia-users] Re: [ANN] MathGL graphics library wrapper

[lewis . hein]

Oooh. Now that should not be happening. The mimetypes.jl file is definitely 
part of the package on my end

As for the error with missing Splines.jl, I didn't realize that that was 
still a dependency. I will post the Splines.jl package as soon as I can, 
probably later today. I will also try to add installation instructions.

All of your testing is very much appreciated. I think I'd better set up a 
"permanently clean" VM where I can test some of this stuff and create fewer 
annoying hassles for others trying to use my code

Lewis

On Friday, March 4, 2016 at 3:36:51 PM UTC-7, Christoph Ortner wrote:
>
> Further down the source file, there is an include
>
> include("mimetypes.jl")
>
> but there is no file mimetypes.jl anywhere. Is it possible that the 
> repository is incomplete?
>
> C
>

Re: [julia-users] Re: [ANN] MathGL graphics library wrapper

[lewis . hein]

OK, I apologize for the problems. The reason for the error on "using 
Splines" is that I also am working on a splines package that I need and 
added some plotting features to plot my splines. So, you can either delete 
that line and any functions that take a Spline as an argument (There aren't 
many) , or you can wait until I get my splines library posted. (I don't 
know how long this will be; hopefully soon). A third option would be to 
take a look here:

https://github.com/thomastaudt/MathGL.jl

It appears that someone else had the same idea that I did. His probably has 
way fewer rough edges, too. If you want my plotOpStack functionality, then 
that would be trivially easy (a few dozen lines of code) to add to any 
julia-based plotting system, MathGL.jl, gadfly, etc.

As for installing MathGL, it should be in the repo for most mainstream 
linux distros; If you are running windows then I think they have an 
installer.

It should be possible to have MathGL.jl install the MathGL library 
automatically, given a developer who knows their way around the julia 
packaging system sufficiently. The one thing that gave me pause is (other 
than that I don't know my way around the packaging system that well) is 
that MathGL is a 20 Mb download, which would hugely increase the size of my 
package

Lewis
On Friday, March 4, 2016 at 12:27:22 PM UTC-7, Christoph Ortner wrote:
>
>
> looks really nice, but I just spent 30 minutes trying to install MathML. 
> This is when I lose interest. Sorry. 
>
> Any chance to have MathML.jl install MathML automatically?
>
> Christoph
>
>

[julia-users] Re: [ANN] MathGL graphics library wrapper

[Christoph Ortner]

Further down the source file, there is an include

include("mimetypes.jl")

but there is no file mimetypes.jl anywhere. Is it possible that the 
repository is incomplete?

C

[julia-users] Re: [ANN] MathGL graphics library wrapper

[Christoph Ortner]

Would be nice to post installation istrunctions on the MathGL.jl README. It 
took me a while to realise that MathGL can just be installed via home-brew.

brew install homebrew/science/mathgl

Anyhow, there is still the problem with the error message. What is 
"Splines"?

Re: [julia-users] poor performance of threads

[Sam Kaplan]

Hello,

Perhaps it might help the discussion to try running one of the Julia 
threading performance benchmark codes on your MacBook Air.  For example:

https://github.com/JuliaLang/julia/blob/master/test/perf/threads/laplace3d/laplace3d.jl

Presumably that code is type-stable and compute-bound since it was written 
for the purpose of testing the threading code.  Anyways, if you had time to 
do that, I would certainly be interested in the result.  Also, the 
following GitHub issue might be of interest:

https://github.com/JuliaLang/julia/issues/14417


Thanks!

Sam

On Friday, March 4, 2016 at 8:14:33 AM UTC-6, pev...@gmail.com wrote:
>
> Thank you very much Tim.
> I am using the profiler and your package ProfileView quite extensively and 
> I know where is my Achille heel in the code, and it is cpu bound.  That's 
> why I am so puzzled with threads.
>
> I will try to use @code_warntype, never use it before. 
>
> Best wishes,
> Tomas
>

Re: [julia-users] Re: [ANN] MathGL graphics library wrapper

[Christoph Ortner]

looks really nice, but I just spent 30 minutes trying to install MathML. 
This is when I lose interest. Sorry. 

Any chance to have MathML.jl install MathML automatically?

Christoph

Re: [julia-users] Re: [ANN] MathGL graphics library wrapper

[Chris]

Oh right, sorry, that's the error message I get as well. The one in my
original reply was from trying to do `Pkg.add("Splines")`

On Fri, Mar 4, 2016 at 1:51 PM Christoph Ortner 
wrote:

> Actually in my case I get the error message
>
> julia> using MathGL
> ERROR: LoadError: ArgumentError: Splines not found in path
>
>
>

[julia-users] Re: [ANN] MathGL graphics library wrapper

[Christoph Ortner]

Actually in my case I get the error message 

julia> using MathGL
ERROR: LoadError: ArgumentError: Splines not found in path

[julia-users] Re: [ANN] MathGL graphics library wrapper

[Christoph Ortner]

I get the same - I assumed one needs to install MathGL first. Have you 
tried this?
C

On Friday, 4 March 2016 18:44:09 UTC, Chris wrote:
>
> Thanks, I haven't heard of MathGL before, and it looks interesting. When I 
> try to use your module, I get "ERROR: unknown package Splines", from the 
> "using Splines" line. There doesn't appear to be a Splines.jl registered in 
> METADATA.
>
> On Friday, March 4, 2016 at 1:30:00 PM UTC-5, Lewis Hein wrote:
>>
>> Hi all 
>>
>> MathGL is a really nice plotting library that I have been using 
>> extensively for the past year. When I took the plunge recently and 
>> switched my research codebase over to julia, I wanted MathGL to follow 
>> it, especially as there are some things such as animations, windowing 
>> toolkit widgets, and the incredible customizeability and diversity of 
>> the plots that I couldn't find in other plotting packages. Therefore, I 
>> took the obvious solution and built a julia interface. The code is on 
>> github here: 
>>
>> https://github.com/LewisHein/MathGL.jl 
>>
>> Lewis 
>>
>

[julia-users] Re: [ANN] MathGL graphics library wrapper

[Chris]

Thanks, I haven't heard of MathGL before, and it looks interesting. When I 
try to use your module, I get "ERROR: unknown package Splines", from the 
"using Splines" line. There doesn't appear to be a Splines.jl registered in 
METADATA.

On Friday, March 4, 2016 at 1:30:00 PM UTC-5, Lewis Hein wrote:
>
> Hi all 
>
> MathGL is a really nice plotting library that I have been using 
> extensively for the past year. When I took the plunge recently and 
> switched my research codebase over to julia, I wanted MathGL to follow 
> it, especially as there are some things such as animations, windowing 
> toolkit widgets, and the incredible customizeability and diversity of 
> the plots that I couldn't find in other plotting packages. Therefore, I 
> took the obvious solution and built a julia interface. The code is on 
> github here: 
>
> https://github.com/LewisHein/MathGL.jl 
>
> Lewis 
>

[julia-users] [ANN] MathGL graphics library wrapper

[Lewis Hein]

Hi all

MathGL is a really nice plotting library that I have been using
extensively for the past year. When I took the plunge recently and
switched my research codebase over to julia, I wanted MathGL to follow
it, especially as there are some things such as animations, windowing
toolkit widgets, and the incredible customizeability and diversity of
the plots that I couldn't find in other plotting packages. Therefore, I
took the obvious solution and built a julia interface. The code is on
github here:

https://github.com/LewisHein/MathGL.jl

Lewis

[julia-users] Re: How to upload file and also specify data options with Requests.jl?

[Tyler Nappy]

Hi,

I would check out the Requests.jl library 
. The following code POSTs a file 
to the OCR Document API and reads the response:

Pkg.add("Requests")

using Requests
import Requests: get, post, put, delete, options

filenamne = "/path/to/file"

*response = post("http://api.havenondemand.com/1/api/sync/ocrdocument/v1;; 
data=Dict("apikey"=>"YOUR_APIKEY", 
"file"=>FileParam(open(filename,"r"),"text/julia","file2","file2.jl",true)))*

*readall(response)*


I work for HPE.

On Wednesday, December 2, 2015 at 8:35:40 PM UTC-5, Ismael Venegas Castelló 
wrote:
>
> Curl:
>
> curl -X POST --form "apikey=" --form "file=@myscan.pdf" --form 
> "mode=document_photo" https://api.idolondemand.com/1/api/async/ocrdocument/v1
>
>
>
> HTML:
>
> 
> 
> Multipart Form Example
> 
> 
> https://api.idolondemand.com/1/api/sync/ocrdocument/v1;
>  method="post" enctype="multipart/form-data">
>  value="your-apikey-here">
> 
> Submit
> 
> 
> 
>
>
> Node:
>
> var needle = require('needle');
> var data = {
>   apikey: myapikey,
>   file: { file: 'myscan.pdf', content_type: 'multipart/form-data' }
> }
>
> needle.post('https://api.idolondemand.com/1/api/async/ocrdocument/v1', data, 
> { multipart: true }, function(err, resp, body) {
>   console.log(body)
>   // needle will read the file and include it in the form-data as binary
> });
>
>
> Ruby:
>
> require 'httpclient'
> require 'json'
>
> data={file:open("myscan.pdf"), mode:"document_photo", apikey:"apikey"}
> clnt = HTTPClient.new
> resp=clnt.post(url, data)
> body=JSON.parse(resp.body)
>
>
> Python:
>
> files = {'file': open('myscan.pdf', 'rb')}
> data= {'apikey':myapikey,'mode':'document_photo'}
> requests.post("https://api.idolondemand.com/1/api/async/ocrdocument/v1",data=data,files=files)
> print r.json()
>
>
> Julia ...how to???
>
> El miércoles, 2 de diciembre de 2015, 19:20:51 (UTC-6), Ismael VC escribió:
>>
>> Hello everyone!
>>
>> I've been trying/learning how to wrap HPE Haven OnDemand API 
>>  with Julia: HavenOnDemand.jl. but I 
>> havent figured out how to send multipart data, plus other parameters, it 
>> seems to me that one can only do one or the other:
>>
>> "Main `HavenOnDemand` function, used to call **Haven OnDemand** API."
>> function call_HOD(
>> endpoint:: String,
>> async   :: Bool,# Some endpoints are `async_only`.
>> files   :: Vector{FileParam},
>> options :: Dict;
>> api_url :: String = "https://api.havenondemand.com;,
>> version :: Int= 1,
>> default_version :: Int= 1,
>> )
>>
>> try
>> options["apikey"] = _HOD_API_KEY
>> catch
>> error("Use `HavenOnDemand.set_api_key(api_key::AbstractString)` 
>> first.")
>> end
>>
>> sync_str = async ? "async" : "sync"
>> r = post(
>> 
>> "$(api_url)/$(version)/api/$(sync_str)/$(endpoint)/v$(default_version)",
>> files = files,# <--- HERE'S THE PROBLEM!
>> data = options
>> )
>> return r.status == 200 ? json(r) : throw(HODException(r))
>> end
>>
>> "`DataFrame` that holds the `HavenOnDemand` data used wrap the API."
>> const _HOD_API = readtable(
>> joinpath(Pkg.dir("HavenOnDemand"), "src", "api.data"),
>> separator = ' '
>> )
>>
>> # Meta wrap most of the API:
>> for row in eachrow(_HOD_API::DataFrame)
>> func_name, endpoint, async_only, description = [v for (k, v) in row]
>> title = join([ucfirst(s) for s in split(func_name, '_')], ' ')
>> docstring = """
>> **HPE Haven OnDemand: $(title)**
>>
>> `$(func_name)([kwargs...])`
>>
>> $description
>>
>> All the arguments are optional and they must be supplied as 
>> keyword
>> arguments, non valid keyword names are ignored.
>>
>> For information about valid arguments, visit:
>>
>> * https://dev.havenondemand.com/apis/$(endpoint)
>> """
>>
>> @eval begin
>> @doc $docstring ->
>> function $(symbol(func_name))(; file = [], kwargs...)
>> return call_HOD(
>> $endpoint,
>> $async_only,
>> [FileParam(open(f)) for f in file],
>> Dict(kwargs)
>> )
>> end
>> end
>> end
>>
>> The error I get is:
>>
>> ERROR: Multiple body options specified. Please only specify one
>>  in do_stream_request at 
>> C:\Users\Peter\.julia\v0.4\Requests\src\Requests.jl:268
>>
>> If I try to use both *`file`* and *`data`* in the *`Requests.post`* 
>> function. I need both because I send the *`apikey`* within the *`data`* 
>> parameter, if I just use *`file`*, then the *`apikey`* is not passed to 
>> the API and the request fails
>>
>> These are some examples of what I'm trying to achieve: 
>> 

Re: [julia-users] Julia programs in pipes

[Stefan Karpinski]

Just read from STDIN and write to STDOUT and this should work as in any
other language. You may need to call `flush(STDOUT)` sometimes since output
is buffered – depends on what kind of buffering you need.

On Fri, Mar 4, 2016 at 12:05 PM, Ben Ward  wrote:

> Hi,
>
> I'm wondering how easy it is to make a Julia script that can be part of a
> shell pipeline `cat file > julia myscript.jl > output.txt`?
>
> When reading the documentation, I get the impression most io is designed
> to be done all at once. I know how to loop over a file handle or stream,
> and I think
> STDOUT and STDIN are accessible from Julia, I'm just unsure about how to
> make it act as one link in a longer shell pipe.
> For example, if I were to loop using a stream of STDIN, waiting for a
> piece of data from the output of previous step instead of just thinking
> STDIN is finished or just waiting for all of it,
> I'd like things to move through piecemeal and so on, like a node stream or
> unix pipe does. Are there any good examples of people doing this?
>
> Until now I've just read in whole files or iterated over lines of files,
> and analysed them.
>
> Thanks,
> Ben.
>

[julia-users] Re: Incomplete parametric types

[Christopher Alexander]

Ah, yep.  I figured I might have to at least know the type ahead of time. 
 This should work though, thanks!!

Chris

On Friday, March 4, 2016 at 12:26:49 PM UTC-5, John Myles White wrote:
>
> Is this what you want?
>
>
> julia> abstract ABC
>
>
> julia> type A <: ABC end
>
>
> julia> type B <: ABC end
>
>
> julia> 
>
>
> julia> type TestType{T <:ABC}
>
>a::Float64
>
>b::T
>
>
>
>TestType(a::Float64) = new(a)
>
>end
>
>
> julia> myT = TestType{A}(4.0)
>
> TestType{A}(4.0,#undef)
>
>
> julia> myT.b = A()
>
> A()
>
>
> julia> myT
>
> TestType{A}(4.0,A())
>
> On Friday, March 4, 2016 at 9:19:42 AM UTC-8, Christopher Alexander wrote:
>>
>> Hi all,
>>
>> Is there anyway to do something like the following?
>>
>> abstract ABC
>>
>> type A <: ABC end
>>
>> type B <: ABC end
>>
>> type TestType{T <:ABC}
>> a::Float64
>> b::T
>> 
>> TestType(a::Float64) = new(a)
>> end
>>
>> myT = TestType(4.0)
>> myT.b = A()
>>
>> I am wondering if you can incompletely initialize a parametric type, and 
>> then set the actual value needed later.  The above code doesn't work, but 
>> that is what I'm trying to do.  The alternative is I guess to have some 
>> default Null Type, but then to get the performance gains I have to copy the 
>> object when I want to actually set it with the value that I want.
>>
>> Thanks!
>>
>> Chris
>>
>

[julia-users] Re: Incomplete parametric types

[John Myles White]

 

Is this what you want?


julia> abstract ABC


julia> type A <: ABC end


julia> type B <: ABC end


julia> 


julia> type TestType{T <:ABC}

   a::Float64

   b::T

   

   TestType(a::Float64) = new(a)

   end


julia> myT = TestType{A}(4.0)

TestType{A}(4.0,#undef)


julia> myT.b = A()

A()


julia> myT

TestType{A}(4.0,A())

On Friday, March 4, 2016 at 9:19:42 AM UTC-8, Christopher Alexander wrote:
>
> Hi all,
>
> Is there anyway to do something like the following?
>
> abstract ABC
>
> type A <: ABC end
>
> type B <: ABC end
>
> type TestType{T <:ABC}
> a::Float64
> b::T
> 
> TestType(a::Float64) = new(a)
> end
>
> myT = TestType(4.0)
> myT.b = A()
>
> I am wondering if you can incompletely initialize a parametric type, and 
> then set the actual value needed later.  The above code doesn't work, but 
> that is what I'm trying to do.  The alternative is I guess to have some 
> default Null Type, but then to get the performance gains I have to copy the 
> object when I want to actually set it with the value that I want.
>
> Thanks!
>
> Chris
>

[julia-users] Incomplete parametric types

[Christopher Alexander]

Hi all,

Is there anyway to do something like the following?

abstract ABC

type A <: ABC end

type B <: ABC end

type TestType{T <:ABC}
a::Float64
b::T

TestType(a::Float64) = new(a)
end

myT = TestType(4.0)
myT.b = A()

I am wondering if you can incompletely initialize a parametric type, and 
then set the actual value needed later.  The above code doesn't work, but 
that is what I'm trying to do.  The alternative is I guess to have some 
default Null Type, but then to get the performance gains I have to copy the 
object when I want to actually set it with the value that I want.

Thanks!

Chris

[julia-users] Julia programs in pipes

[Ben Ward]

Hi,

I'm wondering how easy it is to make a Julia script that can be part of a 
shell pipeline `cat file > julia myscript.jl > output.txt`?

When reading the documentation, I get the impression most io is designed to 
be done all at once. I know how to loop over a file handle or stream, and I 
think
STDOUT and STDIN are accessible from Julia, I'm just unsure about how to 
make it act as one link in a longer shell pipe.
For example, if I were to loop using a stream of STDIN, waiting for a piece 
of data from the output of previous step instead of just thinking STDIN is 
finished or just waiting for all of it,
I'd like things to move through piecemeal and so on, like a node stream or 
unix pipe does. Are there any good examples of people doing this?

Until now I've just read in whole files or iterated over lines of files, 
and analysed them.

Thanks,
Ben.

Re: [julia-users] ERROR: LoadError: syntax: incomplete: premature end of input

[Stefan Karpinski]

That would do it :-)

On Thursday, March 3, 2016, Fei Ma  wrote:

> Many thanks for the reply. I re-checked my code. It seems like that I have
> left out a ")".
>
>
>
> On Friday, 4 March 2016 02:37:58 UTC+11, Stefan Karpinski wrote:
>>
>> It means what entered in new.jl isn't a complete Julia expression.
>> Perhaps you're missing a closing `end` or something?
>>
>> On Thu, Mar 3, 2016 at 6:58 AM, Fei Ma  wrote:
>>
>>> Hello. Here I have a problem.
>>>
>>> I run a program, but stopped  it when it is running, and then the
>>> terminal output this :
>>>
>>> ERROR: LoadError: syntax: incomplete: premature end of input
>>>  in include at ./boot.jl:261
>>>  in include_from_node1 at ./loading.jl:304
>>> while loading /home/i/Documents/Julia/new.jl, in expression starting on
>>> line 80
>>>
>>> I don't know what  it means. Does anyone have such problems?
>>>
>>
>>

Re: [julia-users] poor performance of threads

[pevnak]

Thank you very much Tim.
I am using the profiler and your package ProfileView quite extensively and 
I know where is my Achille heel in the code, and it is cpu bound.  That's 
why I am so puzzled with threads.

I will try to use @code_warntype, never use it before. 

Best wishes,
Tomas

[julia-users] Re: can't get pyjulia to work

[Páll Haraldsson]

 

> I'm not looking to call python libs from julia.  I have a large investment 
> in python code.  The obvious (to me) path to try out some julia is to move 
> a 
> function or 2 into julia, and be able to call julia from my python main 
> program.  So I would need a working pyjulia to do this.


Yes, my PyCall.jl, suggestion, was probably not to helpful.. While you can 
use it to call TO Julia, using callbacks I guess, I assume you would be 
reimplementing pyjulia on your own from scratch..

Calling from Python, for sure worked at some point even recursively (might 
not be well tested, even less that only pyjulia) back and forth:

Julia Calling Python Calling Julia...
http://blog.leahhanson.us/post/julia/julia-calling-python.html

Now, I'm not sure what changed, at least Julia 0.4 came along and something 
broke pyjulia. You could help or wait for it to get fixed. OR if you are 
just trying out, you could use what did work. I think that would then be 
Julia 0.3.12:

A.

http://julialang.org/downloads/oldreleases.html

Is it preferred. No, but ok, to start out trying Julia.. 0.4 sure got 
better, e.g. faster/incremental GC, but Julia sure was fast in 0.3, and 
even 0.2.

These major version upgrades DID break some syntax (why they are major 
versions). Some are trivial changes, such as case of integer types. It 
would get annoying having to change but not you can use the newer syntax 
using Compat.jl. And you do not have to actually specify the type that 
often (never for speed).

B.
Another option is https://github.com/wavexx/Polyglot.jl

PyCall or pyjulia would almost always be preferred to that, except when 
it's not :) e.g. say running on different machines, or when, say either, 
e.g. the latter pyjulia doesn't work. This is still a neat project (at 
least for other languages), that I haven't tried. Hopefully 0.4 didn't 
break it..

-- 
Palli.

[julia-users] Re: can't get pyjulia to work

[Neal Becker]

Páll Haraldsson wrote:

> Since I'm more of a Julia user (or including calling Python) I'm not sure
> I can help you too much. I would however find it interesting to know
> if/when pyjulia works. If been following commits and see a big pull
> request here:
> 
> https://github.com/JuliaLang/pyjulia/pull/52
> 
> Maybe this is the one, that will help you when it is committed. I do not
> know git[hub] enough to know if you can checkout [pyjulia] project with
> these additions:
> 
> EQt wants to merge 17 commits into JuliaLang:master from
> EQt:clean-julia-0.4
> 
> [or others for that matter.. there are many other pull requests (and
> issues) open.]
> 
> At EQt's page I saw (but not his own repository with the code):
> 
> Repositories contributed to
> 
> https://github.com/benmoran/pyjulia
> 
> I guess someone must review these changes (could be you?), or possibly
> waiting for them to get committed (by those who have the privileges) is
> the correct option.
> 
> PyCall.jl as you may know allows for calling in the other direction.
> pyjulia reuses it, but I saw something about it needing it's own PyCall..
> (so they can develop in independent directions(?), seems like a bad idea);
> maybe I misunderstood..
> 
> 
> May I ask what you are doing with your own code? Or at least what Python
> Libraries/frameworks you use? In particular, if you use Django or similar?
> I would like to know if Django could be used with Julia, and then it seems
> pyjulia would be needed (not just PyCall).
> 
> I do not think you need to compile your own Julia. I could be wrong, just
> do not see the reason.
> 
> Maybe this, wasn't too helpful.. Is there another possibility, that you
> could just use PyCall?
> 
> On Tuesday, March 1, 2016 at 3:40:03 PM UTC, Neal Becker wrote:
>>
>> From time-to-time, I get interested in trying out moving some of my work
>> from python to julia.  Before I can even start, I need to be able to call
>> from python to julia.  But I've never gotten pyjulia to work on
>> linux/fedora
>> (currently 23).  I've tried the fedora version of julia (0.4.3), and I've
>> built my own julia today from master, and in both cases I get:
>>
>> j = julia.Julia (jl_init_path='/home/nbecker/julia')
>> ERROR: UndefVarError: dlpath not defined
>>  in eval(::Module, ::Any) at ./boot.jl:267
>>  [inlined code] from ./sysimg.jl:14
>>  in process_options(::Base.JLOptions) at ./client.jl:239
>>  in _start() at ./client.jl:318
>> Traceback (most recent call last):
>>   File "/home/nbecker/pyjulia/julia/core.py", line 238, in __init__
>> """])
>>   File "/usr/lib64/python3.4/subprocess.py", line 620, in check_output
>> raise CalledProcessError(retcode, process.args, output=output)
>> subprocess.CalledProcessError: Command
>> '['/home/nbecker/julia/usr/bin/julia', '-e', '\n
>> println(JULIA_HOME)\n
>> println(Sys.dlpath(dlopen("libjulia")))\n ']'
>> returned non-zero exit status 1
>>
>> During handling of the above exception, another exception occurred:
>>
>> Traceback (most recent call last):
>>   File "", line 1, in 
>>   File "/home/nbecker/pyjulia/julia/core.py", line 244, in __init__
>> raise JuliaError('error starting up the Julia process')
>> julia.core.JuliaError: error starting up the Julia process
>>
>>

I'm not looking to call python libs from julia.  I have a large investment 
in python code.  The obvious (to me) path to try out some julia is to move a 
function or 2 into julia, and be able to call julia from my python main 
program.  So I would need a working pyjulia to do this.

[julia-users] Memory increase in parallel processing when using Julia in Linux (Fedora 23)

[Eduardo Lenz]

Hi.

I have a question about parallel processing in Linux versus parallel 
processing in Windows using Julia. I have been using it without issues for 
some time in both operational systems without any problem. Yesterday, using 
fedora 23 and Julia 0.4.3-1 (fedora version) and 0.4.3-6 (nalimilan/julia 
Copr) I  observed a consistent increase in memory usage at every iteration 
of my program.  Reverting it to the serial version solves this issue. It 
does not happens in windows, since there is no increase in memory usage 
along the iterations.

Does anyone else observed this issue ? Unfortunenly, the code is very 
large, comprising finite element, homogenization and topology optimization, 
but it is clear that the problem is associated to parallel processing. I 
will try to write a small prototype in order to make further studies about 
this behavior.

Thank for your help.

Re: [julia-users] poor performance of threads

[Tim Holy]

The point still holds: if your single-threaded benchmark is not type-stable, 
you're absolutely, 100% wasting your time. Depending on your CPU, you might 
get perhaps a 4- to 8-fold improvement by using threads. But you might get a 
30-fold speed improvement by making it type-stable. Which would you rather 
have?

Now, maybe this isn't an issue with your bigger problem, but the fact that you 
posted a toy problem with this issue makes me suspect that it's likely to be 
worth your time to first ensure that your single-threaded performance is as 
good as it can be. So start by using the profiler and @code_warntype, and 
extinguish any performance problems. If you still need more performance, then 
we can talk about threads. I've not played with them myself, so it's likely 
others will have to comment.

Best,
--Tim

On Thursday, March 03, 2016 09:40:11 PM pev...@gmail.com wrote:
> Thanks a lot for the suggestions.
> As I have mentioned, it was really a toy problem, but I am not getting a
> significant speedup on a bigger problem, where threads are nicely separated
> either and the problem is very CPU bound either. I would be very interested
> to know about tool that would point out to problems with cache and memory
> access.
> 
> Tomas

[julia-users] Re: What wrong in (D'*D)/k.-(mean(D,1)'*mean(D,1)) ?

[Dan]

The matrix `D` is based on type `Int8` which overflows and wraps (without 
error) very quickly. The `cov` function automatically converts to a `Float64` 
but the manual calculation does not. To make the calculations match, try:
D = convert(Matrix{Int},D)

On Friday, March 4, 2016 at 10:42:02 AM UTC+2, paul analyst wrote:
>
> Why (D'*D)/k.-(mean(D,1)'*mean(D,1)) compute var = -0.018 if var must 
> be 0.124326 ?**
>
>
> julia> using HDF5, JLD
>
> julia> D=load("D_test.jld","D_test");
>
> julia> k,l=size(D)
> (100,10)
>
> julia> cov(D[:,1],D[:,1],corrected=false)
> 0.12432634086422582
>
> julia> cov(D,corrected=false)
> 10x10 Array{Float64,2}:
> 0.124326 0.0239205 -0.0119771 -0.0201163 0.013
> 0.0239205 0.154538 -0.0156671 -0.0263335 -0.023
> -0.0119771 -0.0156671 0.0853087 -0.0158088 0.002
> -
>
> julia> (D'*D)/k.-(mean(D,1)'*mean(D,1))
> 10x10 Array{Float64,2}:
> -0.018 -0.0244635 -0.0119771 -0.0247243 -0.01791
> -0.0244635 -0.0320863 -0.0156671 -0.0324775 -0.02332
> -0.0119771 -0.0156671 -0.00761925 -0.0158088 -0.01140
>
> julia> (D[:,1]'*D[:,1])/k.-(mean(D[:,1],1)'*mean(D[:,1],1))
> 1-element Array{Float64,1}:
> 0.124326
>
> julia> mean(D,1)'
> 10x1 Array{Float64,2}:
> 0.136944
> 0.179135
> 0.08746
> 0.180755
> 0.130206
> 0.015541
> 0.036076
> 0.101094
> 0.155723
> 0.045505
>
> julia> mean(D,1)
> 1x10 Array{Float64,2}:
> 0.136944 0.179135 0.08746 0.180755 0.130206 0.015541 0
>
> julia> ee=mean(D[:,1],1)'*mean(D[:,1],1)
> 1-element Array{Float64,1}:
> 0.0187537
>
> julia> dd=(D[:,1]'*D[:,1])/k
> 1-element Array{Float64,1}:
> 0.14308
>
> julia> dd-ee
> 1-element Array{Float64,1}:
> 0.124326
>
>
> file: 
>
> https://drive.google.com/open?id=0B9xW5VtANWhDOHlwVlZxZk1xanc
>
>
>

[julia-users] Re: can't get pyjulia to work

[Páll Haraldsson]

Since I'm more of a Julia user (or including calling Python) I'm not sure I 
can help you too much. I would however find it interesting to know if/when 
pyjulia works. If been following commits and see a big pull request here:

https://github.com/JuliaLang/pyjulia/pull/52

Maybe this is the one, that will help you when it is committed. I do not 
know git[hub] enough to know if you can checkout [pyjulia] project with 
these additions:

EQt wants to merge 17 commits into JuliaLang:master from EQt:clean-julia-0.4

[or others for that matter.. there are many other pull requests (and 
issues) open.]

At EQt's page I saw (but not his own repository with the code):

Repositories contributed to

https://github.com/benmoran/pyjulia

I guess someone must review these changes (could be you?), or possibly 
waiting for them to get committed (by those who have the privileges) is the 
correct option.

PyCall.jl as you may know allows for calling in the other direction. 
pyjulia reuses it, but I saw something about it needing it's own PyCall.. 
(so they can develop in independent directions(?), seems like a bad idea); 
maybe I misunderstood..


May I ask what you are doing with your own code? Or at least what Python 
Libraries/frameworks you use? In particular, if you use Django or similar? 
I would like to know if Django could be used with Julia, and then it seems 
pyjulia would be needed (not just PyCall).

I do not think you need to compile your own Julia. I could be wrong, just 
do not see the reason.

Maybe this, wasn't too helpful.. Is there another possibility, that you 
could just use PyCall?

On Tuesday, March 1, 2016 at 3:40:03 PM UTC, Neal Becker wrote:
>
> From time-to-time, I get interested in trying out moving some of my work 
> from python to julia.  Before I can even start, I need to be able to call 
> from python to julia.  But I've never gotten pyjulia to work on 
> linux/fedora 
> (currently 23).  I've tried the fedora version of julia (0.4.3), and I've 
> built my own julia today from master, and in both cases I get: 
>
> j = julia.Julia (jl_init_path='/home/nbecker/julia') 
> ERROR: UndefVarError: dlpath not defined 
>  in eval(::Module, ::Any) at ./boot.jl:267 
>  [inlined code] from ./sysimg.jl:14 
>  in process_options(::Base.JLOptions) at ./client.jl:239 
>  in _start() at ./client.jl:318 
> Traceback (most recent call last): 
>   File "/home/nbecker/pyjulia/julia/core.py", line 238, in __init__ 
> """]) 
>   File "/usr/lib64/python3.4/subprocess.py", line 620, in check_output 
> raise CalledProcessError(retcode, process.args, output=output) 
> subprocess.CalledProcessError: Command 
> '['/home/nbecker/julia/usr/bin/julia', '-e', '\n 
> println(JULIA_HOME)\n 
> println(Sys.dlpath(dlopen("libjulia")))\n ']' returned 
> non-zero exit status 1 
>
> During handling of the above exception, another exception occurred: 
>
> Traceback (most recent call last): 
>   File "", line 1, in  
>   File "/home/nbecker/pyjulia/julia/core.py", line 244, in __init__ 
> raise JuliaError('error starting up the Julia process') 
> julia.core.JuliaError: error starting up the Julia process 
>
>

[julia-users] Bug in daxpy! ???

[pevnak]

Hello all,
I was polishing my call and I have found the following definition of daxpy! 
I was not aware of


function axpy!{Ti<:Integer,Tj<:Integer}(α, x::AbstractArray, 
rx::AbstractArray{Ti}, y::AbstractArray, ry::AbstractArray{Tj})
if length(x) != length(y)
throw(DimensionMismatch("x has length $(length(x)), but y has 
length $(length(y))"))
elseif minimum(rx) < 1 || maximum(rx) > length(x)
throw(BoundsError(x, rx))
elseif minimum(ry) < 1 || maximum(ry) > length(y)
throw(BoundsError(y, ry))
elseif length(rx) != length(ry)
throw(ArgumentError("rx has length $(length(rx)), but ry has length 
$(length(ry))"))
end
for i = 1:length(rx)
@inbounds y[ry[i]] += x[rx[i]]*α
end
y
end

Is the first check
 length(x) != length(y)
really an intended behavior? 

The multiplication goes over indexes rx and ry, should not be the check 
 length(rx) != length(ry) ?

Thanks for the clarification.
Tomas

[julia-users] What wrong in (D'*D)/k.-(mean(D,1)'*mean(D,1)) ?

[paul analyst]

 

Why (D'*D)/k.-(mean(D,1)'*mean(D,1)) compute var = -0.018 if var must 
be 0.124326 ?**


julia> using HDF5, JLD

julia> D=load("D_test.jld","D_test");

julia> k,l=size(D)
(100,10)

julia> cov(D[:,1],D[:,1],corrected=false)
0.12432634086422582

julia> cov(D,corrected=false)
10x10 Array{Float64,2}:
0.124326 0.0239205 -0.0119771 -0.0201163 0.013
0.0239205 0.154538 -0.0156671 -0.0263335 -0.023
-0.0119771 -0.0156671 0.0853087 -0.0158088 0.002
-

julia> (D'*D)/k.-(mean(D,1)'*mean(D,1))
10x10 Array{Float64,2}:
-0.018 -0.0244635 -0.0119771 -0.0247243 -0.01791
-0.0244635 -0.0320863 -0.0156671 -0.0324775 -0.02332
-0.0119771 -0.0156671 -0.00761925 -0.0158088 -0.01140

julia> (D[:,1]'*D[:,1])/k.-(mean(D[:,1],1)'*mean(D[:,1],1))
1-element Array{Float64,1}:
0.124326

julia> mean(D,1)'
10x1 Array{Float64,2}:
0.136944
0.179135
0.08746
0.180755
0.130206
0.015541
0.036076
0.101094
0.155723
0.045505

julia> mean(D,1)
1x10 Array{Float64,2}:
0.136944 0.179135 0.08746 0.180755 0.130206 0.015541 0

julia> ee=mean(D[:,1],1)'*mean(D[:,1],1)
1-element Array{Float64,1}:
0.0187537

julia> dd=(D[:,1]'*D[:,1])/k
1-element Array{Float64,1}:
0.14308

julia> dd-ee
1-element Array{Float64,1}:
0.124326


file: 

https://drive.google.com/open?id=0B9xW5VtANWhDOHlwVlZxZk1xanc