Re: [julia-users] Type-stable global variables?

2016-10-15 Thread Andrei Zh
Thanks, this is exactly what I was looking for. On Saturday, October 15, 2016 at 4:08:48 PM UTC+3, Yichao Yu wrote: > > > > On Sat, Oct 15, 2016 at 8:59 AM, Andrei Zh <faithle...@gmail.com > > wrote: > >> What is the most straightforward way to make a variable in

[julia-users] Type-stable global variables?

2016-10-15 Thread Andrei Zh
What is the most straightforward way to make a variable in the global scope that can change it's value, but not its type? So far I use this: const GLOBAL_VAR = [MyType[]] # array with single element set_global_var(x::MyType) = GLOBAL_VAR[1] = x get_goval_var() = GLOBAL_VAR[1] This works fine

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-11 Thread Andrei Zh
> > > `slotnames` of LambdaInfo should have what you want. > Do note that the name is not unique. > Do you mean that if's, loops, etc. may contain variables of the same name but with different slot?

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-10 Thread Andrei Zh
under Julia 0.5, so I suppose there might be more stable way to do it. On Monday, August 8, 2016 at 10:39:34 PM UTC+3, Andrei Zh wrote: > > Thanks, this makes sense. Just for clarification: > > >> It strongly depend on what you want to do and whether you care about what >

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-08 Thread Andrei Zh
ust 8, 2016 at 1:55:51 AM UTC+3, Yichao Yu wrote: >>> >>> >>> >>> On Mon, Aug 8, 2016 at 3:57 AM, Andrei Zh <faithle...@gmail.com> wrote: >>> >>>> While parsing Julia expressions, I noticed that sometimes cal

Re: [julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-07 Thread Andrei Zh
> On Mon, Aug 8, 2016 at 3:57 AM, Andrei Zh <faithle...@gmail.com > > wrote: > >> While parsing Julia expressions, I noticed that sometimes calls to >> > > This shouldn't happen. > > >> global functions resolve to `GloablRef` and sometimes

[julia-users] GlobalRef(Module, func) vs. :(Module.func)

2016-08-07 Thread Andrei Zh
While parsing Julia expressions, I noticed that sometimes calls to global functions resolve to `GloablRef` and sometimes to getfield(Module, func). Could somebody please clarify: 1. Why do we need both? 2. Is it safe to replace one by the other (assuming only modules and functions are

[julia-users] Re: ReverseDiffSource on v0.5

2016-08-06 Thread Andrei Zh
Just tried it on my Ubuntu 16.04. The last thing I could see was julia process eating all 16Gb or RAM in ~1 second and hanging on my laptop so that I had to do hard reboot. So this package seems to be a good tool for testing stability of Julia 0.5 release candidates :) On Thursday, August 4,

[julia-users] Re: Symbolic differentiation similar to TensorFlow / Theano

2016-07-09 Thread Andrei Zh
ade. Thanks again! On Saturday, July 9, 2016 at 3:02:55 AM UTC+3, Andrei Zh wrote: > > In Python, libraries like TensorFlow or Theano provide possibility to > perform automatic differentiation over their computational graphs. E.g. in > TensorFlow (example from SO > <http://stacko

[julia-users] Symbolic differentiation similar to TensorFlow / Theano

2016-07-08 Thread Andrei Zh
In Python, libraries like TensorFlow or Theano provide possibility to perform automatic differentiation over their computational graphs. E.g. in TensorFlow (example from SO ): data =

Re: [julia-users] Performance of TCPServer

2016-06-16 Thread Andrei Zh
, June 15, 2016, Andrei Zh <faithle...@gmail.com > > wrote: > >> Thanks, Tony. Moving to Julia 0.4.5 improved speed to ~0.025 seconds per >> megabyte (~39Mb / sec). This is still far behind awesome 2.5Gb for Julia >> 0.5, but I think it doesn't make sense to spend ti

[julia-users] Re: Performance of TCPServer

2016-06-15 Thread Andrei Zh
://github.com/JuliaLang/julia/milestones On Thursday, June 16, 2016 at 1:20:46 AM UTC+3, Tony Kelman wrote: > > Use 0.4.4 or 0.4.5. https://github.com/JuliaLang/julia/issues/14467 > > On Wednesday, June 15, 2016 at 1:46:34 PM UTC-7, Andrei Zh wrote: >> >> Seems like resul

[julia-users] Re: Performance of TCPServer

2016-06-15 Thread Andrei Zh
allocations: 1.012 MB) 0.000400 seconds (114 allocations: 1.012 MB) Any suggestions on why the results are so different and how to fix TCP sockets for Julia 0.4? On Wednesday, June 15, 2016 at 5:10:12 PM UTC+3, Andrei Zh wrote: > > I've just tested performance of TCP server/client in Juli

[julia-users] Performance of TCPServer

2016-06-15 Thread Andrei Zh
I've just tested performance of TCP server/client in Julia REPL and find results pretty disappointing. The server is as simple as: server = listen(2000) while true sock = accept(server) begin while true @time readbytes(sock, 1024*1024) end end end It

[julia-users] Re: parallel computing in julia

2016-05-20 Thread Andrei Zh
In addition to what Fred said, `addprocs()` adds new workers, i.e. processes, which are not necessary bound to your CPU cores. On Friday, May 20, 2016 at 6:05:05 AM UTC+3, SHORE SHEN wrote: > > Hi > > Im doing a loop and need parallel computing. > > from the doc, it is said to "julia -p n"

Re: [julia-users] Re: Async file IO?

2016-05-19 Thread Andrei Zh
IO is built on libuv >>> >> >> Julia IO is mostly built on libuv, but file IO uses the internal ios >> library that is part of flisp. >> >> On Thu, May 19, 2016 at 5:43 PM, Yichao Yu <yyc...@gmail.com >> > wrote: >> >>> On Thu, May 19, 2016

[julia-users] Re: Async file IO?

2016-05-19 Thread Andrei Zh
Based on answers to my own question , I believe it's safe to assume that `read()` on file will switch to another task during IO operation. On Tuesday, May 17, 2016 at 2:03:21 AM UTC+3, g wrote: > > Hello All, > > I think

Re: [julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-18 Thread Andrei Zh
up. > > On Tue, May 17, 2016 at 4:36 PM, Andrei Zh <faithle...@gmail.com > > wrote: > >> Hmm, I now think that I could misinterpret exception (EOFError, if I >> remember correctly) - it could be a server who closed connection by >> timeout, not Julia's `read(

Re: [julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-17 Thread Andrei Zh
with timeout > > That sounds like a bug, although I'm not aware of any read code-path that > has a timeout. Can you post an example of when it fails? This should > absolutely be the correct way to handle IO. > > > > On Tue, May 17, 2016 at 3:09 AM Andrei Zh <faithle...@gma

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-17 Thread Andrei Zh
at would remove possible delay is welcome. On Tuesday, May 17, 2016 at 8:18:16 AM UTC+3, Jameson wrote: > > Launch each socket into it's own task (@async) and let Julia's > green-threading manage the poll/select for you internally. > > > On Friday, May 13, 2016 at 8:08:01 PM UTC-4, Andr

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-13 Thread Andrei Zh
o if I manage to create a callback for incoming data, I will be able to borrow some ideas from (1) and come up with a `select` function that satisfies my needs. Does anybody have an example of using TCPSocket's callbacks/conditions? On Thursday, May 12, 2016 at 12:45:44 AM UTC+3, Andrei Zh

[julia-users] Re: select/poll on TCPSocket in Julia?

2016-05-12 Thread Andrei Zh
to a file descriptor? On Thursday, May 12, 2016 at 12:45:44 AM UTC+3, Andrei Zh wrote: > > Given several TCPSocket-s, how can I sequentially poll them and read the > first that becomes readable? > > I believe *nix's poll/select functions provide something similar for files > (and

[julia-users] select/poll on TCPSocket in Julia?

2016-05-11 Thread Andrei Zh
Given several TCPSocket-s, how can I sequentially poll them and read the first that becomes readable? I believe *nix's poll/select functions provide something similar for files (and Julia has `poll_fd`), but what is the best way to approach this task for sockets?

[julia-users] Re: Write immutable with variable-length array field to a stream

2016-05-10 Thread Andrei Zh
2016 at 2:31:59 PM UTC-4, Andrei Zh wrote: >> >> I work on implementing a binary protocol for a service. This protocol is >> based on messages - structs that support integer numbers, variable-length >> arrays and other structs, i.e. something we could implement in Julia as:

Re: [julia-users] How fast is appending an element to an array?

2016-05-10 Thread Andrei Zh
Very detailed explanation, thank you! On Tuesday, May 10, 2016 at 11:35:43 PM UTC+3, Yichao Yu wrote: > > On Tue, May 10, 2016 at 4:24 PM, Andrei Zh <faithle...@gmail.com > > wrote: > > From performance perspective, how bad it is to use `push!(array, > element)` > &

[julia-users] How fast is appending an element to an array?

2016-05-10 Thread Andrei Zh
>From performance perspective, how bad it is to use `push!(array, element)` compared to pre-allocating array and setting elements in it? More generally, how extending arrays works in Julia? (code in array.c is quite readable, but

[julia-users] Write immutable with variable-length array field to a stream

2016-05-07 Thread Andrei Zh
I work on implementing a binary protocol for a service. This protocol is based on messages - structs that support integer numbers, variable-length arrays and other structs, i.e. something we could implement in Julia as: immutable Message1 version::Int16 length::Int32

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-17 Thread Andrei Zh
can manipulate original pointer and also have direct access to the `data` field. On Sunday, April 17, 2016 at 3:35:34 PM UTC+3, Yichao Yu wrote: > > On Sun, Apr 17, 2016 at 6:23 AM, Andrei Zh <faithle...@gmail.com > > wrote: > > A followup question. In the above example I

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-17 Thread Andrei Zh
at 3:49:51 AM UTC+3, Andrei Zh wrote: > > This is exactly the answer I hoped to see! Thanks a lot! > > On Sunday, April 17, 2016 at 2:46:15 AM UTC+3, Yichao Yu wrote: >> >> On Sat, Apr 16, 2016 at 6:55 PM, Andrei Zh <faithle...@gmail.com> wrote: >> >

Re: [julia-users] What is the proper way to allocate and free C struct?

2016-04-16 Thread Andrei Zh
This is exactly the answer I hoped to see! Thanks a lot! On Sunday, April 17, 2016 at 2:46:15 AM UTC+3, Yichao Yu wrote: > > On Sat, Apr 16, 2016 at 6:55 PM, Andrei Zh <faithle...@gmail.com > > wrote: > > I have a question regarding Struct Type correspondences section of t

[julia-users] What is the proper way to allocate and free C struct?

2016-04-16 Thread Andrei Zh
I have a question regarding Struct Type correspondences section of the documentation. Let's say, in a C library we have a struct like this: typedef struct { int rows; int cols;

[julia-users] [ANN] Spark.jl - Julia interface to Apache Spark

2016-04-14 Thread Andrei Zh
Spark.jl provides Julia bindings for Apache Spark - by far the most popular computational framework in Hadoop ecosystem. Find it at: https://github.com/dfdx/Spark.jl There's still *a lot* of work to do (Spark API is *huge*), but Spark.jl already supports: - map and reduce functions, as

Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-12-08 Thread Andrei Zh
UDA 7.5 >> >> If somebody else could confirm the problem on curand in cuda 7.0, we >> could add that in CURAND.jl documentation. >> >> >> >> Em quarta-feira, 25 de novembro de 2015 19:20:48 UTC-2, Andrei Zh >> escreveu: >>> >>> @Joa

Re: [julia-users] How to create array wrapper with constrains on type and number of dimensions

2015-11-30 Thread Andrei Zh
Hmm, seems like this gives an error when constructing a type: julia> ArrayWrapper(rand(3, 4)) ERROR: MethodError: `convert` has no method matching convert(::Type{ ArrayWrapper{T,N,AT<:AbstractArray{T,N}}}, ::Array{Float64,2}) This may have arisen from a call to the constructor

[julia-users] How to create array wrapper with constrains on type and number of dimensions

2015-11-30 Thread Andrei Zh
I'm trying to do something like this (which doesn't compile in its current form): type ArrayWrapper{T,N,AT <: AbstractArray{T,N}} <: AbstractArray{T,N} arr::AT{T,N} end That is: - wrapper around any type AT inherited from AbstractArray{T,N} - wrapper should be itself parametrized by

[julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-11-24 Thread Andrei Zh
While investigating this issue we found that CUDArt and CURAND may behave differently on different platforms. Could somebody from user group with enabled CUDA try the following code in Julia REPL and report: * operating system * whether this code succeeded or failed; if failed, then what is

Re: [julia-users] Re: CUDART and CURAND problem on running the same "do" loop twice

2015-11-10 Thread Andrei Zh
> But I can replicate the bug if I use CURAND. I'd suggest filing an issue > with > that package. Most likely it needs to (and, you need to) initialize > resources > according to these instructions: > https://github.com/JuliaGPU/CUDArt.jl#initializing-and-freeing-ptx-modules > > CURAND.jl

[julia-users] Re: How to escape array of symbols in a macro?

2015-10-19 Thread Andrei Zh
@Cedric: > $(map(esc, args)...) > > works everywhere I've tried it _except_ in function definition argument > lists. > Ha, I tried a couple of scenarios and in most cases it really behaves as I want, but not in this specific case :) Even though I like macro hygiene by default, I should

[julia-users] Re: How to escape array of symbols in a macro?

2015-10-19 Thread Andrei Zh
(generic function with 1 method) > > julia> foo(5, 3.0) > ERROR: MethodError: `foo` has no method matching foo(::Int64, ::Float64) > Closest candidates are: > foo(::Any, ::Int64) > > julia> foo(5, 3) > hello > > > julia> @deffun bar(a::AbstractString

[julia-users] Re: is there a way to access help from within emacs shell?

2015-10-12 Thread Andrei Zh
If you are looking for docstrings: On Julia 0.3: julia> help(qr) INFO: Loading help data... Base.qr(A, [pivot=false,][thin=true]) -> Q, R, [p] Compute the (pivoted) QR factorization of "A" such that either "A = Q*R" or "A[:,p] = Q*R". Also see "qrfact". The default is to compute a

Re: [julia-users] Inner constructor in a type with type parameters

2015-10-12 Thread Andrei Zh
Ah, that was obvious. Example of why late night coding isn't very productive :) Thanks! On Monday, October 12, 2015 at 2:15:02 AM UTC+3, Yichao Yu wrote: > > On Sun, Oct 11, 2015 at 6:54 PM, Andrei Zh <faithle...@gmail.com > > wrote: > > I didn't know about such capa

Re: [julia-users] Inner constructor in a type with type parameters

2015-10-11 Thread Andrei Zh
call{T}(::Type{T}, arg) at essentials.jl:56 call{T}(::Type{T}, args...) at essentials.jl:57 On Monday, October 12, 2015 at 1:06:14 AM UTC+3, Yichao Yu wrote: > > On Sun, Oct 11, 2015 at 5:57 PM, Andrei Zh <faithle...@gmail.com > > wrote: > > Let's consider 2 ty

[julia-users] Inner constructor in a type with type parameters

2015-10-11 Thread Andrei Zh
Let's consider 2 types with inner constructors: type Foo x::Array{Int,1} Foo() = Foo(zeros(Int, 10)) end type Bar{T} x::Array{T,1} Bar() = Bar(zeros(T, 10)) end The only difference between them is that `Bar` has type parameter while `Foo` doesn't. I'd expect their inner

Re: [julia-users] What does `Base.uncompressed_ast(f.code)` return?

2015-10-08 Thread Andrei Zh
nd` directly, and then > look at the form of that expanded expression (hint: it is a > LambdaStaticData). Also see how this is done in `base/serialize.jl` -- or, > ideally, just reuse that functionality. > > > On Wed, Oct 7, 2015 at 7:13 PM, Andrei Zh <faithle...@gmail.com > > w

[julia-users] What does `Base.uncompressed_ast(f.code)` return?

2015-10-07 Thread Andrei Zh
If `f` is anonymous function: f = x -> x + 1 what is the format of AST returned by: Base.uncompressed_ast(f.code) ? I expected it to be pure AST of a lambda function, but it's quite different from quoting lambda directly: julia> dump(Base.uncompressed_ast(f.code)) Expr head: Symbol

[julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
Julia supports multiprocessing pretty well, including map-reduce-like jobs. E.g. in the next example I add 3 processes to a "workgroup", distribute simulation between them and then reduce results via (+) operator: julia> addprocs(3) 3-element Array{Int64,1}: 2 3 4 julia> nheads =

Re: [julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
the major concern of the original MapReduce design. > > On Tue, Oct 6, 2015 at 10:24 AM, Andrei Zh <faithle...@gmail.com > > wrote: > >> Julia supports multiprocessing pretty well, including map-reduce-like >> jobs. E.g. in the next example I add 3 processes to a &qu

Re: [julia-users] Re: Implementing mapreduce parallel model (not general multi-threading) ? easy and enough ?

2015-10-06 Thread Andrei Zh
e only codes that really nail it are carefully handcrafted HPC codes. Could you please elaborate on this? I think I know Spark code quite well, but can't connect it to the notion of handcrafted HPC code. On Tue, Oct 6, 2015 at 12:57 PM, Andrei Zh <faithle...@gmail.com > > wrote: > >>

[julia-users] Re: What's the reason of the Success of Python?

2015-09-30 Thread Andrei Zh
Coming back to the original question about success of Python, I'd stress these points: * it's *simple*. for both - programmers and non-programmers * it's a *system language* for most Unix-like operations systems * it has very strong *scientific stack* (read as: has a lot of mature

[julia-users] Why Julia uses GitHub to maintain package index?

2015-09-30 Thread Andrei Zh
First of all, thanks to all developers of current packaging system in Julia - it has very convenient REPL-based interface, graceful treatment of binary dependencies and it never was that easy to make and submit changes to existing packages. So please don't consider this post as critique, but

Re: [julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-25 Thread Andrei Zh
ps://github.com/JuliaWeb/HttpServer.jl/pull/59. > > On Fri, Sep 25, 2015 at 5:47 AM Páll Haraldsson <pall.ha...@gmail.com > > wrote: > >> On Thursday, September 24, 2015 at 11:07:56 PM UTC, Andrei Zh wrote: >>> >>> >>> >>>> I fin

[julia-users] Re: What are the "strengths" of the Julia "ecosystem" (e.g. for bio and quants/finance)? This is not a very specific question (while sub-questions are), you've been warned..

2015-09-23 Thread Andrei Zh
If you are looking for a best in the class libraries, you probably won't find many. This is implied by a simple fact that most such libraries had already been created in other languages by the time Julia was born. However, if you want something comparable to such best libraries, then I would

[julia-users] Re: What's your favourite editor?

2015-09-22 Thread Andrei Zh
te t) (setq ess-tab-complete-in-script t) On Tuesday, September 22, 2015 at 4:16:33 PM UTC+3, Andrei Zh wrote: > > > >> I'd be grateful to hear from other emacs users regarding your workflows >> for Julia development, e.g. if you want to write a Julia package what emacs >&

[julia-users] Re: What's your favourite editor?

2015-09-22 Thread Andrei Zh
> > I'd be grateful to hear from other emacs users regarding your workflows > for Julia development, e.g. if you want to write a Julia package what emacs > packages, setup and workflow help you to be the most productive? > > I use ESS + autocomplete + few keybindings (I'll post exact .init.el

Re: [julia-users] Emacs, ESS and julia-mode

2015-09-17 Thread Andrei Zh
ESS mode's integration with Julia's REPL is quite simple and can't handle special sequences. It's a pity that `help()` function has been removed from 0.4, I didn't see it before your post. There was a discussion regarding better REPL (network REPL?) recently, but AFAIK nobody is working on

[julia-users] Re: [Concurrency] Julia with/vs. Python 3.5, Go, Erlang/Elixir

2015-09-15 Thread Andrei Zh
It would be more productive, if you could describe what exactly you expect from a good concurrent/parallel programming language. Julia already has built-in coroutines (Tasks) and multiprocessing, and will get shared-memory parallelism (multithreading) in observable future. The only thing in

[julia-users] Re: How to use Apache Spark from Julia

2015-09-05 Thread Andrei Zh
Spark's website provides pretty good documentation to get started. Basically, you just need to download and unpack Spark archive. To use it from Julia, you can try either Spock.jl, or Sparta.jl (Spark.jl isn't a wrapper, but instead an

Re: [julia-users] IDE for Julia

2015-09-01 Thread Andrei Zh
@Cedric Unless you are interested specifically in notebooks, I'd suggest trying ESS mode for Emacs which has support for Julia including REPL. It looks something like this (though this video also uncovers sometimes annoying

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-25 Thread Andrei Zh
:30 AM UTC+3, Andrei Zh wrote: @Jameson: setting UV_THREADPOOL_SIZE to 32 seems to reduce DNS resolution time twice (from 26 to 12 seconds on my latest tests), so thank you. However, DNS seems to be not the only root of the problem: I noticed that with large number of URLs first ones get

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
-performance applications so don't hesitate to file an issue if it gives you any performance problems. On Sunday, August 23, 2015 at 7:40:08 PM UTC-4, Andrei Zh wrote: Hi Steven, thanks for your answer! It turns out I misunderstood @async long time ago, assuming it also makes a remote call

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
-performance applications so don't hesitate to file an issue if it gives you any performance problems. On Sunday, August 23, 2015 at 7:40:08 PM UTC-4, Andrei Zh wrote: Hi Steven, thanks for your answer! It turns out I misunderstood @async long time ago, assuming it also makes a remote call to other

Re: [julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-24 Thread Andrei Zh
dns queries, you may want to try increasing the number that can be run simultaneously but setting the UV_THREADPOOL_SIZE environment variable before starting julia to something larger (default is 4, max is 128). On Mon, Aug 24, 2015 at 9:17 AM Andrei Zh faithle...@gmail.com javascript

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-23 Thread Andrei Zh
More generally, is there anything like futures/callbacks or async IO in Julia? I couldn't find anything, but maybe I just don't know the right keywords.

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-23 Thread Andrei Zh
Hi Steven, thanks for your answer! It turns out I misunderstood @async long time ago, assuming it also makes a remote call to other processes and thus introduces true multi-tasking. So now I need to rethink my approach before going further. Just to clarify: my goal is to perform as many

[julia-users] Re: What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-22 Thread Andrei Zh
, 2015 at 5:52:27 AM UTC+3, Andrei Zh wrote: I'm writing a kind of a web scanner that should retrieve and analyze about 100k URLs as fast as possible. Of course, it will take time anyway, but I'm looking for how to utilize my CPUs and network as much as possible. My initial approach

[julia-users] What is the fastest way to perform 100k blocking IO operations in parallel?

2015-08-22 Thread Andrei Zh
I'm writing a kind of a web scanner that should retrieve and analyze about 100k URLs as fast as possible. Of course, it will take time anyway, but I'm looking for how to utilize my CPUs and network as much as possible. My initial approach was to add all available processors, pack urls into

Re: [julia-users] Benchmarking Julia HttpServer

2015-08-19 Thread Andrei Zh
I've got similar results some time ago [1], but stopped investigating because of lack of time. If you could find out the reason why different tools result in different outcomes, it would really push Julia web stack to production level. [1]: https://github.com/JuliaWeb/HttpServer.jl/issues/48

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-14 Thread Andrei Zh
:55 PM UTC+3, Jake Bolewski wrote: Andrei Zh I'm confused. Have you actually tried? julia io = IOBuffer() IOBuffer(data=UInt8[...], readable=true, writable=true, seekable=true, append=false, size=0, maxsize=Inf, ptr=1, mark=-1) julia foo(x) = x + 1 foo (generic function with 1 method

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-14 Thread Andrei Zh
define the function with @everywhere, it will be defined on all existing workers. Likewise, `using MyPackage` loads the package on all workers. --Tim On Thursday, August 13, 2015 03:10:54 PM Andrei Zh wrote: Ok, after going through serialization code, it's clear that default

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-13 Thread Andrei Zh
PM UTC+3, Andrei Zh wrote: Yes, I incorrectly assumed `serialize` / `deserialize` use JLD format. But anyway, even when I saved the function into example.jls or even plain byte array (using IOBuffer and `takebuf_array`), nothing changed. Am I missing something obvious? On Monday, August

Re: [julia-users] Computer Vision Package

2015-08-10 Thread Andrei Zh
but I might still need a layer between Images.jl and my package because most of tracking algorithm uses only grayscale images, but still not sure about that. Based on my experience (re)implementing active appearance models, I'd say many tracking algorithms work even better with colored

[julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-10 Thread Andrei Zh
I'm afraid it's not quite true, and I found simple way to show it. In the next code snippet I define function `f` and serialize it to a file: julia f(x) = x + 1 f (generic function with 1 method) julia f(5) 6 julia open(example.jld, w) do io serialize(io, f) end Then I close Julia REPL and

Re: [julia-users] Re: Can Julia function be serialized and sent by network?

2015-08-10 Thread Andrei Zh
/JLD.jl#saving-and-loading-variables-in-julia-data-format-jld --Tim On Monday, August 10, 2015 at 12:45:35 PM UTC-7, Stefan Karpinski wrote: JLD doesn't support serializing functions but Julia itself does. On Mon, Aug 10, 2015 at 3:43 PM, Andrei Zh faithle...@gmail.com

[julia-users] Can Julia function be serialized and sent by network?

2015-08-09 Thread Andrei Zh
I'm working on a wrapper for Apache Spark in Julia. Essentially, workflow looks like this: 1. Julia driver creates instance of `JuliaRDD` class in JVM and passes serialized Julia function to it. 2. Spark core copies JuliaRDD to each machine in a cluster and runs its `.compute()` method.

[julia-users] Re: ZMQ Publish, Subscribe

2015-07-15 Thread Andrei Zh
To start with zmq_socket.connect(zmq_tcp1) is not Julia syntax, correct syntax is connect(zmq_socket, zmq_tcp1) In other words, you don't invoke method of an object with 1 argument, but instead invoke a function with 2 arguments. On Wednesday, July 15, 2015 at 1:54:48 AM UTC+3,

Re: [julia-users] Julia Summer of Code

2015-05-25 Thread Andrei Zh
@Jey, I'm willing to help, but I need some initial guidance. Please, check out issues tab in Spock.jl. On Monday, May 25, 2015 at 7:05:01 AM UTC+3, Jey Kottalam wrote: Viral -- Given the clear interest in it, I'll work toward fleshing it out more fully. However, if anyone is interested in

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-22 Thread Andrei Zh
BTW, I've measured throughput of Redis on my machine, and it resulted in only 30K/sec, which is 10 times slower than ZMQ. Possibly, some hybrid solution of several alternative backends will work. On Saturday, May 23, 2015 at 2:04:02 AM UTC+3, Andrei Zh wrote: It still helps a lot, because

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-22 Thread Andrei Zh
It still helps a lot, because you can have many reporting programs, each talking to different processes on the server, and those processes are able to get the transactions done very quickly, with multiple write daemons and journaling daemons doing the actual I/O from the shared buffer

Re: [julia-users] Re: High-performance metric collector in Julia

2015-05-20 Thread Andrei Zh
I was able to get 2M/sec without transactions, and 909K/sec with transactions (so it's durable), on my laptop, using Caché... (from InterSystems... I used to consult for them) They do have a free single user database engine, Globals, that you might be able to use... I don't recall

[julia-users] High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
I'm working on a simple load testing library. Basically, it provides macro `@runtimes` that takes number of parallel users, number of iterations and expression to execute, and repeats that expression specified number of times in different threads / tasks. In code, it looks something like this:

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
that route, is to use Redis as a store. There are a couple of julia Redis drivers around. Regards - Avik On Tuesday, 19 May 2015 17:23:23 UTC+1, Scott Jones wrote: On Tuesday, May 19, 2015 at 11:39:24 AM UTC-4, Andrei Zh wrote: Not quite, I'm more interested in systems for collecting

[julia-users] Re: Using Julia program as computation backend

2015-05-19 Thread Andrei Zh
I would say the easiest way would be to create Julia wrapper around one of existing clients, e.g. use PyCall.jl and Python's pika library [1]. [1]: https://pika.readthedocs.org/en/0.9.14/ On Monday, May 18, 2015 at 5:34:12 PM UTC+3, Roman Kravchik wrote: I am building a web app for DSP

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
@Scott, thanks for your comment, but I think we are talking about different scenarios and different speed levels :) ??? Not on any real RDBMS like MySQL, PostgreSQL... If you want reported message to be durable, you need either to replicate it to another server, or write it to disk.

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
As I mentioned, StatsD is cool, but comes with a lot of dependencies. Also, as far as I know, StatsD doesn't provide storage, but instead pushes data further (e.g. to graphite). Thus, to use it for simple metric collection one will also need to create StatsD backend (in addition to front-end),

[julia-users] Re: High-performance metric collector in Julia

2015-05-19 Thread Andrei Zh
Not quite, I'm more interested in systems for collecting metrics (think of reporting server or application health, for example) rather than storage with random reads and writes. It is possible to use databases for this, but it will be pretty inefficient. For example: 1. If we tale traditional

[julia-users] Why `using` new module shadows old methods?

2015-04-27 Thread Andrei Zh
Let's say, we have 2 modules: module A export foo foo(n::Int) = println(A) end module B export foo foo(s::String) = println(B) end and then in REPL: julia using A julia using B !julia methods(foo) # 1 method for generic function foo:

Re: [julia-users] Why `using` new module shadows old methods?

2015-04-27 Thread Andrei Zh
, Andrei Zh faithle...@gmail.com javascript: wrote: Let's say, we have 2 modules: module A export foo foo(n::Int) = println(A) end module B export foo foo(s::String) = println(B) end and then in REPL: julia using A julia using B !julia methods(foo) # 1 method for generic

[julia-users] Re: Julia and Spark

2015-04-16 Thread Andrei Zh
Julia bindings for Spark would provide much more than just RDD, they will give us access to multiple big data components for streaming, machine learning, SQL capabilities and much more. On Friday, April 17, 2015 at 12:54:32 AM UTC+3, wil...@gmail.com wrote: However, I wonder, how hard it

[julia-users] Re: poly2mask in Julia?

2015-01-31 Thread Andrei Zh
Andrew, thanks for the link! I spent quite some time to rewrite and optimize code for Julia, but it was worth it: for 100x100 matrix intersection-based algorithms gives almost 10 times speedup, and the larger the matrix, the higher is the difference. Here's my new code: function

[julia-users] Is `fetch(RemoteRef)` blocking a thread?

2015-01-29 Thread Andrei Zh
It's clear, that when we call `fetch` on an instance of `RemoteRef`, it blocks the task. But does it mean that the whole thread is blocked (and thus we experience thread context switching) or control is simply passed to another task? For context: I'm trying to understand different ways of

Re: [julia-users] poly2mask in Julia?

2015-01-28 Thread Andrei Zh
}) = inpolygon(P, 1:size(img,1), 1:size(img,2)) inpolygon(P,M::AbstractArray,N::AbstractArray) = Float32[inpolygon(P, m,n) for m in M, n in N] Am 28.01.2015 um 03:50 schrieb Steven G. Johnson steve...@gmail.com javascript:: On Sunday, January 25, 2015 at 11:03:30 AM UTC-5, Andrei Zh

[julia-users] Re: poly2mask in Julia?

2015-01-26 Thread Andrei Zh
@Andreas: do you refer to this thread: https://groups.google.com/forum/#!topic/julia-users/Kwy2Zh-i4ks Yes, it's an option. I was looking for something specifically designed for this purpose, but seems like Scikit Image and some others keep exactly the approach you've described [1], so I

[julia-users] poly2mask in Julia?

2015-01-25 Thread Andrei Zh
Is there something like Matlab's `poly2mask` function in any Julia package? In Matlab, `poly2mask` transforms polygon (given by arrays of `x` and `y` coordinates) into a binary mask, where all values inside polygon get value 1 and all others get value 0. I found Octave implementation [1]

Re: [julia-users] Simple drawing over image?

2015-01-11 Thread Andrei Zh
Ah, my fault. I should have checked it before googling more sophisticated things. Thanks! On Monday, January 12, 2015 at 2:57:50 AM UTC+3, Tim Holy wrote: ImageView has an annotation framework; it's documented in the README. --Tim On Monday, January 12, 2015 02:32:12 AM Andrei wrote:

[julia-users] Re: Test-scoped requirements

2015-01-05 Thread Andrei Zh
Thanks, works like a charm. On Sunday, January 4, 2015 10:39:27 PM UTC+3, Michael Hatherly wrote: You can put a REQUIRE file in the test directory that should do what you want. — Mike ​ On Sunday, 4 January 2015 21:33:37 UTC+2, Andrei Zh wrote: I have a package A that has file-based

[julia-users] Test-scoped requirements

2015-01-04 Thread Andrei Zh
I have a package A that has file-based integration with package B, i.e. A writes files that B can read, but neither A depends on B, nor vise versa. I want to write a test for this integration, but don't want to include B in REQUIRE, since A may be used completely without it. I'm wondering if

Re: [julia-users] DataFrames: convert categorical variables to dummy ones?

2014-10-01 Thread Andrei Zh
Thanks for your suggestions! So far using GLM and pool!() with a list of categorical variables works fine for me. On Wednesday, October 1, 2014 5:27:33 PM UTC+3, John Myles White wrote: Currently, the way to do this is via the GLM package (or at least its strategy for generating design

[julia-users] Resize image to specific size

2014-07-29 Thread Andrei Zh
There's pretty cool package Images by Tim Holy (kudos, Tim!), but it seems to leak one standard and pretty helpful function - resizing image to specific size. There's restrict method (probably imported from Grid.jl), but it only reduces image size by 2 at each dimension. Are there any other

[julia-users] Re: help with improving performance of a matrix multiplication

2014-07-29 Thread Andrei Zh
I had similar problem recently, and in my case solution was to: 1. Make sure that you use OpenBLAS and not other implementation: julia Base.blas_vendor() :openblas 2. Use mutable versions of BLAS matrix multiplication (e.g. A_mul_B! or more general gemm!) and thus not produce

Re: [julia-users] Resize image to specific size

2014-07-29 Thread Andrei Zh
...and Ron, Kevin, Lucas, etc ... Surely, I'm just referring to someone I see on the mailing list very often :) Yeah, it's a pretty obvious omission, but to be truthful it's never come up for me so I've never implemented it. I will get to it eventually, but in the meantime if you

  1   2   >