On Sun, Apr 17, 2016 at 6:23 AM, Andrei Zh <faithlessfri...@gmail.com> wrote:
> A followup question. In the above example I'd like to add method `getindex`
> for `XMatrix`. Previously, I could do it like this:
>
> type XMatrix
> rows::Cint
> cols::Cint
> data::Ptr{Float64}
> end
>
> getindex(mat::XMatrix, i::Integer) = unsafe_load(mat.data, i)
>
> Now after Yichao's suggestion my type looks like:
>
> type XMatrix
> ptr::Ptr{Void}
> end
>
> So I don't have direct access to the `data` field. What is the proper way
> to get it?
>
> So far my best idea is to use something like this:
>
> type XMatrixStruct
> rows::Cint
> cols::Cint
> data::Ptr{Float64}
> end
^^ I would make this an `immutable`
>
> type XMatrix
> ptr::Ptr{XMatrixStruct}
> end
>
> function getindex(mat::XMatrix, i::Integer)
> struct = unsafe_load(mat.ptr) # load copy of XMatrixStruct
> type
And then llvm should optimize out the load of the entire struct.
> value = unsafe_load(struct.data, i) # load value from its data field
> end
There are of course still cases that is inconvenient (e.g. if you want
to store to a field) something like this[1] may help with that. In the
mean time, you can load and store the field directly with an offset
too.
[1] https://github.com/JuliaLang/julia/issues/11902
>
> But it would be nice to avoid intermediate `XMatrixStruct` type.
>
>
>
> On Sunday, April 17, 2016 at 3:49:51 AM UTC+3, Andrei Zh wrote:
>>
>> This is exactly the answer I hoped to see! Thanks a lot!
>>
>> On Sunday, April 17, 2016 at 2:46:15 AM UTC+3, Yichao Yu wrote:
>>>
>>> On Sat, Apr 16, 2016 at 6:55 PM, Andrei Zh <faithle...@gmail.com> wrote:
>>> > I have a question regarding Struct Type correspondences section of the
>>> > documentation. Let's say, in a C library we have a struct like this:
>>> >
>>> > typedef struct {
>>> > int rows;
>>> > int cols;
>>> > double* data;
>>> > } xmatrix;
>>> >
>>> > In C code it can be created using function:
>>> >
>>> > int xmatrix_copy(void* src, xmatrix** mat, int rows, int cols);
>>> >
>>> > Note pointer-to-pointer in second argument. This C function allocates
>>> > new
>>> > region of memory for `xmatrix` and fills in the values (rows, cols and
>>> > pointer to data). I call this function from Julia like this:
>>> >
>>> > type XMatrix
>>> > rows::Cint
>>> > cols::Cint
>>> > data::Ptr{Float64}
>>> > end
>>> >
>>> >
>>> > data = Float64[1, 2, 3, 4, 5, 6]
>>> > matptr = Array(Ptr{XMatrix}, 1)
>>> >
>>> >
>>> > ccall((:xmatrix_copy), Cint,
>>> > (Ptr{Void}, Ptr{Ptr{XMatrix}}, Cint, Cint),
>>> > pointer(data), matptr, Cint(3), Cint(2))
>>> >
>>> > mat = unsafe_load(matptr[1])
>>>
>>> This is wrong, it leaks memory. See below.
>>>
>>> >
>>> > Details to notice in `ccall`:
>>> >
>>> > 1. In type tuple I set type of xmatrix argument to be a
>>> > pointer-to-pointer
>>> > to XMatrix.
>>> > 2. As an argument, I pass 1-element array of pointers to XMatrix and
>>> > let C
>>> > code to fill it in.
>>> >
>>> > This approach works and I get initialized `XMatrix`. Yet, I don't
>>> > really
>>> > understand all details of memory management in this case, In
>>> > particular:
>>> >
>>> > 1. Who owns memory of an instance of `XMatrix`? Is `unsafe_load`
>>> > copying
>>> > fields or just assigns Julia type tag to C-allocated region of memory?
>>>
>>> The owner of the memory is determined by the c library. In this case
>>> it seems that the C library owns the memory (which is generally the
>>> case if the library provide new and free functions)
>>> Julia never (and can never) assign tag to C memory. Unsafe load does
>>> exactly what the name suggests and is equivalent to `*ptr` in C. It
>>> copies the content of the struct and the return value
>>> has no aliasing with the original pointer.
>>>
>>> I assume the C library want the original pointer so you need to keep
>>> the pointer in the wrapper type and define conversion functions to
>>> pass it to C. See this[1] for an example for the proper way to do it
>>> (mostly the ptr field, the cconvert and unsafe_convert definitions and
>>> the finalizer(explained below)).
>>>
>>> > 2. What would be the proper way to free allocated memory given that C
>>> > library provides function `xmatrix_free(xmatrix* mat)`?
>>>
>>> ccall the free function. You usually also want a finalizer on the
>>> wrapper type mentioned above so that the GC can free the memory for
>>> you when the julia object is collected. Note that you should use
>>> finalizer to finalize object only for types that you don't care too
>>> much about their lifetime since the time when a finalizer is called is
>>> undefined.
>>>
>>> > 3. Aforementioned section (and the next one) suggest to use
>>> > `Ref{XMatrix}`
>>> > instead. Can somebody provide an example of corresponding code using
>>> > refs
>>> > instead of wrapping into an array?
>>>
>>> Ref on a bits type is basically a 0-dim array
>>>
>>> matout = Ref{Ptr{XMatrix}}()
>>>
>>> ccall((:xmatrix_copy), Cint,
>>> (Ptr{Void}, Ptr{Ptr{XMatrix}}, Cint, Cint),
>>> data, matout, Cint(3), Cint(2)) # DO NOT USE
>>> `pointer(data)`!!!! This is the single most common mistake when using
>>> ccall. It is fine in the global scope but if this is in a function,
>>> julia is free to collect the data array when you are in/before
>>> entering the ccall. See the manual section about cconvert and
>>> unsafe_convert
>>>
>>> matptr = matout[] # This is the C managed/owned pointer that you
>>> should embed in your julia type.
>>>
>>> >
>>> > Thanks,
>>> > Andrei
>>>
>>>
>>> [1]
>>> https://github.com/yuyichao/LibArchive.jl/blob/4112772e8d1d7124c896bf98c5baee13149d3f8d/src/writer.jl#L28-L58
