Thanks for the suggestions everyone. Unfortunately, neither the code from
Ivar nor João worked for me.
liboauth = dlopen("/usr/local/lib/liboauth.dylib")
Out[8]:
Ptr{Void} @0x00007fdc665c5ca0
In [9]:
function oauth_sign_hmac_sha1(m::String,k::String)
res =
ccall((:oauth_sign_hmac_sha1,liboauth),Ptr{Uint8},(Ptr{Uint8},Ptr{Uint8}),m,k)
if res == C_NULL
error("oauth_sign_hmac_sha1 failed")
end
return string(res)
end
testurl =
"GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal"
testkey = "kd94hf93k423kf44&pfkkdhi9sl3r4s00"
oauth_sign_hmac_sha1(testurl, testkey)
type: oauth_sign_hmac_sha1: in ccall: first argument not a pointer or valid
constant expression, expected DataType, got Type{(Any...,)}
while loading In[9], in expression starting on line 11
in oauth_sign_hmac_sha1 at In[9]:2
testurl =
"GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal"
testkey = "kd94hf93k423kf44&pfkkdhi9sl3r4s00"
b64d = ccall((:oauth_sign_hmac_sha1, liboauth), Ptr{Uint8}, (Ptr{Uint8},
Ptr{Uint8}), testurl, testkey)
bytestring(b64d)
type: anonymous: in ccall: first argument not a pointer or valid constant
expression, expected DataType, got Type{(Any...,)}
while loading In[10], in expression starting on line 3
in anonymous at no file
Does this mean I compiled the library wrong? When I did the 'make' step,
after it was complete, I did 'make installcheck' and the 3 tests reported
that they completed successfully.
At a higher level, I like Isaiah's suggestion of doing this process by hand
to really get to understanding what is going on. Is there an eaier external
library that one of you could suggest that I could use to walk through the
entire process? I've done the examples in the manual using :clock and
:getenv and understand what is going on, so now I want to work with a
user-installed library to make sure I really get the process. I started
with OAuth to see if I could make something usable to work with my Twitter
package, but maybe it's not the best starting place.
On Monday, September 1, 2014 10:47:22 AM UTC-4, João Felipe Santos wrote:
>
> You need to do ccalls using Ptr{Uint8}. What you can do actually is wrap
> the Clang.jl-generated functions with your own functions with an
> alternative signature, and then convert from ASCIIString to Ptr{Uint8}.
> Note that you probably will need to do it anyways to wrap the return types,
> since you probably do not want to work with pointers from Julia :)
>
> In that specific example, if m and k are ASCIIStrings, conversions will be
> done automatically:
>
> testurl = "GET&http%3A%2F%2Fphotos.example.net
> %2Fphotos&file%3Dvacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26oauth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk%26oauth_version%3D1.0%26size%3Doriginal"
> testkey = "kd94hf93k423kf44&pfkkdhi9sl3r4s00"
> b64d = ccall((:oauth_sign_hmac_sha1, liboauth), Ptr{Uint8},
> (Ptr{Uint8}, Ptr{Uint8}), testurl, testkey)
>
> Note that b64d here will be a Ptr{Uint8}, which you can convert to a Julia
> string by using bytestring(b64d).
>
> --
> João Felipe Santos
>
>
> On Mon, Sep 1, 2014 at 9:55 AM, Randy Zwitch <[email protected]
> <javascript:>> wrote:
>
>> To make it a bit more concrete, the C library has this test function:
>>
>> testurl = "GET&http%3A%2F%2Fphotos.example.net%2Fphotos&file%3D"
>> "vacation.jpg%26oauth_consumer_key%3Ddpf43f3p2l4k3l03%26oauth_nonce"
>> "%3Dkllo9940pd9333jh%26oauth_signature_method%3DHMAC-SHA1%26o"
>> "auth_timestamp%3D1191242096%26oauth_token%3Dnnch734d00sl2jdk"
>> "%26oauth_version%3D1.0%26size%3Doriginal";
>> testkey = "kd94hf93k423kf44&pfkkdhi9sl3r4s00";
>> b64d = oauth_sign_hmac_sha1(testurl , testkey);
>>
>> In Julia, I have this generated from Clang.jl:
>>
>> liboauth = dlopen("/usr/local/lib/liboauth.dylib")
>>
>> function oauth_sign_hmac_sha1(m::Ptr{Uint8},k::Ptr{Uint8})
>>
>> ccall((:oauth_sign_hmac_sha1,liboauth),Ptr{Uint8},(Ptr{Uint8},Ptr{Uint8}),m,k)
>> end
>>
>> What modifications do I need to make to the Clang Julia code to run this
>> test?
>>
>>
>>
>> On Monday, September 1, 2014 8:54:59 AM UTC-4, Randy Zwitch wrote:
>>>
>>> Thanks for the detailed response João! As it turns out, I didn't have
>>> any .dylib files as part of the downloaded code, but I was able to figure
>>> out how to compile the code and I believe I've got the library installed
>>> locally now.
>>>
>>> When you reference that most of my Ptr{Uint8} values are really just
>>> looking for string inputs, can I modify the Julia function signature to
>>> take ::ASCIIString instead? Or is this something where I have the leave the
>>> type signatures as they are, but understand what the function is actually
>>> looking for from the documentation?
>>>
>>> On Sunday, August 31, 2014 9:19:21 PM UTC-4, João Felipe Santos wrote:
>>>>
>>>> Hello Randy,
>>>>
>>>> the following comes from my experience with ccall and Julia
>>>> documentation. Please anyone correct me if I explained any of the
>>>> internals
>>>> wrong!
>>>>
>>>> The API seems to be really simple so pretty much everything can be done
>>>> with standard Julia types. Clang.jl seems to have generated correct code
>>>> for it.
>>>>
>>>> Regarding 2, liboauth.dylib (or .dll, or .so on Linux) has to be loaded
>>>> and passed to ccall. You can load it by hand using dlopen, like this:
>>>>
>>>> liboauth = dlopen("/path/of/liboauth.dylib")
>>>>
>>>> This will raise an error if the path is not valid. If the library is in
>>>> your environment's standard path, you can pass its name to ccall as a
>>>> string instead of creating a pointer with dlopen.
>>>>
>>>> Most of the Ptr{Uint8} you got in the code generated by Clang.jl are
>>>> actually mapping char *, which are C strings (ASCII, not Unicode). For
>>>> example, let's say you wanted to call puts from libc, which is a function
>>>> that gets a string as its argument, prints it to the screen followed by a
>>>> newline, and returns the number of printed characters. You could do it
>>>> like
>>>> this:
>>>>
>>>> ccall((:puts, "libc"), Cint, (Ptr{Uint8},), "Hello, Randy")
>>>>
>>>> So, you basically can pass your string as the function argument and it
>>>> should work. You don't need to pass a pointer, but it's good to know that
>>>> your Julia ASCIIString was converted to Ptr{Uint8} internally.
>>>>
>>>> With this you should be able to call functions, but you still need to
>>>> be able to recover their output. The second argument to ccall is the C
>>>> function return type, but note that often C functions do not return their
>>>> output in the return type, but in a variable which was passed by reference
>>>> as one of its arguments. I don't know enough of liboauth to know what's
>>>> the
>>>> case, so you will need to check the documentation. See the sections
>>>> http://docs.julialang.org/en/release-0.3/manual/calling-c-
>>>> and-fortran-code/#accessing-data-through-a-pointer and
>>>> http://docs.julialang.org/en/release-0.3/manual/calling-c-
>>>> and-fortran-code/#passing-pointers-for-modifying-inputs for more
>>>> details on that.
>>>>
>>>> --
>>>> João Felipe Santos
>>>>
>>>>
>>>> On Sun, Aug 31, 2014 at 8:17 PM, Randy Zwitch <[email protected]
>>>> > wrote:
>>>>
>>>>> Hi all -
>>>>>
>>>>> I've been trying to learn more about C and how Julia interacts and
>>>>> decided to play around with Clang.jl. I decided I was going to wrap
>>>>> liboauth from here:
>>>>>
>>>>> http://liboauth.sourceforge.net/oauth_8h_source.html
>>>>>
>>>>> I downloaded the C source, which resided in my OSX Downloads
>>>>> directory. Using the following Julia code generated a bunch of output:
>>>>>
>>>>> [1]:
>>>>>
>>>>> using Clang.wrap_c
>>>>>
>>>>> In [2]:
>>>>>
>>>>> context = wrap_c.init(; output_file="liboauth.jl",
>>>>> header_library=x->"liboauth", common_file="liboauth.jl",
>>>>> clang_diagnostics=true)
>>>>>
>>>>> context.options.wrap_structs = true
>>>>>
>>>>> wrap_c.wrap_c_headers(context,
>>>>> ["/Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h"])
>>>>>
>>>>> WARNING: wrap_c_headers: deprecated
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:112:46: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:138:54: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:138:86: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:315:67: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:315:81: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:320:66: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:320:80: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:519:28: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:532:30: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:670:61: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:688:57: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:715:43: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:717:70: warning:
>>>>> type specifier missing, defaults to 'int' [-Wimplicit-int]
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:717:77: warning:
>>>>> type specifier missing, defaults to 'int' [-Wimplicit-int]
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:717:77: error:
>>>>> redefinition of parameter 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:717:70: note:
>>>>> previous declaration is here
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:741:24: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:771:43: error:
>>>>> unknown type name 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:773:70: warning:
>>>>> type specifier missing, defaults to 'int' [-Wimplicit-int]
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:773:77: warning:
>>>>> type specifier missing, defaults to 'int' [-Wimplicit-int]
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:773:77: error:
>>>>> redefinition of parameter 'size_t'
>>>>> /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h:773:70: note:
>>>>> previous declaration is here
>>>>>
>>>>> WRAPPING HEADER: /Users/randyzwitch/Downloads/liboauth-1.0.3/src/oauth.h
>>>>>
>>>>> WARNING: Not wrapping MacroInstantiation OA_GCC_VERSION_AT_LEAST
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>> WARNING: Not wrapping MacroInstantiation attribute_deprecated
>>>>>
>>>>> writing liboauth.jl
>>>>>
>>>>> Out[2]:
>>>>>
>>>>> 1-element Array{Any,1}:
>>>>> nothing
>>>>>
>>>>>
>>>>>
>>>>> Output:
>>>>>
>>>>>
>>>>> const LIBOAUTH_VERSION = "1.0.3"
>>>>> const LIBOAUTH_VERSION_MAJOR = 1
>>>>> const LIBOAUTH_VERSION_MINOR = 0
>>>>> const LIBOAUTH_VERSION_MICRO = 3
>>>>> const LIBOAUTH_CUR = 8
>>>>> const LIBOAUTH_REV = 7
>>>>> const LIBOAUTH_AGE = 8
>>>>>
>>>>> # Skipping MacroDefinition: OA_GCC_VERSION_AT_LEAST ( x , y ) (
>>>>> __GNUC__ > x || __GNUC__ == x && __GNUC_MINOR__ >= y )
>>>>> # Skipping MacroDefinition: attribute_deprecated __attribute__ ( (
>>>>> deprecated ) )
>>>>>
>>>>> # begin enum ANONYMOUS_1
>>>>> typealias ANONYMOUS_1 Uint32
>>>>> const OA_HMAC = (uint32)(0)
>>>>> const OA_RSA = (uint32)(1)
>>>>> const OA_PLAINTEXT = (uint32)(2)
>>>>> # end enum ANONYMOUS_1
>>>>>
>>>>> # begin enum OAuthMethod
>>>>> typealias OAuthMethod Uint32
>>>>> const OA_HMAC = (uint32)(0)
>>>>> const OA_RSA = (uint32)(1)
>>>>> const OA_PLAINTEXT = (uint32)(2)
>>>>> # end enum OAuthMethod
>>>>> function oauth_sign_hmac_sha1(m::Ptr{Uint8},k::Ptr{Uint8})
>>>>> ccall((:oauth_sign_hmac_sha1,liboauth),Ptr{Uint8},(Ptr{
>>>>> Uint8},Ptr{Uint8}),m,k)
>>>>> end
>>>>>
>>>>> function oauth_sign_hmac_sha1_raw(m::Ptr{Uint8},ml::Cint,k::Ptr{
>>>>> Uint8},kl::Cint)
>>>>> ccall((:oauth_sign_hmac_sha1_raw,liboauth),Ptr{Uint8},(Ptr{
>>>>> Uint8},Cint,Ptr{Uint8},Cint),m,ml,k,kl)
>>>>> end
>>>>>
>>>>> function oauth_sign_plaintext(m::Ptr{Uint8},k::Ptr{Uint8})
>>>>> ccall((:oauth_sign_plaintext,liboauth),Ptr{Uint8},(Ptr{
>>>>> Uint8},Ptr{Uint8}),m,k)
>>>>> end
>>>>>
>>>>> function oauth_sign_rsa_sha1(m::Ptr{Uint8},k::Ptr{Uint8})
>>>>> ccall((:oauth_sign_rsa_sha1,liboauth),Ptr{Uint8},(Ptr{
>>>>> Uint8},Ptr{Uint8}),m,k)
>>>>> end
>>>>>
>>>>> function oauth_verify_rsa_sha1(m::Ptr{Uint8},c::Ptr{Uint8},s::Ptr{
>>>>> Uint8})
>>>>> ccall((:oauth_verify_rsa_sha1,liboauth),Cint,(Ptr{Uint8},
>>>>> Ptr{Uint8},Ptr{Uint8}),m,c,s)
>>>>> end
>>>>>
>>>>> <snip>
>>>>>
>>>>> My questions are:
>>>>>
>>>>> 1) Did I do this right? I got back a Julia file full of Julia code
>>>>> with ccalls, so it seems like I got back what was expected? There were
>>>>> lots
>>>>> of deprecation warnings in Julia however.
>>>>> 2) How do I actually start using this code for something? Other than
>>>>> unzipping the source code for liboauth, I haven't done anything with the
>>>>> C
>>>>> code other than generating the Julia ccalls. How will this Julia code
>>>>> know
>>>>> how to reference the original source liboauth.h file?
>>>>> 3) The Julia functions are using pointers as input types. Do I
>>>>> literally need to pass a pointer, or can I just provide the data type of
>>>>> the pointer (say, Uint8)?
>>>>>
>>>>> So, am I on the right track? What should I be thinking about if I want
>>>>> to take the next step in actually trying to use these functions?
>>>>>
>>>>
>>>>
>
