Re: [Pharo-users] uFFI ExternalAddress challenges

[eftomi]

Hi, 

as a wrap-up for this forum thread, I found an elegant way to take a result
as C wchar_t* from an FFI call on Windows, based on the idea of
ExternalData>>#fromCString. If anybody needs it:
https://github.com/eftomi/pharo-uFFI-readWin32WideString.

Best wishes,
Tomaz



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] uFFI ExternalAddress challenges

[eftomi]

Hi, after some more investigation the easiest way to get Win32WideString from
external module is to pass it as a parameter, like in:

^self ffiCall: #( void myFunction ( Win32WideString aWin32WideString ) ) 

and the external module can access it directly to do r/w. 

There are other possibilities as well - like adapting the method
Win32WideString>>#fromHandle: to translate from ExternalData to
Win32WideString, creating new ExternalType, or even creating a new
FFIWin32WideString. I wonder what would be the best approach :-) 

Best wishes,
Tomaz



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

Hi,

> I don’t know if there is support for that (I don’t have the time to 
check it in detail now).

> But, as a workaround, you may try
>
 > - return a void*
 >
 >  myFunction
 >^self ffiCall: #(void* myFunction ( void ) )
>
>   - and then do the transformation yourself using 
Win32WideString>>#fromHandle:


I checked, Win32WideString class>>#asExternalTypeOn: is defined, and it 
creates FFIExternalObjectType(Win32WideString) nicely with


^self ffiCall: #(Win32WideString myFunction ( void ) )

I also tried the other approach:

^self ffiCall: #(void* myFunction ( void ) )

and the transformation Win32WideString>>#fromHandle

Interestingly, in both cases I get


however I cannot pinpoint the reason for this.

> What you are discovering here would be very useful as a 
section/chapter in

> the uFFI booklet.

Thanks for the invitation, yes, it would be nice to participate, I'll 
try to prepare something when I get to the bottom of the challenge :-)


Thanks and best wishes,
Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Stephan Eggermont]

Tomaž Turk  wrote:
> OK, great, I'll try it out.


Hi Tomaž,

What you are discovering here would be very useful as a section/chapter in
the uFFI booklet. Just posting it as text here will do if you don’t feel up
to translating to Pillar. 

Cheers
  Stephan

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

OK, great, I'll try it out.

Thanks,
Tomaz

-- Original Message --
From: "Guillermo Polito" 
To: "Tomaž Turk" 
Sent: 17.9.2019 11:56:21
Subject: Re: [Pharo-users] uFFI ExternalAddress challenges

I don’t know if there is support for that (I don’t have the time to 
check it in detail now).

But, as a workaround, you may try

 - return a void*

  myFunction
^self ffiCall: #(void* myFunction ( void ) )

  - and then do the transformation yourself using 
Win32WideString>>#fromHandle:

Re: [Pharo-users] uFFI ExternalAddress challenges

[Guillermo Polito]

Hi Tomaz,

Have you checked Win32WideString in Pharo7/8 and its users?
You can do

aWindowsWideString := ‘a wide string’ asWin32WideString.

And then check the senders of Win32WideString, you will find you can declare 
ffi signatures like this

removeEnvironmentVariable: nameString

 ^ self ffiCall: #( int SetEnvironmentVariableW ( Win32WideString 
nameString, 0 ) )

Isn’t this working for you?
If not, I’d like to have more details on why it does not work, so we can figure 
out a solution :)

Cheers,
Guille


> El 13 sept 2019, a las 15:22, Tomaž Turk  escribió:
> 
> OK, I'm proceeding with the library that I mentioned 
> (http://disphelper.sourceforge.net/ <http://disphelper.sourceforge.net/>) and 
> I got nice results. Pablo's package goes directly into the core of OLE/COM 
> automation and it would be too hard to for me to study it and continue with 
> it at this stage. 
> 
> An example below loads Word for Windows, makes it visible, creates a new 
> document and writes some text:
> 
> | w |
> COMEngineAlpha initializeCOM .
> w := COMEngineAlpha createObjectByName: 'Word.Application'.
> COMEngineAlpha callObject: w setProperty: '.Visible = %b' withInteger: 1.
> COMEngineAlpha callObject: w method: '.Documents.Add'.
> COMEngineAlpha callObject: w method: '.Selection.TypeText(%S)' withString: 
> 'This is a message to appear in Word document.'.
> COMEngineAlpha safeReleaseObject: w.
> COMEngineAlpha uninitializeCOM.
> 
> However, I'd like to marshall a WideString through uFFI so that I won't loose 
> UTF-8 support, of course the plain (String aString) signature doesn't work 
> ("Cannot coerce arguments"). Is there anything I can do myself (i.e. without 
> poking into Pharo virtual machine complexity) to solve this challenge?
> 
> Best wishes,
> Tomaz
> 
> -- Original Message --
> From: "Torsten Bergmann" mailto:asta...@gmx.de>>
> To: tomaz.t...@ef.uni-lj.si <mailto:tomaz.t...@ef.uni-lj.si>; 
> pharo-users@lists.pharo.org <mailto:pharo-users@lists.pharo.org>
> Cc: "Any question about pharo is welcome"  <mailto:pharo-users@lists.pharo.org>>
> Sent: 12. 09. 2019 12:07:34
> Subject: Aw: Re: [Pharo-users] uFFI ExternalAddress challenges
> 
>> From your snippets it looks like you want to do COM Automation on Windows.
>> 
>> Maybe you should have a look at: https://github.com/tesonep/pharo-com 
>> <https://github.com/tesonep/pharo-com>
>> 
>> Bye
>> T.
>>  
>> Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
>> Von: "Tomaž Turk" mailto:tomaz.t...@ef.uni-lj.si>>
>> An: "Any question about pharo is welcome" > <mailto:pharo-users@lists.pharo.org>>
>> Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges
>> > We are interested in it 
>> > If I recall well Pablo got one version somewhere.
>>  
>> Me too   Unfortunately, I only have time for 'weekend' 
>> projects, and any help is appreciated 
>>  
>> Regarding the uFFI calls, is it possible to pass Win32WideString in a 
>> similar fashion as a String - or where to look to implement that?
>>  
>> Best wishes,
>> Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[eftomi]

It seems that there's nothing wrong with the primitive, but with Win32Variant
creation in ExternalStructure class>>fromHandle:, specifically with basicNew
and Win32Variant class as receiver.

In what circumstances the debugger/inspector shows 'error printing' message
(on the result of basicNew)? 

Sorry for newbie questions :-)

Best wishes,
Tomaz



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] uFFI ExternalAddress challenges

[eftomi]

A rudimentary disphelper library "approach" is working OK, I succeeded to
create a connection to SQL Server through ADODB, too, which is rather cool,
since I've been looking for this functionality for too long. 

I re-checked Pablo's pharo-com package and it is rather comprehensive - it
successfully creates COM servers and more. However, when trying to invoke
writing to COM server properties, as in:

wordObj propertyNamed: 'Visible' put: true. 

the call fails with: 

Primitive failed: primitive #integerAt:size:signed: in ExternalAddress

Problematic method uses a primitive



which probably doesn't exist. Does anybody know about the purpose/intent and
the history of this primitive in 'SqueakFFIPrims'? 

I think it would be a pitty that we leave Pablo's work unfinished.



--
Sent from: http://forum.world.st/Pharo-Smalltalk-Users-f1310670.html

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

OK, I'm proceeding with the library that I mentioned 
(http://disphelper.sourceforge.net/) and I got nice results. Pablo's 
package goes directly into the core of OLE/COM automation and it would 
be too hard to for me to study it and continue with it at this stage.


An example below loads Word for Windows, makes it visible, creates a new 
document and writes some text:


| w |
COMEngineAlpha initializeCOM .
w := COMEngineAlpha createObjectByName: 'Word.Application'.
COMEngineAlpha callObject: w setProperty: '.Visible = %b' withInteger: 
1.

COMEngineAlpha callObject: w method: '.Documents.Add'.
COMEngineAlpha callObject: w method: '.Selection.TypeText(%S)' 
withString: 'This is a message to appear in Word document.'.

COMEngineAlpha safeReleaseObject: w.
COMEngineAlpha uninitializeCOM.

However, I'd like to marshall a WideString through uFFI so that I won't 
loose UTF-8 support, of course the plain (String aString) signature 
doesn't work ("Cannot coerce arguments"). Is there anything I can do 
myself (i.e. without poking into Pharo virtual machine complexity) to 
solve this challenge?


Best wishes,
Tomaz

-- Original Message --
From: "Torsten Bergmann" 
To: tomaz.t...@ef.uni-lj.si; pharo-users@lists.pharo.org
Cc: "Any question about pharo is welcome" 
Sent: 12. 09. 2019 12:07:34
Subject: Aw: Re: [Pharo-users] uFFI ExternalAddress challenges

From your snippets it looks like you want to do COM Automation on 
Windows.


Maybe you should have a look at: https://github.com/tesonep/pharo-com

Bye
T.

Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
Von: "Tomaž Turk" 
An: "Any question about pharo is welcome" 
Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges
> We are interested in it
:)
> If I recall well Pablo got one version somewhere.

Me too
:-)
 Unfortunately, I only have time for 'weekend' projects, and any help 
is appreciated

:-)

Regarding the uFFI calls, is it possible to pass Win32WideString in a 
similar fashion as a String - or where to look to implement that?


Best wishes,
Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

> Maybe you should have a look at: https://github.com/tesonep/pharo-com

Thanks for the link, I'll check it out - that's probably the work Steph 
has mentioned.


Best wishes,
Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Ben Coman]

On Thu, 12 Sep 2019 at 18:08, Torsten Bergmann  wrote:

> From your snippets it looks like you want to do COM Automation on Windows.
>
> Maybe you should have a look at: https://github.com/tesonep/pharo-com
>

Does that by chance include an implementation of "MS-CFB" Microsoft
Compound File Binary
https://docs.microsoft.com/en-us/openspecs/windows_protocols/ms-cfb/50708a61-81d9-49c8-ab9c-43c98a795242


I was wanting to write a utility to parse headers of MSG files saved from
Outlook,
to process a for a project email archive by bulk renaming each MSG file
with its transmit-date & subject.

cheers -ben

Re: [Pharo-users] uFFI ExternalAddress challenges

[Torsten Bergmann]

From your snippets it looks like you want to do COM Automation on Windows.

Maybe you should have a look at: https://github.com/tesonep/pharo-com


Bye

T.

 

Gesendet: Donnerstag, 12. September 2019 um 11:08 Uhr
Von: "Tomaž Turk" 
An: "Any question about pharo is welcome" 
Betreff: Re: [Pharo-users] uFFI ExternalAddress challenges


> We are interested in it :)

> If I recall well Pablo got one version somewhere.

 

Me too :-)  Unfortunately, I only have time for 'weekend' projects, and any help is appreciated :-)

 

Regarding the uFFI calls, is it possible to pass Win32WideString in a similar fashion as a String - or where to look to implement that?

 

Best wishes,

Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

> We are interested in it :)
> If I recall well Pablo got one version somewhere.

Me too :-)  Unfortunately, I only have time for 'weekend' projects, and 
any help is appreciated :-)


Regarding the uFFI calls, is it possible to pass Win32WideString in a 
similar fashion as a String - or where to look to implement that?


Best wishes,
Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

Excellent, thanks!

Re: [Pharo-users] uFFI ExternalAddress challenges

[Guillermo Polito]

> El 11 sept 2019, a las 11:43, Tomaž Turk  escribió:
> 
> >  Well, the pointer survives, the area of memory pointed by that pointer 
> > will be recycled with subsequent calls in general.
> 
> I'll pay special attention to that, but it's probably a matter of properly 
> creating and releasing COM objects.
> 
> 
> I have another question - in FFI call, how would one send a wide string, as 
> in 
> 
> void CallMethod(void * COMObj, wchar_t * MethodName)
> 
> I might have additional questions about UTF-8 and other code page nuances …

:D Windows is particularly different than other platforms.
In *nixes most APIs require a char* with the encoded bytes, which we can encode 
in Pharo doing something like

‘my string’ utf8Encoded => a byte array with the encoded string

But for windows, I think the best is to use their encoding APIs.
You’ll find the image already has some support for that, that we use to access 
environment variables and the working directory string.

Check the class Win32WideString and its class comment ;)

> 
> Best wishes,
> Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

>  Well, the pointer survives, the area of memory pointed by that 
pointer will be recycled with subsequent calls in general.


I'll pay special attention to that, but it's probably a matter of 
properly creating and releasing COM objects.



I have another question - in FFI call, how would one send a wide string, 
as in


void CallMethod(void * COMObj, wchar_t * MethodName)

I might have additional questions about UTF-8 and other code page 
nuances ...


Best wishes,
Tomaz

Re: [Pharo-users] uFFI ExternalAddress challenges

[Guillermo Polito]

> El 11 sept 2019, a las 11:07, Tomaž Turk  escribió:
> 
> Hi, thanks for your quick reply.

no problem ;)

> I found the culprint while writing this email :-)

Cool!

> 
> Generally, what I'm trying to do is to make a very basic package to create 
> and communicate with COM components on Windows, mostly as an exercise :-) I 
> found this library: http://disphelper.sourceforge.net/ 
>  as a great start, so you don't have to 
> fiddle with OLE/COM headaches.
> 
> The problematic function in C is defined as:
> 
> __declspec(dllexport) void CallMethod(void* myObj) {
> ...
> }
> 
> It takes a pointer to a COM object, which was previously returned by 
> 
> __declspec(dllexport) void* CreateObject(char* szProgId) {
> ...
>IDispatch * myObj = NULL
> ...
>return 
> }
> 
> So, CreateObject() creates COM object and returns a pointer to it, 
> CallMethod() takes this pointer and tries to communicate with it. 
> 
> My error was in returning a pointer to a pointer ... since IDispatch * myObj 
> = NULL was actually hidden in a macro and I missed its true definition, huh. 

Yes, looking at the code above it seems strange returning the , being 
myObj a local variable.
Because that means that you would be returning a pointer to the stack, from a 
frame that already returned...

> 
> I was not sure whether the pointer survives from one FFI call to another

Well, the pointer survives, the area of memory pointed by that pointer will be 
recycled with subsequent calls in general.

> , or how Pharo loads/unloads DLLs - but it works as expected.

In general Pharo will load the library on usage, so you don’t have to care 
about it.
I’m sure the library is not unloaded, unless it is done explicitly (you can 
check VirtualMachine >> unloadModule:)

> 
> Thanks again and best wishes,
> Tomaz

Cheers!

Re: [Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

Hi, thanks for your quick reply. I found the culprint while writing this 
email :-)


Generally, what I'm trying to do is to make a very basic package to 
create and communicate with COM components on Windows, mostly as an 
exercise :-) I found this library: http://disphelper.sourceforge.net/ as 
a great start, so you don't have to fiddle with OLE/COM headaches.


The problematic function in C is defined as:

__declspec(dllexport) void CallMethod(void* myObj) {
...
}

It takes a pointer to a COM object, which was previously returned by

__declspec(dllexport) void* CreateObject(char* szProgId) {
...
   IDispatch * myObj = NULL
...
   return 
}

So, CreateObject() creates COM object and returns a pointer to it, 
CallMethod() takes this pointer and tries to communicate with it.


My error was in returning a pointer to a pointer ... since IDispatch * 
myObj = NULL was actually hidden in a macro and I missed its true 
definition, huh.


I'm running Pharo from PharoLauncher directly on Win10, without 
cygwin/mingw. For DLL I use Visual Studio 2019. I was not sure whether 
the pointer survives from one FFI call to another, or how Pharo 
loads/unloads DLLs - but it works as expected.


Thanks again and best wishes,
Tomaz

-- Original Message --
From: "Guillermo Polito" 
To: "Tomaž Turk" ; "Any question about pharo is 
welcome" 

Sent: 11. 09. 2019 10:29:34
Subject: Re: [Pharo-users] uFFI ExternalAddress challenges


Hi Thomas,

Can you share more about the definition of your CallMethod function in 
C?

From the uFFI point of view the binding looks ok…

How are you opening Pharo? From the command line or the pharo launcher?
Are you on cygwin/mingw? One other possibility (since you’re playing 
with C libraries) is to launch Pharo inside a gdb/lldb so you can check 
the reason behind the error when it crashes.


Keep us posted,
Guille

Re: [Pharo-users] uFFI ExternalAddress challenges

[Guillermo Polito]

Hi Thomas, 

Can you share more about the definition of your CallMethod function in C?
From the uFFI point of view the binding looks ok…

How are you opening Pharo? From the command line or the pharo launcher?
Are you on cygwin/mingw? One other possibility (since you’re playing with C 
libraries) is to launch Pharo inside a gdb/lldb so you can check the reason 
behind the error when it crashes.

Keep us posted,
Guille

> El 10 sept 2019, a las 19:53, Tomaž Turk  escribió:
> 
> Dear all,
> 
> I'm struggling with passing an ExternalAddress to a c function call with 
> uFFI. Firstly, I get the ExternalAddress from this method:
> 
> myFFI>>create: aString
> ^ self 
> ffiCall: #(void * CreateObject (String aString) ) 
> 
> [in c:
> void* CreateObject(char* szProgId)]
> 
> The method that uses the resulting ExternalAddress is defined as:
> 
> myFFI>>ask: anExternalAddress
> ^ self 
> ffiCall: #(void CallMethod (void * anExternalAddress) )
> 
> [in c:
> void CallMethod(void* myObj)]
> 
> In playground I have
> 
> | w |
> w := myFFI create: 'Word.Application'.
> myFFI ask: w .
> 
> w gets a "nice", properly looking external address, however the last line 
> crushes Pharo, its window gets closed.
> 
> I went through the uFFI book, however I cannot find the answer.
> 
> I'm on Windows 10 x64, Pharo 7.0.4 32-bit.
> 
> Best wishes,
> Tomaz

[Pharo-users] uFFI ExternalAddress challenges

[Tomaž Turk]

Dear all,

I'm struggling with passing an ExternalAddress to a c function call with 
uFFI. Firstly, I get the ExternalAddress from this method:


myFFI>>create: aString
^ self
ffiCall: #(void * CreateObject (String aString) )

[in c:
void* CreateObject(char* szProgId)]

The method that uses the resulting ExternalAddress is defined as:

myFFI>>ask: anExternalAddress
^ self
ffiCall: #(void CallMethod (void * anExternalAddress) )

[in c:
void CallMethod(void* myObj)]

In playground I have

| w |
w := myFFI create: 'Word.Application'.
myFFI ask: w .

w gets a "nice", properly looking external address, however the last 
line crushes Pharo, its window gets closed.


I went through the uFFI book, however I cannot find the answer.

I'm on Windows 10 x64, Pharo 7.0.4 32-bit.

Best wishes,
Tomaz