Re: [webkit-dev] WebKit render tree

[Myles C. Maxfield]

From the description, it sounds like you have some native code which needs to 
be able to modify the DOM and you need events from the DOM to be able to call 
code from your library. You can do this today without modifying WebKit.

The way for native code to modify the DOM is for the native code to generate a 
JavaScript string and submit the string to be run inside the JavaScript context 
of the page. You can do this in WK1 by JSEvaluateScript(), the WebScriptObject, 
or by stringByEvaluatingJavaScriptFromString. In WebKit2, WKWebView has a 
method evaluateJavaScript (Or from C you can run 
WKPageRunJavaScriptInMainFrame, but this isn’t considered API)

You can also use Objective-C DOM bindings to manipulate the DOM directly from 
native code, this approach is discouraged.

One way for DOM events to call code from your library is to create a JavaScript 
object or method which is backed by native code. Then, the JavaScript event 
handler can call the functions on this object, which causes your native code to 
run. This can be done with the functions listed inside 
Source/JavaScriptCore/API/JSObjectRef.h.

In WebKit2, this native-backed-object can still be made, but it must be done 
inside the Web process (not the UI process), and therefore must be done with 
the InjectedBundle. Inside the InjectedBundle, you can run native code in the 
web process, and there are facilities to send messages between the Web process 
& UI process. You can find out more if you search for “InjectedBundle” inside 
Source/WebKit2/UIProcess/API. Note, however, that use of the InjectedBundle is 
not considered API.

We rely on the above mechanisms heavily in our testing infrastructure. Our 
Tools/DumpRenderTree and Tools/WebKitTestRunner projects use the techniques 
described above.

The Windows port currently doesn’t support WebKit2, so you’ll have to use the 
WebKit1 approach on Windows.


> On Jun 1, 2016, at 11:49 AM, Andy Somogyi  wrote:
> 
> 
>> On Jun 1, 2016, at 1:37 PM, Myles C. Maxfield > > wrote:
>> 
>> Replies inline.
> 
>> Generating a DOM is usually done in JavaScript. I'm curious about the 
>> problem you are trying to solve where JavaScript is not sufficient.
>> 
>> Drawing into an existing window is usually done by inserting a WebView into 
>> the view 
>> reallyhierarchy of the window. Yet again, I'm interested in why this is not 
>> sufficient.
> 
> I have the source code of the new language, and a parser/compiler for it 
> written in C++. I’d hate to have to introduce a new set of languages into the 
> mix. 
> 
> The new programming language is designed to be visually edited, hence the 
> need for hit detection. So, a component may be dragged from one region to 
> another. This would trigger an event which would cause one branch of the AST 
> to be pruned, modified and re-attached to another branch. This in turn would 
> trigger a re-gen of the render tree, and eventually a repaint. I've done 
> something very similar in the past where I created a visual computer algebra 
> system. The entire runtime, including AST are written in C++, and it would be 
> nice to connect it to an existing rendering/layout engine, and be able to 
> respond to user events directly in my C++ code. 
> 
> I see this as a very dynamic application, and all of the internal logic which 
> is a mix of C and JIT compiled code from the new language needs to interact 
> with both the visual representation, and the physics engine (the Lang is 
> designed for real-time physics simulation). Ideally, I'd like to be able to 
> render a visual representation of a language element to an OpenGL texture, 
> and have this exist in the physics engine. 
> 
> However, being as the DOM is publicly available from WebKit in C++ (I think 
> they’re all exposed publicly as pure virtual interfaces), it would be 
> relatively straightforward for me to to compile the new language to target 
> the C++ v-table ABI. 
> 
> 
>>> Do you think this would be possible using WebCore as a library and have 
>>> these custom render tree / dom tree derived objects live in my own library 
>>> (this is really, really the way I’d prefer to do things), or do you think 
>>> it would be better to add these objects directly into the WebCore library. 
>>> 
>> 
>> Are you planning on committing any changes to trunk?
>> 
>> Of course it is ::possible:: to do something like this - it's all software. 
>> Are you asking if it's easy to do it? Or the best way to do it
> 
> On OSX, I don’t think I’ll have to do any changes at all to WebKit. I’ve got 
> proof of concept working where I create a new RenderElement derived object, 
> and give it a new GraphicsContext. 
> 
> However, on Windows, the classes I use from WebKit in my own code would need 
> to be exported via the WEBCORE_EXPORT macro, this would need to be added to 
> each class definition in WebKit. I suppose what I could do is write a quick 
> script which 

Re: [webkit-dev] WebKit render tree

[Andy Somogyi]

> On Jun 1, 2016, at 1:37 PM, Myles C. Maxfield  wrote:
> 
> Replies inline.

> Generating a DOM is usually done in JavaScript. I'm curious about the problem 
> you are trying to solve where JavaScript is not sufficient.
> 
> Drawing into an existing window is usually done by inserting a WebView into 
> the view hierarchy of the window. Yet again, I'm interested in why this is 
> not sufficient.

I have the source code of the new language, and a parser/compiler for it 
written in C++. I’d really hate to have to introduce a new set of languages 
into the mix. 

The new programming language is designed to be visually edited, hence the need 
for hit detection. So, a component may be dragged from one region to another. 
This would trigger an event which would cause one branch of the AST to be 
pruned, modified and re-attached to another branch. This in turn would trigger 
a re-gen of the render tree, and eventually a repaint. I've done something very 
similar in the past where I created a visual computer algebra system. The 
entire runtime, including AST are written in C++, and it would be nice to 
connect it to an existing rendering/layout engine, and be able to respond to 
user events directly in my C++ code. 

I see this as a very dynamic application, and all of the internal logic which 
is a mix of C and JIT compiled code from the new language needs to interact 
with both the visual representation, and the physics engine (the Lang is 
designed for real-time physics simulation). Ideally, I'd like to be able to 
render a visual representation of a language element to an OpenGL texture, and 
have this exist in the physics engine. 

However, being as the DOM is publicly available from WebKit in C++ (I think 
they’re all exposed publicly as pure virtual interfaces), it would be 
relatively straightforward for me to to compile the new language to target the 
C++ v-table ABI. 


>> Do you think this would be possible using WebCore as a library and have 
>> these custom render tree / dom tree derived objects live in my own library 
>> (this is really, really the way I’d prefer to do things), or do you think it 
>> would be better to add these objects directly into the WebCore library. 
>> 
> 
> Are you planning on committing any changes to trunk?
> 
> Of course it is ::possible:: to do something like this - it's all software. 
> Are you asking if it's easy to do it? Or the best way to do it

On OSX, I don’t think I’ll have to do any changes at all to WebKit. I’ve got 
proof of concept working where I create a new RenderElement derived object, and 
give it a new GraphicsContext. 

However, on Windows, the classes I use from WebKit in my own code would need to 
be exported via the WEBCORE_EXPORT macro, this would need to be added to each 
class definition in WebKit. I suppose what I could do is write a quick script 
which adds this macro to each class that I use as part of my build process. 
That way, I would not have to touch the upstream webkit source at all. 

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit render tree

[Myles C. Maxfield]

Replies inline.

> On May 31, 2016, at 11:19 PM, Andy Somogyi  wrote:
> 
> Hi,
> 
> I’m a programming language researcher, and we are working on a new visual 
> programming language. 
> 
> I’m investigating using WebCore as the rendering component of our language 
> editing/visualization system. 
> 
> Essentially what I’d like to be able to do is to programmatically (C++) 
> generate 1: a DOM tree of a new family of Element derived object, 2: 
> rendering tree, 3: layout the tree, 4: attach a GraphicsContext to an 
> existing window or bitmapped resource, and 5: use WebCore’s GraphicsContext 
> to render the custom rendering tree. I’d also like to create a family of 
> custom Element derived objects that would be attached to the rendering tree 
> and use the rendering tree. I’d also like to respond to the standard events, 
> i.e. mouse, keyboard, etc…

Generating a DOM is usually done in JavaScript. I'm curious about the problem 
you are trying to solve where JavaScript is not sufficient.

Drawing into an existing window is usually done by inserting a WebView into the 
view hierarchy of the window. Yet again, I'm interested in why this is not 
sufficient.

> 
> Do you think this would be possible using WebCore as a library and have these 
> custom render tree / dom tree derived objects live in my own library (this is 
> really, really the way I’d prefer to do things), or do you think it would be 
> better to add these objects directly into the WebCore library. 
> 

Are you planning on committing any changes to trunk?

Of course it is ::possible:: to do something like this - it's all software. Are 
you asking if it's easy to do it? Or the best way to do it?

> I don’t think accessing any of the render tree objects in webcore would cause 
> any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros 
> on each class def. If I added these, what would be the WebKit policy of 
> accepting these changes? 
> 
> What is the, for lack of a better word, the “viability” of WebKit on Windows? 
> Our project fundamentally has to be cross-platform, and currently, I’m not 
> aware of any webkit based browsers on Windows. I’m currently looking into 
> using webkit, firefox or blink. I’ve essentially eliminated blink because 
> their code is very hard to follow, much harder than webkit or firefox, and I 
> think the webkit code is the easiest to understand and use. Webkit is 
> currently my first choice, but my only hesitation is will webkit continued to 
> be supported on Windows. 

Windows is an official port, just like iOS / Mac / EFL / GTK.

> 
> thanks
> 
> -- Andy Somogyi PhD
> School of Informatics and Computing
> Indiana University
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] WebKit render tree

[Konstantin Tokarev]

01.06.2016, 09:19, "Andy Somogyi" :
> Hi,
>
> I’m a programming language researcher, and we are working on a new visual 
> programming language.
>
> I’m investigating using WebCore as the rendering component of our language 
> editing/visualization system.
>
> Essentially what I’d like to be able to do is to programmatically (C++) 
> generate 1: a DOM tree of a new family of Element derived object, 2: 
> rendering tree, 3: layout the tree, 4: attach a GraphicsContext to an 
> existing window or bitmapped resource, and 5: use WebCore’s GraphicsContext 
> to render the custom rendering tree. I’d also like to create a family of 
> custom Element derived objects that would be attached to the rendering tree 
> and use the rendering tree. I’d also like to respond to the standard events, 
> i.e. mouse, keyboard, etc…

Are you really sure you need to go that deep?

Can't your goals be achieved with just custom DOM tree + style sheets + drawing 
that on exisisting resource using standard WebCore rendering machinery?

If this is the case I propose you to take a look at QtWebKit port, where you 
can build DOM in C++ and use custom QPainter/QPaintEngine for low-level drawing 
operations. Qt also provides you a way to simulate user input events like key 
presses or mouse moves, or you can filter out or override real input events.

Here is my fork which uses updated WebKit from 2016:

https://github.com/annulen/webkit
(See https://github.com/annulen/webkit/wiki for more details)

>
> Do you think this would be possible using WebCore as a library and have these 
> custom render tree / dom tree derived objects live in my own library (this is 
> really, really the way I’d prefer to do things), or do you think it would be 
> better to add these objects directly into the WebCore library.
>
> I don’t think accessing any of the render tree objects in webcore would cause 
> any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros 
> on each class def. If I added these, what would be the WebKit policy of 
> accepting these changes?

I don't think exporting private APIs will be accepted, and I don't think it 
makes much sense because they may be changed at any time anyway. If you need 
them, you will be safer with your customized WebKit fork.


>
> What is the, for lack of a better word, the “viability” of WebKit on Windows? 
> Our project fundamentally has to be cross-platform, and currently, I’m not 
> aware of any webkit based browsers on Windows. I’m currently looking into 
> using webkit, firefox or blink. I’ve essentially eliminated blink because 
> their code is very hard to follow, much harder than webkit or firefox, and I 
> think the webkit code is the easiest to understand and use. Webkit is 
> currently my first choice, but my only hesitation is will webkit continued to 
> be supported on Windows.

AFAIK, there is no indication that Windows support is going to be dropped 
anytime soon.

>
> thanks
>
> -- Andy Somogyi PhD
> School of Informatics and Computing
> Indiana University
> ,
>
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev


-- 
Regards,
Konstantin
___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

[webkit-dev] WebKit render tree

[Andy Somogyi]

Hi,

I’m a programming language researcher, and we are working on a new visual 
programming language. 

I’m investigating using WebCore as the rendering component of our language 
editing/visualization system. 

Essentially what I’d like to be able to do is to programmatically (C++) 
generate 1: a DOM tree of a new family of Element derived object, 2: rendering 
tree, 3: layout the tree, 4: attach a GraphicsContext to an existing window or 
bitmapped resource, and 5: use WebCore’s GraphicsContext to render the custom 
rendering tree. I’d also like to create a family of custom Element derived 
objects that would be attached to the rendering tree and use the rendering 
tree. I’d also like to respond to the standard events, i.e. mouse, keyboard, 
etc…

Do you think this would be possible using WebCore as a library and have these 
custom render tree / dom tree derived objects live in my own library (this is 
really, really the way I’d prefer to do things), or do you think it would be 
better to add these objects directly into the WebCore library. 

I don’t think accessing any of the render tree objects in webcore would cause 
any issues on Mac/Unix, however, Windows sadly requires those EXPORT macros on 
each class def. If I added these, what would be the WebKit policy of accepting 
these changes? 

What is the, for lack of a better word, the “viability” of WebKit on Windows? 
Our project fundamentally has to be cross-platform, and currently, I’m not 
aware of any webkit based browsers on Windows. I’m currently looking into using 
webkit, firefox or blink. I’ve essentially eliminated blink because their code 
is very hard to follow, much harder than webkit or firefox, and I think the 
webkit code is the easiest to understand and use. Webkit is currently my first 
choice, but my only hesitation is will webkit continued to be supported on 
Windows. 

thanks

-- Andy Somogyi PhD
School of Informatics and Computing
Indiana University___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev

Re: [webkit-dev] getComputedStyle does not handle over-constrained properties correctly

[Myles C. Maxfield]

It looks like WebKit visually renders the result correctly according to the 
spec text. Therefore, we are only interested here with the computed style of 
the over-specified element.

The spec text uses the verb “becomes.” I don’t know if this means that either 
1) the rendering and the computed style should reflect this, or 2) just the 
rendering should reflect this.

Do you know if this issue has been discussed in the W3C?

Thanks,
Myles
> On May 27, 2016, at 5:59 AM, 김덕진  wrote:
> 
> Hello, 
> 
> I'm working on blink engine as deokjin81@samsung.com 
> .
> And I have a question about implementation plan for getComputedStyle.
> As I know, getComputedStyle does not handle over-constrained properties 
> correctly.
> So I implemented 
> it(https://bugs.chromium.org/p/chromium/issues/detail?id=601118 
> ) according to 
> spec(https://www.w3.org/TR/2011/REC-CSS2-20110607/visuren.html#relative-positioning
>  
> )
>  on blink engine.
> 
> Below paragraphs(from spec) describe detail operation to handle 
> over-constrained properties.
> 
> If neither 'left' nor 'right' is 'auto', the position is over-constrained, 
> and one of them has to be ignored. If the 'direction' property of the 
> containing block is 'ltr', the value of 'left' wins and 'right' becomes 
> -'left'. If 'direction' of the containing block is 'rtl', 'right' wins and 
> 'left' is ignored.
> 
> The 'top' and 'bottom' properties move relatively positioned element(s) up or 
> down without changing their size. 'Top' moves the boxes down, and 'bottom' 
> moves them up. Since boxes are not split or stretched as a result of 'top' or 
> 'bottom', the used values are always: top = -bottom. If both are 'auto', 
> their used values are both '0'. If one of them is 'auto', it becomes the 
> negative of the other. If neither is 'auto', 'bottom' is ignored (i.e., the 
> used value of 'bottom' will be minus the value of 'top').
> 
> I would like to know Webkit  have any plan for this.
> 
> Thank you in advance,
> Deokjin Kim
> 
> 
> 
> ___
> webkit-dev mailing list
> webkit-dev@lists.webkit.org
> https://lists.webkit.org/mailman/listinfo/webkit-dev

___
webkit-dev mailing list
webkit-dev@lists.webkit.org
https://lists.webkit.org/mailman/listinfo/webkit-dev