Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Peter B. West]

No shortages here.

Peter

John Austin wrote:
On Wed, 2003-11-19 at 19:50, Peter B. West wrote:

John Austin wrote:
...

My apologies to everyone on the list for the testy tone  ...
You mean I might have help p*ss*ng people off ?
--
Peter B. West 

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Peter B. West]

Victor Mote wrote:
Peter B. West wrote:


The last assessment of the situation that I can recall on this list was
that property handling is working satisfactorily so there is no need to
pay any attention to it.  Yes, really.


I don't think this is quite right. It is a matter of priorities. Property
handling works, layout does not.
OK.  Except that I would say, "Property handling works as far as it 
goes."  If you want a conforming implementation, property handling has 
to be complete.

I can't speak for others, but my hangup on alt-design properties that has
prevented progress is that it so far has been presented as inextricably
linked to pull-parsing and the other parts of alt-design. Until either 1)
the case for that approach has been made more satisfactorily than it has
until now, or 2) alt-design property handling can be unbundled from
pull-parsing, it really is too much to ask to take the alt-design approach.
Not if it works.  That is a bone of contention, of course.  One of the 
problems with the Rec, I think, is that only by attempting an 
implementation do you gain an understanding of it.

Especially with the FO Tree isolated, I am glad to have someone step in and
improve its performance, if (and only if) we nail down its API to the rest
of FOP so that the rest of FOP doesn't have to know what is going on here.
Peter
--
Peter B. West 

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[John Austin]

On Wed, 2003-11-19 at 19:50, Peter B. West wrote:
> John Austin wrote:
...
> My apologies to everyone on the list for the testy tone  ...
You mean I might have help p*ss*ng people off ?
> Peter
-- 
John Austin <[EMAIL PROTECTED]>

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Peter B. West]

John Austin wrote:
On Tue, 2003-11-18 at 22:36, Peter B. West wrote:

...

Overnight I ran another test, using xml-fop_20031118231549.tar.gz
and the final summary of the call tree SUGGESTS that things are
better (although the results may be incorrect).
...
My focus is on performance. I came to the properties discussion from a
measurement of program performance. In my experience, measurement is
the most reliable approach to optimization.
Entirely understandable.  Let me know what conclusions you come to.  I 
will fill you in off-line on the approach I am taking.

My apologies to everyone on the list for the testy tone of my previous 
posting.

Peter
--
Peter B. West 

RE: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Victor Mote]

Peter B. West wrote:

> The last assessment of the situation that I can recall on this list was
> that property handling is working satisfactorily so there is no need to
> pay any attention to it.  Yes, really.

I don't think this is quite right. It is a matter of priorities. Property
handling works, layout does not.

I can't speak for others, but my hangup on alt-design properties that has
prevented progress is that it so far has been presented as inextricably
linked to pull-parsing and the other parts of alt-design. Until either 1)
the case for that approach has been made more satisfactorily than it has
until now, or 2) alt-design property handling can be unbundled from
pull-parsing, it really is too much to ask to take the alt-design approach.

Especially with the FO Tree isolated, I am glad to have someone step in and
improve its performance, if (and only if) we nail down its API to the rest
of FOP so that the rest of FOP doesn't have to know what is going on here.

Victor Mote

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Glen Mazza]

--- John Austin <[EMAIL PROTECTED]> wrote:
>
> I just spent a while trying to understand
> PropertyList
> and PropertyListBuilder and found out that I need to
> understand Property and Property.Maker as well. I
> think
> I am going to have to help with this part of the
> project
> but it is going to take a while.
> 
> I offered (off-line) to look merging the Alt-Design
> code
> in to the main branch but I suspect that there are
> some
> different directions associated with this. Perhaps
> this
> is the reason it has not been done so far. I am
> still willing
> to work on this but I don't want to walk in to a
> firefight.
> 

It would be great if you can help us out--you may be
one of the few that can understand both Alt-Design and
our current approach in HEAD--plus you seem to be very
knowledgable in code optimization--what we need here.

May I suggest, start tentatively sketching out your
ideas for properties on our FOP wiki page--you'll
probably get comments from many of the other
committers.

> I believe the measurements I did yesterday and I
> feel that
> a bit of algorithm replacement should produce a
> significant
> improvement in the program. I would also like to
> suggest
> that anyone interested in performance look at Java
> Memory
> Profiler at
> http://www.khelekore.org/jmp/performance.html
> 

That looks like a fantastic product for us--I did a
Google search on it as soon as I saw your screen shots
on it--thanks--we possibly should even add it to our
team page so others can critique our code as well.

> I suspect there are still major memory leaks in FOP
> and this
> is one tool that will help you track them down.
> 

Yes, this is something that will help us get to
them--Thanks!

Glen  


__
Do you Yahoo!?
Protect your identity with Yahoo! Mail AddressGuard
http://antispam.yahoo.com/whatsnewfree

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Peter B. West]

John Austin wrote:
Looks like I really put my foot into it this time ;-)
What makes you say that?

I have repeated the measurements I did yesterday and I
think that it is a pretty reasonable conclusion that a
lot of resources are consumed by FOP in its rather
Byzantine property management code. 

I just spent a while trying to understand PropertyList
and PropertyListBuilder and found out that I need to
understand Property and Property.Maker as well. I think
I am going to have to help with this part of the project
but it is going to take a while.
I offered (off-line) to look merging the Alt-Design code
in to the main branch but I suspect that there are some
different directions associated with this. Perhaps this
is the reason it has not been done so far. I am still willing
to work on this but I don't want to walk in to a firefight.
That way lies madness.  When you look into the *actual* requirements for 
property processing, you may well draw the same conclusions as I did. 
In fact, when I got to the point of running some comparative timing 
tests, I had already decided that there were serious flaws in my design 
which I needed to address before proceeding.  Some of those I have 
already mentioned.  I will give you the benefit of my experiences 
off-line.  You may be able to see a way to integrate the work I have 
done with the current directions of HEAD.

I believe the measurements I did yesterday and I feel that
a bit of algorithm replacement should produce a significant
improvement in the program.
The last assessment of the situation that I can recall on this list was 
that property handling is working satisfactorily so there is no need to 
pay any attention to it.  Yes, really.

 I would also like to suggest
that anyone interested in performance look at Java Memory
Profiler at http://www.khelekore.org/jmp/performance.html
I suspect there are still major memory leaks in FOP and this
is one tool that will help you track them down.
Peter
--
Peter B. West 

Re: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[John Austin]

Looks like I really put my foot into it this time ;-)

I have repeated the measurements I did yesterday and I
think that it is a pretty reasonable conclusion that a
lot of resources are consumed by FOP in its rather
Byzantine property management code. 

I just spent a while trying to understand PropertyList
and PropertyListBuilder and found out that I need to
understand Property and Property.Maker as well. I think
I am going to have to help with this part of the project
but it is going to take a while.

I offered (off-line) to look merging the Alt-Design code
in to the main branch but I suspect that there are some
different directions associated with this. Perhaps this
is the reason it has not been done so far. I am still willing
to work on this but I don't want to walk in to a firefight.

I believe the measurements I did yesterday and I feel that
a bit of algorithm replacement should produce a significant
improvement in the program. I would also like to suggest
that anyone interested in performance look at Java Memory
Profiler at http://www.khelekore.org/jmp/performance.html

I suspect there are still major memory leaks in FOP and this
is one tool that will help you track them down.

-- 
John Austin <[EMAIL PROTECTED]>

RE: FOP ~ PropertyList search gives linear performance (FROM: fop-user)

[Victor Mote]

Peter B. West wrote:

[This message was originally on fop-user. I have moved it here as it is
mostly a dev topic.]

> The approach I took was to resolve all of the properties at every node
> in the tree during the parsing.  As the FO tree builder descends to the
> leaf nodes, a complete properties array is maintained at each level.
> Before the tree builder exits a node though, all properties not relevant
> to that particular node type are discarded (i.e. the property sets are
> "reduced") so that the maximal data arrays are short-lived.  The
> relevant properties are maintained in sparse arrays.

FWIW, we now have the FO Tree tied together in a more tree-like way. We have
several methods which rely on going up the tree to an appropriate level to
get information. Right now, that is mostly high-level stuff like the logger
& document-level information. For properties, I recommend that we use
appropriately-named "get" methods for each individual property. Then we can
hide the data structures behind that & modify them as needed. So if a "get"
method sees that the property is inherited, it can get it from the parent
node instead of needing to walk through the tree to resolve the value.

> There is a flaw in this approach which is exposed when markers are
> processed.  Because marker subtrees are conceptually "re-parented" in
> the static-content tree of the page on which they are laid out, the
> properties within the marker tree cannot initially be resolved, and the
> properties of the static-content trees cannot be "reduced".  This
> situation is not catered for in the existing code, but is quite easy to
> accommodate by not reducing the property sets of nodes in the
> static-content, and by slicing the marker subtrees out and storing them
>   unresolved for later processing, including property resolution, in the
> context of static-content.
>
> All of this assumes that property resolution cannot be completed
> independently of layout.  This is a major point of difference between me
> and everyone else on the FOP team, but the implication is there in the
> Rec. if you read it carefully.

I think I agree with your statement that "property resolution cannot be
completed independently of layout" for the specific case that you mention. I
think where we differ is about whether the FO Tree can simply store a "To Be
Determined At Layout" value and go on about life, or whether the concept of
the FO Tree should be scrapped entirely because of this issue.

Victor Mote