Malia, pretty interesting stylesheet. Thanks for the look. At some point
it will be a very good case to feed through the profiler. If you're
inclined, you might consider putting it and your three files in a zip file
and sending to me offline, so we can do some non-speculative diagnosis and
see what might be done. We can also add it to our regular performance
regression tests, if you're OK with that.
I suspect use of the preceding:: axis is the main culprit.
I'm not sure which version of Xalan you're using. Recent versions should
be able to optimize some of the "/document/record" partial paths away by
doing redundent expression elimination. Of course, it's possible that
we're doing something wrong at the rewrite stage and making things worse in
this case.
-scott
Malia Zaheer
<[EMAIL PROTECTED] To: Joseph Kesselman
<[EMAIL PROTECTED]>, "Frank E. Weiss" <[EMAIL PROTECTED]>
corp.com> cc: [EMAIL PROTECTED],
(bcc: Scott Boag/Cambridge/IBM)
Subject: RE: FW: transformer
hangs
07/25/2002 10:54
AM
After using SALT instead of interpretive XSLT and using SAXSource instead
of
StreamSource for input, the performance increased significantly, but not
yet
optimal. Here are the results:
25KB xml - took 14 sec
50KB xml - took 70 sec
75KB xml - took 220 sec
I know that the stylesheet that I am using is not efficient. That is
because I am generating that stylesheet programmatically, not by hand so
that I can optimize it for each input. Here is the stylesheet (I know it's
hard to read, but it's just nested for-each, if and variable statements
followed by choose statement). I have also attached it to this email if
you
want to open it in IE, it's easier to read.
<?xml version="1.0" encoding="UTF-8"?>
<xsl:stylesheet xmlns:xsl="http://www.w3.org/1999/XSL/Transform";
exclude-result-prefixes="java" version="1.0"
xmlns:java="http://xml.apache.org/xslt/java";><xsl:output indent="yes"
method="xml"/><xsl:template match="/"><xsl:element
name="document"><xsl:call-template
name="template_1"/></xsl:element></xsl:template>
<xsl:template name="template_1">
<xsl:for-each select="/document/record/Flags[not(preceding::Flags=.)]">
<xsl:variable name="Flags_1" select="."/>
<xsl:if test="/document[record[Flags=$Flags_1]]">
<xsl:for-each
select="/document/record/SPM_RegioId[not(preceding::SPM_RegioId=.)]">
<xsl:variable name="SPM_RegioId_2" select="."/>
<xsl:if
test="/document[record[Flags=$Flags_1][SPM_RegioId=$SPM_RegioId_2]]">
<xsl:for-each
select="/document/record/SPM_DeviceId[not(preceding::SPM_DeviceId=.)]">
<xsl:variable name="SPM_DeviceId_3" select="."/>
<xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SPM_Reg
ioId=$SPM_RegioId_2]]">
<xsl:for-each
select="/document/record/SUB_Instance[not(preceding::SUB_Instance=.)]">
<xsl:variable name="SUB_Instance_4" select="."/>
<xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SUB_Ins
tance=$SUB_Instance_4][SPM_RegioId=$SPM_RegioId_2]]">
<xsl:for-each
select="/document/record/SPM_SubId[not(preceding::SPM_SubId=.)]">
<xsl:variable name="SPM_SubId_5" select="."/>
<xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SUB_Ins
tance=$SUB_Instance_4][SPM_SubId=$SPM_SubId_5][SPM_RegioId
=$SPM_RegioId_2]]"
>
<xsl:for-each
select="/document/record/SPM_IspId[not(preceding::SPM_IspId=.)]">
<xsl:variable name="SPM_IspId_6" select="."/><xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SUB_Ins
tance=$SUB_Instance_4][SPM_SubId=$SPM_SubId_5][SPM_RegioId
=$SPM_RegioId_2][S
PM_IspId=$SPM_IspId_6]]">
<xsl:for-each
select="/document/record/TimeStamp[not(preceding::TimeStamp
=.)]"><xsl:variab
le name="TimeStamp_7" select="."/>
<xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SUB_Ins
tance=$SUB_Instance_4][SPM_SubId=$SPM_SubId_5][SPM_RegioId
=$SPM_RegioId_2][T
imeStamp=$TimeStamp_7][SPM_IspId=$SPM_IspId_6]]">
<xsl:for-each
select="/document/record/SPM_TRUNKID[not(preceding::SPM_TRUNKID
=.)]"><xsl:va
riable name="SPM_TRUNKID_8" select="."/>
<xsl:if
test="/document[record[SPM_DeviceId=$SPM_DeviceId_3][Flags
=$Flags_1][SPM_TRU
NKID=$SPM_TRUNKID_8][SUB_Instance=$SUB_Instance_4][SPM_SubId
=$SPM_SubId_5][S
PM_RegioId=$SPM_RegioId_2][TimeStamp=$TimeStamp_7][SPM_IspId
=$SPM_IspId_6]]"
>
<xsl:for-each
select="/document/record/IFI_IPACKETS[not(preceding::IFI_IPACKETS
=.)]"><xsl:
variable name="IFI_IPACKETS_9" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[Flags=$Flags_1][SPM_RegioId=$SPM_RegioId_2][SPM_TRUNKID
=$SPM_TRUNKID_8][SUB
_Instance=$SUB_Instance_4][SPM_DeviceId=$SPM_DeviceId_3][SPM_IspId
=$SPM_IspI
d_6][TimeStamp=$TimeStamp_7]]">
<xsl:for-each
select="/document/record/IFI_OPACKETS[not(preceding::IFI_OPACKETS=.)]">
<xsl:variable name="IFI_OPACKETS_10" select="."/><xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OPACKETS=$IFI_OPACKETS_10][Flags=$Flags_1][SPM_RegioId=$SPM_RegioId_2]
[
SPM_TRUNKID=$SPM_TRUNKID_8][SUB_Instance=$SUB_Instance_4][SPM_DeviceId
=$SPM_
DeviceId_3][SPM_IspId=$SPM_IspId_6][TimeStamp=$TimeStamp_7]]">
<xsl:for-each
select="/document/record/IFI_IBYTES[not(preceding::IFI_IBYTES
=.)]"><xsl:vari
able name="IFI_IBYTES_11" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OPACKETS=$IFI_OPACKETS_10][Flags=$Flags_1][SPM_RegioId=$SPM_RegioId_2]
[
IFI_IBYTES=$IFI_IBYTES_11][SPM_TRUNKID=$SPM_TRUNKID_8][SUB_Instance
=$SUB_Ins
tance_4][SPM_DeviceId=$SPM_DeviceId_3][SPM_IspId=$SPM_IspId_6][TimeStamp
=$Ti
meStamp_7]]">
<xsl:for-each
select="/document/record/IFI_OBYTES[not(preceding::IFI_OBYTES=.)]">
<xsl:variable name="IFI_OBYTES_12" select="."/><xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OBYTES=$IFI_OBYTES_12][IFI_OPACKETS=$IFI_OPACKETS_10][Flags
=$Flags_1][S
PM_RegioId=$SPM_RegioId_2][IFI_IBYTES=$IFI_IBYTES_11][SPM_TRUNKID
=$SPM_TRUNK
ID_8][SUB_Instance=$SUB_Instance_4][SPM_DeviceId=$SPM_DeviceId_3][SPM_IspId
=
$SPM_IspId_6][TimeStamp=$TimeStamp_7]]"><xsl:for-each
select="/document/record/IFI_IQDROPS[not(preceding::IFI_IQDROPS
=.)]"><xsl:va
riable name="IFI_IQDROPS_13" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OBYTES=$IFI_OBYTES_12][IFI_OPACKETS=$IFI_OPACKETS_10][Flags
=$Flags_1][I
FI_IQDROPS=$IFI_IQDROPS_13][SPM_RegioId=$SPM_RegioId_2][IFI_IBYTES
=$IFI_IBYT
ES_11][SPM_TRUNKID=$SPM_TRUNKID_8][SUB_Instance
=$SUB_Instance_4][SPM_DeviceI
d=$SPM_DeviceId_3][SPM_IspId=$SPM_IspId_6][TimeStamp
=$TimeStamp_7]]"><xsl:fo
r-each select="/document/record/IFI_OQDROPS[not(preceding::IFI_OQDROPS
=.)]">
<xsl:variable name="IFI_OQDROPS_14" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OBYTES=$IFI_OBYTES_12][IFI_OPACKETS=$IFI_OPACKETS_10][Flags
=$Flags_1][I
FI_IQDROPS=$IFI_IQDROPS_13][SPM_RegioId=$SPM_RegioId_2][IFI_IBYTES
=$IFI_IBYT
ES_11][SPM_TRUNKID=$SPM_TRUNKID_8][SUB_Instance
=$SUB_Instance_4][SPM_DeviceI
d=$SPM_DeviceId_3][SPM_IspId=$SPM_IspId_6][TimeStamp
=$TimeStamp_7][IFI_OQDRO
PS=$IFI_OQDROPS_14]]">
<xsl:for-each
select="/document/record/PKTS_DROP_ERR[not(preceding::PKTS_DROP_ERR
=.)]"><xs
l:variable name="PKTS_DROP_ERR_15" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[IFI_OBYTES=$IFI_OBYTES_12][IFI_OPACKETS=$IFI_OPACKETS_10][Flags
=$Flags_1][I
FI_IQDROPS=$IFI_IQDROPS_13][SPM_RegioId=$SPM_RegioId_2][IFI_IBYTES
=$IFI_IBYT
ES_11][PKTS_DROP_ERR=$PKTS_DROP_ERR_15][SPM_TRUNKID
=$SPM_TRUNKID_8][SUB_Inst
ance=$SUB_Instance_4][SPM_DeviceId=$SPM_DeviceId_3][SPM_IspId=$SPM_IspId_6]
[
TimeStamp=$TimeStamp_7][IFI_OQDROPS=$IFI_OQDROPS_14]]"><xsl:for-each
select="/document/record/MULTICAST_IN_PKTS[not(preceding::MULTICAST_IN_PKTS
=
.)]">
<xsl:variable name="MULTICAST_IN_PKTS_16" select="."/>
<xsl:if
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[MULTICAST_IN_PKTS=$MULTICAST_IN_PKTS_16][IFI_OBYTES
=$IFI_OBYTES_12][IFI_OPA
CKETS=$IFI_OPACKETS_10][Flags=$Flags_1][IFI_IQDROPS
=$IFI_IQDROPS_13][SPM_Reg
ioId=$SPM_RegioId_2][IFI_IBYTES=$IFI_IBYTES_11][PKTS_DROP_ERR
=$PKTS_DROP_ERR
_15][SPM_TRUNKID=$SPM_TRUNKID_8][SUB_Instance=$SUB_Instance_4][SPM_DeviceId
=
$SPM_DeviceId_3][SPM_IspId=$SPM_IspId_6][TimeStamp
=$TimeStamp_7][IFI_OQDROPS
=$IFI_OQDROPS_14]]">
<xsl:for-each
select
="/document/record/MULTICAST_OUT_PKTS[not(preceding::MULTICAST_OUT_PKT
S=.)]"><xsl:variable name="MULTICAST_OUT_PKTS_17" select="."/>
<xsl:choose>
<xsl:when
test="/document[record[SPM_SubId=$SPM_SubId_5][IFI_IPACKETS
=$IFI_IPACKETS_9]
[MULTICAST_IN_PKTS=$MULTICAST_IN_PKTS_16][IFI_OBYTES
=$IFI_OBYTES_12][IFI_OPA
CKETS=$IFI_OPACKETS_10][Flags=$Flags_1][MULTICAST_OUT_PKTS
=$MULTICAST_OUT_PK
TS_17][IFI_IQDROPS=$IFI_IQDROPS_13][SPM_RegioId=$SPM_RegioId_2][IFI_IBYTES
=$
IFI_IBYTES_11][PKTS_DROP_ERR=$PKTS_DROP_ERR_15][SPM_TRUNKID=$SPM_TRUNKID_8]
[
SUB_Instance=$SUB_Instance_4][SPM_DeviceId=$SPM_DeviceId_3][SPM_IspId
=$SPM_I
spId_6][TimeStamp=$TimeStamp_7][IFI_OQDROPS=$IFI_OQDROPS_14]]">
<xsl:element name="record"><xsl:element name="Flags"><xsl:value-of
select="$Flags_1"/></xsl:element><xsl:element
name="SPM_RegioId"><xsl:value-of select="$SPM_RegioId_2"/></xsl:element>
<xsl:element name="SPM_DeviceId"><xsl:value-of
select="$SPM_DeviceId_3"/></xsl:element>
<xsl:element name="SUB_Instance"><xsl:value-of
select="$SUB_Instance_4"/></xsl:element>
<xsl:element name="SPM_SubId"><xsl:value-of
select="$SPM_SubId_5"/></xsl:element>
<xsl:element name="SPM_IspId"><xsl:value-of
select="$SPM_IspId_6"/></xsl:element>
<xsl:element name="TimeStamp"><xsl:value-of
select="$TimeStamp_7"/></xsl:element>
<xsl:element name="SPM_TRUNKID"><xsl:value-of
select="$SPM_TRUNKID_8"/></xsl:element>
<xsl:element name="IFI_IPACKETS"><xsl:value-of
select="$IFI_IPACKETS_9"/></xsl:element>
<xsl:element name="IFI_OPACKETS"><xsl:value-of
select="$IFI_OPACKETS_10"/></xsl:element>
<xsl:element name="IFI_IBYTES"><xsl:value-of
select="$IFI_IBYTES_11"/></xsl:element>
<xsl:element name="IFI_OBYTES"><xsl:value-of
select="$IFI_OBYTES_12"/></xsl:element>
<xsl:element name="IFI_IQDROPS"><xsl:value-of
select="$IFI_IQDROPS_13"/></xsl:element>
<xsl:element name="IFI_OQDROPS"><xsl:value-of
select="$IFI_OQDROPS_14"/></xsl:element>
<xsl:element name="PKTS_DROP_ERR"><xsl:value-of
select="$PKTS_DROP_ERR_15"/></xsl:element><xsl:element
name="MULTICAST_IN_PKTS"><xsl:value-of
select="$MULTICAST_IN_PKTS_16"/></xsl:element><xsl:element
name="MULTICAST_OUT_PKTS"><xsl:value-of
select="$MULTICAST_OUT_PKTS_17"/></xsl:element></xsl:element>
</xsl:when>
</xsl:choose>
</xsl:for-each>
</xsl:if>
</xsl:for-each></xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl:if></xs
l:for-each></xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl:if></xsl:fo
r-each></xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl:if></xsl:for-ea
ch></xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl:if></xsl:for-each>
<
/xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl:if></xsl:for-each></xsl
:template></xsl:stylesheet>
And sample data record is:
<record><Flags>0</Flags>
<SPM_RegioId>1</SPM_RegioId>
<SPM_DeviceId>1</SPM_DeviceId>
<SUB_Instance>-1</SUB_Instance>
<SPM_SubId>-1</SPM_SubId>
<SPM_IspId>6</SPM_IspId>
<TimeStamp>Thu Jul 04 17:40:30 EDT 2002</TimeStamp>
<SPM_TRUNKID>45</SPM_TRUNKID>
<IFI_IPACKETS>113</IFI_IPACKETS>
<IFI_OPACKETS>219</IFI_OPACKETS>
<IFI_IBYTES>7002</IFI_IBYTES>
<IFI_OBYTES>13038</IFI_OBYTES>
<IFI_IQDROPS>0</IFI_IQDROPS>
<IFI_OQDROPS>0</IFI_OQDROPS>
<PKTS_DROP_ERR>0</PKTS_DROP_ERR>
<MULTICAST_IN_PKTS>6760</MULTICAST_IN_PKTS>
<MULTICAST_OUT_PKTS>0</MULTICAST_OUT_PKTS>
</record>
Do you know of a good site that focus on stylesheet efficiency?
-----Original Message-----
From: Joseph Kesselman [mailto:[EMAIL PROTECTED]
Sent: Thursday, July 25, 2002 9:46 AM
To: Frank E. Weiss
Cc: [EMAIL PROTECTED]
Subject: Re: FW: transformer hangs
>If a lot of cross-referencing of the input document is occuring,
>then the process can increase super-linearly, that is to say,
exponentially or >factorially.
Note that there isn't much in XSLT or XPath, per se, which scales worse
than linearly -- but there are many ways to write a path or stylesheet
which does. Also, of course, if you're processing something large enough
to drive your system into heavy swapping to disk you can expect to see a
knee in the curve.
So I'm still inclined to agree that this is most likely an inefficient
stylesheet. It may well be one that _can't_ be written efficiently;
convolving multiple lists against each other is sometimes necessary and
always expensive... but it might be a good idea to look at one of the
websites which suggest ways of improving stylesheet efficiency, and see
whether any of those tips apply to what you're trying to do.
______________________________________
Joe Kesselman / IBM Research
