Hi Zach,
a quick answer would be to look around if there is an other runtime vm for
your Linux environement and compare the time consumption for your test
classes.
The harder way is to look some titles in the Bookstore (aka amazon), scan
the topics of the 'table of contents' of titles with Java + performance (+
optional
(L|U)nix) in the search and figure out where to start performance tuning.
The lastest figures may point in the I/O area - try a system with an other
file
system type since writing is so time costly in 2.5.1. Do the test always
twice
to archive the leverage effects of any caching et alt.
May the source be with you (SCR ;=),
good luck, Stefan
P.S.: Try to figure out if there is a chance to use the Window$ within your
architectur in order to get rid with that annoyance.
> -----Ursprüngliche Nachricht-----
> Von: Zach Engel [SMTP:[EMAIL PROTECTED]
> Gesendet am: Donnerstag, 18. August 2005 13:51
> An: 'POI Users List'
> Betreff: RE: bad performance in linux
>
> Hi
>
> Sadly, I am not a unix / linux guy, so can anyone (Marx Stefan?) refer me
> to
> some info / best practices on the subject of Linux environment tuning?
>
> in the mean time I created a simple java class that creates a simple
> workbook. (see code below)
>
> Look at the differences between windows and linux
>
>
> performance in windows
> ======================
>
> Using poi version 1.5.1
> D:\JbuilderProjects\src\general\POI-SRC\run>java -cp
> .;jakarta-poi-1.5.1-final-20020615.jar TestWorkbook_v151 400 10
> create workBook millis=562
> write workBook millis=125
> entire test millis=687
>
>
> Using poi version 2.5.1
> D:\JbuilderProjects\src\general\POI-SRC\run>java -cp
> .;poi-2.5.1-final-20040804.jar TestWorkbook_v251 400 10
> create workBook millis=500
> write workBook millis=218
> entire test millis=718
>
>
>
>
>
>
> performance in linux
> ======================
>
> Using poi version 1.5.1
> [EMAIL PROTECTED] run]# java -cp .:jakarta-poi-1.5.1-final-20020615.jar
> TestWorkbook_v151 400 10
> create workBook millis=1237
> write workBook millis=275
> entire test millis=1512
>
>
> Using poi version 2.5.1
> [EMAIL PROTECTED] run]# java -cp .:poi-2.5.1-final-20040804.jar
> TestWorkbook_v251 400 10
> create workBook millis=1099
> write workBook millis=18959
> entire test millis=20058
>
>
>
>
>
>
>
> as you can see, in linux,
> generating a sheet with 400 rows, 10 cells in each row takes 1512
> milliseconds while using a very old version of POI (1.5.1),
> however, when using the newest version (2.5.1) -
> it takes 20058 milliseconds!
>
>
> I don't know if it is important, but here are the environment variables of
> both environments:
>
>
> linux
> =======================
> [EMAIL PROTECTED] run]# uname -srm
> Linux 2.4.20-13.7 i686
>
> [EMAIL PROTECTED] run]# env
> PWD=/tmp/zach/run
> VENDOR=intel
> REMOTEHOST=zaksrv2.checkpoint.com
> HOSTNAME=harrison-dev.checkpoint.com
> LESSOPEN=|/usr/bin/lesspipe.sh %s
> USER=zak
> LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=4
> 0;
> 33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com
> =0
> 1;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;3
> 1:
> *.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=0
> 1;
> 31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:
> *.
> gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
> MACHTYPE=i386
> MAIL=/var/mail/zak
> LANG=en_US.UTF8
> HOST=harrison-dev.checkpoint.com
> SSH_CLIENT=194.29.44.79 2642 22
> LOGNAME=zak
> SHLVL=4
> GROUP=unknown
> SUPPORTED=en_US.UTF8:en_US:en:en_US.iso885915
> SHELL=/bin/bash
> HOSTTYPE=i386-linux
> OSTYPE=linux
> HOME=/root
> TERM=vt100
> SSH_AUTH_SOCK=/tmp/ssh-XXXHtDKX/agent.14463
> PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
> SSH_TTY=/dev/pts/3
> OLDPWD=/tmp/zach
> _=/usr/bin/env
>
>
>
>
>
>
>
>
>
> windows
> =======================
> D:\JbuilderProjects\src\general\POI-SRC\run>set
> ALLUSERSPROFILE=C:\Documents and Settings\All Users
> ANT_HOME=c:\apache-ant-1.6.5
> APPDATA=C:\Documents and Settings\zak\Application Data
> CLIENTNAME=Console
> COMMONHOME=C:\Program Files\Rational\common
> CommonProgramFiles=C:\Program Files\Common Files
> COMPUTERNAME=ZAKSRV2
> ComSpec=C:\WINDOWS\system32\cmd.exe
> DISPLAY=:0.0
> FORREST_HOME=c:\apache-forrest-0.7
> FP_NO_HOST_CHECK=NO
> HOME=C:\Documents and Settings\zak
> HOMEDRIVE=C:
> HOMEPATH=\Documents and Settings\zak
> HSSF.testdata.path=D:\RequiredLibs\poi\source\src\testcases\org\apache\poi
> \h
> ssf\
> data\
> J2EE_HOME=c:\j2sdkee1.3.1\
> JAVA_HOME=C:\j2sdk1.4.2_08
> JAVA_HOME_1.4.1_02=C:\j2sdk1.4.1_02
> LOGONSERVER=\\HAMPSON
> NUMBER_OF_PROCESSORS=1
> NUTCROOT=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT
> NUTSUFFIX=1
> NUT_SUFFIXED_SEARCHING=1
> OS=Windows_NT
> Path=C:\j2sdk1.4.2_08\bin;D:\sapjcorfcLocation\;c:\apache-ant-1.6.5\bin;c:
> \a
> pach
> e-forrest-0.7\bin;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\bin;C:\PROGRA~1\R
> AT
> IONA
> L\RATION~1\NUTCROOT\bin\x11;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\mksnt;d
> :\
> sapd
> b\programs\bin;d:\sapdb\programs\pgm;C:\ora817\bin;C:\Program
> Files\Oracle\jre\1
> .1.7\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Progra
> m
> File
> s\Rational\common;C:\Program Files\Rational\ClearQuest;C:\Program
> Files\Rational
> \Rose\TopLink\;C:\Program Files\Rational\Rational
> Test;C:\JBuilder7\bin;C:\Progr
> am Files\SecureCRT\;C:\PROGRA~1\ULTRAE~1
> PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
> PROCESSOR_ARCHITECTURE=x86
> PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 4, GenuineIntel
> PROCESSOR_LEVEL=15
> PROCESSOR_REVISION=0204
> ProgramFiles=C:\Program Files
> PROMPT=$P$G
> RATL_RTHOME=C:\Program Files\Rational\Rational Test
> ROOTDIR=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT
> RQAHOME=C:\Program Files\Rational\Rational Test\QualityArchitect
> SAPJCORFC_LOCATION=D:\sapjcorfcLocation\
> SESSIONNAME=Console
> SHELL=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\mksnt\sh.exe
> SystemDrive=C:
> SystemRoot=C:\WINDOWS
> TEMP=C:\DOCUME~1\zak\LOCALS~1\Temp
> TERM=nutc
> TERMCAP=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\etc\termcap
> TERMINFO=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\usr\lib\terminfo
> TMP=C:\DOCUME~1\zak\LOCALS~1\Temp
> TMPDIR=C:\DOCUME~1\zak\LOCALS~1\Temp
> USERDNSDOMAIN=AD.CHECKPOINT.COM
> USERDOMAIN=AD
> USERNAME=zak
> USERPROFILE=C:\Documents and Settings\zak
> windir=C:\WINDOWS
>
>
>
> and here is the code for the test class (I built one for POI 1.5.1 and one
> for POI 2.5.1, as there is a slight difference in the POI interfaces...
>
> /*************************************
> ** start of v 2.5.1 class
> *************************************/
>
> import java.io.*;
> import org.apache.poi.hssf.usermodel.*;
>
> public class TestWorkbook_v251 {
>
> public TestWorkbook_v251() {
> }
>
>
> public static void main(String[] args) {
>
> long[] millis = new long[2];
>
> int NUM_OF_ROWS = 10;
> int NUM_OF_COLS = 10;
>
>
> // get num of rows and columns from command line
> if (args != null && args.length > 0) {
> NUM_OF_ROWS = Integer.parseInt(args[0]);
> NUM_OF_COLS = NUM_OF_ROWS;
>
> if (args.length > 1) {
> NUM_OF_COLS = Integer.parseInt(args[1]);
> }
> }
>
>
>
> TestWorkbook_v251 testLargeWorkbook1 = new TestWorkbook_v251();
>
> testLargeWorkbook1.testCreateAndWriteHSSFWorkbook(NUM_OF_ROWS,
> NUM_OF_COLS);
>
> }
>
>
>
>
> public void testCreateAndWriteHSSFWorkbook(final int NUM_OF_ROWS, final
> int NUM_OF_COLS) {
> try {
> long[] millis = new long[4];
> millis[0] = System.currentTimeMillis();
> HSSFWorkbook hssfwb = createHSSFWorkBook(NUM_OF_ROWS, NUM_OF_COLS);
> millis[1] = System.currentTimeMillis();
>
> millis[2] = System.currentTimeMillis();
> OutputStream out = getOutputStream();
> hssfwb.write(out);
> millis[3] = System.currentTimeMillis();
>
> System.out.println("create workBook millis=" +
> (millis[1]-millis[0]));
> System.out.println("write workBook millis=" +
> (millis[3]-millis[2]));
> System.out.println("entire test millis=" +
> (millis[3]-millis[0]));
>
> }
> catch (IOException ex) {
> System.err.println("testLargeWorkbook1 failed");
> ex.printStackTrace();
> }
>
> }
>
>
>
>
>
>
> private OutputStream getOutputStream() throws IOException,
> FileNotFoundException {
> final String fileLocation = "testXLS.xls";
>
> File file = new File (fileLocation);
> if (file.exists() == false) {
> file.createNewFile();
> }
>
> BufferedOutputStream outStream = new BufferedOutputStream(new
> FileOutputStream(fileLocation));
>
> return outStream;
> }
>
> public HSSFWorkbook createHSSFWorkBook(final int NUM_OF_ROWS, final int
> NUM_OF_COLS) {
> HSSFWorkbook wb = new HSSFWorkbook();
> HSSFSheet hssfSheet1 = wb.createSheet("sheet1");
>
> for (int i = 0; i < NUM_OF_ROWS; i++) {
> HSSFRow row = hssfSheet1.createRow(i) ;
> for (int j = 0; j < NUM_OF_COLS; j++) {
> HSSFCell cell = row.createCell((short)j);
> cell.setCellValue("cell (" + i + "," + j + ")");
> }
> }
>
> return wb;
> }
> }
>
> /*************************************
> ** end of v 2.5.1 class
> *************************************/
>
>
>
>
>
> /*************************************
> ** start of v 1.5.1 class
> *************************************/
>
> import java.io.*;
> import org.apache.poi.hssf.usermodel.*;
>
> public class TestWorkbook_v151 {
>
> public TestWorkbook_v151() {
> }
>
>
> public static void main(String[] args) {
>
> long[] millis = new long[2];
>
> int NUM_OF_ROWS = 10;
> int NUM_OF_COLS = 10;
>
>
> // get num of rows and columns from command line
> if (args != null && args.length > 0) {
> NUM_OF_ROWS = Integer.parseInt(args[0]);
> NUM_OF_COLS = NUM_OF_ROWS;
>
> if (args.length > 1) {
> NUM_OF_COLS = Integer.parseInt(args[1]);
> }
> }
>
>
>
> TestWorkbook_v151 testLargeWorkbook1 = new TestWorkbook_v151();
>
> testLargeWorkbook1.testCreateAndWriteHSSFWorkbook(NUM_OF_ROWS,
> NUM_OF_COLS);
>
> }
>
>
>
>
> public void testCreateAndWriteHSSFWorkbook(final int NUM_OF_ROWS, final
> int NUM_OF_COLS) {
> try {
> long[] millis = new long[4];
> millis[0] = System.currentTimeMillis();
> HSSFWorkbook hssfwb = createHSSFWorkBook(NUM_OF_ROWS, NUM_OF_COLS);
> millis[1] = System.currentTimeMillis();
>
> millis[2] = System.currentTimeMillis();
> OutputStream out = getOutputStream();
> hssfwb.write(out);
> millis[3] = System.currentTimeMillis();
>
> System.out.println("create workBook millis=" +
> (millis[1]-millis[0]));
> System.out.println("write workBook millis=" +
> (millis[3]-millis[2]));
> System.out.println("entire test millis=" +
> (millis[3]-millis[0]));
>
> }
> catch (IOException ex) {
> System.err.println("testLargeWorkbook1 failed");
> ex.printStackTrace();
> }
>
> }
>
>
>
>
>
>
> private OutputStream getOutputStream() throws IOException,
> FileNotFoundException {
> final String fileLocation = "testXLS.xls";
>
> File file = new File (fileLocation);
> if (file.exists() == false) {
> file.createNewFile();
> }
>
> BufferedOutputStream outStream = new BufferedOutputStream(new
> FileOutputStream(fileLocation));
>
> return outStream;
> }
>
> public HSSFWorkbook createHSSFWorkBook(final int NUM_OF_ROWS, final int
> NUM_OF_COLS) {
> HSSFWorkbook wb = new HSSFWorkbook();
> HSSFSheet hssfSheet1 = wb.createSheet("sheet1");
>
> for (int i = 0; i < NUM_OF_ROWS; i++) {
> HSSFRow row = hssfSheet1.createRow((short)i) ;
> for (int j = 0; j < NUM_OF_COLS; j++) {
> HSSFCell cell = row.createCell((short)j);
> cell.setCellValue("cell (" + i + "," + j + ")");
> }
> }
>
> return wb;
> }
> }
>
>
> /*************************************
> ** end of v 1.5.1 class
> *************************************/
>
>
>
> -----Original Message-----
> From: Marx, Stefan ext. MA Fa. Lynx
> [mailto:[EMAIL PROTECTED]
> Sent: Thursday, August 18, 2005 2:00 PM
> To: 'POI Users List'
> Subject: AW: bad performance in linux
>
> Hi Zach,
>
> 'cause you didn't the machine data I assume that you measured
> the times on the same machine.
> After you investigated in the direction of file systems (blocksize,
> journaling (Reiser, ext3), give your vm a try. Are you using a hot-
> spot version. Andy Oliver from the core team wrote about huge
> differences in the runtime behaviour of different flavours with SUNs
> runtime set as most performant among other (Blackbird etc.).
>
> May be you can tune your Linux enviroment in this area.
>
> cu Stefan
>
> > -----Ursprüngliche Nachricht-----
> > Von: Zach Engel [SMTP:[EMAIL PROTECTED]
> > Gesendet am: Mittwoch, 17. August 2005 21:08
> > An: 'POI Users List'
> > Betreff: bad performance in linux
> >
> > Hi
> >
> > Need some help with the following performance problem:
> >
> > I have some code that creates a HSSFWorkBook
> > (about 400 rows of 10 cells each) and then writes it
> > to an output stream (out = HttpServletResponse.getOutputStream())
> >
> > This takes 2 seconds on a WINDOWS machine.
> > Here is some info from my log
> > [17 Aug 2005 18:12:30 DEBUG] - hssf entire write millis=1908
> > [17 Aug 2005 18:12:30 DEBUG] - hssf getbytes millis=497
> > [17 Aug 2005 18:12:30 DEBUG] - hssf createDocument millis=713
> > [17 Aug 2005 18:12:30 DEBUG] - hssf writeFilesystem millis=698
> >
> > When I run the same code on a linux machine (Linux 2.4.20-13.7),
> > it takes 7-10 seconds.
> > Here is some info from my log
> > [2005-08-17 19:07:21 DEBUG] - hssf entire write millis=7051
> > [2005-08-17 19:07:21 DEBUG] - hssf getbytes millis=4441
> > [2005-08-17 19:07:21 DEBUG] - hssf createDocument millis=46
> > [2005-08-17 19:07:21 DEBUG] - hssf writeFilesystem millis=2562
> >
> > After downloading the most current POI sources, and debugging a little,
> > I discovered that when the getBytes() method (in HSSFWorkbook.write)
> tries
> > to serialize the SSTRecord record, it takes about 4000 milliseconds,
> > While it takes only about 47 milliseconds on windows
> >
> >
> > In addition to that, while investigating the writeFilesystem method,
> > I discovered that most of the time is spent writing block2
> > (indeed shhet2 contains the 400 rows of data)
> >
> > From the Linux log:
> > [2005-08-17 19:28:02 DEBUG] - hssf writeBlock[1] millis=0
> > [2005-08-17 19:28:04 DEBUG] - hssf writeBlock[2] millis=2300
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[3] millis=210
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[4] millis=0
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[5] millis=0
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[6] millis=0
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[7] millis=0
> > [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[8] millis=1
> >
> >
> > From the WINDOWS log:
> > [17 Aug 2005 18:33:31 DEBUG] - hssf writeBlock[1] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[2] millis=1807
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[3] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[4] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[5] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[6] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[7] millis=0
> > [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[8] millis=0
> >
> >
> >
> >
> >
> >
> > One last thing regarding the LANG environment variable:
> > While looking for answers on the web I noticed there have been some
> issues
> > with encoding settings.
> > My LANG on the linux is set to en_US.UTF8
> > I changed it to ISO-8859-1, and then ran the tomcat,
> > but the results where the same.
> >
> > I hope you can offer some kind of explanation to the wide difference
> > between
> > the 2 environments.
> >
> > Thank you
> >
> > Zach Engel
> > Check Point
> >
> >
> >
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > Mailing List: http://jakarta.apache.org/site/mail2.html#poi
> > The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
Mailing List: http://jakarta.apache.org/site/mail2.html#poi
The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
