Hi
Sadly, I am not a unix / linux guy, so can anyone (Marx Stefan?) refer me to
some info / best practices on the subject of Linux environment tuning?
in the mean time I created a simple java class that creates a simple
workbook. (see code below)
Look at the differences between windows and linux
performance in windows
======================
Using poi version 1.5.1
D:\JbuilderProjects\src\general\POI-SRC\run>java -cp
.;jakarta-poi-1.5.1-final-20020615.jar TestWorkbook_v151 400 10
create workBook millis=562
write workBook millis=125
entire test millis=687
Using poi version 2.5.1
D:\JbuilderProjects\src\general\POI-SRC\run>java -cp
.;poi-2.5.1-final-20040804.jar TestWorkbook_v251 400 10
create workBook millis=500
write workBook millis=218
entire test millis=718
performance in linux
======================
Using poi version 1.5.1
[EMAIL PROTECTED] run]# java -cp .:jakarta-poi-1.5.1-final-20020615.jar
TestWorkbook_v151 400 10
create workBook millis=1237
write workBook millis=275
entire test millis=1512
Using poi version 2.5.1
[EMAIL PROTECTED] run]# java -cp .:poi-2.5.1-final-20040804.jar
TestWorkbook_v251 400 10
create workBook millis=1099
write workBook millis=18959
entire test millis=20058
as you can see, in linux,
generating a sheet with 400 rows, 10 cells in each row takes 1512
milliseconds while using a very old version of POI (1.5.1),
however, when using the newest version (2.5.1) -
it takes 20058 milliseconds!
I dont know if it is important, but here are the environment variables of
both environments:
linux
=======================
[EMAIL PROTECTED] run]# uname -srm
Linux 2.4.20-13.7 i686
[EMAIL PROTECTED] run]# env
PWD=/tmp/zach/run
VENDOR=intel
REMOTEHOST=zaksrv2.checkpoint.com
HOSTNAME=harrison-dev.checkpoint.com
LESSOPEN=|/usr/bin/lesspipe.sh %s
USER=zak
LS_COLORS=no=00:fi=00:di=01;34:ln=01;36:pi=40;33:so=01;35:bd=40;33;01:cd=40;
33;01:or=01;05;37;41:mi=01;05;37;41:ex=01;32:*.cmd=01;32:*.exe=01;32:*.com=0
1;32:*.btm=01;32:*.bat=01;32:*.sh=01;32:*.csh=01;32:*.tar=01;31:*.tgz=01;31:
*.arj=01;31:*.taz=01;31:*.lzh=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.gz=01;
31:*.bz2=01;31:*.bz=01;31:*.tz=01;31:*.rpm=01;31:*.cpio=01;31:*.jpg=01;35:*.
gif=01;35:*.bmp=01;35:*.xbm=01;35:*.xpm=01;35:*.png=01;35:*.tif=01;35:
MACHTYPE=i386
MAIL=/var/mail/zak
LANG=en_US.UTF8
HOST=harrison-dev.checkpoint.com
SSH_CLIENT=194.29.44.79 2642 22
LOGNAME=zak
SHLVL=4
GROUP=unknown
SUPPORTED=en_US.UTF8:en_US:en:en_US.iso885915
SHELL=/bin/bash
HOSTTYPE=i386-linux
OSTYPE=linux
HOME=/root
TERM=vt100
SSH_AUTH_SOCK=/tmp/ssh-XXXHtDKX/agent.14463
PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin
SSH_TTY=/dev/pts/3
OLDPWD=/tmp/zach
_=/usr/bin/env
windows
=======================
D:\JbuilderProjects\src\general\POI-SRC\run>set
ALLUSERSPROFILE=C:\Documents and Settings\All Users
ANT_HOME=c:\apache-ant-1.6.5
APPDATA=C:\Documents and Settings\zak\Application Data
CLIENTNAME=Console
COMMONHOME=C:\Program Files\Rational\common
CommonProgramFiles=C:\Program Files\Common Files
COMPUTERNAME=ZAKSRV2
ComSpec=C:\WINDOWS\system32\cmd.exe
DISPLAY=:0.0
FORREST_HOME=c:\apache-forrest-0.7
FP_NO_HOST_CHECK=NO
HOME=C:\Documents and Settings\zak
HOMEDRIVE=C:
HOMEPATH=\Documents and Settings\zak
HSSF.testdata.path=D:\RequiredLibs\poi\source\src\testcases\org\apache\poi\h
ssf\
data\
J2EE_HOME=c:\j2sdkee1.3.1\
JAVA_HOME=C:\j2sdk1.4.2_08
JAVA_HOME_1.4.1_02=C:\j2sdk1.4.1_02
LOGONSERVER=\\HAMPSON
NUMBER_OF_PROCESSORS=1
NUTCROOT=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT
NUTSUFFIX=1
NUT_SUFFIXED_SEARCHING=1
OS=Windows_NT
Path=C:\j2sdk1.4.2_08\bin;D:\sapjcorfcLocation\;c:\apache-ant-1.6.5\bin;c:\a
pach
e-forrest-0.7\bin;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\bin;C:\PROGRA~1\RAT
IONA
L\RATION~1\NUTCROOT\bin\x11;C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\mksnt;d:\
sapd
b\programs\bin;d:\sapdb\programs\pgm;C:\ora817\bin;C:\Program
Files\Oracle\jre\1
.1.7\bin;C:\WINDOWS\system32;C:\WINDOWS;C:\WINDOWS\System32\Wbem;C:\Program
File
s\Rational\common;C:\Program Files\Rational\ClearQuest;C:\Program
Files\Rational
\Rose\TopLink\;C:\Program Files\Rational\Rational
Test;C:\JBuilder7\bin;C:\Progr
am Files\SecureCRT\;C:\PROGRA~1\ULTRAE~1
PATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.VBE;.JS;.JSE;.WSF;.WSH
PROCESSOR_ARCHITECTURE=x86
PROCESSOR_IDENTIFIER=x86 Family 15 Model 2 Stepping 4, GenuineIntel
PROCESSOR_LEVEL=15
PROCESSOR_REVISION=0204
ProgramFiles=C:\Program Files
PROMPT=$P$G
RATL_RTHOME=C:\Program Files\Rational\Rational Test
ROOTDIR=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT
RQAHOME=C:\Program Files\Rational\Rational Test\QualityArchitect
SAPJCORFC_LOCATION=D:\sapjcorfcLocation\
SESSIONNAME=Console
SHELL=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\mksnt\sh.exe
SystemDrive=C:
SystemRoot=C:\WINDOWS
TEMP=C:\DOCUME~1\zak\LOCALS~1\Temp
TERM=nutc
TERMCAP=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\etc\termcap
TERMINFO=C:\PROGRA~1\RATIONAL\RATION~1\NUTCROOT\usr\lib\terminfo
TMP=C:\DOCUME~1\zak\LOCALS~1\Temp
TMPDIR=C:\DOCUME~1\zak\LOCALS~1\Temp
USERDNSDOMAIN=AD.CHECKPOINT.COM
USERDOMAIN=AD
USERNAME=zak
USERPROFILE=C:\Documents and Settings\zak
windir=C:\WINDOWS
and here is the code for the test class (I built one for POI 1.5.1 and one
for POI 2.5.1, as there is a slight difference in the POI interfaces...
/*************************************
** start of v 2.5.1 class
*************************************/
import java.io.*;
import org.apache.poi.hssf.usermodel.*;
public class TestWorkbook_v251 {
public TestWorkbook_v251() {
}
public static void main(String[] args) {
long[] millis = new long[2];
int NUM_OF_ROWS = 10;
int NUM_OF_COLS = 10;
// get num of rows and columns from command line
if (args != null && args.length > 0) {
NUM_OF_ROWS = Integer.parseInt(args[0]);
NUM_OF_COLS = NUM_OF_ROWS;
if (args.length > 1) {
NUM_OF_COLS = Integer.parseInt(args[1]);
}
}
TestWorkbook_v251 testLargeWorkbook1 = new TestWorkbook_v251();
testLargeWorkbook1.testCreateAndWriteHSSFWorkbook(NUM_OF_ROWS,
NUM_OF_COLS);
}
public void testCreateAndWriteHSSFWorkbook(final int NUM_OF_ROWS, final
int NUM_OF_COLS) {
try {
long[] millis = new long[4];
millis[0] = System.currentTimeMillis();
HSSFWorkbook hssfwb = createHSSFWorkBook(NUM_OF_ROWS, NUM_OF_COLS);
millis[1] = System.currentTimeMillis();
millis[2] = System.currentTimeMillis();
OutputStream out = getOutputStream();
hssfwb.write(out);
millis[3] = System.currentTimeMillis();
System.out.println("create workBook millis=" + (millis[1]-millis[0]));
System.out.println("write workBook millis=" + (millis[3]-millis[2]));
System.out.println("entire test millis=" + (millis[3]-millis[0]));
}
catch (IOException ex) {
System.err.println("testLargeWorkbook1 failed");
ex.printStackTrace();
}
}
private OutputStream getOutputStream() throws IOException,
FileNotFoundException {
final String fileLocation = "testXLS.xls";
File file = new File (fileLocation);
if (file.exists() == false) {
file.createNewFile();
}
BufferedOutputStream outStream = new BufferedOutputStream(new
FileOutputStream(fileLocation));
return outStream;
}
public HSSFWorkbook createHSSFWorkBook(final int NUM_OF_ROWS, final int
NUM_OF_COLS) {
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet hssfSheet1 = wb.createSheet("sheet1");
for (int i = 0; i < NUM_OF_ROWS; i++) {
HSSFRow row = hssfSheet1.createRow(i) ;
for (int j = 0; j < NUM_OF_COLS; j++) {
HSSFCell cell = row.createCell((short)j);
cell.setCellValue("cell (" + i + "," + j + ")");
}
}
return wb;
}
}
/*************************************
** end of v 2.5.1 class
*************************************/
/*************************************
** start of v 1.5.1 class
*************************************/
import java.io.*;
import org.apache.poi.hssf.usermodel.*;
public class TestWorkbook_v151 {
public TestWorkbook_v151() {
}
public static void main(String[] args) {
long[] millis = new long[2];
int NUM_OF_ROWS = 10;
int NUM_OF_COLS = 10;
// get num of rows and columns from command line
if (args != null && args.length > 0) {
NUM_OF_ROWS = Integer.parseInt(args[0]);
NUM_OF_COLS = NUM_OF_ROWS;
if (args.length > 1) {
NUM_OF_COLS = Integer.parseInt(args[1]);
}
}
TestWorkbook_v151 testLargeWorkbook1 = new TestWorkbook_v151();
testLargeWorkbook1.testCreateAndWriteHSSFWorkbook(NUM_OF_ROWS,
NUM_OF_COLS);
}
public void testCreateAndWriteHSSFWorkbook(final int NUM_OF_ROWS, final
int NUM_OF_COLS) {
try {
long[] millis = new long[4];
millis[0] = System.currentTimeMillis();
HSSFWorkbook hssfwb = createHSSFWorkBook(NUM_OF_ROWS, NUM_OF_COLS);
millis[1] = System.currentTimeMillis();
millis[2] = System.currentTimeMillis();
OutputStream out = getOutputStream();
hssfwb.write(out);
millis[3] = System.currentTimeMillis();
System.out.println("create workBook millis=" + (millis[1]-millis[0]));
System.out.println("write workBook millis=" + (millis[3]-millis[2]));
System.out.println("entire test millis=" + (millis[3]-millis[0]));
}
catch (IOException ex) {
System.err.println("testLargeWorkbook1 failed");
ex.printStackTrace();
}
}
private OutputStream getOutputStream() throws IOException,
FileNotFoundException {
final String fileLocation = "testXLS.xls";
File file = new File (fileLocation);
if (file.exists() == false) {
file.createNewFile();
}
BufferedOutputStream outStream = new BufferedOutputStream(new
FileOutputStream(fileLocation));
return outStream;
}
public HSSFWorkbook createHSSFWorkBook(final int NUM_OF_ROWS, final int
NUM_OF_COLS) {
HSSFWorkbook wb = new HSSFWorkbook();
HSSFSheet hssfSheet1 = wb.createSheet("sheet1");
for (int i = 0; i < NUM_OF_ROWS; i++) {
HSSFRow row = hssfSheet1.createRow((short)i) ;
for (int j = 0; j < NUM_OF_COLS; j++) {
HSSFCell cell = row.createCell((short)j);
cell.setCellValue("cell (" + i + "," + j + ")");
}
}
return wb;
}
}
/*************************************
** end of v 1.5.1 class
*************************************/
-----Original Message-----
From: Marx, Stefan ext. MA Fa. Lynx
[mailto:[EMAIL PROTECTED]
Sent: Thursday, August 18, 2005 2:00 PM
To: 'POI Users List'
Subject: AW: bad performance in linux
Hi Zach,
'cause you didn't the machine data I assume that you measured
the times on the same machine.
After you investigated in the direction of file systems (blocksize,
journaling (Reiser, ext3), give your vm a try. Are you using a hot-
spot version. Andy Oliver from the core team wrote about huge
differences in the runtime behaviour of different flavours with SUNs
runtime set as most performant among other (Blackbird etc.).
May be you can tune your Linux enviroment in this area.
cu Stefan
> -----Ursprüngliche Nachricht-----
> Von: Zach Engel [SMTP:[EMAIL PROTECTED]
> Gesendet am: Mittwoch, 17. August 2005 21:08
> An: 'POI Users List'
> Betreff: bad performance in linux
>
> Hi
>
> Need some help with the following performance problem:
>
> I have some code that creates a HSSFWorkBook
> (about 400 rows of 10 cells each) and then writes it
> to an output stream (out = HttpServletResponse.getOutputStream())
>
> This takes 2 seconds on a WINDOWS machine.
> Here is some info from my log
> [17 Aug 2005 18:12:30 DEBUG] - hssf entire write millis=1908
> [17 Aug 2005 18:12:30 DEBUG] - hssf getbytes millis=497
> [17 Aug 2005 18:12:30 DEBUG] - hssf createDocument millis=713
> [17 Aug 2005 18:12:30 DEBUG] - hssf writeFilesystem millis=698
>
> When I run the same code on a linux machine (Linux 2.4.20-13.7),
> it takes 7-10 seconds.
> Here is some info from my log
> [2005-08-17 19:07:21 DEBUG] - hssf entire write millis=7051
> [2005-08-17 19:07:21 DEBUG] - hssf getbytes millis=4441
> [2005-08-17 19:07:21 DEBUG] - hssf createDocument millis=46
> [2005-08-17 19:07:21 DEBUG] - hssf writeFilesystem millis=2562
>
> After downloading the most current POI sources, and debugging a little,
> I discovered that when the getBytes() method (in HSSFWorkbook.write) tries
> to serialize the SSTRecord record, it takes about 4000 milliseconds,
> While it takes only about 47 milliseconds on windows
>
>
> In addition to that, while investigating the writeFilesystem method,
> I discovered that most of the time is spent writing block2
> (indeed shhet2 contains the 400 rows of data)
>
> From the Linux log:
> [2005-08-17 19:28:02 DEBUG] - hssf writeBlock[1] millis=0
> [2005-08-17 19:28:04 DEBUG] - hssf writeBlock[2] millis=2300
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[3] millis=210
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[4] millis=0
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[5] millis=0
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[6] millis=0
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[7] millis=0
> [2005-08-17 19:28:05 DEBUG] - hssf writeBlock[8] millis=1
>
>
> From the WINDOWS log:
> [17 Aug 2005 18:33:31 DEBUG] - hssf writeBlock[1] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[2] millis=1807
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[3] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[4] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[5] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[6] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[7] millis=0
> [17 Aug 2005 18:33:32 DEBUG] - hssf writeBlock[8] millis=0
>
>
>
>
>
>
> One last thing regarding the LANG environment variable:
> While looking for answers on the web I noticed there have been some issues
> with encoding settings.
> My LANG on the linux is set to en_US.UTF8
> I changed it to ISO-8859-1, and then ran the tomcat,
> but the results where the same.
>
> I hope you can offer some kind of explanation to the wide difference
> between
> the 2 environments.
>
> Thank you
>
> Zach Engel
> Check Point
>
>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> Mailing List: http://jakarta.apache.org/site/mail2.html#poi
> The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
Mailing List: http://jakarta.apache.org/site/mail2.html#poi
The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
Mailing List: http://jakarta.apache.org/site/mail2.html#poi
The Apache Jakarta Poi Project: http://jakarta.apache.org/poi/
