Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mark,

On 12/1/11 9:50 AM, Mark H. Wood wrote:
> On Thu, Dec 01, 2011 at 12:38:01PM +0100, Mikolaj Rydzewski wrote:
>> On the other hand, increasing java heap size is not always the
>> best option. It heavily depends on memory usage pattern in your
>> application. In general: the bigger heap, the longer GC will
>> run.
> 
> I was thinking that someone should bring this up.  When a program
> uses unexpectedly huge amounts of memory in practice, the *first*
> thing to consider is:
> 
> 1.  does it actually need that much?

+1 !!

> 2.  ...or is it leaking dynamically created objects? 3.  ...or has
> cheap allocation and garbage collection lured me into doing
> something suboptimal, like sucking down an entire database table
> into an array or list and then walking it sequentially, when I
> could have used an iterator and let the DBMS code work out 
> near-optimal buffering?
> 
> IOW "is my problem fundamentally this big, or is something else
> going on?"

The 2 times our production servers have suffered OOMEs, it's been
because we were running with fairly small, (intentionally) restricted
heaps (64MiB at first, then 192MiB) and our traffic simply increased
beyond our heap size: we had a legitimate reason to increase the heap
size (and plenty of physical RAM available to do it).

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7YLEwACgkQ9CaO5/Lv0PCGTQCfSwBVBLSKIW2OMjYZWVobxrKY
JzkAoJQmi4JK2CHqo23DCuMRGE5Fzq/0
=Qte1
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Mikolaj,

On 12/1/11 6:38 AM, Mikolaj Rydzewski wrote:
> On Thu, 1 Dec 2011 12:29:14 +0100, Casper Wandahl Schmidt wrote:
> 
>> That didn't quite help me understand, because how can the OS map
>> from ie. 0-4GB to 4-8GB (the window is moved) when it can only
>> use a 32bit register to tell the machine where to look in the
>> psysical memory, that is where my knowledge ends :) So I read
>> about PAE and found out that it uses 2 registers (36 bits due to
>> some bits being used as flags) and that makes good sense, but how
>> can the cpu calculate an address without overflow and send a 
>> command to the bus containing a 36bit address (or whatever
>> fetches the bits from RAM)? That is where I'm puzzled but I guess
>> it is because I'm not at all into ISA-level and below :)
> 
> Well, it's rather out of the scope of this list.
> 
> On the other hand, increasing java heap size is not always the
> best option. It heavily depends on memory usage pattern in your
> application. In general: the bigger heap, the longer GC will run.

That's a rather sweeping generalization. The heap size doesn't matter
directly.. it's the number of objects being managed within that heap
that matters. Of course, with a larger heap, you can fit more objects
into it before a major collection is required. Generational heap
strategies are fairly efficient, and performance depends upon the
number of LIVE objects, not just the total number of objects. Oddly
enough, most "garbage collection" is really collecting non-garbage and
ignoring the actual garbage.

It's a bit like moving to a different house when yours gets too
cluttered: you just take the things you want to keep and leave
everything else behind.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7YK88ACgkQ9CaO5/Lv0PDzdACgmYJEuWHFNkFyEVWRcucJo4Yu
6uwAoK2JWcjX0SRY6PPIWwd1m7Fhx+f8
=XY04
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Casper,

On 12/1/11 3:39 AM, Casper Wandahl Schmidt wrote:
> Aha so I learned something new today :) I'm still puzzled as to
> how a 32 bit CPU can compute and fetch a memory cell with address
> above 4GB since it cannot hold this large value.

OS != CPU

Also, OS != process

While the chips and OSs are officially 32-bit, both are able to handle
integers that don't fit into 32-bit registers in various ways. Usually,
CPUs have registers that are larger than their architecture would
suggest, and uses them even to perform computations on 32-bit data.

The real issue here is that in a 32-bit environment, word-sized
pointers are 32-bits and therefore an individual process gets a 4GiB
maximum process space, which can be mapped-into a much larger space by
the kernel, and even by the underlying hardware if it's in on the deal.

> Anyway that is just too much low-level computer science for me, all
> I ever had was a seven week course on architecture and networking
> (a single week out of the seven) :)

It never hurts to learn more. Unless your brain is full. Then it
*really* hurts.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7YKp0ACgkQ9CaO5/Lv0PDpDgCgwNXVZ1k43CrOFDjcDryl3JTw
dSkAoK5XWk47MjE+fbsNnOS3CbGBdjxb
=nuE/
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Mark H. Wood
On Thu, Dec 01, 2011 at 12:38:01PM +0100, Mikolaj Rydzewski wrote:
>  On the other hand, increasing java heap size is not always the best 
>  option. It heavily depends on memory usage pattern in your application. 
>  In general: the bigger heap, the longer GC will run.

I was thinking that someone should bring this up.  When a program uses
unexpectedly huge amounts of memory in practice, the *first* thing to
consider is:

1.  does it actually need that much?
2.  ...or is it leaking dynamically created objects?
3.  ...or has cheap allocation and garbage collection lured me into
doing something suboptimal, like sucking down an entire database
table into an array or list and then walking it sequentially, when
I could have used an iterator and let the DBMS code work out
near-optimal buffering?

IOW "is my problem fundamentally this big, or is something else going on?"

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Asking whether markets are efficient is like asking whether people are smart.


pgpIowkiM39ep.pgp
Description: PGP signature


Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Mark H. Wood
The OS has little to do with the calculation.  The CPU hardware is
doing it.  The processor's address logic uses registers which are
wider than 32 bits.  Just as you can add a 1-digit number to a 3-digit
number and get a 3-digit result, the widget that maps a process'
virtual address space to the hardware's physical address space can add
the content of a 32-bit register to the content of a 36-bit register
and get a 36-bit result.

(I'm ignoring the possibility of overflow, like adding 1 to 999 in
a 3-digit field.  With good management they can be avoided.)

Only a tiny bit of the OS kernel, and nothing in any process, needs to
know about physical memory.  The hardware is set up by that bit and
makes processes, and the rest of the kernel, think they each live in a
block of memory that starts at 0 and ends at, say, 3GB.  In physical
memory they live side-by-side (to oversimplify a bit).

For how it does that, track down a little story called The Paging Game.

-- 
Mark H. Wood, Lead System Programmer   mw...@iupui.edu
Asking whether markets are efficient is like asking whether people are smart.


pgpjKSIiRF27q.pgp
Description: PGP signature


RE: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Caldarale, Charles R
> From: André Warnier [mailto:a...@ice-sa.com] 
> Subject: Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows 
> platform

> > In general: the bigger heap, the longer GC will run.

Not strictly true, and hasn't been true for many years.  GC time is 
proportional to the number of live (reachable) objects, not the size of the 
heap.  If the app is making heavy use of weak references, this may allow more 
live objects to persist in a larger heap until GC gets fed up with the mess and 
throws them all away. 

> Why do I feel that a comment from Chuck is going to follow 
> that one later on ?

Just had to wake up first.

 - Chuck


THIS COMMUNICATION MAY CONTAIN CONFIDENTIAL AND/OR OTHERWISE PROPRIETARY 
MATERIAL and is thus for use only by the intended recipient. If you received 
this in error, please contact the sender and delete the e-mail and its 
attachments from all computers.



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread André Warnier

Mikolaj Rydzewski wrote:

On Thu, 1 Dec 2011 12:29:14 +0100, Casper Wandahl Schmidt wrote:


That didn't quite help me understand, because how can the OS map from ie.
0-4GB to 4-8GB (the window is moved) when it can only use a 32bit 
register
to tell the machine where to look in the psysical memory, that is 
where my
knowledge ends :) So I read about PAE and found out that it uses 2 
registers

(36 bits due to some bits being used as flags) and that makes good sense,
but how can the cpu calculate an address without overflow and send a 
command

to the bus containing a 36bit address (or whatever fetches the bits from
RAM)? That is where I'm puzzled but I guess it is because I'm not at all
into ISA-level and below :)


Well, it's rather out of the scope of this list.

On the other hand, increasing java heap size is not always the best 
option. It heavily depends on memory usage pattern in your application. 
In general: the bigger heap, the longer GC will run.




Why do I feel that a comment from Chuck is going to follow that one later on ?
;-)


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Mikolaj Rydzewski

On Thu, 1 Dec 2011 12:29:14 +0100, Casper Wandahl Schmidt wrote:

That didn't quite help me understand, because how can the OS map from 
ie.
0-4GB to 4-8GB (the window is moved) when it can only use a 32bit 
register
to tell the machine where to look in the psysical memory, that is 
where my
knowledge ends :) So I read about PAE and found out that it uses 2 
registers
(36 bits due to some bits being used as flags) and that makes good 
sense,
but how can the cpu calculate an address without overflow and send a 
command
to the bus containing a 36bit address (or whatever fetches the bits 
from
RAM)? That is where I'm puzzled but I guess it is because I'm not at 
all

into ISA-level and below :)


Well, it's rather out of the scope of this list.

On the other hand, increasing java heap size is not always the best 
option. It heavily depends on memory usage pattern in your application. 
In general: the bigger heap, the longer GC will run.


--
Mikolaj Rydzewski 

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Casper Wandahl Schmidt


-Original Message-
From: Francis GALIEGUE [mailto:f...@one2team.com] 
Sent: 1. december 2011 12:33
To: Tomcat Users List
Subject: Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows 
platform

On Thu, Dec 1, 2011 at 12:29, Casper Wandahl Schmidt  
wrote:
[...]
>
> That didn't quite help me understand, because how can the OS map from ie.
> 0-4GB to 4-8GB (the window is moved) when it can only use a 32bit 
> register to tell the machine where to look in the psysical memory, 
> that is where my knowledge ends :) So I read about PAE and found out 
> that it uses 2 registers
> (36 bits due to some bits being used as flags) and that makes good 
> sense, but how can the cpu calculate an address without overflow and 
> send a command to the bus containing a 36bit address (or whatever 
> fetches the bits from RAM)? That is where I'm puzzled but I guess it 
> is because I'm not at all into ISA-level and below :)
>

It is the role of the MMU to do that. At any one time, it can map a "virtual", 
32-bit wide, address to a "real", 36-bit wide address. It uses TLBs 
(Translation Lookaside Buffers) for that, and it is the OS'
role to have the correct TLB in place at any time.

Nice to know :) That explained it all :)

-Casper

--
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Francis GALIEGUE
On Thu, Dec 1, 2011 at 12:29, Casper Wandahl Schmidt
 wrote:
[...]
>
> That didn't quite help me understand, because how can the OS map from ie.
> 0-4GB to 4-8GB (the window is moved) when it can only use a 32bit register
> to tell the machine where to look in the psysical memory, that is where my
> knowledge ends :) So I read about PAE and found out that it uses 2 registers
> (36 bits due to some bits being used as flags) and that makes good sense,
> but how can the cpu calculate an address without overflow and send a command
> to the bus containing a 36bit address (or whatever fetches the bits from
> RAM)? That is where I'm puzzled but I guess it is because I'm not at all
> into ISA-level and below :)
>

It is the role of the MMU to do that. At any one time, it can map a
"virtual", 32-bit wide, address to a "real", 36-bit wide address. It
uses TLBs (Translation Lookaside Buffers) for that, and it is the OS'
role to have the correct TLB in place at any time.

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Casper Wandahl Schmidt


-Original Message-
From: Ronald Klop (Mailing List) [mailto:ronald-mailingl...@base.nl] 
Sent: 1. december 2011 12:06
To: Tomcat Users List
Subject: Re: [OT]RE: Maximum memory that can be assigned to Tomcat on
windows platform




Op donderdag, 1 december 2011 09:39 schreef Casper Wandahl Schmidt
:
> 
>   
>  See below. I hope MS Outlook does some decent indend so my response 
> is clear -.-
>  
>  -Original Message-
>  From: Christopher Schultz [mailto:ch...@christopherschultz.net]
>  Sent: 30. november 2011 18:51
>  To: Tomcat Users List
>  Subject: Re: Maximum memory that can be assigned to Tomcat on windows 
> platform
>  
>  -BEGIN PGP SIGNED MESSAGE-
>  Hash: SHA1
>  
>  Casper,
>  
>  On 11/30/11 3:37 AM, Casper Wandahl Schmidt wrote:
>  > Another question to ask is, why do you have 8GB memory when running  
> > 32bit? That is just stupid since 32bit cannot address more than 4GB 
> of  > memory no matter what you do. Any sysadmin should know that right?
>  
>  That's per process. All reasonably recent 32-bit OSs can address way more
than 4GiB internally.
>  
>  For example:
>  
> http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=v
> s.85%29.aspx#memory_limits
>  
>  This is generally done through PAE
>  (http://en.wikipedia.org/wiki/Physical_Address_Extension) which allows
32-bit OSs to access more than 4GiB at the kernel level, though each process
is still limited to 4GiB.
>  
>  Aha so I learned something new today :) I'm still puzzled as to how a 
> 32 bit CPU can compute and fetch a memory cell with address above 4GB 
> since it cannot hold this large value. Anyway that is just too much 
> low-level computer science for me, all I ever had was a seven week 
> course on architecture and networking (a single week out of the seven) 
> :)
>  
>  -Casper
>  
>  Running a machine with more than 4GiB in 32-bit mode isn't stupid at all
IMO. If you have relatively small processes, there's no need for the
overhead of 64-bit even if you have 16GiB or more.
>  
>  - -chris
>   
>  
> 
> 
>   
 I have an analogy for you.
 If you look out of your window you only see a small part of the world. If
you move your window you wil see another part of the world.
 This is what the OS does with PAE. It moves the window on your RAM
frequently. That is why a 32 bits application only sees max. 4GB. That is
the size of its window.

 Ronald.

That didn't quite help me understand, because how can the OS map from ie.
0-4GB to 4-8GB (the window is moved) when it can only use a 32bit register
to tell the machine where to look in the psysical memory, that is where my
knowledge ends :) So I read about PAE and found out that it uses 2 registers
(36 bits due to some bits being used as flags) and that makes good sense,
but how can the cpu calculate an address without overflow and send a command
to the bus containing a 36bit address (or whatever fetches the bits from
RAM)? That is where I'm puzzled but I guess it is because I'm not at all
into ISA-level and below :)

-Casper


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Ronald Klop (Mailing List)




Op donderdag, 1 december 2011 09:39 schreef Casper Wandahl Schmidt 
:


  
 See below. I hope MS Outlook does some decent indend so my response is clear -.-
 
 -Original Message-

 From: Christopher Schultz [mailto:ch...@christopherschultz.net]
 Sent: 30. november 2011 18:51
 To: Tomcat Users List
 Subject: Re: Maximum memory that can be assigned to Tomcat on windows platform
 
 -BEGIN PGP SIGNED MESSAGE-

 Hash: SHA1
 
 Casper,
 
 On 11/30/11 3:37 AM, Casper Wandahl Schmidt wrote:

 > Another question to ask is, why do you have 8GB memory when running
 > 32bit? That is just stupid since 32bit cannot address more than 4GB of
 > memory no matter what you do. Any sysadmin should know that right?
 
 That's per process. All reasonably recent 32-bit OSs can address way more than 4GiB internally.
 
 For example:

 
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=vs.85%29.aspx#memory_limits
 
 This is generally done through PAE

 (http://en.wikipedia.org/wiki/Physical_Address_Extension) which allows 32-bit 
OSs to access more than 4GiB at the kernel level, though each process is still 
limited to 4GiB.
 
 Aha so I learned something new today :) I'm still puzzled as to how a 32 bit CPU can compute and fetch a memory cell with address above 4GB since it cannot hold this large value. Anyway that is just too much low-level computer science for me, all I ever had was a seven week course on architecture and networking (a single week out of the seven) :)
 
 -Casper
 
 Running a machine with more than 4GiB in 32-bit mode isn't stupid at all IMO. If you have relatively small processes, there's no need for the overhead of 64-bit even if you have 16GiB or more.
 
 - -chris
  
 



  

I have an analogy for you.
If you look out of your window you only see a small part of the world. If you 
move your window you wil see another part of the world.
This is what the OS does with PAE. It moves the window on your RAM frequently. 
That is why a 32 bits application only sees max. 4GB. That is the size of its 
window.

Ronald.

[OT]RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-12-01 Thread Casper Wandahl Schmidt
See below. I hope MS Outlook does some decent indend so my response is clear -.-

-Original Message-
From: Christopher Schultz [mailto:ch...@christopherschultz.net] 
Sent: 30. november 2011 18:51
To: Tomcat Users List
Subject: Re: Maximum memory that can be assigned to Tomcat on windows platform

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Casper,

On 11/30/11 3:37 AM, Casper Wandahl Schmidt wrote:
> Another question to ask is, why do you have 8GB memory when running 
> 32bit? That is just stupid since 32bit cannot address more than 4GB of 
> memory no matter what you do. Any sysadmin should know that right?

That's per process. All reasonably recent 32-bit OSs can address way more than 
4GiB internally.

For example:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=vs.85%29.aspx#memory_limits

This is generally done through PAE
(http://en.wikipedia.org/wiki/Physical_Address_Extension) which allows 32-bit 
OSs to access more than 4GiB at the kernel level, though each process is still 
limited to 4GiB.

Aha so I learned something new today :) I'm still puzzled as to how a 32 bit 
CPU can compute and fetch a memory cell with address above 4GB since it cannot 
hold this large value. Anyway that is just too much low-level computer science 
for me, all I ever had was a seven week course on architecture and networking 
(a single week out of the seven) :)

-Casper

Running a machine with more than 4GiB in 32-bit mode isn't stupid at all IMO. 
If you have relatively small processes, there's no need for the overhead of 
64-bit even if you have 16GiB or more.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7WbQYACgkQ9CaO5/Lv0PBsWwCgnifhHtqrLUBi7K4PeDjp4hnC
JMkAn0gilsNy2hv3zu3nzUkrmrzxoYWF
=AZpI
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT] Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Francis GALIEGUE
On Wed, Nov 30, 2011 at 22:38, André Warnier  wrote:
[...]
>
> I am not knowledgeable at all in such questions, and while you are at it let
> me ask a question :
> Does the fact of having a system with a 64-bit CPU (and OS) necessarily (or
> usually) imply that data transfers between CPU and RAM happen also 64-bit in
> parallel ?
>

No, this depends on the bus width, not the address space. And bus
widths nowadays are rather in the range of 256/512 bits.

> For example, let' suppose that I have an application which handles a very
> large table of integers (the integers themselves being < 2exp31), and that
> periodically all elements of this table have to be updated.
> My understanding is that under a 32-bit platform, each integer will occupy
> 32 bit, while on a 64-bit platform each will occupy 64 bit (the source
> program remaining the same).
>
> If there is not a corresponding 64-bit parallel transfer between CPU and
> memory, then using a 64-bit CPU would be detrimental, no ?
> Or is this a non-sensical case nowadays ?
>

This depends on the alignment the JVM uses. If it aligns to 8 bytes,
then yes, any integer will take up 8 bytes, but so will any byte. In
general, the alignment is 4 bytes, except for long values of course.

With a 4 byte alignment, this means you can store two ints in a
register, and it's then only a matter of logical ands/shifting to
obtain the value you want. And these operations are ultra fast ;)

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: [OT] Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Ilya Kazakevich
>>Does the fact of having a system with a 64-bit CPU (and OS) necessarily
(or usually) imply 
>>that data transfers between CPU and RAM happen also 64-bit in parallel ?
That depends on front bus width. Modern PCs has 64-bit bus AFAIK.

In "64-bit CPU" 64 is register size and nothing else (although 86x64 mode
has some differences with IA32)

>>My understanding is that under a 32-bit platform, each integer will occupy
32 bit, while 
>>on a 64-bit platform each will occupy 64 bit (the source program remaining
the same).
Java integers are always 32bit, are not they?

C int may vary from 16 (in BCC 3.1 for DOS) to 64. 

64-bit processor mode gives you ability to:
1) address more memory (2^64 bytes theoretically).
2) store more data in register so CPU could now operate 64bit numbers and
you need one operation to do something with two 64bit objects opposite to
two operations on 32bit.
3) v86 mode is disabled so you can't run 16bit apps :)
4) use new registers, new SSE instructions and so on:
http://en.wikipedia.org/wiki/X86-64#Architectural_features



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: [OT] Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread André Warnier

Francis GALIEGUE wrote:

On Wed, Nov 30, 2011 at 18:51, Christopher Schultz
 wrote:
[...]

Running a machine with more than 4GiB in 32-bit mode isn't stupid at
all IMO. If you have relatively small processes, there's no need for
the overhead of 64-bit even if you have 16GiB or more.



This is quite the opposite: a 32bit OS has _more_ overhead than a
64bit OS when running on a 64bit CPU.

Just consider filesystem calls such as read(), write(), seek(): their
arguments are 64bits, not 32, and have been so before 64bit was even
common (2+ GB files are common place today). With a 32bit OS on a
64bit CPU, you eat up two registers whereas a 64bit OS will take only
one.

And of course, this is without considering wasted TLB space or the
sheer time to address just one memory page.



I am not knowledgeable at all in such questions, and while you are at it let me ask a 
question :
Does the fact of having a system with a 64-bit CPU (and OS) necessarily (or usually) imply 
that data transfers between CPU and RAM happen also 64-bit in parallel ?


For example, let' suppose that I have an application which handles a very large table of 
integers (the integers themselves being < 2exp31), and that periodically all elements of 
this table have to be updated.
My understanding is that under a 32-bit platform, each integer will occupy 32 bit, while 
on a 64-bit platform each will occupy 64 bit (the source program remaining the same).
If there is not a corresponding 64-bit parallel transfer between CPU and memory, then 
using a 64-bit CPU would be detrimental, no ?

Or is this a non-sensical case nowadays ?



-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Stefan Mayr

Am 30.11.2011 12:08, schrieb j...@gniffelnieuws.net:

On Wed, 30 Nov 2011 16:14:45 +0530, Choudhury wrote

Hello ,
 The question is not why I would use 32 bit JVM , the
question is whether there is any maximum limit on memory for Tomcat
and if yes why ? Regards,


The limit is the JVM, not Tomcat itself


It depends on the windows version used actually. From what I remember this is
limited to maximum 2GB for user processes. The easiest way to test is, is with
a small app that shows some memory information like:
public class MaxMemory {
 public static void main(String[] args) {
 Runtime rt = Runtime.getRuntime();
 long totalMem = rt.totalMemory();
 long maxMem = rt.maxMemory();
 long freeMem = rt.freeMemory();
 double megs = 1048576.0;

 System.out.println ("Total Memory: " + totalMem + " (" +
(totalMem/megs) + " MiB)");
 System.out.println ("Max Memory:   " + maxMem + " (" + (maxMem/megs) +
" MiB)");
 System.out.println ("Free Memory:  " + freeMem + " (" + (freeMem/megs)
+ " MiB)");
 }
}

Try to run it with different options for the heap. I would be surprised if you
would get over 1600MB.


This is what we experienced: up to 1GB Heap (Xmx) was stable on all servers.
Heaps larger were very prone to crashes. We've seen a maximum of 1.2GB 
Heap on Windows with 32bit JVMs


That's why we have switched to 64bit JVMs on all machines. No need to 
swap JVM editions when applications require more memory - just increase 
Xmx as long as RAM is available.


  Stefan

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



[OT] Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Francis GALIEGUE
On Wed, Nov 30, 2011 at 18:51, Christopher Schultz
 wrote:
[...]
>
> Running a machine with more than 4GiB in 32-bit mode isn't stupid at
> all IMO. If you have relatively small processes, there's no need for
> the overhead of 64-bit even if you have 16GiB or more.
>

This is quite the opposite: a 32bit OS has _more_ overhead than a
64bit OS when running on a 64bit CPU.

Just consider filesystem calls such as read(), write(), seek(): their
arguments are 64bits, not 32, and have been so before 64bit was even
common (2+ GB files are common place today). With a 32bit OS on a
64bit CPU, you eat up two registers whereas a 64bit OS will take only
one.

And of course, this is without considering wasted TLB space or the
sheer time to address just one memory page.

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



RE: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Ilya Kazakevich
>> That is just stupid since 32bit cannot address more
>> than 4GB of memory no matter what you do. Any sysadmin should know
>> that right?

>That's per process. All reasonably recent 32-bit OSs can address way
>more than 4GiB internally.

Yes, but one region in memory is reserved for IO. Drivers use high memory
addreses (>3Gb) to speak with their devices.
To support more than 3Gb on Win/32bit PAE could be used so IO would reside
higher than 4Gb. 
But in this situation drivers need to deal with 64bit pointers. 
Microsoft believes it would lead to driver fails and BSODs, so it denied it
for desktop versions of OSes. But it works in server versions.
Idea was: "4Gb is rarely needed on desktop and desktop has cheap hardware
and poorly-written drivers.  And on server they have good hardware and good
drivers and often require more than 4Gb."

You could use +4Gb on Win2008/32bit but you _can't_ use more than about
3.5Gb on Win7@32bit. 


BTW process could access more than 4Gb too (see AWE:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366527(v=vs.85).as
px) but AFAIK Sun JVM does not use it

>If you have relatively small processes, there's no need for
>the overhead of 64-bit even if you have 16GiB or more.

You can run 32bit apps on 64bit windows (that's called WoW64). Processor and
Windows support it and it works pretty well.

Ilya Kazakevich,
Developer
JetBrains Inc
http://www.jetbrains.com
"Develop with pleasure!"


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Casper,

On 11/30/11 3:37 AM, Casper Wandahl Schmidt wrote:
> Another question to ask is, why do you have 8GB memory when
> running 32bit? That is just stupid since 32bit cannot address more
> than 4GB of memory no matter what you do. Any sysadmin should know
> that right?

That's per process. All reasonably recent 32-bit OSs can address way
more than 4GiB internally.

For example:
http://msdn.microsoft.com/en-us/library/windows/desktop/aa366778%28v=vs.85%29.aspx#memory_limits

This is generally done through PAE
(http://en.wikipedia.org/wiki/Physical_Address_Extension) which allows
32-bit OSs to access more than 4GiB at the kernel level, though each
process is still limited to 4GiB.

Running a machine with more than 4GiB in 32-bit mode isn't stupid at
all IMO. If you have relatively small processes, there's no need for
the overhead of 64-bit even if you have 16GiB or more.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7WbQYACgkQ9CaO5/Lv0PBsWwCgnifhHtqrLUBi7K4PeDjp4hnC
JMkAn0gilsNy2hv3zu3nzUkrmrzxoYWF
=AZpI
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Christopher Schultz
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Subhrajyoti,

On 11/30/11 1:28 AM, choudh...@labware.com wrote:
> Is there any cap on maximum memory that can be assigned to tomcat 
> on 32 bit Windows machines?

See this week's thread called "Server crash for memory limit" for some
information about that.

- -chris
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.17 (Darwin)
Comment: GPGTools - http://gpgtools.org
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7Wak0ACgkQ9CaO5/Lv0PAOzACgoFatvhsCE8sMoDnWFAKOFqpp
5IsAn2MvLYj5kszAFXfaZstjdQaQ/8sU
=hvCT
-END PGP SIGNATURE-

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Pid *
On 30 Nov 2011, at 10:45, "choudh...@labware.com"  wrote:

> Hello ,
>The question is not why I would use 32 bit JVM , the question is
> whether there is any maximum limit on memory for Tomcat and if yes why

No. Tomcat has no knowledge of the memory capacity of the JVM.


p


> ?
> Regards,
>
> Subhrajyoti
> Mobile: +919830079545
> Mail: choudh...@labware.com
> Web: www.labware.com
>
> LabWare LIMS Solutions - Results Count
>
>
>
> From:   Francis GALIEGUE 
> To: Tomcat Users List 
> Date:   11/30/2011 01:51 PM
> Subject:    Re: Maximum memory that can be assigned to Tomcat on
> windows platform
>
>
>
> On Wed, Nov 30, 2011 at 07:28,   wrote:
>> Hi ,
>>Is there any cap on maximum memory that can be assigned to
> tomcat
>> on 32 bit Windows machines ? I have found out that even if we have 8GB
>> memory in the server , can not assign more 1.4/1.5 GB to Tomcat . I also
>> found this thread which reinforces what I have been seeing ? We are
> using
>> Tomcat 7.0.19 .
>>
>> www.theserverside.com/discussions/thread.tss?thread_id=48793
>>
>
> In any event, a 32bit JVM (1.6, I don't know for lower VMs) cannot
> allocate more than 3 GB heap space.
>
> "With so much RAM, why do you still use a 32bit OS and JVM" is the
> question here.
>
> --
> Francis Galiegue
> ONE2TEAM
> Ingénieur système
> Mob : +33 (0) 683 877 875
> Tel : +33 (0) 178 945 552
> f...@one2team.com
> 40 avenue Raymond Poincaré
> 75116 Paris
>
> -
> To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
> For additional commands, e-mail: users-h...@tomcat.apache.org
>
>

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread john
On Wed, 30 Nov 2011 16:14:45 +0530, Choudhury wrote
> Hello , 
> The question is not why I would use 32 bit JVM , the 
> question is whether there is any maximum limit on memory for Tomcat 
> and if yes why ? Regards,
>
It depends on the windows version used actually. From what I remember this is
limited to maximum 2GB for user processes. The easiest way to test is, is with
a small app that shows some memory information like:
public class MaxMemory {
public static void main(String[] args) {
Runtime rt = Runtime.getRuntime();
long totalMem = rt.totalMemory();
long maxMem = rt.maxMemory();
long freeMem = rt.freeMemory();
double megs = 1048576.0;

System.out.println ("Total Memory: " + totalMem + " (" +
(totalMem/megs) + " MiB)");
System.out.println ("Max Memory:   " + maxMem + " (" + (maxMem/megs) +
" MiB)");
System.out.println ("Free Memory:  " + freeMem + " (" + (freeMem/megs)
+ " MiB)");
}
}

Try to run it with different options for the heap. I would be surprised if you
would get over 1600MB.  
 
--
/(bb|[^b]{2})/


-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Francis GALIEGUE
On Wed, Nov 30, 2011 at 11:44,   wrote:
> Hello ,
>        The question is not why I would use 32 bit JVM , the question is
> whether there is any maximum limit on memory for Tomcat and if yes why ?


The question _is_ why you use a 32bit OS and JVM with 8 GB RAM. Tomcat
is not limited in memory usage by itself, it is the JVM which limits
what it can consume.

In the best of cases (32bit Linux and 32bit Sun 1.6 JVM), you are
limited to a 3 GB heap size.

Use a 64bit OS and JVM.

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Choudhury
Hello , 
The question is not why I would use 32 bit JVM , the question is 
whether there is any maximum limit on memory for Tomcat and if yes why ?
Regards,

Subhrajyoti 
Mobile: +919830079545
Mail: choudh...@labware.com
Web: www.labware.com

LabWare LIMS Solutions - Results Count



From:   Francis GALIEGUE 
To: Tomcat Users List 
Date:   11/30/2011 01:51 PM
Subject:Re: Maximum memory that can be assigned to Tomcat on 
windows platform



On Wed, Nov 30, 2011 at 07:28,   wrote:
> Hi ,
> Is there any cap on maximum memory that can be assigned to 
tomcat
> on 32 bit Windows machines ? I have found out that even if we have 8GB
> memory in the server , can not assign more 1.4/1.5 GB to Tomcat . I also
> found this thread which reinforces what I have been seeing ? We are 
using
> Tomcat 7.0.19 .
>
> www.theserverside.com/discussions/thread.tss?thread_id=48793
>

In any event, a 32bit JVM (1.6, I don't know for lower VMs) cannot
allocate more than 3 GB heap space.

"With so much RAM, why do you still use a 32bit OS and JVM" is the
question here.

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org




Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Francis GALIEGUE
On Wed, Nov 30, 2011 at 09:37, Casper Wandahl Schmidt
 wrote:
>
[...]
>>
> Another question to ask is, why do you have 8GB memory when running 32bit?
> That is just stupid since 32bit cannot address more than 4GB of memory no
> matter what you do. Any sysadmin should know that right?
>
> disclaimer: I'm not a sysadmin, if I'm wrong please teach me :)
>

With PAE[1] you actually can address up to 2^36 RAM (therefore 64 GB).
But not all OSes support that, and anyway the addressable memory at
one time (and therefore max memory theoretically accessible to one
process at any given moment) is still 2^32.

[1] Physical Address Extension

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Casper Wandahl Schmidt



Den 30-11-2011 09:21, Francis GALIEGUE skrev:

On Wed, Nov 30, 2011 at 07:28,  wrote:

Hi ,
 Is there any cap on maximum memory that can be assigned to tomcat
on 32 bit Windows machines ? I have found out that even if we have 8GB
memory in the server , can not assign more 1.4/1.5 GB to Tomcat . I also
found this thread which reinforces what I have been seeing ? We are using
Tomcat 7.0.19 .

www.theserverside.com/discussions/thread.tss?thread_id=48793


In any event, a 32bit JVM (1.6, I don't know for lower VMs) cannot
allocate more than 3 GB heap space.

"With so much RAM, why do you still use a 32bit OS and JVM" is the
question here.

Another question to ask is, why do you have 8GB memory when running 
32bit? That is just stupid since 32bit cannot address more than 4GB of 
memory no matter what you do. Any sysadmin should know that right?


disclaimer: I'm not a sysadmin, if I'm wrong please teach me :)

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Re: Maximum memory that can be assigned to Tomcat on windows platform

2011-11-30 Thread Francis GALIEGUE
On Wed, Nov 30, 2011 at 07:28,   wrote:
> Hi ,
>         Is there any cap on maximum memory that can be assigned to tomcat
> on 32 bit Windows machines ? I have found out that even if we have 8GB
> memory in the server , can not assign more 1.4/1.5 GB to Tomcat . I also
> found this thread which reinforces what I have been seeing ? We are using
> Tomcat 7.0.19 .
>
> www.theserverside.com/discussions/thread.tss?thread_id=48793
>

In any event, a 32bit JVM (1.6, I don't know for lower VMs) cannot
allocate more than 3 GB heap space.

"With so much RAM, why do you still use a 32bit OS and JVM" is the
question here.

-- 
Francis Galiegue
ONE2TEAM
Ingénieur système
Mob : +33 (0) 683 877 875
Tel : +33 (0) 178 945 552
f...@one2team.com
40 avenue Raymond Poincaré
75116 Paris

-
To unsubscribe, e-mail: users-unsubscr...@tomcat.apache.org
For additional commands, e-mail: users-h...@tomcat.apache.org



Maximum memory that can be assigned to Tomcat on windows platform

2011-11-29 Thread Choudhury
Hi , 
 Is there any cap on maximum memory that can be assigned to tomcat 
on 32 bit Windows machines ? I have found out that even if we have 8GB 
memory in the server , can not assign more 1.4/1.5 GB to Tomcat . I also 
found this thread which reinforces what I have been seeing ? We are using 
Tomcat 7.0.19 . 

www.theserverside.com/discussions/thread.tss?thread_id=48793

Regards,

Subhrajyoti  Choudhury 
WebLIMS Developer 
Mobile: +919830079545
Mail: choudh...@labware.com
Web: www.labware.com

LabWare LIMS Solutions - Results Count