Excessive memory consumation in SuperServer 3.0 while establishing multiple
attachments. Memory can be not released if output of abend messages is
interrupted by Ctrl-C
------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Key: CORE-4857
URL: http://tracker.firebirdsql.org/browse/CORE-4857
Project: Firebird Core
Issue Type: Bug
Components: Engine
Affects Versions: 3.0 Beta 2
Reporter: Pavel Zotov
Attachments: fb-memory-consumation.PNG
FB version: WI-T3.0.0.31916
Config:
===
AuthClient = Legacy_Auth,Srp,Win_Sspi
AuthServer = Legacy_Auth,Srp
FileSystemCacheThreshold = 65536K
LockHashSlots = 22111
MaxUnflushedWrites = -1
MaxUnflushedWriteTime = -1
MaxUserTraceLogSize = 99999
RemoteServicePort = 3333
TempCacheLimit = 2147483647
TempDirectories = c:\temp
UserManager = Legacy_UserManager
WireCrypt = Disabled
===
NOTE: using architecture is SuperServer.
Extremely huge memory consumption was explored during developing test on Python
for core-4439, while trying to establish 2000 attachments:
{
'id': 'bugs.core_4439',
'qmid': '',
'tracker_id': 'CORE-4439',
'title': 'Raise the 1024 connections limit (FD_SETSIZE) on Windows SS/SC',
'description': '',
'min_versions': None,
'versions': [
{
'firebird_version': '3.0',
'platform': 'All',
'init_script':
"""
""",
'test_type': 'Python',
'test_script': """db_conn.close()
i = 1
while i <= 2000:
runexpr='c%i = kdb.connect(dsn=dsn,user=user_name,password=user_password)'
% i
exec runexpr
print ('Established connection, sequential N=',i)
i += 1
""",
'expected_stdout':
"""
"""
}
]
}
The same can be obtained using following java class:
=== begin ===
import java.sql.*;
import org.firebirdsql.jdbc.*;
public class MultipleAttaches {
private static final String URL = "jdbc:firebirdsql://localhost:3333/e30";
private static final String USR = "SYSDBA";
private static final String PSW = "masterke";
public static void main(String[] args) {
FirebirdConnection[] ca = new FirebirdConnection[ 2000 ];
long ms;
for(int i=0; i < ca.length; i++ ) {
try {
ms = System.currentTimeMillis();
ca[i] = (FirebirdConnection)java.sql.DriverManager.getConnection(URL,
USR, PSW);
System.out.printf( "\nEtsablished connection %4d at %6d ms", 1+i,
System.currentTimeMillis() - ms );
}
catch (Exception x) {
x.printStackTrace();
System.exit(1); --------------------------------- [ 1 ]
}
}
try { Thread.sleep(1000000); } catch (InterruptedException e) { }
}
}
=== end ===
Steps to reproduce (assuming that JRE was installed):
1) correct values of constants URL, USR and PSW to yours.
2) download jaybird-full-2.x.x.jar and save it to the c:\temp\
3) create environment variable with the name = CLASSPATH and value =
.;c:\temp\* // note about leading dot and trailing asterisk
4) compile: javac MultipleAttaches.java // output should be empty
5) running: java MultipleAttaches).
When I run this class, I get:
===
Etsablished connection 1 at 719 ms
Etsablished connection 2 at 15 ms
Etsablished connection 3 at 31 ms
Etsablished connection 4 at 0 ms
. . .
Etsablished connection 1524 at 31 ms
Etsablished connection 1525 at 31 ms
org.firebirdsql.jdbc.FBSQLException: GDS Exception. 335544430. unable to
allocate memory from o
perating system
at
org.firebirdsql.jdbc.FBDataSource.getConnection(FBDataSource.java:123)
at org.firebirdsql.jdbc.AbstractDriver.connect(AbstractDriver.java:126)
at java.sql.DriverManager.getConnection(Unknown Source)
at java.sql.DriverManager.getConnection(Unknown Source)
at MultipleAttaches.main(MultipleAttaches.java:20)
Caused by: org.firebirdsql.gds.GDSException: unable to allocate memory from
operating system
at
org.firebirdsql.gds.impl.wire.AbstractJavaGDSImpl.readStatusVector(AbstractJavaGDSImpl.java:2092)
. . .
===
There are 2 GB memory on my machine (OS = Win XP), all not-necessary
applications and services have been terminated before this test was launched.
When exception occurs, FB consumes about 1.8 Gb memory - see attached
screenshot.
firebird.log will contain:
===
CSPROG Fri Jul 03 20:29:39 2015
INET/inet_error: read errno = 10054
CSPROG Fri Jul 03 20:29:39 2015
INET/inet_error: read errno = 10054
CSPROG Fri Jul 03 20:29:39 2015
INET/inet_error: read errno = 10054
CSPROG Fri Jul 03 20:29:39 2015
Operating system call _beginthreadex failed. Error code 8
CSPROG Fri Jul 03 20:29:39 2015
INET/inet_error: read errno = 10054
. . .
===
The question is: what forces FB to consume so much memory when only new
attachments are done (and no any actions within them) ?
PS. Furthermore: if I comment statement:
System.exit(1); --------------------------------- [ 1 ]
-- and run this class again, then bulk of messages about unable to allocate
memory appear. If I interrupt this output, than FB does not release memory, and
no further attachments can be done to that service.
--
This message is automatically generated by JIRA.
-
If you think it was sent incorrectly contact one of the administrators:
http://tracker.firebirdsql.org/secure/Administrators.jspa
-
For more information on JIRA, see: http://www.atlassian.com/software/jira
------------------------------------------------------------------------------
Don't Limit Your Business. Reach for the Cloud.
GigeNET's Cloud Solutions provide you with the tools and support that
you need to offload your IT needs and focus on growing your business.
Configured For All Businesses. Start Your Cloud Today.
https://www.gigenetcloud.com/
Firebird-Devel mailing list, web interface at
https://lists.sourceforge.net/lists/listinfo/firebird-devel
