Indeed, now I googled this (https://stackoverflow.com/questions/1434779/maximum-java-heap-size-of-a-32-bit-jvm-on-a-64-bit-os) and learned that 32-bit windows is only able to allocate 1.4GB to 1.6GB in practice. I was under the impression that at least 2GB should be usable. So if this is root cause of the problem it would of course be the best solution to switch to 64bit or otherwise (and besides that...) you might want to use a proper OS which doesn't restrict your usable heap size so much ;)
Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht Frankfurt am Main - Reg.-Nr.: HRB 105546 Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: Daniel Gr?zinger -----Urspr?ngliche Nachricht----- Von: basex-talk-boun...@mailman.uni-konstanz.de [mailto:basex-talk-boun...@mailman.uni-konstanz.de] Im Auftrag von Imsieke, Gerrit, le-tex Gesendet: Mittwoch, 16. August 2017 08:04 An: basex-talk@mailman.uni-konstanz.de Betreff: Re: [basex-talk] Could not reserve enough space for object heap It wasn't clear to me from the OP whether the issue of *slowness* persisted after raising max heap to 2 GB, or whether the issue of *not being able to allocate this amount of space at all* persisted. If Bram set max heap to more than 1.5 GB and immediately received the message "Could not reserve enough space ...", this could be an indication that he is running a 32-bit Java on his 64-bit machine. Bram, what is the output of 'java -version'? Mine is: java version "1.8.0_144" Java(TM) SE Runtime Environment (build 1.8.0_144-b01) Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode) If it does not contain '64-Bit', it's 32 bit, and you need to install a different binary from Oracle's site. Gerrit On 16.08.2017 07:53, Kirsten, Dirk wrote: > Hi Bram > > How did you set the java heap space? Usual way would be by using the > command line and setting e.g. > > java -Xmx2g > > However, the object which throws the error is already nearly 1,5GB in > size, so if other stuff has to be allocates as well it could simply be > that it is still not enough. Try increasing it to at least 4GB. > > Indicating the memory leak: Well, it could certainly be, but I > personally doubt it. First of all, having millions of databases is a > lot, so some performance impact is expected. And given the scale of > your databases I wouldn't call 600ms "terribly slow", you just seem to > have a lot of data. When you create a new database there is also stuff > which has to be checked (I would guess for example that BaseX checks > that the database not already exists, which runtime obviously depends > on the number of databases you already have), so I would guess the > performance is somewhat expected. Of course, it might be possible to > optimize this and try to make BaseX more performant for your given scenario. > > Cheers > Dirk > > ** > Senacor Technologies Aktiengesellschaft - Sitz: Eschborn - Amtsgericht > Frankfurt am Main - Reg.-Nr.: HRB 105546 > Vorstand: Matthias Tomann, Marcus Purzer - Aufsichtsratsvorsitzender: > Daniel Gr?zinger > > > -----Urspr?ngliche Nachricht----- > Von: basex-talk-boun...@mailman.uni-konstanz.de > [mailto:basex-talk-boun...@mailman.uni-konstanz.de] Im Auftrag von > Bram Vanroy > Gesendet: Dienstag, 15. August 2017 22:22 > An: basex-talk@mailman.uni-konstanz.de > Betreff: [basex-talk] Could not reserve enough space for object heap > > Hi all > > I'm running into an issue with many databases. I.e. one server > instance with millions of databases. When creating all of these, I > found that the more databases are included on the instance, the slower > further database generation got. For instance, I could see in the logs > that in the first < 10.000 databases the creation happened smoothly > with around 50ms per file of 1-4kB. However, when having more and more > databases for this server instance, things got very slow: for an XML > file of 1-4kB the logs show ~600ms. This is terribly slow, as you can imagine. > > At first I thought something was wrong with my hardware, but I checked > on another system and the same issues arises. Then I thought maybe > Java is doing something strange, so I figured I'd reboot and see if > that cleared some stuff up. But now when I try to launch 'basex' or > 'basexserver', I get the following message: > > Could not reserve enough space for 1433600KB object heap > > I googled the issue, and it was suggested that I added a JAVA option > to my system's variable (I'm on Windows 10 64 bit, BaseX 8.6.4) > indicating the memory it could use. I set that to 2048MB. But still > the same issue persists. > > I have contacted the list before, with issues of generating millions > of database with the same server instance, and this seems another one > related to the problem. I am no expert AT ALL, but isn't it possible > there is some sort of micro memory leak that only becomes apparent > when creating an amount of databases of this magnitude? If not, other > ideas are welcome as well. At least on how to get rid of the Java > error mentioned above. > > > Kind regards > > Bram Vanroy > -- Gerrit Imsieke Gesch?ftsf?hrer / Managing Director le-tex publishing services GmbH Weissenfelser Str. 84, 04229 Leipzig, Germany Phone +49 341 355356 110, Fax +49 341 355356 510 gerrit.imsi...@le-tex.de, http://www.le-tex.de Registergericht / Commercial Register: Amtsgericht Leipzig Registernummer / Registration Number: HRB 24930 Gesch?ftsf?hrer: Gerrit Imsieke, Svea Jelonek, Thomas Schmidt, Dr. Reinhard V?ckler
