Disk space is cheap, yes, but of course there is a performance cost to read from it. More importantly, do the same record sizes apply to the in-memory representation of properties? If so, that might be an issue, since memory is a more precious commodity. If not, then doing a "pre-load" of much of the graph would help mitigate the disk space and I/O concerns.
Can you confirm whether or not the in-memory structures use a fixed-record size model also? -------- Original Message -------- Subject: Re: [Neo4j] Node creation limit From: Mattias Persson <[1][email protected]> Date: Mon, June 07, 2010 7:38 am To: Neo4j user discussions <[2][email protected]> 2010/6/7 Craig Taverner <[3][email protected]>: > Seems that the string store is not optimal for the 'common' usage of > properties for names or labels, which are typically 5 to 20 characters long, > leading to about 5x (or more) space utilization than needed. By 'names or > labels' I mean things like username, tags, categorizations, product names, > etc. These seem like very common scenarios. You're quite right about that, however: Is there a specific constrain on disk space? Normally disk space isn't a problem... it's cheap and there's usually loads of it. > > I guess there is no simple way of optimizing for this? At an app level it is > possible to pack a bunch of strings into a single property using, for > example, JSON. But that only makes sense if the usage is always to work with > them all as a unit. > > On Mon, Jun 7, 2010 at 10:32 AM, Johan Svensson <[4][email protected]>wrote: > >> Hi, >> >> These are the current record sizes in bytes that can be used to >> calculate the actual store size: >> >> nodestore: 9 >> relationshipstore: 33 >> propertystore: 25 >> stringstore: 133 >> arraystore: 133 >> >> All properties except strings and arrays will take a single >> propertystore record (25 bytes). A string or array property will use >> one record from the propertystore then as many blocks needed from the >> string/array store file (each block of 133 bytes can store 120 bytes >> of data). This means if all your strings are in 120 bytes multiples in >> size you will make very efficient use of the store file while if they >> are empty you will not make very good use of the space (exactly like a >> normal filesystem taking up space for empty files). >> >> -Johan >> >> On Fri, Jun 4, 2010 at 9:15 AM, Mattias Persson >> <[5][email protected]> wrote: >> > That formula is correct regarding nodes and relationships, yes. When >> > properties comes into play another formula would, of course, have to >> > be applied. Depending on property types and length of keys/string >> > values it is different. It could be good though with a formula/tool to >> > calculate that. >> > >> > 2010/6/4, Biren Gandhi <[6][email protected]>: >> >> In that case, what are the ways to estimate storage capacity numbers? >> Basic >> >> formula of nodes*9 + edges*33 doesn't seem like a practical one. >> >> >> >> On Wed, Jun 2, 2010 at 11:26 PM, Mattias Persson >> >> <[7][email protected]>wrote: >> >> >> >>> String properties are stored in blocks so even if you have tiny string >> >>> values each property value will occupy a full block (30 or 60 bytes, >> >>> can someone correct me here?). That's what taking most of your space >> >>> IMHO >> _______________________________________________ >> Neo4j mailing list >> [8][email protected] >> [9]https://lists.neo4j.org/mailman/listinfo/user >> > _______________________________________________ > Neo4j mailing list > [10][email protected] > [11]https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [[12][email protected]] Hacker, Neo Technology [13]www.neotechnology.com _______________________________________________ Neo4j mailing list [14][email protected] [15]https://lists.neo4j.org/mailman/listinfo/user References 1. mailto://[email protected]/ 2. mailto://[email protected]/ 3. mailto://[email protected]/ 4. mailto://[email protected]/ 5. mailto://[email protected]/ 6. mailto://[email protected]/ 7. mailto://[email protected]/ 8. mailto://[email protected]/ 9. https://lists.neo4j.org/mailman/listinfo/user 10. mailto://[email protected]/ 11. https://lists.neo4j.org/mailman/listinfo/user 12. mailto://[email protected]/ 13. http://www.neotechnology.com/ 14. mailto://[email protected]/ 15. https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list [email protected] https://lists.neo4j.org/mailman/listinfo/user
