Hi John, which kind of "transaction logging did you turn off" ?
Would you be able to share the queries you are using? each double property takes 8 bytes of storage in the property-record (which are linked in a chain, each property-record can hold up to 4 4-byte-storage properties). But arrays are optimized, esp. if you have small values in your weights it tries to use only the significant bits to encode values in an array (but I think it might only do that for integer values). Would you be able to run a test where instead of having 5-10 individual properties you just use an array with that many entries? And perhaps even better project the the floating point values to integer values in that array. I also ask our kernel engineers for other tips in this regard. HTH, Michael On Mon, Aug 8, 2016 at 6:56 PM, John Fry <[email protected]> wrote: > Hello Michael, > > the graph is used as follows: > > - ~10M nodes; ~200M relationships > - Each relationship requires multiple floating properties that can be > considered connecting strength weights. These multiple weights make up a > weight vector - upto ~20 weights per vector > - The weights on the relationship are static (or at least they rarely > change) > - The weight vector is used to compute custom (very algorithmic in > nature) costs per link to drive node-to-node traversals, expansions and to > find cost based n-shortest paths > - The costs per link are calculated in as close to real time as > possible and are always different and are never stored or written back to > the relationships in the graph > > Regards, John. > > On Monday, August 8, 2016 at 12:12:13 AM UTC-7, Michael Hunger wrote: >> >> Hi John, >> >> Do you have more details on the properties that you add as well as your >> graph model and queries? Without these details it will be hard to help. >> >> It sounds a bit as if your property heavy relationships might be nodes in >> hiding. >> >> Cheers Michael >> >> >> Von meinem iPhone gesendet >> >> Am 08.08.2016 um 06:05 schrieb John Fry <[email protected]>: >> >> Hi All, >> >> In ne04j 2.3 what / where are the limits when storing properties on >> relationships? >> >> I have a graph with about 200M relationships and for each relationship I >> want to add floating point attributes as properties. >> Here is what I am experiencing: >> >> - adding 2 properties per rel - all works fine; very good performance >> - adding 5 properties per rel - start to see exceptions/crashes - can >> be fixed by turning off transaction logging - good performance >> - adding ~7 properties per rel - performance dramatically fades (10x >> slower) - occasional exceptions/crashes >> - adding ~10 properties per real - performance stalls/stops - >> eventually will crash >> >> What is a realistic set of expectations for storing this many properties >> where the relationship store could easily exceed > 20GB? >> >> Regards and thanks for any advice, John. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Neo4j" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> For more options, visit https://groups.google.com/d/optout. >> >> -- > You received this message because you are subscribed to the Google Groups > "Neo4j" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Neo4j" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
