Smalyshev added a comment. @Fceller many thanks to you for your explanations!
I certainly understand the reasons for your choices, however I'm still not sure how given our data sizes - ~16M vertices, ~100M edges, 2-3 thousands indexed fields - can be supported in current ArangoDB model. Maybe I am missing some opportunity or capability, I assume that you are using a fulltext index in your example, right? No, I used two hash indexes and two skiplist indexes, on 2961954 documents (only vertices, no edges). Of course, it is a rough test, but we'd have to have much more indexes on about order of magnitude more data, so given that the loading time right now is over a hour, is is concerning, as it means if a server goes down, the system may be out of commission for hours at least. TASK DETAIL https://phabricator.wikimedia.org/T88549 REPLY HANDLER ACTIONS Reply to comment or attach files, or !close, !claim, !unsubscribe or !assign <username>. EMAIL PREFERENCES https://phabricator.wikimedia.org/settings/panel/emailpreferences/ To: Smalyshev Cc: Fceller, JanZerebecki, Aklapper, Manybubbles, jkroll, Smalyshev, Wikidata-bugs, aude, GWicke, daniel _______________________________________________ Wikidata-bugs mailing list [email protected] https://lists.wikimedia.org/mailman/listinfo/wikidata-bugs
