I think this 'schema mode' is a good addition. Robert Dale
On Sun, Jan 14, 2018 at 3:41 AM, Michael Pollmeier < [email protected]> wrote: > First of all: if you're not manually enabling the memory-efficient mode, > you get the exact same behaviour as now. > > With the memory-efficient mode enabled, a few semantics change (by design): > * setting a property that is not schema-conform may (IMO *should*, but > that depends on the implementation) throw an error > * generic properties (and meta properties) change conceptually, but you > can still have them > > There's no changes to the underlying queries necessary. We didn't have > to adapt ours, anyway. > > On 13/01/18 11:34, Robert Dale wrote: > > Michael, are there any limitations other considerations? That is, are > there > > any graph features that are lost or schema doesn't apply to, e.g. > > meta-properties? > > > > Robert Dale > > > > On Thu, Jan 11, 2018 at 4:03 PM, Michael Pollmeier < > > [email protected]> wrote: > > > >> Thanks Stephen. Just adding a few notes: > >> > >> * As mentioned before I'm happy to provide a PR to the Apache > >> Tinkergraph and thereby close down our fork. If others prefer to not > >> dilute the waters, I'll equally happily maintain the fork and bring in > >> changes from Tinkerpop releases (as I've just done with 3.3.1) > >> > >> * By default, even our fork still uses the same (generic, non-strict) > >> mode. The user can enable the specialised/strict/memory-efficient mode > >> by providing factories for their specialised elements > >> > >> * It's all-or-nothing: either all elements are specialised or all > >> elements are generic. Technically this can be changed, I just didn't > >> have the need and time to do it. > >> > >> Cheers > >> Michael > >> > >> > >> > >> On 12/01/18 00:50, Stephen Mallette wrote: > >>> Michael Pollmeier recently authored a blog post that described how > their > >>> fork of TinkerGraph memory usage could be reduced by 70% assuming usage > >> of > >>> a strict schema: > >>> > >>> https://blog.shiftleft.io/open-sourcing-our-specialized- > >> tinkergraph-with-70-memory-reduction-and-strict-schema- > >> validation-fa5cfb3dd82d > >>> > >>> The question at this point, is whether or not similar enhancements > should > >>> be made to TinkerPop's TinkerGraph. I've had a few minutes to get to > >>> understand the changes that make the memory improvement possible - > here's > >>> my thoughts: > >>> > >>> 1. This was a clever way to extend TinkerGraph. > >>> 2. The schema is implemented by way of concrete graph element classes > >> shown > >>> here: > >>> > >>> https://github.com/ShiftLeftSecurity/tinkergraph- > >> gremlin/tree/master/src/test/java/org/apache/tinkerpop/ > >> gremlin/tinkergraph/structure/specialized/gratefuldead > >>> > >>> 3. Given that approach, you give up some flexibility in favor of > improved > >>> memory usage > >>> 4. This approach started to feel sufficiently different to me to > warrant > >>> this improvement as being more than just a "performance enhancement" > and > >>> more like a fundamental change to what TinkerGraph is about. > >>> 5. Of course, Michael had said on the user mailing list that the strict > >>> schema was optional - though you needed to use it to get the improved > >>> memory usage. > >>> 6. So....I think the questions forming in my mind are: (a) do we want a > >>> major new feature (schema support) on TinkerGraph? (b) if so, is the > >> schema > >>> support implemented in the manner in which we would want it (i.e. > >> concrete > >>> classes)? (c) is this new feature changing TinkerGraph's mission in > >>> TinkerPop? (d) if so, should it simply remain as a fork (presumably > >> under a > >>> different name to avoid confusion) so that it is not bound by what > >>> TinkerGraph is meant to be and can develop more freely?. > >>> > >>> I'll leave it at that for now and see what other people think. > >>> > >>> Irrespective of how this ends, I'd quickly offer another round of > thanks > >> to > >>> Michael and his colleagues for coming up with a neat feature and > >>> performance enhancement that could be useful to the TinkerPop > community. > >>> > >> > >> > > > >
