Hi Michael I try to describe the changes proposed by the PoC in https://issues.apache.org/jira/browse/OAK-6073?focusedCommentId=15965623&pa ge=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment -15965623. Additionally added some step-by-step instruction on how we proceeded.
What is still missing is additional extra summary of impact it has on 'oak-core' and the cleanup of the 'oak.util' package. will add that tomorrow. More comments and answers to your questions inline: On 12/04/17 13:56, "Michael Dürig" <[email protected]> wrote: > >Thanks for driving this and coming up with a PoC. I like the direction >this is taking. The module boundaries make sense to me and having >certain dependencies de-tangled will certainly be helpful going forward. >Could you share a bit of your experience doing this refactoring? First of all it wasn't just me but a group of oak developers discussing and working on this :-) And I am sure they have more things to add here, looking at the whole exercise from a different angle. When we first looked at it on Tuesday last week, I thought that we would end the exercise with a "tried hard but failed" summary. So, I am quite pleased that actually ended up with a working PoC. >What were the main difficulties? Up to now we were looking for the hanging fruits. I guess the main difficulties lay ahead of us with plugins.tree and some of the query/index classes where exported stuff refers to private classes. > Looking back I thing the biggest issues are - putting everything in oak-core was obviously convenient but it turned out to be impossible to protect against boundary violations - packages sometimes contain classes that not really belong together, e.g. - org.apache.jackrabbit.oak.spi.lifecycle containing OakInitializer - oak.apache.jackrabbit.oak.spi.whiteboard containing classes that should be located with the corresponding feature (e.g. user mgt, index) - impl specific methods that are not defined by an API contract such as e.g. ValueImpl.getBlob, ValueImpl.getOakString... this was actually the only place where we added a new interface and modified existing code - somehow i got the impression that we didn't manage to make consistent decision wrt package naming - what should go into a 'spi' package? - what should go into 'plugins'-something? and how is that different from spi? (and what is e.g. the diff between spi.blob and plugins.blob?) - when do we create a new package space oak.somethingnew and how are those packages intended to be used. Moving forward I think it would help a lot if we had a common understanding here and come up with some description what is used for what. Maybe we also need to take a closer look when adding new stuff to oak-core while moving forward. >Quick wins? Well... for me the biggest win is the fact that 'oak-blob-azure', 'oak-blob-cloud' and 'oak-segment-tar' no longer would depend on oak-core. Working on modularisation also is quite helpful to spot 'dumpster'-packages like e.g. oak.util Also, working on the modularisation helped to identify unused imports, the kind of things that come in naturally when trying to clean up. >Is there anything that could be >controversial? We had a discussion in the group on whether it really makes sense to have 'oak-blob' (existing) and 'oak-blob-plugins' (the blob.plugins part of oak-core)... I don't have a strong feeling here and would leave that primarily to those working on the blob handling... at the end we kept it separate to see the effective dependencies to and from those 2 modules first and then decide on whether we should merge it. after all merging should be fairly easy compared to the split. > >Looking at the list of modules, its size and the names, did you consider >switching to a hierarchical module structure? No, we didn't discuss that. >Or could this make sense later on? I don't have any strong preference here. We had some discussion on how we should align the svn structure in general and what would be the best when we want to start releasing modules individually. >Otherwise can we come up with a naming scheme that implies >grouping (e.g. node store implementations, blob store implementations, >etc.) Sure, makes a lot of sense to me. :-) > >Re. oak-base and oak-commons, these are probably separated to avoid >circular dependencies. Is there anyway to otherwise clarify the >difference between the two? I.e. if I implement a new class, which >module it should go into? Would oak-base be something like oak-core-spi >or even oak-spi? This would nicely dual the oak-store-spi module. Exactly... and actually I like the name 'oak-core-spi' a lot better. In OAK-6073 I stated that IMO that module might be a tmp solution as it currently contains a somewhat loose collection of packages that were in 'oak-core' and didn't really fit into 'oak-commons' from my point of view. After all I wanted to avoid converting 'oak-commons' into a second 'oak-core' :-). That module is the one with the least consistency IMO. But things may clarify if we move on... I definitely would love to move oak.spi.security and oak.security.* out of oak-core... but that probably requires a second round *wishful thinking*. > >Is there plans to move document/rdb stores to separate modules or is >this beyond the current scope? I guess that would be a natural step as we move on... but during the last week we didn't look into this. Kind regards Angela PS: will attach simplified picture to OAK-6073 to illustrated the big picture. > >Michael > >On 12.04.17 11:21, Angela Schreiber wrote: >> Hi >> >> As mentioned my Marcel this morning [0] we had some offline discussions >> related to the oak-blob-azure module and how we could independently >> release it. While we didn't see a satisfying solution for the 1.6 >>branch, >> we concluded that we should pick up the modularisation discussion for >> address this in the near future. >> >> Consequently a group of oak devs started to work on a PoC on how to >> improve modularisation of Oak (in particular oak-core). As we managed to >> get rid of the dependency of oak-blob-azure (and oak-segment-tar for >>that >> matter) from oak-core with a reasonable effort, we would like move >>forward >> with this in oak-trunk. >> >> For that matter I created a new epic "Modularisation of Oak" (OAK-6069 >>at >> [1]) and added/linked a initial bunch of issues spotted during the >> workshop and earlier. For the 'oak-blob-azure' topic I create a >>dedicated >> task OAK-6073 [2], where I will also add some detailed summary of the >> initial effort. The latter can also be looked at on a github fork at >>[3]. >> >> Kind regards >> Angela >> >> [0] >> >>https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fmarkmail. >>org%2Fmessage%2Fneoiyv5qsffo424e%3Fq%3Dazure%2Blist%3Aorg%252Eapache%252E >>ja&data=02%7C01%7C%7Cdcbd2fbcce2d43eed98208d4819aeae0%7Cfa7b1b5a7b3443879 >>4aed2c178decee1%7C0%7C0%7C636275949762279890&sdata=sSEdgDegV4Sigh5%2B5R1y >>uZQiMUY%2F7pOmvBhxojSpDV8%3D&reserved=0 >> ckrabbit%2Eoak-dev+from:%22Marcel+Reutegger%22&page=1 >> [1] >>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a >>pache.org%2Fjira%2Fbrowse%2FOAK-6069&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9 >>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622 >>89898&sdata=Gxu0MBcOz7zuobBRVSpofWBDJTV36T60bLH3Wmn1v5Q%3D&reserved=0 >> [2] >>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fissues.a >>pache.org%2Fjira%2Fbrowse%2FOAK-6073&data=02%7C01%7C%7Cdcbd2fbcce2d43eed9 >>8208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C6362759497622 >>89898&sdata=Xmxhr0gndLBTf4UvWiM50j7Q71yt13wpENm5SAeTVwo%3D&reserved=0 >> [3] >>https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.c >>om%2Fmreutegg%2Fjackrabbit-oak%2Ftree%2Fm12n&data=02%7C01%7C%7Cdcbd2fbcce >>2d43eed98208d4819aeae0%7Cfa7b1b5a7b34438794aed2c178decee1%7C0%7C0%7C63627 >>5949762289898&sdata=7gBCJs5NTQkAfvE6X0aAHSjaJsXlze9bPrTxBRFxXZE%3D&reserv >>ed=0. >>
