Hi Arek,
My mistake, my initial message (below) was intended for oak-dev.
Michael
On 10.05.17 11:01, Arek Kita wrote:
Hi Michael,
sorry, but I noticed right now, that the email was directed *only* to
me (not sure if on purpose).
Yes, I guess the approach should be not in the way how to create and
load such file, but starting from the other way round, how we can glue
all NodeStores (DataStores) together first based on what we have now
using one Builder/Factory interfaces that will create and link
everything. Then we can think about higher-level configuration format
(human configurable, not only dev configurable).
I guess this would be a good topic for an Oakathon because I'm not
involved in Oak heavily right now at any particular story. I guess
however that it would be easier for me to be focused and involved once
for that particular improvement.
Thanks,
Arek
2017-05-02 10:42 GMT+02:00 Michael Dürig <[email protected]>:
Hi Arek,
I agree that Oak would benefit from being simpler and more uniform to
configure and run. The current approach with the Oak and Jcr builder classes
has somehow outgrown its initial purpose.
I am somewhat sceptic about (starting with) configurations files thought.
Configuration files tend to suffer from weak semantics that is not well
understood and poorly documented. Going forward it can be difficult to
evolve the format in a consistent way while keeping backward compatibility.
I would therefore rather suggest to start experimenting with simpler ways to
setup Oak in code through an internal DSL. From there we can start a
discussion whether it is worth to introduce external bindings and how these
should look. Maybe such bindings would just expose the most common subset of
features and you would need to turn to the internal configuration DSL for
the full expressive power.
Michael
On 28.04.17 12:56, Arek Kita wrote:
Hi,
I've noticed recently that with many different NodeStore
implementation (Segment, Document, Multiplexing) but also DataStore
implementation (File, S3, Azure) and some composite ones like
(Hierarchical, Federated - that was already mentioned in [0]) it
becomes more and more difficult to set up everything correctly and be
able to know the current persistence state of repository (especially
with pretty aged repos).
Moreover, the configuration pattern that is based on individual PID of
one service becomes problematic (i.e. recent change for
SegmentNodeStoreService).
From the operations and user perspective everything should be treated
as a whole IMHO no matter which service handles which fragment of
persistence layout. Oak should know itself how to "autowire" different
parts, obviously with some hints and pointers from users as they want
to run Oak in their own preferred layout.
My proposal would be to integrate everything together to a pretty old
concept called "fstab". For our purposes I would call it "nstab".
This could look like [1] for the most simple case (with internal
blobs), [2] for typical SegmentMK + FDS, [3] for SegmentMK + S3DS, [4]
for MultiplexingNodeStore with some areas of repo set as read only. I
think we could also model Hierarchical and Federated DataStores as
well in the future.
Examples are for illustration purposes but I guess such setup will
help changing layout without a need to inspect many OSGi
configurations in a current setup and making sure some conflicting
ones aren't active.
The schema is also similar to an UNIX-way of configuring filesystem so
it will help Oak users to understand the layout (at least better than
it is now). I see also advantage for automated tooling like
oak-upgrade for complex cases in the future - user just provides
source nstab and target nstab in order to migrate repository.
The config should be also simpler avoiding things like customBlobStore
(it will be inferred from context).
WDYT? I have some thoughts how could this be implemented but first I
would like to know your opinions on that.
Thanks in advance for feedback!
Arek
[0] http://oak.markmail.org/thread/22dvuo6b7ab5ib7m
[1]
https://gist.githubusercontent.com/kitarek/f755dab6e889d1dfc5a1c595727f0171/raw/53d41ac7f935886783afd6c85d60e38e565a9259/nstab.1
[2]
https://gist.githubusercontent.com/kitarek/f755dab6e889d1dfc5a1c595727f0171/raw/53d41ac7f935886783afd6c85d60e38e565a9259/nstab.2
[3]
https://gist.githubusercontent.com/kitarek/f755dab6e889d1dfc5a1c595727f0171/raw/53d41ac7f935886783afd6c85d60e38e565a9259/nstab.3
[4]
https://gist.githubusercontent.com/kitarek/f755dab6e889d1dfc5a1c595727f0171/raw/53d41ac7f935886783afd6c85d60e38e565a9259/nstab.4
