Good questions!
> What does this mean in terms of the original goals behind the storage system
> refactor?
My current effort is targeting goals (b), (c), and (d) in the above list.
Are we confident that Jordan's work for hot-followers will alleviate the
> problems w/ long failovers?
My plans
What does this mean in terms of the original goals behind the storage
system refactor? Are we confident that Jordan's work for hot-followers will
alleviate the problems w/ long failovers? I'm definitely in favor of
killing the H2 code if its goals can never be realized and it's just a
maintenance b
That’s right, nothing fancy.
On Oct 2, 2017, 4:24 AM -0700, Erb, Stephan ,
wrote:
> What do you have in mind for the in-memory replacement? Revert back to the
> usage of thrift objects within plain Java containers like we do for the task
> store?
>
> On 02.10.17, 00:59, "Bill Farner" wrote:
>
What do you have in mind for the in-memory replacement? Revert back to the
usage of thrift objects within plain Java containers like we do for the task
store?
On 02.10.17, 00:59, "Bill Farner" wrote:
I would like to revive this discussion in light of some work i have been
doing around
I would like to revive this discussion in light of some work i have been
doing around the storage system. The fruits of the DB storage system will
require a lot of additional effort to reach the beneficial outcomes i laid
out above, and i agree that we should cut our losses.
I plan to introduce p
H2 could give us fine granular data access. However, most of our code
performs massive joins to reconstruct fully hydrated thrift objects.
Most of the time we are then only interested in very few properties of
those thrift structs. This applies to internal usage, but also how we
use the API.
I the
So it sounds like before we make any decisions around removing the work
done in H2 so far, we should figure out what is remaining to move to
external storage (or if it's even still a goal).
I may still play around with reviving the in-memory stores, but will
separate that work from any goal to rem
Adding some background - there were several motivators to using SQL that come
to mind:
a) well-understood transaction isolation guarantees leading to a simpler
programming model w.r.t. concurrency
b) ability to offload storage to a separate system (e.g. Postgres) and scale it
separately
c) relie
My understanding of the H2-backed stores is that at least part of the
original rationale behind them was that they were meant to be an interim
point on the way to external SQL-backed stores which should theoretically
provide significant benefits w.r.t. to GC (obviously unproven, especially
at scale
I don't object to changes to storage so long as we have a migration plan
and a design doc. I'm also not opposed to radical revisits of storage,
including overhauling what we store and where we store it. For example,
instead of storing our `TaskConfig` objects could we store Mesos `TaskInfo`
objects
10 matches
Mail list logo
