[
https://issues.apache.org/jira/browse/KUDU-2321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Wong updated KUDU-2321:
------------------------------
Description:
Today, Kudu will refuse to start up when its FS layout isn't as expected.
Before 1.6.0, users could not add data directories; before 1.7.0, users could
not remove data directories; today, to do either of the above, users must use
the `update_dirs` tool before starting up.
Prior to this, the preferred way to start up a Kudu node with a different FS
layout was to rmrf the entirety of the node's FS layout and start a new node
with a new UUID at the same location. While Kudu is designed to be resilient to
such removal of data (automatically replicating the removed tablets to other
nodes), this has lead to problems, as, e.g., wiping multiple nodes without
waiting ample time in between wipes could lead to data loss.
While the `update_dirs` tool removes the need for rmrf altogether, that may not
stop unknowing users from wiping their nodes clean upon failure to startup
(e.g. if a user adds a data dir through some cluster management software like
Cloudera Manager (which doesn't run `update_dirs`), and receives an error upon
starting up). Now that we have a solution to safe removal and addition of data
directories with known constraints to its usage, it's not unthinkable that we
extend Kudu itself to start up with a different FS layout subject to the same
constraints.
was:
Today, Kudu will refuse to start up when its FS layout isn't as expected.
Before 1.6.0, users could not add data directories; before 1.7.0, users could
not remove data directories; today, to do either of the above, users must use
the `update_dirs` tool before starting up.
Prior to this, the preferred way to start up a Kudu node with a different FS
layout was to rmrf the entirety of the node's FS layout and start a new node
with a new UUID at the same location. While Kudu is designed to be resilient to
such removal of data (automatically replicating the removed tablets to other
nodes), this has lead to problems, as, e.g., wiping multiple nodes without
waiting ample time in between wipes could lead to data loss.
While the `update_dirs` tool removes the need for rmrf altogether, that may not
stop unknowing users from wiping their nodes clean. Now that we have a solution
to safe removal and addition of data directories with known constraints to its
usage, it's not unthinkable that we extend Kudu itself to start up with a
different FS layout subject to the same constraints.
> Allow Kudu to start up with different data directories
> ------------------------------------------------------
>
> Key: KUDU-2321
> URL: https://issues.apache.org/jira/browse/KUDU-2321
> Project: Kudu
> Issue Type: Improvement
> Components: fs, supportability
> Reporter: Andrew Wong
> Priority: Major
>
> Today, Kudu will refuse to start up when its FS layout isn't as expected.
> Before 1.6.0, users could not add data directories; before 1.7.0, users could
> not remove data directories; today, to do either of the above, users must use
> the `update_dirs` tool before starting up.
> Prior to this, the preferred way to start up a Kudu node with a different FS
> layout was to rmrf the entirety of the node's FS layout and start a new node
> with a new UUID at the same location. While Kudu is designed to be resilient
> to such removal of data (automatically replicating the removed tablets to
> other nodes), this has lead to problems, as, e.g., wiping multiple nodes
> without waiting ample time in between wipes could lead to data loss.
> While the `update_dirs` tool removes the need for rmrf altogether, that may
> not stop unknowing users from wiping their nodes clean upon failure to
> startup (e.g. if a user adds a data dir through some cluster management
> software like Cloudera Manager (which doesn't run `update_dirs`), and
> receives an error upon starting up). Now that we have a solution to safe
> removal and addition of data directories with known constraints to its usage,
> it's not unthinkable that we extend Kudu itself to start up with a different
> FS layout subject to the same constraints.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
