Re: create or setData in transaction?
Hi Ted,
Thanks a lot for your email.
I don't stick to a general atomic if-else capability. It comes
from a function in etcd community[1] and I'm curious whether ZooKeepr
has cover this capability.
Follow this thread I got the idea that a multi operation is like
a transaction, succeed or fail together, and inherent "isolated".
The case you take about "lock" implementation and do what I mean
with holding the lock is what I'm actually facing. Since it is diverged from
the title of this thread. I would rephrase and start a new thread later.
Thank you again :-)
Best,
tison.
[1] https://godoc.org/github.com/coreos/etcd/clientv3#OpTxn
Ted Dunning 于2019年8月16日周五 上午1:28写道：
> So, getting back to the original problem, I think that you need to specify
> it more tightly in order to solve it. Otherwise, when you find problems in
> a solution you won't know whether those are problems with the solution or
> the original problem statement.
>
> As I see it, there are several possible meanings of what you are asking
> for.
>
> One possible specification is that you want a createOrSet such that it has
> one value if the znode was created and a different value if it was
> modified. It is important to allow various other operations before or after
> this operation.
>
> Another possible specification that I would dismiss for now is a general
> atomic if-else capability. That is a bit much for me to handle in the
> morning. :-)
>
>
> So...
>
> firstly, you could just do this:
>
> while (true) {
> if (create(...)) break;
> if (set(...)) break; // could not succeed
> }
>
> This is actually a very practical solution that will only ever iterate if
> it collides with another mutation in flight and it will have all of the
> atomic properties that we would like because only one of the create or set
> will ever succeed.
>
> If you have almost *any* constraints on the updates that other machines
> will make to this znode, then you can prove that this loop is a good
> implementation. For instance, if there is a limit on the frequency of
> updates. Or if nodes will never delete the znode. Or ... many other kinds
> of constraint.
>
> You should, of course, check the returned values to be sure that the
> failures are not due to permissions, but are, in fact, due to the node
> existing at the time of the create or not existing at the time of the set.
>
> So this is a practical solution even if not a theoretically nice one.
>
>
> If you really want a solution that has theoretical guarantees, you can use
> this instead:
>
> lock (znode-twin) {
> create(...) || set(...)
> }
>
> Here you would use a standard ZK idiom to create a lock. One simple and
> pretty standard version is to try to create a node with a watch. If the
> create fails, wait for the watch to trigger and try again. Once the create
> succeeds, do your stuff and delete the znode as you leave. There are lock
> implementations that implement fair waiting at the cost of just a bit more
> complexity.
>
> This alternative requires more ZK operations (create lock, create, maybe
> set, delete lock) and requires two znodes. If you use a fair lock
> implementation, you are have better completion guarantees.
>
> Does this do what you want?
>
>
>
>
>
>
> On Wed, Aug 14, 2019 at 11:04 PM Zili Chen wrote:
>
> > Thanks for your explanation Michael and Ted :-)
> >
> > Best,
> > tison.
> >
> >
> > Ted Dunning 于2019年8月15日周四 下午1:22写道：
> >
> > > As Michael correctly said, isolation only makes sense when you allow
> > > concurrent queries. Of the four ACID properties, the multi op satisfies
> > A,
> > > C and D while I is essentially irrelevant (or could be said to be
> > trivially
> > > satisfied since there are no concurrent queries.
> > >
> > >
> > > On Wed, Aug 14, 2019 at 6:45 PM Zili Chen
> wrote:
> > >
> > > > Thanks for your reply Ted.
> > > >
> > > > I cannot understand the statement "That leaves isolated which is kind
> > of
> > > > hard to talk about with ZK since all operations are fast and
> > sequential."
> > > > well. Could you explain a bit? What is "that" means and where is the
> > > "hard"
> > > > comes from?
> > > >
> > > > Best,
> > > > tison.
> > > >
> > > >
> > > > Ted Dunning 于2019年8月15日周四 上午9:40写道：
> > > >
> > > > > The multi op is atomic (all other operations will be before or
> after
> > > teh
> > > > > multi), consistent (all viewers will see all the effects or none,
> and
> > > > > durable (because ZK is linearized anyway).
> > > > >
> > > > > That leaves isolated which is kind of hard to talk about with ZK
> > since
> > > > all
> > > > > operations are fast and sequential.
> > > > >
> > > > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han
> wrote:
> > > > >
> > > > > > ...
> > > > > > Ted can correct me if I am wrong, since he added the multi op
> > > feature,
> > > > > but
> > > > > > my understanding is "multi op" is branded from day one as the
> > > > transaction
> > > > > > support for
Re: create or setData in transaction?
So, getting back to the original problem, I think that you need to specify
it more tightly in order to solve it. Otherwise, when you find problems in
a solution you won't know whether those are problems with the solution or
the original problem statement.
As I see it, there are several possible meanings of what you are asking for.
One possible specification is that you want a createOrSet such that it has
one value if the znode was created and a different value if it was
modified. It is important to allow various other operations before or after
this operation.
Another possible specification that I would dismiss for now is a general
atomic if-else capability. That is a bit much for me to handle in the
morning. :-)
So...
firstly, you could just do this:
while (true) {
if (create(...)) break;
if (set(...)) break; // could not succeed
}
This is actually a very practical solution that will only ever iterate if
it collides with another mutation in flight and it will have all of the
atomic properties that we would like because only one of the create or set
will ever succeed.
If you have almost *any* constraints on the updates that other machines
will make to this znode, then you can prove that this loop is a good
implementation. For instance, if there is a limit on the frequency of
updates. Or if nodes will never delete the znode. Or ... many other kinds
of constraint.
You should, of course, check the returned values to be sure that the
failures are not due to permissions, but are, in fact, due to the node
existing at the time of the create or not existing at the time of the set.
So this is a practical solution even if not a theoretically nice one.
If you really want a solution that has theoretical guarantees, you can use
this instead:
lock (znode-twin) {
create(...) || set(...)
}
Here you would use a standard ZK idiom to create a lock. One simple and
pretty standard version is to try to create a node with a watch. If the
create fails, wait for the watch to trigger and try again. Once the create
succeeds, do your stuff and delete the znode as you leave. There are lock
implementations that implement fair waiting at the cost of just a bit more
complexity.
This alternative requires more ZK operations (create lock, create, maybe
set, delete lock) and requires two znodes. If you use a fair lock
implementation, you are have better completion guarantees.
Does this do what you want?
On Wed, Aug 14, 2019 at 11:04 PM Zili Chen wrote:
> Thanks for your explanation Michael and Ted :-)
>
> Best,
> tison.
>
>
> Ted Dunning 于2019年8月15日周四 下午1:22写道：
>
> > As Michael correctly said, isolation only makes sense when you allow
> > concurrent queries. Of the four ACID properties, the multi op satisfies
> A,
> > C and D while I is essentially irrelevant (or could be said to be
> trivially
> > satisfied since there are no concurrent queries.
> >
> >
> > On Wed, Aug 14, 2019 at 6:45 PM Zili Chen wrote:
> >
> > > Thanks for your reply Ted.
> > >
> > > I cannot understand the statement "That leaves isolated which is kind
> of
> > > hard to talk about with ZK since all operations are fast and
> sequential."
> > > well. Could you explain a bit? What is "that" means and where is the
> > "hard"
> > > comes from?
> > >
> > > Best,
> > > tison.
> > >
> > >
> > > Ted Dunning 于2019年8月15日周四 上午9:40写道：
> > >
> > > > The multi op is atomic (all other operations will be before or after
> > teh
> > > > multi), consistent (all viewers will see all the effects or none, and
> > > > durable (because ZK is linearized anyway).
> > > >
> > > > That leaves isolated which is kind of hard to talk about with ZK
> since
> > > all
> > > > operations are fast and sequential.
> > > >
> > > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote:
> > > >
> > > > > ...
> > > > > Ted can correct me if I am wrong, since he added the multi op
> > feature,
> > > > but
> > > > > my understanding is "multi op" is branded from day one as the
> > > transaction
> > > > > support for zookeeper (we even provide an API with exact name:
> > > > > Transaction). If we use the traditional semantic for transaction in
> > > > > database context, the ACID properties multi-op satisfies at least
> > > > atomicity
> > > > > and durability. So saying zookeeper does not support transaction
> > seems
> > > a
> > > > > strong argument that against the properties of multi-op and
> existing
> > > > > literatures related to zookeeper. On the other side, typically bulk
> > > > > operations does not support atomicity, which will not take care of
> > > > rolling
> > > > > back failed operations.
> > > > >
> > > >
> > > >
> > > > >
> > > >
> > >
> >
>
Re: create or setData in transaction?
Thanks for your explanation Michael and Ted :-) Best, tison. Ted Dunning 于2019年8月15日周四 下午1:22写道： > As Michael correctly said, isolation only makes sense when you allow > concurrent queries. Of the four ACID properties, the multi op satisfies A, > C and D while I is essentially irrelevant (or could be said to be trivially > satisfied since there are no concurrent queries. > > > On Wed, Aug 14, 2019 at 6:45 PM Zili Chen wrote: > > > Thanks for your reply Ted. > > > > I cannot understand the statement "That leaves isolated which is kind of > > hard to talk about with ZK since all operations are fast and sequential." > > well. Could you explain a bit? What is "that" means and where is the > "hard" > > comes from? > > > > Best, > > tison. > > > > > > Ted Dunning 于2019年8月15日周四 上午9:40写道： > > > > > The multi op is atomic (all other operations will be before or after > teh > > > multi), consistent (all viewers will see all the effects or none, and > > > durable (because ZK is linearized anyway). > > > > > > That leaves isolated which is kind of hard to talk about with ZK since > > all > > > operations are fast and sequential. > > > > > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote: > > > > > > > ... > > > > Ted can correct me if I am wrong, since he added the multi op > feature, > > > but > > > > my understanding is "multi op" is branded from day one as the > > transaction > > > > support for zookeeper (we even provide an API with exact name: > > > > Transaction). If we use the traditional semantic for transaction in > > > > database context, the ACID properties multi-op satisfies at least > > > atomicity > > > > and durability. So saying zookeeper does not support transaction > seems > > a > > > > strong argument that against the properties of multi-op and existing > > > > literatures related to zookeeper. On the other side, typically bulk > > > > operations does not support atomicity, which will not take care of > > > rolling > > > > back failed operations. > > > > > > > > > > > > > > > > > > > >
Re: create or setData in transaction?
As Michael correctly said, isolation only makes sense when you allow concurrent queries. Of the four ACID properties, the multi op satisfies A, C and D while I is essentially irrelevant (or could be said to be trivially satisfied since there are no concurrent queries. On Wed, Aug 14, 2019 at 6:45 PM Zili Chen wrote: > Thanks for your reply Ted. > > I cannot understand the statement "That leaves isolated which is kind of > hard to talk about with ZK since all operations are fast and sequential." > well. Could you explain a bit? What is "that" means and where is the "hard" > comes from? > > Best, > tison. > > > Ted Dunning 于2019年8月15日周四 上午9:40写道： > > > The multi op is atomic (all other operations will be before or after teh > > multi), consistent (all viewers will see all the effects or none, and > > durable (because ZK is linearized anyway). > > > > That leaves isolated which is kind of hard to talk about with ZK since > all > > operations are fast and sequential. > > > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote: > > > > > ... > > > Ted can correct me if I am wrong, since he added the multi op feature, > > but > > > my understanding is "multi op" is branded from day one as the > transaction > > > support for zookeeper (we even provide an API with exact name: > > > Transaction). If we use the traditional semantic for transaction in > > > database context, the ACID properties multi-op satisfies at least > > atomicity > > > and durability. So saying zookeeper does not support transaction seems > a > > > strong argument that against the properties of multi-op and existing > > > literatures related to zookeeper. On the other side, typically bulk > > > operations does not support atomicity, which will not take care of > > rolling > > > back failed operations. > > > > > > > > > > > > >
Re: create or setData in transaction?
>> where is the "hard" comes from? Isolation levels is only meaningful when concurrent transactions are allowed. For zookeeper, there is no concurrent transactions, as every transaction (and write operations in general) is serialized. On Wed, Aug 14, 2019 at 6:45 PM Zili Chen wrote: > Thanks for your reply Ted. > > I cannot understand the statement "That leaves isolated which is kind of > hard to talk about with ZK since all operations are fast and sequential." > well. Could you explain a bit? What is "that" means and where is the "hard" > comes from? > > Best, > tison. > > > Ted Dunning 于2019年8月15日周四 上午9:40写道： > > > The multi op is atomic (all other operations will be before or after teh > > multi), consistent (all viewers will see all the effects or none, and > > durable (because ZK is linearized anyway). > > > > That leaves isolated which is kind of hard to talk about with ZK since > all > > operations are fast and sequential. > > > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote: > > > > > ... > > > Ted can correct me if I am wrong, since he added the multi op feature, > > but > > > my understanding is "multi op" is branded from day one as the > transaction > > > support for zookeeper (we even provide an API with exact name: > > > Transaction). If we use the traditional semantic for transaction in > > > database context, the ACID properties multi-op satisfies at least > > atomicity > > > and durability. So saying zookeeper does not support transaction seems > a > > > strong argument that against the properties of multi-op and existing > > > literatures related to zookeeper. On the other side, typically bulk > > > operations does not support atomicity, which will not take care of > > rolling > > > back failed operations. > > > > > > > > > > > > >
Re: create or setData in transaction?
Thanks for your reply Ted. I cannot understand the statement "That leaves isolated which is kind of hard to talk about with ZK since all operations are fast and sequential." well. Could you explain a bit? What is "that" means and where is the "hard" comes from? Best, tison. Ted Dunning 于2019年8月15日周四 上午9:40写道： > The multi op is atomic (all other operations will be before or after teh > multi), consistent (all viewers will see all the effects or none, and > durable (because ZK is linearized anyway). > > That leaves isolated which is kind of hard to talk about with ZK since all > operations are fast and sequential. > > On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote: > > > ... > > Ted can correct me if I am wrong, since he added the multi op feature, > but > > my understanding is "multi op" is branded from day one as the transaction > > support for zookeeper (we even provide an API with exact name: > > Transaction). If we use the traditional semantic for transaction in > > database context, the ACID properties multi-op satisfies at least > atomicity > > and durability. So saying zookeeper does not support transaction seems a > > strong argument that against the properties of multi-op and existing > > literatures related to zookeeper. On the other side, typically bulk > > operations does not support atomicity, which will not take care of > rolling > > back failed operations. > > > > > > >
Re: create or setData in transaction?
OuO
Let me take a closer look...
Best,
tison.
Ted Dunning 于2019年8月15日周四 上午9:37写道：
> The multi operation *is* like a transaction. All the operations will
> succeed or none.
>
>
>
> On Wed, Aug 14, 2019 at 9:33 AM Andor Molnar wrote:
>
> > "it's said that ZooKeeper has a transaction mechanism”
> >
> > I’m still confused with this. ZooKeeper doesn’t support transactions to
> my
> > best knowledge. It has a `multi` operation feature, but that’s more like
> a
> > bulk operation, not transaction.
> >
> > "I want to tell ZooKeeper that the check and setData should be
> successful”
> >
> > I don’t think you can do that. ZK has no check-and-set support either.
> >
> > Maybe we should step back first and see what’s your use case exactly that
> > you’re trying to solve with ZooKeeper. I suspect that you’re trying to
> > follow the wrong approach or misusing ZooKeeper.
> >
> > Have you checked our tutorial and recipes page?
> > You can find some recommended usage patterns:
> > https://zookeeper.apache.org/doc/r3.5.5/recipes.html
> > https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html
> >
> > If that’s not enough, you could also try Curator which has even more
> > built-in high level functionalities on top of basic ZK commands.
> >
> > Andor
> >
> >
> >
> > > On 2019. Aug 14., at 17:52, Zili Chen wrote:
> > >
> > > Hi Andor,
> > >
> > > Thanks for your attention.
> > >
> > > The problem is that in concurrent scenario zk.setData() could still
> > failed
> > > if there is another thread delete the node. I know with proper lock
> > strategy
> > > and ownership separation this can be avoid but it's said that ZooKeeper
> > has
> > > a transaction mechanism so I'd like to see whether I can make use of
> it.
> > >
> > > There is where I turn to
> > >
> > > zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> > path2
> > > is irrelevant
> > >
> > > when the existence of a mark node(path1) guarded a condition and I want
> > to
> > > make
> > > sure that setData successes only if the mark node exist. If I check the
> > > existence
> > > first and commit setData, a remove to the node could break the guard.
> In
> > > other
> > > words, I want to tell ZooKeeper that the check and setData should be
> > > successful
> > > committed or fail to be committed atomically.
> > >
> > > Best,
> > > tison.
> > >
> > >
> > > Andor Molnar 于2019年8月14日周三 下午11:12写道：
> > >
> > >> Hi Zili,
> > >>
> > >> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> > >> think your multi example (zk.multi(Op.check(path), Op.setData(path,
> > data)))
> > >> is already a usage pattern which multi is not designed to support.
> > >>
> > >> Why do you need to do this in “transactions” (there’s no transaction
> in
> > >> ZK)?
> > >>
> > >> In Java you can do:
> > >>
> > >> try {
> > >> zk.create();
> > >> } catch (NodeExistsException e) {
> > >> // swallow exception
> > >> }
> > >> zk.setData();
> > >> …
> > >>
> > >> Regards,
> > >> Andor
> > >>
> > >>
> > >>
> > >>
> > >>> On 2019. Aug 6., at 14:44, Zili Chen wrote:
> > >>>
> > >>> Hi Enrico,
> > >>>
> > >>> Thanks for your reply.
> > >>>
> > In this case usually you use conditional setData, using the
> 'version'
> > of
> > thr znode
> > >>>
> > >>>
> > >>> what if the caller has no idea on whether the node exist?(see also
> > >>> my if-else pseudo-code above.)
> > >>>
> > >>> IIRC if we call `setData` on a non-exist path a NoNodeException
> > >>> will be thrown.
> > >>> Best,
> > >>> tison.
> > >>>
> > >>>
> > >>> Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> > >>>
> > Il mar 6 ago 2019, 13:47 Zili Chen ha
> scritto:
> >
> > > Any ideas?
> > >
> > >
> > > Zili Chen 于2019年7月29日周一 上午11:12写道：
> > >
> > >> Hi ZooKeepers,
> > >>
> > >> Currently our transaction mechanism supports doing
> > >> create/setData/checkExist/delete in transaction. However, taking
> > this
> > >> scenario into consideration, we want to put data in path "/path"
> but
> > >> don't know whether the znode exists or not. Let's say we program
> as
> > >> below
> > >>
> > >> if (zk.exist(path)) {
> > >> zk.setData(path, data);
> > >> } else {
> > >> zk.create(path, data);
> > >> }
> > >
> >
> > Do you need to perform other ops in the same transaction?
> > In this case usually you use conditional setData, using the
> 'version'
> > of
> > thr znode
> >
> >
> > Enrico
> >
> > >
> > >> if we want to do the check and "put" in transaction, it would be
> > like
> > >>
> > >> zk.multi(Op.check(path), Op.setData(path, data));
> > >>
> > >> but we cannot add a "else" branch. ZooKeeper's transaction would
> all
> > >> success or fail.
> > >>
> > >> Is there any way to do an "if-else" transaction?
> > >>
> > >> Best,
> > >> tison.
> > >>
> > >
> >
> > >>
> > >>
> >
> >
>
Re: create or setData in transaction?
The multi op is atomic (all other operations will be before or after teh multi), consistent (all viewers will see all the effects or none, and durable (because ZK is linearized anyway). That leaves isolated which is kind of hard to talk about with ZK since all operations are fast and sequential. On Wed, Aug 14, 2019 at 3:12 PM Michael Han wrote: > ... > Ted can correct me if I am wrong, since he added the multi op feature, but > my understanding is "multi op" is branded from day one as the transaction > support for zookeeper (we even provide an API with exact name: > Transaction). If we use the traditional semantic for transaction in > database context, the ACID properties multi-op satisfies at least atomicity > and durability. So saying zookeeper does not support transaction seems a > strong argument that against the properties of multi-op and existing > literatures related to zookeeper. On the other side, typically bulk > operations does not support atomicity, which will not take care of rolling > back failed operations. > >
Re: create or setData in transaction?
The multi operation *is* like a transaction. All the operations will
succeed or none.
On Wed, Aug 14, 2019 at 9:33 AM Andor Molnar wrote:
> "it's said that ZooKeeper has a transaction mechanism”
>
> I’m still confused with this. ZooKeeper doesn’t support transactions to my
> best knowledge. It has a `multi` operation feature, but that’s more like a
> bulk operation, not transaction.
>
> "I want to tell ZooKeeper that the check and setData should be successful”
>
> I don’t think you can do that. ZK has no check-and-set support either.
>
> Maybe we should step back first and see what’s your use case exactly that
> you’re trying to solve with ZooKeeper. I suspect that you’re trying to
> follow the wrong approach or misusing ZooKeeper.
>
> Have you checked our tutorial and recipes page?
> You can find some recommended usage patterns:
> https://zookeeper.apache.org/doc/r3.5.5/recipes.html
> https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html
>
> If that’s not enough, you could also try Curator which has even more
> built-in high level functionalities on top of basic ZK commands.
>
> Andor
>
>
>
> > On 2019. Aug 14., at 17:52, Zili Chen wrote:
> >
> > Hi Andor,
> >
> > Thanks for your attention.
> >
> > The problem is that in concurrent scenario zk.setData() could still
> failed
> > if there is another thread delete the node. I know with proper lock
> strategy
> > and ownership separation this can be avoid but it's said that ZooKeeper
> has
> > a transaction mechanism so I'd like to see whether I can make use of it.
> >
> > There is where I turn to
> >
> > zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> path2
> > is irrelevant
> >
> > when the existence of a mark node(path1) guarded a condition and I want
> to
> > make
> > sure that setData successes only if the mark node exist. If I check the
> > existence
> > first and commit setData, a remove to the node could break the guard. In
> > other
> > words, I want to tell ZooKeeper that the check and setData should be
> > successful
> > committed or fail to be committed atomically.
> >
> > Best,
> > tison.
> >
> >
> > Andor Molnar 于2019年8月14日周三 下午11:12写道：
> >
> >> Hi Zili,
> >>
> >> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> >> think your multi example (zk.multi(Op.check(path), Op.setData(path,
> data)))
> >> is already a usage pattern which multi is not designed to support.
> >>
> >> Why do you need to do this in “transactions” (there’s no transaction in
> >> ZK)?
> >>
> >> In Java you can do:
> >>
> >> try {
> >> zk.create();
> >> } catch (NodeExistsException e) {
> >> // swallow exception
> >> }
> >> zk.setData();
> >> …
> >>
> >> Regards,
> >> Andor
> >>
> >>
> >>
> >>
> >>> On 2019. Aug 6., at 14:44, Zili Chen wrote:
> >>>
> >>> Hi Enrico,
> >>>
> >>> Thanks for your reply.
> >>>
> In this case usually you use conditional setData, using the 'version'
> of
> thr znode
> >>>
> >>>
> >>> what if the caller has no idea on whether the node exist?(see also
> >>> my if-else pseudo-code above.)
> >>>
> >>> IIRC if we call `setData` on a non-exist path a NoNodeException
> >>> will be thrown.
> >>> Best,
> >>> tison.
> >>>
> >>>
> >>> Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> >>>
> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
>
> > Any ideas?
> >
> >
> > Zili Chen 于2019年7月29日周一 上午11:12写道：
> >
> >> Hi ZooKeepers,
> >>
> >> Currently our transaction mechanism supports doing
> >> create/setData/checkExist/delete in transaction. However, taking
> this
> >> scenario into consideration, we want to put data in path "/path" but
> >> don't know whether the znode exists or not. Let's say we program as
> >> below
> >>
> >> if (zk.exist(path)) {
> >> zk.setData(path, data);
> >> } else {
> >> zk.create(path, data);
> >> }
> >
>
> Do you need to perform other ops in the same transaction?
> In this case usually you use conditional setData, using the 'version'
> of
> thr znode
>
>
> Enrico
>
> >
> >> if we want to do the check and "put" in transaction, it would be
> like
> >>
> >> zk.multi(Op.check(path), Op.setData(path, data));
> >>
> >> but we cannot add a "else" branch. ZooKeeper's transaction would all
> >> success or fail.
> >>
> >> Is there any way to do an "if-else" transaction?
> >>
> >> Best,
> >> tison.
> >>
> >
>
> >>
> >>
>
>
Re: create or setData in transaction?
Hi Jordan, Thanks for your reply. The problem is that we cannot confirm an action is committed atomically with checking the leadership. The thread shows that we are unable to access the latch znode. There is another thread[1] in Curator list and we could continue the discussion there. Best, tison. [1] https://www.mail-archive.com/user@curator.apache.org/msg01238.html Jordan Zimmerman 于2019年8月15日周四 上午8:23写道： > FYI - that "wart" had a workaround. You can't use the same workaround? > > -Jordan > > > On Aug 14, 2019, at 5:19 PM, Zili Chen wrote: > > > > Actually I want something like LeaderLatch of Curator but the > > implementation in Curator has some wart[1] > >
Re: create or setData in transaction?
FYI - that "wart" had a workaround. You can't use the same workaround? -Jordan > On Aug 14, 2019, at 5:19 PM, Zili Chen wrote: > > Actually I want something like LeaderLatch of Curator but the > implementation in Curator has some wart[1]
Re: create or setData in transaction?
> Actually I want something like LeaderLatch of Curator but the > implementation in Curator has some wart[1] so that I need to > reimplement it by myself. Or we can enhance Curator :D Why not add a new recipe for what you want in Curator. I (or one of the other committers) will help ypu. -Jordan > On Aug 14, 2019, at 5:19 PM, Zili Chen wrote: > > @Enrico > > The problem is that an operation should be rejected not only if > the version mismatched, but also if the mark node did not exist > even if the version matched. > > @Andor > > Thanks for your clarification. Let's then refer it as multi op. > > Actually I want something like LeaderLatch of Curator but the > implementation in Curator has some wart[1] so that I need to > reimplement it by myself. The semantic "bulk operations" can also > fit my case if ZooKeeper ensures the check is immediately followed > by setData. Since check doesn't commit a modification the problem > that it wouldn't roll back doesn't matter. > > @Michael > > Thanks for your clarification, too. Now I know bulk operation won't > take care of rolling back failed operations and thus it is different > from traditional transactions. It is a misunderstanding from my side. > > Best, > tison. > > [1] https://www.mail-archive.com/user@curator.apache.org/msg00912.html > > > Michael Han 于2019年8月15日周四 上午6:12写道： > Is there any way to do an "if-else" transaction? >> >> No for your use case. The only remotely related conditional operation you >> can express with multi-op is by using check operator (Op.check), where you >> can check a zNode's version and only execute subsequent operation in multi >> op when version matches. Though, even this would only allow you to express >> "if" rather than "if / else". >> ZooKeeper doesn’t support transactions to my best knowledge. It has a >> `multi` operation feature, but that’s more like a bulk operation, not >> transaction. >> >> Ted can correct me if I am wrong, since he added the multi op feature, but >> my understanding is "multi op" is branded from day one as the transaction >> support for zookeeper (we even provide an API with exact name: >> Transaction). If we use the traditional semantic for transaction in >> database context, the ACID properties multi-op satisfies at least atomicity >> and durability. So saying zookeeper does not support transaction seems a >> strong argument that against the properties of multi-op and existing >> literatures related to zookeeper. On the other side, typically bulk >> operations does not support atomicity, which will not take care of rolling >> back failed operations. >> >> On Wed, Aug 14, 2019 at 9:33 AM Andor Molnar wrote: >> >>> "it's said that ZooKeeper has a transaction mechanism” >>> >>> I’m still confused with this. ZooKeeper doesn’t support transactions to >> my >>> best knowledge. It has a `multi` operation feature, but that’s more like >> a >>> bulk operation, not transaction. >>> >>> "I want to tell ZooKeeper that the check and setData should be >> successful” >>> >>> I don’t think you can do that. ZK has no check-and-set support either. >>> >>> Maybe we should step back first and see what’s your use case exactly that >>> you’re trying to solve with ZooKeeper. I suspect that you’re trying to >>> follow the wrong approach or misusing ZooKeeper. >>> >>> Have you checked our tutorial and recipes page? >>> You can find some recommended usage patterns: >>> https://zookeeper.apache.org/doc/r3.5.5/recipes.html >>> https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html >>> >>> If that’s not enough, you could also try Curator which has even more >>> built-in high level functionalities on top of basic ZK commands. >>> >>> Andor >>> >>> >>> On 2019. Aug 14., at 17:52, Zili Chen wrote: Hi Andor, Thanks for your attention. The problem is that in concurrent scenario zk.setData() could still >>> failed if there is another thread delete the node. I know with proper lock >>> strategy and ownership separation this can be avoid but it's said that ZooKeeper >>> has a transaction mechanism so I'd like to see whether I can make use of >> it. There is where I turn to zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or != >>> path2 is irrelevant when the existence of a mark node(path1) guarded a condition and I want >>> to make sure that setData successes only if the mark node exist. If I check the existence first and commit setData, a remove to the node could break the guard. >> In other words, I want to tell ZooKeeper that the check and setData should be successful committed or fail to be committed atomically. Best, tison. Andor Molnar 于2019年8月14日周三 下午11:12写道： > Hi Zili, > > There’s no such functionality in ZooKeeper as far as I’m concerned. I > think your multi example
Re: create or setData in transaction?
@Enrico
The problem is that an operation should be rejected not only if
the version mismatched, but also if the mark node did not exist
even if the version matched.
@Andor
Thanks for your clarification. Let's then refer it as multi op.
Actually I want something like LeaderLatch of Curator but the
implementation in Curator has some wart[1] so that I need to
reimplement it by myself. The semantic "bulk operations" can also
fit my case if ZooKeeper ensures the check is immediately followed
by setData. Since check doesn't commit a modification the problem
that it wouldn't roll back doesn't matter.
@Michael
Thanks for your clarification, too. Now I know bulk operation won't
take care of rolling back failed operations and thus it is different
from traditional transactions. It is a misunderstanding from my side.
Best,
tison.
[1] https://www.mail-archive.com/user@curator.apache.org/msg00912.html
Michael Han 于2019年8月15日周四 上午6:12写道：
> >> Is there any way to do an "if-else" transaction?
>
> No for your use case. The only remotely related conditional operation you
> can express with multi-op is by using check operator (Op.check), where you
> can check a zNode's version and only execute subsequent operation in multi
> op when version matches. Though, even this would only allow you to express
> "if" rather than "if / else".
>
> >> ZooKeeper doesn’t support transactions to my best knowledge. It has a
> `multi` operation feature, but that’s more like a bulk operation, not
> transaction.
>
> Ted can correct me if I am wrong, since he added the multi op feature, but
> my understanding is "multi op" is branded from day one as the transaction
> support for zookeeper (we even provide an API with exact name:
> Transaction). If we use the traditional semantic for transaction in
> database context, the ACID properties multi-op satisfies at least atomicity
> and durability. So saying zookeeper does not support transaction seems a
> strong argument that against the properties of multi-op and existing
> literatures related to zookeeper. On the other side, typically bulk
> operations does not support atomicity, which will not take care of rolling
> back failed operations.
>
> On Wed, Aug 14, 2019 at 9:33 AM Andor Molnar wrote:
>
> > "it's said that ZooKeeper has a transaction mechanism”
> >
> > I’m still confused with this. ZooKeeper doesn’t support transactions to
> my
> > best knowledge. It has a `multi` operation feature, but that’s more like
> a
> > bulk operation, not transaction.
> >
> > "I want to tell ZooKeeper that the check and setData should be
> successful”
> >
> > I don’t think you can do that. ZK has no check-and-set support either.
> >
> > Maybe we should step back first and see what’s your use case exactly that
> > you’re trying to solve with ZooKeeper. I suspect that you’re trying to
> > follow the wrong approach or misusing ZooKeeper.
> >
> > Have you checked our tutorial and recipes page?
> > You can find some recommended usage patterns:
> > https://zookeeper.apache.org/doc/r3.5.5/recipes.html
> > https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html
> >
> > If that’s not enough, you could also try Curator which has even more
> > built-in high level functionalities on top of basic ZK commands.
> >
> > Andor
> >
> >
> >
> > > On 2019. Aug 14., at 17:52, Zili Chen wrote:
> > >
> > > Hi Andor,
> > >
> > > Thanks for your attention.
> > >
> > > The problem is that in concurrent scenario zk.setData() could still
> > failed
> > > if there is another thread delete the node. I know with proper lock
> > strategy
> > > and ownership separation this can be avoid but it's said that ZooKeeper
> > has
> > > a transaction mechanism so I'd like to see whether I can make use of
> it.
> > >
> > > There is where I turn to
> > >
> > > zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> > path2
> > > is irrelevant
> > >
> > > when the existence of a mark node(path1) guarded a condition and I want
> > to
> > > make
> > > sure that setData successes only if the mark node exist. If I check the
> > > existence
> > > first and commit setData, a remove to the node could break the guard.
> In
> > > other
> > > words, I want to tell ZooKeeper that the check and setData should be
> > > successful
> > > committed or fail to be committed atomically.
> > >
> > > Best,
> > > tison.
> > >
> > >
> > > Andor Molnar 于2019年8月14日周三 下午11:12写道：
> > >
> > >> Hi Zili,
> > >>
> > >> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> > >> think your multi example (zk.multi(Op.check(path), Op.setData(path,
> > data)))
> > >> is already a usage pattern which multi is not designed to support.
> > >>
> > >> Why do you need to do this in “transactions” (there’s no transaction
> in
> > >> ZK)?
> > >>
> > >> In Java you can do:
> > >>
> > >> try {
> > >> zk.create();
> > >> } catch (NodeExistsException e) {
> > >> // swallow exception
> > >> }
> > >> zk.setData();
> > >> …
> > >>
> > >> Regards,
> >
Re: create or setData in transaction?
>> Is there any way to do an "if-else" transaction?
No for your use case. The only remotely related conditional operation you
can express with multi-op is by using check operator (Op.check), where you
can check a zNode's version and only execute subsequent operation in multi
op when version matches. Though, even this would only allow you to express
"if" rather than "if / else".
>> ZooKeeper doesn’t support transactions to my best knowledge. It has a
`multi` operation feature, but that’s more like a bulk operation, not
transaction.
Ted can correct me if I am wrong, since he added the multi op feature, but
my understanding is "multi op" is branded from day one as the transaction
support for zookeeper (we even provide an API with exact name:
Transaction). If we use the traditional semantic for transaction in
database context, the ACID properties multi-op satisfies at least atomicity
and durability. So saying zookeeper does not support transaction seems a
strong argument that against the properties of multi-op and existing
literatures related to zookeeper. On the other side, typically bulk
operations does not support atomicity, which will not take care of rolling
back failed operations.
On Wed, Aug 14, 2019 at 9:33 AM Andor Molnar wrote:
> "it's said that ZooKeeper has a transaction mechanism”
>
> I’m still confused with this. ZooKeeper doesn’t support transactions to my
> best knowledge. It has a `multi` operation feature, but that’s more like a
> bulk operation, not transaction.
>
> "I want to tell ZooKeeper that the check and setData should be successful”
>
> I don’t think you can do that. ZK has no check-and-set support either.
>
> Maybe we should step back first and see what’s your use case exactly that
> you’re trying to solve with ZooKeeper. I suspect that you’re trying to
> follow the wrong approach or misusing ZooKeeper.
>
> Have you checked our tutorial and recipes page?
> You can find some recommended usage patterns:
> https://zookeeper.apache.org/doc/r3.5.5/recipes.html
> https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html
>
> If that’s not enough, you could also try Curator which has even more
> built-in high level functionalities on top of basic ZK commands.
>
> Andor
>
>
>
> > On 2019. Aug 14., at 17:52, Zili Chen wrote:
> >
> > Hi Andor,
> >
> > Thanks for your attention.
> >
> > The problem is that in concurrent scenario zk.setData() could still
> failed
> > if there is another thread delete the node. I know with proper lock
> strategy
> > and ownership separation this can be avoid but it's said that ZooKeeper
> has
> > a transaction mechanism so I'd like to see whether I can make use of it.
> >
> > There is where I turn to
> >
> > zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> path2
> > is irrelevant
> >
> > when the existence of a mark node(path1) guarded a condition and I want
> to
> > make
> > sure that setData successes only if the mark node exist. If I check the
> > existence
> > first and commit setData, a remove to the node could break the guard. In
> > other
> > words, I want to tell ZooKeeper that the check and setData should be
> > successful
> > committed or fail to be committed atomically.
> >
> > Best,
> > tison.
> >
> >
> > Andor Molnar 于2019年8月14日周三 下午11:12写道：
> >
> >> Hi Zili,
> >>
> >> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> >> think your multi example (zk.multi(Op.check(path), Op.setData(path,
> data)))
> >> is already a usage pattern which multi is not designed to support.
> >>
> >> Why do you need to do this in “transactions” (there’s no transaction in
> >> ZK)?
> >>
> >> In Java you can do:
> >>
> >> try {
> >> zk.create();
> >> } catch (NodeExistsException e) {
> >> // swallow exception
> >> }
> >> zk.setData();
> >> …
> >>
> >> Regards,
> >> Andor
> >>
> >>
> >>
> >>
> >>> On 2019. Aug 6., at 14:44, Zili Chen wrote:
> >>>
> >>> Hi Enrico,
> >>>
> >>> Thanks for your reply.
> >>>
> In this case usually you use conditional setData, using the 'version'
> of
> thr znode
> >>>
> >>>
> >>> what if the caller has no idea on whether the node exist?(see also
> >>> my if-else pseudo-code above.)
> >>>
> >>> IIRC if we call `setData` on a non-exist path a NoNodeException
> >>> will be thrown.
> >>> Best,
> >>> tison.
> >>>
> >>>
> >>> Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> >>>
> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
>
> > Any ideas?
> >
> >
> > Zili Chen 于2019年7月29日周一 上午11:12写道：
> >
> >> Hi ZooKeepers,
> >>
> >> Currently our transaction mechanism supports doing
> >> create/setData/checkExist/delete in transaction. However, taking
> this
> >> scenario into consideration, we want to put data in path "/path" but
> >> don't know whether the znode exists or not. Let's say we program as
> >> below
> >>
> >> if (zk.exist(path)) {
> >> zk.setData(path, data);
> >> } else {
> >> zk.create(path, data);
> >>
Re: create or setData in transaction?
"it's said that ZooKeeper has a transaction mechanism”
I’m still confused with this. ZooKeeper doesn’t support transactions to my best
knowledge. It has a `multi` operation feature, but that’s more like a bulk
operation, not transaction.
"I want to tell ZooKeeper that the check and setData should be successful”
I don’t think you can do that. ZK has no check-and-set support either.
Maybe we should step back first and see what’s your use case exactly that
you’re trying to solve with ZooKeeper. I suspect that you’re trying to follow
the wrong approach or misusing ZooKeeper.
Have you checked our tutorial and recipes page?
You can find some recommended usage patterns:
https://zookeeper.apache.org/doc/r3.5.5/recipes.html
https://zookeeper.apache.org/doc/r3.5.5/zookeeperTutorial.html
If that’s not enough, you could also try Curator which has even more built-in
high level functionalities on top of basic ZK commands.
Andor
> On 2019. Aug 14., at 17:52, Zili Chen wrote:
>
> Hi Andor,
>
> Thanks for your attention.
>
> The problem is that in concurrent scenario zk.setData() could still failed
> if there is another thread delete the node. I know with proper lock strategy
> and ownership separation this can be avoid but it's said that ZooKeeper has
> a transaction mechanism so I'd like to see whether I can make use of it.
>
> There is where I turn to
>
> zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or != path2
> is irrelevant
>
> when the existence of a mark node(path1) guarded a condition and I want to
> make
> sure that setData successes only if the mark node exist. If I check the
> existence
> first and commit setData, a remove to the node could break the guard. In
> other
> words, I want to tell ZooKeeper that the check and setData should be
> successful
> committed or fail to be committed atomically.
>
> Best,
> tison.
>
>
> Andor Molnar 于2019年8月14日周三 下午11:12写道：
>
>> Hi Zili,
>>
>> There’s no such functionality in ZooKeeper as far as I’m concerned. I
>> think your multi example (zk.multi(Op.check(path), Op.setData(path, data)))
>> is already a usage pattern which multi is not designed to support.
>>
>> Why do you need to do this in “transactions” (there’s no transaction in
>> ZK)?
>>
>> In Java you can do:
>>
>> try {
>> zk.create();
>> } catch (NodeExistsException e) {
>> // swallow exception
>> }
>> zk.setData();
>> …
>>
>> Regards,
>> Andor
>>
>>
>>
>>
>>> On 2019. Aug 6., at 14:44, Zili Chen wrote:
>>>
>>> Hi Enrico,
>>>
>>> Thanks for your reply.
>>>
In this case usually you use conditional setData, using the 'version' of
thr znode
>>>
>>>
>>> what if the caller has no idea on whether the node exist?(see also
>>> my if-else pseudo-code above.)
>>>
>>> IIRC if we call `setData` on a non-exist path a NoNodeException
>>> will be thrown.
>>> Best,
>>> tison.
>>>
>>>
>>> Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
>>>
Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
> Any ideas?
>
>
> Zili Chen 于2019年7月29日周一 上午11:12写道：
>
>> Hi ZooKeepers,
>>
>> Currently our transaction mechanism supports doing
>> create/setData/checkExist/delete in transaction. However, taking this
>> scenario into consideration, we want to put data in path "/path" but
>> don't know whether the znode exists or not. Let's say we program as
>> below
>>
>> if (zk.exist(path)) {
>> zk.setData(path, data);
>> } else {
>> zk.create(path, data);
>> }
>
Do you need to perform other ops in the same transaction?
In this case usually you use conditional setData, using the 'version' of
thr znode
Enrico
>
>> if we want to do the check and "put" in transaction, it would be like
>>
>> zk.multi(Op.check(path), Op.setData(path, data));
>>
>> but we cannot add a "else" branch. ZooKeeper's transaction would all
>> success or fail.
>>
>> Is there any way to do an "if-else" transaction?
>>
>> Best,
>> tison.
>>
>
>>
>>
Re: create or setData in transaction?
Il mer 14 ago 2019, 17:55 Zili Chen ha scritto:
> FYI, etcd provides a transaction mechanism which supports if-else like
> commits.[1]
>
As we have setData with version check usually you don't need a
'transaction' with some kind of rowlevel lock.
Maybe I am missing something in your case.
Enrico
> Best,
> tison.
>
> [1] https://godoc.org/github.com/coreos/etcd/clientv3#OpTxn
>
>
> Zili Chen 于2019年8月14日周三 下午11:52写道：
>
> > Hi Andor,
> >
> > Thanks for your attention.
> >
> > The problem is that in concurrent scenario zk.setData() could still
> failed
> > if there is another thread delete the node. I know with proper lock
> > strategy
> > and ownership separation this can be avoid but it's said that ZooKeeper
> has
> > a transaction mechanism so I'd like to see whether I can make use of it.
> >
> > There is where I turn to
> >
> > zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> > path2 is irrelevant
> >
> > when the existence of a mark node(path1) guarded a condition and I want
> to
> > make
> > sure that setData successes only if the mark node exist. If I check the
> > existence
> > first and commit setData, a remove to the node could break the guard. In
> > other
> > words, I want to tell ZooKeeper that the check and setData should be
> > successful
> > committed or fail to be committed atomically.
> >
> > Best,
> > tison.
> >
> >
> > Andor Molnar 于2019年8月14日周三 下午11:12写道：
> >
> >> Hi Zili,
> >>
> >> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> >> think your multi example (zk.multi(Op.check(path), Op.setData(path,
> data)))
> >> is already a usage pattern which multi is not designed to support.
> >>
> >> Why do you need to do this in “transactions” (there’s no transaction in
> >> ZK)?
> >>
> >> In Java you can do:
> >>
> >> try {
> >> zk.create();
> >> } catch (NodeExistsException e) {
> >> // swallow exception
> >> }
> >> zk.setData();
> >> …
> >>
> >> Regards,
> >> Andor
> >>
> >>
> >>
> >>
> >> > On 2019. Aug 6., at 14:44, Zili Chen wrote:
> >> >
> >> > Hi Enrico,
> >> >
> >> > Thanks for your reply.
> >> >
> >> >> In this case usually you use conditional setData, using the 'version'
> >> of
> >> >> thr znode
> >> >
> >> >
> >> > what if the caller has no idea on whether the node exist?(see also
> >> > my if-else pseudo-code above.)
> >> >
> >> > IIRC if we call `setData` on a non-exist path a NoNodeException
> >> > will be thrown.
> >> > Best,
> >> > tison.
> >> >
> >> >
> >> > Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> >> >
> >> >> Il mar 6 ago 2019, 13:47 Zili Chen ha
> scritto:
> >> >>
> >> >>> Any ideas?
> >> >>>
> >> >>>
> >> >>> Zili Chen 于2019年7月29日周一 上午11:12写道：
> >> >>>
> >> Hi ZooKeepers,
> >>
> >> Currently our transaction mechanism supports doing
> >> create/setData/checkExist/delete in transaction. However, taking
> this
> >> scenario into consideration, we want to put data in path "/path"
> but
> >> don't know whether the znode exists or not. Let's say we program as
> >> below
> >>
> >> if (zk.exist(path)) {
> >> zk.setData(path, data);
> >> } else {
> >> zk.create(path, data);
> >> }
> >> >>>
> >> >>
> >> >> Do you need to perform other ops in the same transaction?
> >> >> In this case usually you use conditional setData, using the 'version'
> >> of
> >> >> thr znode
> >> >>
> >> >>
> >> >> Enrico
> >> >>
> >> >>>
> >> if we want to do the check and "put" in transaction, it would be
> like
> >>
> >> zk.multi(Op.check(path), Op.setData(path, data));
> >>
> >> but we cannot add a "else" branch. ZooKeeper's transaction would
> all
> >> success or fail.
> >>
> >> Is there any way to do an "if-else" transaction?
> >>
> >> Best,
> >> tison.
> >>
> >> >>>
> >> >>
> >>
> >>
>
Re: create or setData in transaction?
FYI, etcd provides a transaction mechanism which supports if-else like
commits.[1]
Best,
tison.
[1] https://godoc.org/github.com/coreos/etcd/clientv3#OpTxn
Zili Chen 于2019年8月14日周三 下午11:52写道：
> Hi Andor,
>
> Thanks for your attention.
>
> The problem is that in concurrent scenario zk.setData() could still failed
> if there is another thread delete the node. I know with proper lock
> strategy
> and ownership separation this can be avoid but it's said that ZooKeeper has
> a transaction mechanism so I'd like to see whether I can make use of it.
>
> There is where I turn to
>
> zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or !=
> path2 is irrelevant
>
> when the existence of a mark node(path1) guarded a condition and I want to
> make
> sure that setData successes only if the mark node exist. If I check the
> existence
> first and commit setData, a remove to the node could break the guard. In
> other
> words, I want to tell ZooKeeper that the check and setData should be
> successful
> committed or fail to be committed atomically.
>
> Best,
> tison.
>
>
> Andor Molnar 于2019年8月14日周三 下午11:12写道：
>
>> Hi Zili,
>>
>> There’s no such functionality in ZooKeeper as far as I’m concerned. I
>> think your multi example (zk.multi(Op.check(path), Op.setData(path, data)))
>> is already a usage pattern which multi is not designed to support.
>>
>> Why do you need to do this in “transactions” (there’s no transaction in
>> ZK)?
>>
>> In Java you can do:
>>
>> try {
>> zk.create();
>> } catch (NodeExistsException e) {
>> // swallow exception
>> }
>> zk.setData();
>> …
>>
>> Regards,
>> Andor
>>
>>
>>
>>
>> > On 2019. Aug 6., at 14:44, Zili Chen wrote:
>> >
>> > Hi Enrico,
>> >
>> > Thanks for your reply.
>> >
>> >> In this case usually you use conditional setData, using the 'version'
>> of
>> >> thr znode
>> >
>> >
>> > what if the caller has no idea on whether the node exist?(see also
>> > my if-else pseudo-code above.)
>> >
>> > IIRC if we call `setData` on a non-exist path a NoNodeException
>> > will be thrown.
>> > Best,
>> > tison.
>> >
>> >
>> > Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
>> >
>> >> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
>> >>
>> >>> Any ideas?
>> >>>
>> >>>
>> >>> Zili Chen 于2019年7月29日周一 上午11:12写道：
>> >>>
>> Hi ZooKeepers,
>>
>> Currently our transaction mechanism supports doing
>> create/setData/checkExist/delete in transaction. However, taking this
>> scenario into consideration, we want to put data in path "/path" but
>> don't know whether the znode exists or not. Let's say we program as
>> below
>>
>> if (zk.exist(path)) {
>> zk.setData(path, data);
>> } else {
>> zk.create(path, data);
>> }
>> >>>
>> >>
>> >> Do you need to perform other ops in the same transaction?
>> >> In this case usually you use conditional setData, using the 'version'
>> of
>> >> thr znode
>> >>
>> >>
>> >> Enrico
>> >>
>> >>>
>> if we want to do the check and "put" in transaction, it would be like
>>
>> zk.multi(Op.check(path), Op.setData(path, data));
>>
>> but we cannot add a "else" branch. ZooKeeper's transaction would all
>> success or fail.
>>
>> Is there any way to do an "if-else" transaction?
>>
>> Best,
>> tison.
>>
>> >>>
>> >>
>>
>>
Re: create or setData in transaction?
Hi Andor,
Thanks for your attention.
The problem is that in concurrent scenario zk.setData() could still failed
if there is another thread delete the node. I know with proper lock strategy
and ownership separation this can be avoid but it's said that ZooKeeper has
a transaction mechanism so I'd like to see whether I can make use of it.
There is where I turn to
zk.multi(Op.check(path1), Op.setData(path2, data)); // path1 == or != path2
is irrelevant
when the existence of a mark node(path1) guarded a condition and I want to
make
sure that setData successes only if the mark node exist. If I check the
existence
first and commit setData, a remove to the node could break the guard. In
other
words, I want to tell ZooKeeper that the check and setData should be
successful
committed or fail to be committed atomically.
Best,
tison.
Andor Molnar 于2019年8月14日周三 下午11:12写道：
> Hi Zili,
>
> There’s no such functionality in ZooKeeper as far as I’m concerned. I
> think your multi example (zk.multi(Op.check(path), Op.setData(path, data)))
> is already a usage pattern which multi is not designed to support.
>
> Why do you need to do this in “transactions” (there’s no transaction in
> ZK)?
>
> In Java you can do:
>
> try {
> zk.create();
> } catch (NodeExistsException e) {
> // swallow exception
> }
> zk.setData();
> …
>
> Regards,
> Andor
>
>
>
>
> > On 2019. Aug 6., at 14:44, Zili Chen wrote:
> >
> > Hi Enrico,
> >
> > Thanks for your reply.
> >
> >> In this case usually you use conditional setData, using the 'version' of
> >> thr znode
> >
> >
> > what if the caller has no idea on whether the node exist?(see also
> > my if-else pseudo-code above.)
> >
> > IIRC if we call `setData` on a non-exist path a NoNodeException
> > will be thrown.
> > Best,
> > tison.
> >
> >
> > Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> >
> >> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
> >>
> >>> Any ideas?
> >>>
> >>>
> >>> Zili Chen 于2019年7月29日周一 上午11:12写道：
> >>>
> Hi ZooKeepers,
>
> Currently our transaction mechanism supports doing
> create/setData/checkExist/delete in transaction. However, taking this
> scenario into consideration, we want to put data in path "/path" but
> don't know whether the znode exists or not. Let's say we program as
> below
>
> if (zk.exist(path)) {
> zk.setData(path, data);
> } else {
> zk.create(path, data);
> }
> >>>
> >>
> >> Do you need to perform other ops in the same transaction?
> >> In this case usually you use conditional setData, using the 'version' of
> >> thr znode
> >>
> >>
> >> Enrico
> >>
> >>>
> if we want to do the check and "put" in transaction, it would be like
>
> zk.multi(Op.check(path), Op.setData(path, data));
>
> but we cannot add a "else" branch. ZooKeeper's transaction would all
> success or fail.
>
> Is there any way to do an "if-else" transaction?
>
> Best,
> tison.
>
> >>>
> >>
>
>
Re: create or setData in transaction?
Hi Zili,
There’s no such functionality in ZooKeeper as far as I’m concerned. I think
your multi example (zk.multi(Op.check(path), Op.setData(path, data))) is
already a usage pattern which multi is not designed to support.
Why do you need to do this in “transactions” (there’s no transaction in ZK)?
In Java you can do:
try {
zk.create();
} catch (NodeExistsException e) {
// swallow exception
}
zk.setData();
…
Regards,
Andor
> On 2019. Aug 6., at 14:44, Zili Chen wrote:
>
> Hi Enrico,
>
> Thanks for your reply.
>
>> In this case usually you use conditional setData, using the 'version' of
>> thr znode
>
>
> what if the caller has no idea on whether the node exist?(see also
> my if-else pseudo-code above.)
>
> IIRC if we call `setData` on a non-exist path a NoNodeException
> will be thrown.
> Best,
> tison.
>
>
> Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
>
>> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
>>
>>> Any ideas?
>>>
>>>
>>> Zili Chen 于2019年7月29日周一 上午11:12写道：
>>>
Hi ZooKeepers,
Currently our transaction mechanism supports doing
create/setData/checkExist/delete in transaction. However, taking this
scenario into consideration, we want to put data in path "/path" but
don't know whether the znode exists or not. Let's say we program as
below
if (zk.exist(path)) {
zk.setData(path, data);
} else {
zk.create(path, data);
}
>>>
>>
>> Do you need to perform other ops in the same transaction?
>> In this case usually you use conditional setData, using the 'version' of
>> thr znode
>>
>>
>> Enrico
>>
>>>
if we want to do the check and "put" in transaction, it would be like
zk.multi(Op.check(path), Op.setData(path, data));
but we cannot add a "else" branch. ZooKeeper's transaction would all
success or fail.
Is there any way to do an "if-else" transaction?
Best,
tison.
>>>
>>
Re: create or setData in transaction?
Hi Enrico,
Thanks for your reply.
>In this case usually you use conditional setData, using the 'version' of
>thr znode
what if the caller has no idea on whether the node exist?(see also
my if-else pseudo-code above.)
IIRC if we call `setData` on a non-exist path a NoNodeException
will be thrown.
Best,
tison.
Enrico Olivelli 于2019年8月6日周二 下午8:27写道：
> Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
>
> > Any ideas?
> >
> >
> > Zili Chen 于2019年7月29日周一 上午11:12写道：
> >
> > > Hi ZooKeepers,
> > >
> > > Currently our transaction mechanism supports doing
> > > create/setData/checkExist/delete in transaction. However, taking this
> > > scenario into consideration, we want to put data in path "/path" but
> > > don't know whether the znode exists or not. Let's say we program as
> > > below
> > >
> > > if (zk.exist(path)) {
> > > zk.setData(path, data);
> > > } else {
> > > zk.create(path, data);
> > > }
> >
>
> Do you need to perform other ops in the same transaction?
> In this case usually you use conditional setData, using the 'version' of
> thr znode
>
>
> Enrico
>
> >
> > > if we want to do the check and "put" in transaction, it would be like
> > >
> > > zk.multi(Op.check(path), Op.setData(path, data));
> > >
> > > but we cannot add a "else" branch. ZooKeeper's transaction would all
> > > success or fail.
> > >
> > > Is there any way to do an "if-else" transaction?
> > >
> > > Best,
> > > tison.
> > >
> >
>
Re: create or setData in transaction?
Il mar 6 ago 2019, 13:47 Zili Chen ha scritto:
> Any ideas?
>
>
> Zili Chen 于2019年7月29日周一 上午11:12写道：
>
> > Hi ZooKeepers,
> >
> > Currently our transaction mechanism supports doing
> > create/setData/checkExist/delete in transaction. However, taking this
> > scenario into consideration, we want to put data in path "/path" but
> > don't know whether the znode exists or not. Let's say we program as
> > below
> >
> > if (zk.exist(path)) {
> > zk.setData(path, data);
> > } else {
> > zk.create(path, data);
> > }
>
Do you need to perform other ops in the same transaction?
In this case usually you use conditional setData, using the 'version' of
thr znode
Enrico
>
> > if we want to do the check and "put" in transaction, it would be like
> >
> > zk.multi(Op.check(path), Op.setData(path, data));
> >
> > but we cannot add a "else" branch. ZooKeeper's transaction would all
> > success or fail.
> >
> > Is there any way to do an "if-else" transaction?
> >
> > Best,
> > tison.
> >
>
Re: create or setData in transaction?
Any ideas?
Zili Chen 于2019年7月29日周一 上午11:12写道：
> Hi ZooKeepers,
>
> Currently our transaction mechanism supports doing
> create/setData/checkExist/delete in transaction. However, taking this
> scenario into consideration, we want to put data in path "/path" but
> don't know whether the znode exists or not. Let's say we program as
> below
>
> if (zk.exist(path)) {
> zk.setData(path, data);
> } else {
> zk.create(path, data);
> }
>
> if we want to do the check and "put" in transaction, it would be like
>
> zk.multi(Op.check(path), Op.setData(path, data));
>
> but we cannot add a "else" branch. ZooKeeper's transaction would all
> success or fail.
>
> Is there any way to do an "if-else" transaction?
>
> Best,
> tison.
>
