RE: Exception during exception handling
I don’t know how fix deserialization, it seems very internal. But I can do
something for exception handling, but only on weekend.
Andrey.
От: Denis Magda
Отправлено: 26 сентября 2019 г. в 0:30
Кому: user@ignite.apache.org
Копия: Ilya Kasnacheev
Тема: Re: Exception during exception handling
Andrey,
Would you mind sending a pull-request if you have a clear understanding of how
this needs to be fixed?
-
Denis
On Wed, Sep 25, 2019 at 2:06 PM Andrey Davydov wrote:
There are two root causes.
The first one is deserialization of binary object for “toString” when it is
impossible to deserialize (like IGNITE-12178). I saw stackowerflow discussion
about fails on debug logging while googling current problem.
The second, but it may be more important, it is unsafe exception handling,
information about origin error that cause transaction fail should not be lost
anyway. It seems “try finaly” block should be implemented inside “catch”
section. Because we have chance to get exception during exception handling
anyway.
Andrey.
От: Denis Magda
Отправлено: 25 сентября 2019 г. в 22:36
Кому: user@ignite.apache.org; Ilya Kasnacheev
Тема: Re: Exception during exception handling
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov
wrote:
In ignite 2.7.5 I got following exception:
org.apache.ignite.IgniteException: Failed to create string representation of
binary object.
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
~[ignite-core-2.7.5.jar:2.7.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
at java.lang.StringBuilder.append(StringBuilder.java:131)
~[?:1.8.0_222]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
~[ignite-core-2.7.5.jar:2.7.5]….
As I found in sources, it was error during handling of other error. So this
error masks real problem.
See
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
lines 934-944:
catch (Throwable ex) {
// We are about to initiate transaction rollback when
tx has started to committing.
// Need to remove version from committed list.
cctx.tm().removeCommittedTx(this);
boolean isNodeStopping = X.hasCause(ex,
NodeStoppingException.class);
boolean hasInvalidEnvironmentIssue = X.hasCause(ex,
InvalidEnvironmentException.class);
IgniteCheckedException err0 = new
IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
"(all transaction entries will be invalidated,
however there was a window when " +
"entries for this transaction were visible to
others): " + this, ex);
Exception occurs during IgniteTxHeuristicCheckedException creation. So we got
unhandled error and server halt instead of IgniteTxHeuristicCheckedException
and rollback logic.
And the main issue: we lost any information about origin problem.
Andrey.
Re: Exception during exception handling
Andrey,
Would you mind sending a pull-request if you have a clear understanding of
how this needs to be fixed?
-
Denis
On Wed, Sep 25, 2019 at 2:06 PM Andrey Davydov
wrote:
> There are two root causes.
>
>
>
> The first one is deserialization of binary object for “toString” when it
> is impossible to deserialize (like IGNITE-12178). I saw stackowerflow
> discussion about fails on debug logging while googling current problem.
>
>
>
> The second, but it may be more important, it is unsafe exception
> handling, information about origin error that cause transaction fail should
> not be lost anyway. It seems “try finaly” block should be implemented
> inside “catch” section. Because we have chance to get exception during
> exception handling anyway.
>
>
>
> Andrey.
>
>
>
> *От: *Denis Magda
> *Отправлено: *25 сентября 2019 г. в 22:36
> *Кому: *user@ignite.apache.org; Ilya Kasnacheev
> *Тема: *Re: Exception during exception handling
>
>
>
> Hello Andrey and thanks for reporting!
>
>
>
> This reminds me of this issue that has a similar stack trace:
>
> https://issues.apache.org/jira/browse/IGNITE-12178
>
>
>
> Ilya, looks like the root cause is absolutely the same, doesn't it?
>
>
> -
>
> Denis
>
>
>
>
>
> On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov
> wrote:
>
> In ignite 2.7.5 I got following exception:
>
>
>
> org.apache.ignite.IgniteException: Failed to create string representation
> of binary object.
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
>
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> ~[?:1.8.0_222]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
> ~[ignite-core-2.7.5.jar:2.7.5]….
>
>
>
> As I found in sources, it was error during handling of other error. So
> this error masks real problem.
>
> See
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
> lines 934-944:
>
>
>
> catch (Throwable ex) {
>
> // We are about to initiate transaction rollback
> when tx has started to committing.
>
> // Need to remove version from committed list.
>
> cctx.tm().removeCommittedTx(this);
>
>
>
> boolean isNodeStopping = X.hasCause(ex,
> NodeStoppingException.class);
>
> boolean hasInvalidEnvironmentIssue =
> X.hasCause(ex, InvalidEnvironmentException.class);
>
>
>
> IgniteCheckedException err0 = new
> IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
>
> "(all transaction entries will be invalidated,
> however there was a window when " +
>
> "entries for this transaction were visible to
> others): " + this, ex);
>
>
>
> Exception occurs during IgniteTxHeuristicCheckedException creation. So we
> got unhandled error and server halt instead of
> IgniteTxHeuristicCheckedException
> and rollback logic.
>
>
>
> And the main issue: we lost any information about origin problem.
>
>
>
> Andrey.
>
>
>
>
>
RE: Exception during exception handling
There are two root causes.
The first one is deserialization of binary object for “toString” when it is
impossible to deserialize (like IGNITE-12178). I saw stackowerflow discussion
about fails on debug logging while googling current problem.
The second, but it may be more important, it is unsafe exception handling,
information about origin error that cause transaction fail should not be lost
anyway. It seems “try finaly” block should be implemented inside “catch”
section. Because we have chance to get exception during exception handling
anyway.
Andrey.
От: Denis Magda
Отправлено: 25 сентября 2019 г. в 22:36
Кому: user@ignite.apache.org; Ilya Kasnacheev
Тема: Re: Exception during exception handling
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov
wrote:
In ignite 2.7.5 I got following exception:
org.apache.ignite.IgniteException: Failed to create string representation of
binary object.
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
~[ignite-core-2.7.5.jar:2.7.5]
at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
at java.lang.StringBuilder.append(StringBuilder.java:131)
~[?:1.8.0_222]
at
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
~[ignite-core-2.7.5.jar:2.7.5]
at
org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
~[ignite-core-2.7.5.jar:2.7.5]….
As I found in sources, it was error during handling of other error. So this
error masks real problem.
See
org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
lines 934-944:
catch (Throwable ex) {
// We are about to initiate transaction rollback when
tx has started to committing.
// Need to remove version from committed list.
cctx.tm().removeCommittedTx(this);
boolean isNodeStopping = X.hasCause(ex,
NodeStoppingException.class);
boolean hasInvalidEnvironmentIssue = X.hasCause(ex,
InvalidEnvironmentException.class);
IgniteCheckedException err0 = new
IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
"(all transaction entries will be invalidated,
however there was a window when " +
"entries for this transaction were visible to
others): " + this, ex);
Exception occurs during IgniteTxHeuristicCheckedException creation. So we got
unhandled error and server halt instead of IgniteTxHeuristicCheckedException
and rollback logic.
And the main issue: we lost any information about origin problem.
Andrey.
Re: Exception during exception handling
Hello Andrey and thanks for reporting!
This reminds me of this issue that has a similar stack trace:
https://issues.apache.org/jira/browse/IGNITE-12178
Ilya, looks like the root cause is absolutely the same, doesn't it?
-
Denis
On Wed, Sep 25, 2019 at 10:02 AM Andrey Davydov
wrote:
> In ignite 2.7.5 I got following exception:
>
>
>
> org.apache.ignite.IgniteException: Failed to create string representation
> of binary object.
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toStringImpl(GridToStringBuilder.java:1022)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:762)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.util.tostring.GridToStringBuilder.toString(GridToStringBuilder.java:710)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.toString(IgniteTxLocalAdapter.java:1645)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.toString(GridDhtTxLocalAdapter.java:944)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocal.toString(GridDhtTxLocal.java:650)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at java.lang.String.valueOf(String.java:2994) ~[?:1.8.0_222]
>
> at java.lang.StringBuilder.append(StringBuilder.java:131)
> ~[?:1.8.0_222]
>
> at
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter.userCommit(IgniteTxLocalAdapter.java:942)
> ~[ignite-core-2.7.5.jar:2.7.5]
>
> at
> org.apache.ignite.internal.processors.cache.distributed.dht.GridDhtTxLocalAdapter.localFinish(GridDhtTxLocalAdapter.java:796)
> ~[ignite-core-2.7.5.jar:2.7.5]….
>
>
>
> As I found in sources, it was error during handling of other error. So
> this error masks real problem.
>
> See
> org.apache.ignite.internal.processors.cache.transactions.IgniteTxLocalAdapter
> lines 934-944:
>
>
>
> catch (Throwable ex) {
>
> // We are about to initiate transaction rollback
> when tx has started to committing.
>
> // Need to remove version from committed list.
>
> cctx.tm().removeCommittedTx(this);
>
>
>
> boolean isNodeStopping = X.hasCause(ex,
> NodeStoppingException.class);
>
> boolean hasInvalidEnvironmentIssue =
> X.hasCause(ex, InvalidEnvironmentException.class);
>
>
>
> IgniteCheckedException err0 = new
> IgniteTxHeuristicCheckedException("Failed to locally write to cache " +
>
> "(all transaction entries will be invalidated,
> however there was a window when " +
>
> "entries for this transaction were visible to
> others): " + this, ex);
>
>
>
> Exception occurs during IgniteTxHeuristicCheckedException creation. So we
> got unhandled error and server halt instead of
> IgniteTxHeuristicCheckedException
> and rollback logic.
>
>
>
> And the main issue: we lost any information about origin problem.
>
>
>
> Andrey.
>
>
>
