[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15666811#comment-15666811
]
ASF GitHub Bot commented on IGNITE-3191:
Github user asfgit closed the pull request at:
https://github.com/apache/ignite/pull/1197
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
> Components: binary
>Affects Versions: 1.7
>Reporter: Denis Magda
>Assignee: Vladimir Ozerov
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15663250#comment-15663250
]
Vladimir Ozerov commented on IGNITE-3191:
-
Fixed review comments. Waiting for test results.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Affects Versions: 1.7
>Reporter: Denis Magda
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15622260#comment-15622260
]
Taras Ledkov commented on IGNITE-3191:
--
[Test
results|http://195.239.208.174/project.html?projectId=IgniteTests=projectOverview_IgniteTests=pull%2F1197%2Fhead]
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621888#comment-15621888
]
Taras Ledkov commented on IGNITE-3191:
--
According with the IGNITE-1377 Vladimir's fix is properly. We cannot update
metadata on deserialization.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15621775#comment-15621775
]
Taras Ledkov commented on IGNITE-3191:
--
Root cause:
at the {{BinaryContext.registerUserClassDescriptor}} metadata is not updated on
deserialize.
So, the fail case is:
1. *New node* writes object to the cache, update system cache & local maps with
*new-schema*;
2. *Old node* reads the object from binary. Create local
{{BinaryClassDescriptor}} with *old-schema* as a stable schema. *New-chema* is
available at the schemes registry. Does't update metadata (doesn't merge
*old-schema* & *new-schema*);
3. *Old node* writes object to the cache. The new {{BinaryClassDescriptor}}
isn't created because the descriptor is added to the local maps at the *step*
2. The stable *old-schema* is used.
4. *New-node* cannot deserialize the object with *old-shema*.
The suggestion fix from Vladimir updates metadata on the *step 3*.
So, in case the object has been created on the old node then the object is
created on the new-node all works properly because new node updates the system
cache with *new-schema*.
Conclusion:
1. I guess we have to fix the metadata & system caches update on the *step 2*
(when the local instance of the {{BinaryClassDescriptor}} is created the
first.) to prevent this problem in the future.
2. The fix for fields ordering should be switched by system property until the
backward compatibility is required.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Taras Ledkov
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15497191#comment-15497191
]
Valentin Kulichenko commented on IGNITE-3191:
-
Vladimir,
Agree. Do you have any thoughts on how we can address it?
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Valentin Kulichenko
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15493094#comment-15493094
]
Vladimir Ozerov commented on IGNITE-3191:
-
In general changes looks fine in terms that they should not break logic and
performance. But this appears to be a partial fix to me. We have the same issue
with {{Binarilyzable}}. Pseudo code:
{code}
class MyClass {
writeBinary(Writer w) {
w.write("B", b);
w.write("A", a);
}
}
{code}
And then:
{code}
BinaryObject obj1 = toBinary(new MyClass(a, b)); // Not sorted
BinaryObject obj2 = builder("MyClass").setField("B", b).setField("A",
a).build(); // Sorted
assertEquals(obj1, obj2); // Returns "false".
{code}
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Valentin Kulichenko
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491705#comment-15491705
]
Valentin Kulichenko commented on IGNITE-3191:
-
Made changes in branch {{ignite-3191}}.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Assignee: Valentin Kulichenko
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15491637#comment-15491637
]
Valentin Kulichenko commented on IGNITE-3191:
-
Another issue caused by using {{LinkedHashMap}} in the builder is wrong equals.
If the same set of fields is added with the same values, but in different
order, objects will not be equal, because we compare byte arrays.
We should use {{TreeMap}} in builder and also sort fields in the
{{BinaryClassDescriptor}}, so that serialization of a POJO is also consistent.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
>Priority: Critical
> Fix For: 1.8
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15355072#comment-15355072
]
Denis Magda commented on IGNITE-3191:
-
[~vozerov], the ordered structure makes sense because otherwise the user need
to preserve order on how the fields are set during an object construction. If
the order varies from time to time it will lead to the situation when a new
scheme is created for every unique combination.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
> Fix For: 1.7
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15309970#comment-15309970
]
Vladimir Ozerov commented on IGNITE-3191:
-
{{TreeMap}} is slower than {{LinkedHashMap}}, and we didn't consider many
distinct field combinations as a common use case.
Moreover, even now I don't understand well why we should bother with changing
everything from unordered to ordered structure.
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
> Fix For: 1.7
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (IGNITE-3191) BinaryObjectBuilder: binary schema id depends on the order of fields addition
[
https://issues.apache.org/jira/browse/IGNITE-3191?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15308960#comment-15308960
]
Dmitriy Setrakyan commented on IGNITE-3191:
---
Any reason why {{LinkedHashMap}} was chosen initially?
> BinaryObjectBuilder: binary schema id depends on the order of fields addition
> -
>
> Key: IGNITE-3191
> URL: https://issues.apache.org/jira/browse/IGNITE-3191
> Project: Ignite
> Issue Type: Bug
>Reporter: Denis Magda
> Fix For: 1.7
>
>
> Presently if an object is created using BinaryObjectBuilder then several
> BinarySchemes can be generated for the same set of fields in case when fields
> are added in a different order.
> This happens because {{LinkedHashMap}} is used underneath. However we should
> rely on order-free structure like {{TreeMap}}.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
