Hello, Anton. > or use the POJO from the ignite core.
How it differs from the current implementation? > As I can see you have tests only for one node but what happens if different > nodes have different filters? The error will happen. Please, don’t forget that we are talking about two scenarios: 1. Blocker bug - we delete some class that was written to metastore from the new version. 2. Migration between custom Ignite distributions where one of them has a custom class and the other doesn’t. It’s a very rare incident in my experience. Why do you think that «good» solution should exist for this kind of issue? I don’t think we should limit internal architecture to cover these cases. Maybe we should make metastore fully lazy, so any stored key will not be deserialized before it explicitly queried. > 22 дек. 2020 г., в 14:30, Anton Kalashnikov <kaa....@yandex.ru> написал(а): > > Hello everyone, > > In my opinion, it is indeed better to limit storing to the metastore by > primitive type(map or list are also possible) or use the POJO from the ignite > core. > As Kirill correctly notice, right now, it is a problem not inside of the > distributed metastore but inside of discovery. > In fact, we can rewrite the sending metastore data in such a way that the > discovery would think that there is just a simple byte array which shouldn't > be deserialized. Right now, it understands that it is a serialized java > object and it tries to deserialize it after receiving it. But this way > requires more investigation about possible corner cases. > > Nikolay, I also don't sure that your fix handles metastorage history > correctly. > As I can see you have tests only for one node but what happens if different > nodes have different filters? > or if we need to send history to the joining node but some of the keys don't > pass the filter? > Maybe I wrong but in the first eye, it can lead to different > results/histories on different nodes which is a problem. > I just briefly looked at your PR(so maybe I didn't understand something), I > will try to do it more carefully at the nearest time. > > -- > Best regards, > Anton Kalashnikov > > > > 18.12.2020, 15:33, "Mekhanikov Denis" <dmekhani...@gmail.com>: >> Nikolay, >> >> Thanks for your reply! >> >> I encountered a similar case to what you've described in point #1. I used a >> private plugin that writes some information to the metastorage. >> After that I decided to get rid of that plugin, while the information had >> already been written to the metastorage. >> Following the approach that you described and implemented in the PR, I'll >> need to work with the flag to ignore certain keys in the metastorage >> forever. That's quite inconvenient. >> Wouldn't it be better if we just limited the set of allowed types that can >> be stored in the metastorage? Instead of a POJO, a Map will be accepted. >> >> Denis >> >> On 18.12.2020, 13:59, "ткаленко кирилл" <tkalkir...@yandex.ru> wrote: >> >> Hello everybody! >> >> If you look at the stackTrace, the error is that deserialized objects >> are being sent to listeners. >> It may be more correct to send a raw arrays of bytes, and each plugin >> will be able to process it if needed. >> >> 18.12.2020, 12:18, "Nikolay Izhikov" <nizhi...@apache.org>: >> > Hello, Denis. >> > >> > It’s a known issue for me. >> > Metastore is a private API, isn’t it? >> > AFAICU it can occur for two reasons: >> > >> > * User migrates from custom Ignite fork that has private improvements >> or plugins that write to the metastore. >> > * We have a blocker bug and just remove some internal class that can >> be written into metastore from distribution. >> > >> > I planned to fix it with some system flag. >> > During startup administrator just sets a list of the metastore items >> that should be ignored. >> > Please, take a look at the PR [1] >> > >> > WDYT? >> > >> >> it’s better to limit the metastorage with storing primitives only >> > >> > I think that ability to write object is very useful and should stay. >> > >> > [1] https://github.com/apache/ignite/pull/8221 >> > >> >> 18 дек. 2020 г., в 12:06, Mekhanikov Denis <dmekhani...@gmail.com> >> написал(а): >> >> >> >> Hi everyone! >> >> >> >> Ignite has a limitation that it can’t work with custom classes put >> into metastorage: https://issues.apache.org/jira/browse/IGNITE-13642 >> >> If you put a POJO into the metastorage, then Ignite will try to >> deserialize it using the classes it finds on the classpath. If it can’t do >> the deserialization, then the node will fail. >> >> >> >> There is an opinion that the metastorage wasn’t design for a case >> when classes that can disappear from Ignite distribution. >> >> If we follow this path, then it’s better to limit the metastorage >> with storing primitives only, so that it’s impossible to occasionally put >> anything breaking. >> >> If a piece of configuration is put into the metastorage by a plugin, >> then the plugin will be in charge of deserializing the configuration, and >> not Ignite. >> >> >> >> Alternatively we can try to fix the metastorage and make it ignore >> deserialization errors when they occur. >> >> >> >> What do you think? >> >> >> >> Denis
