On Mon, Nov 11, 2024 at 11:46 AM Victor Cherviakov <[email protected]> wrote:
> Hi guys, I'm pretty new to using protobuf and maybe this question has been > answered multiple times, but > is there a way to check for messages duplications and drop them before > sending them over socket/wire? > > The thing is, my code sometimes generating messages of the same type with > same field values (identical messages) and I want to skip sending those > duplicated message. > > I know that protobuf is unhashable. However, if all the messages are > generated *within same process*, is it possible that: > msg1.SerializeToString() != msg2.SerializeToString() ? > Yes, you can have two equivalent messages that serialize to different bytes. There are many reasons for this. Of course, identical bytes represent the same content. Serialization is non-deterministic, and even if you force determinism it is non-canonical. You can read more about it here: https://protobuf.dev/programming-guides/serialization-not-canonical/ My idea was to use serialized values as dict key and the message as a > value, so I won't have problems with same messages. > If you use the serialized bytes as a key in a cache you will have false negatives and might end up with extra entries representing the same message. The best way might be to create a key yourself with all the fields that matter to you. You could use reflection for this for a more generic approach. > > P.S. I am also trying to remove possibility of duplicate message > generating, however I am not there yet > > Thanks! > > > -- > You received this message because you are subscribed to the Google Groups > "Protocol Buffers" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion visit > https://groups.google.com/d/msgid/protobuf/1c7ad210-742b-488c-be9b-bfa11ed9d75an%40googlegroups.com > <https://groups.google.com/d/msgid/protobuf/1c7ad210-742b-488c-be9b-bfa11ed9d75an%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "Protocol Buffers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion visit https://groups.google.com/d/msgid/protobuf/CAM9aRsKzUQHAKf1k-QNUAbpqXb5kLK-yhyzZ5g6AqL5%3D%2BYJK5g%40mail.gmail.com.
