I’ve update the KIP with suggestions from Gunnar. I’d like to bring this up for a vote.
Brandon Brown > On Oct 22, 2020, at 12:53 PM, Brandon Brown <bran...@bbrownsound.com> wrote: > > Hey Gunnar, > > Those are great questions! > > 1) I went with it only selecting top level fields since it seems like that’s > the way most of the out of the box SMTS work, however I could see a lot of > value in it supporting nested fields. > 2) I had not thought about adding salt but I think that would be a valid > option as well. > > I think I’ll update the KIP to reflect those suggestions. One more, do you > think this should allow a regex for fields or stick with the explicit naming > of the fields? > > Thanks for the great feedback > > Brandon Brown > >> On Oct 22, 2020, at 12:40 PM, Gunnar Morling >> <gunnar.morl...@googlemail.com.invalid> wrote: >> >> Hey Brandon, >> >> I think that's an interesting idea, we got something as a built-in >> connector feature in Debezium, too [1]. Two questions: >> >> * Can "field" select nested fields, e.g. "after.email"? >> * Did you consider an option for specifying salt for the hash functions? >> >> --Gunnar >> >> [1] >> https://debezium.io/documentation/reference/connectors/mysql.html#mysql-property-column-mask-hash >> >> >> >>> Am Do., 22. Okt. 2020 um 12:53 Uhr schrieb Brandon Brown < >>> bran...@bbrownsound.com>: >>> >>> Gonna give this another little bump. :) >>> >>> Brandon Brown >>> >>>> On Oct 15, 2020, at 12:51 PM, Brandon Brown <bran...@bbrownsound.com> >>> wrote: >>>> >>>> >>>> As I mentioned in the KIP, this transformer is slightly different from >>> the current MaskField SMT. >>>> >>>>> Currently there exists a MaskField SMT but that would completely remove >>> the value by setting it to an equivalent null value. One problem with this >>> would be that you’d not be able to know in the case of say a password going >>> through the mask transform it would become "" which could mean that no >>> password was present in the message, or it was removed. However this hash >>> transformer would remove this ambiguity if that makes sense. The proposed >>> hash functions would be MD5, SHA1, SHA256. which are all supported via >>> MessageDigest. >>>> >>>> Given this take on things do you still think there would be value in >>> this smt? >>>> >>>> >>>> Brandon Brown >>>>> On Oct 15, 2020, at 12:36 PM, Ning Zhang <ning2008w...@gmail.com> >>> wrote: >>>>> >>>>> Hello, I think this SMT feature is parallel to >>> https://docs.confluent.io/current/connect/transforms/index.html >>>>> >>>>>>> On 2020/10/15 15:24:51, Brandon Brown <bran...@bbrownsound.com> >>> wrote: >>>>>> Bumping this thread. >>>>>> Please take a look at the KIP and vote or let me know if you have any >>> feedback. >>>>>> >>>>>> KIP: >>> https://cwiki.apache.org/confluence/display/KAFKA/KIP-665%3A+Kafka+Connect+Hash+SMT >>>>>> >>>>>> Proposed: https://github.com/apache/kafka/pull/9057 >>>>>> >>>>>> Thanks >>>>>> >>>>>> Brandon Brown >>>>>> >>>>>>>> On Oct 8, 2020, at 10:30 PM, Brandon Brown <bran...@bbrownsound.com> >>> wrote: >>>>>>> >>>>>>> Just wanted to give another bump on this and see if anyone had any >>> comments. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> Brandon Brown >>>>>>> >>>>>>>> On Oct 1, 2020, at 9:11 AM, "bran...@bbrownsound.com" < >>> bran...@bbrownsound.com> wrote: >>>>>>>> >>>>>>>> Hey Kafka Developers, >>>>>>>> >>>>>>>> I’ve created the following KIP and updated it based on feedback from >>> Mickael. I was wondering if we could get a vote on my proposal and move >>> forward with the proposed pr. >>>>>>>> >>>>>>>> Thanks so much! >>>>>>>> -Brandon >>>>>> >>>
