I wrote up an Improvement Jira to add the property to Validate Field Names to AvroSchemaRegistry: https://issues.apache.org/jira/browse/NIFI-4612
-Matt On Thu, Nov 16, 2017 at 9:15 AM, Matt Burgess <[email protected]> wrote: > Mike, > > That's a very good point, I guess that would have to be emulated > somehow. I wonder how Linkedin and Coursera (who use a Scala binding > for Pegasus called Courier) handle all that? > > Anyway as it turns out Schema.Parser() has a method called > setValidate(boolean) [1] which can enable/disable name validation. > Perhaps we should disable it or add a property to AvroSchemaRegistry > to allow it to be disabled? > > Regards, > Matt > > [1] > https://avro.apache.org/docs/1.8.2/api/java/org/apache/avro/Schema.Parser.html#setValidate(boolean) > > On Wed, Nov 15, 2017 at 5:29 PM, Mike Thomsen <[email protected]> wrote: >> Might want to be careful about that because Avro 1.8 added the support for >> logical types and removing that could break parts of the Record API like the >> date/timestamp functionality. >> >> On Wed, Nov 15, 2017 at 3:41 PM, Matt Burgess <[email protected]> wrote: >>> >>> Thomas, >>> >>> You can file an Improvement or New Feature Jira [1] asking for the >>> enhancement. Ironically Avro 1.4.0's Schema.parse() method does allow >>> the dollar sign, but we use Avro 1.8.x now which is apparently more >>> strict. I am toying around with a PegasusSchemaRegistry using the >>> PegasusSchemaParser from Linkedin's rest.li project [2]. It is very >>> much like Avro but has some nice differences [3] and the parser >>> doesn't choke on invalid Avro identifiers. Another alternative (in a >>> separate NAR) is to provide a LessStrictAvroSchemaRegistry that is >>> exactly like AvroSchemaRegistry but bundles Avro 1.4.0 instead of >>> 1.8.2 (and uses Schema.parse() rather than new >>> Schema.Parser().parse()). >>> >>> Regards, >>> Matt >>> >>> [1] https://issues.apache.org/jira/projects/NIFI >>> [2] https://github.com/linkedin/rest.li >>> [3] >>> https://github.com/linkedin/rest.li/wiki/DATA-Data-Schema-and-Templates >>> >>> On Wed, Nov 15, 2017 at 3:32 PM, DENIMAL Thomas >>> <[email protected]> wrote: >>> > Hello Mike, >>> > >>> > >>> > >>> > Thanks for your answer. >>> > >>> > Do you know how can i ask for an enhancement request for this feature? >>> > >>> > >>> > >>> > >>> > >>> > Regards >>> > >>> > Thomas DENIMAL >>> > >>> > >>> > >>> > >>> > >>> > >>> > >>> > De : Mike Thomsen [mailto:[email protected]] >>> > Envoyé : mercredi 15 novembre 2017 19:58 >>> > À : [email protected] >>> > Objet : Re: NIFI 1.4.0 - PutMongo - How to use composite key for "Update >>> > Query key" parameter ? >>> > >>> > >>> > >>> > Also, I'll use this thread to bring something about PutMongoRecord into >>> > the >>> > mailing list for other Mongo users to see. >>> > >>> > PutMongoRecord cannot support more than document replacement updates >>> > because >>> > the Mongo update operators use "$" which is an illegal starting >>> > character in >>> > Avro. So you cannot define a Record API-compliant schema that >>> > corresponds to >>> > a partial update operation. You also, unfortunately, cannot do that with >>> > extended JSON features like date handling. You have to use the Avro and >>> > Record API-based methods for representing a Mongo date field when >>> > handling >>> > records. >>> > >>> > >>> > >>> > On Wed, Nov 15, 2017 at 1:32 PM, DENIMAL Thomas >>> > <[email protected]> >>> > wrote: >>> > >>> > No one with the same issue ? >>> > >>> > On 2017-11-04 14:18, DENIMAL Thomas <[email protected]> wrote: >>> >> Hello ,> >>> >> I'm using NIFI 1.4.0 with Mongodb 3.2.17.> >>> >> >>> >> >>> >> >>> >> Is it possible to use a composite key for "Update Query key" parameter >>> >> with PutMongo processor ?> >>> >> My collection "primary key" is a composite key of 2 columns, SIREN & >>> >> NIC, >>> >> (I'm using a composite unique index to ensure uniqueness)> >>> >> >>> >> I want to be able to make upsert operations on that collection with >>> >> Nifi.> >>> >> >>> >> Thanks for your help> >>> >> Regards> >>> >> Thomas DENIMAL> >>> >> >>> > >>> > >> >>
