Hi Sean thanks for the additional info. This is really helpful. Some of these constraints are not what I would prefer... and of course mean more work. Powers that be request them ;) Thanks On May 6, 2014 9:06 AM, "Sean Busbey" <[email protected]> wrote:
> Hi Lewis, > > Well, I'm not sure what you're going to gain by isolating each file into > its own namespace. > > However, one way you can achieve that is to take the information you want > included from before and then create a hash of it. You can use the hex > representation of the hash to give you something that > > * meets Avro's requirements for namespace names > * uniquely identifies each namespace based on experiment, channel, and > timestamp > > If you need the file to also contain the information that went into making > that hash, you can attach it as Avro-ignored metadata on the schema. > > -Sean > > > On Mon, May 5, 2014 at 6:49 PM, Lewis John Mcgibbney < > [email protected]> wrote: > >> Hi Sean, >> Arggggh... >> Maybe I can propose a change to the namespace semantics in my project. >> I've been given the following brief. >> Data files should have a logical namespace structure which can be >> globally unified across multiple instruments and receiver platforms into a >> common namespace on the filesystem. >> I have a whole list of other data representation constraints and Avro >> satisfies them all. This one is the only 'issue'. >> Thanks for your help. >> Lewis >> On May 5, 2014 4:21 PM, "Sean Busbey" <[email protected]> wrote: >> >>> Hi Lewis! >>> >>> Avro namespaces don't allow the characters '/', ':', or '-'. So your >>> specific example would not work. The allowed characters for a namespace >>> are defined in the Avro spec[1]. >>> >>> It would help if you could clarify what purpose namespacing serves in >>> the system. >>> >>> [1]: http://avro.apache.org/docs/current/spec.html#Names >>> >>> >>> On Mon, May 5, 2014 at 6:14 PM, Lewis John Mcgibbney < >>> [email protected]> wrote: >>> >>>> Hi Charles >>>> Thanks for reply. >>>> Re: more info >>>> Say I want my namespace to be >>>> >>>> experiment_name-{timestamp}/channel_name/2014-05-05T16:13:29/ >>>> >>>> Where all constituent parts of namespace separated by '/' are >>>> dynamic... is this possible via builder api? >>>> Thanks again >>>> On May 5, 2014 4:04 PM, "Pritchard, Charles X. -ND" < >>>> [email protected]> wrote: >>>> >>>>> Need more info — you can use the Schema builder to do anything you >>>>> like at runtime, >>>>> Schema.createRecord and setFields. >>>>> >>>>> >>>>> >>>>> On May 5, 2014, at 3:54 PM, Lewis John Mcgibbney < >>>>> [email protected]> wrote: >>>>> >>>>> Hi Folks, >>>>> I'm trying to propose Afro for a project I've been drafted on to. >>>>> Namespace declarations are important... and a requirement is that >>>>> namespaces are dynamic in nature... preferably even decided at runtime. >>>>> Is this possible in Avro? >>>>> Thanks >>>>> Lewis >>>>> >>>>> >>>>> >>> > > > -- > Sean >
