Be careful of using compressed JSON, since it isn't splittable. JSON is also very slow for reading.
.. Owen On Tue, Oct 11, 2016 at 4:31 AM, Casey Stella <[email protected]> wrote: > I'd also tack on to this that the configuration for the hdfs writer should > be moved to zookeeper rather than done in flux, IMO > On Tue, Oct 11, 2016 at 07:20 Otto Fowler <[email protected]> wrote: > > > The storage format and retrieval from that format should be configurable, > > that is a ‘boundary’ for Metron so to speak. > > > > On October 10, 2016 at 16:15:12, [email protected] ([email protected]) > > wrote: > > > > Is there a specific reason why the JSON files stored in HDFS are not > > compressed? I looked for some related JIRAs and mail conversations but > > couldn't find this already mentioned. I'm wondering if there was a good > > enough of an argument to keep things uncompressed, or if the subject just > > hadn't been broached yet. > > > > Jon > > -- > > > > Jon > > >
