On Thu, Feb 2, 2017 at 9:28 AM, Justin Bertram <[email protected]> wrote: >> For large stores I think we will need to do something here in terms of >> compression otherwise we could end up with huge XML files. > > What specifically are your concerns and ideas here? Storage appears cheap > and plentiful and files can be compressed after they are created with gzip or > similar. Also, huge message stores are (IMO) an anti-pattern. A message > broker is not a database after all. What use-cases do you have in mind?
I am not sure what CShanon had in mind, but we could (I"m not saying yet we should.. just listing the possibility): - import from KahaDB and do directly what the importer is doing. Instead of dumping an XML, it could add directly to the broker running from the kahaDB storage. I don't mind a huge XML (or compressed gz in such way the importer would read compressed). But it could be an option eventually. @CShanon: I wouldn't do the directly unless you are aware of user demand, meaning the XML is not good for certain cases.
