I'm considering a new Moses branch that would allow users to create proprietary binarized tables that only work in their deployments. The purpose is to protect data models from use with unauthorized systems. The project:
1. after creating binarized models with the processPhraseTable and processLexicalTable steps, would encode the binarized data tables to a proprietary format 2. would modify the Moses PhraseDictionaryTreeAdaptor (PhraseDictionaryTree & other as required) to a) detect open or proprietary format b) if open format, execute to existing code, if proprietary execute branched code to decode proprietary format for every transaction and lookup 3. publish static mods to Moses as open source. Static mods could, however, could point to dynamic linked libraries that could be proprietary. Questions: A) Does a branch like this exist or has anyone considered it? B) What problems, if any, would this create in the smart pointer's ability to search and find what it's looking for? C) Encoding the .xxxtree and .xxx data files would require decoding with every lookup... potentially degrading performance. Instead, would it be possible to encode/decode only the .xxxvoc and .idx files to achieve the same level of protection without the performance hit of decoding the data and tree for each lookup? Thanks, Tom ---------- Forwarded message ---------- From: [email protected] < [email protected]> Date: Fri, Feb 20, 2009 at 6:53 PM Subject: Re: [Moses-support] zlib compression and binarized tables To: Adam Lopez <[email protected]> Excellent. Exactly what I was looking for. Thank you. Tom On Fri, Feb 20, 2009 at 6:42 PM, Adam Lopez <[email protected]> wrote: Yes, though not directly. PhraseDictionaryTreeAdaptor forwards its calls to PDAImp, which is a wrapper on PhraseDictionaryTree, which uses a PrefixTreeF template class. PrefixTreeF reads binary files using a smart pointer defined in FilePtr.h. Adam On Fri, Feb 20, 2009 at 11:25 AM, [email protected] <[email protected]> wrote: > Thanks Adam. > > Following up, it looks like PhraseDictionaryTreeAdaptor.cpp is responsbile > for reading the binarized phrase tables and language model. Is this correct? > > > Tom
_______________________________________________ Moses-support mailing list [email protected] http://mailman.mit.edu/mailman/listinfo/moses-support
