Hi Fabian, For now the data I've been working is plain text, so it is easy to visualize how splitting the content and distributing the enhancement process can help us. But you are right it is a challenge once context comes in. As you said, then the splitting of the content has to be designed carefully.
Thanks, Som On Wed, Mar 6, 2013 at 12:08 AM, Fabian Christ <[email protected] > wrote: > 2013/3/5 Som Satpathy <[email protected]>: > > I am aiming for > > distributing the enhancement request for the posted content over a > cluster > > of nodes. > > And should each node process a single enhancement engine or are you > trying to split the content? > > Splitting the content has to be designed carefully since some > enhancement engines may rely on context information. Therefore, it > depends on the engines' requirements and the kind of content you are > processing to decide how the content could be split into parts. > > For example, you may have only engines that work on sentence level and > the content is plain text. This way you could distribute each > sentence. But if your content is a website and your engines take the > structure of the website into account, it may not be possible to split > at sentence level. > > Best, > - Fabian > > -- > Fabian > http://twitter.com/fctwitt >
