I looked some files and there were for example one email address, and some conversations which had spaces between words but they were still there and in one token (I don't know how they went there...). So in my opinion it would be better if the tokens which occur only once would be left off.
-Joonas 2015-03-04 15:24 GMT+02:00 Tino Didriksen <[email protected]>: > On 4 March 2015 at 14:16, Joonas Kylmälä <[email protected]> wrote: >> >> This looks good! But there is one thing which came to my mind: what if >> people write there something personal and they don't want it to show >> publicly? If the problem is not taken in account yet, maybe we could >> only show the words which occur two or more times? > > > Thought about that, but something secret that you can somehow make out the > context of when you have 1 token to look at? Highly unlikely. > > Sure, if someone writes MyDirtySecretAllInOneTokenAndMyNameIsXAndILiveInY > then that'll show, but I just can't see that being a real world problem. > > -- TD > > ------------------------------------------------------------------------------ > Dive into the World of Parallel Programming The Go Parallel Website, > sponsored > by Intel and developed in partnership with Slashdot Media, is your hub for > all > things parallel software development, from weekly thought leadership blogs > to > news, videos, case studies, tutorials and more. Take a look and join the > conversation now. http://goparallel.sourceforge.net/ > _______________________________________________ > Apertium-stuff mailing list > [email protected] > https://lists.sourceforge.net/lists/listinfo/apertium-stuff > ------------------------------------------------------------------------------ Dive into the World of Parallel Programming The Go Parallel Website, sponsored by Intel and developed in partnership with Slashdot Media, is your hub for all things parallel software development, from weekly thought leadership blogs to news, videos, case studies, tutorials and more. Take a look and join the conversation now. http://goparallel.sourceforge.net/ _______________________________________________ Apertium-stuff mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/apertium-stuff
