On Thu, 19 Mar 2020, Marc Jeurissen wrote:
Pylucene version: 8.1.1
Hi all,
When you have a custom tokenizer (class CustomTokenizer(PythonTokenizer)),
you don?t seem to be able to override any method besides incrementToken
(so not end, reset, close).
Is this correct?
Correct, the only
Pylucene version: 8.1.1
Hi all,
When you have a custom tokenizer (class CustomTokenizer(PythonTokenizer)), you
don’t seem to be able to override any method besides incrementToken (so not
end, reset, close).
Is this correct?
Thank you very much
Met vriendelijke groeten,
Marc Jeurissen
On Jul 17, 2010, at 22:30, Andi Vajda va...@apache.org wrote:
On Jul 17, 2010, at 22:23, Martin mar...@webscio.net wrote:
Hi there,
I'm trying to extend the PythonTokenizer class to build my own
custom tokenizer, but seem to get stuck pretty much soon after
that. I know that I'm
samples.
Thanks again,
Martin
On Jul 17, 2010, at 22:30, Andi Vajda va...@apache.org wrote:
On Jul 17, 2010, at 22:23, Martin mar...@webscio.net wrote:
Hi there,
I'm trying to extend the PythonTokenizer class to build my own
custom tokenizer, but seem to get stuck pretty much soon after
Hi there,
I'm trying to extend the PythonTokenizer class to build my own custom
tokenizer, but seem to get stuck pretty much soon after that. I know
that I'm supposed to extend the incrementToken() method, but what
exactly am I dealing with in there and what should it return? My goal
On Oct 29, 2009, at 10:28, abhinav mishra abhina...@gmail.com wrote:
Hi,
I know that there is a class EdgeNGramTokenizer, which can be used to
extract n-grams. However, I'm not able use it.
What do you mean by not able to use it ?
Andi..
Any pointers or maybe, piece of code in this
It gives an error. May be the problem is with the way I invoked the
function. I think a sample code to tokenize(ngram) would be sufficient.
Abhinav
On Thu, Oct 29, 2009 at 11:26 AM, Andi Vajda va...@apache.org wrote:
On Oct 29, 2009, at 10:28, abhinav mishra abhina...@gmail.com wrote:
Hi,
On Oct 29, 2009, at 11:48, abhinav mishra abhina...@gmail.com wrote:
It gives an error.
And what is the error ?
May be the problem is with the way I invoked the
function.
And what is the way ?
I think a sample code to tokenize(ngram) would be sufficient.
I'm trying to understand if
