[jira] [Commented] (OPENNLP-1387) Fix alphaNumOpt in tokenizer example

[ASF GitHub Bot (Jira)]

    [ 
https://issues.apache.org/jira/browse/OPENNLP-1387?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623161#comment-17623161
 ] 

ASF GitHub Bot commented on OPENNLP-1387:
-----------------------------------------

jzonthemtn commented on code in PR #426:
URL: https://github.com/apache/opennlp/pull/426#discussion_r1003288846


##########
opennlp-docs/src/docbkx/tokenizer.xml:
##########
@@ -258,37 +258,39 @@ Arguments description:
                                To train the english tokenizer use the 
following command:
                                <screen>
                            <![CDATA[
-$ opennlp TokenizerTrainer -model en-token.bin -alphaNumOpt -lang en -data 
en-token.train -encoding UTF-8
+$ opennlp TokenizerTrainer -model en-token.bin -alphaNumOpt isAlphaNumOpt 
-lang en -data en-token.train -encoding UTF-8

Review Comment:
   Looking at this again, should this be `true` instead of `isAlphaNumOpt`? 
Like:
   
   ```
   $ opennlp TokenizerTrainer -model en-token.bin -alphaNumOpt true -lang en 
-data en-token.train -encoding UTF-8
   ```
   
   Looking at `TrainingParams.java`, `getAlphaNumOpt()` returns a `Boolean` and 
defaults to `false`. Since it defaults to `false`, I guess it makes sense for 
the example to be `true`...?





> Fix alphaNumOpt in tokenizer example
> ------------------------------------
>
>                 Key: OPENNLP-1387
>                 URL: https://issues.apache.org/jira/browse/OPENNLP-1387
>             Project: OpenNLP
>          Issue Type: Task
>          Components: Documentation, Tokenizer
>    Affects Versions: 1.9.4, 2.0.0
>            Reporter: Jeff Zemerick
>            Assignee: Atita Arora
>            Priority: Major
>             Fix For: 2.0.1
>
>
> The example command for the tokenizer in the documentation is:
> {{opennlp TokenizerTrainer -model en-token.bin -alphaNumOpt -lang en -data 
> en-token.train -encoding UTF-8}}
> This is an invalid command because the alphaNumOpt parameter must have a 
> value. Running this command as-is will give the error "Number of parameters 
> must be always be even." 
> The documentation for alphaNumOpt specifies it must have a value:
> {quote}
> -alphaNumOpt isAlphaNumOpt
> Optimization flag to skip alpha numeric tokens for further tokenization
> {quote}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)