[
https://issues.apache.org/jira/browse/OPENNLP-1225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16709865#comment-16709865
]
ASF GitHub Bot commented on OPENNLP-1225:
-----------------------------------------
tteofili closed pull request #339: OPENNLP-1225 - add WVT#tokens for tokens
iteration
URL: https://github.com/apache/opennlp/pull/339
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git
a/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/MapWordVectorTable.java
b/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/MapWordVectorTable.java
index 763a69c56..11f687f16 100644
---
a/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/MapWordVectorTable.java
+++
b/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/MapWordVectorTable.java
@@ -17,6 +17,7 @@
package opennlp.tools.util.wordvector;
+import java.util.Iterator;
import java.util.Map;
class MapWordVectorTable implements WordVectorTable {
@@ -32,6 +33,11 @@ public WordVector get(String token) {
return vectors.get(token);
}
+ @Override
+ public Iterator<String> tokens() {
+ return vectors.keySet().iterator();
+ }
+
@Override
public int size() {
return vectors.size();
diff --git
a/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/WordVectorTable.java
b/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/WordVectorTable.java
index 5d5510791..5bd78d82c 100644
---
a/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/WordVectorTable.java
+++
b/opennlp-tools/src/main/java/opennlp/tools/util/wordvector/WordVectorTable.java
@@ -17,6 +17,8 @@
package opennlp.tools.util.wordvector;
+import java.util.Iterator;
+
import opennlp.tools.util.java.Experimental;
/**
@@ -30,6 +32,8 @@
WordVector get(String token);
+ Iterator<String> tokens();
+
int size();
int dimension();
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> Make it possible to iterate through word vector table tokens
> ------------------------------------------------------------
>
> Key: OPENNLP-1225
> URL: https://issues.apache.org/jira/browse/OPENNLP-1225
> Project: OpenNLP
> Issue Type: Improvement
> Components: word vectors
> Reporter: Tommaso Teofili
> Assignee: Tommaso Teofili
> Priority: Major
> Fix For: 1.9.1
>
>
> Currently it's not possible to iterate through the tokens stored in a
> {{WordVectorTable}}, which makes it hard to implement use cases where all or
> a subset of the existing word vectors need to be processed.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
