krickert commented on code in PR #1468:
URL: https://github.com/apache/opennlp/pull/1468#discussion_r4282613154
##########
opennlp-core/opennlp-ml/opennlp-dl/src/main/java/opennlp/dl/vectors/SentenceVectorsDL.java:
##########
@@ -73,14 +75,23 @@
* {@code close()} with inference methods.</p>
*
* <p>{@link #embedAll(List)} runs one batched session per distinct tokenized
length, so a batch
- * of same-length inputs costs one inference instead of one per input.</p>
+ * of same-length inputs costs one inference instead of one per input. A group
larger than
+ * {@value #MAX_BATCH_TOKEN_POSITIONS} token positions is split into several
inferences.</p>
*/
@ThreadSafe
public class SentenceVectorsDL extends AbstractDL implements TextEmbedder {
/** The maximum number of tokens per input if none is given, the BERT
position limit. */
public static final int DEFAULT_MAX_LENGTH = 512;
+ /**
+ * The upper bound on the token positions of one inference, its row count
times its row length.
+ * {@link #embedAll(List)} splits a larger group into consecutive
inferences, so the tensors of
+ * one inference stay bounded however many texts a call has. A single input
longer than the
+ * bound still runs on its own.
+ */
+ static final int MAX_BATCH_TOKEN_POSITIONS = 16384;
Review Comment:
Added: the cap is 32 rows at the BERT length of 512.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
