krickert opened a new pull request, #1468: URL: https://github.com/apache/opennlp/pull/1468
Draft until #1462 merges; this branch is stacked on it, so the first two commits belong to #1462. TextEmbedder.embedAll and the manual say the implementation forms its own batches, so a caller may pass a list of any size. SentenceVectorsDL.embedAll groups texts by tokenized length and runs each group through the session as one inference, however many texts the group has. Each group is now split so that no inference takes more than 16384 token positions (rows times row length). An input longer than 16384 tokens runs in a batch of its own. Vectors do not change: batches are not padded, so every row is computed from the same tensors as its single-input call. The grouping is in a package-private batches(Tokens[]) that returns the index lists embedAll runs, so the tests can check the inference plan directly. MAX_BATCH_TOKEN_POSITIONS is a package-private constant, not an option. Tests: - SentenceVectorsDLEmbedderTest: a group one quarter over the cap splits into two inferences and every row equals the single-input vector; mixed lengths run one batch per length; two inputs wider than the cap run as two batches of one. - SentenceVectorsDLBatchLimitTest: row count boundaries (exact fit, one over, one under, lengths that do and do not divide the cap), a length at and over the cap, an empty call, a mix where only one group splits, input order with alternating lengths, CLS and mean pooling, the pooled-output model and unit length scaling through a split group, and CUDA against CPU for a split group (skipped without a CUDA provider or device). Fixes #1391 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
