[
https://issues.apache.org/jira/browse/FLINK-40684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danish updated FLINK-40684:
---------------------------
Description:
The Triton input and output names are [case
sensitive|https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol#important-notes].
However, both input and output tensor names are uppercased in
{code:java}
TritonInferenceModelFunction.buildInferenceRequest{code}
{code:java}
inputNode.put("name", inputName.toUpperCase());
outputNode.put("name", outputName.toUpperCase());{code}
It only works when the Triton server defines them in upper case. Any other case
doesn't work. It can be difficult in some cases when the users may not have
full control over the casing of the model input and output names.
If the user has to change the case in the pipeline to match the model, they can
control it via `CREATE MODEL`.
In summary, we should be able to define our models according to the actual case
of the model input and output names.
{code:java}
CREATE MODEL torch_model
INPUT (`INPUT` STRING)
OUTPUT (`OutPut` ARRAY<TINYINT>)
WITH (
'provider' = 'triton',
'endpoint' = 'https://localhost:8000',
'model-name' = 'torch-model'
)
{code}
was:
The Triton input and output names are [case
sensitive|https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol#important-notes].
However, both input and output tensor names are uppercased in
`TritonInferenceModelFunction.buildInferenceRequest`.
```
inputNode.put("name", inputName.toUpperCase());
outputNode.put("name", outputName.toUpperCase());
```
It only works when the Triton server defines them in upper case. Any other case
doesn't work. It can be difficult in some cases when the users may not have
full control over the casing of the model input and output names.
If the user has to change the case in the pipeline to match the model, they can
control it via `CREATE MODEL`.
In summary, we should be able to define our models according to the actual case
of the model input and output names.
```
CREATE MODEL torch_model
INPUT (`INPUT` STRING)
OUTPUT (`OutPut` ARRAY<TINYINT>)
WITH (
'provider' = 'triton',
'endpoint' = 'https://localhost:8000',
'model-name' = 'torch-model'
)
```
> [Model] Tensor names are unconditionally uppercased
> ---------------------------------------------------
>
> Key: FLINK-40684
> URL: https://issues.apache.org/jira/browse/FLINK-40684
> Project: Flink
> Issue Type: Bug
> Reporter: Danish
> Priority: Major
> Labels: pull-request-available
>
> The Triton input and output names are [case
> sensitive|https://kserve.github.io/website/docs/concepts/architecture/data-plane/v2-protocol#important-notes].
>
> However, both input and output tensor names are uppercased in
> {code:java}
> TritonInferenceModelFunction.buildInferenceRequest{code}
>
>
> {code:java}
> inputNode.put("name", inputName.toUpperCase());
> outputNode.put("name", outputName.toUpperCase());{code}
>
> It only works when the Triton server defines them in upper case. Any other
> case doesn't work. It can be difficult in some cases when the users may not
> have full control over the casing of the model input and output names.
> If the user has to change the case in the pipeline to match the model, they
> can control it via `CREATE MODEL`.
> In summary, we should be able to define our models according to the actual
> case of the model input and output names.
>
> {code:java}
> CREATE MODEL torch_model
> INPUT (`INPUT` STRING)
> OUTPUT (`OutPut` ARRAY<TINYINT>)
> WITH (
> 'provider' = 'triton',
> 'endpoint' = 'https://localhost:8000',
> 'model-name' = 'torch-model'
> )
>
> {code}
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
