[
https://issues.apache.org/jira/browse/TIKA-3119?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17140962#comment-17140962
]
Thamme Gowda edited comment on TIKA-3119 at 6/20/20, 4:34 AM:
--------------------------------------------------------------
[~tallison]
The test fails because of this Exception:
{code:java}
org.deeplearning4j.exception.DL4JInvalidInputException: Cannot do forward pass
in Convolution layer (layer name = conv2d_1, layer index = 1): input array
channels does not match CNN layer configuration (data format = NHWC, data input
channels = 299, [minibatch, height, width, channels]=[1, 3, 299, 299]; expected
input channels = 3) (layer name: conv2d_1, layer index: 1, layer type:
ConvolutionLayer)
Note: Convolution layers can be configured for either NCHW (channels first) or
NHWC (channels last) format for input images and activations.
Layers can be configured using .dataFormat(CNN2DFormat.NCHW/NHWC) when
constructing the layer, or for the entire net using
.setInputType(InputType.convolutional(height, width, depth,
CNN2DForman.NCHW/NHWC)).
ImageRecordReader and NativeImageLoader can also be configured to load image
data in either NCHW or NHWC format which must match the network
{code}
!Screen Shot 2020-06-19 at 8.56.05 PM.png!
Originally the tensoir dims were of NCHW, but now the default seems to be NHWC
(where N=batch, C=channels=3, H=height=299, W=Width=299)
So we are giving input as [1, 3, 299, 299] but the model in the latest API is
asking [1, 299, 299, 3]
I tried transposing my input from NCHW -> NHWC as a quick fix, but the JVM
crashes.
That is because, (I am guessing) the some of the weights of model's layers are
still NCHW, so it didn't work.
The newer API supposedly has a method to explicitly indicate NCHW input type,
as informed by Exception
{{.setInputType(InputType.convolutional(height, width, depth,
CNN2DForman.NCHW/NHWC))}}
which we should call with these args:
{{.setInputType(InputType.convolutional(299, 299, 1, CNN2DForman.NCHW));}}
Here is where it has to go:
[https://github.com/apache/tika/blob/233a72d45de515a7ecb5988802559690cd91f57a/tika-dl/src/main/java/org/apache/tika/dl/imagerec/DL4JInceptionV3Net.java#L261-L264]
But the KerasModelBuilder class we are using doesn't have that method, nor does
the KerasModel or the class returned by model.getComputationGraph(); So I don't
know where that setInputType should go. I think the KerasModelBuilder class
should have had it, but somehow DL4J folks missed it?
We need help from DL4J team to fix this. I don't know whom to tag or CC.
was (Author: thammegowda):
[~tallison]
The test fails because of this Exception:
{code:java}
org.deeplearning4j.exception.DL4JInvalidInputException: Cannot do forward pass
in Convolution layer (layer name = conv2d_1, layer index = 1): input array
channels does not match CNN layer configuration (data format = NHWC, data input
channels = 299, [minibatch, height, width, channels]=[1, 3, 299, 299]; expected
input channels = 3) (layer name: conv2d_1, layer index: 1, layer type:
ConvolutionLayer)
Note: Convolution layers can be configured for either NCHW (channels first) or
NHWC (channels last) format for input images and activations.
Layers can be configured using .dataFormat(CNN2DFormat.NCHW/NHWC) when
constructing the layer, or for the entire net using
.setInputType(InputType.convolutional(height, width, depth,
CNN2DForman.NCHW/NHWC)).
ImageRecordReader and NativeImageLoader can also be configured to load image
data in either NCHW or NHWC format which must match the network
{code}
!Screen Shot 2020-06-19 at 8.56.05 PM.png!
Originally the tensoir dims were of NCHW, but now the default seems to be NHWC
(where N=batch, C=channels=3, H=height=299, W=Width=299)
So we are giving input as [1, 3, 299, 299] but the model in the latest API is
asking [1, 299, 299, 3]
I tried transposing my input from NCHW -> NHWC as a quick fix, but the JVM
crashes.
That is because, (I am guessing) the some of the weights of model's layers are
still NCHW, so it didn't work.
The newer API supposedly has a method to explicitly indicate NCHW input type,
as informed by Exception
{{.setInputType(InputType.convolutional(height, width, depth,
CNN2DForman.NCHW/NHWC))}}
which we should call with these args:
{{.setInputType(InputType.convolutional(299, 299, 1, CNN2DForman.NCHW));}}
Here is where it has to go:
[https://github.com/apache/tika/blob/233a72d45de515a7ecb5988802559690cd91f57a/tika-dl/src/main/java/org/apache/tika/dl/imagerec/DL4JInceptionV3Net.java#L261-L264]
But the KerasModelBuilder class we are using doesn't have that method (Not does
KerasModel or the class returned by model.getComputationGraph(); So I dont know
where it should go (I think, the KerasModelBuilder class should have had it;
but somehow DL4J folks missed it?)
We need help from DL4J team to fix this. I don't know whom to tag or CC.
> General upgrades for 1.25
> -------------------------
>
> Key: TIKA-3119
> URL: https://issues.apache.org/jira/browse/TIKA-3119
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Major
> Attachments: Screen Shot 2020-06-19 at 8.56.05 PM.png, Screenshot
> from 2020-06-19 14-50-21.png
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
