luoyuxia commented on code in PR #19931:
URL: https://github.com/apache/flink/pull/19931#discussion_r896270154
##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/planner/delegation/hive/copy/HiveParserBaseSemanticAnalyzer.java:
##########
@@ -387,7 +387,9 @@ private static void
processPrimaryKeyInfos(HiveParserASTNode pkNode, List<PKInfo
}
private static void checkColumnName(String columnName) throws
SemanticException {
- if
(VirtualColumn.VIRTUAL_COLUMN_NAMES.contains(columnName.toUpperCase())) {
Review Comment:
Sorry for lacking of detail explanation. Yes, the core reson isn't
relocated, but is type inconsistent . The difference in
`org.apache.hadoop.hive.ql.metadata.VirtualColumn` between hive2-exec and
hive3-exec bring the case.
I decompile `sql-connector-hive-connector-2.3.8.jar`, and the bytecode of
the method `checkColumnName ` looks like:
`
Code:
0: getstatic #419 // Field
org/apache/hadoop/hive/ql/metadata/VirtualColumn.VIRTUAL_COLUMN_NAMES:Lorg/apache/hive/com/google/common/collect/ImmutableSet
`
the type of `VirtualColumn.VIRTUAL_COLUMN_NAMES` should be
`org/apache/hive/com/google/common/collect/ImmutableSet`.
And the bytecode of the `VirtualColumn` is also
`
public static final
org.apache.flink.hive.shaded.com.google.common.collect.ImmutableSet<java.lang.String>
VIRTUAL_COLUMN_NAMES;
`
So, it's fine with hive2.
Then, I decompile `sql-connector-hive-connector-3.1.2.jar`, and the bytecode
of the method `checkColumnName ` looks like:
`
Code:
0: getstatic #419 // Field
org/apache/hadoop/hive/ql/metadata/VirtualColumn.VIRTUAL_COLUMN_NAMES:Lorg/apache/hive/com/google/common/collect/ImmutableSet
`
the type of `VirtualColumn.VIRTUAL_COLUMN_NAMES` should be
`org/apache/hive/com/google/common/collect/ImmutableSet`.
But the bytecode of the `VirtualColumn` is
`
public static final
org.apache.flink.hive.shaded.com.google.common.collect.ImmutableSet<java.lang.String>
VIRTUAL_COLUMN_NAMES;
`
It's relocated by flink, which isn't consistent to
`org/apache/hive/com/google/common/collect/ImmutableSet`.
So, `NoSuchFieldError` throws.
Also, I decompile the hive-exec-2.3.9 and hive-exec-3.1.2 that our hive
connector depend on. I find in hive-exec-2.3.9, the `VIRTUAL_COLUMN_NAMES` is
type of
`org.apache.hive.com.google.common.collect.ImmutableSet<java.lang.String>`. But
in hive-exec-3.1.2, the the `VIRTUAL_COLUMN_NAMES` is type of
`com.google.common.collect.ImmutableSet<java.lang.String>` which is to be
relocated to
`org.apache.flink.hive.shaded.com.google.common.collect.ImmutableSet<java.lang.String>`.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
