luoyuxia commented on code in PR #19931:
URL: https://github.com/apache/flink/pull/19931#discussion_r896270154
##########
flink-connectors/flink-connector-hive/src/main/java/org/apache/flink/table/planner/delegation/hive/copy/HiveParserBaseSemanticAnalyzer.java:
##########
@@ -387,7 +387,9 @@ private static void
processPrimaryKeyInfos(HiveParserASTNode pkNode, List<PKInfo
}
private static void checkColumnName(String columnName) throws
SemanticException {
- if
(VirtualColumn.VIRTUAL_COLUMN_NAMES.contains(columnName.toUpperCase())) {
Review Comment:
Sorry for missing detail explanation. Yes, the core reson isn't relocated,
but is type inconsistent . The difference in
`org.apache.hadoop.hive.ql.metadata.VirtualColumn` between hive2-exec and
hive3-exec bring the case.
I decompile `sql-connector-hive-connector-2.3.9.jar`, and the bytecode of
the method `checkColumnName ` looks like:
`
Code:
0: getstatic #419 // Field
org/apache/hadoop/hive/ql/metadata/VirtualColumn.VIRTUAL_COLUMN_NAMES:Lorg/apache/hive/com/google/common/collect/ImmutableSet
`
the type of `VirtualColumn.VIRTUAL_COLUMN_NAMES` is expected to be
`org/apache/hive/com/google/common/collect/ImmutableSet`.
And the bytecode of the `VirtualColumn` is also
`
org.apache.hive.com.google.common.collect.ImmutableSet<java.lang.String>
VIRTUAL_COLUMN_NAMES;
`
So, it's fine with hive2.
Then, I decompile `sql-connector-hive-connector-3.1.2.jar`, and the bytecode
of the method `checkColumnName ` looks like:
`
Code:
0: getstatic #419 // Field
org/apache/hadoop/hive/ql/metadata/VirtualColumn.VIRTUAL_COLUMN_NAMES:Lorg/apache/hive/com/google/common/collect/ImmutableSet
`
the type of `VirtualColumn.VIRTUAL_COLUMN_NAMES` should be
`org/apache/hive/com/google/common/collect/ImmutableSet`.
But the bytecode of the `VirtualColumn` is
`
public static final
org.apache.flink.hive.shaded.com.google.common.collect.ImmutableSet<java.lang.String>
VIRTUAL_COLUMN_NAMES;
`
It's relocated by flink, which isn't consistent to
`org/apache/hive/com/google/common/collect/ImmutableSet`.
So, `NoSuchFieldError` throws.
After that, I wonder what cause the difference. So, I decompile the
hive-exec-2.3.9 and hive-exec-3.1.2 that our hive connectors for hive2 and
hive3 depend on.
I find in hive-exec-2.3.9, the `VIRTUAL_COLUMN_NAMES` is type of
`org.apache.hive.com.google.common.collect.ImmutableSet<java.lang.String>`. But
in hive-exec-3.1.2, the the `VIRTUAL_COLUMN_NAMES` is type of
`com.google.common.collect.ImmutableSet<java.lang.String>` which is to be
relocated to
`org.apache.flink.hive.shaded.com.google.common.collect.ImmutableSet<java.lang.String>`.
In hive-exec, it's already different.
So, here, we can use a method call to hide such difference.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
