[
https://issues.apache.org/jira/browse/DRILL-5978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16367722#comment-16367722
]
ASF GitHub Bot commented on DRILL-5978:
---------------------------------------
Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/1111#discussion_r167946336
--- Diff:
contrib/storage-hive/core/src/main/java/org/apache/drill/exec/store/hive/HiveUtilities.java
---
@@ -507,5 +512,49 @@ public static boolean
hasHeaderOrFooter(HiveTableWithColumnCache table) {
int skipFooter = retrieveIntProperty(tableProperties,
serdeConstants.FOOTER_COUNT, -1);
return skipHeader > 0 || skipFooter > 0;
}
+
+ /**
+ * This method checks whether the table is transactional and set
necessary properties in {@link JobConf}.
+ * If schema evolution properties aren't set in job conf for the input
format, method sets the column names
+ * and types from table/partition properties or storage descriptor.
+ *
+ * @param job the job to update
+ * @param properties table or partition properties
+ * @param sd storage descriptor
+ */
+ public static void verifyAndAddTransactionalProperties(JobConf job,
StorageDescriptor sd) {
+
+ if (AcidUtils.isTablePropertyTransactional(job)) {
+ AcidUtils.setTransactionalTableScan(job, true);
+
+ // No work is needed, if schema evolution is used
+ if (Utilities.isSchemaEvolutionEnabled(job, true) &&
job.get(IOConstants.SCHEMA_EVOLUTION_COLUMNS) != null &&
+ job.get(IOConstants.SCHEMA_EVOLUTION_COLUMNS_TYPES) != null) {
+ return;
+ }
+
+ String colNames;
+ String colTypes;
+
+ // Try to get get column names and types from table or partition
properties. If they are absent there, get columns
+ // data from storage descriptor of the table
+ colNames = job.get(serdeConstants.LIST_COLUMNS);
+ colTypes = job.get(serdeConstants.LIST_COLUMN_TYPES);
+
+ if (colNames == null || colTypes == null) {
+ List<String> colNamesList = Lists.newArrayList();
+ List<String> colTypesList = Lists.newArrayList();
+ for (FieldSchema col: sd.getCols()) {
+ colNamesList.add(col.getName());
+ colTypesList.add(col.getType());
+ }
+ colNames = Joiner.on(",").join(colNamesList);
--- End diff --
I have changed it. But we need call `input.getName()` and
`input.getType()`, that's why I use two Functions and code became bigger. Once
we will use Java8 it can be smaller. Or did I miss something here?
> Upgrade Hive libraries to 2.1.1 version.
> ----------------------------------------
>
> Key: DRILL-5978
> URL: https://issues.apache.org/jira/browse/DRILL-5978
> Project: Apache Drill
> Issue Type: Improvement
> Components: Storage - Hive
> Affects Versions: 1.11.0
> Reporter: Vitalii Diravka
> Assignee: Vitalii Diravka
> Priority: Major
> Labels: doc-impacting
> Fix For: 1.13.0
>
>
> Currently Drill uses [Hive version 1.2.1
> libraries|https://github.com/apache/drill/blob/master/pom.xml#L53] to perform
> queries on Hive. This version of library can be used for Hive1.x versions and
> Hive2.x versions too, but some features of Hive2.x are broken (for example
> using of ORC transactional tables). To fix that it will be good to update
> drill-hive library version to 2.1 or newer.
> Tasks which should be done:
> - resolving dependency conflicts;
> - investigating backward compatibility of newer drill-hive library with older
> Hive versions (1.x);
> - updating drill-hive version for
> [MapR|https://github.com/apache/drill/blob/master/pom.xml#L1777] profile too.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
