pvary commented on a change in pull request #3242:
URL: https://github.com/apache/iceberg/pull/3242#discussion_r746633793
##########
File path: mr/src/test/java/org/apache/iceberg/mr/TestIcebergInputFormats.java
##########
@@ -52,8 +50,7 @@
import org.apache.iceberg.expressions.Expressions;
import org.apache.iceberg.hadoop.HadoopCatalog;
import org.apache.iceberg.hadoop.HadoopTables;
-import org.apache.iceberg.mr.mapred.Container;
-import org.apache.iceberg.mr.mapred.MapredIcebergInputFormat;
+import org.apache.iceberg.mr.mapred.TestMapredIcebergInputFormat;
Review comment:
Do we still need this class / import, or we can just revert the changes
in this file?
##########
File path:
mr/src/test/java/org/apache/iceberg/mr/hive/TestHiveIcebergStorageHandlerWithEngine.java
##########
@@ -97,6 +98,17 @@
Types.DecimalType.of(3, 1), Types.UUIDType.get(),
Types.FixedType.ofLength(5),
Types.TimeType.get());
+ private static final Schema COMPLEX_SCHEMA = new Schema(
Review comment:
If this is not reused, we might just define these in the test method
##########
File path: mr/src/test/java/org/apache/iceberg/mr/TestIcebergInputFormats.java
##########
@@ -208,21 +205,21 @@ public void testFailedResidualFiltering() throws
Exception {
helper.appendToTable(Row.of("2020-03-20", 0), expectedRecords);
builder.useHiveRows()
- .filter(Expressions.and(
- Expressions.equal("date", "2020-03-20"),
- Expressions.equal("id", 0)));
+ .filter(Expressions.and(
Review comment:
Do we need any of the changes here?
##########
File path:
mr/src/main/java/org/apache/iceberg/mr/mapred/MapredIcebergInputFormat.java
##########
@@ -66,6 +68,13 @@ public MapredIcebergInputFormat() {
@Override
public InputSplit[] getSplits(JobConf job, int numSplits) throws IOException
{
+ TezUtil.setInMemoryDataModel(job);
+ TezUtil.skipResidualFiltering(job);
+ return getSplitsUnchecked(job);
+ }
+
+ @VisibleForTesting
Review comment:
We might not want to split this method as we do not need to override it
##########
File path: .github/workflows/hive-ci.yml
##########
@@ -82,6 +82,8 @@ jobs:
SPARK_LOCAL_IP: localhost
steps:
- uses: actions/checkout@v2
+ - name: Update update-ca-certificates
Review comment:
do we need this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
