Github user vdiravka commented on a diff in the pull request:
https://github.com/apache/drill/pull/687#discussion_r93028597
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/exec/physical/impl/writer/TestCorruptParquetDateCorrection.java
---
@@ -201,68 +203,34 @@ public void
testCorrectDatesAndExceptionWhileParsingCreatedBy() throws Exception
.go();
}
- /**
- * Test reading a directory full of partitioned parquet files with
dates, these files have a drill version
- * number of 1.4.0 in their footers, so we can be certain they are
corrupt. The option to disable the
- * correction is passed, but it will not change the result in the case
where we are certain correction
- * is needed. For more info see DRILL-4203.
- */
- @Test
- public void testReadPartitionedOnCorruptedDates() throws Exception {
- try {
- for (String selection : new String[]{"*", "date_col"}) {
- // for sanity, try reading all partitions without a filter
- TestBuilder builder = testBuilder()
- .sqlQuery("select " + selection + " from table(dfs.`" +
CORRUPTED_PARTITIONED_DATES_1_4_0_PATH + "`" +
- "(type => 'parquet', autoCorrectCorruptDates => false))")
- .unOrdered()
- .baselineColumns("date_col");
- addDateBaselineVals(builder);
- builder.go();
-
- String query = "select " + selection + " from table(dfs.`" +
CORRUPTED_PARTITIONED_DATES_1_4_0_PATH + "` " +
- "(type => 'parquet', autoCorrectCorruptDates => false))" + "
where date_col = date '1970-01-01'";
- // verify that pruning is actually taking place
- testPlanMatchingPatterns(query, new String[]{"numFiles=1"}, null);
-
- // read with a filter on the partition column
- testBuilder()
- .sqlQuery(query)
- .unOrdered()
- .baselineColumns("date_col")
- .baselineValues(new DateTime(1970, 1, 1, 0, 0))
- .go();
- }
- } finally {
- test("alter session reset all");
- }
- }
@Test
public void testReadPartitionedOnCorruptedDates_UserDisabledCorrection()
throws Exception {
try {
for (String selection : new String[]{"*", "date_col"}) {
- // for sanity, try reading all partitions without a filter
- TestBuilder builder = testBuilder()
- .sqlQuery("select " + selection + " from table(dfs.`" +
CORRUPTED_PARTITIONED_DATES_1_2_PATH + "`" +
- "(type => 'parquet', autoCorrectCorruptDates => false))")
- .unOrdered()
- .baselineColumns("date_col");
- addCorruptedDateBaselineVals(builder);
- builder.go();
-
- String query = "select " + selection + " from table(dfs.`" +
CORRUPTED_PARTITIONED_DATES_1_2_PATH + "` " +
- "(type => 'parquet', autoCorrectCorruptDates => false))" + "
where date_col = cast('15334-03-17' as date)";
- // verify that pruning is actually taking place
- testPlanMatchingPatterns(query, new String[]{"numFiles=1"}, null);
-
- // read with a filter on the partition column
- testBuilder()
- .sqlQuery(query)
- .unOrdered()
- .baselineColumns("date_col")
- .baselineValues(new DateTime(15334, 03, 17, 0, 0))
- .go();
+ for (String table : new
String[]{CORRUPTED_PARTITIONED_DATES_1_2_PATH,
CORRUPTED_PARTITIONED_DATES_1_4_0_PATH}) {
--- End diff --
It is necessary to keep indent (211 line) after `try` block and then (212
line) after `for` block.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
