[
https://issues.apache.org/jira/browse/GOBBLIN-1835?focusedWorklogId=865337&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-865337
]
ASF GitHub Bot logged work on GOBBLIN-1835:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 13/Jun/23 18:42
Start Date: 13/Jun/23 18:42
Worklog Time Spent: 10m
Work Description: meethngala commented on code in PR #3697:
URL: https://github.com/apache/gobblin/pull/3697#discussion_r1228556415
##########
gobblin-iceberg/src/test/java/org/apache/gobblin/iceberg/writer/IcebergMetadataWriterTest.java:
##########
@@ -365,12 +364,6 @@ public void testFaultTolerant() throws Exception {
new LongWatermark(52L))));
Assert.assertEquals(gobblinMCEWriter.getDatasetErrorMap().size(), 1);
Assert.assertEquals(gobblinMCEWriter.getDatasetErrorMap().values().iterator().next().size(),
1);
- Assert.assertEquals(gobblinMCEWriter.getDatasetErrorMap()
- .get(new File(tmpDir, "testDB/testTopic").getAbsolutePath())
- .get("hivedb.testTopic").get(0).lowWatermark, 50L);
- Assert.assertEquals(gobblinMCEWriter.getDatasetErrorMap()
- .get(new File(tmpDir, "testDB/testTopic").getAbsolutePath())
- .get("hivedb.testTopic").get(0).highWatermark, 52L);
Review Comment:
yeah... I was also thinking the same if I could still check for the
watermarks. So previously, since a single table was used and we flushed it
using `gobblinMCEWriter` (in other unit test cases outside of testing
completeness) before attempting `writeEnvelope` and could compare watermarks
being unchanged. Now, we use a different table: `testTopicCompleteness` and
writer for testing completeness: `gobblinMCEWriterWithCompletness` and the
error map only contains the exception that we throw when we attempt to write a
record as part of testing the fault tolerance: `Mockito.doThrow(new
IOException("Test failure")).when(mockWriter).writeEnvelope(Mockito.any(),
Mockito.any(), Mockito.any(), Mockito.any())`. I have updated the assert in my
latest commit to verify the content of the error map and still achieve the goal
of testing fault tolerance after we've successfully flushed the metadata
writer. Let me know if my understanding is correct @jack-moseley since I see
you're the last person who worked on this test case!
Issue Time Tracking
-------------------
Worklog Id: (was: 865337)
Time Spent: 1h 10m (was: 1h)
> Upgrade Iceberg Version from 0.11.1 to 1.2.0
> --------------------------------------------
>
> Key: GOBBLIN-1835
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1835
> Project: Apache Gobblin
> Issue Type: New Feature
> Reporter: Meeth Gala
> Priority: Major
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
