[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r356001634
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -248,8 +238,7 @@ public void testScheduleIngestionBeforePendingCompaction()
throws Exception {
metaClient = new HoodieTableMetaClient(jsc.hadoopConfiguration(),
cfg.getBasePath());
HoodieInstant pendingCompactionInstant =
metaClient.getActiveTimeline().filterPendingCompactionTimeline().firstInstant().get();
-assertTrue("Pending Compaction instant has expected instant time",
-pendingCompactionInstant.getTimestamp().equals(compactionInstantTime));
+assertEquals("Pending Compaction instant has expected instant time",
pendingCompactionInstant.getTimestamp(), compactionInstantTime);
Review comment:
done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r356001481
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -552,16 +535,14 @@ private void executeCompaction(String
compactionInstantTime, HoodieWriteClient c
FileStatus[] allFiles =
HoodieTestUtils.listAllDataFilesInPath(table.getMetaClient().getFs(),
cfg.getBasePath());
HoodieTableFileSystemView view =
new HoodieTableFileSystemView(table.getMetaClient(),
table.getCompletedCommitsTimeline(), allFiles);
-List dataFilesToRead =
view.getLatestDataFiles().collect(Collectors.toList());
-return dataFilesToRead;
+return view.getLatestDataFiles().collect(Collectors.toList());
}
private List getCurrentLatestFileSlices(HoodieTable table,
HoodieWriteConfig cfg) throws IOException {
Review comment:
done
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350703269
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -552,16 +535,14 @@ private void executeCompaction(String
compactionInstantTime, HoodieWriteClient c
FileStatus[] allFiles =
HoodieTestUtils.listAllDataFilesInPath(table.getMetaClient().getFs(),
cfg.getBasePath());
HoodieTableFileSystemView view =
new HoodieTableFileSystemView(table.getMetaClient(),
table.getCompletedCommitsTimeline(), allFiles);
-List dataFilesToRead =
view.getLatestDataFiles().collect(Collectors.toList());
-return dataFilesToRead;
+return view.getLatestDataFiles().collect(Collectors.toList());
}
private List getCurrentLatestFileSlices(HoodieTable table,
HoodieWriteConfig cfg) throws IOException {
HoodieTableFileSystemView view = new
HoodieTableFileSystemView(table.getMetaClient(),
table.getMetaClient().getActiveTimeline().reload().getCommitsAndCompactionTimeline());
-List fileSliceList =
Arrays.asList(HoodieTestDataGenerator.DEFAULT_PARTITION_PATHS).stream()
-.flatMap(partition ->
view.getLatestFileSlices(partition)).collect(Collectors.toList());
-return fileSliceList;
+return Arrays.stream(HoodieTestDataGenerator.DEFAULT_PARTITION_PATHS)
+.flatMap(view::getLatestFileSlices).collect(Collectors.toList());
Review comment:
ditto
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350703186 ## File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java ## @@ -552,16 +535,14 @@ private void executeCompaction(String compactionInstantTime, HoodieWriteClient c FileStatus[] allFiles = HoodieTestUtils.listAllDataFilesInPath(table.getMetaClient().getFs(), cfg.getBasePath()); HoodieTableFileSystemView view = new HoodieTableFileSystemView(table.getMetaClient(), table.getCompletedCommitsTimeline(), allFiles); -List dataFilesToRead = view.getLatestDataFiles().collect(Collectors.toList()); -return dataFilesToRead; +return view.getLatestDataFiles().collect(Collectors.toList()); Review comment: created inline variable. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702746
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -327,10 +314,9 @@ public void
testScheduleCompactionWithOlderOrSameTimestamp() throws Exception {
// Schedule with timestamp same as that of committed instant
gotException = false;
-String dupCompactionInstantTime = secondInstantTime;
Review comment:
this variable is redundant.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702988
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -352,15 +338,15 @@ public void
testScheduleCompactionWithOlderOrSameTimestamp() throws Exception {
public void testCompactionAfterTwoDeltaCommits() throws Exception {
// No Delta Commits after compaction request
HoodieWriteConfig cfg = getConfig(true);
-try (HoodieWriteClient client = getHoodieWriteClient(cfg, true);) {
+try (HoodieWriteClient client = getHoodieWriteClient(cfg, true)) {
String firstInstantTime = "001";
String secondInstantTime = "004";
String compactionInstantTime = "005";
int numRecs = 2000;
List records = dataGen.generateInserts(firstInstantTime,
numRecs);
- records = runNextDeltaCommits(client, Arrays.asList(firstInstantTime,
secondInstantTime), records, cfg, true,
Review comment:
assigning the value to records is unnecessary.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702635
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -312,10 +300,9 @@ public void
testScheduleCompactionWithOlderOrSameTimestamp() throws Exception {
int numRecs = 2000;
List records = dataGen.generateInserts(firstInstantTime,
numRecs);
-records = runNextDeltaCommits(client, Arrays.asList(firstInstantTime,
secondInstantTime), records, cfg, true,
+runNextDeltaCommits(client, Arrays.asList(firstInstantTime,
secondInstantTime), records, cfg, true,
new ArrayList<>());
-HoodieTableMetaClient metaClient = new
HoodieTableMetaClient(jsc.hadoopConfiguration(), cfg.getBasePath());
Review comment:
not getting used anywhere.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702243
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -137,11 +134,6 @@ public void testRollbackForInflightCompaction() throws
Exception {
}
}
- private Path getInstantPath(HoodieTableMetaClient metaClient, String
timestamp, String action, State state) {
Review comment:
not getting used anywhere
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702511
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -312,10 +300,9 @@ public void
testScheduleCompactionWithOlderOrSameTimestamp() throws Exception {
int numRecs = 2000;
List records = dataGen.generateInserts(firstInstantTime,
numRecs);
-records = runNextDeltaCommits(client, Arrays.asList(firstInstantTime,
secondInstantTime), records, cfg, true,
Review comment:
assigning the value to records is redundant, it never gets used.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350702096
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -106,17 +105,15 @@ public void testRollbackForInflightCompaction() throws
Exception {
HoodieInstant pendingCompactionInstant =
metaClient.getActiveTimeline().filterPendingCompactionTimeline().firstInstant().get();
- assertTrue("Pending Compaction instant has expected instant time",
-
pendingCompactionInstant.getTimestamp().equals(compactionInstantTime));
- assertTrue("Pending Compaction instant has expected state",
- pendingCompactionInstant.getState().equals(State.REQUESTED));
+ assertEquals("Pending Compaction instant has expected instant time",
pendingCompactionInstant.getTimestamp(),
+ compactionInstantTime);
+ assertEquals("Pending Compaction instant has expected state",
pendingCompactionInstant.getState(), State.REQUESTED);
moveCompactionFromRequestedToInflight(compactionInstantTime, client,
cfg);
// Reload and rollback inflight compaction
metaClient = new HoodieTableMetaClient(jsc.hadoopConfiguration(),
cfg.getBasePath());
HoodieTable hoodieTable = HoodieTable.getHoodieTable(metaClient, cfg,
jsc);
- hoodieTable.rollback(jsc, compactionInstantTime, false);
Review comment:
this call is redundant, hoodieTable.rollback(jsc, compactionInstantTime,
false) gets called as the first line of code in
client.rollbackInflightCompaction(
new HoodieInstant(State.INFLIGHT,
HoodieTimeline.COMPACTION_ACTION, compactionInstantTime), hoodieTable);
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [incubator-hudi] pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code clean up in TestAsyncCompaction class
pratyakshsharma commented on a change in pull request #1050: [HUDI-368]: code
clean up in TestAsyncCompaction class
URL: https://github.com/apache/incubator-hudi/pull/1050#discussion_r350701647
##
File path: hudi-client/src/test/java/org/apache/hudi/TestAsyncCompaction.java
##
@@ -106,17 +105,15 @@ public void testRollbackForInflightCompaction() throws
Exception {
HoodieInstant pendingCompactionInstant =
metaClient.getActiveTimeline().filterPendingCompactionTimeline().firstInstant().get();
- assertTrue("Pending Compaction instant has expected instant time",
-
pendingCompactionInstant.getTimestamp().equals(compactionInstantTime));
- assertTrue("Pending Compaction instant has expected state",
- pendingCompactionInstant.getState().equals(State.REQUESTED));
+ assertEquals("Pending Compaction instant has expected instant time",
pendingCompactionInstant.getTimestamp(),
+ compactionInstantTime);
+ assertEquals("Pending Compaction instant has expected state",
pendingCompactionInstant.getState(), State.REQUESTED);
Review comment:
assertEquals() is the best fit for these assertions.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
