jihoonson commented on a change in pull request #9861:
URL: https://github.com/apache/druid/pull/9861#discussion_r435633285
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/common/task/IndexTaskTest.java
##########
@@ -1783,6 +1788,36 @@ public void
testIndexTaskWithSingleDimPartitionsSpecThrowingException() throws E
task.isReady(createActionClient(task));
}
+ @Test
+ public void testShardSpecSelectionWithNullPartitionDimension()
+ {
+ ShardSpec spec1 = new HashBasedNumberedShardSpec(0, 2, null, jsonMapper);
+ ShardSpec spec2 = new HashBasedNumberedShardSpec(1, 2, null, jsonMapper);
+
+ Map<Interval, List<ShardSpec>> shardSpecMap = new HashMap<>();
+
shardSpecMap.put(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
Lists.newArrayList(spec1, spec2));
+
+ IndexTask.ShardSpecs shardSpecs = new IndexTask.ShardSpecs(shardSpecMap,
Granularity.fromString("HOUR"));
Review comment:
nit: please use Granularities.HOUR.
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/common/task/IndexTaskTest.java
##########
@@ -1783,6 +1788,36 @@ public void
testIndexTaskWithSingleDimPartitionsSpecThrowingException() throws E
task.isReady(createActionClient(task));
}
+ @Test
+ public void testShardSpecSelectionWithNullPartitionDimension()
+ {
+ ShardSpec spec1 = new HashBasedNumberedShardSpec(0, 2, null, jsonMapper);
+ ShardSpec spec2 = new HashBasedNumberedShardSpec(1, 2, null, jsonMapper);
+
+ Map<Interval, List<ShardSpec>> shardSpecMap = new HashMap<>();
+
shardSpecMap.put(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
Lists.newArrayList(spec1, spec2));
+
+ IndexTask.ShardSpecs shardSpecs = new IndexTask.ShardSpecs(shardSpecMap,
Granularity.fromString("HOUR"));
+ String visitor_id = "visitor_id";
+ String client_type = "client_type";
+ long timestamp1 = DateTimes.of("2014-01-01T00:00:00.000Z").getMillis();
+ InputRow row1 = new MapBasedInputRow(timestamp1,
+ Lists.newArrayList(visitor_id, client_type),
+ ImmutableMap.of(visitor_id, "0", client_type, "iphone")
+ );
+
+ long timestamp2 = DateTimes.of("2014-01-01T00:30:20.456Z").getMillis();
+ InputRow row2 = new MapBasedInputRow(timestamp2,
+ Lists.newArrayList(visitor_id, client_type),
+ ImmutableMap.of(visitor_id, "0", client_type, "iphone")
+ );
+
+ ShardSpec spec3 =
shardSpecs.getShardSpec(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
row1);
+ ShardSpec spec4 =
shardSpecs.getShardSpec(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
row2);
+
+ Assert.assertEquals(true, spec3 == spec4);
+ }
Review comment:
Could you please add a test that `shardSpecs.getShardSpec()` returns
different shardSpecs for two rows when they have exactly same values but
different timestamps?
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/common/task/IndexTaskTest.java
##########
@@ -1783,6 +1788,36 @@ public void
testIndexTaskWithSingleDimPartitionsSpecThrowingException() throws E
task.isReady(createActionClient(task));
}
+ @Test
+ public void testShardSpecSelectionWithNullPartitionDimension()
+ {
+ ShardSpec spec1 = new HashBasedNumberedShardSpec(0, 2, null, jsonMapper);
+ ShardSpec spec2 = new HashBasedNumberedShardSpec(1, 2, null, jsonMapper);
+
+ Map<Interval, List<ShardSpec>> shardSpecMap = new HashMap<>();
+
shardSpecMap.put(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
Lists.newArrayList(spec1, spec2));
+
+ IndexTask.ShardSpecs shardSpecs = new IndexTask.ShardSpecs(shardSpecMap,
Granularity.fromString("HOUR"));
+ String visitor_id = "visitor_id";
+ String client_type = "client_type";
+ long timestamp1 = DateTimes.of("2014-01-01T00:00:00.000Z").getMillis();
+ InputRow row1 = new MapBasedInputRow(timestamp1,
+ Lists.newArrayList(visitor_id, client_type),
+ ImmutableMap.of(visitor_id, "0", client_type, "iphone")
+ );
+
+ long timestamp2 = DateTimes.of("2014-01-01T00:30:20.456Z").getMillis();
+ InputRow row2 = new MapBasedInputRow(timestamp2,
+ Lists.newArrayList(visitor_id, client_type),
+ ImmutableMap.of(visitor_id, "0", client_type, "iphone")
+ );
+
+ ShardSpec spec3 =
shardSpecs.getShardSpec(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
row1);
+ ShardSpec spec4 =
shardSpecs.getShardSpec(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
row2);
+
+ Assert.assertEquals(true, spec3 == spec4);
Review comment:
Please use `Assert.assertSame()` instead.
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/common/task/IndexTaskTest.java
##########
@@ -1783,6 +1788,36 @@ public void
testIndexTaskWithSingleDimPartitionsSpecThrowingException() throws E
task.isReady(createActionClient(task));
}
+ @Test
+ public void testShardSpecSelectionWithNullPartitionDimension()
Review comment:
Thank you for adding a unit test! This test doesn't seem testing the
`IndexTask` but testing `ShardSpecs`. Would you please move the `ShardSpecs`
out of `IndexTask` and add a new class `ShardSpecsTest`? Then we can move this
test to `ShardSpecsTest`.
##########
File path:
indexing-service/src/test/java/org/apache/druid/indexing/common/task/IndexTaskTest.java
##########
@@ -1783,6 +1788,36 @@ public void
testIndexTaskWithSingleDimPartitionsSpecThrowingException() throws E
task.isReady(createActionClient(task));
}
+ @Test
+ public void testShardSpecSelectionWithNullPartitionDimension()
+ {
+ ShardSpec spec1 = new HashBasedNumberedShardSpec(0, 2, null, jsonMapper);
+ ShardSpec spec2 = new HashBasedNumberedShardSpec(1, 2, null, jsonMapper);
+
+ Map<Interval, List<ShardSpec>> shardSpecMap = new HashMap<>();
+
shardSpecMap.put(Intervals.of("2014-01-01T00:00:00.000Z/2014-01-02T00:00:00.000Z"),
Lists.newArrayList(spec1, spec2));
+
+ IndexTask.ShardSpecs shardSpecs = new IndexTask.ShardSpecs(shardSpecMap,
Granularity.fromString("HOUR"));
+ String visitor_id = "visitor_id";
+ String client_type = "client_type";
Review comment:
The Java convention is the camel case. Please rename these variables to
`visitorId` and `clientType`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
