szehon-ho commented on code in PR #7324:
URL: https://github.com/apache/iceberg/pull/7324#discussion_r1163395482
##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
Assert.assertEquals("Should have expected order", expected,
table.sortOrder());
}
+ @Test
+ public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+ sql(
+ "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp,
data string) USING iceberg",
+ tableName);
+ Table table = validationCatalog.loadTable(tableIdent);
+ Assert.assertTrue("Table should start unsorted",
table.sortOrder().isUnsorted());
+ Assertions.assertThatThrownBy(() -> {
+ sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));
Review Comment:
Doesn't sql() already do format?
##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
Assert.assertEquals("Should have expected order", expected,
table.sortOrder());
}
+ @Test
+ public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+ sql(
+ "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp,
data string) USING iceberg",
+ tableName);
+ Table table = validationCatalog.loadTable(tableIdent);
+ Assert.assertTrue("Table should start unsorted",
table.sortOrder().isUnsorted());
+ Assertions.assertThatThrownBy(() -> {
+ sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));
Review Comment:
Should we also move the set outside? So that assertThatThrownBy is only
about the alter stmt?
##########
spark/v3.3/spark-extensions/src/main/scala/org/apache/spark/sql/execution/datasources/v2/SetWriteDistributionAndOrderingExec.scala:
##########
@@ -47,6 +48,7 @@ case class SetWriteDistributionAndOrderingExec(
val txn = iceberg.table.newTransaction()
val orderBuilder = txn.replaceSortOrder()
+ orderBuilder.caseSensitive(session.conf.get(SQLConf.CASE_SENSITIVE,
SQLConf.CASE_SENSITIVE.defaultValue.get))
Review Comment:
nit: can this be on same builder?
```
orderBuilder = txn.replaceSortOrder()
.caseSensitive(...)
```
##########
spark/v3.3/spark-extensions/src/test/java/org/apache/iceberg/spark/extensions/TestSetWriteDistributionAndOrdering.java:
##########
@@ -64,6 +67,25 @@ public void testSetWriteOrderByColumn() {
Assert.assertEquals("Should have expected order", expected,
table.sortOrder());
}
+ @Test
+ public void testSetWriteOrderWithCaseSensitiveColumnNames() {
+ sql(
+ "CREATE TABLE %s (Id bigint NOT NULL, Category string, ts timestamp,
data string) USING iceberg",
+ tableName);
+ Table table = validationCatalog.loadTable(tableIdent);
+ Assert.assertTrue("Table should start unsorted",
table.sortOrder().isUnsorted());
+ Assertions.assertThatThrownBy(() -> {
+ sql(String.format("SET %s=true", SQLConf.CASE_SENSITIVE().key()));
+ sql("ALTER TABLE %s WRITE ORDERED BY category, id", tableName);
+ }).isInstanceOf(ValidationException.class);
+
+ Assertions.assertThatNoException()
+ .isThrownBy(() -> {
+ sql(String.format("SET %s=false", SQLConf.CASE_SENSITIVE().key()));
Review Comment:
Same as above
##########
core/src/test/java/org/apache/iceberg/TestSortOrder.java:
##########
@@ -365,4 +367,26 @@ public void testPreservingOrderSortedColumnNames() {
Set<String> sortedCols = SortOrderUtil.orderPreservingSortedColumns(order);
Assert.assertEquals(ImmutableSet.of("data"), sortedCols);
}
+
+ @Test
+ public void testCaseSensitiveSortedColumnNames() {
Review Comment:
Not sure this test is testing any new code? If we want an API level test,
shouldnt it be for testing BaseReplaceSortOrder (which is the new code)?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
