cgivre commented on a change in pull request #2327:
URL: https://github.com/apache/drill/pull/2327#discussion_r730326926
##########
File path:
contrib/storage-jdbc/src/test/java/org/apache/drill/exec/store/jdbc/TestJdbcWriterWithPostgres.java
##########
@@ -210,6 +216,40 @@ public void testWithLargeFile() throws Exception {
assertTrue(dropResults.succeeded());
}
+ @Test
+ @Ignore("This is a slow test. Please run manually.")
+ public void testWithReallyLongFile() throws Exception {
+ Path generatedFile = null;
+ try {
+ generatedFile =
JdbcTestUtils.generateCsvFile("csv/very_large_file.csvh", 10, 100000);
Review comment:
@luocooong Thanks for the comment, but if you look a little further,
you'll see that we aren't actually storing a large file. There is a method
which generates a csv file of a given size for the tests. Once the tests are
done, it deletes the files. There is a `large_csv` file, however it is only
300k, so .... we could get rid of that if you really want to.
For reference, Postgres has a limit of 1000 rows per insert, so I wanted to
see if we'd break the writer with an insert of more than 1000 rows into
Postgres. The good news is that it didn't break!
##########
File path:
contrib/storage-jdbc/src/test/java/org/apache/drill/exec/store/jdbc/TestJdbcWriterWithH2.java
##########
@@ -370,22 +372,73 @@ public void testWithArrayField() throws Exception {
}
}
+ @Test
+ public void testWithLargeFile() throws Exception {
+ String query = "CREATE TABLE h2.tmp.`drill_h2_test`.`t2`
(id,first_name,last_name,email,gender,ip_address) AS " +
+ "SELECT id,first_name,last_name,email,gender,ip_address FROM
cp.`csv/large_csv.csvh`";
+ QuerySummary insertResults = queryBuilder().sql(query).run();
+ assertTrue(insertResults.succeeded());
+
+ query = "SELECT COUNT(*) FROM h2.tmp.`drill_h2_test`.`t2`";
+ long rowCount = queryBuilder().sql(query).singletonLong();
+ assertEquals(6000, rowCount);
+
+ // Now drop the table
+ String dropQuery = "DROP TABLE h2.tmp.`drill_h2_test`.`t2`";
+ QuerySummary dropResults = queryBuilder().sql(dropQuery).run();
+ assertTrue(dropResults.succeeded());
+ }
+
+ @Test
+ @Ignore("This is a slow test. Please run manually.")
Review comment:
YW!
##########
File path: docs/dev/CreatingAWriter.md
##########
@@ -0,0 +1,2 @@
+# Creating a Writer for a Storage Plugin
+This tutorial explains the mostly undocumented features of how to create a
writer for a Drill storage plugin.
Review comment:
Done!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
