paul-rogers commented on a change in pull request #1978: DRILL-7578: HDF5
Metadata Queries Fail with Large Files
URL: https://github.com/apache/drill/pull/1978#discussion_r380884441
##########
File path:
contrib/format-hdf5/src/test/java/org/apache/drill/exec/store/hdf5/TestHDF5Format.java
##########
@@ -98,9 +98,9 @@ public void testStarQuery() throws Exception {
testBuilder()
.sqlQuery("SELECT * FROM dfs.`hdf5/dset.h5`")
- .unOrdered()
- .baselineColumns("path", "data_type", "file_name", "int_data")
- .baselineValues("/dset", "DATASET", "dset.h5", finalList)
+ .ordered()
Review comment:
@cgivre, you are right about my lack of understanding. This thing is a beast
and I'm afraid I don't understand much of it. The config variable is also a
hack: it means the user needs a separate config for each file and data set. The
ease of use of that is perhaps not what it should be. Having a session option
would be a bit better, but still not great. (Of course, Drill provides no way
for a plugin to define a session option. Another good feature request.)
I posted the question on the dev list and learned that we probably can
modify Drill to allow you to specify the data set name as part of the table
path. And, we can work out how to extend the `DESCRIBE` statement to query for
schema rather than data.
All of this is beyond the scope of this PR. If we use HDF5 as a prototype,
we can see that we'd want to do similar things for other format or plugins.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
