paul-rogers commented on PR #13138: URL: https://github.com/apache/druid/pull/13138#issuecomment-1255295404
@gianm, thanks for your analysis. You raise an important issue around testing. It appears we have no tests for the case of the Hadoop task with extensions and that is concerning. It means I was able to break this area with no feedback that something was amiss. The workaround testing done here is to borrow some code from `StatusResource` and log the set of loaded extensions. Since we have no IT that runs this task, testing was done ad-hoc by hand on a local machine. Absolutely not ideal. But, perhaps good enough considering that extension loading is binary: they are either loaded or not. Moving forward, we do have an IT for this task which you identified: `ITHdfsToHdfsParallelIndexTest`. Due to memory limitations on Travis, that is not run as part of the Travis build. It is run elsewhere. But, that test does not also ensure that extensions are loaded. There are other HDFS-related tests in the `HDFS_DEEP_STORAGE` IT test group. None of them are run. We need to find a way to run these Hadoop tasks so that we get code coverage. We're discussing moving the build to a new build system. Perhaps that system can provide sufficient resources that we can run the HDFS (and Hadoop) containers in our ITs and so we can test this area. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
