[
https://issues.apache.org/jira/browse/DRILL-5260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15881573#comment-15881573
]
ASF GitHub Bot commented on DRILL-5260:
---------------------------------------
Github user paul-rogers commented on a diff in the pull request:
https://github.com/apache/drill/pull/753#discussion_r102843167
--- Diff:
exec/java-exec/src/test/java/org/apache/drill/test/ClusterFixture.java ---
@@ -99,33 +111,83 @@
// behavior. Production default is DEFAULT_SCAN_THREADS
put(ExecConstants.SCAN_THREADPOOL_SIZE, 4);
+
+ // Define a useful root location for the ZK persistent
+ // storage. Profiles will go here when running in distributed
+ // mode.
+
+
put(ZookeeperPersistentStoreProvider.DRILL_EXEC_SYS_STORE_PROVIDER_ZK_BLOBROOT,
"/tmp/drill/log");
}
};
public static final String DEFAULT_BIT_NAME = "drillbit";
private DrillConfig config;
- private Map<String,Drillbit> bits = new HashMap<>();
+ private Map<String, Drillbit> bits = new HashMap<>();
private Drillbit defaultDrillbit;
private BufferAllocator allocator;
private boolean ownsZK;
private ZookeeperHelper zkHelper;
private RemoteServiceSet serviceSet;
- private String dfsTestTmpSchemaLocation;
+ private File dfsTestTempDir;
protected List<ClientFixture> clients = new ArrayList<>();
+ private boolean usesZk;
+ private boolean preserveLocalFiles;
+ private boolean isLocal;
+
+ /**
+ * Temporary directories created for this test cluster.
+ * Each is removed when closing the cluster.
+ */
+
+ private List<File> tempDirs = new ArrayList<>();
+
+ ClusterFixture(FixtureBuilder builder) {
+
+ String zkConnect = configureZk(builder);
+ try {
+ createConfig(builder, zkConnect);
+ startDrillbits(builder);
+ applyOptions(builder);
+ } catch (Exception e) {
+ // Translate exceptions to unchecked to avoid cluttering
+ // tests. Failures will simply fail the test itself.
+
+ throw new IllegalStateException( "Cluster fixture setup failed", e );
+ }
+
+ // Some operations need an allocator.
- protected ClusterFixture(FixtureBuilder builder) throws Exception {
+ allocator = RootAllocatorFactory.newRoot(config);
+ }
+
+ private String configureZk(FixtureBuilder builder) {
// Start ZK if requested.
+ String zkConnect = null;
if (builder.zkHelper != null) {
+ // Case where the test itself started ZK and we're only using it.
+
zkHelper = builder.zkHelper;
ownsZK = false;
- } else if (builder.zkCount > 0) {
- zkHelper = new ZookeeperHelper(true);
- zkHelper.startZookeeper(builder.zkCount);
+ } else if (builder.localZkCount > 0) {
+ // Case where we need a local ZK just for this test cluster.
+
+ zkHelper = new ZookeeperHelper("dummy");
+ zkHelper.startZookeeper(builder.localZkCount);
ownsZK = true;
}
+ if (zkHelper != null) {
+ zkConnect = zkHelper.getConnectionString();
+ // Forced to disable this, because currently we leak memory which is
a known issue for query cancellations.
+ // Setting this causes unittests to fail.
+
builder.configProperty(ExecConstants.RETURN_ERROR_FOR_FAILURE_IN_CANCELLED_FRAGMENTS,
true);
+ }
+ return zkConnect;
--- End diff --
Good catch. Fixed.
Lots of this code was adapted from elsewhere and I didn't always follow
through the exact flow. Thanks for doing so.
> Refinements to new "Cluster Fixture" test framework
> ---------------------------------------------------
>
> Key: DRILL-5260
> URL: https://issues.apache.org/jira/browse/DRILL-5260
> Project: Apache Drill
> Issue Type: Improvement
> Affects Versions: 1.10
> Reporter: Paul Rogers
> Assignee: Paul Rogers
> Priority: Minor
> Fix For: 1.10
>
>
> Roll-up of a number of enhancements to the cluster fixture framework.
> * Config option to suppress printing of CSV and other output. (Allows
> printing for single tests, not printing when running from Maven.)
> * Parsing of query profiles to extract plan and run time information.
> * Fix bug in log fixture when enabling logging for a package.
> * Improved ZK support.
> * Set up the new CTTAS default temporary workspace for tests.
> * Revise TestDrillbitResiliance to use the new framework.
> * Revise TestWindowFrame to to use the new framework.
> * Revise TestMergeJoinWithSchemaChanges to use the new framework.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
