[jira] [Comment Edited] (DRILL-5090) JDBC tests silently ignore failure to set up test storage plugin
[
https://issues.apache.org/jira/browse/DRILL-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713647#comment-15713647
]
Paul Rogers edited comment on DRILL-5090 at 12/2/16 5:55 AM:
-
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
The JDBC test uses a {{LocalPersistentStore}} while the working unit tests use
{{NoWriteLocalStore}}. The reason is that {{LocalPersistentStoreProvider}}
checks {{ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE}}
({{drill.exec.sys.store.provider.local.write}}). This is set in {BaseTestQuery}:
{code}
put(ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE, "false");
{code}
Adding this to the {{JdbcAssert.getDefaultProperties()}} solves the problem.
This does not answer the question as to why the tests succeed for others. Do
they delete /tmp/drill?
was (Author: paul-rogers):
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
These may have been created when running other unit tests. How are they being
cleaned up for folks for whom the JDBC unit tests succeed?
Interesting fact: the storage plugin thinks the local storage path is
{{/tmp/drill/sys.storage_plugins}} while {{TestUtilities.createTempDir();}} in
the JDBC driver test code sets the temp dir to {{/var/folders/xyz/abc}} where
xyz and abc are random strings.
The persistent store is getting its directory from the
{{drill.exec.sys.store.provider.local.path}} config parameter defined in
{{drill/exec/java-exec/src/main/resources/drill-module.conf}}. But this value
has remain unchanged for several years.
> JDBC tests silently ignore failure to set up test storage plugin
>
>
> Key: DRILL-5090
> URL: https://issues.apache.org/jira/browse/DRILL-5090
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>
> Run the {{TestJDBCQuery}} test cases using Java 8. The tests will fail with
> error such as:
> {code}
> VALIDATION ERROR ...
> Current default schema: No default schema selected
> {code}
> The problem is that the tests try to set up a test schema, but fail. The code
> that does this setup just silently ignores the error:
> {code}
> try {
> TestUtilities.updateDfsTestTmpSchemaLocation(pluginRegistry,
> tmpDirPath);
> TestUtilities.makeDfsTmpSchemaImmutable(pluginRegistry);
> }
> } catch(Throwable e) {
> // ... This is unlikely to
> // happen, but just a safeguard to avoid failing user applications.
> logger.warn("Failed to update tmp schema locations. This step is purely
> for testing purpose. " +
> "Shouldn't be seen in production code.");
> // Ignore the error and go with defaults
> }
> {code}
> The reason that the error is ignored is that the JDBC driver _itself_
> contains test code in the form of the following check:
> {code}
> if (props != null &&
> "true".equalsIgnoreCase(props.getProperty("drillJDBCUnitTests"))) {
> {code}
> That is, the JDBC driver itself contains the code needed to set up the test
> schemas. This means that test code is shipped in production. And, we must
> handle the failure gracefully in case the user set the property in production.
> Requested changes:
> * Find a way to do test setup without including test code in JDBC.
> * If test setup fails, issue a fatal error to direct the developer to the
> actual problem.
> And, of course, fix the Java 8 error so that the tests pass. Fixing that
> error is a separate issue. If things were to fail again, we need the above
> fixes to avoid wasting developer's time finding the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5090) JDBC tests silently ignore failure to set up test storage plugin
[
https://issues.apache.org/jira/browse/DRILL-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713647#comment-15713647
]
Paul Rogers edited comment on DRILL-5090 at 12/2/16 5:55 AM:
-
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
The JDBC test uses a {{LocalPersistentStore}} while the working unit tests use
{{NoWriteLocalStore}}. The reason is that {{LocalPersistentStoreProvider}}
checks {{ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE}}
({{drill.exec.sys.store.provider.local.write}}). This is set in
{{BaseTestQuery}}:
{code}
put(ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE, "false");
{code}
Adding this to the {{JdbcAssert.getDefaultProperties()}} solves the problem.
This does not answer the question as to why the tests succeed for others. Do
they delete /tmp/drill?
was (Author: paul-rogers):
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
The JDBC test uses a {{LocalPersistentStore}} while the working unit tests use
{{NoWriteLocalStore}}. The reason is that {{LocalPersistentStoreProvider}}
checks {{ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE}}
({{drill.exec.sys.store.provider.local.write}}). This is set in {BaseTestQuery}:
{code}
put(ExecConstants.SYS_STORE_PROVIDER_LOCAL_ENABLE_WRITE, "false");
{code}
Adding this to the {{JdbcAssert.getDefaultProperties()}} solves the problem.
This does not answer the question as to why the tests succeed for others. Do
they delete /tmp/drill?
> JDBC tests silently ignore failure to set up test storage plugin
>
>
> Key: DRILL-5090
> URL: https://issues.apache.org/jira/browse/DRILL-5090
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>
> Run the {{TestJDBCQuery}} test cases using Java 8. The tests will fail with
> error such as:
> {code}
> VALIDATION ERROR ...
> Current default schema: No default schema selected
> {code}
> The problem is that the tests try to set up a test schema, but fail. The code
> that does this setup just silently ignores the error:
> {code}
> try {
> TestUtilities.updateDfsTestTmpSchemaLocation(pluginRegistry,
> tmpDirPath);
> TestUtilities.makeDfsTmpSchemaImmutable(pluginRegistry);
> }
> } catch(Throwable e) {
> // ... This is unlikely to
> // happen, but just a safeguard to avoid failing user applications.
> logger.warn("Failed to update tmp schema locations. This step is purely
> for testing purpose. " +
> "Shouldn't be seen in production code.");
> // Ignore the error and go with defaults
> }
> {code}
> The reason that the error is ignored is that the JDBC driver _itself_
> contains test code in the form of the following check:
> {code}
> if (props != null &&
> "true".equalsIgnoreCase(props.getProperty("drillJDBCUnitTests"))) {
> {code}
> That is, the JDBC driver itself contains the code needed to set up the test
> schemas. This means that test code is shipped in production. And, we must
> handle the failure gracefully in case the user set the property in production.
> Requested changes:
> * Find a way to do test setup without including test code in JDBC.
> * If test setup fails, issue a fatal error to direct the developer to the
> actual problem.
> And, of course, fix the Java 8 error so that the tests pass. Fixing that
> error is a separate issue. If things were to fail again, we need the above
> fixes to avoid wasting developer's time finding the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5090) JDBC tests silently ignore failure to set up test storage plugin
[
https://issues.apache.org/jira/browse/DRILL-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713647#comment-15713647
]
Paul Rogers edited comment on DRILL-5090 at 12/2/16 3:56 AM:
-
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
These may have been created when running other unit tests. How are they being
cleaned up for folks for whom the JDBC unit tests succeed?
Interesting fact: the storage plugin thinks the local storage path is
{{/tmp/drill/sys.storage_plugins}} while {{TestUtilities.createTempDir();}} in
the JDBC driver test code sets the temp dir to {{/var/folders/xyz/abc}} where
xyz and abc are random strings.
The persistent store is getting its directory from the
{{drill.exec.sys.store.provider.local.path}} config parameter defined in
{{drill/exec/java-exec/src/main/resources/drill-module.conf}}. But this value
has remain unchanged for several years.
was (Author: paul-rogers):
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
These may have been created when running other unit tests. How are they being
cleaned up for folks for whom the JDBC unit tests succeed?
Interesting fact: the storage plugin thinks the local storage path is
{{/tmp/drill/sys.storage_plugins}} while {{TestUtilities.createTempDir();}} in
the JDBC driver test code sets the temp dir to {{/var/folders/xyz/abc}} where
xyz and abc are random strings.
> JDBC tests silently ignore failure to set up test storage plugin
>
>
> Key: DRILL-5090
> URL: https://issues.apache.org/jira/browse/DRILL-5090
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>
> Run the {{TestJDBCQuery}} test cases using Java 8. The tests will fail with
> error such as:
> {code}
> VALIDATION ERROR ...
> Current default schema: No default schema selected
> {code}
> The problem is that the tests try to set up a test schema, but fail. The code
> that does this setup just silently ignores the error:
> {code}
> try {
> TestUtilities.updateDfsTestTmpSchemaLocation(pluginRegistry,
> tmpDirPath);
> TestUtilities.makeDfsTmpSchemaImmutable(pluginRegistry);
> }
> } catch(Throwable e) {
> // ... This is unlikely to
> // happen, but just a safeguard to avoid failing user applications.
> logger.warn("Failed to update tmp schema locations. This step is purely
> for testing purpose. " +
> "Shouldn't be seen in production code.");
> // Ignore the error and go with defaults
> }
> {code}
> The reason that the error is ignored is that the JDBC driver _itself_
> contains test code in the form of the following check:
> {code}
> if (props != null &&
> "true".equalsIgnoreCase(props.getProperty("drillJDBCUnitTests"))) {
> {code}
> That is, the JDBC driver itself contains the code needed to set up the test
> schemas. This means that test code is shipped in production. And, we must
> handle the failure gracefully in case the user set the property in production.
> Requested changes:
> * Find a way to do test setup without including test code in JDBC.
> * If test setup fails, issue a fatal error to direct the developer to the
> actual problem.
> And, of course, fix the Java 8 error so that the tests pass. Fixing that
> error is a separate issue. If things were to fail again, we need the above
> fixes to avoid wasting developer's time finding the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (DRILL-5090) JDBC tests silently ignore failure to set up test storage plugin
[
https://issues.apache.org/jira/browse/DRILL-5090?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15713647#comment-15713647
]
Paul Rogers edited comment on DRILL-5090 at 12/2/16 3:46 AM:
-
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
These may have been created when running other unit tests. How are they being
cleaned up for folks for whom the JDBC unit tests succeed?
Interesting fact: the storage plugin thinks the local storage path is
{{/tmp/drill/sys.storage_plugins}} while {{TestUtilities.createTempDir();}} in
the JDBC driver test code sets the temp dir to {{/var/folders/xyz/abc}} where
xyz and abc are random strings.
was (Author: paul-rogers):
Further tests show that the {{StoragePluginRegistryImpl.createPlugins()}}
method does not load the test bootstrap plugins because the local plugin
directory already exists: {{/tmp/drill/sys.storage_plugins}}.
Not sure how this could have ever worked if this directory exists and has
contents:
{code}
cp.sys.drilldfs.sys.drill s3.sys.drill
{code}
These may have been created when running other unit tests. How are they being
cleaned up for folks for whom the JDBC unit tests succeed?
> JDBC tests silently ignore failure to set up test storage plugin
>
>
> Key: DRILL-5090
> URL: https://issues.apache.org/jira/browse/DRILL-5090
> Project: Apache Drill
> Issue Type: Bug
>Affects Versions: 1.8.0
>Reporter: Paul Rogers
>
> Run the {{TestJDBCQuery}} test cases using Java 8. The tests will fail with
> error such as:
> {code}
> VALIDATION ERROR ...
> Current default schema: No default schema selected
> {code}
> The problem is that the tests try to set up a test schema, but fail. The code
> that does this setup just silently ignores the error:
> {code}
> try {
> TestUtilities.updateDfsTestTmpSchemaLocation(pluginRegistry,
> tmpDirPath);
> TestUtilities.makeDfsTmpSchemaImmutable(pluginRegistry);
> }
> } catch(Throwable e) {
> // ... This is unlikely to
> // happen, but just a safeguard to avoid failing user applications.
> logger.warn("Failed to update tmp schema locations. This step is purely
> for testing purpose. " +
> "Shouldn't be seen in production code.");
> // Ignore the error and go with defaults
> }
> {code}
> The reason that the error is ignored is that the JDBC driver _itself_
> contains test code in the form of the following check:
> {code}
> if (props != null &&
> "true".equalsIgnoreCase(props.getProperty("drillJDBCUnitTests"))) {
> {code}
> That is, the JDBC driver itself contains the code needed to set up the test
> schemas. This means that test code is shipped in production. And, we must
> handle the failure gracefully in case the user set the property in production.
> Requested changes:
> * Find a way to do test setup without including test code in JDBC.
> * If test setup fails, issue a fatal error to direct the developer to the
> actual problem.
> And, of course, fix the Java 8 error so that the tests pass. Fixing that
> error is a separate issue. If things were to fail again, we need the above
> fixes to avoid wasting developer's time finding the problem.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
