[
https://issues.apache.org/jira/browse/FLINK-12513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Alex updated FLINK-12513:
-------------------------
Description:
Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the
same Flink directory to configure and launch a Flink cluster for a test.
As part of their setup steps, they may modify Flink config files (application
config, logs config), the directory itself (e.g. by copying some jars into
{{lib}} folder). Also, some tests involve using additional services (like
spinning up Zookeper, Kafka and so on clusters).
The corresponding clean up code (to stop services, to revert Flink directory to
original state) is spread out and a little bit not well-structured. In
particular
* the test runner itself reverts the Flink config (but doesn't revert other
changes in Flink dir);
* some tests use shell's {{trap}} on exit hook for clean up callback. Adding
multiple of such callbacks in one test would result in non-proper test tear
down.
As the result, some tests may have left overs that may affect execution of next
steps.
The proposal is to introduce a helper method for using one global (per test
run) {{trap}} hook that would enable adding multiple clean up callbacks. This
should enable registering "resource" clean up callbacks in the same place where
resource is used/launched.
Optional improvement: -make the test runner create a temporal copy of Flink
directory and launch test using that temporal directory. After the test is
done, the temporal directory would be removed.-
Update: the test runner now rollbacks some Flink distribution folders (conf,
lib and plugins in other PR). The above idea with temporal folder currently
doesn't work for tests that involve dockerized Flink, because the {{build.sh}}
script uses the relative path to the Flink dir under test.
was:
Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the
same Flink directory to configure and launch a Flink cluster for a test.
As part of their setup steps, they may modify Flink config files (application
config, logs config), the directory itself (e.g. by copying some jars into
{{lib}} folder). Also, some tests involve using additional services (like
spinning up Zookeper, Kafka and so on clusters).
The corresponding clean up code (to stop services, to revert Flink directory to
original state) is spread out and a little bit not well-structured. In
particular
* the test runner itself reverts the Flink config (but doesn't revert other
changes in Flink dir);
* some tests use shell's {{trap}} on exit hook for clean up callback. Adding
multiple of such callbacks in one test would result in non-proper test tear
down.
As the result, some tests may have left overs that may affect execution of next
steps.
The proposal is to introduce a helper method for using one global (per test
run) {{trap}} hook that would enable adding multiple clean up callbacks. This
should enable registering "resource" clean up callbacks in the same place where
resource is used/launched.
Optional improvement: make the test runner create a temporal copy of Flink
directory and launch test using that temporal directory. After the test is
done, the temporal directory would be removed.
> Improve end-to-end (bash based) tests
> -------------------------------------
>
> Key: FLINK-12513
> URL: https://issues.apache.org/jira/browse/FLINK-12513
> Project: Flink
> Issue Type: Improvement
> Components: Tests
> Reporter: Alex
> Assignee: Alex
> Priority: Minor
> Labels: pull-request-available
> Fix For: 1.9.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Tests in {{flink-end-to-end-tests/test-scripts}} directory are re-using the
> same Flink directory to configure and launch a Flink cluster for a test.
> As part of their setup steps, they may modify Flink config files (application
> config, logs config), the directory itself (e.g. by copying some jars into
> {{lib}} folder). Also, some tests involve using additional services (like
> spinning up Zookeper, Kafka and so on clusters).
> The corresponding clean up code (to stop services, to revert Flink directory
> to original state) is spread out and a little bit not well-structured. In
> particular
> * the test runner itself reverts the Flink config (but doesn't revert other
> changes in Flink dir);
> * some tests use shell's {{trap}} on exit hook for clean up callback. Adding
> multiple of such callbacks in one test would result in non-proper test tear
> down.
> As the result, some tests may have left overs that may affect execution of
> next steps.
> The proposal is to introduce a helper method for using one global (per test
> run) {{trap}} hook that would enable adding multiple clean up callbacks. This
> should enable registering "resource" clean up callbacks in the same place
> where resource is used/launched.
> Optional improvement: -make the test runner create a temporal copy of Flink
> directory and launch test using that temporal directory. After the test is
> done, the temporal directory would be removed.-
> Update: the test runner now rollbacks some Flink distribution folders (conf,
> lib and plugins in other PR). The above idea with temporal folder currently
> doesn't work for tests that involve dockerized Flink, because the
> {{build.sh}} script uses the relative path to the Flink dir under test.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
