[GitHub] [airflow] mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate integrations

2020-01-13 Thread GitBox
mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate 
integrations
URL: https://github.com/apache/airflow/pull/7091#discussion_r365755950
 
 

 ##
 File path: scripts/ci/in_container/krb5/krb5.conf
 ##
 @@ -17,8 +17,8 @@
 # under the License.
 
 [logging]
-default = FILE:/var/log/krb5libs.log
-kdc = FILE:/var/log/krb5kdc.log
+default = FILE:/var/log/kerberos.log
 
 Review comment:
   Why do we want to have these logs in one file?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate integrations

2020-01-13 Thread GitBox
mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate 
integrations
URL: https://github.com/apache/airflow/pull/7091#discussion_r365755489
 
 

 ##
 File path: scripts/ci/in_container/_in_container_utils.sh
 ##
 @@ -195,4 +195,5 @@ EOF
 
 function stop_output_heartbeat() {
 kill "${HEARTBEAT_PID}"
+wait "${HEARTBEAT_PID}" || true 2> /dev/null
 
 Review comment:
   What do you think to start using `jobs/fg/bg/kill %1` here? Running scripts 
this way is a bit hacky to me.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services


[GitHub] [airflow] mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate integrations

2020-01-13 Thread GitBox
mik-laj commented on a change in pull request #7091: [AIRFLOW-6489] Separate 
integrations
URL: https://github.com/apache/airflow/pull/7091#discussion_r365750352
 
 

 ##
 File path: TESTING.rst
 ##
 @@ -152,46 +152,262 @@ You can also specify individual tests or a group of 
tests:
 
 ./breeze --test-target tests/test_core.py::TestCore
 
-Running Full test suite via scripts from the host
--
 
-To run all tests with default settings (Python 3.6, Sqlite backend, "docker" 
environment), enter:
+Airflow Integration Tests
+=
 
-.. code-block::
+Some of the tests in Airflow are Integration tests. Those tests require not 
only airflow-testing docker
+image but also extra images with integrations (such as redis/mongodb etc.).
 
-  ./scripts/ci/local_ci_run_airflow_testing.sh
 
+Enabling integrations
+-
 
-To select Python 3.6 version, Postgres backend, and a ``docker`` environment, 
specify:
+Running Airflow integration tests cannot be run in local virtualenv. They can 
only run in Breeze
+environment with enabled integrations and in Travis CI.
+
+When you are in Breeze environment, by default all integrations are disabled - 
this way only true unit tests
+can be executed in Breeze. You can enable the integration by passing 
``--integration ``
+switch when starting Breeze. You can specify multiple integrations by 
repeating the ``--integration`` switch
+or by using ``--integration all`` switch which enables all integrations.
+
+Note, that every integration requires separate container with the 
corresponding integration image,
+so they take precious resources on your PC - mainly memory. The integrations 
started are not stopped
+until you stop the Breeze environment with ``--stop-environment`` switch.
+
+The following integrations are available:
+
+.. list-table:: Airflow Test Integrations
+   :widths: 15 80
+   :header-rows: 1
+
+   * - Integration
+ - Description
+   * - cassandra
+ - Integration required for Cassandra hooks
+   * - kerberos
+ - Integration that provides Kerberos authentication
+   * - mongo
+ - Integration required for MongoDB hooks
+   * - openldap
+ - Integration required for OpenLDAP hooks
+   * - rabbitmq
+ - Integration required for Celery executor tests
+   * - redis
+ - Integration required for Celery executor tests
+
+Below command starts mongi integration only:
 
 Review comment:
   ```suggestion
   Below command starts mongo integration only:
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services