[jira] [Assigned] (AIRFLOW-5784) Migrate AWS S3 to /providers/aws [AIP-21]
[ https://issues.apache.org/jira/browse/AIRFLOW-5784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Rastogi reassigned AIRFLOW-5784: -- Assignee: (was: Ishan Rastogi) > Migrate AWS S3 to /providers/aws [AIP-21] > - > > Key: AIRFLOW-5784 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5784 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Affects Versions: 2.0.0 >Reporter: Bas Harenslak >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (AIRFLOW-5784) Migrate AWS S3 to /providers/aws [AIP-21]
[ https://issues.apache.org/jira/browse/AIRFLOW-5784?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ishan Rastogi reassigned AIRFLOW-5784: -- Assignee: Ishan Rastogi > Migrate AWS S3 to /providers/aws [AIP-21] > - > > Key: AIRFLOW-5784 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5784 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Affects Versions: 2.0.0 >Reporter: Bas Harenslak >Assignee: Ishan Rastogi >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow-site] mik-laj closed issue #8: Create a top menu component
mik-laj closed issue #8: Create a top menu component URL: https://github.com/apache/airflow-site/issues/8 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #17: Create a button-group component
mik-laj closed issue #17: Create a button-group component URL: https://github.com/apache/airflow-site/issues/17 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #9: Create a footer component
mik-laj closed issue #9: Create a footer component URL: https://github.com/apache/airflow-site/issues/9 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #28: Create a integration section with search on home page
mik-laj closed issue #28: Create a integration section with search on home page URL: https://github.com/apache/airflow-site/issues/28 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj closed issue #26: Create a main header with animation on home page
mik-laj closed issue #26: Create a main header with animation on home page URL: https://github.com/apache/airflow-site/issues/26 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Add animations with flying cubes (#94)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 16d71d0 Add animations with flying cubes (#94)
16d71d0 is described below
commit 16d71d0456570937f04458d9ff8b8b6ff910c20d
Author: Kamil Breguła
AuthorDate: Wed Oct 30 05:03:59 2019 +0100
Add animations with flying cubes (#94)
---
landing-pages/package.json | 3 +
landing-pages/site/assets/scss/_header.scss| 98
landing-pages/site/assets/scss/_home-page.scss | 5 +
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/layouts/_default/baseof.html| 21 +-
.../site/layouts/partials/hooks/head-end.html | 8 +
landing-pages/src/index.js | 6 +
landing-pages/src/js/handleActiveVideo.js | 2 +-
landing-pages/src/js/headerAnimation.js| 260 +
.../scss/_home-page.scss => src/js/utils.js} | 16 +-
landing-pages/webpack.common.js| 5 +-
landing-pages/webpack.dev.js | 2 +-
landing-pages/webpack.prod.js | 2 +-
landing-pages/yarn.lock| 5 +
14 files changed, 414 insertions(+), 20 deletions(-)
diff --git a/landing-pages/package.json b/landing-pages/package.json
index 9285721..b1f2c72 100644
--- a/landing-pages/package.json
+++ b/landing-pages/package.json
@@ -23,6 +23,9 @@
"start:hugo": "hugo -d ../dist -s site -vw",
"start:webpack": "webpack-dev-server --config webpack.dev.js --hot"
},
+ "dependencies": {
+"p5": "^0.10.2"
+ },
"devDependencies": {
"@babel/core": "^7.5.4",
"@babel/plugin-proposal-object-rest-spread": "^7.5.4",
diff --git a/landing-pages/site/assets/scss/_header.scss
b/landing-pages/site/assets/scss/_header.scss
new file mode 100644
index 000..c89eac1
--- /dev/null
+++ b/landing-pages/site/assets/scss/_header.scss
@@ -0,0 +1,98 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+@import "media";
+
+#header {
+ position: relative;
+ margin: 123px -20px 0;
+ min-height: calc(100vh - 123px);
+}
+
+#header-canvas {
+ padding: 0;
+ margin: 0;
+ position: absolute;
+ width: 100%;
+ height: 100%;
+ top: 0;
+ left: 0;
+ right: 0;
+ bottom: 0;
+
+ .text-area {
+max-width: 706px;
+width: 100%;
+display: flex;
+flex-direction: column;
+justify-content: center;
+align-items: center;
+position: absolute;
+top: 50%;
+left: 50%;
+transform: translate(-50%, -50%);
+
+&--header {
+ @extend .header__large--greyish-brown;
+ text-align: center;
+ margin-bottom: 20px;
+}
+
+&--subheader {
+ @extend .subtitle__large--brownish-grey;
+ text-align: center;
+ margin-bottom: 20px;
+}
+ }
+
+ canvas {
+position: absolute;
+top: 0;
+left: 0;
+width: 100%;
+height: 100%;
+z-index: -1;
+ }
+}
+
+@media (max-width: $mobile) {
+ #header {
+margin: 77px -20px 0;
+min-height: calc(100vh - 77px);
+ }
+
+ #header-canvas {
+.text-area {
+ max-width: 290px;
+
+ &--header {
+font-size: 48px !important;
+line-height: 1.25 !important;
+margin-bottom: 14px;
+ }
+
+ &--subheader {
+font-size: 16px !important;
+font-weight: normal !important;
+line-height: 1.63 !important;
+text-align: center;
+margin-bottom: 26px;
+ }
+}
+ }
+}
diff --git a/landing-pages/site/assets/scss/_home-page.scss
b/landing-pages/site/assets/scss/_home-page.scss
index 7bc6d31..feaea7c 100644
--- a/landing-pages/site/assets/scss/_home-page.scss
+++ b/landing-pages/site/assets/scss/_home-page.scss
@@ -18,6 +18,11 @@
*/
@import "media";
+.home-page-layout {
+ &.container {
+padding-top: 80px;
+ }
+}
.principles-header {
margin-top: 20px;
margin-bottom: 4px;
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/main-custom.scss
index 9470988..7425907 100644
---
[GitHub] [airflow-site] mik-laj merged pull request #94: Add animations with flying cubes
mik-laj merged pull request #94: Add animations with flying cubes URL: https://github.com/apache/airflow-site/pull/94 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] yegeniy commented on a change in pull request #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]
yegeniy commented on a change in pull request #6281: Run batches of
(self-terminating) EMR JobFlows [AIRFLOW-5763]
URL: https://github.com/apache/airflow/pull/6281#discussion_r340422591
##
File path: airflow/contrib/sensors/emr_run_job_flows.py
##
@@ -0,0 +1,185 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""EmrRunJobFlows manages cluster queue by implementing an EMR sensor."""
+
+from airflow import AirflowException
+from airflow.contrib.hooks.emr_hook import EmrHook
+from airflow.contrib.sensors.emr_base_sensor import EmrBaseSensor
+from airflow.contrib.sensors.emr_job_flow_sensor import EmrJobFlowSensor
+from airflow.utils.decorators import apply_defaults
+
+
+class EmrRunJobFlows(EmrBaseSensor):
+"""
+Submits batches of self-terminating EMR Job Flows and waits for their steps
+to complete. This operator submits a list of EMR clusters in batches, where
+each Job Flow is expected to be self-terminating and list all the EMR steps
+it is expected to execute. Only basic retry logic.
+
+Implementation Note: For each cluster, we submit all the steps at cluster
+creation time. This partially frees the cluster from the vagaries of the
+Airflow scheduler. Since we rely on EMR to terminate its clusters, any
+failed step will need to terminate the cluster and the cluster itself
should
+auto-terminate as per [1]. In other words, each JobFlow must auto-terminate
+(likely via the `job_flows` parameter) by setting its Instances'
+`"KeepJobFlowAliveWhenNoSteps": False`. Additionally, consider setting each
+Step's `"ActionOnFailure": "TERMINATE_CLUSTER"` to allow failing-fast if
+your workflow allows for it.
+
+[1]: https://docs.aws.amazon.com/emr/latest/ManagementGuide/\
+UsingEMR_TerminationProtection.html#emr-termination-protection-steps
+
+TODO: The utility of the EmrBaseSensor that we extend is somewhat limited.
+Currently, it asks for the state of its JobFlow until that JobFlow reaches
a
+terminal state. If the EMR JobFlow fails, the sensor will mark the task as
+failed. If custom EMR sensor logic is pursued, we could set up step-wise
+monitoring and timeouts, which would allow for context-specific retries
+using XComs, and maybe able to extend the implementation to allow for
+cross-cluster logic, such as waiting for all clusters in a batch to finish
+even when some fail.
+
+:param job_flows: a queue of EMR JobFlows. It's a list of dicts, each one
+mapping job_flow names to their configurations:
+[{job_flow_name: job_flow_overrides}]. Each dict in the list represents
+the job flows which should run in parallel, and every cluster in the
+preceding dict is expected to have come to a successful terminal state,
+prior to submitting the next dict. See boto3's job_flow_overrides EMR
+details in
+https://boto3.amazonaws.com/v1/documentation/api/latest/reference/\
+services/emr.html#EMR.Client.run_job_flow (templated)
+:type job_flows: list
+"""
+
+template_fields = ['job_flows']
+template_ext = ()
+# EMR logo... ~RGB(237,165,83)
+ui_color = "#eda553"
+
+# Overrides for EmrBaseSensor
+NON_TERMINAL_STATES = EmrJobFlowSensor.NON_TERMINAL_STATES
+FAILED_STATE = EmrJobFlowSensor.FAILED_STATE
+
+@apply_defaults
+def __init__(
+self,
+job_flows,
+emr_conn_id='emr_default',
+# require_auto_termination = False,
+*args, **kwargs):
+"""
+C0111
+"""
+super().__init__(*args, **kwargs)
+self.job_flows = job_flows
+self.emr_conn_id = emr_conn_id
+# These two fields will be filled in as clusters are requested and
poked
+self.current_batch = {}
+self.statuses = []
+
+def execute(self, context):
+"""
+C0111
+"""
+self.log.info(
+"The clusters will be submitted across the following batches: %s",
+[set(batch.keys()) for batch in self.job_flows])
+# TODO: Verify all clusters set `"KeepJobFlowAliveWhenNoSteps": False`
+# if
[GitHub] [airflow] yegeniy commented on a change in pull request #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]
yegeniy commented on a change in pull request #6281: Run batches of
(self-terminating) EMR JobFlows [AIRFLOW-5763]
URL: https://github.com/apache/airflow/pull/6281#discussion_r340421899
##
File path: airflow/contrib/sensors/emr_run_job_flows.py
##
@@ -0,0 +1,185 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""EmrRunJobFlows manages cluster queue by implementing an EMR sensor."""
+
+from airflow import AirflowException
+from airflow.contrib.hooks.emr_hook import EmrHook
+from airflow.contrib.sensors.emr_base_sensor import EmrBaseSensor
+from airflow.contrib.sensors.emr_job_flow_sensor import EmrJobFlowSensor
+from airflow.utils.decorators import apply_defaults
+
+
+class EmrRunJobFlows(EmrBaseSensor):
+"""
+Submits batches of self-terminating EMR Job Flows and waits for their steps
+to complete. This operator submits a list of EMR clusters in batches, where
+each Job Flow is expected to be self-terminating and list all the EMR steps
+it is expected to execute. Only basic retry logic.
+
+Implementation Note: For each cluster, we submit all the steps at cluster
+creation time. This partially frees the cluster from the vagaries of the
+Airflow scheduler. Since we rely on EMR to terminate its clusters, any
+failed step will need to terminate the cluster and the cluster itself
should
+auto-terminate as per [1]. In other words, each JobFlow must auto-terminate
+(likely via the `job_flows` parameter) by setting its Instances'
+`"KeepJobFlowAliveWhenNoSteps": False`. Additionally, consider setting each
+Step's `"ActionOnFailure": "TERMINATE_CLUSTER"` to allow failing-fast if
+your workflow allows for it.
+
+[1]: https://docs.aws.amazon.com/emr/latest/ManagementGuide/\
+UsingEMR_TerminationProtection.html#emr-termination-protection-steps
+
+TODO: The utility of the EmrBaseSensor that we extend is somewhat limited.
+Currently, it asks for the state of its JobFlow until that JobFlow reaches
a
+terminal state. If the EMR JobFlow fails, the sensor will mark the task as
+failed. If custom EMR sensor logic is pursued, we could set up step-wise
+monitoring and timeouts, which would allow for context-specific retries
+using XComs, and maybe able to extend the implementation to allow for
+cross-cluster logic, such as waiting for all clusters in a batch to finish
+even when some fail.
+
+:param job_flows: a queue of EMR JobFlows. It's a list of dicts, each one
+mapping job_flow names to their configurations:
+[{job_flow_name: job_flow_overrides}]. Each dict in the list represents
+the job flows which should run in parallel, and every cluster in the
+preceding dict is expected to have come to a successful terminal state,
+prior to submitting the next dict. See boto3's job_flow_overrides EMR
+details in
+https://boto3.amazonaws.com/v1/documentation/api/latest/reference/\
+services/emr.html#EMR.Client.run_job_flow (templated)
+:type job_flows: list
+"""
+
+template_fields = ['job_flows']
+template_ext = ()
+# EMR logo... ~RGB(237,165,83)
+ui_color = "#eda553"
+
+# Overrides for EmrBaseSensor
+NON_TERMINAL_STATES = EmrJobFlowSensor.NON_TERMINAL_STATES
+FAILED_STATE = EmrJobFlowSensor.FAILED_STATE
+
+@apply_defaults
+def __init__(
+self,
+job_flows,
+emr_conn_id='emr_default',
+# require_auto_termination = False,
+*args, **kwargs):
+"""
+C0111
+"""
+super().__init__(*args, **kwargs)
+self.job_flows = job_flows
+self.emr_conn_id = emr_conn_id
+# These two fields will be filled in as clusters are requested and
poked
+self.current_batch = {}
+self.statuses = []
+
+def execute(self, context):
+"""
+C0111
+"""
+self.log.info(
+"The clusters will be submitted across the following batches: %s",
+[set(batch.keys()) for batch in self.job_flows])
+# TODO: Verify all clusters set `"KeepJobFlowAliveWhenNoSteps": False`
+# if
[GitHub] [airflow] yegeniy commented on a change in pull request #6281: Run batches of (self-terminating) EMR JobFlows [AIRFLOW-5763]
yegeniy commented on a change in pull request #6281: Run batches of
(self-terminating) EMR JobFlows [AIRFLOW-5763]
URL: https://github.com/apache/airflow/pull/6281#discussion_r340421899
##
File path: airflow/contrib/sensors/emr_run_job_flows.py
##
@@ -0,0 +1,185 @@
+# -*- coding: utf-8 -*-
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+"""EmrRunJobFlows manages cluster queue by implementing an EMR sensor."""
+
+from airflow import AirflowException
+from airflow.contrib.hooks.emr_hook import EmrHook
+from airflow.contrib.sensors.emr_base_sensor import EmrBaseSensor
+from airflow.contrib.sensors.emr_job_flow_sensor import EmrJobFlowSensor
+from airflow.utils.decorators import apply_defaults
+
+
+class EmrRunJobFlows(EmrBaseSensor):
+"""
+Submits batches of self-terminating EMR Job Flows and waits for their steps
+to complete. This operator submits a list of EMR clusters in batches, where
+each Job Flow is expected to be self-terminating and list all the EMR steps
+it is expected to execute. Only basic retry logic.
+
+Implementation Note: For each cluster, we submit all the steps at cluster
+creation time. This partially frees the cluster from the vagaries of the
+Airflow scheduler. Since we rely on EMR to terminate its clusters, any
+failed step will need to terminate the cluster and the cluster itself
should
+auto-terminate as per [1]. In other words, each JobFlow must auto-terminate
+(likely via the `job_flows` parameter) by setting its Instances'
+`"KeepJobFlowAliveWhenNoSteps": False`. Additionally, consider setting each
+Step's `"ActionOnFailure": "TERMINATE_CLUSTER"` to allow failing-fast if
+your workflow allows for it.
+
+[1]: https://docs.aws.amazon.com/emr/latest/ManagementGuide/\
+UsingEMR_TerminationProtection.html#emr-termination-protection-steps
+
+TODO: The utility of the EmrBaseSensor that we extend is somewhat limited.
+Currently, it asks for the state of its JobFlow until that JobFlow reaches
a
+terminal state. If the EMR JobFlow fails, the sensor will mark the task as
+failed. If custom EMR sensor logic is pursued, we could set up step-wise
+monitoring and timeouts, which would allow for context-specific retries
+using XComs, and maybe able to extend the implementation to allow for
+cross-cluster logic, such as waiting for all clusters in a batch to finish
+even when some fail.
+
+:param job_flows: a queue of EMR JobFlows. It's a list of dicts, each one
+mapping job_flow names to their configurations:
+[{job_flow_name: job_flow_overrides}]. Each dict in the list represents
+the job flows which should run in parallel, and every cluster in the
+preceding dict is expected to have come to a successful terminal state,
+prior to submitting the next dict. See boto3's job_flow_overrides EMR
+details in
+https://boto3.amazonaws.com/v1/documentation/api/latest/reference/\
+services/emr.html#EMR.Client.run_job_flow (templated)
+:type job_flows: list
+"""
+
+template_fields = ['job_flows']
+template_ext = ()
+# EMR logo... ~RGB(237,165,83)
+ui_color = "#eda553"
+
+# Overrides for EmrBaseSensor
+NON_TERMINAL_STATES = EmrJobFlowSensor.NON_TERMINAL_STATES
+FAILED_STATE = EmrJobFlowSensor.FAILED_STATE
+
+@apply_defaults
+def __init__(
+self,
+job_flows,
+emr_conn_id='emr_default',
+# require_auto_termination = False,
+*args, **kwargs):
+"""
+C0111
+"""
+super().__init__(*args, **kwargs)
+self.job_flows = job_flows
+self.emr_conn_id = emr_conn_id
+# These two fields will be filled in as clusters are requested and
poked
+self.current_batch = {}
+self.statuses = []
+
+def execute(self, context):
+"""
+C0111
+"""
+self.log.info(
+"The clusters will be submitted across the following batches: %s",
+[set(batch.keys()) for batch in self.job_flows])
+# TODO: Verify all clusters set `"KeepJobFlowAliveWhenNoSteps": False`
+# if
[GitHub] [airflow-site] mik-laj commented on issue #95: Add search blog posts
mik-laj commented on issue #95: Add search blog posts URL: https://github.com/apache/airflow-site/pull/95#issuecomment-547723616 Can you build the whole page once with the `./site.sh build-site` command? This should build a search engine index. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] kgabryje commented on issue #95: Add search blog posts
kgabryje commented on issue #95: Add search blog posts URL: https://github.com/apache/airflow-site/pull/95#issuecomment-547723299 I'm getting this error. I think it didn't occur before. I built index with `node create-index.js`  This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on a change in pull request #96: Add integration search
mik-laj commented on a change in pull request #96: Add integration search
URL: https://github.com/apache/airflow-site/pull/96#discussion_r340414175
##
File path: landing-pages/src/js/integrationList.js
##
@@ -0,0 +1,153 @@
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import debounce from "lodash/debounce";
+import shuffle from "lodash/shuffle";
+
+const root = document.querySelector("#integrations");
+
+const templateText = root.querySelector("#integration-template").innerText;
+const templateElement = document.createElement("div");
+templateElement.innerHTML = templateText;
+
+const loadingIndicator = root.querySelector(".loading");
+const listItems = root.querySelector(".list-items");
+const searchBox = root.querySelector('[type="search"]');
+const moreButton = root.querySelector("#show-more-integration");
+
+let currentPage = 1;
+let currentQeury = "";
+function setIndicatorVisibility(visible) {
+ loadingIndicator.style.display = visible ? "" : "none";
+ listItems.style.display = visible ? "none" : "";
+}
+setIndicatorVisibility(false);
+function setMoreButtonVisibility(visible) {
+ moreButton.style.display = visible ? "" : "none";
+}
+setMoreButtonVisibility(true);
+
+const sortByLogoAvailability = (a, b) => {
+ const a_key = a.logo ? 1 : -1;
+ const b_key = b.logo ? 1 : -1;
+ return a_key - b_key;
+};
+
+const sortByName = (a, b) => {
+ return a.name.toLowerCase().localeCompare(b.name.toLowerCase());
+};
+
+const sortByIndex = (a, b) => {
+ return a.index - b.index;
+};
+
+let fetchIntegrationRequest = null;
+
+function fetchIntegration() {
+ if (fetchIntegrationRequest) {
+return fetchIntegrationRequest;
+ }
+ const request = Promise.all([
+fetch("/integrations.json"),
+() => {
+ setIndicatorVisibility(true);
+}
+ ])
+.then(([resp]) => resp.json())
+.then((integrations) => {
+ setIndicatorVisibility(false);
+ return Promise.resolve(integrations);
+})
+.then((integrations) => {
+ integrations = shuffle(integrations);
+ integrations.forEach((i, index) => i.inddex = index);
+ integrations.sort(sortByLogoAvailability);
+ return Promise.resolve(integrations);
+});
+ fetchIntegrationRequest = request;
+ return request;
+}
+
+function createElement(item) {
+ const element = templateElement.cloneNode(true);
+ element.querySelector('[data-name="name"]').innerText = item.name;
+ element.querySelector("a").href = item.url;
+ return element.firstElementChild;
+}
+
+function setItems(items) {
+ if (items.length === 0) {
+listItems.innerText = "No items";
+ } else {
+while (listItems.firstChild) {
+ listItems.removeChild(listItems.firstChild);
+}
+items.forEach((item) => {
+ const element = createElement(item);
+ listItems.append(element);
+});
+ }
+}
+
+function showItems(keyword, page) {
+ const showMoreButtonIfNeeded = (integrations) => {
+setMoreButtonVisibility(integrations.length > (page * 8));
+ };
+
+ const filterMatchingItems = (integrations) => {
+if (!keyword) {
+ return Promise.resolve(integrations);
+}
+const selectedIntegration = integrations.filter(
+ (integration) => integration.name.indexOf(keyword) >= 0
+);
+return Promise.resolve(selectedIntegration);
+ };
+
+ const filterVisible = (integrations) => {
+if (currentPage === 1) {
+ integrations.sort(sortByIndex);
+} else {
+ integrations.sort(sortByName);
Review comment:
Viewing a long random list will be more confusing than reordering a few
items after clicking, but I can change that and ask for feedback tomorrow.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow-site] kgabryje commented on issue #96: Add integration search
kgabryje commented on issue #96: Add integration search URL: https://github.com/apache/airflow-site/pull/96#issuecomment-547719985 I see, thank you for explanation This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] kgabryje commented on a change in pull request #96: Add integration search
kgabryje commented on a change in pull request #96: Add integration search
URL: https://github.com/apache/airflow-site/pull/96#discussion_r340413637
##
File path: landing-pages/src/js/integrationList.js
##
@@ -0,0 +1,153 @@
+
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+import debounce from "lodash/debounce";
+import shuffle from "lodash/shuffle";
+
+const root = document.querySelector("#integrations");
+
+const templateText = root.querySelector("#integration-template").innerText;
+const templateElement = document.createElement("div");
+templateElement.innerHTML = templateText;
+
+const loadingIndicator = root.querySelector(".loading");
+const listItems = root.querySelector(".list-items");
+const searchBox = root.querySelector('[type="search"]');
+const moreButton = root.querySelector("#show-more-integration");
+
+let currentPage = 1;
+let currentQeury = "";
+function setIndicatorVisibility(visible) {
+ loadingIndicator.style.display = visible ? "" : "none";
+ listItems.style.display = visible ? "none" : "";
+}
+setIndicatorVisibility(false);
+function setMoreButtonVisibility(visible) {
+ moreButton.style.display = visible ? "" : "none";
+}
+setMoreButtonVisibility(true);
+
+const sortByLogoAvailability = (a, b) => {
+ const a_key = a.logo ? 1 : -1;
+ const b_key = b.logo ? 1 : -1;
+ return a_key - b_key;
+};
+
+const sortByName = (a, b) => {
+ return a.name.toLowerCase().localeCompare(b.name.toLowerCase());
+};
+
+const sortByIndex = (a, b) => {
+ return a.index - b.index;
+};
+
+let fetchIntegrationRequest = null;
+
+function fetchIntegration() {
+ if (fetchIntegrationRequest) {
+return fetchIntegrationRequest;
+ }
+ const request = Promise.all([
+fetch("/integrations.json"),
+() => {
+ setIndicatorVisibility(true);
+}
+ ])
+.then(([resp]) => resp.json())
+.then((integrations) => {
+ setIndicatorVisibility(false);
+ return Promise.resolve(integrations);
+})
+.then((integrations) => {
+ integrations = shuffle(integrations);
+ integrations.forEach((i, index) => i.inddex = index);
+ integrations.sort(sortByLogoAvailability);
+ return Promise.resolve(integrations);
+});
+ fetchIntegrationRequest = request;
+ return request;
+}
+
+function createElement(item) {
+ const element = templateElement.cloneNode(true);
+ element.querySelector('[data-name="name"]').innerText = item.name;
+ element.querySelector("a").href = item.url;
+ return element.firstElementChild;
+}
+
+function setItems(items) {
+ if (items.length === 0) {
+listItems.innerText = "No items";
+ } else {
+while (listItems.firstChild) {
+ listItems.removeChild(listItems.firstChild);
+}
+items.forEach((item) => {
+ const element = createElement(item);
+ listItems.append(element);
+});
+ }
+}
+
+function showItems(keyword, page) {
+ const showMoreButtonIfNeeded = (integrations) => {
+setMoreButtonVisibility(integrations.length > (page * 8));
+ };
+
+ const filterMatchingItems = (integrations) => {
+if (!keyword) {
+ return Promise.resolve(integrations);
+}
+const selectedIntegration = integrations.filter(
+ (integration) => integration.name.indexOf(keyword) >= 0
+);
+return Promise.resolve(selectedIntegration);
+ };
+
+ const filterVisible = (integrations) => {
+if (currentPage === 1) {
+ integrations.sort(sortByIndex);
+} else {
+ integrations.sort(sortByName);
Review comment:
I agree, but we achieve randomness by shuffling integrations on line 79.
What I mean here is maybe we shouldn't change the type of sorting after
clicking button (so it would just be sorted by index).
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow-site] kgabryje commented on issue #96: Add integration search
kgabryje commented on issue #96: Add integration search URL: https://github.com/apache/airflow-site/pull/96#issuecomment-547718031 Does it work well for you? I see that searchbar is un-styled. Also, there is an error logged from `handleActiveVideo.js`, which I think I fixed in previous commits by changing line 29 from ` if (!videosList) return` to `if (videosList.length === 0) return` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21]
mingrammer edited a comment on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21] URL: https://github.com/apache/airflow/pull/6455#issuecomment-547716869 I did https://github.com/apache/airflow/pull/6465#issuecomment-547707539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer commented on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21]
mingrammer commented on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21] URL: https://github.com/apache/airflow/pull/6455#issuecomment-547716869 https://github.com/apache/airflow/pull/6465#issuecomment-547707539 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547707539 I sent a new thread on the mailing list about it! https://lists.apache.org/thread.html/5ed0ce4dfd77640d7464dc57d4fafc75e8aac44e016f87ca1be3a4a3@%3Cdev.airflow.apache.org%3E This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547707539 I sent a new thread on the mailing list about it! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547707539 I sent a new thread about it! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on issue #96: Add integration search
mik-laj commented on issue #96: Add integration search URL: https://github.com/apache/airflow-site/pull/96#issuecomment-547711217 @kgabryje Can i ask for review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on issue #94: [WIP] Add animations with flying cubes
mik-laj commented on issue #94: [WIP] Add animations with flying cubes URL: https://github.com/apache/airflow-site/pull/94#issuecomment-547711031 @kgabryje Can i ask for review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on issue #95: [WIP] Add search blog posts
mik-laj commented on issue #95: [WIP] Add search blog posts URL: https://github.com/apache/airflow-site/pull/95#issuecomment-547711072 @kgabryje Can i ask for review? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] kgabryje commented on a change in pull request #103: [depends on #96] Search meetups
kgabryje commented on a change in pull request #103: [depends on #96] Search
meetups
URL: https://github.com/apache/airflow-site/pull/103#discussion_r340405744
##
File path: landing-pages/src/js/meetupsList.js
##
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+
+const runMeetups = () => {
+ const root = document.querySelector(".meetups");
+ if (!root) {
+return;
+ }
+ const templateText = root.querySelector("#meetup-template").innerText;
+ const templateElement = document.createElement("div");
+ templateElement.innerHTML = templateText;
+
+ const listItems = root.querySelector(".list-items");
+ const searchBox = root.querySelector("[type='search']");
+ const moreButton = root.querySelector(".more");
+
+ let currentPage = 1;
+ let currentQuery = "";
+ fetch("/data/meetups.json")
Review comment:
I created it manually (/dist/data/meetups) for testing purposes
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547707539 I sent a new thread about it. But it is not listed in https://lists.apache.org/list.html?d...@airflow.apache.org:2019-10. Are these `d...@airflow.apache.org` and `dev-subscr...@airflow.apache.org` different? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on a change in pull request #103: [depends on #96] Search meetups
mik-laj commented on a change in pull request #103: [depends on #96] Search
meetups
URL: https://github.com/apache/airflow-site/pull/103#discussion_r340405072
##
File path: landing-pages/src/js/meetupsList.js
##
@@ -0,0 +1,118 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+
+const runMeetups = () => {
+ const root = document.querySelector(".meetups");
+ if (!root) {
+return;
+ }
+ const templateText = root.querySelector("#meetup-template").innerText;
+ const templateElement = document.createElement("div");
+ templateElement.innerHTML = templateText;
+
+ const listItems = root.querySelector(".list-items");
+ const searchBox = root.querySelector("[type='search']");
+ const moreButton = root.querySelector(".more");
+
+ let currentPage = 1;
+ let currentQuery = "";
+ fetch("/data/meetups.json")
Review comment:
For me, this file does not exist in the static directory, so it cannot be
accessed from the browser.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mingrammer commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547707539 I sent a new thread about it. But it is not listed in https://lists.apache.org/list.html?d...@airflow.apache.org:2019-10 Hmm..? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5811) Add metric for externally killed task count
[
https://issues.apache.org/jira/browse/AIRFLOW-5811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962628#comment-16962628
]
ASF GitHub Bot commented on AIRFLOW-5811:
-
houqp commented on pull request #6466: [AIRFLOW-5811] add metric for externally
killed task count
URL: https://github.com/apache/airflow/pull/6466
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
We found this to be a very good signal to alert for any init container
related issue. Our worker cluster was affected by a bug in EKS CNI plugin. This
metric would have caught the issue automatically.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Looks like we don't test for stats?
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add metric for externally killed task count
> ---
>
> Key: AIRFLOW-5811
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5811
> Project: Apache Airflow
> Issue Type: Improvement
> Components: scheduler
>Affects Versions: 2.0.0
>Reporter: QP Hou
>Assignee: QP Hou
>Priority: Trivial
>
> Number of externally killed task would be very useful for monitoring init
> container error when using k8s executor.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] houqp opened a new pull request #6466: [AIRFLOW-5811] add metric for externally killed task count
houqp opened a new pull request #6466: [AIRFLOW-5811] add metric for externally
killed task count
URL: https://github.com/apache/airflow/pull/6466
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-XXX
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
We found this to be a very good signal to alert for any init container
related issue. Our worker cluster was affected by a bug in EKS CNI plugin. This
metric would have caught the issue automatically.
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
Looks like we don't test for stats?
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5811) Add metric for externally killed task count
QP Hou created AIRFLOW-5811: --- Summary: Add metric for externally killed task count Key: AIRFLOW-5811 URL: https://issues.apache.org/jira/browse/AIRFLOW-5811 Project: Apache Airflow Issue Type: Improvement Components: scheduler Affects Versions: 2.0.0 Reporter: QP Hou Assignee: QP Hou Number of externally killed task would be very useful for monitoring init container error when using k8s executor. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5562) Skip grant single DAG permissions for Admin role
[ https://issues.apache.org/jira/browse/AIRFLOW-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962599#comment-16962599 ] ASF subversion and git services commented on AIRFLOW-5562: -- Commit d800ed66320e45c5bcf6782176553ceb82050011 in airflow's branch refs/heads/master from Liu Xuesi [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d800ed6 ] [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. (#6199) * [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. - Admin role have all permissions so it does not need to be re-authorized. - Too many permissions for role is not good for view and performance. * [AIRFLOW-5562] Fix typo in last change. > Skip grant single DAG permissions for Admin role > > > Key: AIRFLOW-5562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5562 > Project: Apache Airflow > Issue Type: Improvement > Components: security >Affects Versions: 1.10.3, 1.10.4, 1.10.5 >Reporter: Liu Xuesi >Priority: Major > Labels: security, security-groups > Attachments: admin_permission_full_of_dags.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > From AIRFLOW-2267，there is a function named *update_admin_perm_view* will > refresh admin permission then add ALL permission to Admin role. > But, DAG level access make each DAG a MenuView, these views will be grant to > Admin role. As Admin role already have access to *all_dags*, these > permissions actually make Admin role's permission more chaotic. > In my project, it is hard to check permissions in webUI and actually this > lead to some performance issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5562) Skip grant single DAG permissions for Admin role
[ https://issues.apache.org/jira/browse/AIRFLOW-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962601#comment-16962601 ] ASF subversion and git services commented on AIRFLOW-5562: -- Commit d800ed66320e45c5bcf6782176553ceb82050011 in airflow's branch refs/heads/master from Liu Xuesi [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d800ed6 ] [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. (#6199) * [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. - Admin role have all permissions so it does not need to be re-authorized. - Too many permissions for role is not good for view and performance. * [AIRFLOW-5562] Fix typo in last change. > Skip grant single DAG permissions for Admin role > > > Key: AIRFLOW-5562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5562 > Project: Apache Airflow > Issue Type: Improvement > Components: security >Affects Versions: 1.10.3, 1.10.4, 1.10.5 >Reporter: Liu Xuesi >Assignee: Liu Xuesi >Priority: Major > Labels: security, security-groups > Fix For: 1.10.7 > > Attachments: admin_permission_full_of_dags.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > From AIRFLOW-2267，there is a function named *update_admin_perm_view* will > refresh admin permission then add ALL permission to Admin role. > But, DAG level access make each DAG a MenuView, these views will be grant to > Admin role. As Admin role already have access to *all_dags*, these > permissions actually make Admin role's permission more chaotic. > In my project, it is hard to check permissions in webUI and actually this > lead to some performance issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5562) Skip grant single DAG permissions for Admin role
[ https://issues.apache.org/jira/browse/AIRFLOW-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tao Feng resolved AIRFLOW-5562. --- Fix Version/s: 1.10.7 Assignee: Liu Xuesi Resolution: Fixed > Skip grant single DAG permissions for Admin role > > > Key: AIRFLOW-5562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5562 > Project: Apache Airflow > Issue Type: Improvement > Components: security >Affects Versions: 1.10.3, 1.10.4, 1.10.5 >Reporter: Liu Xuesi >Assignee: Liu Xuesi >Priority: Major > Labels: security, security-groups > Fix For: 1.10.7 > > Attachments: admin_permission_full_of_dags.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > From AIRFLOW-2267，there is a function named *update_admin_perm_view* will > refresh admin permission then add ALL permission to Admin role. > But, DAG level access make each DAG a MenuView, these views will be grant to > Admin role. As Admin role already have access to *all_dags*, these > permissions actually make Admin role's permission more chaotic. > In my project, it is hard to check permissions in webUI and actually this > lead to some performance issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5562) Skip grant single DAG permissions for Admin role
[ https://issues.apache.org/jira/browse/AIRFLOW-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962600#comment-16962600 ] ASF subversion and git services commented on AIRFLOW-5562: -- Commit d800ed66320e45c5bcf6782176553ceb82050011 in airflow's branch refs/heads/master from Liu Xuesi [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=d800ed6 ] [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. (#6199) * [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. - Admin role have all permissions so it does not need to be re-authorized. - Too many permissions for role is not good for view and performance. * [AIRFLOW-5562] Fix typo in last change. > Skip grant single DAG permissions for Admin role > > > Key: AIRFLOW-5562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5562 > Project: Apache Airflow > Issue Type: Improvement > Components: security >Affects Versions: 1.10.3, 1.10.4, 1.10.5 >Reporter: Liu Xuesi >Priority: Major > Labels: security, security-groups > Fix For: 1.10.7 > > Attachments: admin_permission_full_of_dags.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > From AIRFLOW-2267，there is a function named *update_admin_perm_view* will > refresh admin permission then add ALL permission to Admin role. > But, DAG level access make each DAG a MenuView, these views will be grant to > Admin role. As Admin role already have access to *all_dags*, these > permissions actually make Admin role's permission more chaotic. > In my project, it is hard to check permissions in webUI and actually this > lead to some performance issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] feng-tao merged pull request #6199: [AIRFLOW-5562] Skip grant single DAG permissions for Admin role.
feng-tao merged pull request #6199: [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. URL: https://github.com/apache/airflow/pull/6199 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5562) Skip grant single DAG permissions for Admin role
[ https://issues.apache.org/jira/browse/AIRFLOW-5562?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962598#comment-16962598 ] ASF GitHub Bot commented on AIRFLOW-5562: - feng-tao commented on pull request #6199: [AIRFLOW-5562] Skip grant single DAG permissions for Admin role. URL: https://github.com/apache/airflow/pull/6199 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Skip grant single DAG permissions for Admin role > > > Key: AIRFLOW-5562 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5562 > Project: Apache Airflow > Issue Type: Improvement > Components: security >Affects Versions: 1.10.3, 1.10.4, 1.10.5 >Reporter: Liu Xuesi >Priority: Major > Labels: security, security-groups > Attachments: admin_permission_full_of_dags.jpg > > Original Estimate: 168h > Remaining Estimate: 168h > > From AIRFLOW-2267，there is a function named *update_admin_perm_view* will > refresh admin permission then add ALL permission to Admin role. > But, DAG level access make each DAG a MenuView, these views will be grant to > Admin role. As Admin role already have access to *all_dags*, these > permissions actually make Admin role's permission more chaotic. > In my project, it is hard to check permissions in webUI and actually this > lead to some performance issues. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[airflow-site] branch aip-11 updated: Add navbar (#99)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new ccc6c45 Add navbar (#99)
ccc6c45 is described below
commit ccc6c456a09de22150af750e94252baa30f8c0f4
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 30 01:28:49 2019 +0100
Add navbar (#99)
---
landing-pages/site/assets/icons/airflow-logo.svg | 25
landing-pages/site/assets/icons/close-icon.svg | 10 ++
landing-pages/site/assets/icons/hamburger-icon.svg | 13 ++
landing-pages/site/assets/scss/_base-layout.scss | 16 ++-
landing-pages/site/assets/scss/_navbar.scss| 154 +
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/content/en/blog/_index.md | 2 +-
.../site/content/en/case-studies/_index.html | 2 +-
landing-pages/site/content/en/docs/_index.md | 2 +-
landing-pages/site/content/en/install/_index.html | 2 +-
landing-pages/site/content/en/meetups/_index.html | 2 +-
landing-pages/site/layouts/partials/navbar.html| 77 +++
landing-pages/src/index.js | 2 +
.../scss/main-custom.scss => src/js/drawer.js} | 39 +++---
landing-pages/src/{index.js => js/navbarScroll.js} | 12 +-
15 files changed, 322 insertions(+), 37 deletions(-)
diff --git a/landing-pages/site/assets/icons/airflow-logo.svg
b/landing-pages/site/assets/icons/airflow-logo.svg
new file mode 100644
index 000..bd3a762
--- /dev/null
+++ b/landing-pages/site/assets/icons/airflow-logo.svg
@@ -0,0 +1,25 @@
+http://www.w3.org/2000/svg; width="155.314" height="60" viewBox="0
0 155.314 60">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+http://www.w3.org/2000/svg; width="19.799" height="19.799"
viewBox="0 0 19.799 19.799">
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/hamburger-icon.svg
b/landing-pages/site/assets/icons/hamburger-icon.svg
new file mode 100644
index 000..0c7567c
--- /dev/null
+++ b/landing-pages/site/assets/icons/hamburger-icon.svg
@@ -0,0 +1,13 @@
+http://www.w3.org/2000/svg; width="26" height="20" viewBox="0 0 26
20">
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/scss/_base-layout.scss
b/landing-pages/site/assets/scss/_base-layout.scss
index 5fe0023..be4dfef 100644
--- a/landing-pages/site/assets/scss/_base-layout.scss
+++ b/landing-pages/site/assets/scss/_base-layout.scss
@@ -20,7 +20,7 @@
@import "fonts";
.base-layout {
- padding: 44px 0 40px;
+ padding: 123px 0 40px;
&--button {
display: flex;
@@ -34,7 +34,7 @@
@media (max-width: $mobile) {
.base-layout {
-padding: 0 0 60px;
+padding: 76px 0 60px;
&--button {
display: none;
@@ -98,3 +98,15 @@
margin-top: 30px;
}
}
+
+.desktop-only {
+ @media (max-width: $mobile) {
+display: none;
+ }
+}
+
+.mobile-only {
+ @media (min-width: calc(#{$mobile} + 1px)) {
+display: none;
+ }
+}
diff --git a/landing-pages/site/assets/scss/_navbar.scss
b/landing-pages/site/assets/scss/_navbar.scss
new file mode 100644
index 000..f4f1659
--- /dev/null
+++ b/landing-pages/site/assets/scss/_navbar.scss
@@ -0,0 +1,154 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+@import "colors";
+@import "media";
+
+.navbar {
+ position: fixed;
+ top: 0;
+ width: 100%;
+ background-color: white;
+ justify-content: flex-start;
+ border-bottom: solid 1px map-get($colors, very-light-pink);
+ z-index: 32;
+ padding: 30px 60px;
+
+ &__menu-container {
+flex-grow: 1;
+ }
+
+ &__menu-content {
+display: flex;
+justify-content: space-between;
+align-items: center;
+padding-top: 16px;
+padding-left: 88px;
+ }
+
+ &__links-container {
+
[GitHub] [airflow-site] mik-laj merged pull request #99: Add navbar
mik-laj merged pull request #99: Add navbar URL: https://github.com/apache/airflow-site/pull/99 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj merged pull request #98: Add footer and sticky suggest a change button
mik-laj merged pull request #98: Add footer and sticky suggest a change button URL: https://github.com/apache/airflow-site/pull/98 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Add footer and sticky suggest a change button (#98)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 47bfe5d Add footer and sticky suggest a change button (#98)
47bfe5d is described below
commit 47bfe5d635c5e77eb5ae07ad8fb25644e97a66a2
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 30 00:36:58 2019 +0100
Add footer and sticky suggest a change button (#98)
---
landing-pages/site/assets/icons/github-white.svg | 3 +
landing-pages/site/assets/icons/jira.svg | 7 ++
landing-pages/site/assets/icons/slack.svg | 20 +++
landing-pages/site/assets/icons/stackoverflow.svg | 6 +
landing-pages/site/assets/icons/twitter-white.svg | 5 +
landing-pages/site/assets/icons/youtube.svg| 3 +
landing-pages/site/assets/scss/_base-layout.scss | 7 +-
landing-pages/site/assets/scss/_buttons.scss | 7 --
landing-pages/site/assets/scss/_footer.scss| 134 +
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/config.toml | 69 ++-
landing-pages/site/layouts/_default/baseof.html| 2 +-
landing-pages/site/layouts/blog/baseof.html| 4 +-
.../site/layouts/case-studies/baseof.html | 2 +-
landing-pages/site/layouts/community/baseof.html | 2 +-
landing-pages/site/layouts/install/baseof.html | 2 +-
landing-pages/site/layouts/meetups/baseof.html | 2 +-
landing-pages/site/layouts/partials/footer.html| 58 +
.../site/layouts/partials/lang-selector.html | 28 +
19 files changed, 313 insertions(+), 49 deletions(-)
diff --git a/landing-pages/site/assets/icons/github-white.svg
b/landing-pages/site/assets/icons/github-white.svg
new file mode 100644
index 000..3bf8a77
--- /dev/null
+++ b/landing-pages/site/assets/icons/github-white.svg
@@ -0,0 +1,3 @@
+http://www.w3.org/2000/svg; width="46.14" height="45" viewBox="0 0
46.14 45">
+http://www.w3.org/2000/svg; width="45" height="45" viewBox="0 0 45
45">
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/slack.svg
b/landing-pages/site/assets/icons/slack.svg
new file mode 100644
index 000..c804682
--- /dev/null
+++ b/landing-pages/site/assets/icons/slack.svg
@@ -0,0 +1,20 @@
+http://www.w3.org/2000/svg; width="45.073" height="45.073"
viewBox="0 0 45.073 45.073">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/stackoverflow.svg
b/landing-pages/site/assets/icons/stackoverflow.svg
new file mode 100644
index 000..e4dc7da
--- /dev/null
+++ b/landing-pages/site/assets/icons/stackoverflow.svg
@@ -0,0 +1,6 @@
+http://www.w3.org/2000/svg; width="37.647" height="44.6"
viewBox="0 0 37.647 44.6">
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/twitter-white.svg
b/landing-pages/site/assets/icons/twitter-white.svg
new file mode 100644
index 000..ba99e0b
--- /dev/null
+++ b/landing-pages/site/assets/icons/twitter-white.svg
@@ -0,0 +1,5 @@
+http://www.w3.org/2000/svg; width="44.355" height="44.355"
viewBox="0 0 44.355 44.355">
+
+http://www.w3.org/2000/svg; width="49.594" height="34.941"
viewBox="0 0 49.594 34.941">
+http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied. See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+@import "colors";
+@import "media";
+
+footer {
+ min-height: unset;
+
+ .footer-section {
+display: flex;
+flex-wrap: wrap;
+justify-content: space-between;
+align-items: center;
+
+span {
+ @extend .bodytext__medium--white;
+}
+
+&__media-section {
+ padding: 60px 60px 30px;
+ background-color: map-get($colors, greyish-brown);
+
+ &--link {
+margin-right: 30px;
+ }
+
+ &--button-with-text {
+display: flex;
+align-items: center;
+ }
+
+ &--text {
+margin-right: 20px;
+ }
+}
+
+&__policies-section {
+ padding: 30px 60px;
+ background-color: map-get($colors, slate-grey);
+
+ &--policies {
+display: flex;
+ }
+
+ &--policy-item {
+&::before {
+ content: "\00a0\00a0";
+}
+&::after {
+ content: "\00a0\00a0|";
+ color: white;
+}
+ }
+}
+ }
+
+ .dropdown-toggle {
+&::after {
+ color: white;
+ vertical-align: middle;
+}
[airflow-site] branch aip-11 updated: Improve case-study/person pagination (#102)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new c92f732 Improve case-study/person pagination (#102)
c92f732 is described below
commit c92f7328ed84809bca17412df047e9854bcb05a8
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 30 00:25:01 2019 +0100
Improve case-study/person pagination (#102)
---
landing-pages/site/assets/scss/_base-layout.scss | 9 +++
.../site/assets/scss/_community-page.scss | 5 --
.../site/content/en/case-studies/example-case7.md | 23 ++
.../site/content/en/case-studies/example-case8.md | 23 ++
.../site/content/en/case-studies/example-case9.md | 23 ++
landing-pages/site/data/commiters.json | 40 ++
landing-pages/site/layouts/case-studies/list.html | 5 +-
landing-pages/site/layouts/community/list.html | 92 +++---
landing-pages/src/index.js | 7 +-
landing-pages/src/js/showAllCommiters.js | 33 +---
10 files changed, 196 insertions(+), 64 deletions(-)
diff --git a/landing-pages/site/assets/scss/_base-layout.scss
b/landing-pages/site/assets/scss/_base-layout.scss
index e235cb6..e09d0e3 100644
--- a/landing-pages/site/assets/scss/_base-layout.scss
+++ b/landing-pages/site/assets/scss/_base-layout.scss
@@ -88,3 +88,12 @@
padding-left: 20px;
padding-right: 20px;
}
+
+.show-more-button {
+ width: fit-content;
+ margin: 60px auto 0;
+
+ @media (max-width: 640px) {
+margin-top: 30px;
+ }
+}
diff --git a/landing-pages/site/assets/scss/_community-page.scss
b/landing-pages/site/assets/scss/_community-page.scss
index 093a28d..e7a5cc1 100644
--- a/landing-pages/site/assets/scss/_community-page.scss
+++ b/landing-pages/site/assets/scss/_community-page.scss
@@ -51,11 +51,6 @@
margin-bottom: 40px;
}
}
-
- &--button-container {
-width: fit-content;
-margin: 60px auto 0;
- }
}
@media (max-width: $mobile) {
diff --git a/landing-pages/site/content/en/case-studies/example-case7.md
b/landing-pages/site/content/en/case-studies/example-case7.md
new file mode 100644
index 000..9100be7
--- /dev/null
+++ b/landing-pages/site/content/en/case-studies/example-case7.md
@@ -0,0 +1,23 @@
+---
+title: "Example 7"
+linkTitle: "Example 7"
+quote:
+text: "A great ecosystem and community that comes together to address
about any batch data pipeline need."
+author: "Austin Benett, CTO at Spotify"
+logo_path: "icons/dish-logo.svg"
+---
+
+# What was the problem?
+We faced increasing complexity managing lengthy crontabs with scheduling being
an issue, this required carefully planning timing due to resource constraints,
usage patterns, and especially custom code needed for retry logic. In the last
case, having to verify success of previous jobs and/or steps prior to running
the next. Furthermore, time to results is important, but we were increasingly
relying on buffers for processing, where things were effectively sitting idle
and not processing, [...]
+
+# How did Apache Airflow help to solve this problem?
+Relying on community built and existing hooks and operators to the majority of
cloud services we use has allowed us to focus on business outcomes.
+
+# What are the results?
+Airflow helps us manage many of our pain-points, letting us benefit from the
overall ecosystem and
+community. We are able to reduce time-to-end delivery of data products by
being event-driven in our
+processing flows (in our first usage, for example, we were able to take out
over 2 hours - on average - of various
+waiting between stages). Furthermore, we are able to arrive at and iterate on
products quicker as a result of
+not needing as much custom or roll-our-own solutions. For Our code base is
smaller and simpler, it is easier to
+follow, and to a large extent our DAGs serve as sufficient documentation for
new contributors to understand
+what is going on.
diff --git a/landing-pages/site/content/en/case-studies/example-case8.md
b/landing-pages/site/content/en/case-studies/example-case8.md
new file mode 100644
index 000..5b23c08
--- /dev/null
+++ b/landing-pages/site/content/en/case-studies/example-case8.md
@@ -0,0 +1,23 @@
+---
+title: "Example 8"
+linkTitle: "Example 8"
+quote:
+text: "A great ecosystem and community that comes together to address
about any batch data pipeline need."
+author: "Austin Benett, CTO at Spotify"
+logo_path: "icons/dish-logo.svg"
+---
+
+# What was the problem?
+We faced increasing complexity managing lengthy crontabs with scheduling being
an issue, this required carefully planning timing due to resource constraints,
usage patterns, and especially custom code needed for retry logic. In the last
case, having to verify success of previous jobs and/or steps prior to running
the next.
[GitHub] [airflow-site] mik-laj merged pull request #102: Improve case-study/person pagination
mik-laj merged pull request #102: Improve case-study/person pagination URL: https://github.com/apache/airflow-site/pull/102 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-2243) airflow seems to load modules multiple times
[
https://issues.apache.org/jira/browse/AIRFLOW-2243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962525#comment-16962525
]
Eric Gustavson commented on AIRFLOW-2243:
-
+1 I have been experiencing unexpected import behavior with airflow
(specifically composer 1.9.0 with python 3)
> airflow seems to load modules multiple times
>
>
> Key: AIRFLOW-2243
> URL: https://issues.apache.org/jira/browse/AIRFLOW-2243
> Project: Apache Airflow
> Issue Type: Bug
>Reporter: sulphide
>Priority: Major
>
> airflow uses the builtin imp.load_source to load modules, but the
> documentation (for python 2) says:
> [https://docs.python.org/2/library/imp.html#imp.load_source]
> {{imp.}}{{load_source}}(_name_, _pathname_[, _file_])
> Load and initialize a module implemented as a Python source file and return
> its module object. If the module was already initialized, it will be
> initialized _again_. The _name_ argument is used to create or access a module
> object. The_pathname_ argument points to the source file. The _file_ argument
> is the source file, open for reading as text, from the beginning. It must
> currently be a real file object, not a user-defined class emulating a file.
> Note that if a properly matching byte-compiled file (with suffix {{.pyc}} or
> {{.pyo}}) exists, it will be used instead of parsing the given source file.
>
>
> this means that airflow behaves differently from a typical python program in
> that a module may be imported multiple times, which could have unexpected
> effects for those relying on the typical python import semantics.
> https://github.com/apache/incubator-airflow/blob/master/airflow/models.py#L300
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow-site] mik-laj merged pull request #97: Add mobile layouts
mik-laj merged pull request #97: Add mobile layouts URL: https://github.com/apache/airflow-site/pull/97 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Add mobile layouts (#97)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 696f839 Add mobile layouts (#97)
696f839 is described below
commit 696f839c803b7b1ec263c03a752ce905818681bf
Author: Kamil Gabryjelski
AuthorDate: Tue Oct 29 23:53:24 2019 +0100
Add mobile layouts (#97)
---
landing-pages/site/assets/scss/_accordion.scss | 25 +++
landing-pages/site/assets/scss/_base-layout.scss | 45 ++-
landing-pages/site/assets/scss/_blog-page.scss | 38 ++
landing-pages/site/assets/scss/_case-study.scss| 4 +
.../site/assets/scss/_community-page.scss | 39 +-
landing-pages/site/assets/scss/_feature.scss | 45 +++
.../assets/scss/{_meetups.scss => _fonts.scss} | 10 +--
landing-pages/site/assets/scss/_home-page.scss | 6 ++
landing-pages/site/assets/scss/_install-page.scss | 87 --
landing-pages/site/assets/scss/_list-boxes.scss| 61 +++
.../assets/scss/{_case-study.scss => _media.scss} | 6 +-
landing-pages/site/assets/scss/_meetups.scss | 6 ++
landing-pages/site/assets/scss/_quote.scss | 7 ++
.../site/assets/scss/_text-with-icon.scss | 24 ++
landing-pages/site/assets/scss/_typography.scss| 5 +-
landing-pages/site/assets/scss/_video.scss | 11 +++
.../site/content/en/community/_index.html | 2 +-
landing-pages/site/content/en/install/_index.html | 2 +-
landing-pages/site/layouts/blog/list.html | 2 +-
landing-pages/site/layouts/blog/single.html| 4 +-
.../site/layouts/case-studies/single.html | 2 +-
landing-pages/site/layouts/community/list.html | 66
landing-pages/site/layouts/install/list.html | 27 +++
landing-pages/site/layouts/meetups/list.html | 42 +--
.../site/layouts/partials/boxes/blogpost.html | 28 +++
.../site/layouts/partials/boxes/case-study.html| 19 ++---
.../site/layouts/partials/boxes/commiter.html | 54 +++---
.../site/layouts/partials/boxes/event.html | 33
.../site/layouts/partials/boxes/integration.html | 10 +--
landing-pages/site/layouts/partials/feature.html | 4 +-
.../site/layouts/shortcodes/accordion.html | 6 +-
31 files changed, 516 insertions(+), 204 deletions(-)
diff --git a/landing-pages/site/assets/scss/_accordion.scss
b/landing-pages/site/assets/scss/_accordion.scss
index c5410e7..6bdcce6 100644
--- a/landing-pages/site/assets/scss/_accordion.scss
+++ b/landing-pages/site/assets/scss/_accordion.scss
@@ -17,6 +17,7 @@
* under the License.
*/
@import "colors";
+@import "media";
details.accordion {
padding: 40px 30px;
@@ -38,6 +39,7 @@ details.accordion {
.accordion__summary-content {
display: flex;
max-width: 750px;
+margin-right: 40px;
&--icon {
width: 60px;
@@ -77,9 +79,32 @@ details.accordion {
.accordion__content {
margin-top: 30px;
+margin-right: 36px;
&.indented {
margin-left: 102px;
}
}
}
+
+@media (max-width: $mobile) {
+ details.accordion {
+padding: 30px 0;
+
+.accordion__summary-content {
+ &--icon {
+margin-right: 20px;
+ }
+}
+
+.accordion__content {
+ &.indented {
+margin-left: 80px;
+ }
+
+ ol.counter-blue {
+margin-left: -38px !important;
+ }
+}
+ }
+}
diff --git a/landing-pages/site/assets/scss/_base-layout.scss
b/landing-pages/site/assets/scss/_base-layout.scss
index c94204e..e235cb6 100644
--- a/landing-pages/site/assets/scss/_base-layout.scss
+++ b/landing-pages/site/assets/scss/_base-layout.scss
@@ -16,9 +16,11 @@
* specific language governing permissions and limitations
* under the License.
*/
+@import "media";
+@import "fonts";
.base-layout {
- padding: 64px 0 40px;
+ padding: 44px 0 40px;
&--button {
display: flex;
@@ -26,19 +28,42 @@
margin-right: 45px;
margin-top: 80px
}
+
+}
+
+@media (max-width: $mobile) {
+ .base-layout {
+padding: 0 0 60px;
+
+&--button {
+ display: none;
+}
+ }
}
.page-header {
@extend .header__medium--greyish-brown;
text-align: center;
margin-bottom: 16px;
+
+ @media (max-width: $mobile) {
+font-size: 36px !important;
+line-height: 1.22 !important;
+ }
}
.page-subtitle {
@extend .subtitle__large--brownish-grey;
text-align: center;
- font-weight: normal;
+ font-weight: normal !important;
margin-bottom: 80px;
+
+ @media (max-width: $mobile) {
+font-family: $primary-font !important;
+font-size: 16px !important;
+line-height: 1.63 !important;
+margin-bottom: 30px;
+ }
}
.container {
@@ -46,4 +71,20 @@
@media(min-width: 1200px) {
max-width: 1200px;
}
+
+ @media
[GitHub] [airflow] KyleJamesWalker edited a comment on issue #4931: [AIRFLOW-4109] Added support for passing the airflow fernet key to th…
KyleJamesWalker edited a comment on issue #4931: [AIRFLOW-4109] Added support
for passing the airflow fernet key to th…
URL: https://github.com/apache/airflow/pull/4931#issuecomment-547254353
I'm having the same issue but if I set the following:
```
AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__AIRFLOW__CORE__FERNET_KEY: "xxx="
AIRFLOW__KUBERNETES_ENVIRONMENT_VARIABLES__FERNET_KEY: "xxx="
```
The worker gets the variables set, but the key is lowercase, so it's not
being picked up...
So airflow doesn't pick them up:
```
fernet_key: xxx=
airflow__core__fernet_key: xxx=
```
Note: Previous env values did not have the `S` at the end of `VARIABLES`
For now, I added a hack and my DAG and it works just fine:
``` python
if os.getenv('fernet_key'):
os.environ['FERNET_KEY'] = os.environ['fernet_key']
```
Does anyone know how to get these to be set with uppercase values?
I'm assuming it's from here:
https://github.com/apache/airflow/blob/88989200a66291580088188f06a6db503ac823e2/airflow/configuration.py#L336-L339
Honestly, I think this code should have a special check for the k8s env
section, something like:
``` python
section_prefix = 'AIRFLOW__{S}__'.format(S=section.upper())
for env_var in sorted(os.environ.keys()):
if env_var.startswith(section_prefix):
if section == "kubernetes_environment_variables":
key = env_var.replace(section_prefix, '')
else:
key = env_var.replace(section_prefix, '').lower()
_section[key] = self._get_env_var_option(section, key)
```
Update we had to add the following for our k8s setup and it worked!!! the
config map has FERNET_KEY, and now the workers are happy <3
```
AIRFLOW__KUBERNETES__ENV_FROM_CONFIGMAP_REF: "airflow-env"
```
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Created] (AIRFLOW-5810) Using `-int` with `airflow run` with a kubernetesexecutor does not pass -int to container run command
Francesco Macagno created AIRFLOW-5810: -- Summary: Using `-int` with `airflow run` with a kubernetesexecutor does not pass -int to container run command Key: AIRFLOW-5810 URL: https://issues.apache.org/jira/browse/AIRFLOW-5810 Project: Apache Airflow Issue Type: Bug Components: executor-kubernetes Affects Versions: 1.10.3 Reporter: Francesco Macagno Assignee: Daniel Imberman Running `airflow run` with a task using a kubernetes executor does output the logs of the k8s status messages, but the resulting pod arguments should also include the `-int` tag so that the child pod logs include the output of the task. This not being the case makes testing k8s much harder. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (AIRFLOW-5809) Kubernetes executor never returns when run with 'airflow run ...`
Francesco Macagno created AIRFLOW-5809:
--
Summary: Kubernetes executor never returns when run with 'airflow
run ...`
Key: AIRFLOW-5809
URL: https://issues.apache.org/jira/browse/AIRFLOW-5809
Project: Apache Airflow
Issue Type: Bug
Components: executor-kubernetes
Affects Versions: 1.10.3
Reporter: Francesco Macagno
Assignee: Daniel Imberman
When running a task using `airflow run dag_id task_id date` from a minikube
instance, the command never returns, even after the child pod has completed and
been deleted.
{code:java}
airflow@airflow-85464bb66-c4j9m:~$ airflow run k8s-test python 20191010 -int
[2019-10-26 13:31:32,258] {settings.py:182} INFO - settings.configure_orm():
Using pool settings. pool_size=5, pool_recycle=1800, pid=38
/usr/local/lib/python3.6/site-packages/psycopg2/__init__.py:144: UserWarning:
The psycopg2 wheel package will be renamed from release 2.8; in order to keep
installing from binary please use "pip install psycopg2-binary" instead. For
details see:
.
""")
[2019-10-26 13:31:33,745] {__init__.py:51} INFO - Using executor
KubernetesExecutor
[2019-10-26 13:31:34,019] {__init__.py:305} INFO - Filling up the DagBag from
/usr/local/airflow/dags
[2019-10-26 13:31:34,235] {cli.py:517} INFO - Running on host
airflow-85464bb66-c4j9m
[2019-10-26 13:31:34,235] {kubernetes_executor.py:690} INFO - Start Kubernetes
executor
[2019-10-26 13:31:34,338] {kubernetes_executor.py:304} INFO - Event: and now my
watch begins starting at resource_version: 0
[2019-10-26 13:31:34,371] {kubernetes_executor.py:627} INFO - When executor
started up, found 0 queued task instances
Sending to executor.
[2019-10-26 13:31:34,378] {base_executor.py:59} INFO - Adding to queue:
['airflow', 'run', 'k8s-test', 'python', '2019-10-10T00:00:00+00:00',
'--local', '-sd', 'DAGS_FOLDER/hdfs_example.py']
[2019-10-26 13:31:34,381] {kubernetes_executor.py:711} INFO - Add task
('k8s-test', 'python', , 2) with command
['airflow', 'run', 'k8s-test', 'python', '2019-10-10T00:00:00+00:00',
'--local', '-sd', 'DAGS_FOLDER/hdfs_example.py'] with executor_config
{'KubernetesExecutor': {'image': 'registry.prod.factual.com/airflow-cdh5:1',
'image_pull_policy': 'Always', 'annotations': {'injector.factual.com/request':
'airflow-kinit'}}}
[2019-10-26 13:31:34,383] {kubernetes_executor.py:405} INFO - Kubernetes job is
(('k8s-test', 'python', , 2), ['airflow',
'run', 'k8s-test', 'python', '2019-10-10T00:00:00+00:00', '--local', '-sd',
'DAGS_FOLDER/hdfs_example.py'],
KubernetesExecutorConfig(image=registry.prod.factual.com/airflow-cdh5:1,
image_pull_policy=Always, request_memory=None, request_cpu=None,
limit_memory=None, limit_cpu=None, gcp_service_account_key=None,
node_selectors=None, affinity=None,
annotations={'injector.factual.com/request': 'airflow-kinit'}, volumes=[],
volume_mounts=[], tolerations=None))
/usr/local/lib/python3.6/site-packages/airflow/contrib/kubernetes/kubernetes_request_factory/pod_request_factory.py:45:
YAMLLoadWarning: calling yaml.load() without Loader=... is deprecated, as the
default Loader is unsafe. Please read https://msg.pyyaml.org/load for full
details.
req = yaml.load(self._yaml)
[2019-10-26 13:31:34,408] {kubernetes_executor.py:318} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e had an event of type ADDED
[2019-10-26 13:31:34,409] {kubernetes_executor.py:350} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e Pending
[2019-10-26 13:31:34,422] {kubernetes_executor.py:770} INFO - Shutting down
Kubernetes executor
[2019-10-26 13:31:34,427] {kubernetes_executor.py:318} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e had an event of type MODIFIED
[2019-10-26 13:31:34,427] {kubernetes_executor.py:350} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e Pending
[2019-10-26 13:31:34,438] {kubernetes_executor.py:318} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e had an event of type MODIFIED
[2019-10-26 13:31:34,439] {kubernetes_executor.py:350} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e Pending
[2019-10-26 13:31:37,402] {kubernetes_executor.py:318} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e had an event of type MODIFIED
[2019-10-26 13:31:37,403] {kubernetes_executor.py:358} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e is Running
[2019-10-26 13:31:40,443] {kubernetes_executor.py:318} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e had an event of type MODIFIED
[2019-10-26 13:31:40,444] {kubernetes_executor.py:355} INFO - Event:
k8stestpython-b8743ba7b8904b8482e364dccfdd879e Succeeded
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] BasPH commented on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21]
BasPH commented on issue #6455: [WIP][AIRFLOW-5777] Migrate AWS DynamoDB to /providers/aws [AIP-21] URL: https://github.com/apache/airflow/pull/6455#issuecomment-547640771 AFAIK there is currently no convention for class naming. For consistency we definitely need consistent naming, and we might as well do it right now because users have to migrate anyways. @potiuk thoughts? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Improve tags list (#104)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 4d8b46d Improve tags list (#104)
4d8b46d is described below
commit 4d8b46d7078c8dcad1dc25394994d345f665c69b
Author: Kamil Breguła
AuthorDate: Tue Oct 29 22:33:10 2019 +0100
Improve tags list (#104)
---
landing-pages/site/assets/scss/_blog-page.scss | 10 --
landing-pages/site/layouts/blog/content.html| 10 ++
landing-pages/site/layouts/blog/list.html | 12 +++-
landing-pages/site/layouts/partials/boxes/blogpost.html | 10 ++
landing-pages/site/layouts/taxonomy/tag.html| 14 --
5 files changed, 35 insertions(+), 21 deletions(-)
diff --git a/landing-pages/site/assets/scss/_blog-page.scss
b/landing-pages/site/assets/scss/_blog-page.scss
index 09f92ba..7f3d936 100644
--- a/landing-pages/site/assets/scss/_blog-page.scss
+++ b/landing-pages/site/assets/scss/_blog-page.scss
@@ -33,15 +33,21 @@
}
}
-.all-tags-container {
+.tags-container {
display: flex;
justify-content: center;
flex-wrap: wrap;
- margin: 30px 15px;
+ width: fit-content;
+ margin-top: -14px;
+}
+
+.all-tags-container {
+ margin: 30px 0;
}
.new-entry {
margin: 30px 0 20px;
+
&--link {
@extend .bodytext__medium--cerulean-blue;
font-weight: 500;
diff --git a/landing-pages/site/layouts/blog/content.html
b/landing-pages/site/layouts/blog/content.html
index a43beb9..d587707 100644
--- a/landing-pages/site/layouts/blog/content.html
+++ b/landing-pages/site/layouts/blog/content.html
@@ -19,11 +19,13 @@
-{{ range $tag := .Params.tags }}
-{{ with $.Site.GetPage (printf "/tags/%s" $tag) }}
-{{ humanize $tag }}
+
+{{ range $tag := .Params.tags }}
+{{ with $.Site.GetPage (printf "/tags/%s" $tag) }}
+{{ humanize $tag
}}
+{{ end }}
{{ end }}
-{{ end }}
+
{{ .Date.Format "Mon,
Jan 2, 2006" }}
{{ .Title }}
diff --git a/landing-pages/site/layouts/blog/list.html
b/landing-pages/site/layouts/blog/list.html
index c0bba46..4891c43 100644
--- a/landing-pages/site/layouts/blog/list.html
+++ b/landing-pages/site/layouts/blog/list.html
@@ -21,12 +21,14 @@
Blog
-All
-{{ range $name, $taxonomy := .Site.Taxonomies.tags }}
-{{ with $.Site.GetPage (printf "/tags/%s" $name) }}
-{{ humanize $name
}}
+
+All
+{{ range $name, $taxonomy := .Site.Taxonomies.tags }}
+{{ with $.Site.GetPage (printf "/tags/%s" $name) }}
+{{ humanize
$name }}
+{{ end }}
{{ end }}
-{{ end }}
+
https://github.com;>ADD YOUR
ENTRY
diff --git a/landing-pages/site/layouts/partials/boxes/blogpost.html
b/landing-pages/site/layouts/partials/boxes/blogpost.html
index 91b1497..f59859c 100644
--- a/landing-pages/site/layouts/partials/boxes/blogpost.html
+++ b/landing-pages/site/layouts/partials/boxes/blogpost.html
@@ -19,11 +19,13 @@
-{{ range $tag := .Params.tags }}
-{{ with $.Site.GetPage (printf "/tags/%s" $tag) }}
-{{ humanize $tag }}
+
+{{ range $tag := .Params.tags }}
+{{ with $.Site.GetPage (printf "/tags/%s" $tag) }}
+{{ humanize $tag
}}
+{{ end }}
{{ end }}
-{{ end }}
+
{{ .Date.Format "Mon,
Jan 2, 2006" }}
{{ .Params.title }}
diff --git a/landing-pages/site/layouts/taxonomy/tag.html
b/landing-pages/site/layouts/taxonomy/tag.html
index 81e55e8..03d12ab 100644
--- a/landing-pages/site/layouts/taxonomy/tag.html
+++ b/landing-pages/site/layouts/taxonomy/tag.html
@@ -22,13 +22,15 @@
Blog
-All
-{{ range $name, $taxonomy := .Site.Taxonomies.tags }}
-{{ with $.Site.GetPage (printf "/tags/%s" $name) }}
-{{ humanize $name }}
+
+All
+{{ range $name, $taxonomy := .Site.Taxonomies.tags }}
+{{ with $.Site.GetPage (printf "/tags/%s" $name) }}
+{{ humanize $name }}
+{{ end }}
{{ end }}
-{{ end }}
+
https://github.com;>ADD YOUR
ENTRY
[GitHub] [airflow-site] mik-laj merged pull request #104: Improve tags list
mik-laj merged pull request #104: Improve tags list URL: https://github.com/apache/airflow-site/pull/104 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj opened a new pull request #104: Improve tags list
mik-laj opened a new pull request #104: Improve tags list URL: https://github.com/apache/airflow-site/pull/104 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] 03/05: Create community page
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit a73b44ad3560004c2ff13d1a46c77dccc33e3816
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 23 14:14:28 2019 +0200
Create community page
---
.../site/assets/icons/contribute-icon.svg | 24 ++
.../site/assets/icons/documentation-icon.svg | 10 +++
landing-pages/site/assets/icons/fix-bug-icon.svg | 18 +
.../site/assets/icons/new-feature-icon.svg | 18 +
landing-pages/site/assets/icons/project-icon.svg | 22 ++
.../assets/scss/_community-page.scss} | 33 +++-
landing-pages/site/assets/scss/_ol-ul.scss | 21 ++---
landing-pages/site/assets/scss/main-custom.scss| 1 +
.../site/content/en/community/_index.html | 89 ++
landing-pages/site/data/commiters.json | 5 +-
landing-pages/site/layouts/community/baseof.html | 46 +++
landing-pages/site/layouts/community/list.html | 58 ++
landing-pages/src/index.js | 3 +-
landing-pages/src/js/showAllCommiters.js | 6 +-
14 files changed, 336 insertions(+), 18 deletions(-)
diff --git a/landing-pages/site/assets/icons/contribute-icon.svg
b/landing-pages/site/assets/icons/contribute-icon.svg
new file mode 100644
index 000..ee3ef37
--- /dev/null
+++ b/landing-pages/site/assets/icons/contribute-icon.svg
@@ -0,0 +1,24 @@
+http://www.w3.org/2000/svg; width="60" height="43.846" viewBox="0
0 60 43.846">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/documentation-icon.svg
b/landing-pages/site/assets/icons/documentation-icon.svg
new file mode 100644
index 000..b4f909e
--- /dev/null
+++ b/landing-pages/site/assets/icons/documentation-icon.svg
@@ -0,0 +1,10 @@
+http://www.w3.org/2000/svg; width="60" height="42.748" viewBox="0
0 60 42.748">
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/fix-bug-icon.svg
b/landing-pages/site/assets/icons/fix-bug-icon.svg
new file mode 100644
index 000..442ff23
--- /dev/null
+++ b/landing-pages/site/assets/icons/fix-bug-icon.svg
@@ -0,0 +1,18 @@
+http://www.w3.org/2000/svg; width="60" height="56.368" viewBox="0
0 60 56.368">
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/new-feature-icon.svg
b/landing-pages/site/assets/icons/new-feature-icon.svg
new file mode 100644
index 000..1d5e795
--- /dev/null
+++ b/landing-pages/site/assets/icons/new-feature-icon.svg
@@ -0,0 +1,18 @@
+http://www.w3.org/2000/svg; width="59.393" height="43.404"
viewBox="0 0 59.393 43.404">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/site/assets/icons/project-icon.svg
b/landing-pages/site/assets/icons/project-icon.svg
new file mode 100644
index 000..a8ab6da
--- /dev/null
+++ b/landing-pages/site/assets/icons/project-icon.svg
@@ -0,0 +1,22 @@
+http://www.w3.org/2000/svg; width="43.847" height="60" viewBox="0
0 43.847 60">
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/landing-pages/src/index.js
b/landing-pages/site/assets/scss/_community-page.scss
similarity index 62%
copy from landing-pages/src/index.js
copy to landing-pages/site/assets/scss/_community-page.scss
index 375f57c..12ac871 100644
--- a/landing-pages/src/index.js
+++ b/landing-pages/site/assets/scss/_community-page.scss
@@ -17,8 +17,33 @@
* under the License.
*/
-import {showAllCommiters} from "./js/showAllCommiters";
-import {handleActiveVideo} from "./js/handleActiveVideo";
+.list-link {
+ @extend .bodytext__medium--greyish-brown;
+ text-decoration: underline;
+}
-showAllCommiters(8);
-handleActiveVideo();
+.community {
+ &--header-container {
+max-width: 936px;
+margin: 0 auto;
+ }
+
+ &--accordion-container {
+margin: 60px 18px;
+ }
+
+ &--commiters-header {
+@extend .header__small--greyish-brown;
+text-align: center;
+margin: 70px 0 40px;
+
+&.large-margin {
+ margin: 100px 0 40px;
+}
+ }
+
+ &--button-container {
+width: fit-content;
+margin: 60px auto 0;
+ }
+}
diff --git
[airflow-site] branch aip-11 updated (8181f19 -> b73cdbf)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a change to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git.
from 8181f19 Improve accordion spacings and arrow rotation (#90)
new 6a06c9e Put components in home page
new 6ff928e Put components in meetups page
new a73b44a Create community page
new 6ef1122 Create install page
new b73cdbf Remove About and Examples pages
The 5 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails. The revisions
listed as "add" were already present in the repository and have only
been added to this reference.
Summary of changes:
.../site/assets/icons/contribute-icon.svg | 24
.../site/assets/icons/documentation-icon.svg | 10 ++
landing-pages/site/assets/icons/fix-bug-icon.svg | 18 +++
.../site/assets/icons/new-feature-icon.svg | 18 +++
landing-pages/site/assets/icons/project-icon.svg | 22
landing-pages/site/assets/scss/_base-layout.scss | 2 +-
.../scss/{_quote.scss => _community-page.scss} | 45
.../assets/scss/{_pager.scss => _home-page.scss} | 11 +-
.../scss/{_quote.scss => _install-page.scss} | 50 -
.../scss/{_case-study.scss => _meetups.scss} | 9 +-
landing-pages/site/assets/scss/_ol-ul.scss | 21 ++--
.../site/assets/scss/_text-with-icon.scss | 2 +-
landing-pages/site/assets/scss/main-custom.scss| 4 +
landing-pages/site/content/en/_index.html | 67 ---
landing-pages/site/content/en/about/_index.html| 38 ---
.../site/content/en/about/featured-background.jpg | Bin 496231 -> 0 bytes
.../site/content/en/community/_index.html | 89 +++
landing-pages/site/content/en/community/_index.md | 8 --
landing-pages/site/content/en/examples/_index.html | 83 --
landing-pages/site/content/en/install/_index.html | 124 +
landing-pages/site/content/en/meetups/_index.html | 7 ++
landing-pages/site/data/commiters.json | 5 +-
landing-pages/site/data/videos.json| 30 ++---
landing-pages/site/layouts/_default/baseof.html| 2 +-
.../layouts/{taxonomy => community}/baseof.html| 0
landing-pages/site/layouts/community/list.html | 58 ++
.../layouts/{examples/list.html => index.html} | 83 +++---
.../site/layouts/{taxonomy => install}/baseof.html | 0
.../layouts/{case-studies => install}/list.html| 19 ++--
.../site/layouts/{taxonomy => meetups}/baseof.html | 0
landing-pages/site/layouts/meetups/list.html | 46
landing-pages/site/static/icons/docker-logo.png| Bin 0 -> 7297 bytes
landing-pages/src/index.js | 3 +-
landing-pages/src/js/showAllCommiters.js | 6 +-
34 files changed, 538 insertions(+), 366 deletions(-)
create mode 100644 landing-pages/site/assets/icons/contribute-icon.svg
create mode 100644 landing-pages/site/assets/icons/documentation-icon.svg
create mode 100644 landing-pages/site/assets/icons/fix-bug-icon.svg
create mode 100644 landing-pages/site/assets/icons/new-feature-icon.svg
create mode 100644 landing-pages/site/assets/icons/project-icon.svg
copy landing-pages/site/assets/scss/{_quote.scss => _community-page.scss} (65%)
copy landing-pages/site/assets/scss/{_pager.scss => _home-page.scss} (85%)
copy landing-pages/site/assets/scss/{_quote.scss => _install-page.scss} (60%)
copy landing-pages/site/assets/scss/{_case-study.scss => _meetups.scss} (90%)
delete mode 100644 landing-pages/site/content/en/about/_index.html
delete mode 100644 landing-pages/site/content/en/about/featured-background.jpg
create mode 100644 landing-pages/site/content/en/community/_index.html
delete mode 100644 landing-pages/site/content/en/community/_index.md
delete mode 100644 landing-pages/site/content/en/examples/_index.html
create mode 100644 landing-pages/site/content/en/install/_index.html
create mode 100644 landing-pages/site/content/en/meetups/_index.html
copy landing-pages/site/layouts/{taxonomy => community}/baseof.html (100%)
create mode 100644 landing-pages/site/layouts/community/list.html
rename landing-pages/site/layouts/{examples/list.html => index.html} (52%)
copy landing-pages/site/layouts/{taxonomy => install}/baseof.html (100%)
copy landing-pages/site/layouts/{case-studies => install}/list.html (54%)
copy landing-pages/site/layouts/{taxonomy => meetups}/baseof.html (100%)
create mode 100644 landing-pages/site/layouts/meetups/list.html
create mode 100644 landing-pages/site/static/icons/docker-logo.png
[airflow-site] 02/05: Put components in meetups page
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit 6ff928eae02405e4be774242b59ef8466eb366d2
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 23 14:07:00 2019 +0200
Put components in meetups page
---
.../scss/{main-custom.scss => _meetups.scss} | 25 +++-
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/content/en/community/_index.md | 8
landing-pages/site/content/en/meetups/_index.html | 7
landing-pages/site/data/videos.json| 30 ++
landing-pages/site/layouts/meetups/baseof.html | 46 ++
landing-pages/site/layouts/meetups/list.html | 46 ++
7 files changed, 118 insertions(+), 45 deletions(-)
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/_meetups.scss
similarity index 58%
copy from landing-pages/site/assets/scss/main-custom.scss
copy to landing-pages/site/assets/scss/_meetups.scss
index f12c1e7..fa63a23 100644
--- a/landing-pages/site/assets/scss/main-custom.scss
+++ b/landing-pages/site/assets/scss/_meetups.scss
@@ -17,23 +17,10 @@
* under the License.
*/
-@import url('https://fonts.googleapis.com/css?family=Rubik:500=swap');
-@import
url('https://fonts.googleapis.com/css?family=Roboto:400,400i,500,700=swap');
-@import
url('https://fonts.googleapis.com/css?family=Roboto+Mono=swap');
+.video-section-container {
+ margin: 80px 0;
+}
-@import "typography";
-@import "accordion";
-@import "buttons";
-@import "ol-ul";
-@import "list-boxes";
-@import "avatar";
-@import "quote";
-@import "pager";
-@import "case-study";
-@import "markdown-content";
-@import "base-layout";
-@import "feature";
-@import "text-with-icon";
-@import "video";
-@import "blog-page";
-@import "home-page";
+.host-header {
+ margin-bottom: 6px;
+}
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/main-custom.scss
index f12c1e7..a15338f 100644
--- a/landing-pages/site/assets/scss/main-custom.scss
+++ b/landing-pages/site/assets/scss/main-custom.scss
@@ -37,3 +37,4 @@
@import "video";
@import "blog-page";
@import "home-page";
+@import "meetups";
diff --git a/landing-pages/site/content/en/community/_index.md
b/landing-pages/site/content/en/community/_index.md
deleted file mode 100644
index cdade16..000
--- a/landing-pages/site/content/en/community/_index.md
+++ /dev/null
@@ -1,8 +0,0 @@
-title: Community
-menu:
- main:
-weight: 40
-
-
diff --git a/landing-pages/site/content/en/meetups/_index.html
b/landing-pages/site/content/en/meetups/_index.html
new file mode 100644
index 000..391f69c
--- /dev/null
+++ b/landing-pages/site/content/en/meetups/_index.html
@@ -0,0 +1,7 @@
+---
+title: "Meetups"
+linkTitle: "Meetups"
+menu:
+main:
+weight: 3
+---
diff --git a/landing-pages/site/data/videos.json
b/landing-pages/site/data/videos.json
index 64dd841..2e93cf4 100644
--- a/landing-pages/site/data/videos.json
+++ b/landing-pages/site/data/videos.json
@@ -1,56 +1,50 @@
[
{
"name": "video",
-"date": "2019-01-23",
-"title": "Airflow Meetup, London 23 Jan 2019",
-"videoID": "E0asAgpHvaI"
+"date": "2019-10-17",
+"title": "Airflow Meetup, Warsaw 17 Oct 2019",
+"videoID": "Nr4Pp1SNXeU"
},
{
"name": "video",
-"date": "2019-04-05",
-"title": "Airflow Meetup, London 05 Apr 2019",
-"videoID": "uN-TvWzeEvA"
+"date": "2019-09-24",
+"title": "Airflow Meetup, London 24 Sep 2019",
+"videoID": "kmD_u8VfuT0"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "nUfb4UxnvJk"
+"videoID": "Vkprq5F6jkM"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "OhWZavn2OvM"
+"videoID": "jX3iLfcMDCw"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "SRhueFPsCeY"
+"videoID": "K7UkKBLq7pk"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "wH533kbXm2c"
+"videoID": "JNsKvZo6MDs"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "cY9kbO4GY_s"
+"videoID": "w0AOGeqOnFY"
},
{
"name": "video",
"date": "2019-09-30",
"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "d4DEHs_KAzs"
- },
- {
-"name": "video",
-"date": "2019-09-30",
-"title": "Airflow Meetup, London 30 Sep 2019",
-"videoID": "mznO_E3_BQo"
+"videoID": "VK8LH_KeG7A"
}
]
diff --git a/landing-pages/site/layouts/meetups/baseof.html
b/landing-pages/site/layouts/meetups/baseof.html
new file
[GitHub] [airflow-site] mik-laj merged pull request #91: [depends on #90] Add static pages
mik-laj merged pull request #91: [depends on #90] Add static pages URL: https://github.com/apache/airflow-site/pull/91 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] 04/05: Create install page
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit 6ef112239416619573670c7bfcfcc320031fd4d8
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 23 14:39:35 2019 +0200
Create install page
---
.../scss/{main-custom.scss => _install-page.scss} | 53 +
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/content/en/install/_index.html | 124 +
landing-pages/site/layouts/install/baseof.html | 46
landing-pages/site/layouts/install/list.html | 35 ++
landing-pages/site/static/icons/docker-logo.png| Bin 0 -> 7297 bytes
6 files changed, 238 insertions(+), 21 deletions(-)
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/_install-page.scss
similarity index 56%
copy from landing-pages/site/assets/scss/main-custom.scss
copy to landing-pages/site/assets/scss/_install-page.scss
index fc9eff9..0c6b5e9 100644
--- a/landing-pages/site/assets/scss/main-custom.scss
+++ b/landing-pages/site/assets/scss/_install-page.scss
@@ -17,25 +17,36 @@
* under the License.
*/
-@import url('https://fonts.googleapis.com/css?family=Rubik:500=swap');
-@import
url('https://fonts.googleapis.com/css?family=Roboto:400,400i,500,700=swap');
-@import
url('https://fonts.googleapis.com/css?family=Roboto+Mono=swap');
+.install__accordions {
+ &--wrapper {
+margin: 60px 18px;
+ }
-@import "typography";
-@import "accordion";
-@import "buttons";
-@import "ol-ul";
-@import "list-boxes";
-@import "avatar";
-@import "quote";
-@import "pager";
-@import "case-study";
-@import "markdown-content";
-@import "base-layout";
-@import "feature";
-@import "text-with-icon";
-@import "video";
-@import "blog-page";
-@import "home-page";
-@import "meetups";
-@import "community-page";
+ &-content {
+&--header {
+ @extend .bodytext__medium--greyish-brown;
+ font-weight: 500;
+}
+
+&--list-wrapper {
+ margin-bottom: 40px;
+}
+
+&--methods-wrapper {
+ display: flex;
+ margin-bottom: 20px;
+}
+
+&--method-box {
+ display: flex;
+ width: 270px;
+ height: 160px;
+ flex-direction: column;
+ align-items: center;
+ justify-content: center;
+ border: solid 1px #cbcbcb;
+ border-radius: 5px;
+ margin-right: 40px;
+}
+ }
+}
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/main-custom.scss
index fc9eff9..f69bb75 100644
--- a/landing-pages/site/assets/scss/main-custom.scss
+++ b/landing-pages/site/assets/scss/main-custom.scss
@@ -39,3 +39,4 @@
@import "home-page";
@import "meetups";
@import "community-page";
+@import "install-page";
diff --git a/landing-pages/site/content/en/install/_index.html
b/landing-pages/site/content/en/install/_index.html
new file mode 100644
index 000..ed70fee
--- /dev/null
+++ b/landing-pages/site/content/en/install/_index.html
@@ -0,0 +1,124 @@
+---
+title: Install
+menu:
+main:
+weight: 20
+---
+
+
+{{< accordion title="Install Apache Airflow locally" description="Working
on an Open Source project such as Apache Airflow is very demanding but also
equally rewarding when you realize how many businesses use it every day." >}}
+When installing locally is
a good option for you?
+
+
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+
+
+Available methods
+
+
+
+
+
+PIP
installation
+https://github.com;>
+Quick
start
+
+
+
+{{< /accordion >}}
+{{< accordion title="Install Apache Airflow on server"
description="Working on an Open Source project such as Apache Airflow is very
demanding but also equally rewarding when you realize how many businesses use
it every day." >}}
+When installing locally is
a good option for you?
+
+
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+If you don’t want to do it by yourself
+
+
+Available methods
+
+
+
+
+
+PIP
installation
+https://github.com;>
+Quick
start
+
+
+
+{{< /accordion >}}
+{{< accordion title="Install Apache Airflow on cluster"
description="Working on an Open Source project such as Apache Airflow is very
demanding but also equally rewarding when you realize how many businesses use
it every day." >}}
+When installing locally is
a good option for you?
+
+
+If you
[airflow-site] 05/05: Remove About and Examples pages
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit b73cdbf254f77aa5e2f9550b02d38cb411456ba9
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 23 14:50:02 2019 +0200
Remove About and Examples pages
---
landing-pages/site/content/en/about/_index.html| 38
.../site/content/en/about/featured-background.jpg | Bin 496231 -> 0 bytes
landing-pages/site/content/en/examples/_index.html | 83
landing-pages/site/layouts/examples/list.html | 105 -
4 files changed, 226 deletions(-)
diff --git a/landing-pages/site/content/en/about/_index.html
b/landing-pages/site/content/en/about/_index.html
deleted file mode 100644
index c264436..000
--- a/landing-pages/site/content/en/about/_index.html
+++ /dev/null
@@ -1,38 +0,0 @@
-title: About Goldydocs
-linkTitle: About
-menu:
- main:
-weight: 10
-
-
-
-{{< blocks/cover title="About Goldydocs" image_anchor="bottom" height="min" >}}
-
-A sample site using the Docsy Hugo theme.
-
-
-{{< /blocks/cover >}}
-
-{{% blocks/lead %}}
-Goldydocs is a sample site using the https://github.com/google/docsy;>Docsy Hugo theme that shows what it
can do and provides you with a template site structure. It’s designed for you
to clone and edit as much as you like. See the different sections of the
documentation and site for more ideas.
-{{% /blocks/lead %}}
-
-
-{{< blocks/section >}}
-
-This is another section
-
-
-{{< /blocks/section >}}
-
-
-
-{{< blocks/section >}}
-
-
-This is another section
-
-
-{{< /blocks/section >}}
diff --git a/landing-pages/site/content/en/about/featured-background.jpg
b/landing-pages/site/content/en/about/featured-background.jpg
deleted file mode 100644
index 7476923..000
Binary files a/landing-pages/site/content/en/about/featured-background.jpg and
/dev/null differ
diff --git a/landing-pages/site/content/en/examples/_index.html
b/landing-pages/site/content/en/examples/_index.html
deleted file mode 100644
index 4d6085b..000
--- a/landing-pages/site/content/en/examples/_index.html
+++ /dev/null
@@ -1,83 +0,0 @@
-title: Components
-linkTitle: Components
-menu:
-main:
-weight: 5
-
-
-{{< blocks/section >}}
-
-This is a large
greyish-brown header
-This is a medium
greyish-brown header
-This is a small
greyish-brown header
-This is a xsmall
greyish-brown header
-
-This is a large
brownish-grey subtitle
-This is a large normal-weight brownish-grey subtitle
-This is a large
greyish-brown subtitle
-This is a large
cerulean-blue subtitle
-This is a medium bold brownish-grey subtitle
-This is a medium bold greyish-brown subtitle
-
-This is a medium
greyish-brown bodytext
-This is a medium medium-weight greyish-brown bodytext
-This is a medium
brownish-grey bodytext
-This is a medium medium-weight brownish-grey bodytext
-This is
a medium italic brownish-grey bodytext
-This is a medium
normal-weight cerulean-blue bodytext
-This is a medium medium-weight cerulean-blue bodytext
-This is a medium white
bodytext
-This is a small
brownish-grey bodytext
-This is a xsmall white
bodytext
-
-This is a cerulean-blue
monotext
-This is a brownish-grey
monotext
-This is a
bold brownish-grey monotext
-This is a shamrock monotext
-This is a vermillion
monotext
-
-{{< /blocks/section >}}
-
-
-{{< blocks/section color="white" >}}
-
-{{< accordion title="Install Apache Airflow locally" description="Working
on an Open Source project such as Apache Airflow is very demanding but also
equally rewarding when you realize how many businesses use it every day." >}}
-Magna ac placerat vestibulum lectus mauris ultrices. Nullam non nisi est
sit amet facilisis magna etiam tempor. Aliquet nec ullamcorper sit amet risus
nullam eget felis. Rhoncus aenean vel elit scelerisque mauris pellentesque.
-{{< /accordion >}}
-{{< accordion title="Install Apache Airflow on server"
description="Working on an Open Source project such as Apache Airflow is very
demanding but also equally rewarding when you realize how many businesses use
it every day." >}}
-Amet mauris commodo quis imperdiet massa tincidunt nunc. Leo duis ut diam
quam nulla porttitor massa id neque.
-{{< /accordion >}}
-{{< accordion title="Install Apache Airflow on cluster"
description="Working on an Open Source project such as Apache Airflow is very
demanding but also equally rewarding when you realize how many businesses use
it every day." >}}
-Amet mauris commodo quis imperdiet massa tincidunt nunc. Leo duis ut diam
quam nulla porttitor massa id neque.
-{{< /accordion >}}
-
-
-{{< accordion title="Join the devlist" description="If you want to stay up
to date with what is going on in the project, want to discuss the features,
talk about
[airflow-site] 01/05: Put components in home page
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
commit 6a06c9ee59d2817664a2b03aa1bf7882cd88e677
Author: Kamil Gabryjelski
AuthorDate: Wed Oct 23 13:56:55 2019 +0200
Put components in home page
---
landing-pages/site/assets/scss/_base-layout.scss | 2 +-
.../scss/{_text-with-icon.scss => _home-page.scss} | 27 ++---
.../site/assets/scss/_text-with-icon.scss | 2 +-
landing-pages/site/assets/scss/main-custom.scss| 1 +
landing-pages/site/content/en/_index.html | 67 --
landing-pages/site/layouts/_default/baseof.html| 2 +-
landing-pages/site/layouts/index.html | 50
7 files changed, 58 insertions(+), 93 deletions(-)
diff --git a/landing-pages/site/assets/scss/_base-layout.scss
b/landing-pages/site/assets/scss/_base-layout.scss
index 2c560ec..c94204e 100644
--- a/landing-pages/site/assets/scss/_base-layout.scss
+++ b/landing-pages/site/assets/scss/_base-layout.scss
@@ -44,6 +44,6 @@
.container {
margin-top: 44px;
@media(min-width: 1200px) {
-max-width: 1230px;
+max-width: 1200px;
}
}
diff --git a/landing-pages/site/assets/scss/_text-with-icon.scss
b/landing-pages/site/assets/scss/_home-page.scss
similarity index 63%
copy from landing-pages/site/assets/scss/_text-with-icon.scss
copy to landing-pages/site/assets/scss/_home-page.scss
index e27b137..beb35e3 100644
--- a/landing-pages/site/assets/scss/_text-with-icon.scss
+++ b/landing-pages/site/assets/scss/_home-page.scss
@@ -17,29 +17,10 @@
* under the License.
*/
-.text-with-icon-list {
- display: flex;
- flex-wrap: wrap;
- max-width: 940px;
- margin: 0 auto 100px;
- justify-content: space-between;
+.principles-header {
+ margin-bottom: 4px;
}
-.text-with-icon-item {
- display: flex;
- flex-direction: column;
- max-width: 410px;
- align-items: center;
- margin-top: 65px;
-
- &--header {
-@extend .header__xsmall--greyish-brown;
-text-align: center;
-margin-top: 23px;
- }
-
- &--text {
-@extend .bodytext__medium--brownish-grey;
-text-align: center;
- }
+.integrations-header {
+ margin-bottom: 60px; // to be changed when searchbox is added
}
diff --git a/landing-pages/site/assets/scss/_text-with-icon.scss
b/landing-pages/site/assets/scss/_text-with-icon.scss
index e27b137..f508d91 100644
--- a/landing-pages/site/assets/scss/_text-with-icon.scss
+++ b/landing-pages/site/assets/scss/_text-with-icon.scss
@@ -30,7 +30,7 @@
flex-direction: column;
max-width: 410px;
align-items: center;
- margin-top: 65px;
+ margin-top: 54px;
&--header {
@extend .header__xsmall--greyish-brown;
diff --git a/landing-pages/site/assets/scss/main-custom.scss
b/landing-pages/site/assets/scss/main-custom.scss
index bcf14f7..f12c1e7 100644
--- a/landing-pages/site/assets/scss/main-custom.scss
+++ b/landing-pages/site/assets/scss/main-custom.scss
@@ -36,3 +36,4 @@
@import "text-with-icon";
@import "video";
@import "blog-page";
+@import "home-page";
diff --git a/landing-pages/site/content/en/_index.html
b/landing-pages/site/content/en/_index.html
index dbdcffd..d9fe535 100644
--- a/landing-pages/site/content/en/_index.html
+++ b/landing-pages/site/content/en/_index.html
@@ -3,70 +3,3 @@ title = "Goldydocs"
linkTitle = "Goldydocs"
+++
-
-{{< blocks/cover title="Welcome to Goldydocs: A Docsy Example Project!"
image_anchor="top" height="full" color="orange" >}}
-
-}}">
-Learn More
-
-https://github.com/google/docsy-example;>
-Download
-
-Porridge temperature assessment - in the cloud!
-
-{{< blocks/link-down color="info" >}}
-
-
-{{< /blocks/cover >}}
-
-
-{{% blocks/lead color="primary" %}}
-Goldydocs provides a single web UI providing visibility into porridge
temperature, chair size, and bed softness metrics! You can even find out who's
been eating **your** porridge.
-
-(Sadly, Goldydocs isn't a real project, but you can use this site as an
example to create your own real websites with [Docsy](http://docsy.dev))
-{{% /blocks/lead %}}
-
-{{< blocks/section color="dark" >}}
-{{% blocks/feature icon="fa-lightbulb" title="New chair metrics!" %}}
-The Goldydocs UI now shows chair size metrics by default.
-
-Please follow this space for updates!
-{{% /blocks/feature %}}
-
-
-{{% blocks/feature icon="fab fa-github" title="Contributions welcome!"
url="https://github.com/google/docsy-example; %}}
-We do a [Pull Request](https://github.com/gohugoio/hugo/pulls) contributions
workflow on **GitHub**. New users are always welcome!
-{{% /blocks/feature %}}
-
-
-{{% blocks/feature icon="fab fa-twitter" title="Follow us on Twitter!"
url="https://twitter.com/docsydocs; %}}
-For announcement of latest features etc.
-{{% /blocks/feature %}}
-
-
-{{< /blocks/section >}}
-
-
-{{< blocks/section >}}
-
-This is the second Section
-
-
-{{<
[airflow-site] branch aip-11 updated: Improve accordion spacings and arrow rotation (#90)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 8181f19 Improve accordion spacings and arrow rotation (#90)
8181f19 is described below
commit 8181f199aa6fbb4ab4389a8258ab79b951207574
Author: Kamil Gabryjelski
AuthorDate: Tue Oct 29 21:40:11 2019 +0100
Improve accordion spacings and arrow rotation (#90)
---
landing-pages/site/assets/scss/_accordion.scss | 9 +++--
.../site/layouts/shortcodes/accordion.html | 21 +
2 files changed, 16 insertions(+), 14 deletions(-)
diff --git a/landing-pages/site/assets/scss/_accordion.scss
b/landing-pages/site/assets/scss/_accordion.scss
index 392e3e5..c5410e7 100644
--- a/landing-pages/site/assets/scss/_accordion.scss
+++ b/landing-pages/site/assets/scss/_accordion.scss
@@ -37,8 +37,9 @@ details.accordion {
.accordion__summary-content {
display: flex;
+max-width: 750px;
-svg {
+&--icon {
width: 60px;
margin-top: 28px;
margin-right: 42px;
@@ -67,7 +68,7 @@ details.accordion {
}
&[open] {
-.accordion__icon {
+.accordion__arrow {
svg {
transform: rotate(180deg);
}
@@ -76,5 +77,9 @@ details.accordion {
.accordion__content {
margin-top: 30px;
+
+&.indented {
+ margin-left: 102px;
+}
}
}
diff --git a/landing-pages/site/layouts/shortcodes/accordion.html
b/landing-pages/site/layouts/shortcodes/accordion.html
index 2484392..78846ac 100644
--- a/landing-pages/site/layouts/shortcodes/accordion.html
+++ b/landing-pages/site/layouts/shortcodes/accordion.html
@@ -25,18 +25,15 @@
{{ if $icon }}
-{{ with resources.Get $icon }}{{ ( . | minify).Content |
safeHTML }}{{ end }}
+
+{{ with resources.Get $icon }}{{ ( . | minify).Content |
safeHTML }}{{ end }}
+
{{ end }}
-
-{{ .Get "title" }}
-{{ .Get
"description" }}
-
-
+
+{{ .Get "title" }}
+{{ .Get "description"
}}
+
+
-{{ .Inner }}
+{{ .Inner
}}
[GitHub] [airflow-site] mik-laj merged pull request #90: Improve accordion spacings and arrow rotation
mik-laj merged pull request #90: Improve accordion spacings and arrow rotation URL: https://github.com/apache/airflow-site/pull/90 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] mik-laj commented on a change in pull request #90: Improve accordion spacings and arrow rotation
mik-laj commented on a change in pull request #90: Improve accordion spacings
and arrow rotation
URL: https://github.com/apache/airflow-site/pull/90#discussion_r340312950
##
File path: landing-pages/site/layouts/shortcodes/accordion.html
##
@@ -25,7 +25,9 @@
{{ if $icon }}
-{{ with resources.Get $icon }}{{ ( . | minify).Content |
safeHTML }}{{ end }}
+
+{{ with resources.Get $icon }}{{ ( . | minify).Content |
safeHTML }}{{ end }}
Review comment:
I fixed it.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Resolved] (AIRFLOW-5082) add subject in aws sns hook
[ https://issues.apache.org/jira/browse/AIRFLOW-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kamil Bregula resolved AIRFLOW-5082. Resolution: Fixed > add subject in aws sns hook > --- > > Key: AIRFLOW-5082 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5082 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Affects Versions: 1.10.4 >Reporter: MOHAMMD SHAKEEL SHAIK >Assignee: MOHAMMD SHAKEEL SHAIK >Priority: Major > Fix For: 2.0.0 > > > While sending SNS notification to AWS. The subject is an optional field. If > we don't send Subject AWS will add default SNS Subject to email "*AWS > Notification Message*". If anyone wants to add a different Subject. They can > send Subject parameter in AWS SNS hook. > > It is also optional only -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5808) DAG timezones given as datetime.timezone cause odd errors.
[
https://issues.apache.org/jira/browse/AIRFLOW-5808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ash Berlin-Taylor updated AIRFLOW-5808:
---
Summary: DAG timezones given as datetime.timezone cause odd errors. (was:
DAG timezones from datetime.timezone not converted to Pendulm)
> DAG timezones given as datetime.timezone cause odd errors.
> --
>
> Key: AIRFLOW-5808
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5808
> Project: Apache Airflow
> Issue Type: Bug
> Components: DAG
>Affects Versions: 1.10.5
>Reporter: Ash Berlin-Taylor
>Priority: Minor
> Fix For: 1.10.7
>
>
> The airflow docs mention that you should use Pendulum timezones, but we don't
> enforce this, and the stdlib timezone is mostly, but not entirely API
> compatible with this.
> For instance if you create your DAG like this:
> {code:python}
> from airflow import DAG
> from datetime import datetime, timedelta, timezone
> dag = DAG('my_dag', start_date=datetime(2019, 2, 6, hour=0,
> tzinfo=timezone(timedelta(0), name='UTC'))
> {code}
> It will _mostly_ work, except this will break the Tooltip generation in the
> graph page as it expects a pendulm.timezone.
> To make everything predictable in Airflow the DAG constructor should convert
> non-pendulum timezones when it encounters them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (AIRFLOW-5808) DAG timezones from datetime.timezone not converted to Pendulm
Ash Berlin-Taylor created AIRFLOW-5808:
--
Summary: DAG timezones from datetime.timezone not converted to
Pendulm
Key: AIRFLOW-5808
URL: https://issues.apache.org/jira/browse/AIRFLOW-5808
Project: Apache Airflow
Issue Type: Bug
Components: DAG
Affects Versions: 1.10.5
Reporter: Ash Berlin-Taylor
Fix For: 1.10.7
The airflow docs mention that you should use Pendulum timezones, but we don't
enforce this, and the stdlib timezone is mostly, but not entirely API
compatible with this.
For instance if you create your DAG like this:
{code:python}
from airflow import DAG
from datetime import datetime, timedelta, timezone
dag = DAG('my_dag', start_date=datetime(2019, 2, 6, hour=0,
tzinfo=timezone(timedelta(0), name='UTC'))
{code}
It will _mostly_ work, except this will break the Tooltip generation in the
graph page as it expects a pendulm.timezone.
To make everything predictable in Airflow the DAG constructor should convert
non-pendulum timezones when it encounters them.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5082) add subject in aws sns hook
[ https://issues.apache.org/jira/browse/AIRFLOW-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962394#comment-16962394 ] ASF subversion and git services commented on AIRFLOW-5082: -- Commit 919bb8c1cbb36679c02ca3f8890c300e1527c08b in airflow's branch refs/heads/master from SHAKEEL [ https://gitbox.apache.org/repos/asf?p=airflow.git;h=919bb8c ] [AIRFLOW-5082] Add subject in AwsSnsHook (#5694) > add subject in aws sns hook > --- > > Key: AIRFLOW-5082 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5082 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Affects Versions: 1.10.4 >Reporter: MOHAMMD SHAKEEL SHAIK >Assignee: MOHAMMD SHAKEEL SHAIK >Priority: Major > Fix For: 2.0.0 > > > While sending SNS notification to AWS. The subject is an optional field. If > we don't send Subject AWS will add default SNS Subject to email "*AWS > Notification Message*". If anyone wants to add a different Subject. They can > send Subject parameter in AWS SNS hook. > > It is also optional only -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5082) add subject in aws sns hook
[ https://issues.apache.org/jira/browse/AIRFLOW-5082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962382#comment-16962382 ] ASF GitHub Bot commented on AIRFLOW-5082: - mik-laj commented on pull request #5694: [AIRFLOW-5082] Add subject in AwsSnsHook URL: https://github.com/apache/airflow/pull/5694 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > add subject in aws sns hook > --- > > Key: AIRFLOW-5082 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5082 > Project: Apache Airflow > Issue Type: Improvement > Components: aws >Affects Versions: 1.10.4 >Reporter: MOHAMMD SHAKEEL SHAIK >Assignee: MOHAMMD SHAKEEL SHAIK >Priority: Major > Fix For: 2.0.0 > > > While sending SNS notification to AWS. The subject is an optional field. If > we don't send Subject AWS will add default SNS Subject to email "*AWS > Notification Message*". If anyone wants to add a different Subject. They can > send Subject parameter in AWS SNS hook. > > It is also optional only -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [airflow] mik-laj merged pull request #5694: [AIRFLOW-5082] Add subject in AwsSnsHook
mik-laj merged pull request #5694: [AIRFLOW-5082] Add subject in AwsSnsHook URL: https://github.com/apache/airflow/pull/5694 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340297464 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: Well, then this change looks good. I accept it This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340278526 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: @mik-laj we need if condition because. boto3 is throwing an error when you call publish_to_target method with subject as None. Error message from boto3: ``` botocore.exceptions.ParamValidationError: Parameter validation failed: ``` Because boto3 expects the subject to be a string. not None. To handle this case I added if condition. Usage: 1. when you want to send SNS notification with the subject ``` AwsSnsHook().publish_to_target(target_arn='arn', message='test', subject='subject') ``` 2. when you want to send SNS notification witout subject (AWS will allocate default subject) ``` AwsSnsHook().publish_to_target(target_arn='arn', message='test') ``` Sorry, my understanding was wrong from your comment. Let me know if you have any thoughts! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow-site] kgabryje opened a new pull request #103: [depends on #96] Search meetups
kgabryje opened a new pull request #103: [depends on #96] Search meetups URL: https://github.com/apache/airflow-site/pull/103 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340278526 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: @mik-laj we need if condition because. boto3 is throwing an error when you call publish_to_target method with subject as None. Error message from boto3: ``` botocore.exceptions.ParamValidationError: Parameter validation failed: ``` Because boto3 expects the subject to be a string. not None. To handle this case I added if condition. Example: 1. when you want to send SNS notification with the subject ``` AwsSnsHook().publish_to_target(target_arn='arn', message='test', subject='subject') ``` 2. when you want to send SNS notification witout subject (AWS will allocate default subject) ``` AwsSnsHook().publish_to_target(target_arn='arn', message='test') ``` Sorry, my understanding was wrong from your comment. Let me know if you have any thoughts! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340270086 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: let me test one more time @mik-laj This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340268321 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: So we don't need one additional "If" instruction in the code? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mik-laj commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547577638 I am not sure of the correct solution. Can you write about this problem on mailing list? I don't think there has been an official discussion yet about ordering class names for AWS. The only document that describes it is AIP-21 > Case #6 Other isolated cases > There are other random cases of inconsistencies in the naming of classes. It is necessary to review the list of all classes and prepare a plan of change. Support from major cloud service providers will be useful. My team has support from Google, so we have a plan that was presented in the integration guide. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] potiuk commented on issue #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db
potiuk commented on issue #6440: [AIRFLOW-5768] GCP cloud sql don't store ephemeral connection in db URL: https://github.com/apache/airflow/pull/6440#issuecomment-547576412 Yep. Was on my list . Will do it latest tomorrow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling
dstandish commented on a change in pull request #5787: [AIRFLOW-5172] Add choice of interval edge scheduling URL: https://github.com/apache/airflow/pull/5787#discussion_r340259297 ## File path: airflow/jobs/scheduler_job.py ## @@ -648,7 +648,7 @@ def create_dag_run(self, dag, session=None): if dag.schedule_interval == '@once': period_end = next_run_date elif next_run_date: -period_end = dag.following_schedule(next_run_date) +period_end = dag.period_end(next_run_date) Review comment: it seems there is an inconsistency in the language here. The function is called `period_end`, yet sometimes it returns the start of the interval. The name `period_end` for this variable reflects an assumption that dags are scheduled after the _end_ of the interval. That's why the code checks that `period_end <= timezone.utcnow()`: end of interval is in the past. But language of this PR is in conflict with that. The parameter `schedule_at_interval_end` implies that the _interval_ doesn't change, but _where we schedule_ does. So, we may schedule at start or end of "the interval". But as written, if `schedule_at_interval_end=False`, it will in general be the case that `period_end==excecution_date`, which implies that exec date is the end of "the interval" and not the start, and this is a contradiction. It seems that what `period_end` represents in this code is more like `run_after_dttm` -- the datetime before which the dag may not be scheduled. When `schedule_at_interval_end` is True, we can run after exec date + 1 interval; otherwise, we can run after exec date. So in this bit of code, we probably don't even need `period_end()` because we could do this: ``` run_after_dttm = None if dag.schedule_interval == '@once': run_after_dttm = next_run_date elif next_run_date and not self.schedule_at_interval_end: run_after_dttm = next_run_date elif next_run_date and self.schedule_at_interval_end: run_after_dttm = dag.following_schedule(next_run_date) ``` And this: ``` if next_run_date and run_after_dttm and run_after_dttm <= timezone.utcnow(): ``` But elsewhere, it seems that `period_end()` function is used to mean `min_run_date` or `target_run_date` or `run_after_date`. Perhaps a name like this would be clearer. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[airflow-site] branch aip-11 updated: Use system hugo to build (#93)
This is an automated email from the ASF dual-hosted git repository.
kamilbregula pushed a commit to branch aip-11
in repository https://gitbox.apache.org/repos/asf/airflow-site.git
The following commit(s) were added to refs/heads/aip-11 by this push:
new 4b5921b Use system hugo to build (#93)
4b5921b is described below
commit 4b5921bb44e471d46f8a1a2a0cb1bfb468f2fca8
Author: Kamil Breguła
AuthorDate: Tue Oct 29 19:46:52 2019 +0100
Use system hugo to build (#93)
---
landing-pages/package.json | 4 -
landing-pages/yarn.lock| 715 +
2 files changed, 8 insertions(+), 711 deletions(-)
diff --git a/landing-pages/package.json b/landing-pages/package.json
index d8fc5ef..9285721 100644
--- a/landing-pages/package.json
+++ b/landing-pages/package.json
@@ -42,7 +42,6 @@
"exports-loader": "^0.7.0",
"fancy-log": "^1.3.2",
"file-loader": "^4.1.0",
-"hugo-bin": "^0.46.3",
"imports-loader": "^0.8.0",
"mini-css-extract-plugin": "^0.8.0",
"node-sass": "^4.12.0",
@@ -67,9 +66,6 @@
"webpack-merge": "^4.1.4",
"whatwg-fetch": "^3.0.0"
},
- "hugo-bin": {
-"buildTags": "extended"
- },
"resolutions": {
"natives": "1.1.6"
},
diff --git a/landing-pages/yarn.lock b/landing-pages/yarn.lock
index 3dbdba9..36bf6fb 100644
--- a/landing-pages/yarn.lock
+++ b/landing-pages/yarn.lock
@@ -725,11 +725,6 @@
resolved
"https://registry.yarnpkg.com/@nodelib/fs.stat/-/fs.stat-1.1.3.tgz#2b5a3ab3f918cca48a8c754c08168e3f03eba61b;
integrity
sha512-shAmDyaQC4H92APFoIaVDHCx5bStIocgvbwQyxPRrbUY20V1EYTbSDchWbuwlMG3V17cprZhA6+78JfB+3DTPw==
-"@sindresorhus/is@^0.7.0":
- version "0.7.0"
- resolved
"https://registry.yarnpkg.com/@sindresorhus/is/-/is-0.7.0.tgz#9a06f4f137ee84d7df0460c1fdb1135ffa6c50fd;
- integrity
sha512-ONhaKPIufzzrlNbqtWFFd+jlnemX6lJAgq9ZeiZtS7I1PIf/la7CW4m83rTXRnVnsMbW2k56pGYu7AUFJD9Pow==
-
"@types/anymatch@*":
version "1.3.1"
resolved
"https://registry.yarnpkg.com/@types/anymatch/-/anymatch-1.3.1.tgz#336badc1beecb9dacc38bea2cf32adf627a8421a;
@@ -1112,18 +1107,6 @@ aproba@^1.0.3, aproba@^1.1.1:
resolved
"https://registry.yarnpkg.com/aproba/-/aproba-1.2.0.tgz#6802e6264efd18c790a1b0d517f0f2627bf2c94a;
integrity
sha512-Y9J6ZjXtoYh8RnXVCMOU/ttDmk1aBjunq9vO0ta5x85WDQiQfUF9sIPBITdbiiIVcBo03Hi3jMxigBtsddlXRw==
-arch@^2.1.0:
- version "2.1.1"
- resolved
"https://registry.yarnpkg.com/arch/-/arch-2.1.1.tgz#8f5c2731aa35a30929221bb0640eed65175ec84e;
- integrity
sha512-BLM56aPo9vLLFVa8+/+pJLnrZ7QGGTVHWsCwieAWT9o9K8UeGaQbzZbGoabWLOo2ksBCztoXdqBZBplqLDDCSg==
-
-archive-type@^4.0.0:
- version "4.0.0"
- resolved
"https://registry.yarnpkg.com/archive-type/-/archive-type-4.0.0.tgz#f92e72233056dfc6969472749c267bdb046b1d70;
- integrity sha1-+S5yIzBW38aWlHJ0nCZ72wRrHXA=
- dependencies:
-file-type "^4.2.0"
-
are-we-there-yet@~1.1.2:
version "1.1.5"
resolved
"https://registry.yarnpkg.com/are-we-there-yet/-/are-we-there-yet-1.1.5.tgz#4b35c2944f062a8bfcda66410760350fe9ddfc21;
@@ -1189,11 +1172,6 @@ array-uniq@^1.0.1:
resolved
"https://registry.yarnpkg.com/array-uniq/-/array-uniq-1.0.3.tgz#af6ac877a25cc7f74e058894753858dfdb24fdb6;
integrity sha1-r2rId6Jcx/dOBYiUdThY39sk/bY=
-array-uniq@^2.1.0:
- version "2.1.0"
- resolved
"https://registry.yarnpkg.com/array-uniq/-/array-uniq-2.1.0.tgz#46603d5e28e79bfd02b046fcc1d77c6820bd8e98;
- integrity
sha512-bdHxtev7FN6+MXI1YFW0Q8mQ8dTJc2S8AMfju+ZR77pbg2yAdVyDlwkaUI7Har0LyOMRFPHrJ9lYdyjZZswdlQ==
-
array-unique@^0.3.2:
version "0.3.2"
resolved
"https://registry.yarnpkg.com/array-unique/-/array-unique-0.3.2.tgz#a894b75d4bc4f6cd679ef3244a9fd8f46ae2d428;
@@ -1380,56 +1358,11 @@ big.js@^5.2.2:
resolved
"https://registry.yarnpkg.com/big.js/-/big.js-5.2.2.tgz#65f0af382f578bcdc742bd9c281e9cb2d7768328;
integrity
sha512-vyL2OymJxmarO8gxMr0mhChsO9QGwhynfuu4+MHTAW6czfq9humCB7rKpUjDd9YUiDPU4mzpyupFSvOClAwbmQ==
-bin-check@^4.1.0:
- version "4.1.0"
- resolved
"https://registry.yarnpkg.com/bin-check/-/bin-check-4.1.0.tgz#fc495970bdc88bb1d5a35fc17e65c4a149fc4a49;
- integrity
sha512-b6weQyEUKsDGFlACWSIOfveEnImkJyK/FGW6FAG42loyoquvjdtOIqO6yBFzHyqyVVhNgNkQxxx09SFLK28YnA==
- dependencies:
-execa "^0.7.0"
-executable "^4.1.0"
-
-bin-version-check@^4.0.0:
- version "4.0.0"
- resolved
"https://registry.yarnpkg.com/bin-version-check/-/bin-version-check-4.0.0.tgz#7d819c62496991f80d893e6e02a3032361608f71;
- integrity
sha512-sR631OrhC+1f8Cvs8WyVWOA33Y8tgwjETNPyyD/myRBXLkfS/vl74FmH/lFcRl9KY3zwGh7jFhvyk9vV3/3ilQ==
- dependencies:
-bin-version "^3.0.0"
-semver "^5.6.0"
-semver-truncate "^1.1.2"
-
-bin-version@^3.0.0:
- version "3.1.0"
- resolved
"https://registry.yarnpkg.com/bin-version/-/bin-version-3.1.0.tgz#5b09eb280752b1bd28f0c9db3f96f2f43b6c0839;
- integrity
sha512-Mkfm4iE1VFt4xd4vH+gx+0/71esbfus2LsnCGe8Pi4mndSPyT+NGES/Eg99jx8/lUGWfu3z2yuB/bt5UB+iVbQ==
- dependencies:
-execa "^1.0.0"
-
[GitHub] [airflow-site] mik-laj merged pull request #93: Use system hugo to build
mik-laj merged pull request #93: Use system hugo to build URL: https://github.com/apache/airflow-site/pull/93 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
shaikshakeel commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook URL: https://github.com/apache/airflow/pull/5694#discussion_r340261974 ## File path: airflow/contrib/hooks/aws_sns_hook.py ## @@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message): 'default': message } +if subject is None: Review comment: yes @mik-laj. Am I missing anything? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject
in aws sns hook
URL: https://github.com/apache/airflow/pull/5694#discussion_r340255704
##
File path: airflow/contrib/hooks/aws_sns_hook.py
##
@@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message):
'default': message
}
+if subject is None:
Review comment:
```python
def publish_to_target(self, target_arn, message, subject=None):
"""
Publish a message to a topic or an endpoint.
:param target_arn: either a TopicArn or an EndpointArn
:type target_arn: str
:param message: the default message you want to send
:param message: str
:param subject: subject of message
:type subject: str
"""
conn = self.get_conn()
messages = {
'default': message
}
return conn.publish(
TargetArn=target_arn,
Message=json.dumps(messages),
MessageStructure='json',
Subject=subject
)
```
Will this code work properly in all cases?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject in aws sns hook
mik-laj commented on a change in pull request #5694: [AIRFLOW-5082] add subject
in aws sns hook
URL: https://github.com/apache/airflow/pull/5694#discussion_r340255704
##
File path: airflow/contrib/hooks/aws_sns_hook.py
##
@@ -53,8 +55,16 @@ def publish_to_target(self, target_arn, message):
'default': message
}
+if subject is None:
Review comment:
```
def publish_to_target(self, target_arn, message, subject=None):
"""
Publish a message to a topic or an endpoint.
:param target_arn: either a TopicArn or an EndpointArn
:type target_arn: str
:param message: the default message you want to send
:param message: str
:param subject: subject of message
:type subject: str
"""
conn = self.get_conn()
messages = {
'default': message
}
return conn.publish(
TargetArn=target_arn,
Message=json.dumps(messages),
MessageStructure='json',
Subject=subject
)
```
Will this code work properly?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] [airflow] mik-laj edited a comment on issue #6437: [AIRFLOW-5765] When ENV == kubernetes kerberos is also initialized
mik-laj edited a comment on issue #6437: [AIRFLOW-5765] When ENV == kubernetes kerberos is also initialized URL: https://github.com/apache/airflow/pull/6437#issuecomment-547566342 Why do we need Kerberos for Kubernetes testing? Is it not needed only for integration with Hadoop? Kerberos does not work now and tests are green. Shouldn't we delete variables so that tests don't try to connect with Kerberos? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6437: [AIRFLOW-5765] When ENV == kubernetes kerberos is also initialized
mik-laj commented on issue #6437: [AIRFLOW-5765] When ENV == kubernetes kerberos is also initialized URL: https://github.com/apache/airflow/pull/6437#issuecomment-547566342 Why do we need Kerberos for Kubernetes testing? Is it not needed only for integration with Hadoop? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible
mik-laj commented on issue #6460: [AIRFLOW-4691](https://issues.apache.org/jira/browse/AIRFLOW-4691) Make tests/cli Pylint compatible URL: https://github.com/apache/airflow/pull/6460#issuecomment-547565126 Do the tests really require docstrings? It seems to me that it has been disabled. https://github.com/apache/airflow/blob/10d39eff25657c78c2e4d99eb8f4a4779938b5c8/scripts/ci/in_container/_in_container_utils.sh#L178 https://github.com/apache/airflow/blob/47801057989046dfcf7b424ce54afee103803815/scripts/ci/in_container/run_pylint_tests.sh This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer edited a comment on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547562865 As I mentioned here https://github.com/apache/airflow/pull/6455#issuecomment-547451363, I just followed the current class naming conventions for other AWS operators/hooks. So I'm a bit confused about which conventions I should follow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers
mik-laj commented on issue #6421: [AIRFLOW-5741] Move Cloud Natural Language to providers URL: https://github.com/apache/airflow/pull/6421#issuecomment-547563917 Unfortunately. There is a conflict. Can you do rebase? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on a change in pull request #6458: [AIRFLOW-4707](https://issues.apache.org/jira/browse/AIRFLOW-4708) tests/plugins made pylint compliant
mik-laj commented on a change in pull request #6458: [AIRFLOW-4707](https://issues.apache.org/jira/browse/AIRFLOW-4708) tests/plugins made pylint compliant URL: https://github.com/apache/airflow/pull/6458#discussion_r340250197 ## File path: tests/plugins/test_plugin.py ## @@ -16,6 +16,7 @@ # KIND, either express or implied. See the License for the # specific language governing permissions and limitations # under the License. +"""Module containing """ Review comment: Can you expand this description? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mingrammer commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547562865 As I mentioned here https://github.com/apache/airflow/pull/6455#issuecomment-547451363, I just followed the current class name conventions for other AWS operators/hooks. So I'm a bit confused about which conventions I should follow. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6464: [AIRFLOW-5807] Move SFTP from contrib to core
mik-laj commented on issue #6464: [AIRFLOW-5807] Move SFTP from contrib to core URL: https://github.com/apache/airflow/pull/6464#issuecomment-547561556 @TobKed Travis is sad. Can you fix it? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [airflow] mik-laj commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mik-laj commented on issue #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths URL: https://github.com/apache/airflow/pull/6465#issuecomment-547557961 Why did you add the AWS prefix? This is not in line with GCP operator naming guidelines. We have no general rules, so we can use the GCP if they make sense. Maybe you would like to help with creating a document describing the general principles of naming operators. I will gladly add my comments. A spreadsheet with planned changes would also be helpful. I have these for GCP changes. CC: @BasPH @potiuk https://docs.google.com/document/d/1_rTdJSLCt0eyrAylmmgYc3yZr-_h51fVlnvMmWqhCkY/edit?ts=5bb72dfd# This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[jira] [Commented] (AIRFLOW-5803) Rename S3Hook to AWSS3Hook
[
https://issues.apache.org/jira/browse/AIRFLOW-5803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962272#comment-16962272
]
ASF GitHub Bot commented on AIRFLOW-5803:
-
mingrammer commented on pull request #6465: [AIRFLOW-5803] Rename S3Hook to
AWSS3Hook and update import paths
URL: https://github.com/apache/airflow/pull/6465
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5803
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Rename S3Hook to AWSS3Hook
> --
>
> Key: AIRFLOW-5803
> URL: https://issues.apache.org/jira/browse/AIRFLOW-5803
> Project: Apache Airflow
> Issue Type: Improvement
> Components: hooks
>Affects Versions: 2.0.0
>Reporter: MinJae Kwon
>Assignee: MinJae Kwon
>Priority: Minor
> Fix For: 2.0.0
>
>
> S3Hook class should be AWSS3Hook by conventions.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [airflow] mingrammer opened a new pull request #6465: [AIRFLOW-5803] Rename S3Hook to AWSS3Hook and update import paths
mingrammer opened a new pull request #6465: [AIRFLOW-5803] Rename S3Hook to
AWSS3Hook and update import paths
URL: https://github.com/apache/airflow/pull/6465
Make sure you have checked _all_ steps below.
### Jira
- [x] My PR addresses the following [Airflow
Jira](https://issues.apache.org/jira/browse/AIRFLOW/) issues and references
them in the PR title. For example, "\[AIRFLOW-XXX\] My Airflow PR"
- https://issues.apache.org/jira/browse/AIRFLOW-5803
- In case you are fixing a typo in the documentation you can prepend your
commit with \[AIRFLOW-XXX\], code changes always need a Jira issue.
- In case you are proposing a fundamental code change, you need to create
an Airflow Improvement Proposal
([AIP](https://cwiki.apache.org/confluence/display/AIRFLOW/Airflow+Improvements+Proposals)).
- In case you are adding a dependency, check if the license complies with
the [ASF 3rd Party License
Policy](https://www.apache.org/legal/resolved.html#category-x).
### Description
- [x] Here are some details about my PR, including screenshots of any UI
changes:
### Tests
- [x] My PR adds the following unit tests __OR__ does not need testing for
this extremely good reason:
### Commits
- [x] My commits all reference Jira issues in their subject lines, and I
have squashed multiple commits if they address the same issue. In addition, my
commits follow the guidelines from "[How to write a good git commit
message](http://chris.beams.io/posts/git-commit/)":
1. Subject is separated from body by a blank line
1. Subject is limited to 50 characters (not including Jira issue reference)
1. Subject does not end with a period
1. Subject uses the imperative mood ("add", not "adding")
1. Body wraps at 72 characters
1. Body explains "what" and "why", not "how"
### Documentation
- [x] In case of new functionality, my PR adds documentation that describes
how to use it.
- All the public functions and the classes in the PR contain docstrings
that explain what it does
- If you implement backwards incompatible changes, please leave a note in
the [Updating.md](https://github.com/apache/airflow/blob/master/UPDATING.md) so
we can assign it to a appropriate release
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[jira] [Updated] (AIRFLOW-5442) Druid broker hook get pandas DataFrame implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-5442: --- Fix Version/s: (was: 1.10.7) 1.10.6 > Druid broker hook get pandas DataFrame implementation > - > > Key: AIRFLOW-5442 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5442 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Affects Versions: 1.10.5 >Reporter: Sayed Mohammad Hossein Torabi >Assignee: Sayed Mohammad Hossein Torabi >Priority: Minor > Fix For: 1.10.6 > > > The *get_pandas_df* of *DruidDbApiHook* returns NotImplementedError. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (AIRFLOW-5442) Druid broker hook get pandas DataFrame implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai resolved AIRFLOW-5442. Resolution: Fixed > Druid broker hook get pandas DataFrame implementation > - > > Key: AIRFLOW-5442 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5442 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Affects Versions: 1.10.5 >Reporter: Sayed Mohammad Hossein Torabi >Assignee: Sayed Mohammad Hossein Torabi >Priority: Minor > Fix For: 1.10.6 > > > The *get_pandas_df* of *DruidDbApiHook* returns NotImplementedError. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (AIRFLOW-5442) Druid broker hook get pandas DataFrame implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chao-Han Tsai updated AIRFLOW-5442: --- Fix Version/s: 1.10.7 > Druid broker hook get pandas DataFrame implementation > - > > Key: AIRFLOW-5442 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5442 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Affects Versions: 1.10.5 >Reporter: Sayed Mohammad Hossein Torabi >Assignee: Sayed Mohammad Hossein Torabi >Priority: Minor > Fix For: 1.10.7 > > > The *get_pandas_df* of *DruidDbApiHook* returns NotImplementedError. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (AIRFLOW-5442) Druid broker hook get pandas DataFrame implementation
[ https://issues.apache.org/jira/browse/AIRFLOW-5442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16962245#comment-16962245 ] ASF GitHub Bot commented on AIRFLOW-5442: - milton0825 commented on pull request #6057: [AIRFLOW-5442] implementing get_pandas_df method for druid broker hook URL: https://github.com/apache/airflow/pull/6057 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Druid broker hook get pandas DataFrame implementation > - > > Key: AIRFLOW-5442 > URL: https://issues.apache.org/jira/browse/AIRFLOW-5442 > Project: Apache Airflow > Issue Type: Improvement > Components: hooks >Affects Versions: 1.10.5 >Reporter: Sayed Mohammad Hossein Torabi >Assignee: Sayed Mohammad Hossein Torabi >Priority: Minor > > The *get_pandas_df* of *DruidDbApiHook* returns NotImplementedError. -- This message was sent by Atlassian Jira (v8.3.4#803005)
