mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-481949651
@ashb @Fokko @mik-laj @XD-DENG -
Please let me know if you have any advice/recommendations for this request.
Thanks
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-479980551
Thanks @zhongjiajie.
@Fokko @XD-DENG @ashb @mik-laj - Please let me know if you have any
additional comments/feedback/recommendations for this
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-479648729
Hi - Just wondering if you have any more comments on this PR ? Thanks
This is an
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-477457895
@zhongjiajie - Thanks for your feedback. I am looking forward to the teams
further comments/recommendations. Mans
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-477355214
Hi Folks - Please let me know if you have any other advice/recommendation
for me. Thanks
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475980713
@zhongjiajie - Thanks for your feedback and advice.
@feng-tao and team - let me know if there is any additional recommendations.
Thanks
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475940400
@zhongjiajie - I've corrected to use the conn rather the hook in the poke
method. Thanks again for your help.
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475895414
@zhongjiajie - Thanks for your comments. I generally try to keep conditions
separate so that they can their logs for easy debugging. The reason, I
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475864788
@zhongjiajie - I've updated the code based on your review comments. Please
let me know if there are any more suggestions. Thanks again for your
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475799403
@mik-laj @zhongjiajie @feng-tao team - Please let me know if you have any
additional comments for me. Thanks for your advice.
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-475063699
@mik-laj - I've updated the code to reflect your recommendations (added
default conn id, and moved hook to the poke method. Please let me know if
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-474626878
@zhongjiajie - I've updated the code based on your comments. Please let me
know if you have additional advice. Thanks
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-473506861
@mik-laj -
I've updated the documentation based on your feedback. Please let me know
if that is ok and if you have any additional
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-473133742
@mik-laj @XD-DENG @feng-tao
Please let me know if you have any additional advice for me.
Thanks
Mans
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471737248
@XD-DENG -
My use case is that we have files dropped on S3 and need to process them as
soon as possible. Since I do not know the time when
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471399392
@XD-DENG - I was thinking that passing the message via xcom will allow the
subsequent operators to act on the message received. Let me know if you
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471375942
Hi @XD-DENG - My thought was to keep it simple and make sensor just get the
messages and pass it to the next operator. But let me know your
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471355820
@Fokko @feng-tao @mik-laj @zhongjiajie - Let me know if you have any
comments on this PR. Thanks
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471243168
@mik-laj - You are right - the kubernetes tests are failing.
> ERROR: test_integration_run_dag
mans2singh commented on issue #4887: [AIRFLOW-4055] Add AWS SQS Sensor
URL: https://github.com/apache/airflow/pull/4887#issuecomment-471229829
Hi Folks:
I've kicked another build from travis-ci and it is passing
(https://travis-ci.org/mans2singh/airflow).
Can you please
20 matches
Mail list logo
