[
https://issues.apache.org/jira/browse/AIRFLOW-1324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16133534#comment-16133534
]
ASF subversion and git services commented on AIRFLOW-1324:
----------------------------------------------------------
Commit de99aa20f4ffaaf0757d339abcc96961172d238c in incubator-airflow's branch
refs/heads/master from [~Fokko]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-airflow.git;h=de99aa2 ]
[AIRFLOW-1324] Generalize Druid operator and hook
Make the druid operator and hook more specific.
This allows us to
have a more flexible configuration, for example
ingest parquet.
Also get rid of the PyDruid extension since it is
more focussed on
querying druid, rather than ingesting data. Just
requests is
sufficient to submit an indexing job. Add a test
to the hive_to_druid
operator to make sure it behaves as we expect.
Furthermore cleaned
up the docstring a bit
Closes #2378 from Fokko/AIRFLOW-1324-make-more-
general-druid-hook-and-operator
> Make the Druid operator/hook more general
> -----------------------------------------
>
> Key: AIRFLOW-1324
> URL: https://issues.apache.org/jira/browse/AIRFLOW-1324
> Project: Apache Airflow
> Issue Type: Bug
> Reporter: Fokko Driesprong
> Fix For: 1.9.0
>
>
> Hi guys,
> Right now the Druid operator is quite specific with respect to the indexing
> spec. This is predefined and does not fit our use case. For example, we
> ingest parquet files instead of flat files. This is not possible right now
> and therefore a more general druid operator would be nice.
> Right now I have changed the files, we'll check them on our own cluster the
> upcoming days to make sure that they work properly.
> Cheers, Fokko
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
