I use https://github.com/kubeflow/spark-operator rather than bitnami chart,
but https://medium.com/@kayvan.sol2/spark-on-kubernetes-d566158186c6 shows
running spark submit from a master pod exec. Might be something to try.
On Mon, Aug 26, 2024 at 12:22 PM karan alang wrote:
> We are currently us
ably be much faster on ELK. If your queries are more interactive and
> not about batch processing then it does not make so much sense. I am not
> sure why you plan to use Presto.
>
> On 23 May 2016, at 07:28, Mat Schaffer wrote:
>
> I'm curious about trying to use spark as
I'm curious about trying to use spark as a cheap/slow ELK
(ElasticSearch,Logstash,Kibana) system. Thinking something like:
- instances rotate local logs
- copy rotated logs to s3
(s3://logs/region/grouping/instance/service/*.logs)
- spark to convert from raw text logs to parquet
- maybe presto to
