Re: Avro file question
It depends on your usage (when and how u read). the smaller files you were thinking about are also larger than the HDFS block size? I would not go for something smaller than a block. Usually (if relevant to the way you read the data) the partitioning helps determine that. Yaniv Harpaz [ yaniv.harpaz at gmail.com ] On Mon, Nov 4, 2019 at 7:03 PM Sam wrote: > Hi, > > How do we choose between single large avro file (size much larger than > HDFS block size) vs multiple smaller avro files (close to HDFS block size? > > Since avro is splittable, is there even a need to split a very large avro > file into smaller files? > > I’m assuming that a single large avro file can also be split into multiple > mappers/reducers/executors during processing. > > Thanks. >
Re: web access to sparkUI on docker or k8s pods
thank you, I will check it out
Yaniv Harpaz
[ yaniv.harpaz at gmail.com ]
On Wed, Aug 28, 2019 at 7:14 AM Rao, Abhishek (Nokia - IN/Bangalore) <
abhishek@nokia.com> wrote:
> Hi,
>
>
>
> We have seen this issue when we tried to bringup the UI on custom ingress
> path (default ingress path “/” works). Do you also have similar
> configuration?
>
> We tired setting spark.ui.proxyBase and spark.ui.reverseProxy but did not
> help.
>
>
>
> As a workaround, we’re using ingress port (port on edge node) for now.
> There is option of using nodeport as well. That also works.
>
>
>
> Thanks and Regards,
>
> Abhishek
>
>
>
> *From:* Yaniv Harpaz
> *Sent:* Tuesday, August 27, 2019 7:34 PM
> *To:* user@spark.apache.org
> *Subject:* web access to sparkUI on docker or k8s pods
>
>
>
> hello guys,
>
> when I launch driver pods or even when I use docker run with the spark
> image,
>
> the spark master UI (8080) works great,
>
> but the sparkUI (4040) is loading w/o the CSS
>
>
>
> when I dig a bit deeper I see
>
> "Refused to apply style from '' because its MIME type ('text/html')
> is not supported stylesheet MIME type, and strict MIME checking is enabled."
>
>
>
> what am I missing here?
>
> Yaniv
>
>
> Yaniv Harpaz
> [ yaniv.harpaz at gmail.com ]
>
web access to sparkUI on docker or k8s pods
hello guys,
when I launch driver pods or even when I use docker run with the spark
image,
the spark master UI (8080) works great,
but the sparkUI (4040) is loading w/o the CSS
when I dig a bit deeper I see
"Refused to apply style from '' because its MIME type ('text/html') is
not supported stylesheet MIME type, and strict MIME checking is enabled."
what am I missing here?
Yaniv
Yaniv Harpaz
[ yaniv.harpaz at gmail.com ]
