Re: pyspark connect to spark thrift server port

By default, Spark uses Apache Derby (running in embedded mode with store content defined in local files) for hosting the Hive metastore.  You can externalize the metastore on a JDBC-compliant database (e.g., PostgreSQL) and use the database authentication provided by the database.  The JDBC

Re: pyspark connect to spark thrift server port

Hello Artemis,   Understand, if i gave hive metastore uri to anyone to connect using pyspark. the port 9083 is open for anyone without authentication feature. The only way pyspark able to connect to hive is through 9083 and not through port 1. On Friday, October 21, 2022 at 04:06:38

pyspark connect to spark thrift server port

Currently my pyspark code able to connect to hive metastore at port 9083. However using this approach i can't put in-place any security mechanism like LDAP and sql authentication control. Is there anyway to connect from pyspark to spark thrift server on port 1 without exposing hive

Spark partitioned By

Hi Team, I have asked this question in our stackoverflow group pyspark - Apache Spark partition by output path - Stack Overflow *Requirement* 1. I have huge data coming from source and loaded into Azure Data