calling rawCountsSDF.createOrReplaceTempView( "rawCounts" ) ? I
> expected to manage spark to manage the cache automatically given that I do
> not explicitly call cache().
>
>
>
>
>
> How come I do not get a similar warning from?
>
> sa
Hi
I am using Airflow in such scenario
Hello Users,
Is there any alternative for https://github.com/databricks/spark-redshift on
scala 2.12.x?
Thanks
--
[image: vshapesaqua11553186012.gif] <https://vungle.com/> *Jun Zhu*
Sr. Engineer I, Data
+86 18565739171
[image: in1552694272.png] <https://www.linkedin.com/compa
tore
> with URI thrift://ip-172-19-104-48.ec2.internal:9083
> 19/06/04 05:58:18 INFO HiveMetaStoreClient: Opened a connection to
> metastore, current connections: 1
> 19/06/04 05:58:18 INFO HiveMetaStoreClient: Connected to metastore.
> 19/06/04 05:58:18 INFO RetryingMetaStoreClient: Retr
plainCommand 'Project [unresolvedalias('count(1), None)], false,
> false, false
> *19/06/04 05:50:15* INFO SparkExecuteStatementOperation: Result Schema:
> StructType(StructField(plan,StringType,true))
Had set thrift server miniresource(10 instance) and initresourc
Never mind, I got the point, spark replace hive parquet with it's own,
Should set spark.sql.hive.convertMetastoreParquet=false to use hive's.
Thanks
On Thu, Apr 25, 2019 at 5:00 PM Jun Zhu wrote:
> Hi,
> We are using plugins from apache hudi which self defined a hive
ailed to recognize self-define inputformat.
Any thoughts? Or can I config the FileScan to HiveTableScan? thanks~
Best,
--
[image: vshapesaqua11553186012.gif] <https://vungle.com/> *Jun Zhu*
Sr. Engineer I, Data
+86 18565739171
[image: in1552694272.png] <https://www.linkedin.co