Re: Got Error Creating permanent view in Postgresql through Pyspark code
Hi
What you are trying to do does not make sense. I suggest you to understand
how Views work in SQL. IMHO you are better off creating a table.
Ayan
On Fri, 6 Jan 2023 at 12:20 am, Stelios Philippou
wrote:
> Vajiha,
>
> I dont see your query working as you hope it will.
>
> spark.sql will execute a query on a database level
>
> to retrieve the temp view you need to go from the sessions.
> i.e
>
> session.sql("SELECT * FROM TEP_VIEW")
>
> You might need to retrieve the data in a collection and iterate over them
> to do batch insertion using spark.sql("INSERt ...");
>
> Hope this helps
>
> Stelios
>
>
> --
> Hi Stelios Philippou,
> I need to create a view table in Postgresql DB using pyspark code. But I'm
> unable to create a view table, I can able to create table through pyspark
> code.
> I need to know Whether through Pyspark code can I create view table in
> postgresql database or not. Thanks for you reply
>
> Pyspark Code:
> df.createOrReplaceTempView("TEMP_VIEW")
> spark.sql("CREATE VIEW TEMP1 AS SELECT * FROM TEMP_VIEW")
>
> On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A <
> vajihabegu...@maestrowiz.com> wrote:
>
>>
>> I have tried to Create a permanent view in Postgresql DB through Pyspark
>> code, but I have received the below error message. Kindly help me to create
>> a permanent view table in the database.How shall create permanent view
>> using Pyspark code. Please do reply.
>>
>> *Error Message::*
>> *Exception has occurred: Analysis Exception*
>> Not allowed to create a permanent view `default`.`TEMP1` by referencing a
>> temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
>> VIEW
>>
>>
>> Regards,
>> Vajiha
>> Research Analyst
>> MW Solutions
>>
> --
Best Regards,
Ayan Guha
Re: Got Error Creating permanent view in Postgresql through Pyspark code
Vajiha,
I dont see your query working as you hope it will.
spark.sql will execute a query on a database level
to retrieve the temp view you need to go from the sessions.
i.e
session.sql("SELECT * FROM TEP_VIEW")
You might need to retrieve the data in a collection and iterate over them
to do batch insertion using spark.sql("INSERt ...");
Hope this helps
Stelios
--
Hi Stelios Philippou,
I need to create a view table in Postgresql DB using pyspark code. But I'm
unable to create a view table, I can able to create table through pyspark
code.
I need to know Whether through Pyspark code can I create view table in
postgresql database or not. Thanks for you reply
Pyspark Code:
df.createOrReplaceTempView("TEMP_VIEW")
spark.sql("CREATE VIEW TEMP1 AS SELECT * FROM TEMP_VIEW")
On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A
wrote:
>
> I have tried to Create a permanent view in Postgresql DB through Pyspark
> code, but I have received the below error message. Kindly help me to create
> a permanent view table in the database.How shall create permanent view
> using Pyspark code. Please do reply.
>
> *Error Message::*
> *Exception has occurred: Analysis Exception*
> Not allowed to create a permanent view `default`.`TEMP1` by referencing a
> temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP
> VIEW
>
>
> Regards,
> Vajiha
> Research Analyst
> MW Solutions
>
Re: Got Error Creating permanent view in Postgresql through Pyspark code
Vajiha, I believe that you might be confusing stuff ? Permanent View in PSQL is a standard view. Temp view or Global View is the Spark View that is internal for Spark. Can we get a snippet of the code please. On Wed, 4 Jan 2023 at 15:10, Vajiha Begum S A wrote: > > I have tried to Create a permanent view in Postgresql DB through Pyspark > code, but I have received the below error message. Kindly help me to create > a permanent view table in the database.How shall create permanent view > using Pyspark code. Please do reply. > > *Error Message::* > *Exception has occurred: Analysis Exception* > Not allowed to create a permanent view `default`.`TEMP1` by referencing a > temporary view TEMP_VIEW. Please create a temp view instead by CREATE TEMP > VIEW > > > Regards, > Vajiha > Research Analyst > MW Solutions >
