RE: Running Spark/YARN on AWS EMR - Issues finding file on hdfs?

I'm trying to get some workaround for this issue. Thanks and Regards, Neeraj Garg From: H4ml3t [via Apache Spark User List] [mailto:ml-node+s1001560n16379...@n3.nabble.com] Sent: Tuesday, October 14, 2014 6:53 PM To: Neeraj Garg02 Subject: Re: Running Spark/YARN on AWS EMR - Issues finding file

Re: YARN deployment of Spark and Thrift JDBC server

installed. 2. Where can I download/get Spark SQL JDBC/ODBC drivers as I could not find it on databricks site. 3. Could somebody point me to steps to connect Excel with Spark SQL and get some data SQL. Is this possible at all. 4. Which all applications can be used to connect Spark SQL. Regards, Neeraj

Help required on exercise Data Exploratin using Spark SQL

), [] Please help with the possible issue. I'm using pre-built package of Spark with Hadoop 2.4 Please let me know in case of more information is required. Regards, Neeraj -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Help-required-on-exercise-Data-Exploratin

Spark SQL DDL, DML commands

Hi, Does Spark SQL have DDL, DML commands to be executed directly. If yes, please share the link. If No, please help me understand why is it not there? Regards, Neeraj -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Spark-SQL-DDL-DML-commands-tp16572

Re: YARN deployment of Spark and Thrift JDBC server

1. I'm trying to use Spark SQL as data source.. is it possible? 2. Please share the link of ODBC/ JDBC drivers at databricks.. i'm not able to find the same. -- View this message in context:

Re: Help required on exercise Data Exploratin using Spark SQL

is the workaround for this issue. How to read the file through Spark SQL. Is there a need to change the queries? What changes can be made in the queries to get the exact result. Regards, Neeraj -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/Help-required

Steps to connect BI Tools with Spark SQL using Thrift JDBC server

Hi Everybody, I'm looking for information on possible Thrift JDBC/ODBC clients and Thrift JDBC/ODBC servers. For example, how to connect Tableu or other BI/ analytics tools to Spark using ODBC/JDBC servers as mentioned over the Spark SQL page. Thanks and Regards, Neeraj Garg

YARN deployment of Spark and Thrift JDBC server

. Thanks and Regards, Neeraj Garg CAUTION - Disclaimer * This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely for the use of the addressee(s). If you are not the intended recipient, please notify the sender by e-mail and delete the original

Spark SQL API taking longer time than DF API.

understanding, both Spark SQL and DtaaFrame API generate the same code under the hood and execution time has to be similar. Regards, Neeraj

Re: Spark SQL API taking longer time than DF API.

qry_1 and qry_2 are simple select query with groupBy clause. Are there any specific queries which works in a different way for Spark SQL and DataFrame API? Regards, Neeraj On Sat, Mar 30, 2019 at 7:27 PM Jason Nerothin wrote: > Can you please quantify the difference and provide the query c

Re: Spark SQL API taking longer time than DF API.

In Both the cases, I am trying to create a HIVE table based on Union on 2 same queries. Not sure how internally it differs on the process of creation of HIVE table? Regards, Neeraj On Sun, Mar 31, 2019 at 1:29 PM Jörn Franke wrote: > Is the select taking longer or the saving to a file.

Re: Spark SQL API taking longer time than DF API.

Hi All, Can anyone help me here with my query? Regards, Neeraj On Mon, Apr 1, 2019 at 9:44 AM neeraj bhadani wrote: > In Both the cases, I am trying to create a HIVE table based on Union on 2 > same queries. > > Not sure how internally it differs on the process of creation of

unsubscribe

unsubscribe

Re: Converting a date to milliseconds with time zone in Scala

Hi Mich, You can try Spark DateTime function here and see if that helps. https://medium.com/expedia-group-tech/deep-dive-into-apache-spark-datetime-functions-b66de737950a Regards, Neeraj On Tue, Apr 28, 2020 at 5:15 PM Mich Talebzadeh wrote: > Hi, > > I have a date in format l

array_sort function behaviour

----- [image: Screenshot 2020-05-19 12.06.30.png] As you could see the above output is sorted based on the "id" element which is the first numerical element in the struct. Is there any way to specify the element based on which sorting can be done? Regards, Neeraj

Re: Spark Window Documentation

here. If someone can help on how to contribute to the spark docs would be great. Regards, Neeraj On Fri, May 8, 2020 at 12:39 PM Jacek Laskowski wrote: > Hi Neeraj, > > I'm not a committer so I might be wrong, but there is no "blessed way" to > include examples.

Spark Window Documentation

to the spark documentation. Regards, Neeraj

Re: Dataframe to nested json document

Hi, You can follow this to save your DataFrame as JSON file. Regards, Neeraj On Sat, May 30, 2020 at 12:44 PM zakaria benzidalmal wrote: > Hi > > Just save it as json > > Le sam. 30 mai 2020 à 13:15, Chidananda Unchi a > écrit : > >> Hi All, >> >>&g

Re: Dataframe to nested json document

Hi, Apologies for missing link in the previous mail. You can follow the below link to save your DataFrame as JSON file. https://spark.apache.org/docs/latest/api/python/pyspark.sql.html#pyspark.sql.DataFrameWriter.json Regards, Neeraj <https://spark.apache.org/docs/latest/api/pyt

Blog : Apache Spark Window Functions

d feedback. Regards, Neeraj

Re: Online classes for spark topics

I am happy to be a part of this discussion as well. Regards, Neeraj On Wed, 8 Mar 2023 at 22:41, Winston Lai wrote: > +1, any webinar on Spark related topic is appreciated  > > Thank You & Best Regards > Winston Lai > -- > *From:* asma zgol