Re: [Spark SQL]: Does Spark SQL support WAITFOR?
Hi Ram, Have you seen this stackoverflow query and response- https://stackoverflow.com/questions/39685744/apache-spark-how-to-cancel-job-in-code-and-kill-running-tasks if not, please have a look. seems to have a similar problem . *Regards,* *Someshwar Kale* On Fri, May 20, 2022 at 7:34 AM Artemis User wrote: > WAITFOR is part of the Transact-SQL and it's Microsoft SQL server > specific, not supported by Spark SQL. If you want to impose a delay in a > Spark program, you may want to use the thread sleep function in Java or > Scala. Hope this helps... > > On 5/19/22 1:45 PM, K. N. Ramachandran wrote: > > Hi Sean, > > I'm trying to test a timeout feature in a tool that uses Spark SQL. > Basically, if a long-running query exceeds a configured threshold, then the > query should be canceled. > I couldn't see a simple way to make a "sleep" SQL statement to test the > timeout. Instead, I just ran a "select count(*) from table" on a large > table to act as a query with a long duration. > > Is there any way to trigger a "sleep" like behavior in Spark SQL? > > Regards, > Ram > > On Tue, May 17, 2022 at 4:23 PM Sean Owen wrote: > >> I don't think that is standard SQL? what are you trying to do, and why >> not do it outside SQL? >> >> On Tue, May 17, 2022 at 6:03 PM K. N. Ramachandran >> wrote: >> >>> Gentle ping. Any info here would be great. >>> >>> Regards, >>> Ram >>> >>> On Sun, May 15, 2022 at 5:16 PM K. N. Ramachandran >>> wrote: >>> Hello Spark Users Group, I've just recently started working on tools that use Apache Spark. When I try WAITFOR in the spark-sql command line, I just get: Error: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'WAITFOR' expecting (.. list of allowed commands..) 1) Why is WAITFOR not allowed? Is there another way to get a process to sleep for a desired period of time? I'm trying to test a timeout issue and need to simulate a sleep behavior. 2) Is there documentation that outlines why WAITFOR is not supported? I did not find any good matches searching online. Thanks, Ram >>> >>> >>> -- >>> K.N.Ramachandran >>> Ph: 814-441-4279 >>> >> > > -- > K.N.Ramachandran > Ph: 814-441-4279 > > >
Re: [Spark SQL]: Does Spark SQL support WAITFOR?
WAITFOR is part of the Transact-SQL and it's Microsoft SQL server specific, not supported by Spark SQL. If you want to impose a delay in a Spark program, you may want to use the thread sleep function in Java or Scala. Hope this helps... On 5/19/22 1:45 PM, K. N. Ramachandran wrote: Hi Sean, I'm trying to test a timeout feature in a tool that uses Spark SQL. Basically, if a long-running query exceeds a configured threshold, then the query should be canceled. I couldn't see a simple way to make a "sleep" SQL statement to test the timeout. Instead, I just ran a "select count(*) from table" on a large table to act as a query with a long duration. Is there any way to trigger a "sleep" like behavior in Spark SQL? Regards, Ram On Tue, May 17, 2022 at 4:23 PM Sean Owen wrote: I don't think that is standard SQL? what are you trying to do, and why not do it outside SQL? On Tue, May 17, 2022 at 6:03 PM K. N. Ramachandran wrote: Gentle ping. Any info here would be great. Regards, Ram On Sun, May 15, 2022 at 5:16 PM K. N. Ramachandran wrote: Hello Spark Users Group, I've just recently started working on tools that use Apache Spark. When I try WAITFOR in the spark-sql command line, I just get: Error: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'WAITFOR' expecting (.. list of allowed commands..) 1) Why is WAITFOR not allowed? Is there another way to get a process to sleep for a desired period of time? I'm trying to test a timeout issue and need to simulate a sleep behavior. 2) Is there documentation that outlines why WAITFOR is not supported? I did not find any good matches searching online. Thanks, Ram -- K.N.Ramachandran Ph: 814-441-4279 -- K.N.Ramachandran Ph: 814-441-4279
Re: [Spark SQL]: Does Spark SQL support WAITFOR?
Hi Sean, I'm trying to test a timeout feature in a tool that uses Spark SQL. Basically, if a long-running query exceeds a configured threshold, then the query should be canceled. I couldn't see a simple way to make a "sleep" SQL statement to test the timeout. Instead, I just ran a "select count(*) from table" on a large table to act as a query with a long duration. Is there any way to trigger a "sleep" like behavior in Spark SQL? Regards, Ram On Tue, May 17, 2022 at 4:23 PM Sean Owen wrote: > I don't think that is standard SQL? what are you trying to do, and why not > do it outside SQL? > > On Tue, May 17, 2022 at 6:03 PM K. N. Ramachandran > wrote: > >> Gentle ping. Any info here would be great. >> >> Regards, >> Ram >> >> On Sun, May 15, 2022 at 5:16 PM K. N. Ramachandran >> wrote: >> >>> Hello Spark Users Group, >>> >>> I've just recently started working on tools that use Apache Spark. >>> When I try WAITFOR in the spark-sql command line, I just get: >>> >>> Error: Error running query: >>> org.apache.spark.sql.catalyst.parser.ParseException: >>> >>> mismatched input 'WAITFOR' expecting (.. list of allowed commands..) >>> >>> >>> 1) Why is WAITFOR not allowed? Is there another way to get a process to >>> sleep for a desired period of time? I'm trying to test a timeout issue and >>> need to simulate a sleep behavior. >>> >>> >>> 2) Is there documentation that outlines why WAITFOR is not supported? I >>> did not find any good matches searching online. >>> >>> Thanks, >>> Ram >>> >> >> >> -- >> K.N.Ramachandran >> Ph: 814-441-4279 >> > -- K.N.Ramachandran Ph: 814-441-4279
Re: [Spark SQL]: Does Spark SQL support WAITFOR?
I don't think that is standard SQL? what are you trying to do, and why not do it outside SQL? On Tue, May 17, 2022 at 6:03 PM K. N. Ramachandran wrote: > Gentle ping. Any info here would be great. > > Regards, > Ram > > On Sun, May 15, 2022 at 5:16 PM K. N. Ramachandran > wrote: > >> Hello Spark Users Group, >> >> I've just recently started working on tools that use Apache Spark. >> When I try WAITFOR in the spark-sql command line, I just get: >> >> Error: Error running query: >> org.apache.spark.sql.catalyst.parser.ParseException: >> >> mismatched input 'WAITFOR' expecting (.. list of allowed commands..) >> >> >> 1) Why is WAITFOR not allowed? Is there another way to get a process to >> sleep for a desired period of time? I'm trying to test a timeout issue and >> need to simulate a sleep behavior. >> >> >> 2) Is there documentation that outlines why WAITFOR is not supported? I >> did not find any good matches searching online. >> >> Thanks, >> Ram >> > > > -- > K.N.Ramachandran > Ph: 814-441-4279 >
Re: [Spark SQL]: Does Spark SQL support WAITFOR?
Gentle ping. Any info here would be great. Regards, Ram On Sun, May 15, 2022 at 5:16 PM K. N. Ramachandran wrote: > Hello Spark Users Group, > > I've just recently started working on tools that use Apache Spark. > When I try WAITFOR in the spark-sql command line, I just get: > > Error: Error running query: > org.apache.spark.sql.catalyst.parser.ParseException: > > mismatched input 'WAITFOR' expecting (.. list of allowed commands..) > > > 1) Why is WAITFOR not allowed? Is there another way to get a process to > sleep for a desired period of time? I'm trying to test a timeout issue and > need to simulate a sleep behavior. > > > 2) Is there documentation that outlines why WAITFOR is not supported? I > did not find any good matches searching online. > > Thanks, > Ram > -- K.N.Ramachandran Ph: 814-441-4279
[Spark SQL]: Does Spark SQL support WAITFOR?
Hello Spark Users Group, I've just recently started working on tools that use Apache Spark. When I try WAITFOR in the spark-sql command line, I just get: Error: Error running query: org.apache.spark.sql.catalyst.parser.ParseException: mismatched input 'WAITFOR' expecting (.. list of allowed commands..) 1) Why is WAITFOR not allowed? Is there another way to get a process to sleep for a desired period of time? I'm trying to test a timeout issue and need to simulate a sleep behavior. 2) Is there documentation that outlines why WAITFOR is not supported? I did not find any good matches searching online. Thanks, Ram
