[
https://issues.apache.org/jira/browse/SPARK-19015?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15833241#comment-15833241
]
Hyukjin Kwon commented on SPARK-19015:
--------------------------------------
If it is a question, I believe you could get a better answer from mailing lists.
If this is an issue, would you be possible to one of these as below:
- test this against the current master or higher version (such as 2.1.0)
- provide self-contained reproducer
- narrow down the problem
- provide steps to reproduce with Elasticseach and Kafka.
I am willing to help verify this.
> SQL request with transformation cannot be eecuted if not run first a scan
> table
> -------------------------------------------------------------------------------
>
> Key: SPARK-19015
> URL: https://issues.apache.org/jira/browse/SPARK-19015
> Project: Spark
> Issue Type: Bug
> Components: SQL
> Affects Versions: 1.6.0
> Reporter: lakhdar adil
>
> Hello,
> I have a spark streaming wich turn on kafka and send results to ElasticSearch.
> I have an union request between two tables: "statswithrowid" table and
> "queryes" table
> sqlContext.sql(s"select id, rowid,agentId,datecalcul,'KAFKA' as source from
> statswithrowid where id IN ($ids) and agentId = '$agent' UNION select id,
> rowid,agentId,datecalcul, 'ES' as source from queryes where agentId =
> '$agent'")
> This request cannot be executed lonely. Today i need to execute first those
> two requests so that my union request can be working fine. Please find below
> my two requests which must be launched first before union request :
> request on "statswithrowid " table :
> sqlContext.sql(s"select id, rowid,agentId,datecalcul,'KAFKA' as
> source from statswithrowid where id IN ($ids) and agentId = '$agent'").show()
> request on "queryes" table :
> sqlContext.sql(s"select id, rowid,agentId,datecalcul, 'ES' as
> source from queryes where agentId = '$agent'").show()
> For information : if i don't mention .show() on two requests to launch before
> union, nothing can work.
> Why i need to launch that first before making union request ? what is the
> best way to work with union request ? i try union with dataframe, and i have
> the same probleme.
> I look forward your reply. Thank you in advance
> Best regards,
> Adil LAKHDAR
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
