[
https://issues.apache.org/jira/browse/SPARK-12927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15108632#comment-15108632
]
Sasi commented on SPARK-12927:
------------------------------
Just as a comment, thanks for the fast response on all those tickets.
If I create DataFrame I would expect that for every changes the RDD will update
and keep the current data.
In actual the DataFrame kept the old data and didn't refresh or update
according the new data.
So,
1) I have table with data that gets update.
2) The DataFrame contains always old data and never gets updated.
Hope its clear,
Sasi
> Dataframe doesn't display the current status on Aerospike.
> ----------------------------------------------------------
>
> Key: SPARK-12927
> URL: https://issues.apache.org/jira/browse/SPARK-12927
> Project: Spark
> Issue Type: Bug
> Affects Versions: 1.5.1
> Reporter: Sasi
>
> Hi,
> I'm facing with issue on my environment when my dataframe doesn't display the
> real status of my Aerospike bin.
> I have the following code:
> {code}
> SparkContextFunctions sparkContextFunctions = new
> SparkContextFunctions(spark_context);
> String aeroSpikeAddress = host+port;
> String query = "select * from bin";
> dataFrame = sparkContextFunctions.aeroSInput(aeroSpikeAddress,
> query, sqlCtx, 6);
> dataFrame .registerTempTable("testTbl");
> dataFrame .persist(StorageLevel.MEMORY_ONLY());
> {code}
> After create the tmpTbl I do the following:
> {code}
> subscribersDataFrame.where(whereClause).collect();
> {code}
> At the first time I get the same values of the Aerospike bin, but if I add or
> delete row I don't get the same status.
> If I drop the table and create it again then I get the right status.
> Thanks.
> Sasi
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
