[
https://issues.apache.org/jira/browse/SPARK-24199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24199.
----------------------------------
Resolution: Invalid
Questions should go to mailing list rather than filing it as an issue here. I
believe you could have a better answer.
> Structured Streaming
> --------------------
>
> Key: SPARK-24199
> URL: https://issues.apache.org/jira/browse/SPARK-24199
> Project: Spark
> Issue Type: Bug
> Components: DStreams
> Affects Versions: 2.2.0
> Reporter: shuke
> Priority: Major
>
> h3. Hey,when i use the {color:#FF0000}where {color}operate to filter data
> while using Structured Streaming
> I got some problem about it
>
> ========
> get_json_object(col("value"),"$.type").cast(DataTypes.StringType).alias("type"),
>
> get_json_object(col("value"),"$.saleData.type").cast(DataTypes.StringType).alias("saleDataType",get_json_object(col("value"),
> "$.uid").cast(DataTypes.IntegerType).alias(ROI_SHOP_KEY),
> from_unixtime(get_json_object(col("value"),
> "$.time").cast(DataTypes.IntegerType),"yyyy-MM-dd").alias("event_time"),
> //get_json_object(xjson, '$.balanceData.money')/100 as money,
> (get_json_object(col("value"),
> "$.balanceData.money").cast(DataTypes.DoubleType) /
> 100).alias(BUSINESS_AMOUNT),
> get_json_object(col("value"),
> "$.shopData.id").cast(DataTypes.LongType).alias(DARK_ID),
> get_json_object(col("value"),
> "$.balanceData.out_trade_no").cast(DataTypes.StringType).alias(OUT_TRADE_NO),
> get_json_object(col("value"),
> "$.balanceData.type").cast(DataTypes.StringType).alias(BalanceData_type)
> )
> {color:#FF0000}.where("(type = 'residue' and saleDataType = '4' and shop_id
> in ('8610022','5382783')) or type = 'promotion' "{color} )
> .select(col("*"))
> .writeStream
> .trigger(Trigger.ProcessingTime(5000))
> .outputMode("Update")
> .format("console")
> .start()
> =========
> i find that its not while using this way to filter data
> anyone can help
> best wishes
>
>
>
> h1.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
