Re: Spark Streaming concurrent calls

Hi Whether kafka topic's partition number can help ?! 在 2019/8/13 下午10:53, Amit Sharma 写道: I am using kafka spark streming. My UI application send request to streaming through kafka. Problem is streaming handles one request at a time so if multiple users send request at the same time they

Re: help understanding physical plan

Hi, Maybe you can look at the spark ui. The physical plan has no time consuming information. 在 2019/8/13 下午10:45, Marcelo Valle 写道: Hi, I have a job running on AWS EMR. It's basically a join between 2 tables (parquet files on s3), one somehow large (around 50 gb) and other small (less

Re: Spark streaming dataframe extract message to new columns

Hi, Do you mean haven a colum A then you want to extract A1 and A2 from A ? like Cloumn A value   123456,2019-08-07 A1 value is 123456 A2 value is 2019-08-07 If that's the case you can use df.select like this df.select(split('A)(0) as "A1", split('A)(1) as "A2") Good Luck 在 2019/8/12