Andrea Viano created SPARK-33110: ------------------------------------ Summary: array_contains doesn't pick element in the array but just at the end Key: SPARK-33110 URL: https://issues.apache.org/jira/browse/SPARK-33110 Project: Spark Issue Type: Bug Components: ML Affects Versions: 3.0.1, 2.4.7 Environment: Windows 10
Python 3.7.4 Java JDK 8 hadoop 3.0.1 Reporter: Andrea Viano Hello, given a column of array of strings. I used array_contains to find all the array with the string "Beer". array_contains is recognise just the string "Beer" if it is at the end of the array but not if it is in the middle of the array. beer=rep_trans_count.withColumn("keep", array_contains(rep_trans_count.TRANSACTIONSLIST,"Beer")).filter(col("keep")==True) -- This message was sent by Atlassian Jira (v8.3.4#803005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org