I have data which is json in this format myList: array | | |-- elem: struct | | | |-- nm: string (nullable = true) | | | |-- vList: array (nullable = true) | | | | |-- element: string (containsNull = true)
from my kafka stream, i created a dataframe using sqlContext.jsonRDD Then registred it as registerTempTable selected mylist from this table and i see this output. It is a list of rows [Row(nm=u'Apt', vList=[u'image']), Row(nm=u'Agent', vList=[u'Mozilla/5.0 ']), Row(nm=u'Ip', vList=[u'xx.yy.106.25'])] My requirement is to get only rows with nm='IP' and its corresponding value I would need IP, xx.yy.106.25 Please suggest
