I've defined a function that is something like the pseudo code below. It
only works properly if i define the subset1 and subset2 arrays outside of
the function. Any suggestions on how to make it work without having to
define the arrays outside of the function? I've read some of the discussion
regarding scope and I'm still not clear why this doesn't work.
Thanks.
Brandon
Function parse
create db connection
create schema
create sink
define baseurl
for url in urls
pg = xp_parse(url)
subset1 = xpath(pg,subset1 path)
subset2 = xpath(pg,subset2 path)
for s in subset1
build a dataframe
stream dataframe to db
end
same for subset2
end
end