Re: Phoenix spark and dynamic columns

2016-07-27 Thread Paul Jones
Josh, Thank you for your reply. I will take a look at your suggestions. Thanks, Paul Hi Paul, Unfortunately out of the box the Spark integration doesn't support saving to dynamic columns. It's worth filing a JIRA enhancement over, and if you're interested in contributing a patch, here's

Re: Phoenix spark and dynamic columns

2016-07-27 Thread Josh Mahonin
Hi Paul, Unfortunately out of the box the Spark integration doesn't support saving to dynamic columns. It's worth filing a JIRA enhancement over, and if you're interested in contributing a patch, here's the following spots I think would need enhancing: The saving code derives the column names to

Phoenix spark and dynamic columns

2016-07-25 Thread Paul Jones
Is it possible to save a dataframe into a table where the columns are dynamic? For instance, I have a loaded a CSV file with header (key, cat1, cat2) into a dataframe. All values are strings. I created a table like this: create table mytable ("KEY" varchar not null primary key); The code is as