Hi Josh and Josh M,
Thanks for getting back and sharing the info, I will look into dataframe,
and It is good to know the use case of csv, right now what I am doing is
passing in column like seq("col0", "col1:integer") in rdd, then later in upsert
or select construct, put in dynamic column info, but for dataframe, it probably
will be different, will look into that a part more,
and for the part that use "col1:integer", is There any concern?
Thanks for advices!
Great weekend,
Xiaopeng
在 2016年8月19日 下午5:36，Josh Mahonin <[email protected]
Hi Xiaopeng,
You're not the only one who's interested in phoenix-spark and dynamic
columns! I responded to a similar question on the phoenix-users group
recently:
http://search-hadoop.com/m/9UY0h2bNa0FLjndj1&subj=Re+Phoenix+spark+and+dynamic+columns
I'm not aware of any ongoing work for this feature. I don't think it's an
extremely complex enhancement, although it requires getting somewhat
familiar with some of Phoenix's internal Configuration helpers.
If you have any other questions, please feel free to continue posting here
and we'll try help where we can.
Thanks!
Josh
On Fri, Aug 19, 2016 at 11:26 AM, Josh Elser <[email protected]> wrote:
> Hi Xiaopeng,
>
> Thanks for the message. Using the dev list for this type of discussion is
> great.
>
> I am not familiar with the implementation (and the "gotchas" that might
> exist in this improvement), but I would encourage you to try to reach out
> on that JIRA issue you mentioned. Maybe Josh M or Randy would be able to
> give you some review as they have some more context.
>
> Thanks in advance for your contributions.
>
> - Josh
>
>
> Xiaopeng Liao wrote:
>
>> Hello,
>>
>> I just started using Phoenix and found it such a great product that makes
>> my work easier,
>>
>> And the dynamic column feature helps a lot,
>>
>>
>> And now I am trying to store data through spark RDD with dynamic column
>> (and read out),
>>
>> there is a JIRA issue PHOENIX-2648<https://issues.ap
>> ache.org/jira/browse/PHOENIX-2648> related to this, just wondering is
>> there any work being done on this one?
>>
>>
>> I implemented the dynamic column support for RDD (not dataframe yet) for
>> my own use,
>>
>> wondering if it could help in anyway..
>>
>>
>> https://github.com/xiaopeng-liao/phoenix.git
>>
>>
>> my first mail here, if there is anything not align to the guide line,
>> please point out for me, appreciate!
>>
>>
>> Thanks!
>>
>> Xiaopeng
>>
>> [https://avatars3.githubusercontent.com/u/13731092?v=3&s=400]<
>> https://github.com/xiaopeng-liao/phoenix.git>
>>
>> xiaopeng-liao/phoenix<https://github.com/xiaopeng-liao/phoenix.git>
>> github.com
>> Forked from phoenix and trying to test out features of spark connector
>> with dynamic column support based on our job requirements
>>
>>
>>
>>
>>
