Github user jmahonin commented on the pull request:
https://github.com/apache/phoenix/pull/59#issuecomment-88176830
Thanks for the feedback @mravi , point comments below:
1: Right, I'll try get that sorted out. The original phoenix-spark library
would not work with 1.7 for some reason, but that may no longer be the case.
2: Good catch. I think IntelliJ did something a little funny here on me,
that file was supposed to be in the main hierarchy.
3 / 4: It's my first kick at extending Spark (and Phoenix for that matter),
but the naming scheme and file separation was modelled off of DataStax'
Spark-Cassandra connector, which I figured is as good a model as any:
https://github.com/datastax/spark-cassandra-connector/tree/master/spark-cassandra-connector/src/main/scala/com/datastax/spark/connector
In theory, doing it that way means a user can have just one import to get
all the nice implicit definitions:
`import org.apache.spark.phoenix._`
5: I've never had much luck with getting the Scala integration working well
on any IDE, I just run 'mvn test' from the CLI.
Re: Good to haves
1. I totally agree, but I don't think I can afford the cycles at the
moment. My hope was that by modelling after the spark-cassandra-connector, it
would be relatively painless to add for either a third party, or myself in the
hopefully not-too-distant-future.
2. Great idea, I hadn't actually seen that usage with Spark SQL yet. We're
still using the RDD API internally. On a quick glance it looks fairly
straight-forward to implement.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
