[
https://issues.apache.org/jira/browse/PHOENIX-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15629689#comment-15629689
]
Josh Mahonin commented on PHOENIX-3427:
---------------------------------------
Hi [~nico.pappagianis]
Thanks for the info. As I recall, the conf isn't passed through tot he
'partitionConfig' due to the fact that the Configuration object isn't
serializable. Everything that executes in the 'mapPartitions()' call is running
on a worker, but the 'outConfig' lives on the driver.
As a first try, you could attempt to retrieve the 'TenantID' parameter from the
'outConfig' ([1|
https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/ProductRDDFunctions.scala#L32])
to a local variable. Then you could set that parameter again on the
'partitionConfig'
([2|https://github.com/apache/phoenix/blob/master/phoenix-spark/src/main/scala/org/apache/phoenix/spark/ProductRDDFunctions.scala#L40])
It's not very elegant, unfortunately the Phoenix connection settings weren't
really considered with this code. The DataFrame API also has a notion of
options that we could leverage to support the connection properties as well.
Patches are most welcome. :)
(On a personal note, sorry for being a bit negligent on responding to the Spark
stuff, $DAYJOB is nuts and my wife and I just had a baby)
> rdd.saveToPhoenix gives table undefined error when attempting to write to a
> tenant-specific view (TenantId defined in configuration object and passed to
> saveToPhoenix)
> -----------------------------------------------------------------------------------------------------------------------------------------------------------------------
>
> Key: PHOENIX-3427
> URL: https://issues.apache.org/jira/browse/PHOENIX-3427
> Project: Phoenix
> Issue Type: Bug
> Reporter: Nico Pappagianis
>
> Although we can read from a tenant-specific view by passing TenantId in the
> conf object when calling sc.phoenixTableAsRDD the same does not hold for
> rdd.saveToPhoenix. Calling saveToPhoenix with a tenant-specific view as the
> table name gives a table undefined error, even when passing in the TenantId
> with the conf object.
> It appears that TenantId is lost during the execution path of saveToPhoenix.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
