[
https://issues.apache.org/jira/browse/PHOENIX-7377?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17873214#comment-17873214
]
ASF GitHub Bot commented on PHOENIX-7377:
-----------------------------------------
rejeb commented on code in PR #134:
URL:
https://github.com/apache/phoenix-connectors/pull/134#discussion_r1715364148
##########
phoenix5-spark/README.md:
##########
@@ -293,6 +293,17 @@ to executors as a comma-separated list against the key
`phoenixConfigs` i.e (Pho
Note that the same property values will be used for both the driver and
all executors and
these configurations are used each time a connection is made (both on the
driver and executors).
+- As of [PHOENIX-7377](https://issues.apache.org/jira/browse/PHOENIX-7377),
you can pass boolean parameter to avoid mapping
+ non default family columns to `columnFamily.columnName` by setting the key
`escapeColumnFamily` to `true` (default value: `false`), for ex:
Review Comment:
I renamed the property to 'doNotMapColumnFamily'.
> phoenix5-spark dataframe issue with schema inference
> ----------------------------------------------------
>
> Key: PHOENIX-7377
> URL: https://issues.apache.org/jira/browse/PHOENIX-7377
> Project: Phoenix
> Issue Type: Bug
> Reporter: rejeb ben rejeb
> Priority: Major
>
> The fix of the PHOENIX-4981 introduced a bracking change in the way the
> schema was inferred.
> In previous versions of the connector, for non default column family ,
> columns mapped to "columnName" in DataFrame. Now, they are mapped to
> "columnFamily.columnName".
> There are no unit tests that cover this case, all tests uses tables with
> default column family "0".
> The change is made is this [pull
> request|https://github.com/apache/phoenix/pull/402] (the project was moved to
> another git repo since):
> * In previous version code uses `ColumnInfo.getDisplayName` to define the
> name of the column in the DF.
> * The new class SparkSchemaUtil the method used is
> `ColumnInfo.getColumnName` which returns the columnName as
> `columnFamilyName.columnName`.
> The pull request is related to this ticket PHOENIX-4981 the change is not
> documented.
> This change breaks jobs reading from tables having a non default column
> family.
> The saprk3 connector have the same issue since code has been duplicated from
> spark2 module to spark3 module.
> Since V1 api has been modified to use same method to resolve schema it has
> the same behavior and it should not bcause they are now a deprecated classes
> and should not contain a braking change.
>
> *Resolution proposal:*
> The best way to fix the issue is to add a property to have both options for
> schema non default column family column name mapping.
> The issue is in spark connector and it's resolution will not have a side
> effect on other phoenix-connectors like phoenix5-hive for example.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
