betodealmeida commented on a change in pull request #4724: Improve database
type inference
URL:
https://github.com/apache/incubator-superset/pull/4724#discussion_r178435029
##########
File path: superset/dataframe.py
##########
@@ -41,26 +63,50 @@ class SupersetDataFrame(object):
'V': None, # raw data (void)
}
- def __init__(self, df):
- self.__df = df.where((pd.notnull(df)), None)
+ def __init__(self, data, cursor_description, db_engine_spec):
+ column_names = []
+ if cursor_description:
+ column_names = [col[0] for col in cursor_description]
+ self.column_names = dedup(column_names)
+
+ # check whether the result set has any nested dict columns
+ if data:
+ has_dict_col = any([isinstance(c, dict) for c in data[0]])
+ df_data = list(data) if has_dict_col else np.array(data,
dtype=object)
Review comment:
Are you doing this because if a column has dicts all columns will have the
`object` type in the dataframe when initialized from the numpy array? If so,
why not use `infer_objects` here?
```python
df_data = np.array(data or [], dtype=object)
df = pd.DataFrame(df_data, columns=column_names).infer_objects()
```
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
