[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402070128
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
+"default_dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
-"ds": "%Y-%m-%d"
+"ds": "%Y-%m-%d",
+"hour_ts": "epoch_s",
+}
+"expression_by_column_name": {
+"hour_ts": "CAST(hour_ts as INTEGER)",
Review comment:
I think this example is a tad misleading as you wouldn't have the same
column name using both the Python datetime format and expression as these are
mutually exclusive.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402070128
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
+"default_dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
-"ds": "%Y-%m-%d"
+"ds": "%Y-%m-%d",
+"hour_ts": "epoch_s",
+}
+"expression_by_column_name": {
+"hour_ts": "CAST(hour_ts as INTEGER)",
Review comment:
I think this example is a tad misleading as you woudn't have the same column
name using both the Python datetime format and expression as these are mutually
exclusive.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066069
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I sense these default settings (including the
`python_date_format_by_column_name` should be documented outside of the SIP-15
section (this section can then reference said section). This would help to
provide context regarding default database specific fields which may be
inherited either by new Superset tables going forward or used as a fallback (in
the case of SIP-15) if a value isn't defined.
Additionally I was wondering if having all these nested under a `defaults`
section (and possibly scoped to domain) which ensures more consistency (and
thus no need for the `default_` prefix), i.e., something of the form:
```json
{
"defaults": {
"table": {
"main_dttm_col": "ds",
"expression_by_column_name": {
"hour_ts": "CAST(hour_ts as INTEGER)"
},
"dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
"hour_ts": "epoch_s"
}
}
}
}
```
Note if we go down this route the `UPDATING.md` would need to be updated to
reflect the breaking SIP-15 config change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066069
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I sense these default settings (including the
`python_date_format_by_column_name` should be documented outside of the SIP-15
section (this section can then reference said section). This would help to
provide context regarding default database specific fields which may be
inherited either by new Superset tables going forward or used as a fallback (in
the case of SIP-15) if a value isn't defined.
Additionally I was wondering if having all these nested under a `defaults`
section and scoped to tables which ensures more consistency (and thus no need
for the `default_` prefix), i.e., something of the form:
```json
{
"defaults": {
"table": {
"main_dttm_col": "ds",
"expression_by_column_name": {
"hour_ts": "CAST(hour_ts as INTEGER)"
},
"dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
"hour_ts": "epoch_s"
}
}
}
}
```
Note if we go down this route the `UPDATING.md` would need to be updated to
reflect the breaking SIP-15 config change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066069
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I sense these default settings (including the
`python_date_format_by_column_name` should be documented outside of the SIP-15
section (this section can then reference said section). This would help to
provide context regarding default database specific fields which may be
inherited either by new Superset tables going forward or used as a fallback (in
the case of SIP-15) if a value isn't defined.
Additionally I was wondering if having all these nested under a `defaults`
section and scoped to tables which ensures more consistency (and thus no need
for the `default_` prefix), i.e., something of the form:
```json
{
"defaults": {
"tables": {
"main_dttm_col": "ds",
"expression_by_column_name": {
"hour_ts": "CAST(hour_ts as INTEGER)"
},
"dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
"hour_ts": "epoch_s"
}
}
}
}
```
Note if we go down this route the `UPDATING.md` would need to be updated to
reflect the breaking SIP-15 config change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066069
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I sense these default settings (including the
`python_date_format_by_column_name` should be documented outside of the SIP-15
section (this section can then reference said section). This would help to
provide context regarding default database specific fields which may be
inherited either by new Superset tables going forward or used as a fallback (in
the case of SIP-15) if a value isn't defined.
Additionally I was wondering if having all these nested under a `defaults`
section ensures more consistency (and thus no need for the `default_` prefix),
```json
{
"defaults": {
"main_dttm_col": "ds",
"expression_by_column_name": {
"hour_ts": "CAST(hour_ts as INTEGER)"
},
"dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
"hour_ts": "epoch_s"
}
}
}
```
would be better documented in their own section, i.e., outside of SIP-15. It
could then be clearer to the reader that these database level overrides. If we
go down this route the `UPDATING.md` would need to be updated to reflect the
breaking SIP-15 config change.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066155
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
+"default_dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
-"ds": "%Y-%m-%d"
+"ds": "%Y-%m-%d",
+"hour_ts": "epoch_s",
+}
+"expression_by_column_name": {
+"hour_ts": "CAST(hour_ts as INTEGER)",
Review comment:
Nit. No trailing comma in JSON.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066069
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I sense these default settings (including the
`python_date_format_by_column_name` should be documented outside of the SIP-15
section (this section can then reference said section). This would help to
provide context regarding default database specific fields which may be
inherited either by new Superset tables going forward or used as a fallback (in
the case of SIP-15) if a value isn't defined.
Additionally I was wondering if having all these nested under a `defaults`
section ensures more consistency (and thus no need for the `default_` prefix),
```json
{
"defaults": {
"dttm_column_names": ["ds", "hour_ts"],
"main_dttm_col": "ds",
"expression_by_column_name": {
"hour_ts": "CAST(hour_ts as INTEGER)"
},
"python_date_format_by_column_name": {
"ds": "%Y-%m-%d",
"hour_ts": "epoch_s"
}
}
}
```
would be better documented in their own section, i.e., outside of SIP-15. It
could then be clearer to the reader that these database level overrides
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402066173
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
+"default_dttm_column_names": ["ds", "hour_ts"],
"python_date_format_by_column_name": {
-"ds": "%Y-%m-%d"
+"ds": "%Y-%m-%d",
+"hour_ts": "epoch_s",
Review comment:
Nit. No trailing comma in JSON.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402061395
##
File path: docs/installation.rst
##
@@ -1379,8 +1379,14 @@ Prior to SIP-15 SQLAlchemy used inclusive endpoints
however these may behave lik
To remedy this rather than having to define the date/time format for every
non-IS0 8601 date-time column, once can define a default column mapping on a
per database level via the ``extra`` parameter ::
{
+"main_dttm_column": "ds",
Review comment:
I wonder if this should be named `main_dttm_col` to be consistent with the
datasource level field.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
[GitHub] [incubator-superset] john-bodley commented on a change in pull request #9444: Implement dttm column configuration through db extra config
john-bodley commented on a change in pull request #9444: Implement dttm column
configuration through db extra config
URL:
https://github.com/apache/incubator-superset/pull/9444#discussion_r402062452
##
File path: superset/connectors/sqla/models.py
##
@@ -145,6 +146,17 @@ class TableColumn(Model, BaseColumn):
update_from_object_fields = [s for s in export_fields if s not in
("table_id",)]
export_parent = "table"
+def get_expression(self):
+if config["SIP_15_ENABLED"] and not self.expression:
+db_default_expression = (
Review comment:
Why not just `return ( ...`. This results in one less variable, etc. and
thus make the code easier to grok.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: notifications-unsubscr...@superset.apache.org
For additional commands, e-mail: notifications-h...@superset.apache.org
