[
https://issues.apache.org/jira/browse/DRILL-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15027464#comment-15027464
]
Julian Hyde commented on DRILL-4130:
------------------------------------
I agree: properties set at the query level would clearly override those set at
the session level. The principle of least surprise is restored.
My philosophy is that one should be able to set any property at any level above
where it is actually used. If you set it at a high level (e.g. set field
delimiter at system level) it merely becomes the default for where it is used
at a lower level. Some properties only apply at high levels (say system) and it
should be illegal to override them at lower levels.
> Ability to set settings at Table or View level rather than SESSION or SYSTEM
> ----------------------------------------------------------------------------
>
> Key: DRILL-4130
> URL: https://issues.apache.org/jira/browse/DRILL-4130
> Project: Apache Drill
> Issue Type: Improvement
> Components: Metadata
> Affects Versions: 1.3.0
> Environment: All
> Reporter: John Omernik
> Labels: administration, settings
> Fix For: Future
>
>
> There are a number of settings within drill for handling data that due to low
> level of granularity there may be unintended data reading consequences. A few
> examples include:
> store.json.read_numbers_as_double
> and
> store.json.all_text_mode
> (There are likely more, these are some I've worked with)
> The documentation on https://drill.apache.org/docs/json-data-model/ outlines
> how when dealing with certain types of data, that these settings can be
> helpful for reading, and indeed some queries fail with a suggestion to change
> these settings.
> A few points here. 1. The documentation suggests alter system commands. This
> is not ideal as it changes the default way drill handles data for all users
> AND not all users will (should) have the privs to enter this command. The
> documentation at a minimum should show alter session (or provide a clearer
> understanding of the difference)
> But even with alter session, that affects reads for all JSON files for that
> session, when in reality, the reasoning behind the setting is to be able to
> read a specific table that has poorly formed JSON. Thus, issuing a command
> that alters how Drill reads all JSON in order to read one table of JSON could
> have unintended consequences, especially for a user who just wants to be able
> to read things and issues commands without thinking things through.
> Now as an administrator, there are two use cases here. One is I have a table
> of poorly formed JSON that requires one of these settings, and I can't change
> the source, therefore, can I create a view that makes it so all reads of this
> table are done with the more permissive setting? Setting these in a view
> would be very helpful from an administrator perspective for known bad data
> sources. Keep users from having to think about it, and let them do their
> exploration.
> The other use case, is the ability for a user to set a session level read
> that only applies for the table being read. alter session set
> "%tablename%.store.json.read_numbers_as_double = true" (and have the errors
> that display use that as the default suggestion) that way, the user can issue
> the command, but not have downstream consequences in their session while
> reading other tables.
> Either case is valuable to an administrator, and could help prevent data read
> issues.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
