GitHub user zjffdu opened a pull request:
https://github.com/apache/zeppelin/pull/2898
[ZEPPELIN-1967] Passing Z variables to Shell and SQL Interpreters
The code in this PR enables embedding/interpolating Z variables into
command strings passed to Spark's SQL and Shell interpreters. It implements the
functionality described in issue
[ZEPPELIN-1967](https://issues.apache.org/jira/browse/ZEPPELIN-1967)
This PR resumes a fresh effort while taking into consideration all of
the discussion in the context of the earlier
[PR-2502](https://github.com/apache/zeppelin/pull/2502). The earlier PR-2502
was closed due to a corruption in my repo that could not be corrected.
The code in this PR resolves all of the discussion and suggestions in
the body of the earlier
[PR-2502](https://github.com/apache/zeppelin/pull/2502). The following
description is a summary of the current implementation:
Patterns of the form `{var-name}` within commands will be interpolated
only if a predefined object of the specified name exists in `z`. Additionally,
all such patterns within the command line should also be translatable for any
interpolation to occur. Partial translation of a command line (where only some
of the patterns are translated, and others are not) is never performed.
Patterns of the form `{{any-text}}` are translated into `{any-text}`.
This feature is an escaping mechanism that allows `{` and `}` characters to be
passed into a command without invoking the interpolation mechanism.
The translations described above are performed only when all
occurrences of `{`, `}`, `{{`, and `}}` in any command string conform to one of
the two forms described above. A command that contains `{` and/or `}`
characters used in any other way (than `{var-name}` and `{{any-text}}` as
described above) will be used as-is without attempting any translations
whatsoever -- even if the command also contains legal, translate-able and/or
escape-able, constructs of the above two forms.
No error is flagged in any case. This behavior is identical to the
implementation of a similar feature in
Jupyter's shell invocation using the ! magic command.
At present only the SQL and Shell interpreters support object
interpolation.
[Improvement]
* [ ] - Task
https://issues.apache.org/jira/browse/ZEPPELIN-1967
A new unit-test class ZeppCtxtVariableTest.java (see below) has been
added. The attached screenshots below also show tests of the functionality.
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, and detailed documentation has
been added to the part describing ZeppelinContext variables (see file spark.md
below).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/zjffdu/zeppelin ZEPPELIN-1967-0.8
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/zeppelin/pull/2898.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2898
----
commit 800e3c56464e472494724d571d0294540895e0ca
Author: Sanjay Dasgupta <sanjay.dasgupta@...>
Date: 2018-03-26T08:01:09Z
[ZEPPELIN-1967] Passing Z variables to Shell and SQL Interpreters
The code in this PR enables embedding/interpolating Z variables into
command strings passed to Spark's SQL and Shell interpreters. It implements the
functionality described in issue
[ZEPPELIN-1967](https://issues.apache.org/jira/browse/ZEPPELIN-1967)
This PR resumes a fresh effort while taking into consideration all of the
discussion in the context of the earlier
[PR-2502](https://github.com/apache/zeppelin/pull/2502). The earlier PR-2502
was closed due to a corruption in my repo that could not be corrected.
The code in this PR resolves all of the discussion and suggestions in the
body of the earlier [PR-2502](https://github.com/apache/zeppelin/pull/2502).
The following description is a summary of the current implementation:
Patterns of the form `{var-name}` within commands will be interpolated only
if a predefined object of the specified name exists in `z`. Additionally, all
such patterns within the command line should also be translatable for any
interpolation to occur. Partial translation of a command line (where only some
of the patterns are translated, and others are not) is never performed.
Patterns of the form `{{any-text}}` are translated into `{any-text}`. This
feature is an escaping mechanism that allows `{` and `}` characters to be
passed into a command without invoking the interpolation mechanism.
The translations described above are performed only when all occurrences of
`{`, `}`, `{{`, and `}}` in any command string conform to one of the two forms
described above. A command that contains `{` and/or `}` characters used in any
other way (than `{var-name}` and `{{any-text}}` as described above) will be
used as-is without attempting any translations whatsoever -- even if the
command also contains legal, translate-able and/or escape-able, constructs of
the above two forms.
No error is flagged in any case. This behavior is identical to the
implementation of a similar feature in
Jupyter's shell invocation using the ! magic command.
At present only the SQL and Shell interpreters support object interpolation.
[Improvement]
* [ ] - Task
https://issues.apache.org/jira/browse/ZEPPELIN-1967
A new unit-test class ZeppCtxtVariableTest.java (see below) has been added.
The attached screenshots below also show tests of the functionality.
* Does the licenses files need update? No
* Is there breaking changes for older versions? No
* Does this needs documentation? Yes, and detailed documentation has been
added to the part describing ZeppelinContext variables (see file spark.md
below).
Author: Sanjay Dasgupta <[email protected]>
Closes #2834 from sanjaydasgupta/ZEPPELIN-1967 and squashes the following
commits:
77738aa [Sanjay Dasgupta] Changes to comply with Felix Cheung's comment at
https://github.com/apache/zeppelin/pull/2834#discussion_r176976263 and Jeff
Zhang's subsequent clarification
5f8505b [Sanjay Dasgupta] Changes due to Felix Cheung's comments at
https://github.com/apache/zeppelin/pull/2834#pullrequestreview-106738198
d600d86 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
cc3727f [Sanjay Dasgupta] Changes due the Jeff Zhang's comments at
https://github.com/apache/zeppelin/pull/2834/files/1e2c87dd36dc091ca898baf8e9f178d6d1a5e600#r176930418
1e2c87d [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
3dd3dd8 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
a1703b8 [Sanjay Dasgupta] Changes suggested in Felix Cheung's review
https://github.com/apache/zeppelin/pull/2834#pullrequestreview-104805661
b7ddf6b [Sanjay Dasgupta] Implementing configuration (global enable/disable
interpolation) following
https://github.com/apache/zeppelin/pull/2834#issuecomment-373948398
5268803 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
1718e79 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
3b30ea2 [Sanjay Dasgupta] Reversing previous incorrect update
3beebce [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
f43fd99 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
a3215fc [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
ced295c [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
b461c82 [Sanjay Dasgupta] Merge branch 'master' of
https://github.com/apache/zeppelin into ZEPPELIN-1967
2868825 [Sanjay Dasgupta] ZEPPELIN-1967: Initial updates
(cherry picked from commit 4e9d2c449c071871753c8276ec0bcb1165515884)
----
---
