GitHub user HeartSaVioR opened a pull request:
https://github.com/apache/storm/pull/1656
[WIP] STORM-2057 Support JOIN statement in Storm SQL
> * Note for merger
>
>This is on top of STORM-2067 which is not applicable for 1.x-branch. So in
order to make porting back easier, STORM-2067 should be handled first and this
pull request should be rebased to updated master.
This pull request accomplishes supporting JOIN statement in Storm SQL for
Trident mode.
INNER, LEFT OUTER, RIGHT OUTER, FULL OUTER JOIN are supported.
Since this feature is based on Trident join, some limitations follow:
- Only equi-join is supported.
- Joining will be occurred for each batch, not across batches.
- http://storm.apache.org/releases/1.0.2/Trident-API-Overview.html
- Please refer `Merges & Joins` section
- Trident join provides only one value for joining fields (two)
- For inner join, that would be OK
- For outer joins, one of values should be null, which might be not OK
- applied dirty workaround: if whole values of fields are null, value
of joining field is also null
- will try to handle it properly (TODO)
TODO
- [ ] Update documentation (with limitation)
- [ ] Try to fork Trident join feature to fit to SQL join column output
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/HeartSaVioR/storm STORM-2057
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/storm/pull/1656.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #1656
----
commit 743b8e0f88a018c7851be6db2d081d31627d10c8
Author: Jungtaek Lim <[email protected]>
Date: 2016-08-30T01:22:48Z
STORM-2067 Fix "array element type mismatch" from compute-executors in
nimbus.clj
* force set type to Map when calling into-array with seq which has Maps
** Clojure optimization occurs type mismatch: PersistentArrayMap vs
PersistentHashMap
commit 542237bd0a34b45fac348a9e8a84e31cbdfdc469
Author: Jungtaek Lim <[email protected]>
Date: 2016-08-30T00:07:38Z
STORM-2057 Support JOIN statement in Storm SQL
* Support INNER, LEFT / RIGHT / FULL OUTER JOIN based on Trident
* Limitation of feature is just same to Trident join
** join is occurred for each batch (not across batches)
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
