Jungtaek Lim commented on STORM-1443:

I feel we're providing all the controls to users and making the thing too 
complicated. I'm not sure the target of Storm SQL users want to tune all of 
components since the goal of Storm SQL is easy to use.
Unless users include CPU intensive UDF, repartition should be the most 
expensive work and should be avoided. Trident optimizes the Trident topology 
logical plan, and while planning it can pack multiple bolts into one. I didn't 
test how Trident makes physical plan if multiple bolts can be packed but having 
different parallelism, but if we optimize more to pack multiple logical tasks 
into one physical task, repartition matters.

So I'd like to start with spout level parallelism which others will follow just 
before aggregation, and revisit if we find a reason to configure each stage.
I have some ideas in mind to specify / determine spout level parallelism, so 
I'll close this for now and open a new issue.

> Support customizing parallelism in StormSQL
> -------------------------------------------
>                 Key: STORM-1443
>                 URL: https://issues.apache.org/jira/browse/STORM-1443
>             Project: Apache Storm
>          Issue Type: New Feature
>          Components: storm-sql
>            Reporter: Haohui Mai
> Currently all processors in StormSQL have a default parallelism of 1. It is 
> desirable to have the ability to set parallelism for each processor.

This message was sent by Atlassian JIRA

Reply via email to