vikramsinghchandel commented on issue #9820:
URL: https://github.com/apache/druid/issues/9820#issuecomment-662016019
> Hi @vikramsinghchandel, I bet you can workaround this bug by setting
`intervals` in the granularitySpec. Can you try with explicit intervals?
>
> > @sixtus were you able to find any solution/workaround. With hashed
partitioner & forcedGranularityRollUp the performance of indexing is very bad
almost twice as bad compared to EMR based indexing
>
> BTW, this is interesting. Was the setup exactly same for both ingestion
methods? Could you share how they are different in what setup if possible?
@jihoonson
I already have set intervals in granularitySpec
this is how it looks
`"granularitySpec": {
"type": "uniform",
"segmentGranularity": "HOUR",
"queryGranularity": "HOUR",
"rollup": true,
"intervals": [
"2020-07-16/2020-07-18"
]
}`
Sure Following is the setup
EMR:: 22 * m5.2xl nodes == 176 vCPU
Native (K8S indexer nodes) = 50 nodes each with 4 CPU and 26G memory (200
vCPU)
Each node has 3 worker so a total of 150 workers are spawned for native
ingestion.
Let me know if you need detailed numbers for infra and ingestion, can create
a separate ticket with the details.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
