<h3><u>#general</u></h3><br><strong>@ganesh2.mohan: </strong>@ganesh2.mohan has
joined the channel<br><strong>@abfisher0417: </strong>@abfisher0417 has joined
the channel<br><strong>@divcesar: </strong>@divcesar has joined the
channel<br><strong>@npawar: </strong>Good afternoon folks!
Our chapter 2 in the Pinot Segment Lifecycle series is out!
Please enjoy this video on Segment Assignment in Realtime. This also gives an
overview on Pinot’s realtime ingestion architecture.
<https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMcPEfIgwAV3JxzQ4QVYFemG-2BzKYJ3U2AuF63z30fNMHw6uZV_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzz2xHJzo-2Frle0Z-2FPHZ72cSnK1Q4h-2B3T4u52B0VY12KNDCOebmLUoFjPXwZvHmI8gInODbgdDYN5SgrBSY8-2FIrVpfZZMsRKlyasWkU-2F-2BWJ6zTSo5Y-2B4o5KT0tNpUUX9p3aW2W1lzK-2BXtwxWdFQQBNFsOFhCZLZ0glXHC6cAMkZNYCwBbqWFrGGS5kFNvWM-2FdVw-3D><br><strong>@npawar:
</strong>And in case you missed chapter 1 on Segment Assignment in OFFLINE
tables, here’s the link to that.
Cheers!
<https://u17000708.ct.sendgrid.net/ls/click?upn=1BiFF0-2FtVRazUn1cLzaiMcPfgXTAC0ekXTv4YF-2FRhhPKf0df5hsvkFsY-2ByJjj3XLK0Et_vGLQYiKGfBLXsUt3KGBrxeq6BCTMpPOLROqAvDqBeTzz2xHJzo-2Frle0Z-2FPHZ72cS-2BK0yKx0P-2BcP5wWJ6WEAaELKwpvSRiT-2BN9nuzExTD11kIiqPimirdOAplTWoag911oz25jd8cVdO6Csz9ZfhPcXnLt1kKAnsaJSnmTlZSirWvj00urRtO9-2BQgKgdEvt0-2BH-2FJGSWpms7J-2B0U0z7jcbEbxK-2FrQX3aB-2FRLDAqBJaH28-3D><br><h3><u>#random</u></h3><br><strong>@ganesh2.mohan:
</strong>@ganesh2.mohan has joined the channel<br><strong>@abfisher0417:
</strong>@abfisher0417 has joined the channel<br><strong>@divcesar:
</strong>@divcesar has joined the
channel<br><h3><u>#troubleshooting</u></h3><br><strong>@quietgolfer: </strong>I
have a simple k8 job that runs LaunchDataIngestionJob. Is it possible to use
an environment variable in the batch job spec? I'm trying to pass a date
parameter into the input path. Here's the error and example spec.
```Caused by: groovy.lang.MissingPropertyException: No such property:
PINOT_DATE for class: SimpleTemplateScript1
at
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:53)
~[pinot-all-0.6.0-SNAPSHOT-jar-with-dependencies.jar:0.6.0-SNAPSHOT-8782e47b45c945cd02ccb8f06597b0ffa66a735a]```
```apiVersion: v1
kind: ConfigMap
metadata:
name: pinot-s3-data-config
data:
local_batch_job_spec.yaml: |-
executionFrameworkSpec:
...
inputDirURI: '<s3://promoted-event-logs/offlinepinot/dt=$JOB_DATE/>'
...
---
apiVersion: batch/v1
kind: Job
metadata:
name: pinot-populate-from-s3
spec:
template:
spec:
containers:
- name: pinot-populate-from-s3
image: apachepinot/pinot:0.6.0-SNAPSHOT-8782e47b4-20201006-jdk8
env:
- name: JOB_DATE
value: $(date -u +"%Y-%m-%d")
args: [
"LaunchDataIngestionJob",
"-jobSpecFile",
"/home/pinot/pinot-config/local_batch_job_spec.yaml"
]
volumeMounts:
- name: pinot-s3-data-config
mountPath: /home/pinot/pinot-config
restartPolicy: OnFailure
volumes:
- name: pinot-s3-data-config
configMap:
name: pinot-s3-data-config
backoffLimit: 100```<br><h3><u>#community</u></h3><br><strong>@abfisher0417:
</strong>@abfisher0417 has joined the
channel<br><h3><u>#announcements</u></h3><br><strong>@abfisher0417:
</strong>@abfisher0417 has joined the
channel<br><h3><u>#jdbc-connector</u></h3><br><strong>@kharekartik: </strong>Hi
so I am just proceeding with the broker API signature
change<br><strong>@kharekartik: </strong>we decided `/v2/`
r<br><h3><u>#lp-pinot-poc</u></h3><br><strong>@fx19880617: </strong>Hi @andrew
, I’ve tested Pinot on JDK 11 with ZGC, the p99 latency is comparatively better
than jdk8+ g1gc
Below is the graph of switching it.
Just for your reference: the docker image I’m using is
`apachepinot/pinot:0.6.0-SNAPSHOT-1126cac9b-20201002-jdk11`
the `jvmOpts` is
`-javaagent:/opt/pinot/etc/jmx_prometheus_javaagent/jmx_prometheus_javaagent-0.12.0.jar=80:/opt/pinot/etc/jmx_prometheus_javaagent/configs/pinot.yml
-Xms24G -Xmx24G -XX:+UnlockExperimentalVMOptions -XX:+UseZGC -XX:+UseNUMA
-XX:ConcGCThreads=4 -XX:ParallelGCThreads=4
-Xlog:gc*:/opt/pinot/gc-pinot-server.log`<br><strong>@fx19880617:
</strong>However p999 is still around
1sec<br><h3><u>#multi-region-setup</u></h3><br><strong>@ganesh2.mohan:
</strong>@ganesh2.mohan has joined the channel<br>
