snleee commented on a change in pull request #6094:
URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r501879879
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentMergeRollupTaskGenerator.java
##
@@
Jackie-Jiang commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501884005
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/util/TableConfigUtils.java
##
@@ -294,14 +293,19 @@ private static void
snleee commented on pull request #6120:
URL: https://github.com/apache/incubator-pinot/pull/6120#issuecomment-705710970
@pradeepgv42 Can you use Pinot styles for the code formatting?
You can check the following page on how to set up the Pinot stylesheet.
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch offheap_bloom_filter
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from 979828b Implement off-heap bloom filter reader
add 8de5be0 Address comments
No new
fx19880617 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705758333
@npawar @mcvsubbu
This is an automated message from the Apache Git Service.
To respond to the message,
kishoreg commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705785878
This is something we saw in other users as well and we need to figure out a
way to support this. However, you seem to experience this at a low segment
size. Feels like the
mcvsubbu commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705799470
Did you run the provisioning helper tool? Can you paste the output here?
(See https://docs.pinot.apache.org/operators/operating-pinot/tuning/realtime)
That being
Jackie-Jiang commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501884945
##
File path:
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/BloomFilterConfig.java
##
@@ -16,38 +16,26 @@
* specific language
snleee commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501944324
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/OffHeapGuavaBloomFilterReader.java
##
@@ -0,0 +1,81 @@
kishoreg commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705791509
something is not adding up, its 2k msgs/per sec but segments of size 5mb are
getting flushed every 2 hours?
pradeepgv42 opened a new pull request #6120:
URL: https://github.com/apache/incubator-pinot/pull/6120
this is alternative to text_match but consuming less
disk space but more involved and hence flexible.
Realtime matching is using lucene automaton based matching
to keep the
snleee commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501869947
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/OffHeapGuavaBloomFilterReader.java
##
@@ -0,0 +1,81 @@
snleee commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501869947
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/OffHeapGuavaBloomFilterReader.java
##
@@ -0,0 +1,81 @@
snleee commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501864888
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/BloomFilterReaderFactory.java
##
@@ -0,0 +1,36 @@
+/**
+
yupeng9 opened a new issue #6121:
URL: https://github.com/apache/incubator-pinot/issues/6121
We have some use cases that demand very fresh data. For example, display
events aggregates of the **last minute** for decision making.
However, as shown in the diagram below, the ingestion
Jackie-Jiang commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501892316
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/OffHeapGuavaBloomFilterReader.java
##
@@ -0,0
snleee commented on a change in pull request #6094:
URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r501908515
##
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/FileUploadDownloadClient.java
##
@@ -148,6 +153,23 @@ public static URI
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705790392
Correct. The problem exaggerates given the data volume is high.
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a commit to branch test-swagger
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
The following commit(s) were added to refs/heads/test-swagger by this push:
new 65759c6 Add swagger dir and
snleee commented on a change in pull request #6094:
URL: https://github.com/apache/incubator-pinot/pull/6094#discussion_r501876759
##
File path:
pinot-controller/src/main/java/org/apache/pinot/controller/helix/core/minion/generator/SegmentMergeRollupTaskGenerator.java
##
@@
Jackie-Jiang commented on a change in pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118#discussion_r501889067
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/segment/index/readers/bloom/BloomFilterReaderFactory.java
##
@@ -0,0 +1,36 @@
jackjlli merged pull request #6122:
URL: https://github.com/apache/incubator-pinot/pull/6122
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above
jackjlli opened a new pull request #6122:
URL: https://github.com/apache/incubator-pinot/pull/6122
## Description
Currently the pinot-controller jar does not contain the required html file
after building the jars, which makes the api or help page fail to load (404).
This PR creates a
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch offheap_bloom_filter
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 8de5be0 Address comments
discard 979828b Implement off-heap bloom filter reader
add
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705796940
There is some approximation. Each data point is the max value of the 1 min
window. And each line maps to a partition.
mayankshriv commented on a change in pull request #6088:
URL: https://github.com/apache/incubator-pinot/pull/6088#discussion_r501853474
##
File path:
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml
##
@@ -119,6 +119,10 @@
org.apache.pinot
pplaatje opened a new issue #6123:
URL: https://github.com/apache/incubator-pinot/issues/6123
Hi,
We're looking into using Apache Pinot for our OLAP requirements, and we're
currently running into an issue:
Our Kafka Clusters, and as such also our (Confluent) schema registry
deemoliu commented on a change in pull request #6112:
URL: https://github.com/apache/incubator-pinot/pull/6112#discussion_r502022511
##
File path:
pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
##
@@ -80,6 +80,18 @@ public void testFieldSpec() {
fx19880617 opened a new pull request #6125:
URL: https://github.com/apache/incubator-pinot/pull/6125
## Description
This PR is to support any underscore in transform functions. e.g.
`IN_ID_SET`, `ARRAY_SUM`, `ARRAY_AVERAGE`, etc.
Sample query:
```
SELECT
codecov-io commented on pull request #6124:
URL: https://github.com/apache/incubator-pinot/pull/6124#issuecomment-705909906
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6124?src=pr=h1)
Report
> Merging
This is an automated email from the ASF dual-hosted git repository.
jackie pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from 4a60e9b Create swagger dir and put api dir to swagger dir in
pinot-controller resource (#6122)
add
Jackie-Jiang merged pull request #6118:
URL: https://github.com/apache/incubator-pinot/pull/6118
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL
deemoliu commented on a change in pull request #6112:
URL: https://github.com/apache/incubator-pinot/pull/6112#discussion_r502043575
##
File path:
pinot-common/src/test/java/org/apache/pinot/common/data/FieldSpecTest.java
##
@@ -80,6 +80,18 @@ public void testFieldSpec() {
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch
make_transform_functions_support_underscore_in_it
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 937189c823ab05486fa119c108d1962364d09052
Author: Xiang Fu
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705873455
I see. Thanks @mcvsubbu
This is the output of run on one sample segment
```
Memory used per host
numHosts --> 2|4|6|8|10
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705873636
I used `-periodSampleSegmentConsumed 2h`, but not sure the best value for it
This is an automated message
codecov-io commented on pull request #6125:
URL: https://github.com/apache/incubator-pinot/pull/6125#issuecomment-705899115
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6125?src=pr=h1)
Report
> Merging
pradeepgv42 commented on pull request #6120:
URL: https://github.com/apache/incubator-pinot/pull/6120#issuecomment-705899786
> @pradeepgv42 Can you use Pinot styles for the code formatting?
>
> You can check the following page on how to set up the Pinot stylesheet.
>
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705804691
Some more metrics
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705804061
@mcvsubbu No, we have not done that yet. Can look into it
But these are the JVM metrics of one server of this table. Note that this
server also ingests many other tables.
mcvsubbu commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705827639
The tool was there as of 0.1
You can download the latest version and run the tool (it is by itself as a
command line). You don't need to be running the cluster at
npawar opened a new pull request #6124:
URL: https://github.com/apache/incubator-pinot/pull/6124
## Description
https://github.com/apache/incubator-pinot/issues/5753
Here's the final piece of the feature for Pinot managed offline flows.
This is the TaskGenerator which will create
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch
make_transform_functions_support_underscore_in_it
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard f071063 Make transform functions support underscore in the
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705898894
I tried 0.5, but it seems I did not run it correctly
```
Trying to close RealtimeSegmentImpl :
rta_eats_canonical_rt_order_states__1__4457__20201008T2039Z
jackjlli commented on a change in pull request #6088:
URL: https://github.com/apache/incubator-pinot/pull/6088#discussion_r502024417
##
File path:
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml
##
@@ -119,6 +119,10 @@
org.apache.pinot
jackjlli commented on a change in pull request #6088:
URL: https://github.com/apache/incubator-pinot/pull/6088#discussion_r502024417
##
File path:
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml
##
@@ -119,6 +119,10 @@
org.apache.pinot
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705806058
hmm seems this tool is only v0.4 and later
This is an automated message from the Apache Git Service.
To
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a commit to branch
make_transform_functions_support_underscore_in_it
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit f0710637200ef99733e67ecf9aaacac18149cf44
Author: Xiang Fu
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch
make_transform_functions_support_underscore_in_it
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard 937189c Make transform functions support underscore in the
This is an automated email from the ASF dual-hosted git repository.
xiangfu pushed a change to branch
make_transform_functions_support_underscore_in_it
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
at 937189c Make transform functions support underscore in the
codecov-io edited a comment on pull request #6125:
URL: https://github.com/apache/incubator-pinot/pull/6125#issuecomment-705899115
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6125?src=pr=h1)
Report
> Merging
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
from 81028ce Enhance star-tree to skip matching-all predicate on
non-star-tree dimension (#6109)
add
jackjlli commented on a change in pull request #6088:
URL: https://github.com/apache/incubator-pinot/pull/6088#discussion_r502024417
##
File path:
pinot-plugins/pinot-batch-ingestion/v0_deprecated/pinot-hadoop/pom.xml
##
@@ -119,6 +119,10 @@
org.apache.pinot
codecov-io commented on pull request #6120:
URL: https://github.com/apache/incubator-pinot/pull/6120#issuecomment-705891235
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6120?src=pr=h1)
Report
> Merging
mcvsubbu commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705896905
If you run the later version that will be good. It has bugs ironed out
This is an automated message from
yupeng9 commented on a change in pull request #6113:
URL: https://github.com/apache/incubator-pinot/pull/6113#discussion_r502173445
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
##
@@ -224,6 +242,16 @@ public
codecov-io edited a comment on pull request #6112:
URL: https://github.com/apache/incubator-pinot/pull/6112#issuecomment-704563263
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6112?src=pr=h1)
Report
> Merging
Jackie-Jiang commented on a change in pull request #6113:
URL: https://github.com/apache/incubator-pinot/pull/6113#discussion_r502127911
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/data/manager/realtime/RealtimeTableDataManager.java
##
@@ -224,6 +242,16 @@
yupeng9 commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705966809
I see. Why shall we add more hosts? Is it because we have too long
retention, and therefore need to distribute the real-time segments to more
hosts?
cc @chenboat
yupeng9 commented on pull request #6113:
URL: https://github.com/apache/incubator-pinot/pull/6113#issuecomment-705967089
@Jackie-Jiang Addressed the comments. PTAL
This is an automated message from the Apache Git Service.
To
Jackie-Jiang commented on a change in pull request #6119:
URL: https://github.com/apache/incubator-pinot/pull/6119#discussion_r502132569
##
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/helix/HelixHelper.java
##
@@ -474,6 +480,12 @@ public IdealState
yupeng9 commented on a change in pull request #6113:
URL: https://github.com/apache/incubator-pinot/pull/6113#discussion_r502176235
##
File path:
pinot-core/src/main/java/org/apache/pinot/core/upsert/TableUpsertMetadataManager.java
##
@@ -0,0 +1,45 @@
+/**
+ * Licensed to the
codecov-io edited a comment on pull request #6113:
URL: https://github.com/apache/incubator-pinot/pull/6113#issuecomment-704686373
#
[Codecov](https://codecov.io/gh/apache/incubator-pinot/pull/6113?src=pr=h1)
Report
> Merging
mcvsubbu commented on issue #6121:
URL:
https://github.com/apache/incubator-pinot/issues/6121#issuecomment-705940435
This is the correct output. This means that your segment does not fit into
any of these buckets with the kind of memory (48GB usable)and ingestion rate
(1000 events per
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a change to branch test-swagger
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git.
discard da533a5 Remove api support
new 9069b60 Put api directory to dist dir
This update added new
KKcorps opened a new pull request #6119:
URL: https://github.com/apache/incubator-pinot/pull/6119
Add a new `/v2/brokers` API in the controller. This API is an improvement
over the previous API in the sense, that host and port are supplied along with
the instance name. The plan is to
This is an automated email from the ASF dual-hosted git repository.
jlli pushed a commit to branch test-swagger
in repository https://gitbox.apache.org/repos/asf/incubator-pinot.git
commit 9069b60bc2a221318066ae7b26837aacb91be4b6
Author: Jack Li(Analytics Engineering)
AuthorDate: Wed Oct 7
67 matches
Mail list logo
