Hi Xiaoxiang & Chao! I see, thanks a lot for the reply! I think we will give Xiaoxiang's patch a test drive and compare the results with what we expect for the count distinct. Best regards, Andras
On Wed, Sep 4, 2019 at 4:43 AM Xiaoxiang Yu <[email protected]> wrote: > Hi Andras, > > Count distinct(Bitmap) for String type is under development, we hope to > release it at 3.0.0-beta. If you are interested, maybe you could help us > review this PR (https://github.com/apache/kylin/pull/804). > > > > ---------------- > > Best wishes, > > Xiaoxiang Yu > > > > > > *发件人**: *Chao Long <[email protected]> > *答复**: *"[email protected]" <[email protected]> > *日期**: *2019年9月4日 星期三 09:59 > *收件人**: *"[email protected]" <[email protected]> > *主题**: *Re: Real-time OLAP - NPE in stream receiver when processing event > > > > Hi Andras, > > Real-time does not support count-distinct with non-integer datatype > column yet, is there a count-distinct measure with non-integer datatype > column? > > > > On Tue, Sep 3, 2019 at 10:38 PM Andras Nagy <[email protected]> > wrote: > > Dear All, > > > > We seem to have run into an issue with real-time streaming (in the > stream-receiver). We get a NPE in the StreamingConsumerChannel thread when > an event is received from Kafka: > > 2019-09-03 13:50:59,341 ERROR [object_movements_channel] > consumer.StreamingConsumerChannel:119 : error happens when save > event:StreamingMessage{data=[1567492920948, > 502a3dcb-547f-44bd-9f2b-e1e929ce42b0, d56ecaed-d5dd-460f-9e91-ea3fadf6f369, > e6a1760b-d023-4c92-aab5-e848067bca8c, 0f86f96e-2eb6-4271-b7a1-31a24a0b6e8e, > , , 40, 30, 6, 6, 0, 3, 9, 2019, e16a689e-f893-47a6-ab74-9881010333ec, > 2019-09-03, device, c1332a9b-0160-456a-8e43-0a2319efcaa1, > F3DBDC26-B993-4470-9EB7-22E9961B2195, 11.143630707875111], > sourcePosition=org.apache.kylin.stream.source.kafka.KafkaPosition$KafkaPartitionPosition@26b00da, > timestamp=1567518120948, params={}} > java.lang.NullPointerException > at > org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:114) > at > org.apache.kylin.measure.bitmap.BitmapMeasureType$1.valueOf(BitmapMeasureType.java:98) > at > org.apache.kylin.stream.core.storage.columnar.SegmentMemoryStore.buildValueOf(SegmentMemoryStore.java:166) > at > org.apache.kylin.stream.core.storage.columnar.SegmentMemoryStore.buildValue(SegmentMemoryStore.java:138) > at > org.apache.kylin.stream.core.storage.columnar.SegmentMemoryStore.index(SegmentMemoryStore.java:101) > at > org.apache.kylin.stream.core.storage.columnar.ColumnarSegmentStore.addEvent(ColumnarSegmentStore.java:134) > at > org.apache.kylin.stream.core.storage.StreamingCubeSegment.addEvent(StreamingCubeSegment.java:152) > at > org.apache.kylin.stream.core.storage.StreamingSegmentManager.addEvent(StreamingSegmentManager.java:173) > at > org.apache.kylin.stream.core.consumer.StreamingConsumerChannel.run(StreamingConsumerChannel.java:102) > at java.lang.Thread.run(Thread.java:748) > > > This is on our snapshot build from master (unchanged) but as I see the > https://github.com/apache/kylin/blob/master/core-metadata/src/main/java/org/apache/kylin/measure/bitmap/BitmapMeasureType.java > class has last changed on 8th April so is still the same. > > Does anyone have an idea what can be the issue or how we could debug it > further? > > > > Thank you, > > Andras > >
