Re: Using redis cache in flink

2020-01-12 Thread Yun Tang
: user Subject: Re: Using redis cache in flink Hi Yun, Thanks for the update. I can definitely use a redis cluster but what I don't understand is if I use a custom operator then redis cache will instantiated per operator instance. What I would like to ideally have is one redis cache ins

Re: Using redis cache in flink

2020-01-12 Thread Navneeth Krishnan
-blink/src/main/java/org/apache/flink/table/runtime/operators/aggregate/MiniBatchLocalGroupAggFunction.java#L89 > > > -- > *From:* Navneeth Krishnan > *Sent:* Wednesday, January 8, 2020 15:36 > *To:* Yun Tang > *Cc:* user > *Subject:* Re: Using redis cache in flin

Re: Using redis cache in flink

2020-01-08 Thread Yun Tang
ntime/operators/aggregate/MiniBatchLocalGroupAggFunction.java#L89 From: Navneeth Krishnan Sent: Wednesday, January 8, 2020 15:36 To: Yun Tang Cc: user Subject: Re: Using redis cache in flink Hi Yun, Thanks, the way I want to use redis is like a cache not as state ba

Re: Using redis cache in flink

2020-01-08 Thread Yun Tang
flink-table-runtime-blink/src/main/java/org/apache/flink/table/runtime/operators/aggregate/MiniBatchLocalGroupAggFunction.java#L89 Best Yun Tang From: Navneeth Krishnan Sent: Wednesday, January 8, 2020 15:36 To: Yun Tang Cc: user Subject: Re: Using redis cache in

Re: Using redis cache in flink

2020-01-07 Thread Navneeth Krishnan
day, January 8, 2020 12:33 > *To:* user > *Subject:* Using redis cache in flink > > Hi All, > > I want to use redis as near far cache to store data which are common > across slots i.e. share data across slots. This data is required for > processing every single message

Re: Using redis cache in flink

2020-01-07 Thread Yun Tang
Yun Tang From: Navneeth Krishnan Sent: Wednesday, January 8, 2020 12:33 To: user Subject: Using redis cache in flink Hi All, I want to use redis as near far cache to store data which are common across slots i.e. share data across slots. This data is required

Using redis cache in flink

2020-01-07 Thread Navneeth Krishnan
Hi All, I want to use redis as near far cache to store data which are common across slots i.e. share data across slots. This data is required for processing every single message and it's better to store in a in memory cache backed by redis rather than rocksdb since it has to be serialized for ever