creation requests every second.
>
> On Wed, Dec 2, 2020 at 3:20 PM Vasu Gupta wrote:
>
> > Hey folks,
> >
> > While using BigQueryIO for 10k tables insertion, I found that it has an
> > issue in it's local caching technique for table creation. Tables are first
>
Hey folks,
While using BigQueryIO for 10k tables insertion, I found that it has an issue
in it's local caching technique for table creation. Tables are first search in
BigQueryIO's local cache and then checks whether to create a table or not. The
main issue is when inserting to thousands of
Hey folks, why isn't there any capability of creating datasets automatically
just like tables in BigQueryIO? Actually, we at our company have dynamic
dataset architecture which means as the packet arrives, we need to create new
datasets and tables on the go. Since BigQueryIO already have
: 1582994620367, Key : e-3,5
Output : Timestamp : 1582994620368, Key : a-1,7
As you can see c-2 and d-4 are missing and I never received these packets.
On 2020/02/28 18:15:03, Kenneth Knowles wrote:
> What are the timestamps on the elements?
>
> On Fri, Feb 28, 2020 at 8:36 AM Vasu Gup
Hey folks,
Can we use JdbcIO for writing data to multiple Schemas(For Postgres Database)
dynamically using Apache beam Java Framework? Currently, I can't find any
property that I could set to JdbcIO transform for providing schema or maybe I
am missing something.
Thanks
always got correct number of packets with BATCH_SIZE = 1
On 2020/02/27 20:40:16, Kenneth Knowles wrote:
> Can you share some more details? What is the expected output and what
> output are you seeing?
>
> On Thu, Feb 27, 2020 at 9:39 AM Vasu Gupta wrote:
>
> > Hey folks, I
Hey folks, I am using Apache beam Framework in Java with Direction Runner for
local testing purposes. When using GroupIntoBatches with batch size 1 it works
perfectly fine i.e. the output of the transform is consistent and as expected.
But when using with batch size > 1 the output Pcollection
