Re: Re: Beam 2.29.0 throwsing warning/error when reading by using BigQuery Storage Read API

2021-05-18 Thread Filip Popić 
Here it is https://issues.apache.org/jira/browse/BEAM-12359

On 2021/05/18 16:20:10 Chamikara Jayalath wrote:
> Filip, Could you create a Jira with steps to reproduce ?
> 
> +Kenneth Jung 
> 
> Thanks,
> Cham
> 
> On Mon, May 17, 2021 at 10:29 AM Filip Popić  wrote:
> 
> > Hi,
> >
> > I am trying to read data from BigQuery table using BigQuery Storage/Direct
> > <https://cloud.google.com/bigquery/docs/reference/storage> Read API using
> > Beam 2.29.0, and providing simple negated boolean row restriction (`NOT
> > field_x`) but getting warning+errors:
> >
> > ```
> > *~*~*~ Channel ManagedChannelImpl{logId=1, target=
> > bigquerystorage.googleapis.com:443} was not shutdown properly!!! ~*~*~*
> > Make sure to call shutdown()/shutdownNow() and wait until
> > awaitTermination() returns true.
> > ```
> >
> > ```
> > Error
> > 2021-05-12 13:25:49.319 CESTError message from worker:
> > java.lang.NullPointerException
> > org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:105)
> > org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:40)
> > org.apache.beam.runners.dataflow.worker.WorkerCustomSources.splitAndValidate(WorkerCustomSources.java:294)
> > org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplitTyped(WorkerCustomSources.java:216)
> > org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplitWithApiLimit(WorkerCustomSources.java:200)
> > org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplit(WorkerCustomSources.java:179)
> > org.apache.beam.runners.dataflow.worker.WorkerCustomSourceOperationExecutor.execute(WorkerCustomSourceOperationExecutor.java:82)
> > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
> > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
> > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
> > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
> > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
> > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
> > java.util.concurrent.FutureTask.run(FutureTask.java:266)
> > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> > java.lang.Thread.run(Thread.java:748)
> > ```
> >
> > I think it might be related to
> > https://github.com/apache/beam/pull/13896/files but not sure how to fix
> > or make a workaround?
> >
>

Beam 2.29.0 throwsing warning/error when reading by using BigQuery Storage Read API

2021-05-17 Thread Filip Popić 
Hi,

I am trying to read data from BigQuery table using BigQuery Storage/Direct
 Read API using
Beam 2.29.0, and providing simple negated boolean row restriction (`NOT
field_x`) but getting warning+errors:

```
*~*~*~ Channel ManagedChannelImpl{logId=1, target=
bigquerystorage.googleapis.com:443} was not shutdown properly!!! ~*~*~*
Make sure to call shutdown()/shutdownNow() and wait until
awaitTermination() returns true.
```

```
Error
2021-05-12 13:25:49.319 CESTError message from worker:
java.lang.NullPointerException
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageSourceBase.split(BigQueryStorageSourceBase.java:105)
org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageTableSource.split(BigQueryStorageTableSource.java:40)
org.apache.beam.runners.dataflow.worker.WorkerCustomSources.splitAndValidate(WorkerCustomSources.java:294)
org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplitTyped(WorkerCustomSources.java:216)
org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplitWithApiLimit(WorkerCustomSources.java:200)
org.apache.beam.runners.dataflow.worker.WorkerCustomSources.performSplit(WorkerCustomSources.java:179)
org.apache.beam.runners.dataflow.worker.WorkerCustomSourceOperationExecutor.execute(WorkerCustomSourceOperationExecutor.java:82)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389)
org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120)
org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
java.lang.Thread.run(Thread.java:748)
```

I think it might be related to
https://github.com/apache/beam/pull/13896/files but not sure how to fix or
make a workaround?

Re: FYI on Slack Channels

2018-10-08 Thread Filip Popić 
filip.po...@gmail.com

On Mon, 8 Oct 2018 at 16:44, Jean-Baptiste Onofré  wrote:

> Hi
>
> If you don't have a apache.org e-mail address, you have to be invited.
>
> What's your email (to use for slack) ?
>
> Regards
> JB
> Le 8 oct. 2018, à 17:42, "Filip Popić"  a écrit:
>>
>> How can we join the slack workspace?
>>
>> https://the-asf.slack.com/signup  is accepting only apache emails and
>> the join link on
>>
>> https://beam.apache.org/community/contact-us/
>>
>> is not working anymore.
>>
>> On Wed, 27 Jun 2018 at 01:01, Jean-Baptiste Onofré 
>> wrote:
>>
>>> Great idea. Thanks !
>>>
>>> Regards
>>> JB
>>> Le 27 juin 2018, à 06:51, Griselda Cuevas  a écrit:
>>>>
>>>> This is awesome, thanks Rafael!
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, 26 Jun 2018 at 13:46, Scott Wegner  wrote:
>>>>
>>>>> This is great, thanks Rafael!
>>>>>
>>>>> On Tue, Jun 26, 2018 at 6:45 AM Rafael Fernandez 
>>>>> wrote:
>>>>>
>>>>>> Ah! Didn't know -- thanks Romain!
>>>>>>
>>>>>> Done for all channels I could find. Also, here is a list of channels:
>>>>>>
>>>>>> #beam
>>>>>> #beam-events-meetups
>>>>>> #beam-go
>>>>>> #beam-java
>>>>>> #beam-portability
>>>>>> #beam-python
>>>>>> #beam-sql
>>>>>> #beam-testing
>>>>>>
>>>>>>
>>>>>> On Tue, Jun 26, 2018 at 1:18 AM Romain Manni-Bucau <
>>>>>> rmannibu...@gmail.com> wrote:
>>>>>>
>>>>>>> +1 sounds very good
>>>>>>>
>>>>>>> side note: any channel must invite @asfarchivebot, I did it for the
>>>>>>> ones before "etc" but if you add others please ensure it is done
>>>>>>>
>>>>>>> Romain Manni-Bucau
>>>>>>> @rmannibucau <https://twitter.com/rmannibucau> |  Blog
>>>>>>> <https://rmannibucau.metawerx.net/> | Old Blog
>>>>>>> <http://rmannibucau.wordpress.com> | Github
>>>>>>> <https://github.com/rmannibucau> | LinkedIn
>>>>>>> <https://www.linkedin.com/in/rmannibucau> | Book
>>>>>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance>
>>>>>>>
>>>>>>>
>>>>>>> Le mar. 26 juin 2018 à 01:05, Lukasz Cwik  a
>>>>>>> écrit :
>>>>>>>
>>>>>>>> +user@beam.apache.org 
>>>>>>>>
>>>>>>>> On Mon, Jun 25, 2018 at 4:04 PM Rafael Fernandez <
>>>>>>>> rfern...@google.com> wrote:
>>>>>>>>
>>>>>>>>> Hello!
>>>>>>>>>
>>>>>>>>> I took the liberty to create area-specific channels (such as
>>>>>>>>> #beam-java, #beam-python, #beam-go, etc.) As our project and community
>>>>>>>>> grows, I am seeing more and more "organic" interest groups forming -- 
>>>>>>>>> this
>>>>>>>>> may help us chat more online. If they don't, we can delete later.
>>>>>>>>>
>>>>>>>>> Any thoughts? (I am having second thoughts... #beam-go should
>>>>>>>>> probably be #beam-burrow ;p )
>>>>>>>>>
>>>>>>>>> Cheers,
>>>>>>>>> r
>>>>>>>>>
>>>>>>>>

Re: FYI on Slack Channels

2018-10-08 Thread Filip Popić 
How can we join the slack workspace?

https://the-asf.slack.com/signup  is accepting only apache emails and the
join link on

https://beam.apache.org/community/contact-us/

is not working anymore.

On Wed, 27 Jun 2018 at 01:01, Jean-Baptiste Onofré  wrote:

> Great idea. Thanks !
>
> Regards
> JB
> Le 27 juin 2018, à 06:51, Griselda Cuevas  a écrit:
>>
>> This is awesome, thanks Rafael!
>>
>>
>>
>>
>> On Tue, 26 Jun 2018 at 13:46, Scott Wegner  wrote:
>>
>>> This is great, thanks Rafael!
>>>
>>> On Tue, Jun 26, 2018 at 6:45 AM Rafael Fernandez 
>>> wrote:
>>>
 Ah! Didn't know -- thanks Romain!

 Done for all channels I could find. Also, here is a list of channels:

 #beam
 #beam-events-meetups
 #beam-go
 #beam-java
 #beam-portability
 #beam-python
 #beam-sql
 #beam-testing


 On Tue, Jun 26, 2018 at 1:18 AM Romain Manni-Bucau <
 rmannibu...@gmail.com> wrote:

> +1 sounds very good
>
> side note: any channel must invite @asfarchivebot, I did it for the
> ones before "etc" but if you add others please ensure it is done
>
> Romain Manni-Bucau
> @rmannibucau  |  Blog
>  | Old Blog
>  | Github
>  | LinkedIn
>  | Book
> 
>
>
> Le mar. 26 juin 2018 à 01:05, Lukasz Cwik  a écrit :
>
>> +user@beam.apache.org 
>>
>> On Mon, Jun 25, 2018 at 4:04 PM Rafael Fernandez 
>> wrote:
>>
>>> Hello!
>>>
>>> I took the liberty to create area-specific channels (such as
>>> #beam-java, #beam-python, #beam-go, etc.) As our project and community
>>> grows, I am seeing more and more "organic" interest groups forming -- 
>>> this
>>> may help us chat more online. If they don't, we can delete later.
>>>
>>> Any thoughts? (I am having second thoughts... #beam-go should
>>> probably be #beam-burrow ;p )
>>>
>>> Cheers,
>>> r
>>>
>>

Re: Apache Beam Summit in Europe

2018-09-12 Thread Filip Popić 
Any left or returned tickets for the conference (advanced)?

On Thu, 5 Jul 2018 at 09:14, Etienne Chauchot  wrote:

> Hi,
>
> Just a comment, I'm not sure 28-29/09 is very practical because some of
> the Beam community will be at the apachecon in Montreal ending Sept 27th.
>
> Etienne
>
> Le mercredi 04 juillet 2018 à 17:13 +0100, Matthias Baetens a écrit :
>
> Hi everyone!
>
> Thanks for filling out the survey. We are currently looking into the
> practicalities (venue, dates, ...).
> To get a better grasp of the size of the audience, Alex has been so kind
> to start a poll on Twitter:
> https://twitter.com/alexvb/status/1014531906564775937. If you have a
> spare minute, it would be great if you could fill this out (retweets are,
> of course, greatly appreciated as well).
>
> For the dates, we are thinking of:
> - 28-29/09
> - 1-2/10 or 3-4/10
> - 15-16/10
>
> Happy to hear your preferences and comments.
> Best,
> Matthias
>
> On Thu, 31 May 2018 at 07:15 Matthias Baetens 
> wrote:
>
> Hi Beam Community,
>
> We are planning to have an Apache Beam Summit in Europe, pretty similar to
> what we hosted earlier in California on March 15th this year.
> If you'd be interested in attending, helping with organization or
> speaking, please answer this Google form
> 
> !
>
> Cheers,
> Matthias & the events and meetups folks
> --
>
>
> --
>
>
>

Re: Slack Invite

2018-04-26 Thread Filip Popić 
me also

On Thu, Apr 26, 2018, 09:23 Ismaël Mejía  wrote:

> Done, welcome!
>
> On Tue, Apr 24, 2018 at 5:47 PM, Preston Marshall 
> wrote:
> > Can someone send me an invite to the Slack? The invite link on the
> website
> > is broken.
>