Re: [VOTE] Apache Bahir 2.4.0 (RC1)

[Ted Yu]

+1

On Fri, Sep 20, 2019 at 12:37 PM Luciano Resende 
wrote:

> Off course, my +1
> I know a lot of us are just between travels (returning from ApacheCon)
> but are any other volunteers available to review the release?
>
> On Fri, Sep 13, 2019 at 10:38 AM Luciano Resende 
> wrote:
> >
> > Dear community member,
> >
> > Please vote to approve the release of Apache Bahir 2.4.0 (RC1) based on
> > Apache Spark 2.4.0.
> >
> > Tag: v2.4.0-rc1 (f908ec0dc1bbc4c6d11cde446f2bfd89ea39155f)
> >
> > https://github.com/apache/bahir/tree/v2.4.0-rc1
> >
> > Release files:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1031
> >
> > Source distribution:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.4.0-rc1/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> >   [ ] +1 Release this package as Apache Bahir 2.4.0
> >   [ ] -1 Do not release this package because ...
> >
> >
> > Thanks for your vote!
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.3.4 (RC1)

[Ted Yu]

+1

On Fri, Sep 20, 2019 at 12:37 PM Luciano Resende 
wrote:

> Off course, my +1
> I know a lot of us are just between travels (returning from ApacheCon)
> but are any other volunteers available to review the release?
>
> On Wed, Sep 11, 2019 at 5:42 PM Luciano Resende 
> wrote:
> >
> > Dear community member,
> >
> > Please vote to approve the release of Apache Bahir 2.3.4 (RC1) based on
> > Apache Spark 2.3.4.
> >
> > Tag: v2.3.4-rc1 (716107f420ac3e0afd76e61b74069e551d9a7e15)
> >
> > https://github.com/apache/bahir/tree/v2.3.4-rc1
> >
> > Release files:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1030
> >
> > Source distribution:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.3.4-rc1/
> >
> > Also, here is a list of changes between  v2.3.3 and v2.3.4-rc1 tags,
> > which summarize in updating Apache Spark to 2.3.4 and release build
> > preparation.
> >
> > https://github.com/apache/bahir/compare/v2.3.3...v2.3.4-rc1
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> >   [ ] +1 Release this package as Apache Bahir 2.3.4
> >   [ ] -1 Do not release this package because ...
> >
> >
> > Thanks for your vote!
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.2.3 (RC1)

[Ted Yu]

+1

On Thu, May 30, 2019 at 9:01 AM Luciano Resende 
wrote:

> Thanks, Christian,
>
> We are still looking for one more vote. Note that we had two
> concurrent votes going on and this is a different vote thread (e.g.
> 2.2.3 versus 2.3.3)
>
> On Tue, May 28, 2019 at 12:20 AM Christian Kadner 
> wrote:
> >
> > +1
> >
> > Sorry for the delay, I was traveling last week.
> >
> > Thanks Luciano!
> >
> > ~ Christian
> >
> > On 2019/05/19 14:53:32, Luciano Resende  wrote:
> > > Dear community member,
> > >
> > > Please vote to approve the release of Apache Bahir 2.2.3 (RC1) based on
> > > Apache Spark 2.2.3.
> > >
> > > Tag: v2.2.3-rc1 (963c644ff96615bf53ed6570abcf6930d1532776)
> > >
> > > https://github.com/apache/bahir/tree/v2.2.3-rc1
> > >
> > > Release files:
> > >
> > > https://repository.apache.org/content/repositories/orgapachebahir-1028
> > >
> > > Source distribution:
> > >
> > > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.2.3-rc1/
> > >
> > >
> > > The vote is open for at least 72 hours and passes if a majority of at
> least
> > > 3 +1 PMC votes are cast.
> > >
> > >   [ ] +1 Release this package as Apache Bahir 2.2.3
> > >   [ ] -1 Do not release this package because ...
> > >
> > >
> > > Thanks for your vote!
> > >
> > > --
> > > Luciano Resende
> > > http://twitter.com/lresende1975
> > > http://lresende.blogspot.com/
> > >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.3.3 (RC1)

[Ted Yu]

+1

On Sat, May 25, 2019 at 5:51 PM Luciano Resende 
wrote:

> Off course, my +1 for the release.
>
> We are still looking for an extra vote to approve the release, please
> take a little time and help approve the release.
>
> On Sun, May 19, 2019 at 9:15 AM Luciano Resende 
> wrote:
> >
> > Dear community member,
> >
> > Please vote to approve the release of Apache Bahir 2.3.3 (RC1) based on
> > Apache Spark 2.3.3.
> >
> > Tag: v2.3.3-rc1 (e29034cad9bec11da1b81324b1f67118772861d2)
> >
> > https://github.com/apache/bahir/tree/v2.3.3-rc1
> >
> > Release files:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1029
> >
> > Source distribution:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.3.3-rc1/
> >
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> >   [ ] +1 Release this package as Apache Bahir 2.3.3
> >   [ ] -1 Do not release this package because ...
> >
> >
> > Thanks for your vote!
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: Upgrading to Scala 2.12

[Ted Yu]

+1 on keeping backward compatibility with Scala 2.11
 Original message From: Łukasz Antoniak 
 Date: 1/22/19  2:54 AM  (GMT-08:00) To: 
dev@bahir.apache.org Subject: Upgrading to Scala 2.12 
Team,

Since version 2.4.0 Spark team decided to introduce support for Scala 2.12
and remain 2.11 for backward compatibility. As part of BAHIR-107, I have
been working to upgrade code base to Scala 2.12 and Spark 2.4.0 (
https://github.com/apache/bahir/pull/76/files). Do we plan to continue
support for Scala 2.11? In my opinion, Bahir should follow Spark in terms
of Scala binary compatibility. I have already checked that transitive
dependencies that I had to upgrade (org.json4s:json4s-jackson and
com.twitter:algebird-core) provide artifacts for both - Scala 2.11 and 2.12.

Lukasz

Re: [PROPOSAL] Move to gitbox.apache.org

[Ted Yu]

+1
 Original message From: Prashant Sharma  
Date: 12/7/18  9:01 PM  (GMT-08:00) To: dev@bahir.apache.org Subject: Re: 
[PROPOSAL] Move to gitbox.apache.org 
+1



On Sat, Dec 8, 2018 at 2:56 AM Christian Kadner  wrote:

> +1
>
> Thanks JB!
>
> On 2018/12/07 17:20:59, Luciano Resende  wrote:
> > I think we can drive this with consensus (unless infra requires a vote)
> >
> > +1
> > On Fri, Dec 7, 2018 at 9:05 AM Jean-Baptiste Onofré 
> wrote:
> > >
> > > Hi all,
> > >
> > > our repositories are currently located on git-wip-us.apache.org.
> > >
> > > This service will be decommissioned in the coming months.
> > >
> > > I propose to move to gitbox.apache.org.
> > >
> > > I can start a formal vote and if it's OK, I'm volunteer to deal with
> infra.
> > >
> > > Thoughts ?
> > >
> > > Regards
> > > JB
> > > --
> > > Jean-Baptiste Onofré
> > > jbono...@apache.org
> > > http://blog.nanthrax.net
> > > Talend - http://www.talend.com
> >
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>

Re: [VOTE] Apache Bahir 2.3.1 (RC1)

[Ted Yu]

+1

Ran test suite which passed.

On Fri, Nov 30, 2018 at 9:13 PM Jean-Baptiste Onofré 
wrote:

> +1 (binding)
>
> Regards
> JB
>
> On 30/11/2018 17:11, Luciano Resende wrote:
> > Dear community member,
> >
> > Please vote to approve the release of Apache Bahir 2.3.1 (RC1) based on
> > Apache Spark 2.3.1.
> >
> > Tag: v2.3.1-rc1 (6c4d67e0a99bcbbad199b7d1d26f3624491070b4)
> >
> > https://github.com/apache/bahir/tree/v2.3.1-rc1
> >
> > Release files:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1026
> >
> > Source distribution:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.3.1-rc1/
> >
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> >   [ ] +1 Release this package as Apache Bahir 2.3.1
> >   [ ] -1 Do not release this package because ...
> >
> >
> > Thanks for your vote!
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>

Re: [VOTE] Apache Bahir 2.3.2 (RC1)

[Ted Yu]

+1

Ran test suite which passed

On Fri, Nov 30, 2018 at 12:57 PM Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.3.2 (RC1) based on
> Apache Spark 2.3.2.
>
> Tag: v2.3.2-rc1 (6d24b270fd713352faf1c93f08f835e39e496489)
>
> https://github.com/apache/bahir/tree/v2.3.2-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1027
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.3.2-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.3.2
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.3.0 (RC1)

[Ted Yu]

+1

Ran test suite which passed.

On Fri, Nov 30, 2018 at 6:43 AM Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.3.0 (RC1) based on
> Apache Spark 2.3.0.
>
> Tag: v2.3.0-rc1 (e2d138bd2a927d0ab42e45739d8938eefafca352)
>
> https://github.com/apache/bahir/tree/v2.3.0-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1025
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.3.0-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.3.0
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.1.3 (RC1)

[Ted Yu]

+1

On Thu, Nov 8, 2018 at 1:55 PM Luciano Resende  wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.1.3 (RC1) based on
> Apache Spark 2.1.3.
>
> Tag: v2.1.3-rc1 (697d222ced91feae2d21cbc0dfa85d18e7f14b92)
>
> https://github.com/apache/bahir/tree/v2.1.3-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1023
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.1.3-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.1.3
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.2.2 (RC1)

[Ted Yu]

+1

Ran unit test suite which passed.

On Thu, Nov 8, 2018 at 1:34 PM Luciano Resende  wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.2.2 (RC1) based on
> Apache Spark 2.2.2.
>
> Tag: v2.2.2-rc1 (821a8c67c21f4f4ab4a7caa8e2f85a2c396683d4)
>
> https://github.com/apache/bahir/tree/v2.2.2-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1024
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.2.2-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.2.2
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12

See SPARK-14220 .

This JIRA is to track requirements for building Bahir on Scala 2.12.7

  was:
Spark has started effort for accommodating Scala 2.12

See SPARK-14220 .

This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>Priority: Major
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220 .
> This JIRA is to track requirements for building Bahir on Scala 2.12.7



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BAHIR-172) Avoid FileInputStream/FileOutputStream

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-172:
-
Description: 
They rely on finalizers (before Java 11), which create unnecessary GC load.


The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.

  was:
They rely on finalizers (before Java 11), which create unnecessary GC load.

The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.


> Avoid FileInputStream/FileOutputStream
> --
>
> Key: BAHIR-172
> URL: https://issues.apache.org/jira/browse/BAHIR-172
> Project: Bahir
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> They rely on finalizers (before Java 11), which create unnecessary GC load.
> The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
> this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12

See SPARK-14220 .

This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12

See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>Priority: Major
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220 .
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BAHIR-172) Avoid FileInputStream/FileOutputStream

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-172:
-
Description: 
They rely on finalizers (before Java 11), which create unnecessary GC load.

The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.

  was:
They rely on finalizers (before Java 11), which create unnecessary GC load.


The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.


> Avoid FileInputStream/FileOutputStream
> --
>
> Key: BAHIR-172
> URL: https://issues.apache.org/jira/browse/BAHIR-172
> Project: Bahir
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> They rely on finalizers (before Java 11), which create unnecessary GC load.
> The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
> this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BAHIR-172) Avoid FileInputStream/FileOutputStream

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-172:
-
Description: 
They rely on finalizers (before Java 11), which create unnecessary GC load.


The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.

  was:
They rely on finalizers (before Java 11), which create unnecessary GC load.

The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.


> Avoid FileInputStream/FileOutputStream
> --
>
> Key: BAHIR-172
> URL: https://issues.apache.org/jira/browse/BAHIR-172
> Project: Bahir
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> They rely on finalizers (before Java 11), which create unnecessary GC load.
> The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
> this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Catching up on Spark Releases

[Ted Yu]

2.2.2 has been released too:

http://search-hadoop.com/m/Spark/q3RTt4hGjQ15zeqM?subj=+ANNOUNCE+Apache+Spark+2+2+2

On Sun, Jul 1, 2018 at 5:31 PM Ted Yu  wrote:

> Spark 2.1.3 has been released:
>
>
> http://search-hadoop.com/m/Spark/q3RTtDu0Z8V9VMZ?subj=+ANNOUNCE+Apache+Spark+2+1+3
>
> Looking forward to the corresponding Bahir release.
>
> On Fri, Jun 29, 2018 at 5:07 AM, Luciano Resende 
> wrote:
>
>> Just heads up that Spark is currently voting on Spark 2.1.3 and 2.2.2. As
>> soon as they get approved I will start the correspondent Bahir releases.
>>
>> On Tue, May 29, 2018 at 9:48 PM Luciano Resende 
>> wrote:
>>
>> > So, it looks like we can do a release based on Spark 2.1.2 and Spark
>> 2.2.1.
>> >
>> > After that, we will need to migrate some of our extensions to
>> > datasource v2 API and do a release based on Spark 2.3
>> >
>> > I will get started with the Bahir 2.1.2 and 2.2.1 releases.
>> >
>> > On Sun, Apr 22, 2018 at 3:24 PM, Luciano Resende 
>> > wrote:
>> >
>> >> I am planning to start looking at catching up with Spark releases. If
>> >> anyone has anything they want to add to the release, please speak up.
>> >>
>> >> --
>> >> Luciano Resende
>> >> http://twitter.com/lresende1975
>> >> http://lresende.blogspot.com/
>> >>
>> >
>> >
>> >
>> > --
>> > Luciano Resende
>> > http://twitter.com/lresende1975
>> > http://lresende.blogspot.com/
>> >
>>
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>
>

[jira] [Updated] (BAHIR-172) Avoid FileInputStream/FileOutputStream

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-172:
-
Description: 
They rely on finalizers (before Java 11), which create unnecessary GC load.

The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
this issue.

  was:They rely on finalizers (before Java 11), which create unnecessary GC 
load. The alternatives, {{Files.newInputStream}}, are as easy to use and don't 
have this issue.


> Avoid FileInputStream/FileOutputStream
> --
>
> Key: BAHIR-172
> URL: https://issues.apache.org/jira/browse/BAHIR-172
> Project: Bahir
>  Issue Type: Bug
>Reporter: Ted Yu
>Priority: Minor
>
> They rely on finalizers (before Java 11), which create unnecessary GC load.
> The alternatives, {{Files.newInputStream}}, are as easy to use and don't have 
> this issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-173) Upgrade bahir-flink to support Flink 1.5.1

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16547766#comment-16547766
 ] 

Ted Yu commented on BAHIR-173:
--

Please go ahead. 

Thanks

> Upgrade bahir-flink to support Flink 1.5.1
> --
>
> Key: BAHIR-173
> URL: https://issues.apache.org/jira/browse/BAHIR-173
> Project: Bahir
>  Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Major
>
> Flink 1.5.1 has been released.
> We should upgrade flink.version to 1.5.1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BAHIR-173) Upgrade bahir-flink to support Flink 1.5.1

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-173?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16543417#comment-16543417
 ] 

Ted Yu commented on BAHIR-173:
--

Currently I saw:
{code}
[ERROR] warning: [options] bootstrap class path not set in conjunction with 
-source 1.7
[ERROR] 
/bahir-flink/flink-connector-redis/src/test/java/org/apache/flink/streaming/connectors/redis/RedisITCaseBase.java:19:
 error: cannot find symbol
[ERROR] import 
org.apache.flink.streaming.util.StreamingMultipleProgramsTestBase;
[ERROR]   ^
[ERROR]   symbol:   class StreamingMultipleProgramsTestBase
[ERROR]   location: package org.apache.flink.streaming.util
[ERROR] 
/bahir-flink/flink-connector-redis/src/test/java/org/apache/flink/streaming/connectors/redis/RedisITCaseBase.java:28:
 error: cannot find symbol
[ERROR] public abstract class RedisITCaseBase extends 
StreamingMultipleProgramsTestBase {
[ERROR]   ^
[ERROR]   symbol: class StreamingMultipleProgramsTestBase
{code}

> Upgrade bahir-flink to support Flink 1.5.1
> --
>
> Key: BAHIR-173
> URL: https://issues.apache.org/jira/browse/BAHIR-173
> Project: Bahir
>  Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Major
>
> Flink 1.5.1 has been released.
> We should upgrade flink.version to 1.5.1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BAHIR-173) Upgrade bahir-flink to support Flink 1.5.1

[Ted Yu (JIRA)]

Ted Yu created BAHIR-173:


 Summary: Upgrade bahir-flink to support Flink 1.5.1
 Key: BAHIR-173
 URL: https://issues.apache.org/jira/browse/BAHIR-173
 Project: Bahir
  Issue Type: Improvement
Reporter: Ted Yu


Flink 1.5.1 has been released.

We should upgrade flink.version to 1.5.1



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BAHIR-172) Avoid FileInputStream/FileOutputStream

[Ted Yu (JIRA)]

Ted Yu created BAHIR-172:


 Summary: Avoid FileInputStream/FileOutputStream
 Key: BAHIR-172
 URL: https://issues.apache.org/jira/browse/BAHIR-172
 Project: Bahir
  Issue Type: Bug
Reporter: Ted Yu


They rely on finalizers (before Java 11), which create unnecessary GC load. The 
alternatives, {{Files.newInputStream}}, are as easy to use and don't have this 
issue.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Catching up on Spark Releases

[Ted Yu]

Spark 2.1.3 has been released:

http://search-hadoop.com/m/Spark/q3RTtDu0Z8V9VMZ?subj=+ANNOUNCE+Apache+Spark+2+1+3

Looking forward to the corresponding Bahir release.

On Fri, Jun 29, 2018 at 5:07 AM, Luciano Resende 
wrote:

> Just heads up that Spark is currently voting on Spark 2.1.3 and 2.2.2. As
> soon as they get approved I will start the correspondent Bahir releases.
>
> On Tue, May 29, 2018 at 9:48 PM Luciano Resende 
> wrote:
>
> > So, it looks like we can do a release based on Spark 2.1.2 and Spark
> 2.2.1.
> >
> > After that, we will need to migrate some of our extensions to
> > datasource v2 API and do a release based on Spark 2.3
> >
> > I will get started with the Bahir 2.1.2 and 2.2.1 releases.
> >
> > On Sun, Apr 22, 2018 at 3:24 PM, Luciano Resende 
> > wrote:
> >
> >> I am planning to start looking at catching up with Spark releases. If
> >> anyone has anything they want to add to the release, please speak up.
> >>
> >> --
> >> Luciano Resende
> >> http://twitter.com/lresende1975
> >> http://lresende.blogspot.com/
> >>
> >
> >
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.2.1 (RC1)

[Ted Yu]

+1

Ran test suite which passed.

On Thu, Jun 14, 2018 at 2:16 AM, Luciano Resende 
wrote:

> Here is my +1 for the release.
>
> On Wed, Jun 6, 2018 at 5:50 PM Luciano Resende 
> wrote:
>
> > Dear community member,
> >
> > Please vote to approve the release of Apache Bahir 2.2.1 (RC1) based on
> > Apache Spark 2.2.1.
> >
> > Note that there are minimal changes between the 2.1.2 and 2.2.1 releases
> >
> > https://github.com/apache/bahir/compare/v2.1.2-rc1...v2.2.1-rc1
> >
> > Tag: v2.2.1-rc1 (7f96456dcda38479bcdcbfcf4fe789c0f16ae196)
> >
> > https://github.com/apache/bahir/tree/v2.2.1-rc1
> >
> > Release files:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1022
> >
> > Source distribution:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.2.1-rc1/
> >
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> > least
> > 3 +1 PMC votes are cast.
> >
> >   [ ] +1 Release this package as Apache Bahir 2.2.1
> >   [ ] -1 Do not release this package because ...
> >
> >
> > Thanks for your vote!
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.1.2 (RC1)

[Ted Yu]

+1

Downloaded source and built.
Ran test suite which passed.

On Thu, May 31, 2018 at 10:13 PM, Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.1.2 (RC1) based on
> Apache Spark 2.1.2.
>
> Tag: v2.1.2-rc1 (6aa6d19582d86704861cd050d9763e29435ea27a)
>
> https://github.com/apache/bahir/tree/v2.1.2-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1021
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.1.2-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.1.2
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.1.2 (RC1)

[Ted Yu]

I tried to download the source but https://dist.apache.org/repos/
dist/dev/bahir/bahir-spark/2.1.2-rc1/ wasn't accessible using either Chrome
or Firefox.

FYI

On Thu, May 31, 2018 at 10:13 PM, Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.1.2 (RC1) based on
> Apache Spark 2.1.2.
>
> Tag: v2.1.2-rc1 (6aa6d19582d86704861cd050d9763e29435ea27a)
>
> https://github.com/apache/bahir/tree/v2.1.2-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1021
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.1.2-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.1.2
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: Catching up on Spark Releases

[Ted Yu]

bq. catching up with Spark releases

+1

Looking forward to the RC.

On Sun, Apr 22, 2018 at 3:24 PM, Luciano Resende 
wrote:

> I am planning to start looking at catching up with Spark releases. If
> anyone has anything they want to add to the release, please speak up.
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: New Flink connector

[Ted Yu]

+1 on Robert's proposal.

On Sat, Nov 18, 2017 at 6:21 AM, Robert Metzger  wrote:

> I'm really sorry that I didn't respond yet.
>
> Regarding the question "should the code be in the bahir-flink repo, or the
> kudu repo"
> I have the feeling that the kudu repo might actually the better spot,
> because flume, mapreduce, hive, spark, and others are already there. It
> seems that the Kudu project is accepting such contributions and is also
> willing to maintain them.
>
> If the Kudu project rejects the contribution for now, I would suggest to
> provide a script that builds the flink-kudu connector, starts kudu (maybe
> from a docker image?) and then runs a few test jobs.
>
>
> On Fri, Nov 3, 2017 at 11:45 AM, Nacho Garcia Fernandez <
> nachogarci...@gmail.com> wrote:
>
> > Can somebody out there please reply my last question?  Thanks in advance
> :D
> >
> > On 27 October 2017 at 14:23, Nacho Garcia Fernandez <
> > nachogarci...@gmail.com
> > > wrote:
> >
> > > Hello all.
> > >
> > > I'm a little bit stuck with one issue that I hope you can help me with.
> > >
> > > I'm developing a flink-connector-kudu extension that allows to read
> from
> > > Kudu and write to Flink and Kudu. This connector addresses the issue
> > > [BAHIR-99] and is a full re-implemnetation of
> https://github.com/apache/
> > > bahir-flink/pull/17.
> > >
> > > I'm struggle with testing: How is it supposed to be handled when the
> data
> > > storage (kudu) do not provide an embedded driver?
> > >
> > > In the case of Kudu, it does not provide any embedded java-based driver
> > > yet and I need a built Kudu to perform testing against it, otherwise I
> > > cannot test (e2e) this connector with a "real" data storage.
> > >
> > > Because of that I see three main possibilities for this scenario:
> > >
> > > * Create a Mock for Kudu classes (KuduSession, KuduTable, KuduClient,
> > etc).
> > >
> > > * Use MiniKuduCluster utility of Kudu to instantiate a local cluster:
> it
> > > is not possible due to the fact that this needs a real build of Kudu in
> > the
> > > local machine.
> > >
> > > * Update travis.yml to install a Kudu server: it would fix the problem
> > for
> > > CI, but tests would fail locally. Moreover, bulding Kudu takes so long
> > > (more than 20 minutes), which is not feasible for CI.
> > >
> > > * Ignore testing: not an option :)
> > >
> > >
> > > In the case of Kudu, I saw that other connectors for other distributed
> > > analytics platforms (i.e spark) are directly implemented in the Kudu
> > repo (
> > > https://github.com/apache/kudu/tree/master/java/kudu-spark) instead of
> > > using bahir-spark. I think this is good because when you execute the
> > tests
> > > you have a real build of Kudu to perform testing against it.
> > >
> > > What is the best place (kudu vs bahir) for this connector if we take
> into
> > > consideration the abovementioned issues?
> > >
> > > If the answer is bahir-flink, how should I proceed with my tests? :)
> > >
> > > Thanks in advance.
> > >
> > >
> >
>

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12

See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12
See SPARK-14220


This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [DISCUSS][bahir-flink] Drop Java 7 support

[Ted Yu]

+1 on dropping Java 7 support. 
 Original message From: Robert Metzger  
Date: 9/16/17  2:04 AM  (GMT-08:00) To: dev@bahir.apache.org Subject: 
[DISCUSS][bahir-flink] Drop Java 7 support 
Hi,

In the Flink InfluxDB Sink PR [1], there is a little discussion regarding
Java 7 support in bahir-flink.

I'm proposing to drop Java 7 support in bahir-flink. As of the upcoming
Flink 1.4 release, Flink will also drop Java 7 support.

Let me know if somebody has any concerns, otherwise, we'll probably drop
the Java7 travis checks in the mentioned PR.


Regards,
Robert


[1] https://github.com/apache/bahir-flink/pull/21

[jira] [Commented] (BAHIR-136) HBase Spark connector

[Ted Yu (JIRA)]

[ 
https://issues.apache.org/jira/browse/BAHIR-136?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16166884#comment-16166884
 ] 

Ted Yu commented on BAHIR-136:
--

This was created before recent exchanges on the mailing list.

Proposal withdrawn.

> HBase Spark connector
> -
>
> Key: BAHIR-136
> URL: https://issues.apache.org/jira/browse/BAHIR-136
> Project: Bahir
>  Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Critical
> Fix For: Not Applicable
>
>
> Here was related discussion thread:
> http://search-hadoop.com/m/HBase/YGbbPKkqe2dRz3I1?subj=Re+DISCUSS+is+hbase+spark+integration+going+to+make+it+into+HBase+2+0
> It seems consensus is to move hbase-spark connector out of hbase.
> This JIRA would incorporate the connector into Bahir.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Resolved] (BAHIR-136) HBase Spark connector

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-136?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved BAHIR-136.
--
   Resolution: Won't Fix
Fix Version/s: Not Applicable

> HBase Spark connector
> -
>
> Key: BAHIR-136
> URL: https://issues.apache.org/jira/browse/BAHIR-136
> Project: Bahir
>  Issue Type: Improvement
>Reporter: Ted Yu
>Priority: Critical
> Fix For: Not Applicable
>
>
> Here was related discussion thread:
> http://search-hadoop.com/m/HBase/YGbbPKkqe2dRz3I1?subj=Re+DISCUSS+is+hbase+spark+integration+going+to+make+it+into+HBase+2+0
> It seems consensus is to move hbase-spark connector out of hbase.
> This JIRA would incorporate the connector into Bahir.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Created] (BAHIR-136) HBase Spark connector

[Ted Yu (JIRA)]

Ted Yu created BAHIR-136:


 Summary: HBase Spark connector
 Key: BAHIR-136
 URL: https://issues.apache.org/jira/browse/BAHIR-136
 Project: Bahir
  Issue Type: Improvement
Reporter: Ted Yu
Priority: Critical


Here was related discussion thread:

http://search-hadoop.com/m/HBase/YGbbPKkqe2dRz3I1?subj=Re+DISCUSS+is+hbase+spark+integration+going+to+make+it+into+HBase+2+0

It seems consensus is to move hbase-spark connector out of hbase.

This JIRA would incorporate the connector into Bahir.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12
See SPARK-14220


This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Re: [VOTE] Apache Bahir 2.2.0 (RC1)

[Ted Yu]

+1

Checked signatures

Ran thru unit test suite which passed.

On Wed, Aug 16, 2017 at 10:17 PM, Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.2.0 (RC1) based on
> Apache Spark 2.2.0.
>
> Tag: v2.2.0-rc1 (d927289fdad0e66953606869639fe2f153c37b05)
>
> https://github.com/apache/bahir/tree/v2.2.0-rc1
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1020
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.2.0-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.2.0
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: Releasing Apache Bahir for Spark 2.2.0

[Ted Yu]

Has the RC come out ?

Cheers

On Tue, Aug 1, 2017 at 1:57 PM, Christian R Kadner 
wrote:

> Merged the last pending PR last night (except WIP), so good to go!
>
> Thanks Luciano.
>
> Best regards,
> Christian Kadner
>
> [image: Inactive hide details for Ted Yu ---07/27/2017 12:18:59
> AM---Looking forward to the RC.   Original message From]Ted
> Yu ---07/27/2017 12:18:59 AM---Looking forward to the RC.  
> Original message From: Jean-Baptiste Onofré 
> From: Ted Yu 
> To: dev@bahir.apache.org
> Date: 07/27/2017 12:18 AM
> Subject: Re: Releasing Apache Bahir for Spark 2.2.0
> --
>
>
> Looking forward to the RC.
>
>  Original message 
> From: Jean-Baptiste Onofré  Date: 7/27/17  1:48 AM
>  (GMT-05:00)
> To: dev@bahir.apache.org
> Subject: Re: Releasing Apache Bahir for Spark 2.2.0
>
>
> It sounds good to me. Thanks !
>
> Regards
> JB
>
> On 07/26/2017 07:47 PM, Luciano Resende wrote:
> > I am planning to create an RC with Spark 2.2.0 in the next couple days.
> Any
> > important jiras/pr we should wait before cutting the RC?
> >
>
> --
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com
>
>
>
>

Re: Releasing Apache Bahir for Spark 2.2.0

[Ted Yu]

Looking forward to the RC. 
 Original message From: Jean-Baptiste Onofré 
 Date: 7/27/17  1:48 AM  (GMT-05:00) To: 
dev@bahir.apache.org Subject: Re: Releasing Apache Bahir for Spark 2.2.0 
It sounds good to me. Thanks !

Regards
JB

On 07/26/2017 07:47 PM, Luciano Resende wrote:
> I am planning to create an RC with Spark 2.2.0 in the next couple days. Any
> important jiras/pr we should wait before cutting the RC?
> 

-- 
Jean-Baptiste Onofré
jbono...@apache.org
http://blog.nanthrax.net
Talend - http://www.talend.com

Re: failing build question BAHIR-100 PR

[Ted Yu]

Maybe we should upgrade Java 8 - I see 8.0_91 and latest is 8u131.

Other than that, someone with admin privilege can add the suggested ulimit
command at the beginning of the QA run.

Cheers

On Thu, Jul 6, 2017 at 11:16 PM, Rosenstark, David <
david.rosenst...@intel.com> wrote:

> http://169.45.79.58:8080/job/bahir_spark_pr_builder/69/console
>
> On 06/07/2017, 23:08, "Ted Yu"  wrote:
>
> Can you give the URL where the error was reported ?
>
> Thanks
>
> On Thu, Jul 6, 2017 at 1:05 PM, Rosenstark, David <
> david.rosenst...@intel.com> wrote:
>
> > Hi my build passes fine on my machine running oracle java 8.
> > But I see it failed on a different sub-project not affected by
> my PR
> > on Jenkins. But this is the output: (see below)
> > Is this a known intermittent issue? Is there some issue on build
> > machine? Any other hints?
> >
> > 12:48:40 - Send & Receive messages of size 128000 bytes.
> > 12:48:40 AkkaStreamSourceSuite:
> > 12:48:40 BasicAkkaSourceSuite:
> > 12:48:50 - basic usage
> > 12:49:00 - Send and receive 100 messages.
> > 12:49:00 - params not provided
> > 12:49:10 - Recovering offset from the last processed offset
> > 112:49:10 - Recovering offset from the last processed offset
> > 12:49:10 #
> > 12:49:10 # A fatal error has been detected by the Java Runtime
> > Environment:
> > 12:49:10 #
> > 12:49:10 #  SIGSEGV (0xb) at pc=0x7f3970105120, pid=24784,
> > tid=139896151623424
> > 12:49:10 #
> > 12:49:10 # JRE version: OpenJDK Runtime Environment (8.0_91-b14)
> > (build 1.8.0_91-b14)
> > 12:49:10 # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed
> mode
> > linux-amd64 compressed oops)
> > 12:49:10 # Problematic frame:
> > 12:49:10 # C  0x7f3970105120
> > 12:49:10 #
> > 12:49:10 # Failed to write core dump. Core dumps have been
> disabled.
> > To enable core dumping, try "ulimit -c unlimited" before starting
> Java again
> > 12:49:10 #
> > 12:49:10 # An error report file with more information is saved
> as:
> > 12:49:10 # /var/lib/jenkins/workspace/
> bahir_spark_pr_builder/sql-
> > streaming-akka/hs_err_pid24784.log
> > 12:49:10 #
> > 12:49:10 # If you would like to submit a bug report, please
> visit:
> > 12:49:10 #   http://bugreport.java.com/bugreport/crash.jsp
> > 12:49:10 # The crash happened outside the Java Virtual Machine in
> > native code.
> > 12:49:10 # See problematic frame for where to report the bug.
> >
> >
> >
> >
> >
> > 
> -
> > Intel Electronics Ltd.
> >
> > This e-mail and any attachments may contain confidential material for
> > the sole use of the intended recipient(s). Any review or distribution
> > by others is strictly prohibited. If you are not the intended
> > recipient, please contact the sender and delete all copies.
> >
>
>
> -
> Intel Electronics Ltd.
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>

Re: failing build question BAHIR-100 PR

[Ted Yu]

Can you give the URL where the error was reported ?

Thanks

On Thu, Jul 6, 2017 at 1:05 PM, Rosenstark, David <
david.rosenst...@intel.com> wrote:

> Hi my build passes fine on my machine running oracle java 8.
> But I see it failed on a different sub-project not affected by my PR
> on Jenkins. But this is the output: (see below)
> Is this a known intermittent issue? Is there some issue on build
> machine? Any other hints?
>
> 12:48:40 - Send & Receive messages of size 128000 bytes.
> 12:48:40 AkkaStreamSourceSuite:
> 12:48:40 BasicAkkaSourceSuite:
> 12:48:50 - basic usage
> 12:49:00 - Send and receive 100 messages.
> 12:49:00 - params not provided
> 12:49:10 - Recovering offset from the last processed offset
> 112:49:10 - Recovering offset from the last processed offset
> 12:49:10 #
> 12:49:10 # A fatal error has been detected by the Java Runtime
> Environment:
> 12:49:10 #
> 12:49:10 #  SIGSEGV (0xb) at pc=0x7f3970105120, pid=24784,
> tid=139896151623424
> 12:49:10 #
> 12:49:10 # JRE version: OpenJDK Runtime Environment (8.0_91-b14)
> (build 1.8.0_91-b14)
> 12:49:10 # Java VM: OpenJDK 64-Bit Server VM (25.91-b14 mixed mode
> linux-amd64 compressed oops)
> 12:49:10 # Problematic frame:
> 12:49:10 # C  0x7f3970105120
> 12:49:10 #
> 12:49:10 # Failed to write core dump. Core dumps have been disabled.
> To enable core dumping, try "ulimit -c unlimited" before starting Java again
> 12:49:10 #
> 12:49:10 # An error report file with more information is saved as:
> 12:49:10 # /var/lib/jenkins/workspace/bahir_spark_pr_builder/sql-
> streaming-akka/hs_err_pid24784.log
> 12:49:10 #
> 12:49:10 # If you would like to submit a bug report, please visit:
> 12:49:10 #   http://bugreport.java.com/bugreport/crash.jsp
> 12:49:10 # The crash happened outside the Java Virtual Machine in
> native code.
> 12:49:10 # See problematic frame for where to report the bug.
>
>
>
>
>
> -
> Intel Electronics Ltd.
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>

Re: [VOTE] Apache Bahir 2.1.1 (RC2)

[Ted Yu]

I re-ran the test suite and the suite passed.

+1 for RC2.

On Wed, Jun 14, 2017 at 6:36 PM, Ted Yu  wrote:

> When I ran test suite (using jdk 8), I got:
>
> BasicAkkaSourceSuite:
> - basic usage
> - Send and receive 100 messages. *** FAILED ***
>   1 did not equal 100 (AkkaStreamSourceSuite.scala:116)
>
> Has anyone else seen similar test failure ?
>
> Cheers
>
> On Wed, Jun 7, 2017 at 11:24 PM, Luciano Resende 
> wrote:
>
>> Dear community member,
>>
>> Please vote to approve the release of Apache Bahir 2.1.1 (RC2) based on
>> Apache Spark 2.1.1.
>>
>> New Extensions:
>> * CounchDB/Cloudant
>> * Google Pub/Sub
>>
>> Tag: v2.1.1-rc2 (26bdee0bacb9c922abda09314036e9df0e97d88f)
>>
>> https://github.com/apache/bahir/tree/v2.1.1-rc2
>>
>> Release files:
>>
>> https://repository.apache.org/content/repositories/orgapachebahir-1019
>>
>> Source distribution:
>>
>> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.1.1-rc2/
>> <https://dist.apache.org/repos/dist/dev/bahir/2.1.0-rc1/>
>>
>>
>> The vote is open for at least 72 hours and passes if a majority of at
>> least
>> 3 +1 PMC votes are cast.
>>
>>   [ ] +1 Release this package as Apache Bahir 2.1.1
>>   [ ] -1 Do not release this package because ...
>>
>>
>> Thanks for your vote!
>>
>> --
>> Luciano Resende
>> http://twitter.com/lresende1975
>> http://lresende.blogspot.com/
>>
>
>

Re: [VOTE] Apache Bahir 2.1.1 (RC2)

[Ted Yu]

When I ran test suite (using jdk 8), I got:

BasicAkkaSourceSuite:
- basic usage
- Send and receive 100 messages. *** FAILED ***
  1 did not equal 100 (AkkaStreamSourceSuite.scala:116)

Has anyone else seen similar test failure ?

Cheers

On Wed, Jun 7, 2017 at 11:24 PM, Luciano Resende 
wrote:

> Dear community member,
>
> Please vote to approve the release of Apache Bahir 2.1.1 (RC2) based on
> Apache Spark 2.1.1.
>
> New Extensions:
> * CounchDB/Cloudant
> * Google Pub/Sub
>
> Tag: v2.1.1-rc2 (26bdee0bacb9c922abda09314036e9df0e97d88f)
>
> https://github.com/apache/bahir/tree/v2.1.1-rc2
>
> Release files:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1019
>
> Source distribution:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-spark/2.1.1-rc2/
> 
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.1.1
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for your vote!
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir Flink extensions 1.0 (RC5)

[Ted Yu]

I suggest getting BAHIR-88
 fixed
- this may affect future release(s).

On Tue, May 23, 2017 at 12:29 PM, Luciano Resende 
wrote:

> The issue reported was for Spark extensions
> https://issues.apache.org/jira/browse/BAHIR-88
>
>
> And I guess it is also applicable for Flink extensions
>
>
> But the issue is still open.
>
>
> On Tue, May 23, 2017 at 12:08 PM, Mridul Muralidharan <
> mridul...@apache.org>
> wrote:
>
> > Hi Luciano,
> >
> >   I would not consider them blocking - but I wanted clarity about
> > them, since I had seen jira's to the effect that they were fixed ?
> > Not sure if I am misremembering !
> >
> > Regards,
> > Mridul
> >
> >
> > On Tue, May 23, 2017 at 12:01 PM, Luciano Resende 
> > wrote:
> > > On Sun, May 21, 2017 at 11:50 PM, Mridul Muralidharan <
> > mridul...@apache.org>
> > > wrote:
> > >
> > >> Hi,
> > >>
> > >>   From the release tag, clean build and tests work fine, signatures
> > match.
> > >>
> > >> The source at https://dist.apache.org/repos/
> > dist/dev/bahir/bahir-flink/1.
> > >> 0-rc5/
> > >> has additional files which are not in repo:
> > >>
> > >> Not in repo ./distribution/pom.xml.releaseBackup
> > >> Not in repo ./flink-connector-activemq/pom.xml.releaseBackup
> > >> Not in repo ./flink-connector-akka/pom.xml.releaseBackup
> > >> Not in repo ./flink-connector-flume/pom.xml.releaseBackup
> > >> Not in repo ./flink-connector-netty/pom.xml.releaseBackup
> > >> Not in repo ./flink-connector-redis/pom.xml.releaseBackup
> > >> Not in repo ./pom.xml.releaseBackup
> > >> Not in repo ./release.properties
> > >>
> > >> I thought this was fixed, but I could be remembering wrong ?
> > >>
> > >>
> > >>
> > > That's what I thought as well, but it might have been for the Spark
> > > release.
> > >
> > > Are you  considering this blocking ? Or would it be ok to continue with
> > the
> > > approval of the release ?
> > >
> > > --
> > > Luciano Resende
> > > http://twitter.com/lresende1975
> > > http://lresende.blogspot.com/
> >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir Flink extensions 1.0 (RC5)

[Ted Yu]

+1

Ran test suite - passed

On Wed, May 17, 2017 at 1:22 PM, Jean-Baptiste Onofré 
wrote:

> +1 (binding)
>
> Regards
> JB
>
> On May 17, 2017, 16:12, at 16:12, Luciano Resende 
> wrote:
> >Please vote to approve the release of the following candidate as Apache
> >Bahir Flink extensions 1.0 based on Apache Flink 1.2.0.
> >
> >The tag to be voted on is v1.0-rc4
> >(9350cb5f2871df2b2e03f33c5fa1239ca044505a)
> >
> >https://github.com/apache/
> >bahir-flink/commit/9350cb5f2871df2b2e03f33c5fa1239ca044505a
> >
> >The release files, including signatures, digests, etc. can be found at:
> >
> >https://repository.apache.org/content/repositories/orgapachebahir-1017/
> >
> >The source distribution is also available at:
> >
> >https://dist.apache.org/repos/dist/dev/bahir/bahir-flink/1.0-rc5/
> >
> >The vote is open for at least 72 hours and passes if a majority of at
> >least
> >3 +1 PMC votes are cast.
> >
> >[ ] +1 Release this package as Apache Bahir Flink extensions 1.0
> >[ ] -1 Do not release this package because ...
> >
> >--
> >Luciano Resende
> >http://twitter.com/lresende1975
> >http://lresende.blogspot.com/
>

Re: Bahir Flink release

[Ted Yu]

Yes.

Ready for testing.

On Wed, May 3, 2017 at 8:13 AM, Luciano Resende 
wrote:

> I see all PRs related to issues found during the latest RC review are
> merged.
>
> Are we ready for a new RC ?
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12
See SPARK-14220


This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu updated BAHIR-107:
-
Description: 
Spark has started effort for accommodating Scala 2.12
See SPARK-14220


This JIRA is to track requirements for building Bahir on Scala 2.12

  was:
Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12


> Build and test Bahir against Scala 2.12
> ---
>
> Key: BAHIR-107
> URL: https://issues.apache.org/jira/browse/BAHIR-107
> Project: Bahir
>  Issue Type: Improvement
>    Reporter: Ted Yu
>
> Spark has started effort for accommodating Scala 2.12
> See SPARK-14220
> This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [VOTE] Apache Bahir Flink extensions 1.0 (RC3)

[Ted Yu]

+1

Checked signature
Ran "mvn package" which passed

On Thu, Apr 6, 2017 at 4:08 PM, Luciano Resende 
wrote:

> Please vote to approve the release of the following candidate as Apache
> Bahir Flink extensions 1.0 based on Apache Flink 1.2.0.
>
> The tag to be voted on is v1.0-rc3
> (447c2158be7e3c6efb12324b0efc627d276dc828)
>
> https://github.com/apache/bahir-flink/commit/
> 447c2158be7e3c6efb12324b0efc627d276dc828
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1015
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/bahir-flink/1.0-rc3/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir Flink extensions 1.0
> [ ] -1 Do not release this package because ...
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

[jira] [Created] (BAHIR-107) Build and test Bahir against Scala 2.12

[Ted Yu (JIRA)]

Ted Yu created BAHIR-107:


 Summary: Build and test Bahir against Scala 2.12
 Key: BAHIR-107
 URL: https://issues.apache.org/jira/browse/BAHIR-107
 Project: Bahir
  Issue Type: Improvement
Reporter: Ted Yu


Spark has started effort for accommodating Scala 2.12
See SPARK-14220

This JIRA is to track requirements for building Bahir on Scala 2.12



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Flink extensions artifact id

[Ted Yu]

+1

On Thu, Apr 6, 2017 at 11:39 AM, Luciano Resende 
wrote:

> I was looking at some of the flink extensions, which are implemented in
> Java (except akka one), but have the _2.11 as if it was a Scala project.
>
> Should I just remove the _2.11 and leave them with a normal java package
> artifact id ?
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: Java library for Flink-Kudu integration

[Ted Yu]

Ruben:
If you have bandwidth, you can create a Bahir JIRA and port your code over
to:

https://github.com/apache/bahir-flink/

Cheers

On Mon, Mar 27, 2017 at 10:31 AM, Ted Yu  wrote:

> Forwarding to Bahir dev mailing list.
>
> -- Forwarded message --
> From: 
> Date: Mon, Mar 27, 2017 at 9:14 AM
> Subject: Java library for Flink-Kudu integration
> To: u...@flink.apache.org
>
>
> Hi all,
>
>
>
> I apologize for sending the email to both user and dev accounts, but not
> sure where this topic fits better.
>
>
>
> In my team, we have been working in some PoCs and PoVs about new data
> architectures. As part of this work, we have implemented a library to
> connect Kudu and Flink. The library allows reading/writing from/to Kudu
> tablets using DataSet API and also writing to Kudu using DataStream API.
>
>
>
> You can find the code and documentation (including some examples) in
> https://github.com/rubencasado/Flink-Kudu
>
>
>
> Any comment/suggestion/contribution is very welcomed J
>
>
>
> We will try to publish this contribution to the Apache Bahir project.
>
>
>
> Best
>
>
>
> 
>
> *Rubén Casado Tejedor, PhD*
>
> *>* *accenture* *digital*
>
> Big Data Manager
>
> ' + 34 629 009 429 <+34%20629%2000%2094%2029>
>
> * ruben.casado.teje...@accenture.com
>
> --
>
> This message is for the designated recipient only and may contain
> privileged, proprietary, or otherwise confidential information. If you have
> received it in error, please notify the sender immediately and delete the
> original. Any other use of the e-mail by you is prohibited. Where allowed
> by local law, electronic communications with Accenture and its affiliates,
> including e-mail and instant messaging (including content), may be scanned
> by our systems for the purposes of information security and assessment of
> internal compliance with Accenture policy.
> 
> __
>
> www.accenture.com
>
>

Fwd: Java library for Flink-Kudu integration

[Ted Yu]

Forwarding to Bahir dev mailing list.

-- Forwarded message --
From: 
Date: Mon, Mar 27, 2017 at 9:14 AM
Subject: Java library for Flink-Kudu integration
To: u...@flink.apache.org


Hi all,



I apologize for sending the email to both user and dev accounts, but not
sure where this topic fits better.



In my team, we have been working in some PoCs and PoVs about new data
architectures. As part of this work, we have implemented a library to
connect Kudu and Flink. The library allows reading/writing from/to Kudu
tablets using DataSet API and also writing to Kudu using DataStream API.



You can find the code and documentation (including some examples) in
https://github.com/rubencasado/Flink-Kudu



Any comment/suggestion/contribution is very welcomed J



We will try to publish this contribution to the Apache Bahir project.



Best





*Rubén Casado Tejedor, PhD*

*>* *accenture* *digital*

Big Data Manager

' + 34 629 009 429 <+34%20629%2000%2094%2029>

* ruben.casado.teje...@accenture.com

--

This message is for the designated recipient only and may contain
privileged, proprietary, or otherwise confidential information. If you have
received it in error, please notify the sender immediately and delete the
original. Any other use of the e-mail by you is prohibited. Where allowed
by local law, electronic communications with Accenture and its affiliates,
including e-mail and instant messaging (including content), may be scanned
by our systems for the purposes of information security and assessment of
internal compliance with Accenture policy.

__

www.accenture.com

Re: [ANNOUNCE] Apache Bahir 2.1.0 Released

[Ted Yu]

Flink connectors are maintained in the following repo:

https://github.com/apache/bahir-flink/

On Wed, Feb 22, 2017 at 6:12 PM, sebb AT ASF  wrote:

> On 22 February 2017 at 23:36, Christian Kadner  wrote:
> > Hi Sebastian,
> >
> > thanks for your feedback. We will provide more details in future
> > announcement emails.
> >
> > My intention was to avoid superfluous text, since Apache Spark is
> > a well known project and the Apache Bahir extensions to Spark are
> > only of interest to people who are already familiar with Spark.
>
> It does not need much, just a sentence or two.
> For example the ones in the section "What is Apache Bahir" on the website.
>
> > The Bahir extensions for Apache Flink are released separately.
>
> I could not find any download links ...
>
> > Regards,
> > Christian
> >
> > On 22 February 2017 at 07:28:43 AM, Sebastian Bazley wrote:
> >>
> >> What is the project about? Why should I be interested in it?
> >> [rhetorical questions]
> >>
> >>
> >> The Announce emails are sent to people not on the developer or user
> lists.
> >> Most will have no idea what the project is about.
> >>
> >> So the e-mails should contain at least brief details of what the
> >> product does, and some info on why the new release might be of
> >> interest to them.
> >>
> >> Although the mail mentions extensions to Apache Spark, it does not say
> >> what Spark does.
> >>
> >> Also, the Bahir home page mentions Apache Flink, but there do not seem
> >> to be any extensions for Flink.
> >>
> >> If Bahir supports both, I would expect the release to contain both.
> >>
> >> Readers should not have to click the link to find out the basic
> >> information
> >> (although of course it is useful to have such links for further detail).
> >>
> >> Please can you add that information to future announce mails?
> >>
> >> Thanks.
> >>
> >>
> >> On 22 February 2017 at 12:07, Christian Kadner wrote:
> >> > The Apache Bahir community is pleased to announce the release
> >> > of Apache Bahir 2.1.0 which provides the following extensions for
> >> > Apache Spark 2.1.0:
> >> >
> >> >- Akka Streaming
> >> >- MQTT Streaming
> >> >- MQTT Structured Streaming
> >> >- Twitter Streaming
> >> >- ZeroMQ Streaming
> >> >
> >> > For more information about Apache Bahir and to download the
> >> > latest release go to:
> >> >
> >> > http://bahir.apache.org
> >> >
> >> > The Apache Bahir streaming connectors are also available at:
> >> >
> >> > https://spark-packages.org/?q=bahir
> >> >
> >> > ---
> >> > Best regards,
> >> > Christian Kadner
> >>
> >>
>

Re: [ANNOUNCE] Apache Bahir 2.1.0 Released

[Ted Yu]

When would the Maven artifacts be available ?

Thanks 

> On Feb 22, 2017, at 4:07 AM, Christian Kadner  wrote:
> 
> The Apache Bahir community is pleased to announce the release
> of Apache Bahir 2.1.0 which provides the following extensions for
> Apache Spark 2.1.0:
> 
>   - Akka Streaming
>   - MQTT Streaming
>   - MQTT Structured Streaming
>   - Twitter Streaming
>   - ZeroMQ Streaming
> 
> For more information about Apache Bahir and to download the
> latest release go to:
> 
>http://bahir.apache.org
> 
> The Apache Bahir streaming connectors are also available at:
> 
>https://spark-packages.org/?q=bahir
> 
> ---
> Best regards,
> Christian Kadner

Re: [VOTE] Apache Bahir 2.1.0 (RC1)

[Ted Yu]

+1

Checked signature.
Ran test suite which passed:

[INFO] Apache Bahir - Spark SQL Streaming MQTT  SUCCESS [
46.165 s]
[INFO] Apache Bahir - Spark Streaming Twitter . SUCCESS [
25.158 s]
[INFO] Apache Bahir - Spark Streaming ZeroMQ .. SUCCESS [
21.848 s]
[INFO]

[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 02:46 min
[INFO] Finished at: 2017-02-04T12:13:35+00:00
[INFO] Final Memory: 143M/8688M

On Sat, Feb 4, 2017 at 2:21 AM, Christian Kadner  wrote:

> Dear community member,
>
> please vote to approve the release of Apache Bahir 2.1.0 (RC1) based on
> Apache Spark 2.1.0.
>
> Changes:
>
>   https://github.com/apache/bahir/compare/v2.0.2-rc3...v2.1.0-rc1
>
> Tag: v2.1.0-rc1 (e61cc3d)
>
>   https://github.com/apache/bahir/tree/v2.1.0-rc1
>
> Release files:
>
>   https://repository.apache.org/content/repositories/orgapachebahir-1012
>
> Source distribution:
>
>   https://dist.apache.org/repos/dist/dev/bahir/2.1.0-rc1/
>
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
>   [ ] +1 Release this package as Apache Bahir 2.1.0
>   [ ] -1 Do not release this package because ...
>
>
> Thanks for you vote!
>

Re: Release 2.1.0

[Ted Yu]

Don't work too hard, Christian :-)

On Fri, Feb 3, 2017 at 2:29 PM, Christian Kadner  wrote:

> +1
>
> I can prepare the release over the weekend and start a new vote
>
> Best regards,
> Christian Kadner
>
> >
> >
> --------
> -
> > From:Ted Yu 
> > To:  dev@bahir.apache.org
> > Date:02/03/2017 10:20 AM
> > Subject: Re: Release 2.1.0
> >
> 
> -
> >
> > +1
> >
> > On Fri, Feb 3, 2017 at 8:48 AM, Luciano Resende 
> > wrote:
> >
> > > Now that we are done with the Bahir release to support Spark 2.0.2,
> should
> > > we move to the new release supporting 2.1.0 ?
> > >
> > > --
> > > Luciano Resende
> > > http://twitter.com/lresende1975
> > > http://lresende.blogspot.com/
> > >
> >
>

Re: Release 2.1.0

[Ted Yu]

+1

On Fri, Feb 3, 2017 at 8:48 AM, Luciano Resende 
wrote:

> Now that we are done with the Bahir release to support Spark 2.0.2, should
> we move to the new release supporting 2.1.0 ?
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.0.2 (RC3)

[Ted Yu]

Thanks, Christian

On Fri, Jan 27, 2017 at 3:22 PM, Christian R Kadner 
wrote:

> We closed the vote and published the release artifacts to:
> https://dist.apache.org/repos/dist/release/bahir/2.0.2/
>
> We will send an announcement after verifying that the artifacts got
> replicated
> to the Apache download mirrors and to the Maven repositories.
>
> Regards,
> Christian
>
> [image: Inactive hide details for Ted Yu ---01/27/2017 02:29:11 AM---Looks
> like the vote passed. When would the artifacts be ready in]Ted Yu
> ---01/27/2017 02:29:11 AM---Looks like the vote passed. When would the
> artifacts be ready in maven ?
>
> From: Ted Yu 
> To: dev@bahir.apache.org
> Date: 01/27/2017 02:29 AM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC3)
> --
>
>
>
> Looks like the vote passed.
>
> When would the artifacts be ready in maven ?
>
> Thanks
>
> > On Jan 23, 2017, at 12:56 PM, Christian Kadner 
> wrote:
> >
> > Of course, my own (unbinding) vote:
> >
> > +1
> >
> > Thanks,
> > Christian
> >
> >> On 01/21/2017 03:31 AM, Christian Kadner wrote:
> >> Please vote to approve the third release candidate for Apache Bahir
> >> version 2.0.2 based on Apache Spark 2.0.2.
> >>
> >> The difference between 2.0.1 and 2.0.2 RC3 can be visualized at:
> >>
> >> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc3
> >>
> >> The tag to be voted on is v2.0.2-rc3 (477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4)
> >>
> >> https://github.com/apache/bahir/tree/477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4
> >>
> >> The release files, including signatures, digests, etc. can be found at:
> >>
> >> https://repository.apache.org/content/repositories/orgapachebahir-1011
> >>
> >> The source distribution is also available at:
> >>
> >> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc3/
> >>
> >> The vote is open for at least 72 hours and passes if a majority of at
> least
> >> 3 +1 PMC votes are cast.
> >>
> >> [ ] +1 Release this package as Apache Bahir 2.0.2
> >> [ ] -1 Do not release this package because ...
> >>
> >> Thanks for you vote!
> >
> >> On Mon, Jan 23, 2017 at 12:54 PM, Christian Kadner 
> wrote:
> >>> On Fri, Jan 20, 2017 at 6:31 PM, Christian Kadner 
> wrote:
> >>> Please vote to approve the third release candidate for Apache Bahir
> >>> version 2.0.2 based on Apache Spark 2.0.2.
> >>>
> >>> The difference between 2.0.1 and 2.0.2 RC3 can be visualized at:
> >>>
> >>> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc3
> >>>
> >>> The tag to be voted on is v2.0.2-rc3 (477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4)
> >>>
> >>> https://github.com/apache/bahir/tree/477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4
> >>>
> >>> The release files, including signatures, digests, etc. can be found at:
> >>>
> >>> https://repository.apache.org/content/repositories/orgapachebahir-1011
> >>>
> >>> The source distribution is also available at:
> >>>
> >>> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc3/
> >>>
> >>> The vote is open for at least 72 hours and passes if a majority of at
> least
> >>> 3 +1 PMC votes are cast.
> >>>
> >>> [ ] +1 Release this package as Apache Bahir 2.0.2
> >>> [ ] -1 Do not release this package because ...
> >>>
> >>> Thanks for you vote!
>
>
>
>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC3)

[Ted Yu]

Looks like the vote passed. 

When would the artifacts be ready in maven ?

Thanks

> On Jan 23, 2017, at 12:56 PM, Christian Kadner  wrote:
> 
> Of course, my own (unbinding) vote:
> 
> +1
> 
> Thanks,
> Christian
> 
>> On 01/21/2017 03:31 AM, Christian Kadner wrote:
>> Please vote to approve the third release candidate for Apache Bahir
>> version 2.0.2 based on Apache Spark 2.0.2.
>> 
>> The difference between 2.0.1 and 2.0.2 RC3 can be visualized at:
>> 
>> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc3
>> 
>> The tag to be voted on is v2.0.2-rc3 
>> (477ca1b556d2486b8c9ddbfb7a327ed8245066c4)
>> 
>> https://github.com/apache/bahir/tree/477ca1b556d2486b8c9ddbfb7a327ed8245066c4
>> 
>> The release files, including signatures, digests, etc. can be found at:
>> 
>> https://repository.apache.org/content/repositories/orgapachebahir-1011
>> 
>> The source distribution is also available at:
>> 
>> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc3/
>> 
>> The vote is open for at least 72 hours and passes if a majority of at least
>> 3 +1 PMC votes are cast.
>> 
>> [ ] +1 Release this package as Apache Bahir 2.0.2
>> [ ] -1 Do not release this package because ...
>> 
>> Thanks for you vote!
> 
>> On Mon, Jan 23, 2017 at 12:54 PM, Christian Kadner  
>> wrote:
>>> On Fri, Jan 20, 2017 at 6:31 PM, Christian Kadner  
>>> wrote:
>>> Please vote to approve the third release candidate for Apache Bahir
>>> version 2.0.2 based on Apache Spark 2.0.2.
>>> 
>>> The difference between 2.0.1 and 2.0.2 RC3 can be visualized at:
>>> 
>>> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc3
>>> 
>>> The tag to be voted on is v2.0.2-rc3 
>>> (477ca1b556d2486b8c9ddbfb7a327ed8245066c4)
>>> 
>>> https://github.com/apache/bahir/tree/477ca1b556d2486b8c9ddbfb7a327ed8245066c4
>>> 
>>> The release files, including signatures, digests, etc. can be found at:
>>> 
>>> https://repository.apache.org/content/repositories/orgapachebahir-1011
>>> 
>>> The source distribution is also available at:
>>> 
>>> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc3/
>>> 
>>> The vote is open for at least 72 hours and passes if a majority of at least
>>> 3 +1 PMC votes are cast.
>>> 
>>> [ ] +1 Release this package as Apache Bahir 2.0.2
>>> [ ] -1 Do not release this package because ...
>>> 
>>> Thanks for you vote!

Re: [VOTE] Apache Bahir 2.0.2 (RC3)

[Ted Yu]

+1

Checked signature
Ran test suite which passed.

[INFO] Apache Bahir - Spark SQL Streaming MQTT  SUCCESS [
43.237 s]
[INFO] Apache Bahir - Spark Streaming Twitter . SUCCESS [
25.169 s]
[INFO] Apache Bahir - Spark Streaming ZeroMQ .. SUCCESS [
23.989 s]
[INFO]

[INFO] BUILD SUCCESS
[INFO]

[INFO] Total time: 02:39 min
[INFO] Finished at: 2017-01-21T02:45:11+00:00

On Fri, Jan 20, 2017 at 6:31 PM, Christian Kadner 
wrote:

> Please vote to approve the third release candidate for Apache Bahir
> version 2.0.2 based on Apache Spark 2.0.2.
>
> The difference between 2.0.1 and 2.0.2 RC3 can be visualized at:
>
> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc3
>
> The tag to be voted on is v2.0.2-rc3 (477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4)
>
> https://github.com/apache/bahir/tree/477ca1b556d2486b8c9ddbfb7a327e
> d8245066c4
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1011
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc3/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.2
> [ ] -1 Do not release this package because ...
>
> Thanks for you vote!
>

Re: How to deal with sporadically failing test cases in sql-streaming-mqtt BasicMQTTSourceSuite

[Ted Yu]

I think marking these two tests as ignored is fine. 

Cheers

> On Jan 16, 2017, at 9:52 PM, Christian Kadner  wrote:
> 
> The latest two votes for an Apache Bahir 2.0.2 release candidate have
> been side-tracked by failing test cases in sql-streaming-mqtt
> BasicMQTTSourceSuite. The two test cases in question appear to fail
> sporadically for some of us, but consistently fail on under-powered
> machines or on systems under stress.
> 
> Should we mark the two test cases as ignored to guarantee smooth build
> verification for release candidates or ask PMC members who vote on
> release candidates to run build verification on more powerful
> machines?
> 
> Any other recommendations are welcome!  :-)
> 
> Thanks,
> Christian

Re: [VOTE] Apache Bahir 2.0.2 (RC2)

[Ted Yu]

A few more rounds of running the test (based on suggestion from Christian)
but it didn't pass.

I think this is not a blocker for release since other people cannot
reproduce it.

On Thu, Jan 12, 2017 at 1:55 AM, Ted Yu  wrote:

> Running with PR #33, I got:
>
> ^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
> ^[[31m  Buffer("MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
> "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
> queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
> message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
> had size 66 instead of expected size 100 (MQTTStreamSourceSuite.scala:
> 119)^[[0m
>
> Command line:
>
> mvn package
>
> On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner 
> wrote:
>
>> Hi Ted,
>>
>> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
>> and receive 100 messages" failed. Could this be an intermittent problem?
>> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box without
>> any test failures.
>>
>> I opened BAHIR-84 and created PR #33 to suppress the superfluous
>> "...INFO: org.apache.parquet.hadoop..." messages (
>> https://github.com/apache/bahir/pull/33). Could you take a look at that
>> PR and maybe give it a spin to see if that helps.
>>
>> To further reduce logging resources, you could set the root logger
>> threshold to ERROR in sql-streaming-mqtt/src/test/re
>> sources/log4j.properties.
>>
>> Thank you!
>> ~ Christian
>>
>> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
>> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
>> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>>
>> From: Ted Yu 
>> To: dev@bahir.apache.org
>> Date: 01/11/2017 08:24 AM
>> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
>> --
>>
>>
>>
>> ran test suite and saw:
>>
>> http://pastebin.com/xRJaCa6H
>>
>> On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner 
>> wrote:
>>
>> > Please vote to approve the second release candidate for Apache Bahir
>> > version 2.0.2 based on Apache Spark 2.0.2.
>> >
>> > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
>> >
>> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
>> >
>> > The tag to be voted on is v2.0.2-rc2
>> > (dd82363878a6bb4a19aade8010c01ddfbf254aff)
>> >
>> > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
>> > dfbf254aff
>> >
>> > The release files, including signatures, digests, etc. can be found at:
>> >
>> > https://repository.apache.org/content/repositories/orgapachebahir-1008
>> >
>> > The source distribution is also available at:
>> >
>> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
>> >
>> > The vote is open for at least 72 hours and passes if a majority of at
>> least
>> > 3 +1 PMC votes are cast.
>> >
>> > [ ] +1 Release this package as Apache Bahir 2.0.1
>> > [ ] -1 Do not release this package because ...
>> >
>> > Thanks for you vote!
>> >
>>
>>
>>
>>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC2)

[Ted Yu]

Running with PR #33, I got:

^[[31m- Send and receive 100 messages. *** FAILED ***^[[0m
^[[31m  Buffer("MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.")
had size 66 instead of expected size 100
(MQTTStreamSourceSuite.scala:119)^[[0m

Command line:

mvn package

On Thu, Jan 12, 2017 at 1:43 AM, Christian R Kadner 
wrote:

> Hi Ted,
>
> your pastebin (http://pastebin.com/xRJaCa6H) shows that test case "Send
> and receive 100 messages" failed. Could this be an intermittent problem?
> I did several runs on my MacBook Pro and on a Ubuntu 16.4 test box without
> any test failures.
>
> I opened BAHIR-84 and created PR #33 to suppress the superfluous "...INFO:
> org.apache.parquet.hadoop..." messages (https://github.com/apache/
> bahir/pull/33). Could you take a look at that PR and maybe give it a spin
> to see if that helps.
>
> To further reduce logging resources, you could set the root logger
> threshold to ERROR in sql-streaming-mqtt/src/test/
> resources/log4j.properties.
>
> Thank you!
> ~ Christian
>
> [image: Inactive hide details for Ted Yu ---01/11/2017 08:24:52 AM---ran
> test suite and saw: http://pastebin.com/xRJaCa6H]Ted Yu ---01/11/2017
> 08:24:52 AM---ran test suite and saw: http://pastebin.com/xRJaCa6H
>
> From: Ted Yu 
> To: dev@bahir.apache.org
> Date: 01/11/2017 08:24 AM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC2)
> --
>
>
>
> ran test suite and saw:
>
> http://pastebin.com/xRJaCa6H
>
> On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner 
> wrote:
>
> > Please vote to approve the second release candidate for Apache Bahir
> > version 2.0.2 based on Apache Spark 2.0.2.
> >
> > The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
> >
> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
> >
> > The tag to be voted on is v2.0.2-rc2
> > (dd82363878a6bb4a19aade8010c01ddfbf254aff)
> >
> > https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
> > dfbf254aff
> >
> > The release files, including signatures, digests, etc. can be found at:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1008
> >
> > The source distribution is also available at:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Bahir 2.0.1
> > [ ] -1 Do not release this package because ...
> >
> > Thanks for you vote!
> >
>
>
>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC2)

[Ted Yu]

That's one way of making the test stable.

On Wed, Jan 11, 2017 at 10:27 AM, Christian R Kadner 
wrote:

> Yup, this is a noisy test suite. Are you advocating to limit the log
> output?
>
>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC2)

[Ted Yu]

ran test suite and saw:

http://pastebin.com/xRJaCa6H

On Wed, Jan 11, 2017 at 3:33 AM, Christian Kadner 
wrote:

> Please vote to approve the second release candidate for Apache Bahir
> version 2.0.2 based on Apache Spark 2.0.2.
>
> The difference between 2.0.1 and 2.0.2 RC2 can be visualized at:
>
> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc2
>
> The tag to be voted on is v2.0.2-rc2
> (dd82363878a6bb4a19aade8010c01ddfbf254aff)
>
> https://github.com/apache/bahir/tree/dd82363878a6bb4a19aade8010c01d
> dfbf254aff
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1008
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc2/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.1
> [ ] -1 Do not release this package because ...
>
> Thanks for you vote!
>

Re: [VOTE] Apache Bahir 2.0.2 (RC1)

[Ted Yu]

Sounds good.

Looking forward to the next RC.

On Tue, Jan 10, 2017 at 11:14 AM, Christian R Kadner 
wrote:

> Hi Ted,
>
> we decided to ignore the test that failed sporadically when we tested the
> RC in December (BAHIR-83).
>
> So we should be good to give it a second attempt.
>
> Best regards,
> Christian Kadner
>
> [image: Inactive hide details for Ted Yu ---01/10/2017 08:58:06 AM---Is it
> a proper time to start preparing the next RC ? Cheers]Ted Yu
> ---01/10/2017 08:58:06 AM---Is it a proper time to start preparing the next
> RC ? Cheers
>
> From: Ted Yu 
> To: dev@bahir.apache.org
> Date: 01/10/2017 08:58 AM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC1)
> --
>
>
>
> Is it a proper time to start preparing the next RC ?
>
> Cheers
>
> On Sat, Dec 3, 2016 at 10:08 AM, Luciano Resende 
> wrote:
>
> > Please vote to approve the release of the following candidate as Apache
> > Bahir version 2.0.2 based on Apache Spark 2.0.2.
> >
> > The difference between 2.0.1 and 2.0.2 RC1 can be visualized at
> >
> >
> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc1
>
> >
> > The tag to be voted on is v2.0.2-rc1
> > (1c411ea22e60e3f7b3e93fdbc6e2351cda816c90)
> >
> > https://github.com/apache/bahi
> > r/tree/1c411ea22e60e3f7b3e93fdbc6e2351cda816c90
> >
> > The release files, including signatures, digests, etc. can be found at:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1007
> >
> > The source distribution is also available at:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc1/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Bahir 2.0.1
> > [ ] -1 Do not release this package because ...
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
>
>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC1)

[Ted Yu]

Is it a proper time to start preparing the next RC ?

Cheers

On Sat, Dec 3, 2016 at 10:08 AM, Luciano Resende 
wrote:

> Please vote to approve the release of the following candidate as Apache
> Bahir version 2.0.2 based on Apache Spark 2.0.2.
>
> The difference between 2.0.1 and 2.0.2 RC1 can be visualized at
>
> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc1
>
> The tag to be voted on is v2.0.2-rc1
> (1c411ea22e60e3f7b3e93fdbc6e2351cda816c90)
>
> https://github.com/apache/bahi
> r/tree/1c411ea22e60e3f7b3e93fdbc6e2351cda816c90
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1007
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc1/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.1
> [ ] -1 Do not release this package because ...
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.0.2 (RC1)

[Ted Yu]

I ran MQTTStreamSourceSuite alone which passed:

mvn scalatest:test -pl sql-streaming-mqtt -q
-Dsuites='*.MQTTStreamSourceSuite'

On Sun, Dec 4, 2016 at 12:55 AM, Christian R Kadner 
wrote:

> I did repeated runs of that particular test and it failed 6 of 100 runs.
>
> 
> ───
> for i in `seq 100` ; do
>   mvn scalatest:test -pl sql-streaming-mqtt -q \
> -Dsuites='*.BasicMQTTSourceSuite @ Recovering offset from the last
> processed offset.' | \
> grep -q "TEST FAILED" && echo "$i: failed"
> done
> 
> ───
> 1: failed
> 6: failed
> 22: failed
> 46: failed
> 77: failed
> 83: failed
> 
> ───
>
> The complete test suite BasicMQTTSourceSuite failed 56/100 times.
>
> Similar behavior with Spark 2.0.1. so it should not be a regression.
>
> I opened a JIRA to track/fix this:
> https://issues.apache.org/jira/browse/BAHIR-83
>
> Best regards,
> Christian Kadner
>
>
> [image: Inactive hide details for Ted Yu ---12/03/2016 01:41:26 PM---I got
> two test failures: http://pastebin.com/P2J2jQzC]Ted Yu ---12/03/2016
> 01:41:26 PM---I got two test failures: http://pastebin.com/P2J2jQzC
>
> From: Ted Yu 
> To: dev@bahir.apache.org
> Date: 12/03/2016 01:41 PM
> Subject: Re: [VOTE] Apache Bahir 2.0.2 (RC1)
> --
>
>
>
> I got two test failures:
>
> http://pastebin.com/P2J2jQzC
>
> I use maven 3.3.9 and Java 8.
>
> Does anyone else see the above ?
>
> Cheers
>
> On Sat, Dec 3, 2016 at 10:08 AM, Luciano Resende 
> wrote:
>
> > Please vote to approve the release of the following candidate as Apache
> > Bahir version 2.0.2 based on Apache Spark 2.0.2.
> >
> > The difference between 2.0.1 and 2.0.2 RC1 can be visualized at
> >
> > https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc1
> >
> > The tag to be voted on is v2.0.2-rc1
> > (1c411ea22e60e3f7b3e93fdbc6e2351cda816c90)
> >
> > https://github.com/apache/bahi
> > r/tree/1c411ea22e60e3f7b3e93fdbc6e2351cda816c90
> >
> > The release files, including signatures, digests, etc. can be found at:
> >
> > https://repository.apache.org/content/repositories/orgapachebahir-1007
> >
> > The source distribution is also available at:
> >
> > https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc1/
> >
> > The vote is open for at least 72 hours and passes if a majority of at
> least
> > 3 +1 PMC votes are cast.
> >
> > [ ] +1 Release this package as Apache Bahir 2.0.1
> > [ ] -1 Do not release this package because ...
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
>
>
>

Re: [VOTE] Apache Bahir 2.0.2 (RC1)

[Ted Yu]

I got two test failures:

http://pastebin.com/P2J2jQzC

I use maven 3.3.9 and Java 8.

Does anyone else see the above ?

Cheers

On Sat, Dec 3, 2016 at 10:08 AM, Luciano Resende 
wrote:

> Please vote to approve the release of the following candidate as Apache
> Bahir version 2.0.2 based on Apache Spark 2.0.2.
>
> The difference between 2.0.1 and 2.0.2 RC1 can be visualized at
>
> https://github.com/apache/bahir/compare/v2.0.1-rc1...v2.0.2-rc1
>
> The tag to be voted on is v2.0.2-rc1
> (1c411ea22e60e3f7b3e93fdbc6e2351cda816c90)
>
> https://github.com/apache/bahi
> r/tree/1c411ea22e60e3f7b3e93fdbc6e2351cda816c90
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1007
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.2-rc1/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.1
> [ ] -1 Do not release this package because ...
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [ANNOUNCE] New Apache Bahir Committer : Robert Metzger

[Ted Yu]

Congratulations, Robert

On Wed, Oct 19, 2016 at 2:54 PM, Luciano Resende 
wrote:

> The Project Management Committee (PMC) for Apache Bahirhas invited
> Robert Metzger to become a committer and we are pleased to announce
> that he has accepted.
>
> Robert has been a key community member championing the extensions
> related to Apache Flink.
>
>
> Welcome Robert
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [VOTE] Apache Bahir 2.0.1 (RC1)

[Ted Yu]

Running 'mvn package', I got the following two test failures:

Oct 15, 2016 7:59:52 PM INFO:
org.apache.parquet.hadoop.ParquetOutputFormat: Validation ^[[31m- Send and
receive 100 messages. *** FAILED ***^[[0m
^[[31m  Buffer("MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.", "MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a
message queue.", "MQTT is a message queue.", "MQTT is a message queue.",
"MQTT is a message queue.", "MQTT is a message queue.", "MQTT is a message
queue.") had size 69 instead of expected size 100
(MQTTStreamSourceSuite.scala:119)^[[0m
is off
...

Oct 15, 2016 8:00:03 PM INFO:
org.apache.parquet.hadoop.InternalParquetRecordWriter: Flushing mem
columnStore to file. allocated memory^[[31m- Recovering offset from the
last processed offset. *** FAILED ***^[[0m
^[[31m  java.util.NoSuchElementException: None.get^[[0m
^[[31m  at scala.None$.get(Option.scala:347)^[[0m
^[[31m  at scala.None$.get(Option.scala:345)^[[0m
^[[31m  at
org.apache.bahir.sql.streaming.mqtt.BasicMQTTSourceSuite$$anonfun$6.apply$mcV$sp(MQTTStreamSourceSuite.scala:166)^[[0m
^[[31m  at
org.apache.bahir.sql.streaming.mqtt.BasicMQTTSourceSuite$$anonfun$6.apply(MQTTStreamSourceSuite.scala:146)^[[0m
^[[31m  at
org.apache.bahir.sql.streaming.mqtt.BasicMQTTSourceSuite$$anonfun$6.apply(MQTTStreamSourceSuite.scala:146)^[[0m
^[[31m  at
org.scalatest.Transformer$$anonfun$apply$1.apply$mcV$sp(Transformer.scala:22)^[[0m
^[[31m  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)^[[0m
^[[31m  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)^[[0m
^[[31m  at org.scalatest.Transformer.apply(Transformer.scala:22)^[[0m
^[[31m  at org.scalatest.Transformer.apply(Transformer.scala:20)^[[0m
^[[31m  ...^[[0m

Has anyone else seen the above ?

Cheers

On Sat, Oct 15, 2016 at 11:16 AM, Luciano Resende 
wrote:

> Please vote to approve the release of the following candidate as Apache
> Bahir version 2.0.1 based on Apache Spark 2.0.1.
>
> The tag to be voted on is v2.0.1-rc1
> (bd00d2e4aa8c4cc8837bcaa1d0d95d3e8997ac8c)
>
> https://github.com/apache/bahir/tree/
> bd00d2e4aa8c4cc8837bcaa1d0d95d3e8997ac8c
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1004
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.1-rc1/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.1
> [ ] -1 Do not release this package because ...
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: Next Releases

[Ted Yu]

Spark 2.0.1 is in maven:

https://repo1.maven.org/maven2/org/apache/spark/spark-core_2.11/2.0.1/

Time to roll Bahir for Spark 2.0.1 release ?

On Mon, Sep 26, 2016 at 11:48 AM, Ted Yu  wrote:

> +1
>
> > On Sep 26, 2016, at 11:33 AM, Jean-Baptiste Onofré 
> wrote:
> >
> > +1
> >
> > Regards
> > JB
> >
> >> On 09/26/2016 08:32 PM, Luciano Resende wrote:
> >> Looks like Spark is working on 2.0.1 release (RC3 being voted now), and
> I
> >> was wondering if we should use this opportunity to create a Bahir for
> Spark
> >> 2.0.1 release and also a Bahir for Flink 1.0 release.
> >>
> >> Thoughts ?
> >
> > --
> > Jean-Baptiste Onofré
> > jbono...@apache.org
> > http://blog.nanthrax.net
> > Talend - http://www.talend.com
>

Re: Next Releases

[Ted Yu]

+1

> On Sep 26, 2016, at 11:33 AM, Jean-Baptiste Onofré  wrote:
> 
> +1
> 
> Regards
> JB
> 
>> On 09/26/2016 08:32 PM, Luciano Resende wrote:
>> Looks like Spark is working on 2.0.1 release (RC3 being voted now), and I
>> was wondering if we should use this opportunity to create a Bahir for Spark
>> 2.0.1 release and also a Bahir for Flink 1.0 release.
>> 
>> Thoughts ?
> 
> -- 
> Jean-Baptiste Onofré
> jbono...@apache.org
> http://blog.nanthrax.net
> Talend - http://www.talend.com

[jira] [Resolved] (BAHIR-59) Improve ActiveMQ Connector

[Ted Yu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BAHIR-59?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ted Yu resolved BAHIR-59.
-
   Resolution: Fixed
 Assignee: Robert Metzger
Fix Version/s: Flink-0.1

Thanks Robert.

> Improve ActiveMQ Connector
> --
>
> Key: BAHIR-59
> URL: https://issues.apache.org/jira/browse/BAHIR-59
> Project: Bahir
>  Issue Type: Improvement
>  Components: Flink Streaming Connectors
>Affects Versions: Flink-0.1
>Reporter: Robert Metzger
>Assignee: Robert Metzger
> Fix For: Flink-0.1
>
>
> While trying out the ActiveMQ connector for Flink, I found some issues that I 
> would like to address using this JIRA.
> - Make AMQSource constructor public
> - Improve error messages on wrong configuration



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Re: Documentation for Bahir Flink extensions

[Ted Yu]

+1

Sounds reasonable.

On Fri, Aug 26, 2016 at 12:13 PM, Luciano Resende 
wrote:

> For the Spark extensions we currently have a README.md on each extension
> project which helps users accessing github to easily find out how to use
> the extension, and we also use that to "copy" to the website and add some
> style to it, so it can also be used as the website documentation.
>
> Here is a readme.md
>
> https://github.com/apache/bahir/tree/master/sql-streaming-mqtt
>
> And then it gets copied to the website as
>
> http://bahir.apache.org/docs/spark/current/spark-sql-streaming-mqtt/
>
> I would like to propose a similar approach for creating documentation for
> the Bahir flink extensions.
>
> Thoughts ?
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: [DISCUSS] Bahir-flink versioning and releasing

[Ted Yu]

Fine by me.

bq. HBase streaming connector incoming

I will review it.

Thanks

On Thu, Aug 25, 2016 at 5:31 AM, Robert Metzger  wrote:

> Hi,
>
> I was wondering how we want to version the modules in the "bahir-flink"
> repository.
>
> I would actually like to do the versions independent of Flink. One of the
> motivations to move the connectors to Bahir was to allow faster releases.
>
> How about just "major.minor" versions, starting from 1.0. Bugfix releases
> would be 1.1, 1.2, and so on.
> I would also prefer to do frequent major releases (for example when a new
> connector is added) and do bugfix releases only if needed.
>
> Also, there's the ActiveMQ connector PR pending, and a HBase streaming
> connector incoming. Once they are both merged, I'd like to create the first
> Bahir-flink release to get the whole infrastructure in place.
>
>
> What do you think?
>
>
> Regards,
> Robert
>

Re: [DISCUSS] Adding streaming connectors from Apache Flink to Bahir

[Ted Yu]

Having Flink connectors in the same repo seems to make more sense at the
moment.

Certain artifacts can be shared between the two types of connectors.

Flink seems to have more frequent releases recently. But Bahir doesn't have
to follow each Flink patch release.

Just my two cents.

On Thu, Aug 11, 2016 at 9:03 AM, Luciano Resende 
wrote:

> On Thu, Aug 11, 2016 at 5:42 AM, Robert Metzger 
> wrote:
>
> >
> >
> > @Luciano: So the idea is to have separate repositories for each project
> > contributing connectors?
> > I'm wondering if it makes sense to keep the code in the same repository
> to
> > have some synergies (like the release scripts, CI, documentation, a
> common
> > parent pom with rat etc.). Otherwise, it would maybe make more sense to
> > create a Bahir-style project for Flink, to avoid maintaining completely
> > disjunct codebases in the same JIRA, ML, ...
> >
> >
> >
> But we most likely would have very different release schedules with the
> different set of extensions, where Spark extensions will tend to follow
> Spark release cycles, and Flink release cycles. As for the overhead, I
> believe release scripts might be the one piece that would be replicated,
> but I can volunteer the infrastructure overhead for now. All rest, such as
> JIRA, ML, etc will be common. But, anyway, I don't want to make this an
> issue for Flink to bring up the extensions here, so if you have a strong
> preference on having all in the same repo, we could start with that.
>
> Thoughts ?
>

Re: [VOTE] Apache Bahir 2.0.0 (RC1)

[Ted Yu]

+1

Built from source.
Checked license.
Ran unit test suite which passed.

On Sat, Aug 6, 2016 at 3:11 AM, Luciano Resende 
wrote:

> Please vote to approve the release of the following candidate as Apache
> Bahir version 2.0.0 based on Apache Spark 2.0.0.
>
> The tag to be voted on is v2.0.0-rc1
> (368c436ae2ad34b3ca64d11801aee69e478555f7)
>
> https://github.com/apache/bahir/tree/368c436ae2ad34b3ca64d11801aee6
> 9e478555f7
>
> The release files, including signatures, digests, etc. can be found at:
>
> https://repository.apache.org/content/repositories/orgapachebahir-1001
>
> The source distribution is also available at:
>
> https://dist.apache.org/repos/dist/dev/bahir/2.0.0-rc1/
>
> The vote is open for at least 72 hours and passes if a majority of at least
> 3 +1 PMC votes are cast.
>
> [ ] +1 Release this package as Apache Bahir 2.0.0-preview
> [ ] -1 Do not release this package because ...
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>

Re: 2.0.0 Release

[Ted Yu]

Looking at the 5 items listed at the end of the PR, it seems there is some
non-trivial work to be done.

I think releasing 2.0 first without MQTT Structured Streaming connector
would give its author enough time to achieve high quality code.

Cheers

On Mon, Aug 1, 2016 at 1:04 PM, Luciano Resende 
wrote:

> What do others thing about waiting few more days for the release in order
> to have the MQTT Structured Streaming connector being contributed in
> https://github.com/apache/bahir/pull/13 ... Otherwise, another option is
> to
> have RC1 out, and if that pass, we can then have a 2.0.0.1 release right
> when that is available.
>
> Thoughts ?
>
> On Sat, Jul 23, 2016 at 7:33 PM, Luciano Resende 
> wrote:
>
> > Now that the Spark vote for 2.0.0 release has passed, I will wait for the
> > artifacts to be published to maven and start a 2.0.0 release.
> >
> > Please let me know if there are any blocking issues that we should wait
> > before starting that.
> >
> > --
> > Luciano Resende
> > http://twitter.com/lresende1975
> > http://lresende.blogspot.com/
> >
>
>
>
> --
> Luciano Resende
> http://twitter.com/lresende1975
> http://lresende.blogspot.com/
>