Re: Updating Wiki about Hikari Configuration Properties

2022-10-29 Thread Chris Nauroth
That did the trick! Thank you. I have updated the page.

Chris Nauroth


On Fri, Oct 28, 2022 at 11:47 PM Denys Kuzmenko
 wrote:

> Hi Chris,
>
> Previously I just gave you edit permissions for the requested page, but now
> granted Hive space permissions. Please check again and let me know.
>
> On Fri, Oct 28, 2022 at 7:51 PM Chris Nauroth  wrote:
>
> > Thank you, Denys. Unfortunately, I still don't see the Edit button like I
> > do on other projects where I have access (Hadoop, Zookeeper). I just
> > realized my Confluence login is actually just "cnauroth", so maybe I
> caused
> > some confusion by stating it as "cnaur...@apache.org"?
> >
> > Ayush, to further clarify, yes, I am planning to backport HIVE-17317 onto
> > versions 3.1.2 and 3.1.3 of Dataproc's Hive distro. I've already tested
> > locally and confirmed that I can use hikaricp.minimumIdle to limit the
> > number of connections. I'd also be happy to send an upstream pull request
> > against branch-3.1 for the backport if the community would find that
> > valuable. Let me know.
> >
> > When I make the Confluence edit, I'll clarify which versions have this
> > support.
> >
> > Chris Nauroth
> >
> >
> > On Fri, Oct 28, 2022 at 4:37 AM Ayush Saxena  wrote:
> >
> > > Hi Chris,
> > >
> > > Just curious:
> > >>
> > >> BTW, the reason I discovered this is that I recently upgraded a
> cluster
> > >> from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP).
> > >
> > >
> > > As you said the prefix change happened in HIVE-17317 which is in
> > > 4.0.0-alpha-1 [1] and you migrated to Hive 3.x, Then how did you face
> > > this problem? Some patched version of Hive, or some missing FixVersion
> in
> > > the Jira?
> > >
> > > Good to mention the version post which the change in prefix happens in
> > the
> > > wiki as well, to avoid future confusions due to versions.
> > >
> > > -Ayush
> > >
> > > [1] https://issues.apache.org/jira/browse/HIVE-17317
> > >
> > > On Fri, 28 Oct 2022 at 17:02, Denys Kuzmenko
> > >  wrote:
> > >
> > >> Hi Chris,
> > >>
> > >> Please try now
> > >>
> > >> On Fri, Oct 28, 2022 at 12:43 PM Stamatis Zampetakis <
> zabe...@gmail.com
> > >
> > >> wrote:
> > >>
> > >> > Hive PMC members can provide edit rights to the wiki.
> > >> >
> > >> > @Naveen, Dennys, Adam: Can someone please give write privileges to
> > >> Chris?
> > >> >
> > >> > Best,
> > >> > Stamatis
> > >> >
> > >> > On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth 
> > >> wrote:
> > >> >
> > >> > > Hi everyone,
> > >> > >
> > >> > > Regarding this page:
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >>
> >
> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
> > >> > >
> > >> > > It states that the metastore's Hikari connection pool can be
> > >> configured
> > >> > by
> > >> > > specifying properties prefixed as "hikari". This is not quite
> > >> correct. In
> > >> > > HIVE-17317, there was a bug fix made to the Hikari integration
> such
> > >> that
> > >> > > the proper prefix is "hikaricp". For example:
> > >> > >
> > >> > >   
> > >> > > hikaricp.minimumIdle
> > >> > > 4
> > >> > > false
> > >> > > Dataproc Cluster Properties
> > >> > >   
> > >> > >
> > >> > > Could you please grant access to me (cnaur...@apache.org) to
> update
> > >> the
> > >> > > page? If you prefer not to grant access, could a Hive committer
> make
> > >> the
> > >> > > change for me?
> > >> > >
> > >> > > BTW, the reason I discovered this is that I recently upgraded a
> > >> cluster
> > >> > > from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP).
> After
> > >> the
> > >> > > upgrade, I found that HiveMetaStore was generating far more
> database
> > >> > > connections at baseline, putting extra burden on the database. It
> > >> appears
> > >> > > that BoneCP default behavior (4 idle connections) is different
> from
> > >> > > HikariCP default behavior (idle connections equal to max
> connections
> > >> > which
> > >> > > is 10). This put me down the path of wanting to control Hikari's
> > >> > > minimumIdle setting and then finding this discrepancy in the
> > >> > documentation.
> > >> > >
> > >> > > Passing on this information in case others are seeing unusually
> high
> > >> > > connection counts after an upgrade to 3.x.
> > >> > >
> > >> > > Chris Nauroth
> > >> > >
> > >> >
> > >>
> > >
> >
>


Re: Updating Wiki about Hikari Configuration Properties

2022-10-29 Thread Denys Kuzmenko
Hi Chris,

Previously I just gave you edit permissions for the requested page, but now
granted Hive space permissions. Please check again and let me know.

On Fri, Oct 28, 2022 at 7:51 PM Chris Nauroth  wrote:

> Thank you, Denys. Unfortunately, I still don't see the Edit button like I
> do on other projects where I have access (Hadoop, Zookeeper). I just
> realized my Confluence login is actually just "cnauroth", so maybe I caused
> some confusion by stating it as "cnaur...@apache.org"?
>
> Ayush, to further clarify, yes, I am planning to backport HIVE-17317 onto
> versions 3.1.2 and 3.1.3 of Dataproc's Hive distro. I've already tested
> locally and confirmed that I can use hikaricp.minimumIdle to limit the
> number of connections. I'd also be happy to send an upstream pull request
> against branch-3.1 for the backport if the community would find that
> valuable. Let me know.
>
> When I make the Confluence edit, I'll clarify which versions have this
> support.
>
> Chris Nauroth
>
>
> On Fri, Oct 28, 2022 at 4:37 AM Ayush Saxena  wrote:
>
> > Hi Chris,
> >
> > Just curious:
> >>
> >> BTW, the reason I discovered this is that I recently upgraded a cluster
> >> from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP).
> >
> >
> > As you said the prefix change happened in HIVE-17317 which is in
> > 4.0.0-alpha-1 [1] and you migrated to Hive 3.x, Then how did you face
> > this problem? Some patched version of Hive, or some missing FixVersion in
> > the Jira?
> >
> > Good to mention the version post which the change in prefix happens in
> the
> > wiki as well, to avoid future confusions due to versions.
> >
> > -Ayush
> >
> > [1] https://issues.apache.org/jira/browse/HIVE-17317
> >
> > On Fri, 28 Oct 2022 at 17:02, Denys Kuzmenko
> >  wrote:
> >
> >> Hi Chris,
> >>
> >> Please try now
> >>
> >> On Fri, Oct 28, 2022 at 12:43 PM Stamatis Zampetakis  >
> >> wrote:
> >>
> >> > Hive PMC members can provide edit rights to the wiki.
> >> >
> >> > @Naveen, Dennys, Adam: Can someone please give write privileges to
> >> Chris?
> >> >
> >> > Best,
> >> > Stamatis
> >> >
> >> > On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth 
> >> wrote:
> >> >
> >> > > Hi everyone,
> >> > >
> >> > > Regarding this page:
> >> > >
> >> > >
> >> > >
> >> >
> >>
> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
> >> > >
> >> > > It states that the metastore's Hikari connection pool can be
> >> configured
> >> > by
> >> > > specifying properties prefixed as "hikari". This is not quite
> >> correct. In
> >> > > HIVE-17317, there was a bug fix made to the Hikari integration such
> >> that
> >> > > the proper prefix is "hikaricp". For example:
> >> > >
> >> > >   
> >> > > hikaricp.minimumIdle
> >> > > 4
> >> > > false
> >> > > Dataproc Cluster Properties
> >> > >   
> >> > >
> >> > > Could you please grant access to me (cnaur...@apache.org) to update
> >> the
> >> > > page? If you prefer not to grant access, could a Hive committer make
> >> the
> >> > > change for me?
> >> > >
> >> > > BTW, the reason I discovered this is that I recently upgraded a
> >> cluster
> >> > > from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP). After
> >> the
> >> > > upgrade, I found that HiveMetaStore was generating far more database
> >> > > connections at baseline, putting extra burden on the database. It
> >> appears
> >> > > that BoneCP default behavior (4 idle connections) is different from
> >> > > HikariCP default behavior (idle connections equal to max connections
> >> > which
> >> > > is 10). This put me down the path of wanting to control Hikari's
> >> > > minimumIdle setting and then finding this discrepancy in the
> >> > documentation.
> >> > >
> >> > > Passing on this information in case others are seeing unusually high
> >> > > connection counts after an upgrade to 3.x.
> >> > >
> >> > > Chris Nauroth
> >> > >
> >> >
> >>
> >
>


Re: Updating Wiki about Hikari Configuration Properties

2022-10-28 Thread Chris Nauroth
Thank you, Denys. Unfortunately, I still don't see the Edit button like I
do on other projects where I have access (Hadoop, Zookeeper). I just
realized my Confluence login is actually just "cnauroth", so maybe I caused
some confusion by stating it as "cnaur...@apache.org"?

Ayush, to further clarify, yes, I am planning to backport HIVE-17317 onto
versions 3.1.2 and 3.1.3 of Dataproc's Hive distro. I've already tested
locally and confirmed that I can use hikaricp.minimumIdle to limit the
number of connections. I'd also be happy to send an upstream pull request
against branch-3.1 for the backport if the community would find that
valuable. Let me know.

When I make the Confluence edit, I'll clarify which versions have this
support.

Chris Nauroth


On Fri, Oct 28, 2022 at 4:37 AM Ayush Saxena  wrote:

> Hi Chris,
>
> Just curious:
>>
>> BTW, the reason I discovered this is that I recently upgraded a cluster
>> from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP).
>
>
> As you said the prefix change happened in HIVE-17317 which is in
> 4.0.0-alpha-1 [1] and you migrated to Hive 3.x, Then how did you face
> this problem? Some patched version of Hive, or some missing FixVersion in
> the Jira?
>
> Good to mention the version post which the change in prefix happens in the
> wiki as well, to avoid future confusions due to versions.
>
> -Ayush
>
> [1] https://issues.apache.org/jira/browse/HIVE-17317
>
> On Fri, 28 Oct 2022 at 17:02, Denys Kuzmenko
>  wrote:
>
>> Hi Chris,
>>
>> Please try now
>>
>> On Fri, Oct 28, 2022 at 12:43 PM Stamatis Zampetakis 
>> wrote:
>>
>> > Hive PMC members can provide edit rights to the wiki.
>> >
>> > @Naveen, Dennys, Adam: Can someone please give write privileges to
>> Chris?
>> >
>> > Best,
>> > Stamatis
>> >
>> > On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth 
>> wrote:
>> >
>> > > Hi everyone,
>> > >
>> > > Regarding this page:
>> > >
>> > >
>> > >
>> >
>> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
>> > >
>> > > It states that the metastore's Hikari connection pool can be
>> configured
>> > by
>> > > specifying properties prefixed as "hikari". This is not quite
>> correct. In
>> > > HIVE-17317, there was a bug fix made to the Hikari integration such
>> that
>> > > the proper prefix is "hikaricp". For example:
>> > >
>> > >   
>> > > hikaricp.minimumIdle
>> > > 4
>> > > false
>> > > Dataproc Cluster Properties
>> > >   
>> > >
>> > > Could you please grant access to me (cnaur...@apache.org) to update
>> the
>> > > page? If you prefer not to grant access, could a Hive committer make
>> the
>> > > change for me?
>> > >
>> > > BTW, the reason I discovered this is that I recently upgraded a
>> cluster
>> > > from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP). After
>> the
>> > > upgrade, I found that HiveMetaStore was generating far more database
>> > > connections at baseline, putting extra burden on the database. It
>> appears
>> > > that BoneCP default behavior (4 idle connections) is different from
>> > > HikariCP default behavior (idle connections equal to max connections
>> > which
>> > > is 10). This put me down the path of wanting to control Hikari's
>> > > minimumIdle setting and then finding this discrepancy in the
>> > documentation.
>> > >
>> > > Passing on this information in case others are seeing unusually high
>> > > connection counts after an upgrade to 3.x.
>> > >
>> > > Chris Nauroth
>> > >
>> >
>>
>


Re: Updating Wiki about Hikari Configuration Properties

2022-10-28 Thread Ayush Saxena
Hi Chris,

Just curious:
>
> BTW, the reason I discovered this is that I recently upgraded a cluster
> from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP).


As you said the prefix change happened in HIVE-17317 which is in
4.0.0-alpha-1 [1] and you migrated to Hive 3.x, Then how did you face
this problem? Some patched version of Hive, or some missing FixVersion in
the Jira?

Good to mention the version post which the change in prefix happens in the
wiki as well, to avoid future confusions due to versions.

-Ayush

[1] https://issues.apache.org/jira/browse/HIVE-17317

On Fri, 28 Oct 2022 at 17:02, Denys Kuzmenko 
wrote:

> Hi Chris,
>
> Please try now
>
> On Fri, Oct 28, 2022 at 12:43 PM Stamatis Zampetakis 
> wrote:
>
> > Hive PMC members can provide edit rights to the wiki.
> >
> > @Naveen, Dennys, Adam: Can someone please give write privileges to Chris?
> >
> > Best,
> > Stamatis
> >
> > On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth 
> wrote:
> >
> > > Hi everyone,
> > >
> > > Regarding this page:
> > >
> > >
> > >
> >
> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
> > >
> > > It states that the metastore's Hikari connection pool can be configured
> > by
> > > specifying properties prefixed as "hikari". This is not quite correct.
> In
> > > HIVE-17317, there was a bug fix made to the Hikari integration such
> that
> > > the proper prefix is "hikaricp". For example:
> > >
> > >   
> > > hikaricp.minimumIdle
> > > 4
> > > false
> > > Dataproc Cluster Properties
> > >   
> > >
> > > Could you please grant access to me (cnaur...@apache.org) to update
> the
> > > page? If you prefer not to grant access, could a Hive committer make
> the
> > > change for me?
> > >
> > > BTW, the reason I discovered this is that I recently upgraded a cluster
> > > from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP). After
> the
> > > upgrade, I found that HiveMetaStore was generating far more database
> > > connections at baseline, putting extra burden on the database. It
> appears
> > > that BoneCP default behavior (4 idle connections) is different from
> > > HikariCP default behavior (idle connections equal to max connections
> > which
> > > is 10). This put me down the path of wanting to control Hikari's
> > > minimumIdle setting and then finding this discrepancy in the
> > documentation.
> > >
> > > Passing on this information in case others are seeing unusually high
> > > connection counts after an upgrade to 3.x.
> > >
> > > Chris Nauroth
> > >
> >
>


Re: Updating Wiki about Hikari Configuration Properties

2022-10-28 Thread Denys Kuzmenko
Hi Chris,

Please try now

On Fri, Oct 28, 2022 at 12:43 PM Stamatis Zampetakis 
wrote:

> Hive PMC members can provide edit rights to the wiki.
>
> @Naveen, Dennys, Adam: Can someone please give write privileges to Chris?
>
> Best,
> Stamatis
>
> On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth  wrote:
>
> > Hi everyone,
> >
> > Regarding this page:
> >
> >
> >
> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
> >
> > It states that the metastore's Hikari connection pool can be configured
> by
> > specifying properties prefixed as "hikari". This is not quite correct. In
> > HIVE-17317, there was a bug fix made to the Hikari integration such that
> > the proper prefix is "hikaricp". For example:
> >
> >   
> > hikaricp.minimumIdle
> > 4
> > false
> > Dataproc Cluster Properties
> >   
> >
> > Could you please grant access to me (cnaur...@apache.org) to update the
> > page? If you prefer not to grant access, could a Hive committer make the
> > change for me?
> >
> > BTW, the reason I discovered this is that I recently upgraded a cluster
> > from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP). After the
> > upgrade, I found that HiveMetaStore was generating far more database
> > connections at baseline, putting extra burden on the database. It appears
> > that BoneCP default behavior (4 idle connections) is different from
> > HikariCP default behavior (idle connections equal to max connections
> which
> > is 10). This put me down the path of wanting to control Hikari's
> > minimumIdle setting and then finding this discrepancy in the
> documentation.
> >
> > Passing on this information in case others are seeing unusually high
> > connection counts after an upgrade to 3.x.
> >
> > Chris Nauroth
> >
>


Re: Updating Wiki about Hikari Configuration Properties

2022-10-28 Thread Stamatis Zampetakis
Hive PMC members can provide edit rights to the wiki.

@Naveen, Dennys, Adam: Can someone please give write privileges to Chris?

Best,
Stamatis

On Fri, Oct 28, 2022 at 8:41 AM Chris Nauroth  wrote:

> Hi everyone,
>
> Regarding this page:
>
>
> https://cwiki.apache.org/confluence/display/hive/configuration+properties#ConfigurationProperties-HiveMetastoreConnectionPoolingConfiguration
>
> It states that the metastore's Hikari connection pool can be configured by
> specifying properties prefixed as "hikari". This is not quite correct. In
> HIVE-17317, there was a bug fix made to the Hikari integration such that
> the proper prefix is "hikaricp". For example:
>
>   
> hikaricp.minimumIdle
> 4
> false
> Dataproc Cluster Properties
>   
>
> Could you please grant access to me (cnaur...@apache.org) to update the
> page? If you prefer not to grant access, could a Hive committer make the
> change for me?
>
> BTW, the reason I discovered this is that I recently upgraded a cluster
> from Hive 2.x (default BoneCP) to Hive 3.x (default HikariCP). After the
> upgrade, I found that HiveMetaStore was generating far more database
> connections at baseline, putting extra burden on the database. It appears
> that BoneCP default behavior (4 idle connections) is different from
> HikariCP default behavior (idle connections equal to max connections which
> is 10). This put me down the path of wanting to control Hikari's
> minimumIdle setting and then finding this discrepancy in the documentation.
>
> Passing on this information in case others are seeing unusually high
> connection counts after an upgrade to 3.x.
>
> Chris Nauroth
>