Re: Hive 1.1.0 support on hive metastore 2.3.0
Thanks. Will try out MR3. On Mon, Dec 9, 2019 at 3:51 PM Sungwoo Park wrote: > I didn't try to run multiple versions of Hive on the same cluster. If your > installation of Hive uses Tez installed on the Hadoop system, I guess > running multiple versions of Hive might not be easy because different > versions of Hive use different versions of Tez (especially if you want to > run Hive-LLAP). For me, I use Hive on MR3 (well, because we created MR3). > You can run all of Hive 1, 2, 3 concurrently on the same cluster. And there > is no installation process -- just unpack the tar balls and configure > files, and you are all set. > > --- Sungwoo > > On Mon, Dec 9, 2019 at 6:27 PM Priyam Gupta wrote: > >> Thanks Park for sharing the tests that you did. I will try out for the >> specific version for my use cases. >> >> Is there any better way/approach where we can have a single meta store >> and can launch multiple hive clusters of different versions point to the >> same metastore db. >> >> Thanks. >> >> >> >> >> On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park wrote: >> >>> Not a definitive answer, but my test result might help. I tested with >>> HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark >>> (which only read data and never update) run okay. Creating new tables and >>> loading data to tables also work okay. So, I guess for basic uses of Hive, >>> running HiveServer2 1.2 against Metastore 2.3 should be fine. >>> >>> What I am not sure about is whether queries involving transactions work >>> okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I >>> have not tested with Hive 1.1. Perhaps those familiar with internals of >>> Metastore in this mailing list could give more accurate answers. >>> >>> As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However, >>> HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1. >>> >>> Cheers, >>> >>> --- gla >>> >>> On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta wrote: >>> Hi, In order to have single hive metastore supporting different clusters of hive, I have upgraded my hive metastore schema (on postgres) from version 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point it to upgraded metastore db which is as per hive version 2.3.0. Thanks, Priyam >>>
Re: Hive 1.1.0 support on hive metastore 2.3.0
I didn't try to run multiple versions of Hive on the same cluster. If your installation of Hive uses Tez installed on the Hadoop system, I guess running multiple versions of Hive might not be easy because different versions of Hive use different versions of Tez (especially if you want to run Hive-LLAP). For me, I use Hive on MR3 (well, because we created MR3). You can run all of Hive 1, 2, 3 concurrently on the same cluster. And there is no installation process -- just unpack the tar balls and configure files, and you are all set. --- Sungwoo On Mon, Dec 9, 2019 at 6:27 PM Priyam Gupta wrote: > Thanks Park for sharing the tests that you did. I will try out for the > specific version for my use cases. > > Is there any better way/approach where we can have a single meta store and > can launch multiple hive clusters of different versions point to the same > metastore db. > > Thanks. > > > > > On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park wrote: > >> Not a definitive answer, but my test result might help. I tested with >> HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark >> (which only read data and never update) run okay. Creating new tables and >> loading data to tables also work okay. So, I guess for basic uses of Hive, >> running HiveServer2 1.2 against Metastore 2.3 should be fine. >> >> What I am not sure about is whether queries involving transactions work >> okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I >> have not tested with Hive 1.1. Perhaps those familiar with internals of >> Metastore in this mailing list could give more accurate answers. >> >> As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However, >> HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1. >> >> Cheers, >> >> --- gla >> >> On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta wrote: >> >>> >>> >>> Hi, >>> >>> In order to have single hive metastore supporting different clusters of >>> hive, I have upgraded my hive metastore schema (on postgres) from version >>> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point >>> it to upgraded metastore db which is as per hive version 2.3.0. >>> >>> >>> Thanks, >>> Priyam >>> >>
Re: Hive 1.1.0 support on hive metastore 2.3.0
Thanks Park for sharing the tests that you did. I will try out for the specific version for my use cases. Is there any better way/approach where we can have a single meta store and can launch multiple hive clusters of different versions point to the same metastore db. Thanks. On Mon, Dec 9, 2019 at 12:46 PM Sungwoo Park wrote: > Not a definitive answer, but my test result might help. I tested with > HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark > (which only read data and never update) run okay. Creating new tables and > loading data to tables also work okay. So, I guess for basic uses of Hive, > running HiveServer2 1.2 against Metastore 2.3 should be fine. > > What I am not sure about is whether queries involving transactions work > okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I > have not tested with Hive 1.1. Perhaps those familiar with internals of > Metastore in this mailing list could give more accurate answers. > > As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However, > HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1. > > Cheers, > > --- gla > > On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta wrote: > >> >> >> Hi, >> >> In order to have single hive metastore supporting different clusters of >> hive, I have upgraded my hive metastore schema (on postgres) from version >> 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point >> it to upgraded metastore db which is as per hive version 2.3.0. >> >> >> Thanks, >> Priyam >> >
Re: Hive 1.1.0 support on hive metastore 2.3.0
Not a definitive answer, but my test result might help. I tested with HiveServer2 1.2.2 and Metastore 2.3.6. Queries in the TPC-DS benchmark (which only read data and never update) run okay. Creating new tables and loading data to tables also work okay. So, I guess for basic uses of Hive, running HiveServer2 1.2 against Metastore 2.3 should be fine. What I am not sure about is whether queries involving transactions work okay when HiveServer2 1.2 connects to Metastore 2.3. Please note that I have not tested with Hive 1.1. Perhaps those familiar with internals of Metastore in this mailing list could give more accurate answers. As a side note, HiveServer2 2.3 can connect to Metastore 3.1. However, HiveServer2 1.2 seems to have a problem with connecting to Metastore 3.1. Cheers, --- gla On Mon, Dec 9, 2019 at 2:38 PM Priyam Gupta wrote: > > > Hi, > > In order to have single hive metastore supporting different clusters of > hive, I have upgraded my hive metastore schema (on postgres) from version > 1.1.0 to version 2.3.0. Will hive cluster on version 1.1.0 work if I point > it to upgraded metastore db which is as per hive version 2.3.0. > > > Thanks, > Priyam >