I've been watching the RC process here and would like to help verify the release, since I have co-workers wanting to upgrade to 1.11
When I follow the instructions to verify the release I am running into an issue resolving a dependency: Could not resolve dependencies for project org.apache.parquet:parquet-thrift:jar:1.11.0: Could not find artifact com.hadoop.gplcompression:hadoop-lzo:jar:0.4.16 I don't see anything in the README in the repo that explains how I should configure maven to find this dependency. I don’t even see a direct reference to this jar in maven, so would appreciate some guidance. Thanks, Andy, On 3/21/19, 3:40 PM, "Zoltan Ivanfi" <[email protected]> wrote: CAUTION – UNVERIFIED EXTERNAL EMAIL Hi Wes, Here is a list of steps (the first part is probably the same as for parquet-cpp): 1. Download the files from https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.11.0-rc6%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=5WxAct62mkcX2OezQNaj15X9Q%2FQ3zJETYBNt2kKFyWk%3D&reserved=0 2. Download the KEYS file from https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=yy4O7iXv8RZshf4qyarjyzPyIyNq4p3rtkjjNUWcIRY%3D&reserved=0 3. Import the GPG keys by running `gpg --import KEYS`. 4. Verify the signature by running `gpg --verify apache-parquet-1.11.0.tar.gz.asc`. It should say "Good signature", the warning about the key not being trusted can be ignored. 5. Extract the source code: `tar xzf apache-parquet-1.11.0.tar.gz`. 6. Change to the source directory: `cd apache-parquet-1.11.0`. 7. Make sure that you have a thrift compiler and that you have the correct version by issuing `thrift -version`. It should print "Thrift version 0.12.0". If you don't have it, you need to install it in a manner that is specific to your system (for example, `sudo apt install thrift-compiler` or `sudo yum install thrift-compiler`). If your package manager only offers an older version, you may still use that with the workaround shown below. 8. Test the release by issuing `mvn clean verify`. If you have an older thrift compiler, for example 0.9.3, you have to issue `mvn -Dformat.thrift.version=0.9.3 -Dthrift.version=0.9.3 clean install` instead. This may or may not work depending on how old your thrift version is (>= 0.9.0 should work). Running this command can take around 15 minutes and should print "BUILD SUCCESS" at the end (followed by a few extra lines). Br, Zoltan On Thu, Mar 21, 2019 at 7:58 PM Wes McKinney <[email protected]> wrote: > Are there any instructions written down about how to verify this release? > > On Wed, Mar 20, 2019 at 8:50 AM Zoltan Ivanfi <[email protected]> > wrote: > > > > +1 (binding) > > > > signature matches > > git hash matches the git tag > > source tarball matches the git tag > > unit tests and integration tests pass > > > > On Tue, Mar 19, 2019 at 3:00 PM Gabor Szadovszky <[email protected]> > wrote: > > > > > Dear Parquet Users and Developers, > > > > > > I propose the following RC to be released as the official Apache > > > Parquet 1.11.0 release: > > > > > > The commit id is 9756b0e2b35437a09716707a81e2ac0c187112ed > > > * This corresponds to the tag: apache-parquet-1.11.0 > > > * > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F9756b0e2b35437a09716707a81e2ac0c187112ed&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=MoO3FhiXqeXRt0jb0lvBqsNELuUUwKfjekX6DSvOr%2Fg%3D&reserved=0 > > > > > > The release tarball, signature, and checksums are here: > > > * > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.11.0-rc6%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=5WxAct62mkcX2OezQNaj15X9Q%2FQ3zJETYBNt2kKFyWk%3D&reserved=0 > > > > > > You can find the KEYS file here: > > > * https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fformatdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=e6W8VpBIil0xTYrzoJpNRR%2FCiKNClA87nxY3Y6UNmvM%3D&reserved=0 > <https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=yy4O7iXv8RZshf4qyarjyzPyIyNq4p3rtkjjNUWcIRY%3D&reserved=0> > > > > > > Binary artifacts are staged in Nexus here: > > > * > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2Fparquet%2F1.11.0%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C94c5ce21c66047bee55208d6ae45d8e2%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636888012357234811&sdata=NAe4dRJv%2BNuFXnidnLD2Ljc7WuQxlryqbVZlwsNKLag%3D&reserved=0 > > > > > > This release includes the following new features: > > > - PARQUET-1201 - Column indexes > > > - PARQUET-1253 - Support for new logical type representation > > > - PARQUET-1381 - Add merge blocks command to parquet-tools > > > - PARQUET-1388 - Nanosecond precision time and timestamp - parquet-mr > > > > > > The release also includes bug fixes, including: > > > - PARQUET-1472: Dictionary filter fails on FIXED_LEN_BYTE_ARRAY. > > > - PARQUET-1510: Fix notEq for optional columns with null values. > > > - PARQUET-1533: TestSnappy() throws OOM exception with Parquet-1485 > change > > > - PARQUET-1531: Page row count limit causes empty pages to be written > from > > > MessageColumnIO > > > - PARQUET-1544: Possible over-shading of modules > > > > > > The following change has been reverted so it is not part of any public > > > release: > > > - PARQUET-1381: Add merge blocks command to parquet-tools > > > > > > Please download, verify, and test. The vote will be open for at least > 72 > > > hours. > > > > > > Thanks, > > > Gabor > > > >
