Hi, I have made some experiments as well and this gets more and more interesting. Let's move this conversation to the JIRA though (thanks for creating it by the way).
Br, Zoltan On Wed, Apr 3, 2019 at 5:17 PM Andy Grove <[email protected]> wrote: > It works after configuring the twitter Maven repo. Here is the JIRA: > https://issues.apache.org/jira/browse/PARQUET-1556 > > On 4/3/19, 9:07 AM, "Andy Grove" <[email protected]> wrote: > > CAUTION – UNVERIFIED EXTERNAL EMAIL > > > This artifact isn't available in maven central. It is in Twitter's > maven repo. > > I am working on setting up my m2 settings to be able to read from > there, but this is something that really needs to be documented. > > Once I figure it out, I will create a JIRA + PR to update the README. > > Thanks. > > On 4/3/19, 8:55 AM, "Zoltan Ivanfi" <[email protected]> wrote: > > CAUTION – UNVERIFIED EXTERNAL EMAIL > > > Hi, > > That error message is quite surprising, Maven should be able to > find that > artifact without any manual configuration. It may have been some > temporary > network issue with accessing the repository. Could you try it > again? If it > keeps failing, Maven may have cached the unavailability of that > artifact, > which can be cleaned by issuing > > mvn dependency:purge-local-repository > -DmanualInclude=com.hadoop.gplcompression:hadoop-lzo > > In case you still continue to get the same failure, you may be > behind a > stale artifact cache or some other kind of proxy or that does not > allow you > to access this artifact directly. > > Br, > > Zoltan > > > On Wed, Apr 3, 2019 at 2:14 PM XU Qinghui <[email protected]> > wrote: > > > Hello, Andy > > > > I think this dependency is in maven repository: > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmvnrepository.com%2Fartifact%2Fcom.hadoop.gplcompression%2Fhadoop-lzo%2F0.4.16&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585976838&sdata=Bpzt4O1bvNXqy%2FWO7KKYh2sECoU6KV94fZXvIZ5M0Tk%3D&reserved=0 > > Maybe you need to configure your `.m2/settings.xml` to have the > repository, > > here's some doc from maven: > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaven.apache.org%2Fguides%2Fmini%2Fguide-multiple-repositories.html&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585976838&sdata=DwAmqo7IPC7EYNZE03b76EKYdzruQ%2FI6NFpPmLEELOo%3D&reserved=0 > > > > Best wishes, > > > > Le mer. 3 avr. 2019 à 13:33, Andy Grove <[email protected]> a > écrit : > > > > > I've been watching the RC process here and would like to help > verify the > > > release, since I have co-workers wanting to upgrade to 1.11 > > > > > > When I follow the instructions to verify the release I am > running into an > > > issue resolving a dependency: > > > > > > Could not resolve dependencies for project > > > org.apache.parquet:parquet-thrift:jar:1.11.0: Could not find > artifact > > > com.hadoop.gplcompression:hadoop-lzo:jar:0.4.16 > > > > > > I don't see anything in the README in the repo that explains > how I should > > > configure maven to find this dependency. I don’t even see a > direct > > > reference to this jar in maven, so would appreciate some > guidance. > > > > > > Thanks, > > > > > > Andy, > > > > > > > > > On 3/21/19, 3:40 PM, "Zoltan Ivanfi" <[email protected]> > wrote: > > > > > > CAUTION – UNVERIFIED EXTERNAL EMAIL > > > > > > > > > Hi Wes, > > > > > > Here is a list of steps (the first part is probably the > same as for > > > parquet-cpp): > > > > > > 1. Download the files from > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.11.0-rc6%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585976838&sdata=09cNlF5%2FzvJwIqAT9tsixh82aoJEna6h1voR7C3S%2BXc%3D&reserved=0 > > > 2. Download the KEYS file from > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585976838&sdata=Dad6aN4TzDgVmoRDrAxDvTxEO6qFyVYGkHlVvK6ez%2Bo%3D&reserved=0 > > > 3. Import the GPG keys by running `gpg --import KEYS`. > > > 4. Verify the signature by running `gpg --verify > > > apache-parquet-1.11.0.tar.gz.asc`. It should say "Good > signature", > > the > > > warning about the key not being trusted can be ignored. > > > 5. Extract the source code: `tar xzf > apache-parquet-1.11.0.tar.gz`. > > > 6. Change to the source directory: `cd > apache-parquet-1.11.0`. > > > 7. Make sure that you have a thrift compiler and that you > have the > > > correct > > > version by issuing `thrift -version`. It should print > "Thrift version > > > 0.12.0". If you don't have it, you need to install it in a > manner > > that > > > is > > > specific to your system (for example, `sudo apt install > > > thrift-compiler` or > > > `sudo yum install thrift-compiler`). If your package > manager only > > > offers an > > > older version, you may still use that with the workaround > shown > > below. > > > 8. Test the release by issuing `mvn clean verify`. If you > have an > > older > > > thrift compiler, for example 0.9.3, you have to issue `mvn > > > -Dformat.thrift.version=0.9.3 -Dthrift.version=0.9.3 clean > install` > > > instead. This may or may not work depending on how old > your thrift > > > version > > > is (>= 0.9.0 should work). Running this command can take > around 15 > > > minutes > > > and should print "BUILD SUCCESS" at the end (followed by a > few extra > > > lines). > > > > > > Br, > > > > > > Zoltan > > > > > > On Thu, Mar 21, 2019 at 7:58 PM Wes McKinney < > [email protected]> > > > wrote: > > > > > > > Are there any instructions written down about how to > verify this > > > release? > > > > > > > > On Wed, Mar 20, 2019 at 8:50 AM Zoltan Ivanfi > > > <[email protected]> > > > > wrote: > > > > > > > > > > +1 (binding) > > > > > > > > > > signature matches > > > > > git hash matches the git tag > > > > > source tarball matches the git tag > > > > > unit tests and integration tests pass > > > > > > > > > > On Tue, Mar 19, 2019 at 3:00 PM Gabor Szadovszky < > > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Dear Parquet Users and Developers, > > > > > > > > > > > > I propose the following RC to be released as the > official > > Apache > > > > > > Parquet 1.11.0 release: > > > > > > > > > > > > The commit id is > 9756b0e2b35437a09716707a81e2ac0c187112ed > > > > > > * This corresponds to the tag: apache-parquet-1.11.0 > > > > > > * > > > > > > > > > > > > > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fapache%2Fparquet-mr%2Ftree%2F9756b0e2b35437a09716707a81e2ac0c187112ed&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585976838&sdata=ZKYiD%2F%2FDwP7zDadpxBvtSF4T%2FjXCdwIsAT8w%2FBQcxAQ%3D&reserved=0 > > > > > > > > > > > > The release tarball, signature, and checksums are > here: > > > > > > * > > > > > > > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2Fapache-parquet-1.11.0-rc6%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585986852&sdata=v9PAI9BCY0HTGld7Zlb%2F8gJYbQH%2Fy1WmOw87ZzWymUU%3D&reserved=0 > > > > > > > > > > > > You can find the KEYS file here: > > > > > > * > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fformatdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585986852&sdata=VqkSY5roEqYKo%2B09o6Wh2NZQWKkdbAKDvtWIdbLjccE%3D&reserved=0 > > > > < > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdist.apache.org%2Frepos%2Fdist%2Fdev%2Fparquet%2FKEYS&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585986852&sdata=2VCbJrdrUE1g3dN3VUqN%2FAq1Gt6CcBAGmGfx3CjlauI%3D&reserved=0 > > > > > > > > > > > > > > > > Binary artifacts are staged in Nexus here: > > > > > > * > > > > > > > > > > > > > > > > > > > > > > https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Frepository.apache.org%2Fcontent%2Fgroups%2Fstaging%2Forg%2Fapache%2Fparquet%2Fparquet%2F1.11.0%2F&data=02%7C01%7CAndy.Grove%40rms.com%7C0cd3c43dceca4c34f53108d6b8461b90%7Cd43fb8a804da4990b86cc4ba9ba4511f%7C0%7C0%7C636899008585986852&sdata=Ff9wOSvCIG25l9N89STbYJfod7Oo0cb9pWqfOlaRD9s%3D&reserved=0 > > > > > > > > > > > > This release includes the following new features: > > > > > > - PARQUET-1201 - Column indexes > > > > > > - PARQUET-1253 - Support for new logical type > representation > > > > > > - PARQUET-1381 - Add merge blocks command to > parquet-tools > > > > > > - PARQUET-1388 - Nanosecond precision time and > timestamp - > > > parquet-mr > > > > > > > > > > > > The release also includes bug fixes, including: > > > > > > - PARQUET-1472: Dictionary filter fails on > > FIXED_LEN_BYTE_ARRAY. > > > > > > - PARQUET-1510: Fix notEq for optional columns with > null > > values. > > > > > > - PARQUET-1533: TestSnappy() throws OOM exception > with > > > Parquet-1485 > > > > change > > > > > > - PARQUET-1531: Page row count limit causes empty > pages to be > > > written > > > > from > > > > > > MessageColumnIO > > > > > > - PARQUET-1544: Possible over-shading of modules > > > > > > > > > > > > The following change has been reverted so it is not > part of any > > > public > > > > > > release: > > > > > > - PARQUET-1381: Add merge blocks command to > parquet-tools > > > > > > > > > > > > Please download, verify, and test. The vote will be > open for at > > > least > > > > 72 > > > > > > hours. > > > > > > > > > > > > Thanks, > > > > > > Gabor > > > > > > > > > > > > > > > > > > > > > > > > > >
