jiayuasu commented on pull request #536:
URL: https://github.com/apache/incubator-sedona/pull/536#issuecomment-905314053
@swamirishi
Thank you again for your patience! I traveled back to WSU and the new
semester just started. So I was kind of overwhelmed by many things on my plate.
Now I am ready to work with you!
I personally are very excited about this PR but I think the reason why
several committers hesitate to approve this PR is that
(1) it does not have a detailed proposal to explain the proposed file
structure and how it works. We know the overall idea of having a geospatial
Parquet file and it is an awesome idea for sure. But it is unclear to us how
this is achieved in this PR.
(2) it contains 44 file changes which are too many to be reviewed by a
human... It touches several critical places in Sedona. For instance, it adds a
new module called Parquet, adds some Sedona exceptions, change some
dependencies. More importantly, it changes the core/SpatialRDD.java file which
will affect many places.
Therefore, I believe here are some action items
(1) would you please add a Sedona website doc in this PR to explain
* The usage of Sedona Parquet reader and writer
* The structure of the proposed geo Parquet file, especially the
metadata structure
* the algorithm: how it skips irrelevant data chunks by comparing the
spatial query predicate and Parquet metadata
You can put this doc at https://sedona.apache.org/tutorial/ as a programming
guide for geospatial Parquet
(2) for the file changes themselves
* please fix the comments raised by @Imbruced
* Please address my comment for SpatialRDD change
* please remove all whitespace file changes, and changes that just
include "re-formatting the code"
Once these changes + doc are done and we fully understand what this PR is
doing exactly, I will approve this PR. Let the customers give it a try. I
believe once this PR is accepted, your follow-up PRs for the Parquet file will
quickly pass our review.
Thank you again for your contribution. We really appreciate your
contribution!
Jia Yu
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
