kbendick commented on issue #2676: URL: https://github.com/apache/iceberg/issues/2676#issuecomment-854604098
@rymurr and I had some initial discussion in this ticket https://github.com/apache/iceberg/pull/2674, where there are also links to some of the stuff that he and some of the Nessie folks have put together that runs a notebook for the basics. So there will likely be some overlap, but at least from a first pass, there's less overlap than one might think. Mine runs entirely locally via docker-compose (HDFS, Hive Metastore backed by Postgres, Spark, Zookeeper, and WIP on Kafka for streaming) and has been really useful for mounting in my local iceberg codebase to test things without having to publish jars beyond maven local repo. Theirs also uses docker-compose, but runs via notebooks (which I think is great). There's is more Nessie focused, but has the added advantage of being able to run on Google code lab without the need for your own infra as local docker development can definitely slow down a laptop. Please feel free to comment here and I will prioritize pushing what I have (I just need to remove some internal stuff) so that we can then iterate from there or regroup and come up with a better design. I'm sure given the large number of components that this will be an on going effort. I would love to champion this as I do a lot of container work already. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
