littleDrew opened a new issue, #13438:
URL: https://github.com/apache/iceberg/issues/13438
### Query engine
Spark engine
### Question
hi @pvary, it seems that you are working intergrate lance into iceberg, can
you show the detail implementation plan/solution.
Here i want to have a discuss or know previous analysis about select which
way to intergrate with iceberg:
1st way: use lance file format to intergrate with iceberg, thus like
following implementation by @westonpace
2st way: use lance table format to intergrate with iceberg
Here i thought, considering about performance issue, it seems the 2st
way(use lance table format) will be better for good performance
Thus want to have a dissuss as for implementation way. if use 1st way, can
we achive good performance, or have we evalute the pros and cons of both
implementation way.
public enum FileFormat {
PUFFIN("puffin", false),
ORC("orc", true),
PARQUET("parquet", true),
AVRO("avro", true),
LANCE("lance", true) // add lance as an file format of iceberg instead of
table format
}
https://github.com/westonpace/iceberg/commit/84bf5c53bc5ea19101bb7f21d72f24666c2b3804
Hope we have a disscus for this together, as i also have noticed lance have
a good performance in proccessing/reading multi-dimentional data. @dacort
@electrum @martint @massdosage @sullis
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
