JigaoLuo commented on issue #7723: URL: https://github.com/apache/arrow-rs/issues/7723#issuecomment-3026800672
Thanks for the insights! I’m passionate about disaggregated architectures and GPUs, which led me to pursue my PhD at TUDa after finishing TUM. The idea began around 2020 when I first read about memory disaggregation via RDMA. Regarding Parquet in disaggregated systems: I’d add to your idea that data tiers could evolve from cold (unoptimized Parquet) to warm (read-optimized Parquet) to hot-liquid (LiquidCache format, **it is so hot to melt into liquid**) to scalding (in-memory formats). Promotion between tiers should be driven by access frequency at least —there’s no need to invest in rewriting Parquet files that are never queried. This approach aligns with cost-benefit principles in cloud object storage and also data management. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
