Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-14 Thread via GitHub
XorSum commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2054066305 @dongjoon-hyun. Thank you. I've edited the description according to your comments. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-14 Thread via GitHub
dongjoon-hyun commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2054061782 1. Could you put that info into the PR description first to get further reviews? 2. We need both `spark.driver.extraLibraryPath` and `spark.executor.extraLibraryPath` in

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-13 Thread via GitHub
XorSum commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2053870360 The manual test steps to avoid native zStandard library not available error: 1. a x86_64 linux machine 2. download and extract

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-13 Thread via GitHub
dongjoon-hyun commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2053821293 Gentle ping, @XorSum . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] [SPARK-47829] Text Datasource supports Zstd compression codec [spark]

2024-04-12 Thread via GitHub
yaooqinn commented on PR #46026: URL: https://github.com/apache/spark/pull/46026#issuecomment-2051928714 Is the Hadoop native zstd library still missing? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above