mattcasters opened a new pull request, #8607: URL: https://github.com/apache/hop/pull/8607
Avro Record fields in sampled rows could not be read back from an execution information location. Avro File Input and Kafka Consumer leave the schema off the value metadata, and the row codec wrote a raw Avro encoder onto the shared stream. Reading that blob threw `An Avro schema is needed to read a GenericRecord from an input stream` inside `rowsBinaryGzipBase64Encoded`. Loading the cache entry then failed, so drilling from a workflow into a stored Kafka pipeline never opened. Each Avro value now stores its writer schema and a length-prefixed datum. A column before or after the Avro field stays aligned, and a field with no metadata schema round-trips. Data profiling compares an Avro column by the record text it already shows, and it copies the sample row instead of keeping the live buffer. Neo4j stores the same bytes, because a `GenericRecord` is not a property value. An execution already saved with the old layout still opens. The schema was never written, so those Avro cells cannot be rebuilt. Complete rows read before the bad value are kept, and the log says the rest of that sample set was dropped. fixes #8600 ------------------------ Thank you for your contribution! Follow this checklist to help us incorporate your contribution quickly and easily: - [ ] Run `mvn clean install apache-rat:check` to make sure basic checks pass. A more thorough check will be performed on your pull request automatically. - [x] If you have a group of commits related to the same change, please squash your commits into one and force push your branch using `git rebase -i`. - [x] Mention the appropriate issue in your description (for example: `addresses #123`), if applicable. To make clear that you license your contribution under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) you have to acknowledge this by using the following check-box. - [x] I hereby declare this contribution to be licensed under the [Apache License Version 2.0, January 2004](http://www.apache.org/licenses/LICENSE-2.0) - [ ] In any other case, please file an [Apache Individual Contributor License Agreement](https://www.apache.org/licenses/icla.pdf). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
