Hi Stefan,
I think for the java processors the order should also be kept. But I am not sure if it is good to split up the array into multiple events and merge them into a single one within a processor. I would rather recommend a specific adapter that is able to interpret your format to ingest the events into StreamPipes. What do you think about that? Philipp Von: "Obermeier. Stefan" <[email protected]> Antworten an: "[email protected]" <[email protected]> Datum: Montag, 20. Dezember 2021 um 14:47 An: "[email protected]" <[email protected]> Betreff: AW: Event Schema multiple objects in array [Kafka-Adapter] Hi Philipp, thank you for your answer this helped me a lot. Yes in this case all events have the same properties. Splitting the array to multiple events is a good option for me, since I can merge them in the processor. In this case the temporal order between the different arrays is important. For Kafka this seems to be the case since the events are stored in one partition. Is this the case for MQTT and JSM too? Is it a good idea to relay on the temporal order of events? What do you think about adding an option to the pipeline to say that the ordering should be kept. Defining a partion key is too technical and technology dependent!? Stefan Von: Philipp Zehnder <[email protected]> Gesendet: Montag, 20. Dezember 2021 08:07 An: [email protected] Betreff: Re: Event Schema multiple objects in array [Kafka-Adapter] Hi Sefan, the format ‘Json Array No Key’ expects that there is an array of events , where each event has the same event schema. In your case the array represents the whole event, right? Unfortunately, we do not yet support such a format. There are two options to integrate this, you can either write a specific adapter for this data source or you can add a new format for this kind of data. I think the first option would be the simpler one, since the format API needs some refactoring. Do all events have the same properties (e.g. timestamp, start, maintenance, error)? Because our event schema expects a fixed set of properties. If this is not the case, we should discuss how we can solve this. Philipp Von: "Obermeier. Stefan" <[email protected]> Antworten an: "[email protected]" <[email protected]> Datum: Sonntag, 19. Dezember 2021 um 23:32 An: "[email protected]" <[email protected]> Betreff: Event Schema multiple objects in array [Kafka-Adapter] Hi all, I have some data like this in a Kafka topic [ { "timestamp":"1610989200", "uid":"a555fda6-5743-11eb-ae93-0242ac130002", "dataType":"string" }, { "start":"true", "uid":"9778f090-5743-11eb-ae93-0242ac130002", "dataType":"bool" }, { "maintenance":"true", "uid":"9f8cc73e-5743-11eb-ae93-0242ac130002", "dataType":"bool" }, { "error":"false", "uid":"a7f7fda6-5743-11eb-ae93-0242ac130002", "dataType":"bool" } ] An array with some objects. I consume this kind of data with the existing Kakfa-Adapter. If I use ‘Json Array No Key’ as format in the configuration wizard, I can access the first object of the array. Is there a way to access the other objects? BR, Stefan SEEBURGER AG Vorstand/SEEBURGER Executive Board: Sitz der Gesellschaft/Registered Office: Axel Haas, Michael Kleeberg, Axel Otto, Dr. Martin Kuntz, Matthias Feßenbecker Edisonstr. 1 D-75015 BrettenVorsitzende des Aufsichtsrats/Chairperson of the SEEBURGER Supervisory Board: Tel.: 07252 / 96 - 0Prof. Dr. Simone Zeuchner Fax: 07252 / 96 - 2222 Internet: http://www.seeburger.deRegistergericht/Commercial Register: e-mail: [email protected] 240708 Mannheim Dieses E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder Meinung der SEEBURGER AG dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser E-Mail ist strengstens untersagt. Weder die SEEBURGER AG noch der Absender (Obermeier. Stefan) übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren Anhänge auf Viren zu prüfen. This email is intended only for the recipient(s) to whom it is addressed. This email may contain confidential material that may be protected by professional secrecy. Any fact or opinion contained, or expression of the material herein, does not necessarily reflect that of SEEBURGER AG. If you are not the addressee or if you have received this email in error, any use, publication or distribution including forwarding, copying or printing is strictly prohibited. Neither SEEBURGER AG, nor the sender (Obermeier. Stefan) accept liability for viruses; it is your responsibility to check this email and its attachments for viruses. SEEBURGER AG Vorstand/SEEBURGER Executive Board: Sitz der Gesellschaft/Registered Office: Axel Haas, Michael Kleeberg, Axel Otto, Dr. Martin Kuntz, Matthias Feßenbecker Edisonstr. 1 D-75015 BrettenVorsitzende des Aufsichtsrats/Chairperson of the SEEBURGER Supervisory Board: Tel.: 07252 / 96 - 0Prof. Dr. Simone Zeuchner Fax: 07252 / 96 - 2222 Internet: http://www.seeburger.deRegistergericht/Commercial Register: e-mail: [email protected] 240708 Mannheim Dieses E-Mail ist nur für den Empfänger bestimmt, an den es gerichtet ist und kann vertrauliches bzw. unter das Berufsgeheimnis fallendes Material enthalten. Jegliche darin enthaltene Ansicht oder Meinungsäußerung ist die des Autors und stellt nicht notwendigerweise die Ansicht oder Meinung der SEEBURGER AG dar. Sind Sie nicht der Empfänger, so haben Sie diese E-Mail irrtümlich erhalten und jegliche Verwendung, Veröffentlichung, Weiterleitung, Abschrift oder jeglicher Druck dieser E-Mail ist strengstens untersagt. Weder die SEEBURGER AG noch der Absender (Obermeier. Stefan) übernehmen die Haftung für Viren; es obliegt Ihrer Verantwortung, die E-Mail und deren Anhänge auf Viren zu prüfen. This email is intended only for the recipient(s) to whom it is addressed. This email may contain confidential material that may be protected by professional secrecy. Any fact or opinion contained, or expression of the material herein, does not necessarily reflect that of SEEBURGER AG. If you are not the addressee or if you have received this email in error, any use, publication or distribution including forwarding, copying or printing is strictly prohibited. Neither SEEBURGER AG, nor the sender (Obermeier. Stefan) accept liability for viruses; it is your responsibility to check this email and its attachments for viruses.
