Hi Philipp, Thank you for your answer, me and the other Michele tried the File Stream and not File Set but I think that isn't a problem because I've tried on our server that is running via docker-compose on the code tree and had problems, not starting the adapter. Today I've started SP using the installer and so it downloaded all the images not using those compiled on the code, and it actually works, so our problem is on compiling and maybe retrieving the right branch/tag of the code, I guess. I think the right tag or branch to checkout is something that confuses me, sorry, my lack. I retrieved the code cloning it from GIT and another time downloading it as a zip file, but I would prefer the first way in order to be able to patch it in case of bugs. Thank you again for your help guys! Regards, Michele
Michele Canteri _________________________________________________________________ GFT Italia S.r.l. VIA SILE 18 / floor 4 20139 Milano, Italy T +39 02 58158503 F +39 02 58158621 [email protected] www.gft.com/it<http://www.gft.com/it> https://blog.gft.com https://www.linkedin.com/company/gft-group/ www.twitter.com/gft_it <http://www.twitter.com/gft_it> [cid:ActoForTheFuture_GFT@1,2x_68f73108-d76e-4497-aa5c-ff8a5693ce57.png]<https://www.gft.com/it/it/about-us/Sustainability> Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy) VAT Code and number of registration with the Companies' Register of Milan: 00819200478 Corporate capital: Euro 1,000,000 fully paid-in Company subject to the direction and coordination activity of GFT Technologies SE Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda. In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company. _________________________________________________________________ From: Philipp Zehnder <[email protected]> Sent: mercoledì 29 giugno 2022 18:13 To: [email protected] Subject: AW: StreamPipes File Stream Creation Problem [EXTERNAL EMAIL] This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe. Hi Michele, thanks for providing sample data. I was able to create an adapter with it on my machine. I used the "File Set" adapter with the format "Json" -> "Array Field" -> "Key: data". For the timestamp I selected the field "date_unix_ts". I could start the adapter, stored the results in the data lake and visualized them in the data explorer. Does the error occur on your systems with the same configurations? One minor comment: I saw that your values in the data stream have different units. Depending on your use case, my suggestion would be to either create one data streams for each property, or if it is possible to create a bigger event with all the values at one point in time. This often eases the analytics part of the event streams. Cheers, Philipp Von: Iorio, Michele <[email protected]<mailto:[email protected]>> Datum: Montag, 27. Juni 2022 um 18:26 An: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Betreff: RE: StreamPipes File Stream Creation Problem Hi Philipp, in attachment you will find a sample of data to try to recreate the problem. Regarding version 0.69.1-SNAPSHOT, do you have any other information? Thanks Michele Michele Iorio _________________________________________________________________ GFT Italia S.r.l. Via A. Rimassa, 51/2 16129 Genova, Italy T +39 010 583158 [email protected]<mailto:[email protected]> www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=iCCrTklKM4OzTKZBgFpvUryOn0Y%2FYlegBUTpxIPGRXI%3D&reserved=0> https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=vqGlP4r1%2BX4DbSadWDmVWyURz77MhxJ5a6UG2onRau0%3D&reserved=0> https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=l7gOWRftGrZCtq8pU9ZGO8la4X0l9cscLPcN3uNQPwg%3D&reserved=0> www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=YFGDXOHDDpGEuVnlUCoAVliWZLgvbKryMBD6P0DA4%2Fg%3D&reserved=0> [cid:[email protected]]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003491799%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=3w7LrTeJocuKFb83g%2FN0LJcG%2FERHfomlccuSW3635Bg%3D&reserved=0> Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy) VAT Code and number of registration with the Companies' Register of Milan: 00819200478 Corporate capital: Euro 1,000,000 fully paid-in Company subject to the direction and coordination activity of GFT Technologies SE Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda. In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company. _________________________________________________________________ From: Iorio, Michele Sent: venerdì 24 giugno 2022 16:24 To: [email protected]<mailto:[email protected]> Subject: RE: StreamPipes File Stream Creation Problem Hi Philipp, the problem occurs even if I upload a file with 100 entries. The situation is strange, because sometimes it works with the small file, but other times it gives the same error. At the moment I'm checking if I can send you a small sample of data to reproduce the problem. Regarding the last part of Federico's email, why is version 0.69.1-SNAPSHOT no longer installed? Could this be the cause of the problem? Thanks Michele From: Philipp Zehnder <[email protected]<mailto:[email protected]>> Sent: giovedì 23 giugno 2022 19:53 To: [email protected]<mailto:[email protected]> Subject: AW: StreamPipes File Stream Creation Problem Non si ricevono spesso messaggi di posta elettronica da [email protected]<mailto:[email protected]>. Informazioni sul perché è importante<https://aka.ms/LearnAboutSenderIdentification> [EXTERNAL EMAIL] This email originated from outside of the organization or was sent unauthenticated. Do not click links or open attachments unless you confirm the sender and know the content is safe. Hi Frederico, is there anything logged in the docker container running the adapter? Can you try it with a very small file containing only a few events? If you want you can also send me a small sample, so I can try to recreate the problem. Cheers, Philipp Von: Rullo, Federico <[email protected]<mailto:[email protected]>> Datum: Donnerstag, 23. Juni 2022 um 17:44 An: [email protected]<mailto:[email protected]> <[email protected]<mailto:[email protected]>> Betreff: StreamPipes File Stream Creation Problem Hello, I am trying to upload a Json file of 9.2MB of size, containing 50000 entries, to create a datastream. I previously uploaded a smaller version of the file, and everything went fine, then I had to reinstall SP, and uploading the same file, gave me the same error I am having right now. The error is that it creates the DataStream, but no data is being produced. To create the DataStream, I am using the Data Stream creator, and a JSON Array key for the file format. What can be the problem in your opinion? Also, I saw that using the rel/0.69.0 branch yields 0.69.0 as the version and not 0.69.1-SNAPSHOT anymore, the previous installation of SP I had was using the latter, do you think this can be the problem? Thanks in advance for your reply, Best, Federico. Federico Rullo _________________________________________________________________ GFT Italia S.r.l. Via A. Rimassa, 51/2 16129 Genova, Italy T +39 025 8158169 [email protected]<mailto:[email protected]> www.gft.com/it<https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.gft.com%2Fit&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=nziW761WhSbBY1yc9gvjs4U0GssKnsaHsTbxuPQanHs%3D&reserved=0> https://blog.gft.com<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fblog.gft.com%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=AYUzWAd582X6otG0k7kqiOm4Dr0pL73o%2BS4ilhKp2aU%3D&reserved=0> https://www.linkedin.com/company/gft-group/<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fcompany%2Fgft-group%2F&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BbnQ2jQgASKCc864tR7GwW4gMukkODo4Xq0edEDX5gA%3D&reserved=0> www.twitter.com/gft_it <https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.twitter.com%2Fgft_it%2520&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=BkLAfnbDjcbP7kAHalTpTlQgaMI%2BLHt%2Bgo0N%2B3%2FDAhs%3D&reserved=0> [cid:[email protected]]<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gft.com%2Fit%2Fit%2Fabout-us%2FSustainability&data=05%7C01%7CMichele.Canteri%40gft.com%7C5a1c281076614d1e74d408da59ea4722%7C55a2bc67aec14ad29a9c5b2457b91dcd%7C0%7C0%7C637921160003647251%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=%2BAKCBcMct1JE0DW8ty%2FeQXUswiQ6bpJnNFseeshxigY%3D&reserved=0> Registered office: GFT Italia S.r.l., Via Sile 18, 20139 Milan (Italy) VAT Code and number of registration with the Companies' Register of Milan: 00819200478 Corporate capital: Euro 1,000,000 fully paid-in Company subject to the direction and coordination activity of GFT Technologies SE Ai sensi del Regolamento UE 2016/679 (GDPR) e delle Linee Guida per posta elettronica ed Internet del Garante Privacy si rende noto che le informazioni contenute nella presente comunicazione e i relativi allegati possono essere riservate e sono, comunque, destinate esclusivamente alle persone o alla Società sopraindicati. Le stesse non sono da considerarsi comunicazioni personali, quindi eventuali risposte potranno essere conosciute da persone appartenenti all'azienda. In accordance with Regulation EU 2016/679 (General Data Protection Regulation) and Guidelines Applying to the Use of E-Mails and the Internet in the Employment Context of Italian Data Protection Authority it is announced that the information in this communication (which includes any documents with it) is confidential and meant only for above-mentioned persons and/or Companies; the information is not personal therefore, It is intended for the addressee only and any replies will be known by people belonging to the company. _________________________________________________________________
