tenthe commented on issue #2106: URL: https://github.com/apache/streampipes/issues/2106#issuecomment-1807662704
Hi @muyangye thank you for initiating work on this issue. I agree we need a better solution for this. However, I am not quite sure yet what the best solution might be. The current challenge is that simply removing properties removes the user's ability to persist them. On the other hand, using a special encoding for arrays and serializing them as strings raises concerns about the use of the data for downstream processes such as visualization. To move forward, it would be beneficial to collectively define the user's goal when dealing with array data. I see two potential options: **Option 1:** Enforce users to transform arrays into an alternative representation before storing the data. **Option 2:** Modify the representation of an array to a string format for storage in Influx. And as @bossenti stated, we need to consider the side effects on other functions (such as the automatic pipeline generation) I'm open to both options but would also appreciate any additional ideas or insights you might have. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
