Hi Dominik, hi Johannes, first, thank you both for your thoughts on the Data Lake API adaptions and improvements.
@Dominik: Great, since you're already working on harmonizing authentication, we can focus primarily on the REST interface and endpoint definition here for now. Using query parameters instead of long paths should definitely increase the user convenience. That's a good point. I started this morning with working on a concept with adapted and simplified endpoint definitions. If you want to provide input there, Dominik, you are very welcome to do so. Then I can incorporate this one right away. Otherwise I can make a first draft, which we can then discuss and adapt in accordance to your input. Regards, Daniel -----Ursprüngliche Nachricht----- Von: Dominik Riemer <rie...@apache.org> Gesendet: Montag, 26. April 2021 09:27 An: dev@streampipes.apache.org Betreff: RE: Review of Data Lake Sink and REST API Hi Daniel, thanks for bringing this up! I added some comments, in my opinion the redesign of the resource endpoint is the most important thing right now - the authentication concept is something I'm currently working on within the scope of STREAMPIPES-319, where I'm first harmonizing service discovery and one of the next step is to inject an instance of the StreamPipes client into the EventProcessorRuntimeContext, so that pipeline elements can easily request data from the core APIs. Dominik -----Original Message----- From: Johannes Tex <t...@apache.org> Sent: Saturday, April 24, 2021 4:40 PM To: dev@streampipes.apache.org Subject: Re: Review of Data Lake Sink and REST API Hi Daniel, I added some comments. Greetings Johannes On 2021/04/23 08:09:54, Daniel Ebi <e...@fzi.de> wrote: > Hi all, > > I have spent the last few days reviewing the implementation of the Data Lake > sink and in particular the corresponding REST API. The goal was to identify > open TODOs and some possible adaptations or extensions for the REST API. > I've created a new page in Confluence for the results of the review, so that > we can discuss further ideas there in the comments [1]. > > The page is thematically divided into two parts. First, I have put together a > concise description of the status quo of the sink implementation and the > specification of the REST API including open TODOs. Then follows a part with > the ideas for possible adaptations and extensions of the interface. > > It would be great if you leave me feedback and your ideas for adjustments of > the sink / REST API there in the comments, so that we can discuss the ideas > together. Because in a next step I would like to work out a draft of the API > with updated endpoint definitions. > > Thanks and regards, > Daniel > > > [1] https://cwiki.apache.org/confluence/display/STREAMPIPES/Data+Lake+Sink > >
