Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Thanks Danny! I've written a FLIP doc and will send out a new email thread. On Wed, Mar 1, 2023 at 2:43 PM Danny Cranmer wrote: > Hey Daniel, I have granted you access. Please try again. > > Thanks, > Danny > > On Wed, Mar 1, 2023 at 6:48 PM Daniel Collins > > wrote: > > > Hello, > > > > It appears that I still do not have the ability to create Confluence > pages? > > Could someone possibly grant me access? > > > > -Daniel > > > > On Tue, Feb 28, 2023 at 10:20 AM Daniel Collins > > wrote: > > > > > Thanks! My Jira username is `dpcollins-google`. > > > > > > -Daniel > > > > > > On Tue, Feb 28, 2023 at 10:19 AM Martijn Visser < > > martijnvis...@apache.org> > > > wrote: > > > > > >> Hi Daniel, > > >> > > >> If you can provide your Jira username, those permissions can be > granted. > > >> > > >> Best regards, > > >> > > >> Martijn > > >> > > >> On Tue, Feb 28, 2023 at 3:13 PM Daniel Collins > > >> > > >> wrote: > > >> > > >> > Hello, > > >> > > > >> > Absent any comments, I was attempting to create a connector FLIP as > > >> > described above, however it appears that I do not have permissions > to > > >> edit > > >> > confluence to do so. Can someone please add access for me? > > >> > > > >> > -Daniel > > >> > > > >> > On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins < > dpcoll...@google.com > > > > > >> > wrote: > > >> > > > >> > > Hello Martijn, > > >> > > > > >> > > Thanks for the redirect! > > >> > > > > >> > > > The process for contributing a connector is to create a > Connector > > >> FLIP > > >> > > [1], which needs to be discussed and voted on in the Dev mailing > > list > > >> [2] > > >> > > > > >> > > I don't mind doing this, but would like to get a read of the room > > >> before > > >> > > doing so. If people object to it, then it probably doesn't make > > sense. > > >> > > > > >> > > > One thing in particular is who can help with the maintenance of > > the > > >> > > connector: will there be more volunteers who can help with bug > > fixes, > > >> new > > >> > > features etc. > > >> > > > > >> > > In this case, the product team is willing to handle feature > requests > > >> and > > >> > > bug reports for the connector, as we currently do for our beam and > > >> spark > > >> > > connectors. I don't know if there is any mechanism for sending > > emails > > >> > when > > >> > > jira tickets with a specific tag are opened? But if there is I can > > >> ensure > > >> > > that any tickets get routed to a member of my team to look at. > > >> > > > > >> > > What would people's thoughts be about this? > > >> > > > > >> > > -Daniel > > >> > > > > >> > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser < > > >> martijnvis...@apache.org > > >> > > > > >> > > wrote: > > >> > > > > >> > >> Hi Daniel, > > >> > >> > > >> > >> Thanks for reaching out. Keep in mind that you weren't subscribed > > to > > >> the > > >> > >> Flink Dev mailing list, I've just pushed this through the mailing > > >> list > > >> > >> moderation. > > >> > >> > > >> > >> The process for contributing a connector is to create a Connector > > >> FLIP > > >> > >> [1], > > >> > >> which needs to be discussed and voted on in the Dev mailing list > > [2]. > > >> > One > > >> > >> thing in particular is who can help with the maintenance of the > > >> > connector: > > >> > >> will there be more volunteers who can help with bug fixes, new > > >> features > > >> > >> etc. As we've seen with the current PubSub connector, that's > > already > > >> > quite > > >> > >> hard: it's currently lacking volunteers overall. I do recall a > > >> proposal > > >> > to > > >> > >> contribute a PubSub Lite connector a while back, but that > > ultimately > > >> was > > >> > >> not followed through. > > >> > >> > > >> > >> Best regards, > > >> > >> > > >> > >> Martijn > > >> > >> > > >> > >> [1] > > >> > >> > > >> > > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template > > >> > >> [2] > https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > > >> > >> > > >> > >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins > > >> > >> > > >> > >> wrote: > > >> > >> > > >> > >> > Hello flink devs, > > >> > >> > > > >> > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite > > >> > streaming > > >> > >> > system, which is a lower cost streaming data system with > > semantics > > >> > more > > >> > >> > similar to OSS streaming solutions. I've authored a source and > > sink > > >> > >> > connector for flink and load tested it at GiB/s scale- what > would > > >> be > > >> > the > > >> > >> > process for contributing this to flink? > > >> > >> > > > >> > >> > I've opened a JIRA to do this > > >> > >> > > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229 > > , > > >> if > > >> > >> this > > >> > >> > seems like a reasonable thing to do could someone assign it to > > me? > > >> > >> > > > >> > >> > The code for the connector currently lives here > > >> > >> > https://github.com/googleapis/java-pubsublite-flink, I
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hey Daniel, I have granted you access. Please try again. Thanks, Danny On Wed, Mar 1, 2023 at 6:48 PM Daniel Collins wrote: > Hello, > > It appears that I still do not have the ability to create Confluence pages? > Could someone possibly grant me access? > > -Daniel > > On Tue, Feb 28, 2023 at 10:20 AM Daniel Collins > wrote: > > > Thanks! My Jira username is `dpcollins-google`. > > > > -Daniel > > > > On Tue, Feb 28, 2023 at 10:19 AM Martijn Visser < > martijnvis...@apache.org> > > wrote: > > > >> Hi Daniel, > >> > >> If you can provide your Jira username, those permissions can be granted. > >> > >> Best regards, > >> > >> Martijn > >> > >> On Tue, Feb 28, 2023 at 3:13 PM Daniel Collins > >> > >> wrote: > >> > >> > Hello, > >> > > >> > Absent any comments, I was attempting to create a connector FLIP as > >> > described above, however it appears that I do not have permissions to > >> edit > >> > confluence to do so. Can someone please add access for me? > >> > > >> > -Daniel > >> > > >> > On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins > > >> > wrote: > >> > > >> > > Hello Martijn, > >> > > > >> > > Thanks for the redirect! > >> > > > >> > > > The process for contributing a connector is to create a Connector > >> FLIP > >> > > [1], which needs to be discussed and voted on in the Dev mailing > list > >> [2] > >> > > > >> > > I don't mind doing this, but would like to get a read of the room > >> before > >> > > doing so. If people object to it, then it probably doesn't make > sense. > >> > > > >> > > > One thing in particular is who can help with the maintenance of > the > >> > > connector: will there be more volunteers who can help with bug > fixes, > >> new > >> > > features etc. > >> > > > >> > > In this case, the product team is willing to handle feature requests > >> and > >> > > bug reports for the connector, as we currently do for our beam and > >> spark > >> > > connectors. I don't know if there is any mechanism for sending > emails > >> > when > >> > > jira tickets with a specific tag are opened? But if there is I can > >> ensure > >> > > that any tickets get routed to a member of my team to look at. > >> > > > >> > > What would people's thoughts be about this? > >> > > > >> > > -Daniel > >> > > > >> > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser < > >> martijnvis...@apache.org > >> > > > >> > > wrote: > >> > > > >> > >> Hi Daniel, > >> > >> > >> > >> Thanks for reaching out. Keep in mind that you weren't subscribed > to > >> the > >> > >> Flink Dev mailing list, I've just pushed this through the mailing > >> list > >> > >> moderation. > >> > >> > >> > >> The process for contributing a connector is to create a Connector > >> FLIP > >> > >> [1], > >> > >> which needs to be discussed and voted on in the Dev mailing list > [2]. > >> > One > >> > >> thing in particular is who can help with the maintenance of the > >> > connector: > >> > >> will there be more volunteers who can help with bug fixes, new > >> features > >> > >> etc. As we've seen with the current PubSub connector, that's > already > >> > quite > >> > >> hard: it's currently lacking volunteers overall. I do recall a > >> proposal > >> > to > >> > >> contribute a PubSub Lite connector a while back, but that > ultimately > >> was > >> > >> not followed through. > >> > >> > >> > >> Best regards, > >> > >> > >> > >> Martijn > >> > >> > >> > >> [1] > >> > >> > >> > > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template > >> > >> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > >> > >> > >> > >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins > >> > >> > >> > >> wrote: > >> > >> > >> > >> > Hello flink devs, > >> > >> > > >> > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite > >> > streaming > >> > >> > system, which is a lower cost streaming data system with > semantics > >> > more > >> > >> > similar to OSS streaming solutions. I've authored a source and > sink > >> > >> > connector for flink and load tested it at GiB/s scale- what would > >> be > >> > the > >> > >> > process for contributing this to flink? > >> > >> > > >> > >> > I've opened a JIRA to do this > >> > >> > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229 > , > >> if > >> > >> this > >> > >> > seems like a reasonable thing to do could someone assign it to > me? > >> > >> > > >> > >> > The code for the connector currently lives here > >> > >> > https://github.com/googleapis/java-pubsublite-flink, I believe > it > >> is > >> > >> > following the FLIP-27 guidelines, but please let me know if I'm > >> > >> > implementing the wrong interfaces. Which repo and in what folder > >> > should > >> > >> I > >> > >> > move this code into? > >> > >> > > >> > >> > -Daniel > >> > >> > > >> > >> > >> > > > >> > > >> > > >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hello, It appears that I still do not have the ability to create Confluence pages? Could someone possibly grant me access? -Daniel On Tue, Feb 28, 2023 at 10:20 AM Daniel Collins wrote: > Thanks! My Jira username is `dpcollins-google`. > > -Daniel > > On Tue, Feb 28, 2023 at 10:19 AM Martijn Visser > wrote: > >> Hi Daniel, >> >> If you can provide your Jira username, those permissions can be granted. >> >> Best regards, >> >> Martijn >> >> On Tue, Feb 28, 2023 at 3:13 PM Daniel Collins >> >> wrote: >> >> > Hello, >> > >> > Absent any comments, I was attempting to create a connector FLIP as >> > described above, however it appears that I do not have permissions to >> edit >> > confluence to do so. Can someone please add access for me? >> > >> > -Daniel >> > >> > On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins >> > wrote: >> > >> > > Hello Martijn, >> > > >> > > Thanks for the redirect! >> > > >> > > > The process for contributing a connector is to create a Connector >> FLIP >> > > [1], which needs to be discussed and voted on in the Dev mailing list >> [2] >> > > >> > > I don't mind doing this, but would like to get a read of the room >> before >> > > doing so. If people object to it, then it probably doesn't make sense. >> > > >> > > > One thing in particular is who can help with the maintenance of the >> > > connector: will there be more volunteers who can help with bug fixes, >> new >> > > features etc. >> > > >> > > In this case, the product team is willing to handle feature requests >> and >> > > bug reports for the connector, as we currently do for our beam and >> spark >> > > connectors. I don't know if there is any mechanism for sending emails >> > when >> > > jira tickets with a specific tag are opened? But if there is I can >> ensure >> > > that any tickets get routed to a member of my team to look at. >> > > >> > > What would people's thoughts be about this? >> > > >> > > -Daniel >> > > >> > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser < >> martijnvis...@apache.org >> > > >> > > wrote: >> > > >> > >> Hi Daniel, >> > >> >> > >> Thanks for reaching out. Keep in mind that you weren't subscribed to >> the >> > >> Flink Dev mailing list, I've just pushed this through the mailing >> list >> > >> moderation. >> > >> >> > >> The process for contributing a connector is to create a Connector >> FLIP >> > >> [1], >> > >> which needs to be discussed and voted on in the Dev mailing list [2]. >> > One >> > >> thing in particular is who can help with the maintenance of the >> > connector: >> > >> will there be more volunteers who can help with bug fixes, new >> features >> > >> etc. As we've seen with the current PubSub connector, that's already >> > quite >> > >> hard: it's currently lacking volunteers overall. I do recall a >> proposal >> > to >> > >> contribute a PubSub Lite connector a while back, but that ultimately >> was >> > >> not followed through. >> > >> >> > >> Best regards, >> > >> >> > >> Martijn >> > >> >> > >> [1] >> > >> >> > >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template >> > >> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws >> > >> >> > >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins >> > >> >> > >> wrote: >> > >> >> > >> > Hello flink devs, >> > >> > >> > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite >> > streaming >> > >> > system, which is a lower cost streaming data system with semantics >> > more >> > >> > similar to OSS streaming solutions. I've authored a source and sink >> > >> > connector for flink and load tested it at GiB/s scale- what would >> be >> > the >> > >> > process for contributing this to flink? >> > >> > >> > >> > I've opened a JIRA to do this >> > >> > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, >> if >> > >> this >> > >> > seems like a reasonable thing to do could someone assign it to me? >> > >> > >> > >> > The code for the connector currently lives here >> > >> > https://github.com/googleapis/java-pubsublite-flink, I believe it >> is >> > >> > following the FLIP-27 guidelines, but please let me know if I'm >> > >> > implementing the wrong interfaces. Which repo and in what folder >> > should >> > >> I >> > >> > move this code into? >> > >> > >> > >> > -Daniel >> > >> > >> > >> >> > > >> > >> >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Thanks! My Jira username is `dpcollins-google`. -Daniel On Tue, Feb 28, 2023 at 10:19 AM Martijn Visser wrote: > Hi Daniel, > > If you can provide your Jira username, those permissions can be granted. > > Best regards, > > Martijn > > On Tue, Feb 28, 2023 at 3:13 PM Daniel Collins > > wrote: > > > Hello, > > > > Absent any comments, I was attempting to create a connector FLIP as > > described above, however it appears that I do not have permissions to > edit > > confluence to do so. Can someone please add access for me? > > > > -Daniel > > > > On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins > > wrote: > > > > > Hello Martijn, > > > > > > Thanks for the redirect! > > > > > > > The process for contributing a connector is to create a Connector > FLIP > > > [1], which needs to be discussed and voted on in the Dev mailing list > [2] > > > > > > I don't mind doing this, but would like to get a read of the room > before > > > doing so. If people object to it, then it probably doesn't make sense. > > > > > > > One thing in particular is who can help with the maintenance of the > > > connector: will there be more volunteers who can help with bug fixes, > new > > > features etc. > > > > > > In this case, the product team is willing to handle feature requests > and > > > bug reports for the connector, as we currently do for our beam and > spark > > > connectors. I don't know if there is any mechanism for sending emails > > when > > > jira tickets with a specific tag are opened? But if there is I can > ensure > > > that any tickets get routed to a member of my team to look at. > > > > > > What would people's thoughts be about this? > > > > > > -Daniel > > > > > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser < > martijnvis...@apache.org > > > > > > wrote: > > > > > >> Hi Daniel, > > >> > > >> Thanks for reaching out. Keep in mind that you weren't subscribed to > the > > >> Flink Dev mailing list, I've just pushed this through the mailing list > > >> moderation. > > >> > > >> The process for contributing a connector is to create a Connector FLIP > > >> [1], > > >> which needs to be discussed and voted on in the Dev mailing list [2]. > > One > > >> thing in particular is who can help with the maintenance of the > > connector: > > >> will there be more volunteers who can help with bug fixes, new > features > > >> etc. As we've seen with the current PubSub connector, that's already > > quite > > >> hard: it's currently lacking volunteers overall. I do recall a > proposal > > to > > >> contribute a PubSub Lite connector a while back, but that ultimately > was > > >> not followed through. > > >> > > >> Best regards, > > >> > > >> Martijn > > >> > > >> [1] > > >> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template > > >> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > > >> > > >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins > > >> > > >> wrote: > > >> > > >> > Hello flink devs, > > >> > > > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite > > streaming > > >> > system, which is a lower cost streaming data system with semantics > > more > > >> > similar to OSS streaming solutions. I've authored a source and sink > > >> > connector for flink and load tested it at GiB/s scale- what would be > > the > > >> > process for contributing this to flink? > > >> > > > >> > I've opened a JIRA to do this > > >> > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, > if > > >> this > > >> > seems like a reasonable thing to do could someone assign it to me? > > >> > > > >> > The code for the connector currently lives here > > >> > https://github.com/googleapis/java-pubsublite-flink, I believe it > is > > >> > following the FLIP-27 guidelines, but please let me know if I'm > > >> > implementing the wrong interfaces. Which repo and in what folder > > should > > >> I > > >> > move this code into? > > >> > > > >> > -Daniel > > >> > > > >> > > > > > >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hi Daniel, If you can provide your Jira username, those permissions can be granted. Best regards, Martijn On Tue, Feb 28, 2023 at 3:13 PM Daniel Collins wrote: > Hello, > > Absent any comments, I was attempting to create a connector FLIP as > described above, however it appears that I do not have permissions to edit > confluence to do so. Can someone please add access for me? > > -Daniel > > On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins > wrote: > > > Hello Martijn, > > > > Thanks for the redirect! > > > > > The process for contributing a connector is to create a Connector FLIP > > [1], which needs to be discussed and voted on in the Dev mailing list [2] > > > > I don't mind doing this, but would like to get a read of the room before > > doing so. If people object to it, then it probably doesn't make sense. > > > > > One thing in particular is who can help with the maintenance of the > > connector: will there be more volunteers who can help with bug fixes, new > > features etc. > > > > In this case, the product team is willing to handle feature requests and > > bug reports for the connector, as we currently do for our beam and spark > > connectors. I don't know if there is any mechanism for sending emails > when > > jira tickets with a specific tag are opened? But if there is I can ensure > > that any tickets get routed to a member of my team to look at. > > > > What would people's thoughts be about this? > > > > -Daniel > > > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser > > > wrote: > > > >> Hi Daniel, > >> > >> Thanks for reaching out. Keep in mind that you weren't subscribed to the > >> Flink Dev mailing list, I've just pushed this through the mailing list > >> moderation. > >> > >> The process for contributing a connector is to create a Connector FLIP > >> [1], > >> which needs to be discussed and voted on in the Dev mailing list [2]. > One > >> thing in particular is who can help with the maintenance of the > connector: > >> will there be more volunteers who can help with bug fixes, new features > >> etc. As we've seen with the current PubSub connector, that's already > quite > >> hard: it's currently lacking volunteers overall. I do recall a proposal > to > >> contribute a PubSub Lite connector a while back, but that ultimately was > >> not followed through. > >> > >> Best regards, > >> > >> Martijn > >> > >> [1] > >> > https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template > >> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > >> > >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins > >> > >> wrote: > >> > >> > Hello flink devs, > >> > > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite > streaming > >> > system, which is a lower cost streaming data system with semantics > more > >> > similar to OSS streaming solutions. I've authored a source and sink > >> > connector for flink and load tested it at GiB/s scale- what would be > the > >> > process for contributing this to flink? > >> > > >> > I've opened a JIRA to do this > >> > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, if > >> this > >> > seems like a reasonable thing to do could someone assign it to me? > >> > > >> > The code for the connector currently lives here > >> > https://github.com/googleapis/java-pubsublite-flink, I believe it is > >> > following the FLIP-27 guidelines, but please let me know if I'm > >> > implementing the wrong interfaces. Which repo and in what folder > should > >> I > >> > move this code into? > >> > > >> > -Daniel > >> > > >> > > >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hello, Absent any comments, I was attempting to create a connector FLIP as described above, however it appears that I do not have permissions to edit confluence to do so. Can someone please add access for me? -Daniel On Mon, Feb 27, 2023 at 10:28 AM Daniel Collins wrote: > Hello Martijn, > > Thanks for the redirect! > > > The process for contributing a connector is to create a Connector FLIP > [1], which needs to be discussed and voted on in the Dev mailing list [2] > > I don't mind doing this, but would like to get a read of the room before > doing so. If people object to it, then it probably doesn't make sense. > > > One thing in particular is who can help with the maintenance of the > connector: will there be more volunteers who can help with bug fixes, new > features etc. > > In this case, the product team is willing to handle feature requests and > bug reports for the connector, as we currently do for our beam and spark > connectors. I don't know if there is any mechanism for sending emails when > jira tickets with a specific tag are opened? But if there is I can ensure > that any tickets get routed to a member of my team to look at. > > What would people's thoughts be about this? > > -Daniel > > On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser > wrote: > >> Hi Daniel, >> >> Thanks for reaching out. Keep in mind that you weren't subscribed to the >> Flink Dev mailing list, I've just pushed this through the mailing list >> moderation. >> >> The process for contributing a connector is to create a Connector FLIP >> [1], >> which needs to be discussed and voted on in the Dev mailing list [2]. One >> thing in particular is who can help with the maintenance of the connector: >> will there be more volunteers who can help with bug fixes, new features >> etc. As we've seen with the current PubSub connector, that's already quite >> hard: it's currently lacking volunteers overall. I do recall a proposal to >> contribute a PubSub Lite connector a while back, but that ultimately was >> not followed through. >> >> Best regards, >> >> Martijn >> >> [1] >> https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template >> [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws >> >> On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins >> >> wrote: >> >> > Hello flink devs, >> > >> > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite streaming >> > system, which is a lower cost streaming data system with semantics more >> > similar to OSS streaming solutions. I've authored a source and sink >> > connector for flink and load tested it at GiB/s scale- what would be the >> > process for contributing this to flink? >> > >> > I've opened a JIRA to do this >> > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, if >> this >> > seems like a reasonable thing to do could someone assign it to me? >> > >> > The code for the connector currently lives here >> > https://github.com/googleapis/java-pubsublite-flink, I believe it is >> > following the FLIP-27 guidelines, but please let me know if I'm >> > implementing the wrong interfaces. Which repo and in what folder should >> I >> > move this code into? >> > >> > -Daniel >> > >> >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hello Martijn, Thanks for the redirect! > The process for contributing a connector is to create a Connector FLIP [1], which needs to be discussed and voted on in the Dev mailing list [2] I don't mind doing this, but would like to get a read of the room before doing so. If people object to it, then it probably doesn't make sense. > One thing in particular is who can help with the maintenance of the connector: will there be more volunteers who can help with bug fixes, new features etc. In this case, the product team is willing to handle feature requests and bug reports for the connector, as we currently do for our beam and spark connectors. I don't know if there is any mechanism for sending emails when jira tickets with a specific tag are opened? But if there is I can ensure that any tickets get routed to a member of my team to look at. What would people's thoughts be about this? -Daniel On Mon, Feb 27, 2023 at 3:53 AM Martijn Visser wrote: > Hi Daniel, > > Thanks for reaching out. Keep in mind that you weren't subscribed to the > Flink Dev mailing list, I've just pushed this through the mailing list > moderation. > > The process for contributing a connector is to create a Connector FLIP [1], > which needs to be discussed and voted on in the Dev mailing list [2]. One > thing in particular is who can help with the maintenance of the connector: > will there be more volunteers who can help with bug fixes, new features > etc. As we've seen with the current PubSub connector, that's already quite > hard: it's currently lacking volunteers overall. I do recall a proposal to > contribute a PubSub Lite connector a while back, but that ultimately was > not followed through. > > Best regards, > > Martijn > > [1] > https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template > [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws > > On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins > > wrote: > > > Hello flink devs, > > > > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite streaming > > system, which is a lower cost streaming data system with semantics more > > similar to OSS streaming solutions. I've authored a source and sink > > connector for flink and load tested it at GiB/s scale- what would be the > > process for contributing this to flink? > > > > I've opened a JIRA to do this > > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, if > this > > seems like a reasonable thing to do could someone assign it to me? > > > > The code for the connector currently lives here > > https://github.com/googleapis/java-pubsublite-flink, I believe it is > > following the FLIP-27 guidelines, but please let me know if I'm > > implementing the wrong interfaces. Which repo and in what folder should I > > move this code into? > > > > -Daniel > > >
Re: Contributing a Google Cloud Pub/Sub Lite source and sink?
Hi Daniel, Thanks for reaching out. Keep in mind that you weren't subscribed to the Flink Dev mailing list, I've just pushed this through the mailing list moderation. The process for contributing a connector is to create a Connector FLIP [1], which needs to be discussed and voted on in the Dev mailing list [2]. One thing in particular is who can help with the maintenance of the connector: will there be more volunteers who can help with bug fixes, new features etc. As we've seen with the current PubSub connector, that's already quite hard: it's currently lacking volunteers overall. I do recall a proposal to contribute a PubSub Lite connector a while back, but that ultimately was not followed through. Best regards, Martijn [1] https://cwiki.apache.org/confluence/display/FLINK/FLIP+Connector+Template [2] https://cwiki.apache.org/confluence/display/FLINK/Flink+Bylaws On Mon, Feb 27, 2023 at 9:44 AM Daniel Collins wrote: > Hello flink devs, > > My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite streaming > system, which is a lower cost streaming data system with semantics more > similar to OSS streaming solutions. I've authored a source and sink > connector for flink and load tested it at GiB/s scale- what would be the > process for contributing this to flink? > > I've opened a JIRA to do this > https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, if this > seems like a reasonable thing to do could someone assign it to me? > > The code for the connector currently lives here > https://github.com/googleapis/java-pubsublite-flink, I believe it is > following the FLIP-27 guidelines, but please let me know if I'm > implementing the wrong interfaces. Which repo and in what folder should I > move this code into? > > -Daniel >
Contributing a Google Cloud Pub/Sub Lite source and sink?
Hello flink devs, My name is Daniel, I'm the tech lead for Google's Pub/Sub Lite streaming system, which is a lower cost streaming data system with semantics more similar to OSS streaming solutions. I've authored a source and sink connector for flink and load tested it at GiB/s scale- what would be the process for contributing this to flink? I've opened a JIRA to do this https://issues.apache.org/jira/projects/FLINK/issues/FLINK-31229, if this seems like a reasonable thing to do could someone assign it to me? The code for the connector currently lives here https://github.com/googleapis/java-pubsublite-flink, I believe it is following the FLIP-27 guidelines, but please let me know if I'm implementing the wrong interfaces. Which repo and in what folder should I move this code into? -Daniel
