Good meeting and thanks for the notes Kaxil!
On 2021/01/08 21:55:13, Kaxil Naik <kaxiln...@apache.org> wrote: > Please ignore the last bullet around "breaking changes" -- I mixed that up > with the Airflow Dev call summary ;) > > Regards, > Kaxil > > On Fri, Jan 8, 2021 at 9:52 PM Kaxil Naik <kaxiln...@apache.org> wrote: > > > Hi all, > > > > Here are the notes from our Kibble dev call from yesterday. Please feel > > free to add anything I have missed. > > > > *Attendees:* > > > > - Kaxil Naik > > - Tomek Urbaszek > > - Michał Słowikowski > > - Sharan Foga > > - Daniel Gruno > > > > > > Here is the summary of the call. > > > > - *Kibble v2*: > > - Rewrite the implementation from Scratch for v2 to improve code > > and to cater for new ideas > > - Cherrypick / copy any code from the current Repo > > - New repo allows us to work with a TDD approach. The current repo > > does not have tests. > > - *Supporting more DBs* > > - For now, we would just support Elasticsearch. > > - Supporting just one DB allows us to maintain it in a better way > > - Would be good to find some kind of ORM to talk to Elasticsearch > > which will allow static Schema and use Class attributes instead of > > dealing > > with dicts > > - *Migration* > > - Make migration tool or make whatever we build new as > > backwards-compatible so that we don’t need to re-scan all the data > > for ASF > > Kibble -- which can take aleast a week if started from scratch. > > (contains 50million records) > > - *Scanners* > > - Build Base Scanner for v2 that will allow easily creating new > > Scanners and encourage more contributions > > - Write docs around how to build a new scanner inheriting > > *BaseScanner* > > - Some of the new Scanners that users have requested are: > > - Gitlab > > - Social Media Scanners: Twitter, Discord, Slack > > - Create a Github Issue template for Scanners so users can request > > new Scanners > > - *Dashboard* > > - Do a POC to see if Apache Superset can replace our current UI > > - Make Apache Superset or the existing UI optional as some Users just > > rely on the API > > - That will allow us to focus on the CORE (Scanner, Server and ES) > > - > > - What happens to the PR that introduces breaking changes?? > > - We should not merge that PR until it is a breaking change, > > instead we should make the PR backwards compatible > > - Or wait until next major release (assign appropriate Github > > Milestone) > > > > > > Let me know if anyone has any thoughts on any of the items listed above. > > > > Best regards, > > Kaxil > > >
