2022-04-20 Minutes of the bi-weekly meeting of Apache Linkis(incubating) 1. Apache Linkis incubation & version progress is synchronized. —— Wang Zhen (1) Each version needs to regularly pull and merge the master code to resolve some conflicts between versions in time (2) Using the joint test development environment, the home IP is not fixed and the company VPN has multiple exit IPs. For users with multiple egress IPs, it is recommended to consult the company's network department to provide a complete IP list. At the same time, the community will investigate the method of cloud server proxy, and Jie Longping is responsible for follow-up. (3) The git action unit test coverage tool has been used, and community users need to pay attention to some indicators of unit test coverage when submitting PR.
2. Apache Linkis1.1.1 progress synchronization. —— Xia Chen (1) Progress: The self-testing test is completed. When sorting out the documents, it is necessary to provide the upgrade guide document (due to the change of the udf table). It is expected to start voting before 5.1. (2) Related discussion: If each version involves major changes and changes with tables, it is necessary to provide an upgrade guide process document 3. Apache Linkis1.1.2 progress synchronization. - Wang Heping (1) Progress: Go to HDFS and pass the test, most of the features have been submitted for testing, and are expected to be released in May (2) Related discussion: After implementing HDFS, you can start to start debugging linkis on windows 4. Apache Linkis1.2.0 progress synchronization. —— Wang Zhen (1) Progress: Most of the code has been completed, the test encountered problems to be solved, lack of documentation and unit tests (2) Related discussion: The Presto engine will be compatible with Presto and Trino by implementing different EngineConnExecutors 5. Discussion on some features of Linkis - Wang Heping (1) The parameter supports real-time modification to take effect, and there is no need to restart the service. Now modifying the linkis parameter requires restarting the service. It is recommended to support refresh Discussion: Investigating Spring Cloud config and the nacos configuration center. (Wang Zhen is responsible for follow-up) (2) The task supports interface re-pulling. Now, if the entry service hangs up, the task will generally be marked as canceled. However, in many scenarios, these tasks need to be re-pulled. It is recommended to add this function. Discussion: Automatically or provide an interface to restart the abnormally failed task. (Wang Heping is responsible for follow-up) (3) The client also supports task retry. Now Orchestrator supports LinkisRetryException to retry 10 times, but it is best to also support retry when calling through the client. Retry is performed by judging whether the task supports retry. Need to modify the task record table plus retry field. Discussion: Since task submission may also fail, all the best clients also add retries (Xu Jie is responsible for follow-up) (4) Linkis supports grayscale update, based on eureka offline mechanism Discussion: Through the eureka offline mechanism, the service is offline in a grayscale. It should be noted that the Entrance service may not be directly offline due to the status. (Wang Zhen is responsible for follow-up) (5) The management console supports the update and viewing of engine material resources, and the interfaced installation of ec Discussion: It is mainly about the development of front-end pages. Further design and discussion are needed to consider whether it should be considered as a project topic in the summer of open source. (Xia Chen is responsible for follow-up) 6. The progress of Apache Linkis community operations is synchronized. Li Wen explained and analyzed recent indicators related to community operations. 7. The host of the next regular meeting, welcome to claim. Lu Xiaolong will be the host of the next regular meeting. 8. The dynamic synchronization on GitHub in the past two weeks, whether it should be discussed as a fixed item in the future (such as those new bugs, the newly merged pr feature repair instructions, etc.) In the later stage of the discussion, some needs, features or problems can be collected in the community group in advance and discussed in the follow-up meeting. And considering the scope of the open meeting, you can try to notify the user group as well. --- 2022-04-20 Apache Linkis(incubating) 双周例会会议纪要 1. Apache Linkis 孵化&版本 进展同步。 —— 王震 （1）各个版本需要定期拉取合并 master 代码，及时解决版本间的一些冲突 （2）使用联合测试开发环境，遇到了家庭 IP 不固定，公司 VPN 存在多个出口 IP 的情况。对于有多个出口 IP 有的用户，建议向公司网络部门进行咨询，提供完整的IP列表。同时社区将调研云服务器代理的方法，介龙平负责跟进。 （3）git action 单元测试覆盖率工具开始使用，社区用户提交 PR 时需要关注单元测试覆盖率的一些指标。 2. Apache Linkis1.1.1 进展同步。 —— 夏晨 （1）进度：自测测试完成，在梳理文档，需要提供升级指引文档（由于 udf 表变动），预计 5.1 之前开始投票 （2）相关讨论：每个版本如果涉及大的变动，以及有表的变更时，需要提供升级指引流程文档 3. Apache Linkis1.1.2 进展同步。 —— 王和平 （1）进展：去 HDFS 测试通过，大部分特性已经提交带测试，预计 5 月份发布 （2）相关讨论：实现去 HDFS 后，可以开始实现 windows 上面启动调试 linkis 4. Apache Linkis1.2.0 进展同步。 —— 王震 （1）进展：大部分代码已经完成，测试遇到问题待解决，缺少文档和单元测试 （2）相关讨论：Presto 引擎将通过实现，不同的 EngineConnExecutor 来兼容 Presto 和 Trino 5. Linkis 部分特性讨论 —— 王和平 （1）参数支持实时修改生效，不需要重启服务，现在修改linkis参数需要重启服务，建议支持刷新 讨论：调研 Spring Cloud config 和 nacos 配置中心。（王震负责跟进） （2）任务支持接口重拉，现在如果entrance服务挂掉后，一般会将任务标记为取消，但是很多场景需要将这些任务进行重拉，建议加上这个功能 讨论：自动或者提供接口，重新拉起异常失败任务。（王和平负责跟进） （3）客户端也支持任务重试，现在Orchestrator支持LinkisRetryException重试10次，但是很多时候通过客户端调用的时候最好也支持重试，通过判断任务是否支持重试进行重试。需要修改任务记录表加上重试字段。 讨论：由于任务提交也可能会失败，所有最好客户端也添加重试（徐杰负责跟进） （4）Linkis支持灰度更新，基于 eureka 下线机制 讨论：通过 eureka 下线机制，对服务进行灰度下线，需要注意 Entrance 服务有状态可能不能直接下线。（王震负责跟进） （5）管理台支持引擎物料资源更新、查看，ec的界面化安装 讨论：主要是前端页面的开发，需要进一步设计讨论，考虑是否作为开源之夏的项目议题。（夏晨负责跟进） 6. Apache Linkis 社区运营 进展同步。 李文对近期的社区运营相关指标进行说明分析。 7. 下一场例会的主持人，欢迎认领。 陆小龙将作为下一场例会的主持人。 8. 最近两周的GitHub上的动态同步，后续要不要作为一个固定事项讨论(如新增了那些bug，新合入的pr特性修复说明等) 讨论后期可以提前在社区群中收集一些需求、特性或问题，在后续的会议中进行讨论。并考虑开放会议的范围，可尝试在用户群里面也进行通知。 -- Kind Regards, Zhen Wang
