[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-516105901 Rebased and squashed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-516033759 Sorry about that @NicoK, resolved. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-515499428 Thanks for taking a look @fhueske, I believe I have addressed all of your points. In particular, I've added descriptions and sample output after each modification to the job. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-515142560 Sorry about that @NicoK, should be taken care of now. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-514765009 @NicoK I believe I have addressed all of your comments This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-514694835 @NicoK Thank you for the review, my intention was to add table_api.zh.md at the very end so I wouldn't need to keep both files in sync. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-511398012 This thread has gotten a little unwieldy so I'd like to summarize where everything stands. There are three new modules: * `flink-walkthrough-common`: Pre-defined sources/sinks/UDFs that users will use when following along * `flink-walkthrough-table-java`: A java maven archetype for initializing the walkthrough * `flink-walkthrough-table-scala`: A scala maven archetype for initializing the walkthrough * End-To-End tests that validate that both archetypes compile and run * Walkthrough documentation that covers both java and scala using the table api * The python table api is not currently feature-complete enough to be included * The walkthrough contains code snippits throughout with the full runnable code at the end This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-510909640 @morsapaes I believe I've addressed all of your comments @knaufk I'm working on adding testing to the archetypes, in particular the quickstarts have [archetype integration tests](https://maven.apache.org/archetype/maven-archetype-plugin/integration-test-mojo.html) but it doesn't seem like they actually run and that the quickstarts are covered by end to end tests. I am working to see if I can get the plugin to work since it is much lighter weight but worst case I will add the e2e tests. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-508115738 I believe I have addressed all the comments including adding a scala version. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-506360114 Thank you for taking a look @knaufk . > I would be in favor of simply adding this as an example to flink-examples-table without a maven archtype. Firstly, we already have quickstart modules to create program skeletons. Adding an additional option for the walkthroughs adds extra burden in terms of ongoing maintenance and might confuse users. I believe the best walkthroughs are the ones users can easily run and make small modifications to. In practice, this means we need an easy way to import the initial project into their IDE. If the code is in `flink-examples-table` it is non-trivial to extricate it from the rest of flink. This archetype is not meant to replace the quickstart archetypes in any way, and only serves as a way to quickly and easily create a new project with the right dependencies and initial set up for following along with the tutorial. We can make the `Setting up a Maven Project` more clear about what we are doing and direct users towards the generic quickstarts if they want to build their own projects. > Secondly, in my opinion, a "code walkthrough" should focus on explaining the code of an example so that I can easily follow it. I don't necessarily expect the utilities to bootstrap a new program here. (We should have a separate dedicated subsection in the "Getting Started" section for the quickstarts/project setup.) Again, not the intention but I take this to mean what I have is not clear enough. I would also support quickly moving [Application Development -> Project Build Setup](https://github.com/apache/flink/tree/master/docs/dev/projectsetup) to the getting started section with a nice introduction. This way it should be more clear to users that if they want to start a new project look here, and if they want a follow along look there. The fact that both these components use maven archetypes to facilitate their goals is just a coincidence. > Personally, I think, the example for a code walkthrough should be no-magic and I should be able to understand as much as possible as a new user. Here this means to go without the custom TableSource, which is not easy to understand and it is not clear why this can be a streaming and batch source at the same time etc.. Alternatively, one could simply read from a file of transactions (batch case) or read from a message queue (streaming case; maybe provide a minimal data generator). This would also go well with the future vision of "simply replace the source and your job turns into a streaming program", although we are not there yet. 1) This is not a dockerized environment and so I think we should avoid any external dependencies. 2) I should better explain that tables are logical views on top of external systems such as file systems or message queues but I don't see them as magic. > Do we also want to add SQL to this example (probably as a section in the end implementing the same business logic)? Currently, we don't plan to have dedicated "SQL Walkthrough". Do we need an SQL walkthrough at all? My thought was this would be just focused on the Table Api and we could do some sort of SQL walkthrough with the docker environment. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] [flink] sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough
sjwiesman commented on issue #8903: [FLINK-12747][docs] Getting Started - Table API Example Walkthrough URL: https://github.com/apache/flink/pull/8903#issuecomment-506024995 cc @knaufk @fhueske This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
