Github user zentol commented on the issue:
https://github.com/apache/flink/pull/5823
1. You should create a project _within the test_. This project must **not**
be based on `https://flink.apache.org/q/quickstart.sh`, is has to be created
with the archetype that is installed locally by `flink-quickstarts`. For this
you will have to use the maven commands as outlined in the
[documentation](https://ci.apache.org/projects/flink/flink-docs-master/quickstart/java_api_quickstart.html).
2. Correct, you should verify that none of the core flink classes are
contained in the jar. The only classes that should be contained are those of
the project.
3. No, this is not related to checkpointing. The point of this test is to
ensure that job-jars created by a quickstart project actually work when
submitted to a flink cluster. For example, they have to contain the job
classes, like `StreamingJob`, as otherwise the job will fail outright since
there's nothing to run. So you have to verify that the job was successfully
run, the easiest way being to write some data to some file (like the WordCount
tests), and verifying the contents.
---
