Re: Few question about upgrade from 1.4 to 1.5 flink ( some very basic )

Chesnay Schepler Mon, 25 Jun 2018 07:23:19 -0700

The watermark issue is know and will be fixed in 1.5.1


On 25.06.2018 15:03, Vishal Santoshi wrote:

Thank you....

One addition

I do not see WM info on the UI  ( Attached )

Is this a know issue. The same pipe on our production has the WM ( Infact never had an issue with Watermarks not appearing ) . Am Imissing something ?

On Mon, Jun 25, 2018 at 4:15 AM, Fabian Hueske <fhue...@gmail.com<mailto:fhue...@gmail.com>> wrote:


    Hi Vishal,

    1. I don't think a rolling update is possible. Flink 1.5.0 changed
    the process orchestration and how they communicate. IMO, the way
    to go is to start a Flink 1.5.0 cluster, take a savepoint on the
    running job, start from the savepoint on the new cluster and shut
    the old job down.
    2. Savepoints should be compatible.
    3. You can keep the slot configuration as before.
    4. As I said before, mixing 1.5 and 1.4 processes does not work
    (or at least, it was not considered a design goal and nobody paid
    attention that it is possible).

    Best, Fabian


    2018-06-23 13:38 GMT+02:00 Vishal Santoshi
    <vishal.santo...@gmail.com <mailto:vishal.santo...@gmail.com>>:


        1.

Can or has any one done a rolling upgrade from 1.4 to 1.5 ?I am not sure we can. It seems that JM cannot recover jobs

        with this exception

        Caused by: java.io.InvalidClassException:
        
org.apache.flink.runtime.jobgraph.tasks.CheckpointCoordinatorConfiguration;
        local class incompatible: stream classdesc serialVersionUID =
        -647384516034982626, local class serialVersionUID = 2




        2.
        Does SP on 1.4, resume on 1.5 ( pretty basic but no harm
        asking ) ?



        3.
        
https://ci.apache.org/projects/flink/flink-docs-release-1.5/release-notes/flink-1.5.html#update-configuration-for-reworked-job-deployment
        
<https://ci.apache.org/projects/flink/flink-docs-release-1.5/release-notes/flink-1.5.html#update-configuration-for-reworked-job-deployment>
        The taskmanager.numberOfTaskSlots: What would be the desired
        setting in a stand alone ( non mesos/yarn ) cluster ?


        4. I suspend all jobs and establish 1.5 on the JM ( the TMs
        are still running with 1.4 ) . JM refuse to start  with

        Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net
        <http://flink-ad21ac07.bf2.tumblr.net> docker[3395]:
        2018-06-23 11:34:23 ERROR JobManager:116 - Failed to recover
        job 454cd84a519f3b50e88bcb378d8a1330.

        Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net
        <http://flink-ad21ac07.bf2.tumblr.net> docker[3395]:
        java.lang.InstantiationError:
        org.apache.flink.runtime.blob.BlobKey

        Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net
        <http://flink-ad21ac07.bf2.tumblr.net> docker[3395]: at
        
sun.reflect.GeneratedSerializationConstructorAccessor51.newInstance(Unknown
        Source)

        Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net
        <http://flink-ad21ac07.bf2.tumblr.net> docker[3395]: at
        java.lang.reflect.Constructor.newInstance(Constructor.java:423)

        Jun 23 07:34:23 flink-ad21ac07.bf2.tumblr.net
        <http://flink-ad21ac07.bf2.tumblr.net> docker[3395]: at
        java.io.ObjectStreamClass.newInstance(ObjectStreamClass.java:1079)

        Jun

        .....



        Any feedback would be highly appreciated...

Re: Few question about upgrade from 1.4 to 1.5 flink ( some very basic )

Reply via email to