[jira] [Commented] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17071439#comment-17071439 ] yanxiaobin commented on FLINK-16791: ok, thanks. > Could not deploy Yarn job clust

[jira] [Commented] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17068293#comment-17068293 ] yanxiaobin commented on FLINK-16791: Hi,  thanks everyone. Just as [~kkl0u] said, It

[jira] [Commented] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067618#comment-17067618 ] yanxiaobin commented on FLINK-16791: It does't work for now. The unshaded com.google

[jira] [Commented] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067576#comment-17067576 ] yanxiaobin commented on FLINK-16791: Hi Yang Wang, thanks. I have already put flink-

[jira] [Updated] (FLINK-16792) flink-s3-fs-hadoop cannot access s3

[ https://issues.apache.org/jira/browse/FLINK-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-16792: --- Description: When flink-s3-fs-hadoop-1.10.0.jar is placed in the lib directory of Flink distributio

[jira] [Commented] (FLINK-16792) flink-s3-fs-hadoop cannot access s3

[ https://issues.apache.org/jira/browse/FLINK-16792?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067389#comment-17067389 ] yanxiaobin commented on FLINK-16792: The issue is related to https://issues.apache.o

[jira] [Created] (FLINK-16792) flink-s3-fs-hadoop cannot access s3

yanxiaobin created FLINK-16792: -- Summary: flink-s3-fs-hadoop cannot access s3 Key: FLINK-16792 URL: https://issues.apache.org/jira/browse/FLINK-16792 Project: Flink Issue Type: Bug Com

[jira] [Commented] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17067380#comment-17067380 ] yanxiaobin commented on FLINK-16791: I checked that the application does not contain

[jira] [Updated] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

[ https://issues.apache.org/jira/browse/FLINK-16791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-16791: --- Description: Since aws s3 is needed, so copy the flink-s3-fs-hadoop-1.10.0.jar JAR file to the {{li

[jira] [Created] (FLINK-16791) Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar

yanxiaobin created FLINK-16791: -- Summary: Could not deploy Yarn job cluster when using flink-s3-fs-hadoop-1.10.0.jar Key: FLINK-16791 URL: https://issues.apache.org/jira/browse/FLINK-16791 Project: Flin

[jira] [Commented] (FLINK-12992) All host(s) tried for query failed (tried com.datastax.driver.core.exceptions.TransportException: Error writing...)

[ https://issues.apache.org/jira/browse/FLINK-12992?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16876018#comment-16876018 ] yanxiaobin commented on FLINK-12992: At present, I have upgraded the netty version o

[jira] [Created] (FLINK-12992) All host(s) tried for query failed (tried com.datastax.driver.core.exceptions.TransportException: Error writing...)

yanxiaobin created FLINK-12992: -- Summary: All host(s) tried for query failed (tried com.datastax.driver.core.exceptions.TransportException: Error writing...) Key: FLINK-12992 URL: https://issues.apache.org/jira/brows

[jira] [Updated] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8500: -- Summary: Get the timestamp of the Kafka message from kafka consumer (was: Get the timestamp of the Kaf

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16749861#comment-16749861 ] yanxiaobin commented on FLINK-8500: --- I have changed the title. > Get the timestamp of

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496136#comment-16496136 ] yanxiaobin edited comment on FLINK-8500 at 5/31/18 5:42 AM: [

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496136#comment-16496136 ] yanxiaobin commented on FLINK-8500: --- [~tzulitai]   thanks, greate.  > Get the timestam

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16496006#comment-16496006 ] yanxiaobin commented on FLINK-8500: --- So far, it can solve the current problems, but in

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16488779#comment-16488779 ] yanxiaobin commented on FLINK-8500: --- I prefer to  to expose Kafka's ConsumerRecord direc

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16451808#comment-16451808 ] yanxiaobin commented on FLINK-8500: --- hi [~aljoscha] ：What is your idea? I now solve the

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417174#comment-16417174 ] yanxiaobin commented on FLINK-8794: --- Other problems have nothing to do with Flink. > Wh

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416734#comment-16416734 ] yanxiaobin commented on FLINK-8794: --- thanks! indeed so.  Enable consistent-view can caus

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415131#comment-16415131 ] yanxiaobin commented on FLINK-8794: --- The underlying implementation is com.amazon.ws.emr

[jira] [Commented] (FLINK-8939) Provide better support for saving streaming data to s3

[ https://issues.apache.org/jira/browse/FLINK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16415112#comment-16415112 ] yanxiaobin commented on FLINK-8939: --- I'm using AWS S3. This problem has been plaguing me

[jira] [Commented] (FLINK-8939) Provide better support for saving streaming data to s3

[ https://issues.apache.org/jira/browse/FLINK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413319#comment-16413319 ] yanxiaobin commented on FLINK-8939: --- thanks! Because S3 follows the *eventually consiste

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16404584#comment-16404584 ] yanxiaobin commented on FLINK-8794: --- S3 follows the *eventually consistent* principle, s

[jira] [Comment Edited] (FLINK-8939) Provide better support for saving streaming data to s3

[ https://issues.apache.org/jira/browse/FLINK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398211#comment-16398211 ] yanxiaobin edited comment on FLINK-8939 at 3/14/18 8:07 AM: I

[jira] [Commented] (FLINK-8939) Provide better support for saving streaming data to s3

[ https://issues.apache.org/jira/browse/FLINK-8939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398211#comment-16398211 ] yanxiaobin commented on FLINK-8939: --- I am also using BucketingSink API for saving stream

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16395286#comment-16395286 ] yanxiaobin commented on FLINK-8794: --- I will open a case with AWS for the root cause of t

[jira] [Comment Edited] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393956#comment-16393956 ] yanxiaobin edited comment on FLINK-8794 at 3/10/18 1:53 AM: Ab

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16393956#comment-16393956 ] yanxiaobin commented on FLINK-8794: --- About : 1.What I described above is that there will

[jira] [Updated] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8794: -- Issue Type: Improvement (was: Bug) > When using BucketingSink, it happens that one of the files is alwa

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390914#comment-16390914 ] yanxiaobin edited comment on FLINK-8500 at 3/8/18 8:47 AM: --- Sorr

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390914#comment-16390914 ] yanxiaobin commented on FLINK-8500: --- Sorry to reply so late. I see. We really should be

[jira] [Comment Edited] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384954#comment-16384954 ] yanxiaobin edited comment on FLINK-8794 at 3/4/18 4:13 AM: --- yes.

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16384954#comment-16384954 ] yanxiaobin commented on FLINK-8794: --- yes. If we allow for different directory that shoul

[jira] [Comment Edited] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383356#comment-16383356 ] yanxiaobin edited comment on FLINK-8794 at 3/2/18 8:40 AM: --- hi,

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383356#comment-16383356 ] yanxiaobin commented on FLINK-8794: --- hi, [~pnowojski] ! Thank you for your suggestion！ T

[jira] [Commented] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383209#comment-16383209 ] yanxiaobin commented on FLINK-8794: --- hi, [~aljoscha] Thank you for your reply！   There a

[jira] [Updated] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8794: -- Description: When using BucketingSink, it happens that one of the files is always in the [.in-progress]

[jira] [Updated] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8794: -- Description: When using BucketingSink, it happens that one of the files is always in the [.in-progress]

[jira] [Updated] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

[ https://issues.apache.org/jira/browse/FLINK-8794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8794: -- Description: When using BucketingSink, it happens that one of the files is always in the [.in-progress]

[jira] [Created] (FLINK-8794) When using BucketingSink, it happens that one of the files is always in the [.in-progress] state

yanxiaobin created FLINK-8794: - Summary: When using BucketingSink, it happens that one of the files is always in the [.in-progress] state Key: FLINK-8794 URL: https://issues.apache.org/jira/browse/FLINK-8794

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16376428#comment-16376428 ] yanxiaobin commented on FLINK-8500: --- hi, [~aljoscha] ! I can solve that problem like tha

[jira] [Comment Edited] (FLINK-5479) Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions

[ https://issues.apache.org/jira/browse/FLINK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343115#comment-16343115 ] yanxiaobin edited comment on FLINK-5479 at 2/1/18 10:30 AM: I'

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347951#comment-16347951 ] yanxiaobin edited comment on FLINK-8500 at 2/1/18 3:28 AM: --- Grea

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16347951#comment-16347951 ] yanxiaobin commented on FLINK-8500: --- Greate！I think it's reasonable.But I think it's bes

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346550#comment-16346550 ] yanxiaobin commented on FLINK-8500: --- hi, [~aljoscha] .  What do you think? > Get the ti

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346180#comment-16346180 ] yanxiaobin edited comment on FLINK-8500 at 1/31/18 3:09 AM: hi

[jira] [Comment Edited] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346180#comment-16346180 ] yanxiaobin edited comment on FLINK-8500 at 1/31/18 3:01 AM: hi

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346180#comment-16346180 ] yanxiaobin commented on FLINK-8500: --- hi,[~aljoscha] , thank you for your reply!Please lo

[jira] [Updated] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8500: -- Attachment: image-2018-01-31-10-48-59-633.png > Get the timestamp of the Kafka message from kafka consum

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16344591#comment-16344591 ] yanxiaobin commented on FLINK-8500: --- As shown below（Kafka010Fetcher）： !image-2018-01-30-

[jira] [Updated] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8500: -- Attachment: image-2018-01-30-14-58-58-167.png > Get the timestamp of the Kafka message from kafka consum

[jira] [Commented] (FLINK-5479) Per-partition watermarks in FlinkKafkaConsumer should consider idle partitions

[ https://issues.apache.org/jira/browse/FLINK-5479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343115#comment-16343115 ] yanxiaobin commented on FLINK-5479: --- I've also met this problem at the moment. > Per-pa

[jira] [Commented] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

[ https://issues.apache.org/jira/browse/FLINK-8500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338958#comment-16338958 ] yanxiaobin commented on FLINK-8500: --- I mean that I want to use Kafka message timestamp a

[jira] [Created] (FLINK-8500) Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher)

yanxiaobin created FLINK-8500: - Summary: Get the timestamp of the Kafka message from kafka consumer(Kafka010Fetcher) Key: FLINK-8500 URL: https://issues.apache.org/jira/browse/FLINK-8500 Project: Flink

[jira] [Updated] (FLINK-8321) Generate _SUCCESS (map-reduce style) when folder has been written！

[ https://issues.apache.org/jira/browse/FLINK-8321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yanxiaobin updated FLINK-8321: -- Description: There is no success file generation, and the downstream processing can't determine when the

[jira] [Created] (FLINK-8321) Generate _SUCCESS (map-reduce style) when folder has been written！

yanxiaobin created FLINK-8321: - Summary: Generate _SUCCESS (map-reduce style) when folder has been written！ Key: FLINK-8321 URL: https://issues.apache.org/jira/browse/FLINK-8321 Project: Flink I