[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16746569#comment-16746569
 ] 

Matthias J. Sax commented on KAFKA-4218:


Done. Re-opened a ticket, updated a another ticket, and created two new ones. 
Also linked all tickets to each other and updated the KIP page accordingly.

> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>Priority: Major
>  Labels: api, kip
> Fix For: 1.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[Guozhang Wang (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744656#comment-16744656
 ] 

Guozhang Wang commented on KAFKA-4218:
--

You are right :) Mind creating a new JIRA and fixing this one's version tag?

> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>Priority: Major
>  Labels: api, kip
> Fix For: 2.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16737524#comment-16737524
 ] 

Matthias J. Sax commented on KAFKA-4218:


I guess fixed version 2.1 is not correct. the KIP is partially implemented in 
1.1 via this ticket but no progress since then. Thus, for this ticket 1.1 
should be the correct version, but we need further JIRAs to track the open 
parts of the KIP? Thoughts?

> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>Priority: Major
>  Labels: api, kip
> Fix For: 2.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[Matthias J. Sax (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16634781#comment-16634781
 ] 

Matthias J. Sax commented on KAFKA-4218:


[~lindong] There was no progress on KIP-149 for 2.1 release.

> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>Priority: Major
>  Labels: api, kip
> Fix For: 1.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284488#comment-16284488
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

GitHub user jeyhunkarimov opened a pull request:

https://github.com/apache/kafka/pull/4309

KAFKA-4218: Enable access to key in ValueTransformer and ValueMapper

This PR is the partial implementation for KIP-149. As the discussion for 
this KIP is still ongoing, I made a PR on the "safe" portions of the KIP (so 
that it can be included in the next release) which are 1) `ValueMapperWithKey`, 
2) `ValueTransformerWithKeySupplier`, and 3) `ValueTransformerWithKey`.




You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jeyhunkarimov/kafka KIP-149_hope_last

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/4309.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #4309


commit 91c1108c72f2f3b3097dcff3bd2aad237789215e
Author: Jeyhun Karimov 
Date:   2017-12-09T00:56:36Z

Submit the first version of KIP-149




> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16284486#comment-16284486
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

Github user jeyhunkarimov closed the pull request at:

https://github.com/apache/kafka/pull/3570


> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.1.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106435#comment-16106435
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

GitHub user jeyhunkarimov opened a pull request:

https://github.com/apache/kafka/pull/3601

KAFKA-4218: Add withkey methods to KGroupedTable

This PR aims to add withKey methods to KGroupedTable interface. 
This PR is part of 
[KIP-149](https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner).
I separated the complete PR into 4 parts as discussed in 
[here](https://github.com/apache/kafka/pull/3570#issuecomment-317645747). So, 
this PR concentrates on adding withKey methods to KGroupedTable interface. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jeyhunkarimov/kafka KIP-149_KGroupedTable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3601.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3601


commit 00f10288fac503764c8c6af8ae1907864d042dae
Author: Jeyhun Karimov 
Date:   2017-07-28T23:25:59Z

Add withkey interfaces

Add withkey interfaces

Add withKey converters in AbstractStream

Add withKey interface to KGroupedTable

Remove unnecessary lines

Remove unnecessary lines




> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.0.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106432#comment-16106432
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

GitHub user jeyhunkarimov opened a pull request:

https://github.com/apache/kafka/pull/3600

KAFKA-4218 : Add withkey methods to KTable

This PR aims to add withKey methods to KTable interface. 
This PR is part of 
[KIP-149](https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner).
I separated the complete PR into 4 parts as discussed in 
[here](https://github.com/apache/kafka/pull/3570#issuecomment-317645747). So, 
this PR concentrates on adding withKey methods to KTable interface. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jeyhunkarimov/kafka KIP-149_KTable

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3600.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3600


commit 8e34c603d2db4a71127972cfd2c2efa35b32cc6a
Author: Jeyhun Karimov 
Date:   2017-07-28T23:25:59Z

Add withkey interfaces

Add withkey interfaces

Add withKey converters in AbstractStream

Add withKey interfaces to KTable

Remove unnecessary lines




> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.0.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106428#comment-16106428
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

GitHub user jeyhunkarimov opened a pull request:

https://github.com/apache/kafka/pull/3599

KAFKA-4218: Add withkey methods to KGroupedStream

This PR aims to add withKey methods to KGroupedStream interface. 
This PR is part of 
[KIP-149](https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner).
I separated the complete PR into 4 parts as discussed in 
[here](https://github.com/apache/kafka/pull/3570#issuecomment-317645747). So, 
this PR concentrates on adding withKey methods to KGroupedStream interface. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jeyhunkarimov/kafka KIP-149_KGroupedStream

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3599.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3599


commit 6af52ab0822a9c557fbfd45a67d6584b506287ad
Author: Jeyhun Karimov 
Date:   2017-07-28T23:25:59Z

Add withkey interfaces

Add withkey interfaces

Add withKey converters in AbstractStream

Add tests

Remove unnecessary changes




> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.0.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16106421#comment-16106421
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

GitHub user jeyhunkarimov opened a pull request:

https://github.com/apache/kafka/pull/3598

KAFKA-4218: Add withkey methods to KStream

This PR aims to add withKey methods to KStream interface. 
This PR is part of 
[KIP-149](https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner).
I separated the complete PR into 4 parts as discussed in 
[here](https://github.com/apache/kafka/pull/3570#issuecomment-317645747). So, 
this PR concentrates on adding withKey methods to KStream interface. 

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/jeyhunkarimov/kafka KIP-149_KStream

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/3598.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #3598


commit c37be59024ea7fd69e1a8ffe2bae1272c3035aec
Author: Jeyhun Karimov 
Date:   2017-07-28T23:25:59Z

Add withkey interfaces

Add withkey interfaces

Add withKey converters in AbstractStream

Add KStream withKey interfaces

Remove unnecessary lines




> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.0.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

[jira] [Commented] (KAFKA-4218) Enable access to key in ValueTransformer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16099492#comment-16099492
 ] 

ASF GitHub Bot commented on KAFKA-4218:
---

Github user jeyhunkarimov closed the pull request at:

https://github.com/apache/kafka/pull/2946


> Enable access to key in ValueTransformer
> 
>
> Key: KAFKA-4218
> URL: https://issues.apache.org/jira/browse/KAFKA-4218
> Project: Kafka
>  Issue Type: Improvement
>  Components: streams
>Affects Versions: 0.10.0.1
>Reporter: Elias Levy
>Assignee: Jeyhun Karimov
>  Labels: api, kip
> Fix For: 1.0.0
>
>
> While transforming values via {{KStream.transformValues}} and 
> {{ValueTransformer}}, the key associated with the value may be needed, even 
> if it is not changed.  For instance, it may be used to access stores.  
> As of now, the key is not available within these methods and interfaces, 
> leading to the use of {{KStream.transform}} and {{Transformer}}, and the 
> unnecessary creation of new {{KeyValue}} objects.
> KIP-149: 
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-149%3A+Enabling+key+access+in+ValueTransformer%2C+ValueMapper%2C+and+ValueJoiner



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)