[jira] [Resolved] (KAFKA-3531) support subnet in ACL tool

[Manikumar Reddy (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-3531?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Manikumar Reddy resolved KAFKA-3531.

Resolution: Duplicate
  Assignee: (was: Parth Brahmbhatt)

Resolving this as duplicate, as PR is submitted for KAFKA-4759

> support subnet in ACL tool
> --
>
> Key: KAFKA-3531
> URL: https://issues.apache.org/jira/browse/KAFKA-3531
> Project: Kafka
>  Issue Type: Wish
>Affects Versions: 0.9.0.1
>Reporter: Jun Rao
>
> In the ACL tool, we currently support individual ip or all ips. It will be 
> useful to add support for things like a subnet.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: kafka-trunk-jdk8 #1270

[Apache Jenkins Server]

See 

Changes:

[becket.qin] KAFKA-4340; Change default message.timestamp.difference.max.ms to 
the

--
Started by an SCM change
[EnvInject] - Loading node environment variables.
Building remotely on ubuntu-us1 (ubuntu) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url 
 > https://git-wip-us.apache.org/repos/asf/kafka.git # timeout=10
Fetching upstream changes from https://git-wip-us.apache.org/repos/asf/kafka.git
 > git --version # timeout=10
 > git -c core.askpass=true fetch --tags --progress 
 > https://git-wip-us.apache.org/repos/asf/kafka.git 
 > +refs/heads/*:refs/remotes/origin/*
 > git rev-parse refs/remotes/origin/trunk^{commit} # timeout=10
 > git rev-parse refs/remotes/origin/origin/trunk^{commit} # timeout=10
Checking out Revision 8bc9d583930110cf209f8ab20f9ff640ace3ed35 
(refs/remotes/origin/trunk)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8bc9d583930110cf209f8ab20f9ff640ace3ed35
 > git rev-list 601d4040d446599bb01c45389b3e90645245e9af # timeout=10
Setting 
GRADLE_2_4_RC_2_HOME=/home/jenkins/jenkins-slave/tools/hudson.plugins.gradle.GradleInstallation/Gradle_2.4-rc-2
[kafka-trunk-jdk8] $ /bin/bash -xe /tmp/hudson7155099535483202046.sh
+ rm -rf 
+ 
/home/jenkins/jenkins-slave/tools/hudson.plugins.gradle.GradleInstallation/Gradle_2.4-rc-2/bin/gradle

ERROR: JAVA_HOME is set to an invalid directory: 
/home/jenkins/tools/java/latest1.8

Please set the JAVA_HOME variable in your environment to match the
location of your Java installation.

Build step 'Execute shell' marked build as failure
Recording test results
Setting 
GRADLE_2_4_RC_2_HOME=/home/jenkins/jenkins-slave/tools/hudson.plugins.gradle.GradleInstallation/Gradle_2.4-rc-2
ERROR: Step ‘Publish JUnit test result report’ failed: Test reports were found 
but none of them are new. Did tests run? 
For example, 

 is 1 day 5 hr old

Setting 
GRADLE_2_4_RC_2_HOME=/home/jenkins/jenkins-slave/tools/hudson.plugins.gradle.GradleInstallation/Gradle_2.4-rc-2

[jira] [Commented] (KAFKA-4340) Change the default value of log.message.timestamp.difference.max.ms to the same as log.retention.ms

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862654#comment-15862654
 ] 

ASF GitHub Bot commented on KAFKA-4340:
---

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/2071


> Change the default value of log.message.timestamp.difference.max.ms to the 
> same as log.retention.ms
> ---
>
> Key: KAFKA-4340
> URL: https://issues.apache.org/jira/browse/KAFKA-4340
> Project: Kafka
>  Issue Type: Improvement
>  Components: core
>Affects Versions: 0.10.1.0
>Reporter: Jiangjie Qin
>Assignee: Jiangjie Qin
> Fix For: 0.10.3.0
>
>
> [~junrao] brought up the following scenario: 
> If users are pumping data with timestamp already passed log.retention.ms into 
> Kafka, the messages will be appended to the log but will be immediately 
> rolled out by log retention thread when it kicks in and the messages will be 
> deleted. 
> To avoid this produce-and-deleted scenario, we can set the default value of 
> log.message.timestamp.difference.max.ms to be the same as log.retention.ms.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] kafka pull request #2071: KAFKA-4340: Change default message.timestamp.diffe...

[asfgit]

Github user asfgit closed the pull request at:

https://github.com/apache/kafka/pull/2071


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Commented] (KAFKA-4759) Add support for subnet masks in SimpleACLAuthorizer

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/KAFKA-4759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15862612#comment-15862612
 ] 

ASF GitHub Bot commented on KAFKA-4759:
---

GitHub user takebayashi opened a pull request:

https://github.com/apache/kafka/pull/2541

KAFKA-4759: Add support for subnet masks in SimpleACLAuthorizer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/takebayashi/kafka simpleacl-subnet

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/2541.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2541


commit 001297a68d0052bcbf9dbd46717954de32711c8b
Author: Shun Takebayashi 
Date:   2017-02-11T02:19:39Z

Add commons-net as a dependency

commit 3edf8f99208ac80dd66d3a06994fc0e96611ad37
Author: Shun Takebayashi 
Date:   2017-02-12T00:24:32Z

Support subnect matching in SimpleAclAuthorizer

commit 8d67144bdb8fd08e5ed8300048a60dccacaad792
Author: Shun Takebayashi 
Date:   2017-02-12T00:44:41Z

Add tests for subnet matcher




> Add support for subnet masks in SimpleACLAuthorizer
> ---
>
> Key: KAFKA-4759
> URL: https://issues.apache.org/jira/browse/KAFKA-4759
> Project: Kafka
>  Issue Type: Improvement
>  Components: security
>Reporter: Shun Takebayashi
>
> SimpleACLAuthorizer currently accepts only single IP addresses.
> Supporting subnet masks with SimpleACLAuthorizer can make ACL configurations 
> simpler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[GitHub] kafka pull request #2541: KAFKA-4759: Add support for subnet masks in Simple...

[takebayashi]

GitHub user takebayashi opened a pull request:

https://github.com/apache/kafka/pull/2541

KAFKA-4759: Add support for subnet masks in SimpleACLAuthorizer



You can merge this pull request into a Git repository by running:

$ git pull https://github.com/takebayashi/kafka simpleacl-subnet

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/kafka/pull/2541.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #2541


commit 001297a68d0052bcbf9dbd46717954de32711c8b
Author: Shun Takebayashi 
Date:   2017-02-11T02:19:39Z

Add commons-net as a dependency

commit 3edf8f99208ac80dd66d3a06994fc0e96611ad37
Author: Shun Takebayashi 
Date:   2017-02-12T00:24:32Z

Support subnect matching in SimpleAclAuthorizer

commit 8d67144bdb8fd08e5ed8300048a60dccacaad792
Author: Shun Takebayashi 
Date:   2017-02-12T00:44:41Z

Add tests for subnet matcher




---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---

[jira] [Created] (KAFKA-4759) Add support for subnet masks in SimpleACLAuthorizer

[Shun Takebayashi (JIRA)]

Shun Takebayashi created KAFKA-4759:
---

 Summary: Add support for subnet masks in SimpleACLAuthorizer
 Key: KAFKA-4759
 URL: https://issues.apache.org/jira/browse/KAFKA-4759
 Project: Kafka
  Issue Type: Improvement
  Components: security
Reporter: Shun Takebayashi


SimpleACLAuthorizer currently accepts only single IP addresses.
Supporting subnet masks with SimpleACLAuthorizer can make ACL configurations 
simpler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: [DISCUSS] KIP-117: Add a public AdministrativeClient API for Kafka admin operations

[Dong Lin]

If we require use non-batching API for purge, client needs to send one
PurgeRequest per partition and broker needs to have one delayed purgatory
event per partition. This is inefficient if the broker has 10s of
partitions that user wants to purge. This is a general problem for all APIs
(e.g. offsetsForTimes) which are typically used for operation on multiple
partitions at a time.


On Fri, Feb 10, 2017 at 5:05 PM, Dong Lin  wrote:

> Hi Jun,
>
> Currently KIP-107 uses this API:
>
> Future> 
> purgeDataBefore(Map Long> offsetForPartition)
>
> Are you suggesting that we should provide this:
>
> Future purgeDataBefore(TopicPartition, Long)
>
> I think the second solution works and the resulting implementation of
> KIP-107 will be simpler. The only concern is that this will be a bit
> inconvenience to use. User will need to have a "for" loop in their own code
> to call purgeDataBefore for every partition, get results for every
> partition, and aggregate results.
>
> KIP-117 currently only provides topic management operation and it is
> reasonable to call create(..) once for every topic because user generally
> creates one topic a time. But operations such as purgeDataBefore(..) seems
> different in the sense that user generally wants to purge data of all
> partitions of a topic.
>
> I can change change KIP-107 to use the second API if we decide to avoid
> batch API in AdminClient in the long term with the awareness that user
> needs to do a bit extra work. Maybe we can minimize this extra work by
> providing a utility class to combine multiple Future objects into one
> Future() object.
>
> Thanks,
> Dong
>
>
>
> On Fri, Feb 10, 2017 at 4:18 PM, Jun Rao  wrote:
>
>> Hi, Dong,
>>
>> For KIP-107, the purgeDataBefore() api will eventually be added to the
>> AdminClient too, right? It would be useful to make the apis consistent.
>> Currently, in KIP-107, we do batching in purgeDataBefore(). In Colin's
>> current proposal, there is no batching.
>>
>> Thanks,
>>
>> Jun
>>
>> On Thu, Feb 9, 2017 at 10:54 AM, Dong Lin  wrote:
>>
>> > Thanks for the explanation. This makes sense.
>> >
>> > Best,
>> > Dong
>> >
>> > On Thu, Feb 9, 2017 at 10:51 AM, Colin McCabe 
>> wrote:
>> >
>> > > On Wed, Feb 8, 2017, at 19:02, Dong Lin wrote:
>> > > > I am not aware of any semantics that will be caused by sharing
>> > > > NetworkClient between producer/consumer and AdminClient. But I agree
>> > that
>> > > > there is currently no good way to share such an internal class
>> between
>> > > > them. And yes, goal is to reduce number of connections. For example,
>> > say
>> > > > we
>> > > > want to enable auto data purge based on committed offset using
>> > > > AdminClient.purgeDataBefore(...) in a stream processing
>> application,
>> > > then
>> > > > in addition to producer or consumer, we will now have AdminClient in
>> > > > every
>> > > > job. It means that the the number of connection between server and
>> > client
>> > > > will double.
>> > > >
>> > > > I have another comment on the KIP. Is AdminClient API supposed to be
>> > > > thread
>> > > > safe?
>> > >
>> > > Yes.  The wiki page states: "The client will be multi-threaded;
>> multiple
>> > > threads will be able to safely make calls using the same AdminClient
>> > > object."
>> > >
>> > > > If so, should we mark private variables such as clientTimeoutMs to
>> > > > be @volatile? Would it be a concern if two threads call
>> > > > TopicsContext.setServerTimeout(...) concurrently to use different
>> > > timeout
>> > > > for their own use-case?
>> > >
>> > > The context objects are extremely lightweight and are not intended to
>> be
>> > > shared between multiple threads.  So each thread would just do
>> > > client.topics().setClientTimeout(...).create(), and operate on its
>> own
>> > > TopicsContext.  I will add that to the wiki.
>> > >
>> > > best,
>> > > Colin
>> > >
>> > > >
>> > > > Thanks,
>> > > > Dong
>> > > >
>> > > > On Wed, Feb 8, 2017 at 6:50 PM, Jason Gustafson > >
>> > > > wrote:
>> > > >
>> > > > > I'm not too sure sharing NetworkClient is a good idea. The
>> consumer
>> > > and the
>> > > > > producer both have request semantics which would be more
>> difficult to
>> > > > > reason about if the connections are shared with another client.
>> Also,
>> > > the
>> > > > > NetworkClient is an internal class which is not really meant for
>> > > users. Do
>> > > > > we really want to open that up? Is the only benefit saving the
>> number
>> > > of
>> > > > > connections? Seems not worth it in my opinion.
>> > > > >
>> > > > > -Jason
>> > > > >
>> > > > > On Wed, Feb 8, 2017 at 6:43 PM, Dong Lin 
>> > wrote:
>> > > > >
>> > > > > > BTW, the idea to share NetworkClient is suggested by Radai and I
>> > like
>> > > > > this
>> > > > > > idea.
>> > > > > >
>> > > > > > On Wed, Feb 8, 2017 at 6:39 

[jira] [Created] (KAFKA-4758) Connect WorkerSinkTask is missing checks for NO_TIMESTAMP

[Jason Gustafson (JIRA)]

Jason Gustafson created KAFKA-4758:
--

 Summary: Connect WorkerSinkTask is missing checks for NO_TIMESTAMP
 Key: KAFKA-4758
 URL: https://issues.apache.org/jira/browse/KAFKA-4758
 Project: Kafka
  Issue Type: Bug
  Components: KafkaConnect
Reporter: Jason Gustafson
Assignee: Ryan P


The current check for NO_TIMESTAMP_TYPE is not sufficient. Upconverted messages 
will have a timestamp type, but if the topic is set to use CREAT_TIME, the 
timestamp will be NO_TIMESTAMP (-1). We should use {{null}} in that case.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Request to add to Contributor list

[Jason Gustafson]

Done. Thanks for contributing!

-Jason

On Sat, Feb 11, 2017 at 7:10 AM, Kamal C 
wrote:

> Hi Dev Team,
>
> I want to contribute to Apache Kafka. Could you please add my id in the
> contributor list ?
>
> JIRA Id - ckamal
>
> -- Kamal
>

Request to add to Contributor list

[Kamal C]

Hi Dev Team,

I want to contribute to Apache Kafka. Could you please add my id in the
contributor list ?

JIRA Id - ckamal

-- Kamal