We should look at hooking up rat, if that's the commonly used checker for ASF releases.
Before doing that, we should update all files to the expected header, and add the headers to the non-java files. Chetan, do you know if the license:format can handle the non-java files, too? On Wed, Sep 23, 2015 at 4:33 PM, Chetan Narsude <[email protected]> wrote: > Yes Ram, this is handled by license plugin already. Do you think we should > hook up the license:check with the pull request? > > -- > Chetan > > > On Wed, Sep 23, 2015 at 4:17 PM, Munagala Ramanath <[email protected]> > wrote: > > > If we have a solution that we are happy with, there is no need to resort > to > > my script. > > > > I wrote the script to address what I thought was a one-time need as > > described in the message > > from Thomas. Our files have a lot of quirks in the the way the copyrights > > notices are present, > > for example, I found these variants (among others): > > > > /* > > * Copyright (c) 2012-2013 DataTorrent, Inc. > > * All Rights Reserved. > > */ > > > > /* > > * Copyright (c) 2014 DataTorrent, Inc. ALL Rights Reserved. > > */ > > > > /** > > * Put your copyright and license info here. > > */ > > > > A custom script that recognizes and deals with these variants will likely > > do a better job that > > a standard plugin (for example, we need to preserve the original date, we > > cannot change everything > > to 2015). I'm not suggesting it for regular use, just a one-time > > conversion. > > > > So, to repeat, if we already have a solution we like, please disregard my > > script. > > > > Ram > > > > On Wed, Sep 23, 2015 at 4:06 PM, Chetan Narsude <[email protected]> > > wrote: > > > > > Ram, > > > > > > It does have a licensed header which violates the generally followed > > > licensing rules. The first line is blank (there is a issue open to be > > > lenient in that case as I remember I saw sometime ago). There are a few > > > more files with the same viloation. Regardless, I am not saying that > it's > > > perfect. But why reinvent the wheel? You can work with the plugin > writer > > > like 100 others have done and make it better if you feel that it's not > > good > > > enough. > > > > > > A better way to find all the files which violate the formatting is to > > > format the files which do not follow the rule, I get the following. It > > has > > > a few more files than you mentioned: > > > > > > chetan@chubi:~/work/apex$ mvn license:format -Dlicense.skip=false > > > [INFO] Scanning for projects... > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Build Order: > > > [INFO] > > > [INFO] Realtime Stream Processing Framework > > > [INFO] DataTorrent API > > > [INFO] Base Library > > > [INFO] Buffer Server > > > [INFO] Streaming Application Manager > > > [INFO] Apex Application Maven archetype > > > [INFO] Apex App Configuration Maven archetype > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Realtime Stream Processing Framework 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ > dt-framework > > > --- > > > [INFO] Updating license headers... > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building DataTorrent API 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-api --- > > > [INFO] Updating license headers... > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/api/src/main/java/com/datatorrent/api/annotation/Name.java > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Base Library 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-common > --- > > > [INFO] Updating license headers... > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/SingleMetricAggregator.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/sum/LongSumAggregator.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/common/src/main/java/com/datatorrent/common/metric/sum/DoubleSumAggregator.java > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Buffer Server 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ > > dt-bufferserver > > > --- > > > [INFO] Updating license headers... > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/bufferserver/src/main/java/com/datatorrent/bufferserver/util/VarInt.java > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Streaming Application Manager 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ dt-engine > --- > > > [INFO] Updating license headers... > > > [WARNING] Unknown file extension: > > > /home/chetan/work/apex/engine/src/main/scripts/dtcli > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/engine/Slider.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/FieldSignatureVisitor.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/CompactAnnotationNode.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/webapp/asm/CompactFieldNode.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/api/AppDataSource.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/main/java/com/datatorrent/stram/util/PubSubWebSocketServlet.java > > > [INFO] Updating license header in: > > > > > > > > > /home/chetan/work/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java > > > [WARNING] Unable to find a comment style definition for some files. You > > may > > > want to add a custom mapping for the relevant file extensions. > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Apex Application Maven archetype 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ > > > apex-app-archetype --- > > > [INFO] Updating license headers... > > > [INFO] > > > > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Building Apex App Configuration Maven archetype 3.2.0-SNAPSHOT > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] > > > [INFO] --- license-maven-plugin:2.11:format (default-cli) @ > > > apex-conf-archetype --- > > > [INFO] Updating license headers... > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Reactor Summary: > > > [INFO] > > > [INFO] Realtime Stream Processing Framework ............... SUCCESS [ > > > 0.889 s] > > > [INFO] DataTorrent API .................................... SUCCESS [ > > > 0.162 s] > > > [INFO] Base Library ....................................... SUCCESS [ > > > 0.093 s] > > > [INFO] Buffer Server ...................................... SUCCESS [ > > > 0.072 s] > > > [INFO] Streaming Application Manager ...................... SUCCESS [ > > > 0.408 s] > > > [INFO] Apex Application Maven archetype ................... SUCCESS [ > > > 0.233 s] > > > [INFO] Apex App Configuration Maven archetype ............. SUCCESS [ > > > 0.013 s] > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] BUILD SUCCESS > > > [INFO] > > > > ------------------------------------------------------------------------ > > > [INFO] Total time: 2.521 s > > > [INFO] Finished at: 2015-09-23T16:00:02-07:00 > > > [INFO] Final Memory: 13M/471M > > > [INFO] > > > > ------------------------------------------------------------------------ > > > > > > > > > > > > > > > > > > On Wed, Sep 23, 2015 at 4:00 PM, Munagala Ramanath < > [email protected]> > > > wrote: > > > > > > > When I run that command, I get the same error about Name.java but > that > > > file > > > > _does_ have a > > > > license header. > > > > > > > > The files identified by my script as having issues are not shown: > > > > > > > > ============================================================ > > > > 3 Java files with no comments > > > > 0: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java > > > > 1: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java > > > > 2: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java > > > > ============================================================ > > > > 4 Java files with no matching copyright comment > > > > 0: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java > > > > 1: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java > > > > 2: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java > > > > 3: > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java > > > > > > > > > > > > Ram > > > > > > > > On Wed, Sep 23, 2015 at 3:46 PM, Thomas Weise < > [email protected]> > > > > wrote: > > > > > > > > > Ram, > > > > > > > > > > You need to run > > > > > > > > > > mvn license:check -Dlicense.skip=false > > > > > > > > > > I get the following: > > > > > > > > > > [INFO] --- license-maven-plugin:2.11:check (default-cli) @ dt-api > --- > > > > > [INFO] Checking licenses... > > > > > [WARNING] Missing header in: > > > > > > > > > > > > > > > > > > > > /home/hdev/devel/dt/Apex/api/src/main/java/com/datatorrent/api/annotation/Name.java > > > > > > > > > > Thomas > > > > > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 23, 2015 at 3:34 PM, Munagala Ramanath < > > > [email protected]> > > > > > wrote: > > > > > > > > > > > It's in the output.txt file uploaded with my earlier message but > > here > > > > is > > > > > an > > > > > > extract from the end of it: > > > > > > > > > > > > ============================================================ > > > > > > 3 Java files with no comments > > > > > > 0: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/plan/logical/StreamCodecWrapperForPersistance.java > > > > > > 1: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/main/java/com/datatorrent/stram/stream/PartitionAwareSinkForPersistence.java > > > > > > 2: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/java/com/datatorrent/stram/plan/StreamPersistanceTests.java > > > > > > ============================================================ > > > > > > 4 Java files with no matching copyright comment > > > > > > 0: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java > > > > > > 1: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/src/test/resources/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java > > > > > > 2: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/RandomNumberGenerator.java > > > > > > 3: > > > > > > > > > > > > > > > > > > > > > > > > > > > /home/ram/src/apex/apex/engine/target/test-classes/testAppPackage/mydtapp/src/main/java/com/example/mydtapp/StdoutOperator.java > > > > > > > > > > > > > > > > > > On Wed, Sep 23, 2015 at 2:44 PM, Chetan Narsude < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > which files are those? > > > > > > > > > > > > > > -- > > > > > > > chetan > > > > > > > > > > > > > > > > > > > > > On Wed, Sep 23, 2015 at 12:51 PM, Munagala Ramanath < > > > > > [email protected] > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Not sure what it's checking but "*mvn license:check*" shows > me > > no > > > > > list > > > > > > of > > > > > > > > files in core. > > > > > > > > Yet, my script finds at least 3 java files with no comments > at > > > all. > > > > > > > > > > > > > > > > Ram > > > > > > > > > > > > > > > > On Wed, Sep 23, 2015 at 11:54 AM, Chetan Narsude < > > > > > > [email protected] > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > mvn license:check > > > > > > > > > mvn license:format > > > > > > > > > > > > > > > > > > At the top level pom does magic for you. > > > > > > > > > > > > > > > > > > On Wednesday, September 23, 2015, Munagala Ramanath < > > > > > > > [email protected] > > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Thomas and Chris: > > > > > > > > > > > > > > > > > > > > I wrote a Ruby script to automate this process. > Currently, > > it > > > > > makes > > > > > > > no > > > > > > > > > > changes and just > > > > > > > > > > reports what it finds. The script is attached. > > > > > > > > > > > > > > > > > > > > I ran it like this: ruby -w fix-license.rb -f > > > ~/src/apex/apex > > > > > > > > > > output.txt > > > > > > > > > > The output is also attached. > > > > > > > > > > > > > > > > > > > > Let me know if you'd like me to enhance this to actually > > add > > > > the > > > > > > new > > > > > > > > > > copyright notice to the top of > > > > > > > > > > the files where it does detect some form of copyright > > notice > > > > > (there > > > > > > > > will > > > > > > > > > > still be a few files where > > > > > > > > > > no such notice was found and those will need manual > > > > > augmentation). > > > > > > > > > > > > > > > > > > > > Ram > > > > > > > > > > > > > > > > > > > > On Tue, Sep 22, 2015 at 9:30 AM, Chris Nauroth < > > > > > > > > [email protected] > > > > > > > > > > <javascript:_e(%7B%7D,'cvml','[email protected] > > ');>> > > > > > wrote: > > > > > > > > > > > > > > > > > > > >> Hi Thomas, > > > > > > > > > >> > > > > > > > > > >> I think you are correct. Here is the official Apache > > > > statement > > > > > on > > > > > > > > what > > > > > > > > > to > > > > > > > > > >> do with the source headers: > > > > > > > > > >> > > > > > > > > > >> http://www.apache.org/legal/src-headers.html > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> I think your result (the second example below) matches > up > > > with > > > > > > that. > > > > > > > > > >> > > > > > > > > > >> As a reminder, I have a work-in-progress patch for the > > > > non-Java > > > > > > > files > > > > > > > > in > > > > > > > > > >> Apex here: > > > > > > > > > >> > > > > > > > > > >> https://issues.apache.org/jira/browse/APEXCORE-1 > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> It would be nice if we could come up with a script to > chop > > > out > > > > > the > > > > > > > > > >> DataTorrent header and replace it with the ASF header. > > > Then, > > > > we > > > > > > > could > > > > > > > > > >> simply run that across *.java in both Apex and Malhar. > > > > > > > > > >> > > > > > > > > > >> --Chris Nauroth > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > >> On 9/22/15, 9:24 AM, "Thomas Weise" < > > [email protected] > > > > > > > > > >> <javascript:_e(%7B%7D,'cvml','[email protected] > ');>> > > > > > wrote: > > > > > > > > > >> > > > > > > > > > >> >Hit the wrong button... > > > > > > > > > >> > > > > > > > > > > >> >This question is about the license header for the > source > > > > files. > > > > > > > > > Currently > > > > > > > > > >> >we have: > > > > > > > > > >> > > > > > > > > > > >> >/** > > > > > > > > > >> > * Copyright (C) 2015 DataTorrent, Inc. > > > > > > > > > >> > * > > > > > > > > > >> > * Licensed under the Apache License, Version 2.0 (the > > > > > > "License"); > > > > > > > > > >> > * you may not use this file except in compliance with > > the > > > > > > License. > > > > > > > > > >> > * You may obtain a copy of the License at > > > > > > > > > >> > * > > > > > > > > > >> > * http://www.apache.org/licenses/LICENSE-2.0 > > > > > > > > > >> > * > > > > > > > > > >> > * Unless required by applicable law or agreed to in > > > writing, > > > > > > > > software > > > > > > > > > >> > * distributed under the License is distributed on an > "AS > > > IS" > > > > > > > BASIS, > > > > > > > > > >> > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either > > > > express > > > > > > or > > > > > > > > > >> >implied. > > > > > > > > > >> > * See the License for the specific language governing > > > > > > permissions > > > > > > > > and > > > > > > > > > >> > * limitations under the License. > > > > > > > > > >> > */ > > > > > > > > > >> > > > > > > > > > > >> >The first line needs to go. And a reference to NOTICE > > needs > > > > to > > > > > be > > > > > > > > added > > > > > > > > > >> as > > > > > > > > > >> >shown below? > > > > > > > > > >> > > > > > > > > > > >> >/* > > > > > > > > > >> > * Licensed to the Apache Software Foundation (ASF) > under > > > one > > > > > or > > > > > > > more > > > > > > > > > >> > * contributor license agreements. See the NOTICE file > > > > > > distributed > > > > > > > > > with > > > > > > > > > >> > * this work for additional information regarding > > copyright > > > > > > > > ownership. > > > > > > > > > >> > * The ASF licenses this file to You under the Apache > > > > License, > > > > > > > > Version > > > > > > > > > >> 2.0 > > > > > > > > > >> > * (the "License"); you may not use this file except in > > > > > > compliance > > > > > > > > with > > > > > > > > > >> > * the License. You may obtain a copy of the License > at > > > > > > > > > >> > * > > > > > > > > > >> > * http://www.apache.org/licenses/LICENSE-2.0 > > > > > > > > > >> > * > > > > > > > > > >> > * Unless required by applicable law or agreed to in > > > writing, > > > > > > > > software > > > > > > > > > >> > * distributed under the License is distributed on an > "AS > > > IS" > > > > > > > BASIS, > > > > > > > > > >> > * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either > > > > express > > > > > > or > > > > > > > > > >> >implied. > > > > > > > > > >> > * See the License for the specific language governing > > > > > > permissions > > > > > > > > and > > > > > > > > > >> > * limitations under the License. > > > > > > > > > >> > */ > > > > > > > > > >> > > > > > > > > > > >> >On Tue, Sep 22, 2015 at 9:19 AM, Thomas Weise < > > > > > > > > [email protected] > > > > > > > > > >> <javascript:_e(%7B%7D,'cvml','[email protected] > ');>> > > > > > > > > > >> >wrote: > > > > > > > > > >> > > > > > > > > > > >> >> There > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> >> > > > > > > > > > >> > > > > > > > > > >> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
