I can’t see it. Could you share a direct link to it? If we can add one more script like that and it will work then the question is closed.
Denis On Monday, October 2, 2017, Dmitriy Setrakyan <dsetrak...@apache.org> wrote: > Denis, I can see the download.cgi script in SVN. Am I missing something? > > On Mon, Oct 2, 2017 at 5:31 PM, Denis Magda <dma...@apache.org > <javascript:;>> wrote: > > > Dmitriy, > > > > That’s the rule. See replies in the ticket [1] > > > > *Background: the TLP server is already pretty darned busy just serving > > *static* sites. Dynamic operation for near-200 PMCs would bury the > machine. > > Our policy of "static-only websites" has been in place since the > Foundation > > started* > > > > Download scripts seem to be the only exception and we as PMC don’t even > > have access to them. > > > > If you want to keep pushing this direction let’s craft a message to Greg > > and Daniel directly. I don’t know what else to ask for here rather than a > > virtual machine that’s conceivably to much for a single script like that. > > > > [1] https://issues.apache.org/jira/browse/INFRA-15182 < > > https://issues.apache.org/jira/browse/INFRA-15182> > > > > — > > Denis > > > > > On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <dsetrak...@apache.org > <javascript:;>> > > wrote: > > > > > > On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov <voze...@gridgain.com > <javascript:;>> > > > wrote: > > > > > >> I am not sure it is good idea to send requests to 3rd-party addresses > > from > > >> Ignite node. Let's do not make the same mistakes again. > > >> > > > > > > Agree with Vladimir. > > > > > > We obviously have CGI support on the website. Can someone explain why > CGI > > > is not possible to use? > > > > > > > > >> > > >> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov < > anovi...@gridgain.com <javascript:;>> > > >> wrote: > > >> > > >>> We may directly send request to GA from Ignite node: > > >>> https://developers.google.com/analytics/devguides/ > > >> collection/protocol/v1/ > > >>> <https://developers.google.com/analytics/devguides/ > > >> collection/protocol/v1/ > > >>>> > > >>> Latest version can be received from maven central: > > >>> https://repo1.maven.org/maven2/org/apache/ignite/ > > >>> ignite-core/maven-metadata.xml <https://repo1.maven.org/ > > >>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml> > > >>> > > >>> > > >>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <dsetrak...@apache.org > <javascript:;>> > > >>> написал(а): > > >>>> > > >>>> Denis, > > >>>> > > >>>> I am not sure I understand. We already do have CGI enabled for > > >>>> download.cgi. Is there something else we need? > > >>>> > > >>>> D. > > >>>> > > >>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dma...@gridgain.com > <javascript:;>> > > >> wrote: > > >>>> > > >>>>> There is an obstacle. There is no way to execute the script using > PHP > > >> or > > >>>>> similar sever side language and trigger GA as discussed earlier: > > >>>>> https://issues.apache.org/jira/browse/INFRA-15182 > > >>>>> > > >>>>> How else can we tackle this? > > >>>>> > > >>>>> Denis > > >>>>> > > >>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan < > > >>> dsetrak...@apache.org <javascript:;>> > > >>>>> wrote: > > >>>>> > > >>>>>> I think it is safe to assume at this point that everyone is in > > >> general > > >>>>>> agreement, since there are no active objections. > > >>>>>> > > >>>>>> I have filed a ticket for the 2.3 release. Let's try to make it > > >> happen: > > >>>>>> https://issues.apache.org/jira/browse/IGNITE-6305 > > >>>>>> > > >>>>>> D. > > >>>>>> > > >>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan < > > >>>>> dsetrak...@apache.org <javascript:;> > > >>>>>> <javascript:;>> > > >>>>>> wrote: > > >>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani < > > >> raul....@evosent.com <javascript:;> > > >>>>>> <javascript:;>> > > >>>>>>> wrote: > > >>>>>>> > > >>>>>>>> Yeah, I guess that's doable as well and requires less management > > >>>>> effort > > >>>>>>>> than my suggestion. We could use events [1] to store payload > data > > >>>>> (e.g. > > >>>>>>>> IP, > > >>>>>>>> version, etc.) > > >>>>>>> > > >>>>>>> > > >>>>>>> Yes, we could use events or some other similar API provided by > GA. > > >>>>>>> > > >>>>>>> > > >>>>>>>> What the download page CGI developed in? PHP? > > >>>>>>>> > > >>>>>>> > > >>>>>>> To be honest, no clue. I guess someone in the community can > figure > > >> it > > >>>>>> out: > > >>>>>>> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html > > >>>>>>> > > >>>>>>> > > >>>>>>>> However, I'm not sure whether storing this data in a 3rd party > > >>>>> (Google) > > >>>>>> is > > >>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if > > >> there's > > >>>>>>>> doubt > > >>>>>>>> in the PMC, it's better to ask legal@. > > >>>>>>> > > >>>>>>> > > >>>>>>> I am not sure there is anything to ask about. The whole Ignite > > >> website > > >>>>> is > > >>>>>>> GA enabled, and all we are doing is accessing a standard web page > > >> from > > >>>>>> the > > >>>>>>> Ignite web site. The information gathered from GA is available > only > > >> to > > >>>>>> the > > >>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by > this > > >>>>>>> question. > > >>>>>>> > > >>>>>>> Even ASF website itself uses GA: https://www.apache.org/ > > >>>>>>> foundation/policies/privacy.html > > >>>>>>> > > >>>>>>> > > >>>>>>>> I think Cos said it's OK; maybe Roman can pitch in. > > >>>>>>>> > > >>>>>>> > > >>>>>>> Sure, would be nice to hear from Roman as well. > > >>>>>>> > > >>>>>>> > > >>>>>>>> Cheers. > > >>>>>>>> > > >>>>>>>> [1] > > >>>>>>>> https://developers.google.com/analytics/devguides/collection > > >>>>>>>> /analyticsjs/events > > >>>>>>>> > > >>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan < > > >>>>>>>> dsetrak...@apache.org <javascript:;> <javascript:;>> > > >>>>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> Raul, > > >>>>>>>>> > > >>>>>>>>> Could point about Javascript, it will not work because it > > executes > > >>>>> in > > >>>>>>>> the > > >>>>>>>>> browser. This means we need a server-side script, like CGI we > are > > >>>>>> using > > >>>>>>>> on > > >>>>>>>>> our download page. > > >>>>>>>>> > > >>>>>>>>> How about this approach. We create something like > > >> ignite-version.cgi > > >>>>>>>> script > > >>>>>>>>> which will invoke a call to GA and then return the latest > > version. > > >>>>>>>>> > > >>>>>>>>> This should work, right? > > >>>>>>>>> > > >>>>>>>>> D. > > >>>>>>>>> > > >>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani < > > >>>>> raul....@evosent.com <javascript:;> > > >>>>>> <javascript:;>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> Hey Dmitriy and all > > >>>>>>>>>> > > >>>>>>>>>> Also, since we have GA enabled for the website, we can track > how > > >>>>>> many > > >>>>>>>>> times > > >>>>>>>>>>> this page was accessed, which will be equal to the number of > > >>>>>> starts. > > >>>>>>>>> This > > >>>>>>>>>>> way, the counter information is tracked and monitored by the > > >>>>>> Ignite > > >>>>>>>>> PMC. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Unfortunately this won't work because GA is loaded via > > Javascript > > >>>>> on > > >>>>>>>> the > > >>>>>>>>>> browser, so Google will never receive the page hit. > > >>>>>>>>>> > > >>>>>>>>>> Given the constraints, the most viable solution is an HTTPS > > >>>>> endpoint > > >>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST > > >>>>> request. > > >>>>>>>> The > > >>>>>>>>>> simplest thing is to write each request in a log file, along > > with > > >>>>>> the > > >>>>>>>>>> timestamp, the version reported by the client, maybe the IP > (not > > >>>>>> sure > > >>>>>>>>> about > > >>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK > if > > >>>>> you > > >>>>>>>> make > > >>>>>>>>> it > > >>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the node > > >>>>>> creates > > >>>>>>>> on > > >>>>>>>>>> first startup or something. > > >>>>>>>>>> > > >>>>>>>>>> This endpoint would need some basic DDoS protection and > > >>>>> blacklisting > > >>>>>>>> to > > >>>>>>>>>> prevent data spoofing. > > >>>>>>>>>> > > >>>>>>>>>> If we'll be implementing this endpoint anyway, then there's no > > >>>>> point > > >>>>>>>>>> placing another file on Git or elsewhere for reporting the > > latest > > >>>>>>>>> versions: > > >>>>>>>>>> the endpoint itself can return them. > > >>>>>>>>>> > > >>>>>>>>>> WDYT? > > >>>>>>>>>> > > >>>>>>>>>> Cheers. > > >>>>>>>>>> > > >>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan < > > >>>>>>>>> dsetrak...@apache.org <javascript:;> <javascript:;>> > > >>>>>>>>>> wrote: > > >>>>>>>>>> > > >>>>>>>>>>> Cos, Raul, > > >>>>>>>>>>> > > >>>>>>>>>>> Thanks for the feedback. I completely agree about Maven > Central > > >>>>>>>> being a > > >>>>>>>>>> 3rd > > >>>>>>>>>>> party repo (did not think about that initially). All your > > >>>>>>>> suggestions > > >>>>>>>>>> make > > >>>>>>>>>>> sense, but I would like to keep it as simple as possible, and > > so > > >>>>>> far > > >>>>>>>>>>> everything suggested required GIT dependencies and extra > work. > > >>>>>>>>>>> > > >>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the > > Ignite > > >>>>>>>>> website > > >>>>>>>>>>> which will have only the latest version. Every time a node > > >>>>> starts, > > >>>>>>>> it > > >>>>>>>>>>> receives the latest version from the page and suggests that > > >>>>> users > > >>>>>>>>> upgrade > > >>>>>>>>>>> if needed. > > >>>>>>>>>>> > > >>>>>>>>>>> Also, since we have GA enabled for the website, we can track > > how > > >>>>>>>> many > > >>>>>>>>>> times > > >>>>>>>>>>> this page was accessed, which will be equal to the number of > > >>>>>> starts. > > >>>>>>>>> This > > >>>>>>>>>>> way, the counter information is tracked and monitored by the > > >>>>>> Ignite > > >>>>>>>>> PMC. > > >>>>>>>>>>> > > >>>>>>>>>>> This approach looks pretty innocent to me and everything is > > kept > > >>>>>> and > > >>>>>>>>>>> managed within Apache. > > >>>>>>>>>>> > > >>>>>>>>>>> Thoughts? > > >>>>>>>>>>> > > >>>>>>>>>>> D. > > >>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik < > > >>>>>>>> c...@apache.org <javascript:;> <javascript:;>> > > >>>>>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>>> I agree with Raul. > > >>>>>>>>>>>> - providing a ping-back address to a 3rd party might be > frown > > >>>>>>>> upon by > > >>>>>>>>>>> some. > > >>>>>>>>>>>> And might have a consequences like stats collection about > > >>>>>> users' > > >>>>>>>>>>>> infrastructure. > > >>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any > > >>>>> binary > > >>>>>>>>> data: > > >>>>>>>>>>>> everything is clear text and could be easily monitored by > > >>>>> any > > >>>>>> of > > >>>>>>>>> the > > >>>>>>>>>>>> network > > >>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a > > >>>>> bit > > >>>>>> of > > >>>>>>>>> the > > >>>>>>>>>>>> release > > >>>>>>>>>>>> discipline. > > >>>>>>>>>>>> - the binary data download in the runtime is my main > concern. > > >>>>>>>> This is > > >>>>>>>>>> the > > >>>>>>>>>>>> vector of MMA. What if someone gains the control over the > > >>>>>>>>> repository > > >>>>>>>>>>> and > > >>>>>>>>>>>> replaces the file with some malicious content. > > >>>>>>>>>>>> > > >>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make a > > >>>>> call > > >>>>>>>> to > > >>>>>>>>> an > > >>>>>>>>>>>> external script to check upon the atest software version > > >>>>>> available > > >>>>>>>>> for > > >>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd > party > > >>>>>>>>> server, > > >>>>>>>>>> I > > >>>>>>>>>>>> believe the best way would be to put this script on ASF > infra > > >>>>>> and > > >>>>>>>>> have > > >>>>>>>>>>> the > > >>>>>>>>>>>> "update checker" running in a completely controlled > > >>>>> environment. > > >>>>>>>>>>> Actually, > > >>>>>>>>>>>> I > > >>>>>>>>>>>> recall we had this very discussion during the Incubation; I > > >>>>> can > > >>>>>>>>>> probably > > >>>>>>>>>>>> dig > > >>>>>>>>>>>> out the corresponding thread. > > >>>>>>>>>>>> > > >>>>>>>>>>>> Thoughts? > > >>>>>>>>>>>> Cok > > >>>>>>>>>>>> > > >>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote: > > >>>>>>>>>>>>> Hey guys > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party > > >>>>>>>>> (Sonatype). > > >>>>>>>>>>> We > > >>>>>>>>>>>>> don't know what kind of data analysis or intelligence > > >>>>>> extraction > > >>>>>>>>> they > > >>>>>>>>>>>> run. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> If Ignite servers all over the world were hitting > maven.org > > >>>>>>>>>>> periodically > > >>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a > > >>>>> clear > > >>>>>>>>>>> indication > > >>>>>>>>>>>>> of who is running an Ignite server. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> They could reverse lookup the IP address and find out what > > >>>>>>>>>> corporation > > >>>>>>>>>>> it > > >>>>>>>>>>>>> is. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> How about having Ignite check the ASF Git directly? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> We could use the Git ssh API, but that would require a new > > >>>>>>>>>> dependency, > > >>>>>>>>>>>>> which I advise against. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new > Git > > >>>>>>>> tags: > > >>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Another option is to place a txt file in the root of the > > >>>>>> master > > >>>>>>>>>> branch > > >>>>>>>>>>>> (e.g > > >>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for > > >>>>> each > > >>>>>>>> major > > >>>>>>>>>>>> version > > >>>>>>>>>>>>> line (1.x, 2.x). > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I would advocate this last option, but it requires somebody > > >>>>>>>>>> remembering > > >>>>>>>>>>>> to > > >>>>>>>>>>>>> update the file with every release, unless we automate it > > >>>>>> with a > > >>>>>>>>>> Maven > > >>>>>>>>>>>>> plugin. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Hope that helps! > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dma...@apache.org > <javascript:;> > > >>>>>> <javascript:;>> wrote: > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> I see nothing wrong with this approach. > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? Is > > >>>>> it > > >>>>>>>> good > > >>>>>>>>>> to > > >>>>>>>>>>>> go? > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> — > > >>>>>>>>>>>>> Denis > > >>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan < > > >>>>>>>>>>> dsetrak...@apache.org <javascript:;> <javascript:;> > > >>>>>>>>>>>>> > > >>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a ticket > > >>>>> on > > >>>>>>>> it? > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan < > > >>>>>>>>>>>> dsetrak...@apache.org <javascript:;> <javascript:;>> > > >>>>>>>>>>>>>> wrote: > > >>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Igniters, > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> There has been lots of talk of proposals about various > > >>>>>> usage > > >>>>>>>>>> metrics > > >>>>>>>>>>>> for > > >>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to resurrect > > >>>>>> the > > >>>>>>>>> topic > > >>>>>>>>>>> and > > >>>>>>>>>>>>>>> propose something very simple and non-intrusive. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> 1. Update Checker > > >>>>>>>>>>>>>>> The main purpose of the update checker is not to collect > > >>>>>>>>> metrics, > > >>>>>>>>>>> but > > >>>>>>>>>>>> to > > >>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing > > >>>>>>>>> maven.org > > >>>>>>>>>>> and > > >>>>>>>>>>>>>>> getting the version out of the metadata file: > > >>>>>>>>>>>>>>> http://repo2.maven.org/maven2/ > > >>>>>> org/apache/ignite/ignite-core/ > > >>>>>>>>>>>>>>> maven-metadata.xml > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> This way we do not send any information anywhere and, at > > >>>>>> the > > >>>>>>>>> same > > >>>>>>>>>>>> time, > > >>>>>>>>>>>>>>> urge our users to download and start using the latest > > >>>>>>>> version of > > >>>>>>>>>>>> Ignite. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> 2. Startup Counter > > >>>>>>>>>>>>>>> This piece is optional, but we can also get an insight in > > >>>>>> how > > >>>>>>>>> many > > >>>>>>>>>>>> times > > >>>>>>>>>>>>> a > > >>>>>>>>>>>>>>> certain Ignite release gets started. This is just a cool > > >>>>>>>> metric > > >>>>>>>>>> for > > >>>>>>>>>>>> the > > >>>>>>>>>>>>>>> community to gauge the project popularity. You can think > > >>>>> of > > >>>>>>>> it > > >>>>>>>>> as > > >>>>>>>>>>> of a > > >>>>>>>>>>>>> page > > >>>>>>>>>>>>>>> visit counter shown on many websites. We can even decide > > >>>>> to > > >>>>>>>>>> display > > >>>>>>>>>>>> this > > >>>>>>>>>>>>>>> counter on the Ignite website as well. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every > > >>>>>>>> release, > > >>>>>>>>>> e.g. > > >>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 byte. > > >>>>>>>> Every > > >>>>>>>>>> time > > >>>>>>>>>>>> an > > >>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the > > >>>>>>>> background. > > >>>>>>>>>>> Then > > >>>>>>>>>>>> we > > >>>>>>>>>>>>>>> will be able to view the number of the total downloads > > >>>>> for > > >>>>>>>> this > > >>>>>>>>>> JAR > > >>>>>>>>>>> in > > >>>>>>>>>>>>>>> Maven Central, which is essentially the number of starts > > >>>>> of > > >>>>>>>>> Ignite > > >>>>>>>>>>>> nodes. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> *Note that neither of the above suggestions require > > >>>>> Ignite > > >>>>>> to > > >>>>>>>>> send > > >>>>>>>>>>> or > > >>>>>>>>>>>>>>> track any user information whatsoever.* > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this > > >>>>>>>> approach. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>>>>> D. > > >>>>>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>>> > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>> > > >>> > > >> > > > > >
