Denis, I am not sure I understand. We already do have CGI enabled for download.cgi. Is there something else we need?
D. On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dma...@gridgain.com> wrote: > There is an obstacle. There is no way to execute the script using PHP or > similar sever side language and trigger GA as discussed earlier: > https://issues.apache.org/jira/browse/INFRA-15182 > > How else can we tackle this? > > Denis > > On Thursday, September 7, 2017, Dmitriy Setrakyan <dsetrak...@apache.org> > wrote: > > > I think it is safe to assume at this point that everyone is in general > > agreement, since there are no active objections. > > > > I have filed a ticket for the 2.3 release. Let's try to make it happen: > > https://issues.apache.org/jira/browse/IGNITE-6305 > > > > D. > > > > On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan < > dsetrak...@apache.org > > <javascript:;>> > > wrote: > > > > > > > > > > > On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani <raul....@evosent.com > > <javascript:;>> > > > wrote: > > > > > >> Yeah, I guess that's doable as well and requires less management > effort > > >> than my suggestion. We could use events [1] to store payload data > (e.g. > > >> IP, > > >> version, etc.) > > > > > > > > > Yes, we could use events or some other similar API provided by GA. > > > > > > > > >> What the download page CGI developed in? PHP? > > >> > > > > > > To be honest, no clue. I guess someone in the community can figure it > > out: > > > https://svn.apache.org/repos/asf/ignite/site/trunk/download.html > > > > > > > > >> However, I'm not sure whether storing this data in a 3rd party > (Google) > > is > > >> compliant with the ASF policy. I guess it's no biggie, but if there's > > >> doubt > > >> in the PMC, it's better to ask legal@. > > > > > > > > > I am not sure there is anything to ask about. The whole Ignite website > is > > > GA enabled, and all we are doing is accessing a standard web page from > > the > > > Ignite web site. The information gathered from GA is available only to > > the > > > Ignite PMC. Frankly, I think legal@ will be very confused by this > > > question. > > > > > > Even ASF website itself uses GA: https://www.apache.org/ > > > foundation/policies/privacy.html > > > > > > > > >> I think Cos said it's OK; maybe Roman can pitch in. > > >> > > > > > > Sure, would be nice to hear from Roman as well. > > > > > > > > >> Cheers. > > >> > > >> [1] > > >> https://developers.google.com/analytics/devguides/collection > > >> /analyticsjs/events > > >> > > >> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan < > > >> dsetrak...@apache.org <javascript:;>> > > >> wrote: > > >> > > >> > Raul, > > >> > > > >> > Could point about Javascript, it will not work because it executes > in > > >> the > > >> > browser. This means we need a server-side script, like CGI we are > > using > > >> on > > >> > our download page. > > >> > > > >> > How about this approach. We create something like ignite-version.cgi > > >> script > > >> > which will invoke a call to GA and then return the latest version. > > >> > > > >> > This should work, right? > > >> > > > >> > D. > > >> > > > >> > On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani < > raul....@evosent.com > > <javascript:;>> > > >> > wrote: > > >> > > > >> > > Hey Dmitriy and all > > >> > > > > >> > > Also, since we have GA enabled for the website, we can track how > > many > > >> > times > > >> > > > this page was accessed, which will be equal to the number of > > starts. > > >> > This > > >> > > > way, the counter information is tracked and monitored by the > > Ignite > > >> > PMC. > > >> > > > > >> > > > > >> > > Unfortunately this won't work because GA is loaded via Javascript > on > > >> the > > >> > > browser, so Google will never receive the page hit. > > >> > > > > >> > > Given the constraints, the most viable solution is an HTTPS > endpoint > > >> > > running on ASF infra that Ignite invokes via a GET or POST > request. > > >> The > > >> > > simplest thing is to write each request in a log file, along with > > the > > >> > > timestamp, the version reported by the client, maybe the IP (not > > sure > > >> > about > > >> > > the ASF rules about this concerning privacy, I guess it's OK if > you > > >> make > > >> > it > > >> > > an opt-in) and a unique node identifier, i.e. a UUID the node > > creates > > >> on > > >> > > first startup or something. > > >> > > > > >> > > This endpoint would need some basic DDoS protection and > blacklisting > > >> to > > >> > > prevent data spoofing. > > >> > > > > >> > > If we'll be implementing this endpoint anyway, then there's no > point > > >> > > placing another file on Git or elsewhere for reporting the latest > > >> > versions: > > >> > > the endpoint itself can return them. > > >> > > > > >> > > WDYT? > > >> > > > > >> > > Cheers. > > >> > > > > >> > > On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan < > > >> > dsetrak...@apache.org <javascript:;>> > > >> > > wrote: > > >> > > > > >> > > > Cos, Raul, > > >> > > > > > >> > > > Thanks for the feedback. I completely agree about Maven Central > > >> being a > > >> > > 3rd > > >> > > > party repo (did not think about that initially). All your > > >> suggestions > > >> > > make > > >> > > > sense, but I would like to keep it as simple as possible, and so > > far > > >> > > > everything suggested required GIT dependencies and extra work. > > >> > > > > > >> > > > How about Yakov's suggestion. We simply add a page to the Ignite > > >> > website > > >> > > > which will have only the latest version. Every time a node > starts, > > >> it > > >> > > > receives the latest version from the page and suggests that > users > > >> > upgrade > > >> > > > if needed. > > >> > > > > > >> > > > Also, since we have GA enabled for the website, we can track how > > >> many > > >> > > times > > >> > > > this page was accessed, which will be equal to the number of > > starts. > > >> > This > > >> > > > way, the counter information is tracked and monitored by the > > Ignite > > >> > PMC. > > >> > > > > > >> > > > This approach looks pretty innocent to me and everything is kept > > and > > >> > > > managed within Apache. > > >> > > > > > >> > > > Thoughts? > > >> > > > > > >> > > > D. > > >> > > > > > >> > > > > > >> > > > On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik < > > >> c...@apache.org <javascript:;>> > > >> > > > wrote: > > >> > > > > > >> > > > > I agree with Raul. > > >> > > > > - providing a ping-back address to a 3rd party might be frown > > >> upon by > > >> > > > some. > > >> > > > > And might have a consequences like stats collection about > > users' > > >> > > > > infrastructure. > > >> > > > > - checking an ASF git-repo is easy and won't download any > binary > > >> > data: > > >> > > > > everything is clear text and could be easily monitored by > any > > of > > >> > the > > >> > > > > network > > >> > > > > diagnostic tools, shall it be required. But it involves a > bit > > of > > >> > the > > >> > > > > release > > >> > > > > discipline. > > >> > > > > - the binary data download in the runtime is my main concern. > > >> This is > > >> > > the > > >> > > > > vector of MMA. What if someone gains the control over the > > >> > repository > > >> > > > and > > >> > > > > replaces the file with some malicious content. > > >> > > > > > > >> > > > > As for the particular mechanism: IIRC Ignite used to make a > call > > >> to > > >> > an > > >> > > > > external script to check upon the atest software version > > available > > >> > for > > >> > > > > download. In the past, the endpoint was running on a 3rd party > > >> > server, > > >> > > I > > >> > > > > believe the best way would be to put this script on ASF infra > > and > > >> > have > > >> > > > the > > >> > > > > "update checker" running in a completely controlled > environment. > > >> > > > Actually, > > >> > > > > I > > >> > > > > recall we had this very discussion during the Incubation; I > can > > >> > > probably > > >> > > > > dig > > >> > > > > out the corresponding thread. > > >> > > > > > > >> > > > > Thoughts? > > >> > > > > Cok > > >> > > > > > > >> > > > > On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote: > > >> > > > > > Hey guys > > >> > > > > > > > >> > > > > > In my opinion, maven.org is still owned by a third party > > >> > (Sonatype). > > >> > > > We > > >> > > > > > don't know what kind of data analysis or intelligence > > extraction > > >> > they > > >> > > > > run. > > >> > > > > > > > >> > > > > > If Ignite servers all over the world were hitting maven.org > > >> > > > periodically > > >> > > > > > asking for an Ignite Maven artifact, it gives Sonatype a > clear > > >> > > > indication > > >> > > > > > of who is running an Ignite server. > > >> > > > > > > > >> > > > > > They could reverse lookup the IP address and find out what > > >> > > corporation > > >> > > > it > > >> > > > > > is. > > >> > > > > > > > >> > > > > > How about having Ignite check the ASF Git directly? > > >> > > > > > > > >> > > > > > We could use the Git ssh API, but that would require a new > > >> > > dependency, > > >> > > > > > which I advise against. > > >> > > > > > > > >> > > > > > Alternatively, we could have it scrape this HTML for new Git > > >> tags: > > >> > > > > > https://git-wip-us.apache.org/repos/asf?p=ignite.git > > >> > > > > > > > >> > > > > > Another option is to place a txt file in the root of the > > master > > >> > > branch > > >> > > > > (e.g > > >> > > > > > LATEST), containing a list of the latest GA versions for > each > > >> major > > >> > > > > version > > >> > > > > > line (1.x, 2.x). > > >> > > > > > > > >> > > > > > I would advocate this last option, but it requires somebody > > >> > > remembering > > >> > > > > to > > >> > > > > > update the file with every release, unless we automate it > > with a > > >> > > Maven > > >> > > > > > plugin. > > >> > > > > > > > >> > > > > > Hope that helps! > > >> > > > > > > > >> > > > > > > > >> > > > > > On 24 Aug 2017 19:37, "Denis Magda" <dma...@apache.org > > <javascript:;>> wrote: > > >> > > > > > > > >> > > > > > I see nothing wrong with this approach. > > >> > > > > > > > >> > > > > > Cos, Roman, Raul, as Apache veterans, what do you think? Is > it > > >> good > > >> > > to > > >> > > > > go? > > >> > > > > > > > >> > > > > > — > > >> > > > > > Denis > > >> > > > > > > > >> > > > > > > On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan < > > >> > > > dsetrak...@apache.org <javascript:;> > > >> > > > > > > > >> > > > > > wrote: > > >> > > > > > > > > >> > > > > > > Is everyone OK with this approach? Should I file a ticket > on > > >> it? > > >> > > > > > > > > >> > > > > > > On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan < > > >> > > > > dsetrak...@apache.org <javascript:;>> > > >> > > > > > > wrote: > > >> > > > > > > > > >> > > > > > >> Igniters, > > >> > > > > > >> > > >> > > > > > >> There has been lots of talk of proposals about various > > usage > > >> > > metrics > > >> > > > > for > > >> > > > > > >> Ignite and nothing came of it. I would like to resurrect > > the > > >> > topic > > >> > > > and > > >> > > > > > >> propose something very simple and non-intrusive. > > >> > > > > > >> > > >> > > > > > >> 1. Update Checker > > >> > > > > > >> The main purpose of the update checker is not to collect > > >> > metrics, > > >> > > > but > > >> > > > > to > > >> > > > > > >> notify users about a new version of Ignite by accessing > > >> > maven.org > > >> > > > and > > >> > > > > > >> getting the version out of the metadata file: > > >> > > > > > >> http://repo2.maven.org/maven2/ > > org/apache/ignite/ignite-core/ > > >> > > > > > >> maven-metadata.xml > > >> > > > > > >> > > >> > > > > > >> This way we do not send any information anywhere and, at > > the > > >> > same > > >> > > > > time, > > >> > > > > > >> urge our users to download and start using the latest > > >> version of > > >> > > > > Ignite. > > >> > > > > > >> > > >> > > > > > >> 2. Startup Counter > > >> > > > > > >> This piece is optional, but we can also get an insight in > > how > > >> > many > > >> > > > > times > > >> > > > > > a > > >> > > > > > >> certain Ignite release gets started. This is just a cool > > >> metric > > >> > > for > > >> > > > > the > > >> > > > > > >> community to gauge the project popularity. You can think > of > > >> it > > >> > as > > >> > > > of a > > >> > > > > > page > > >> > > > > > >> visit counter shown on many websites. We can even decide > to > > >> > > display > > >> > > > > this > > >> > > > > > >> counter on the Ignite website as well. > > >> > > > > > >> > > >> > > > > > >> To do this, we can simply add a JAR in maven for every > > >> release, > > >> > > e.g. > > >> > > > > > >> ignite-start-counter.jar, which will contain only 1 byte. > > >> Every > > >> > > time > > >> > > > > an > > >> > > > > > >> Ignite node starts, it will download this JAR in the > > >> background. > > >> > > > Then > > >> > > > > we > > >> > > > > > >> will be able to view the number of the total downloads > for > > >> this > > >> > > JAR > > >> > > > in > > >> > > > > > >> Maven Central, which is essentially the number of starts > of > > >> > Ignite > > >> > > > > nodes. > > >> > > > > > >> > > >> > > > > > >> *Note that neither of the above suggestions require > Ignite > > to > > >> > send > > >> > > > or > > >> > > > > > >> track any user information whatsoever.* > > >> > > > > > >> > > >> > > > > > >> Please reply suggesting weather you are OK with this > > >> approach. > > >> > > > > > >> > > >> > > > > > >> D. > > >> > > > > > >> > > >> > > > > > > >> > > > > > > >> > > > > > >> > > > > >> > > > >> > > > > > > > > >
