Hi Cos, Sure, will see you around there.
Anyway, a short summary is the following. Starting Ignite 2.3 the nodes will be connecting to a *static* file [1] deployed on Ignite site to obtain the most recent version. If the file has a later version than the nodes will print out a message encouraging a user to do an upgrade. On top of that, some community members proposed to trigger GA once the file [1] is loaded. But to achieve this the file has to become dynamic being able to execute simple CGI scripts. The INFRA turned this request down. [1] https://ignite.apache.org/latest [2] https://issues.apache.org/jira/browse/INFRA-15182 — Denis > On Oct 24, 2017, at 11:57 PM, Konstantin Boudnik <c...@apache.org> wrote: > > Guys, > > I had a quick chat with Nikita earlier today and it seems that we have came > across of a blocker of some kind. With my member's hat on I would love to > help to find a resolution that let us (and potentially other communities) to > move forward. > > Let's craft the message and circulate it with Infra and Greg. Denis, could > you find me tomorrow at the IMCS and give me an insight into the technical > side of it? I feel a bit at loss after going through this thread... > -- > Regards, > Cos > > On October 2, 2017 7:31:48 AM PDT, Denis Magda <dma...@apache.org> wrote: >> Dmitriy, >> >> That’s the rule. See replies in the ticket [1] >> >> *Background: the TLP server is already pretty darned busy just serving >> *static* sites. Dynamic operation for near-200 PMCs would bury the >> machine. Our policy of "static-only websites" has been in place since >> the Foundation started* >> >> Download scripts seem to be the only exception and we as PMC don’t even >> have access to them. >> >> If you want to keep pushing this direction let’s craft a message to >> Greg and Daniel directly. I don’t know what else to ask for here rather >> than a virtual machine that’s conceivably to much for a single script >> like that. >> >> [1] https://issues.apache.org/jira/browse/INFRA-15182 >> <https://issues.apache.org/jira/browse/INFRA-15182> >> >> — >> Denis >> >>> On Oct 2, 2017, at 2:48 AM, Dmitriy Setrakyan <dsetrak...@apache.org> >> wrote: >>> >>> On Mon, Oct 2, 2017 at 12:46 PM, Vladimir Ozerov >> <voze...@gridgain.com> >>> wrote: >>> >>>> I am not sure it is good idea to send requests to 3rd-party >> addresses from >>>> Ignite node. Let's do not make the same mistakes again. >>>> >>> >>> Agree with Vladimir. >>> >>> We obviously have CGI support on the website. Can someone explain why >> CGI >>> is not possible to use? >>> >>> >>>> >>>> On Mon, Oct 2, 2017 at 12:42 PM, Andrey Novikov >> <anovi...@gridgain.com> >>>> wrote: >>>> >>>>> We may directly send request to GA from Ignite node: >>>>> https://developers.google.com/analytics/devguides/ >>>> collection/protocol/v1/ >>>>> <https://developers.google.com/analytics/devguides/ >>>> collection/protocol/v1/ >>>>>> >>>>> Latest version can be received from maven central: >>>>> https://repo1.maven.org/maven2/org/apache/ignite/ >>>>> ignite-core/maven-metadata.xml <https://repo1.maven.org/ >>>>> maven2/org/apache/ignite/ignite-core/maven-metadata.xml> >>>>> >>>>> >>>>>> 2 окт. 2017 г., в 12:51, Dmitriy Setrakyan <dsetrak...@apache.org> >>>>> написал(а): >>>>>> >>>>>> Denis, >>>>>> >>>>>> I am not sure I understand. We already do have CGI enabled for >>>>>> download.cgi. Is there something else we need? >>>>>> >>>>>> D. >>>>>> >>>>>> On Mon, Oct 2, 2017 at 8:35 AM, Denis Magda <dma...@gridgain.com> >>>> wrote: >>>>>> >>>>>>> There is an obstacle. There is no way to execute the script using >> PHP >>>> or >>>>>>> similar sever side language and trigger GA as discussed earlier: >>>>>>> https://issues.apache.org/jira/browse/INFRA-15182 >>>>>>> >>>>>>> How else can we tackle this? >>>>>>> >>>>>>> Denis >>>>>>> >>>>>>> On Thursday, September 7, 2017, Dmitriy Setrakyan < >>>>> dsetrak...@apache.org> >>>>>>> wrote: >>>>>>> >>>>>>>> I think it is safe to assume at this point that everyone is in >>>> general >>>>>>>> agreement, since there are no active objections. >>>>>>>> >>>>>>>> I have filed a ticket for the 2.3 release. Let's try to make it >>>> happen: >>>>>>>> https://issues.apache.org/jira/browse/IGNITE-6305 >>>>>>>> >>>>>>>> D. >>>>>>>> >>>>>>>> On Sat, Aug 26, 2017 at 3:06 PM, Dmitriy Setrakyan < >>>>>>> dsetrak...@apache.org >>>>>>>> <javascript:;>> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Sat, Aug 26, 2017 at 3:22 AM, Raúl Kripalani < >>>> raul....@evosent.com >>>>>>>> <javascript:;>> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Yeah, I guess that's doable as well and requires less >> management >>>>>>> effort >>>>>>>>>> than my suggestion. We could use events [1] to store payload >> data >>>>>>> (e.g. >>>>>>>>>> IP, >>>>>>>>>> version, etc.) >>>>>>>>> >>>>>>>>> >>>>>>>>> Yes, we could use events or some other similar API provided by >> GA. >>>>>>>>> >>>>>>>>> >>>>>>>>>> What the download page CGI developed in? PHP? >>>>>>>>>> >>>>>>>>> >>>>>>>>> To be honest, no clue. I guess someone in the community can >> figure >>>> it >>>>>>>> out: >>>>>>>>> >> https://svn.apache.org/repos/asf/ignite/site/trunk/download.html >>>>>>>>> >>>>>>>>> >>>>>>>>>> However, I'm not sure whether storing this data in a 3rd party >>>>>>> (Google) >>>>>>>> is >>>>>>>>>> compliant with the ASF policy. I guess it's no biggie, but if >>>> there's >>>>>>>>>> doubt >>>>>>>>>> in the PMC, it's better to ask legal@. >>>>>>>>> >>>>>>>>> >>>>>>>>> I am not sure there is anything to ask about. The whole Ignite >>>> website >>>>>>> is >>>>>>>>> GA enabled, and all we are doing is accessing a standard web >> page >>>> from >>>>>>>> the >>>>>>>>> Ignite web site. The information gathered from GA is available >> only >>>> to >>>>>>>> the >>>>>>>>> Ignite PMC. Frankly, I think legal@ will be very confused by >> this >>>>>>>>> question. >>>>>>>>> >>>>>>>>> Even ASF website itself uses GA: https://www.apache.org/ >>>>>>>>> foundation/policies/privacy.html >>>>>>>>> >>>>>>>>> >>>>>>>>>> I think Cos said it's OK; maybe Roman can pitch in. >>>>>>>>>> >>>>>>>>> >>>>>>>>> Sure, would be nice to hear from Roman as well. >>>>>>>>> >>>>>>>>> >>>>>>>>>> Cheers. >>>>>>>>>> >>>>>>>>>> [1] >>>>>>>>>> https://developers.google.com/analytics/devguides/collection >>>>>>>>>> /analyticsjs/events >>>>>>>>>> >>>>>>>>>> On Sat, Aug 26, 2017 at 12:23 AM, Dmitriy Setrakyan < >>>>>>>>>> dsetrak...@apache.org <javascript:;>> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Raul, >>>>>>>>>>> >>>>>>>>>>> Could point about Javascript, it will not work because it >> executes >>>>>>> in >>>>>>>>>> the >>>>>>>>>>> browser. This means we need a server-side script, like CGI we >> are >>>>>>>> using >>>>>>>>>> on >>>>>>>>>>> our download page. >>>>>>>>>>> >>>>>>>>>>> How about this approach. We create something like >>>> ignite-version.cgi >>>>>>>>>> script >>>>>>>>>>> which will invoke a call to GA and then return the latest >> version. >>>>>>>>>>> >>>>>>>>>>> This should work, right? >>>>>>>>>>> >>>>>>>>>>> D. >>>>>>>>>>> >>>>>>>>>>> On Fri, Aug 25, 2017 at 2:42 PM, Raúl Kripalani < >>>>>>> raul....@evosent.com >>>>>>>> <javascript:;>> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hey Dmitriy and all >>>>>>>>>>>> >>>>>>>>>>>> Also, since we have GA enabled for the website, we can track >> how >>>>>>>> many >>>>>>>>>>> times >>>>>>>>>>>>> this page was accessed, which will be equal to the number >> of >>>>>>>> starts. >>>>>>>>>>> This >>>>>>>>>>>>> way, the counter information is tracked and monitored by >> the >>>>>>>> Ignite >>>>>>>>>>> PMC. >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Unfortunately this won't work because GA is loaded via >> Javascript >>>>>>> on >>>>>>>>>> the >>>>>>>>>>>> browser, so Google will never receive the page hit. >>>>>>>>>>>> >>>>>>>>>>>> Given the constraints, the most viable solution is an HTTPS >>>>>>> endpoint >>>>>>>>>>>> running on ASF infra that Ignite invokes via a GET or POST >>>>>>> request. >>>>>>>>>> The >>>>>>>>>>>> simplest thing is to write each request in a log file, along >> with >>>>>>>> the >>>>>>>>>>>> timestamp, the version reported by the client, maybe the IP >> (not >>>>>>>> sure >>>>>>>>>>> about >>>>>>>>>>>> the ASF rules about this concerning privacy, I guess it's OK >> if >>>>>>> you >>>>>>>>>> make >>>>>>>>>>> it >>>>>>>>>>>> an opt-in) and a unique node identifier, i.e. a UUID the >> node >>>>>>>> creates >>>>>>>>>> on >>>>>>>>>>>> first startup or something. >>>>>>>>>>>> >>>>>>>>>>>> This endpoint would need some basic DDoS protection and >>>>>>> blacklisting >>>>>>>>>> to >>>>>>>>>>>> prevent data spoofing. >>>>>>>>>>>> >>>>>>>>>>>> If we'll be implementing this endpoint anyway, then there's >> no >>>>>>> point >>>>>>>>>>>> placing another file on Git or elsewhere for reporting the >> latest >>>>>>>>>>> versions: >>>>>>>>>>>> the endpoint itself can return them. >>>>>>>>>>>> >>>>>>>>>>>> WDYT? >>>>>>>>>>>> >>>>>>>>>>>> Cheers. >>>>>>>>>>>> >>>>>>>>>>>> On Fri, Aug 25, 2017 at 9:56 PM, Dmitriy Setrakyan < >>>>>>>>>>> dsetrak...@apache.org <javascript:;>> >>>>>>>>>>>> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> Cos, Raul, >>>>>>>>>>>>> >>>>>>>>>>>>> Thanks for the feedback. I completely agree about Maven >> Central >>>>>>>>>> being a >>>>>>>>>>>> 3rd >>>>>>>>>>>>> party repo (did not think about that initially). All your >>>>>>>>>> suggestions >>>>>>>>>>>> make >>>>>>>>>>>>> sense, but I would like to keep it as simple as possible, >> and so >>>>>>>> far >>>>>>>>>>>>> everything suggested required GIT dependencies and extra >> work. >>>>>>>>>>>>> >>>>>>>>>>>>> How about Yakov's suggestion. We simply add a page to the >> Ignite >>>>>>>>>>> website >>>>>>>>>>>>> which will have only the latest version. Every time a node >>>>>>> starts, >>>>>>>>>> it >>>>>>>>>>>>> receives the latest version from the page and suggests that >>>>>>> users >>>>>>>>>>> upgrade >>>>>>>>>>>>> if needed. >>>>>>>>>>>>> >>>>>>>>>>>>> Also, since we have GA enabled for the website, we can >> track how >>>>>>>>>> many >>>>>>>>>>>> times >>>>>>>>>>>>> this page was accessed, which will be equal to the number >> of >>>>>>>> starts. >>>>>>>>>>> This >>>>>>>>>>>>> way, the counter information is tracked and monitored by >> the >>>>>>>> Ignite >>>>>>>>>>> PMC. >>>>>>>>>>>>> >>>>>>>>>>>>> This approach looks pretty innocent to me and everything is >> kept >>>>>>>> and >>>>>>>>>>>>> managed within Apache. >>>>>>>>>>>>> >>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>> >>>>>>>>>>>>> D. >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> On Fri, Aug 25, 2017 at 11:29 AM, Konstantin Boudnik < >>>>>>>>>> c...@apache.org <javascript:;>> >>>>>>>>>>>>> wrote: >>>>>>>>>>>>> >>>>>>>>>>>>>> I agree with Raul. >>>>>>>>>>>>>> - providing a ping-back address to a 3rd party might be >> frown >>>>>>>>>> upon by >>>>>>>>>>>>> some. >>>>>>>>>>>>>> And might have a consequences like stats collection about >>>>>>>> users' >>>>>>>>>>>>>> infrastructure. >>>>>>>>>>>>>> - checking an ASF git-repo is easy and won't download any >>>>>>> binary >>>>>>>>>>> data: >>>>>>>>>>>>>> everything is clear text and could be easily monitored by >>>>>>> any >>>>>>>> of >>>>>>>>>>> the >>>>>>>>>>>>>> network >>>>>>>>>>>>>> diagnostic tools, shall it be required. But it involves a >>>>>>> bit >>>>>>>> of >>>>>>>>>>> the >>>>>>>>>>>>>> release >>>>>>>>>>>>>> discipline. >>>>>>>>>>>>>> - the binary data download in the runtime is my main >> concern. >>>>>>>>>> This is >>>>>>>>>>>> the >>>>>>>>>>>>>> vector of MMA. What if someone gains the control over the >>>>>>>>>>> repository >>>>>>>>>>>>> and >>>>>>>>>>>>>> replaces the file with some malicious content. >>>>>>>>>>>>>> >>>>>>>>>>>>>> As for the particular mechanism: IIRC Ignite used to make >> a >>>>>>> call >>>>>>>>>> to >>>>>>>>>>> an >>>>>>>>>>>>>> external script to check upon the atest software version >>>>>>>> available >>>>>>>>>>> for >>>>>>>>>>>>>> download. In the past, the endpoint was running on a 3rd >> party >>>>>>>>>>> server, >>>>>>>>>>>> I >>>>>>>>>>>>>> believe the best way would be to put this script on ASF >> infra >>>>>>>> and >>>>>>>>>>> have >>>>>>>>>>>>> the >>>>>>>>>>>>>> "update checker" running in a completely controlled >>>>>>> environment. >>>>>>>>>>>>> Actually, >>>>>>>>>>>>>> I >>>>>>>>>>>>>> recall we had this very discussion during the Incubation; >> I >>>>>>> can >>>>>>>>>>>> probably >>>>>>>>>>>>>> dig >>>>>>>>>>>>>> out the corresponding thread. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thoughts? >>>>>>>>>>>>>> Cok >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Fri, Aug 25, 2017 at 10:41AM, Raul Kripalani wrote: >>>>>>>>>>>>>>> Hey guys >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> In my opinion, maven.org is still owned by a third party >>>>>>>>>>> (Sonatype). >>>>>>>>>>>>> We >>>>>>>>>>>>>>> don't know what kind of data analysis or intelligence >>>>>>>> extraction >>>>>>>>>>> they >>>>>>>>>>>>>> run. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> If Ignite servers all over the world were hitting >> maven.org >>>>>>>>>>>>> periodically >>>>>>>>>>>>>>> asking for an Ignite Maven artifact, it gives Sonatype a >>>>>>> clear >>>>>>>>>>>>> indication >>>>>>>>>>>>>>> of who is running an Ignite server. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> They could reverse lookup the IP address and find out >> what >>>>>>>>>>>> corporation >>>>>>>>>>>>> it >>>>>>>>>>>>>>> is. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> How about having Ignite check the ASF Git directly? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> We could use the Git ssh API, but that would require a >> new >>>>>>>>>>>> dependency, >>>>>>>>>>>>>>> which I advise against. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Alternatively, we could have it scrape this HTML for new >> Git >>>>>>>>>> tags: >>>>>>>>>>>>>>> https://git-wip-us.apache.org/repos/asf?p=ignite.git >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Another option is to place a txt file in the root of the >>>>>>>> master >>>>>>>>>>>> branch >>>>>>>>>>>>>> (e.g >>>>>>>>>>>>>>> LATEST), containing a list of the latest GA versions for >>>>>>> each >>>>>>>>>> major >>>>>>>>>>>>>> version >>>>>>>>>>>>>>> line (1.x, 2.x). >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I would advocate this last option, but it requires >> somebody >>>>>>>>>>>> remembering >>>>>>>>>>>>>> to >>>>>>>>>>>>>>> update the file with every release, unless we automate it >>>>>>>> with a >>>>>>>>>>>> Maven >>>>>>>>>>>>>>> plugin. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Hope that helps! >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> On 24 Aug 2017 19:37, "Denis Magda" <dma...@apache.org >>>>>>>> <javascript:;>> wrote: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> I see nothing wrong with this approach. >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> Cos, Roman, Raul, as Apache veterans, what do you think? >> Is >>>>>>> it >>>>>>>>>> good >>>>>>>>>>>> to >>>>>>>>>>>>>> go? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> — >>>>>>>>>>>>>>> Denis >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Aug 23, 2017, at 11:17 PM, Dmitriy Setrakyan < >>>>>>>>>>>>> dsetrak...@apache.org <javascript:;> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Is everyone OK with this approach? Should I file a >> ticket >>>>>>> on >>>>>>>>>> it? >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Aug 21, 2017 at 2:07 PM, Dmitriy Setrakyan < >>>>>>>>>>>>>> dsetrak...@apache.org <javascript:;>> >>>>>>>>>>>>>>>> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Igniters, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> There has been lots of talk of proposals about various >>>>>>>> usage >>>>>>>>>>>> metrics >>>>>>>>>>>>>> for >>>>>>>>>>>>>>>>> Ignite and nothing came of it. I would like to >> resurrect >>>>>>>> the >>>>>>>>>>> topic >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> propose something very simple and non-intrusive. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 1. Update Checker >>>>>>>>>>>>>>>>> The main purpose of the update checker is not to >> collect >>>>>>>>>>> metrics, >>>>>>>>>>>>> but >>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>> notify users about a new version of Ignite by accessing >>>>>>>>>>> maven.org >>>>>>>>>>>>> and >>>>>>>>>>>>>>>>> getting the version out of the metadata file: >>>>>>>>>>>>>>>>> http://repo2.maven.org/maven2/ >>>>>>>> org/apache/ignite/ignite-core/ >>>>>>>>>>>>>>>>> maven-metadata.xml >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> This way we do not send any information anywhere and, >> at >>>>>>>> the >>>>>>>>>>> same >>>>>>>>>>>>>> time, >>>>>>>>>>>>>>>>> urge our users to download and start using the latest >>>>>>>>>> version of >>>>>>>>>>>>>> Ignite. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2. Startup Counter >>>>>>>>>>>>>>>>> This piece is optional, but we can also get an insight >> in >>>>>>>> how >>>>>>>>>>> many >>>>>>>>>>>>>> times >>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>> certain Ignite release gets started. This is just a >> cool >>>>>>>>>> metric >>>>>>>>>>>> for >>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> community to gauge the project popularity. You can >> think >>>>>>> of >>>>>>>>>> it >>>>>>>>>>> as >>>>>>>>>>>>> of a >>>>>>>>>>>>>>> page >>>>>>>>>>>>>>>>> visit counter shown on many websites. We can even >> decide >>>>>>> to >>>>>>>>>>>> display >>>>>>>>>>>>>> this >>>>>>>>>>>>>>>>> counter on the Ignite website as well. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> To do this, we can simply add a JAR in maven for every >>>>>>>>>> release, >>>>>>>>>>>> e.g. >>>>>>>>>>>>>>>>> ignite-start-counter.jar, which will contain only 1 >> byte. >>>>>>>>>> Every >>>>>>>>>>>> time >>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>> Ignite node starts, it will download this JAR in the >>>>>>>>>> background. >>>>>>>>>>>>> Then >>>>>>>>>>>>>> we >>>>>>>>>>>>>>>>> will be able to view the number of the total downloads >>>>>>> for >>>>>>>>>> this >>>>>>>>>>>> JAR >>>>>>>>>>>>> in >>>>>>>>>>>>>>>>> Maven Central, which is essentially the number of >> starts >>>>>>> of >>>>>>>>>>> Ignite >>>>>>>>>>>>>> nodes. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> *Note that neither of the above suggestions require >>>>>>> Ignite >>>>>>>> to >>>>>>>>>>> send >>>>>>>>>>>>> or >>>>>>>>>>>>>>>>> track any user information whatsoever.* >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Please reply suggesting weather you are OK with this >>>>>>>>>> approach. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> D. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>> >>>>> >>>>
