Hi, I wanted to give an update on my intentions regarding the server's handling of the user agent. My current thoughts are to have 3 actions take place upon receiving a new connection:
1. Write a debug level log message "New Connection on channel <channel id> with user agent <user agent>" or "New Connection on channel <channel id> with no user agent provided" 2. Store the user agent as a netty channel attribute. This can later be extracted from a ChannelHandlerContext or Context object if it is deemed useful for some exception. 3. Create and increment a counter metric for each encountered user agent. Further I propose that the format of the user agent be changed from: [Application Name] [GLV Name]/[Version] [Language Runtime Version] [OS]/[Version] [CPU Architecture] [User-Supplied Extension] to: [Application Name] [GLV Name].[Version] [Language Runtime Version] [OS].[Version] [CPU Architecture] The user supplied extension was initially included when the plan was to use the existing user agent request option. Now that these changes are being targeted towards the handshake, any user supplied values can be left untouched in the request options. Also I would like to replace the '/' with a '.' due to an issue with the CSV reporter for metrics on unix-like systems. If there is a slash in the metric name it cannot correctly generate a CSV file to output to. It seems better to me to avoid sending slashes in the first place instead of replacing them on the server side. Please reach out if you have any questions or comments regarding this approach. Thanks, Cole Greer On Thu, Oct 6, 2022 at 6:52 AM Divij Vaidya <[email protected]> wrote: > 4. Yes. Let’s make it configurable with default to enabled please. > > 5. Sure. Even if language have different format for OS runtime, it should > be fine because I think this information should be opaque to the server. We > don’t want to be in the business of trying normalize the format across > gazillions of runtimes. The user / consumer of this information would > handle the normalization. > > 6. Reg: adding to WS handshake, I agree. It’s a nice idea and it saves per > message overhead. We may want to document our WS frame schema so that > clients could work on a standardized structure. > > 7. Reg: HTTP. We can start with WS for now as that is most prevalent > communication protocol in my opinion. Later we can change the server to > accept HTTP headers with user agent info on them. > > New comments: > > 8. How would a user consume this information for the server? Are we > planning to export a metric? Add it to the logs (this might have privacy > implications)? Perhaps we need to think about a “request metadata” log file > separate from application log which contains metadata information about a > request. > > On Tue, Oct 4, 2022 at 1:00 PM Stephen Mallette <[email protected]> > wrote: > > > I don't know of any clients that don't use websockets. The HTTP stuff is > > mostly for curl like requests afaik. At least to me, I don't think we > care > > about tracking this "agent" over raw HTTP requests. I would imagine that > > having it sent once makes the most sense in any event so putting it on > the > > handshake seems right, but i assume the server would need to keep that > > information in its context for the connection somehow so that it could be > > used for logging in relation to a particular query. > > > > On Mon, Oct 3, 2022 at 9:37 PM Ken Hu <[email protected]> > > wrote: > > > > > Hi Stephen, > > > > > > Thanks for your response. > > > > > > After giving it some more thought, I think that I want to modify the > > > proposal. Rather than using the current "userAgent" RequestOption > > argument, > > > I think it might make more sense instead to add a custom User-Agent > > header > > > to the WebSocket handshake. This should prevent breaking most users > since > > > it doesn't seem to exist in any of the clients. It also allows for it > to > > be > > > sent just once during the handshake instead of per request. One > downside > > > would be that the server won't get this same information if the client > is > > > connecting via HTTP, but given that most GLVs seem to use WebSockets > this > > > may be a minor issue. > > > > > > Any thoughts on this change? Is the lack of standardization for > > HTTP-based > > > clients going to be an issue? > > > > > > Thanks, > > > Ken > > > > > > On Fri, Sep 30, 2022 at 4:11 AM Stephen Mallette <[email protected] > > > > > wrote: > > > > > > > Thanks for the proposal Ken. > > > > > > > > > To allow this change in the 3.5-dev branch, we would probably want > to > > > > prevent breaking current users of the "userAgent" header. > > > > > > > > While we aren't losing any current userAgent values by appending to > the > > > > end, I think that we're still potentially establishing a breaking > > change > > > as > > > > the value that arrives to a server in 3.5.5 won't be the same as the > > one > > > > that arrives from 3.5.4. > > > > > > > > That said, I think that tiny break is acceptable as I believe this > > > feature > > > > to be not widely used or known by anyone. If there are any objections > > to > > > > this change for 3.5.5, I'd say it should go in 3.6.2 as providers are > > > still > > > > consuming that line and the impact lessened. > > > > > > > > > > > > > > > > On Thu, Sep 29, 2022 at 6:36 PM Ken Hu <[email protected] > > > > > > wrote: > > > > > > > > > Hi Divij, > > > > > > > > > > Thanks for the response. My thoughts are below. > > > > > > > > > > 1. Is "space" a good delimiter between fields? > > > > > In my opinion, space should work as a delimiter for these fields as > > > long > > > > as > > > > > we ensure that the fields the driver creates don't contain any > > spaces. > > > > > Though any other delimiter would work fine. I chose space since > that > > is > > > > > what is used in HTTP. Did you have a different delimiter in mind? > > > > > > > > > > 2. Are the individual fields nullable/optional? > > > > > Yes, they are nullable/optional with the exception of [GLV > > > > Name]/[Version]. > > > > > If a particular field can't be populated then we will probably just > > > add a > > > > > value like NotApplicable. > > > > > > > > > > 3. Would we have any client & server side validation to assert the > > > > > structure of UserAgent string? > > > > > The client side would be implemented in a way that all the GLVs > would > > > > > produce a string with the same structure. There is unlikely to be > > > > > validation on the server side for now although that should be > > > relatively > > > > > straightforward to add. > > > > > > > > > > 4. Would users have concern with sharing the details of their > > > environment > > > > > due to privacy reasons? > > > > > This is possible, however, we wouldn't be collecting anything > > > personally > > > > > identifiable. We could also add in a connection option that would > > > disable > > > > > this feature. > > > > > > > > > > 5. Do all languages have methods that provide details of the > runtime > > > > > environment? Is it structured similarly across languages? > > > > > All the current languages have the ability to gather information > for > > > the > > > > > proposed fields with the possible exception of Go which can't > easily > > > > report > > > > > OS details. There are some minor differences that would need to be > > > sorted > > > > > out. For example, some OS/runtimes report the same 64bit > architecture > > > as > > > > > amd64 or x86_64. > > > > > > > > > > Let me know if you have any further questions. > > > > > > > > > > Thanks, > > > > > Ken > > > > > > > > > > On Thu, Sep 29, 2022 at 4:35 AM Divij Vaidya < > > [email protected]> > > > > > wrote: > > > > > > > > > > > Hey Ken > > > > > > > > > > > > Thank you for the proposal. I have a few questions and would > > solicit > > > > your > > > > > > thoughts on these. > > > > > > > > > > > > 1. Is "space" a good delimiter between fields? > > > > > > 2. Are the individual fields nullable/optional? > > > > > > 3. Would we have any client & server side validation to assert > the > > > > > > structure of UserAgent string? > > > > > > 4. Would users have concern with sharing the details of their > > > > > > environment due to privacy reasons? > > > > > > 5. Do all languages have methods that provide details of the > > runtime > > > > > > environment? Is it structured similarly across languages? > > > > > > > > > > > > -- > > > > > > Divij Vaidya > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Sep 29, 2022 at 2:57 AM Ken Hu > > <[email protected] > > > > > > > > > > wrote: > > > > > > > > > > > > > Hi All, > > > > > > > > > > > > > > With regard to TINKERPOP-2480, I propose that we formalize the > > > usage > > > > of > > > > > > the > > > > > > > "userAgent" header in TinkerPop. Currently, this is a > > user-supplied > > > > > value > > > > > > > that can be any custom string. Because this header is often > > > missing, > > > > it > > > > > > > can't be used to determine which GLV is being used to connect > to > > > the > > > > > > > server. Instead, the GLVs should automatically add a value for > > this > > > > > > header. > > > > > > > > > > > > > > The proposed format would be as follows: > > > > > > > [Application Name] [GLV Name]/[Version] [Language Runtime > > Version] > > > > > > > [OS]/[Version] [CPU Architecture] [User-Supplied Extension] > > > > > > > > > > > > > > Spaces would be removed from each part (such as Application > Name) > > > in > > > > > > order > > > > > > > to make it easier to parse the string. > > > > > > > > > > > > > > To allow this change in the 3.5-dev branch, we would probably > > want > > > to > > > > > > > prevent breaking current users of the "userAgent" header. To > > > > accomplish > > > > > > > this, I propose that we append any user-supplied information to > > the > > > > end > > > > > > of > > > > > > > the string (it becomes the User-Supplied Extension). > > > > > > > > > > > > > > Let's walk through a simple example. > > > > > > > > > > > > > > Given this sample code: > > > > > > > client.submit("g.V().count()", > > > > > > > > > > > > > > > > RequestOptions.build().userAgent("myUserSuppliedValue").create()).all(); > > > > > > > > > > > > > > Currently, the args for RequestMessage would contain: > > > > > > > userAgent=myUserSuppliedValue > > > > > > > > > > > > > > The proposed RequestMessage, however, would contain: > > > > > > > userAgent=TestApplication gremlin-driver/3.5.4 11.0.16 > > MacOSX/12.6 > > > > > x86_64 > > > > > > > myUserSuppliedValue > > > > > > > > > > > > > > Any comments or suggestions to this proposal would be > > appreciated. > > > > > > > > > > > > > > Thanks, > > > > > > > Ken > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > Divij Vaidya >
