This is an automated email from the ASF dual-hosted git repository. kenhuuu pushed a commit to branch 3.8-documentation in repository https://gitbox.apache.org/repos/asf/tinkerpop.git
commit c28a1d0650d8f24e1bb7e8a33003e4dd3740171b Author: Ken Hu <[email protected]> AuthorDate: Wed Oct 29 14:47:01 2025 -0700 Re-organize upgrade documentation based on severity --- docs/src/upgrade/release-3.8.x.asciidoc | 970 ++++++++++++++++---------------- 1 file changed, 472 insertions(+), 498 deletions(-) diff --git a/docs/src/upgrade/release-3.8.x.asciidoc b/docs/src/upgrade/release-3.8.x.asciidoc index 9f254b581d..58c053a741 100644 --- a/docs/src/upgrade/release-3.8.x.asciidoc +++ b/docs/src/upgrade/release-3.8.x.asciidoc @@ -30,171 +30,16 @@ complete list of all the modifications that are part of this release. === Upgrading for Users -==== Gremlin MCP Server +==== Breaking Language Changes -Gremlin MCP Server is an experimental application that implements the link:https://modelcontextprotocol.io/[Model Context Protocol] -(MCP) to expose Gremlin Server-backed graph operations to MCP-capable clients such as Claude Desktop, Cursor, or -Windsurf. Through this integration, graph structure can be discovered, and Gremlin traversals can be executed. Basic -health checks are included to validate connectivity. - -A running Gremlin Server that fronts the target TinkerPop graph is required. An MCP client can be configured to connect -to the Gremlin MCP Server endpoint. - -==== Air Routes Dataset - -The Air Routes sample dataset has long been used to help showcase and teach Gremlin. Popularized by the first edition -of link:https://kelvinlawrence.net/book/PracticalGremlin.html[Practical Gremlin], this dataset offers a real-world graph -structure that allows for practical demonstration of virtually every feature that Gremlin syntax has to offer. While it -was easy to simply get the dataset from the Practical Gremlin link:https://github.com/krlawrence/graph[repository], -including it with the TinkerPop distribution makes it much more convenient to use with Gremlin Server, Gremlin Console, -or directly in code that depends on the `tinkergraph-gremlin` package. - -[source,text] ----- -plugin activated: tinkerpop.tinkergraph -gremlin> graph = TinkerFactory.createAirRoutes() -==>tinkergraph[vertices:3619 edges:50148] -gremlin> g = traversal().with(graph) -==>graphtraversalsource[tinkergraph[vertices:3619 edges:50148], standard] -gremlin> g.V().has('airport','code','IAD').valueMap('code','desc','lon','lat') -==>[code:[IAD],lon:[-77.45580292],lat:[38.94449997],desc:[Washington Dulles International Airport]] ----- - -TinkerPop distributes the 1.0 version of the dataset. - -==== Type Predicate - -The new `P.typeOf()` predicate allows filtering traversers based on their runtime type. It accepts either a `GType` -enum constant or a string representation of a simple class name. This predicate is particularly useful for type-safe -filtering in heterogeneous data scenarios. - -[source,text] ----- -// Filter vertices by property type -gremlin> g.V().values("age","name").is(P.typeOf(GType.INT)) -==>29 -==>27 -==>32 -==>35 - -// Type inheritance support - NUMBER matches all numeric types -gremlin> g.union(V(), E()).values().is(P.typeOf(GType.NUMBER)) -==>29 -==>27 -==>32 -==>35 -==>0.5 -==>1.0 -==>0.4 -==>1.0 -==>0.4 -==>0.2 ----- - -The predicate supports type inheritance where `GType.NUMBER` will match any numeric type. Invalid type names will -throw an exception at execution time. - -See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#a-note-on-predicates[Predicates], link:https://issues.apache.org/jira/browse/TINKERPOP-2234[TINKERPOP-2234] - -==== Number Conversion Step - -The new `asNumber()` step provides type casting functionality to Gremlin. It serves as an umbrella step that parses -strings and casts numbers into desired types. For the convenience of remote traversals in GLVs, these available types -are denoted by a set of number tokens (`GType`). - -This new step will allow users to normalize their data by converting string numbers and mixed numeric types to a -consistent format, making it easier to perform downstream mathematical operations. As an example: - -[source,text] ----- -// sum() step can only take numbers -gremlin> g.inject(1.0, 2l, 3, "4", "0x5").sum() -class java.lang.String cannot be cast to class java.lang.Number - -// use asNumber() to avoid casting exceptions -gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum() -==>15.0 - -// given sum() step returned a double, one can use asNumber() to further cast the result into desired type -gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum().asNumber(GType.INT) -==>15 ----- - -Semantically, the `asNumber()` step will convert the incoming traverser to a logical parsable type if no argument is -provided, or to the desired numerical type, based on the number token (`GType`) provided. - -Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown -for any overflow as a result of narrowing of types: - -[source,text] ----- -gremlin> g.inject(5.0).asNumber(GType.INT) -==> 5 // casts double to int -gremlin> g.inject(12).asNumber(GType.BYTE) -==> 12 -gremlin> g.inject(128).asNumber(GType.BYTE) -==> ArithmeticException ----- - -String input will be parsed. By default, the smalled unit of number to be parsed into is `int` if no type token is -provided. `NumberFormatException` will be thrown for any unparsable strings: - -[source,text] ----- -gremlin> g.inject("5").asNumber() -==> 5 -gremlin> g.inject("5.7").asNumber(GType.INT) -==> 5 -gremlin> g.inject("1,000").asNumber(GType.INT) -==> NumberFormatException -gremlin> g.inject("128").asNumber(GType.BYTE) -==> ArithmeticException ----- - -All other input types will result in `IllegalArgumentException`: -[source,text] ----- -gremlin> g.inject([1, 2, 3, 4]).asNumber() -==> IllegalArgumentException ----- - -See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asNumber-step[asNumber()-step], link:https://issues.apache.org/jira/browse/TINKERPOP-3166[TINKERPOP-3166] - -==== Boolean Conversion Step - -The `asBool()` step bridges another gap in Gremlin's casting functionalities. Users now have the ability to parse -strings and numbers into boolean values, both for normalization and to perform boolean logic with numerical values. - -[source,text] ----- -gremlin> g.inject(2, "true", 1, 0, false, "FALSE").asBool().fold() -==>[true,true,true,false,false,false] - -// using the modern graph, we can turn count() results into boolean values -gremlin> g.V().local(outE().count()).fold() -==>[3,0,0,2,0,1] -gremlin> g.V().local(outE().count()).asBool().fold() -==>[true,false,false,true,false,true] -// a slightly more complex one using sack for boolean operations for vertices with both 'person' label and has out edges -gremlin> g.V().sack(assign).by(__.hasLabel('person').count().asBool()).sack(and).by(__.outE().count().asBool()).sack().path() -==>[v[1],true] -==>[v[2],false] -==>[v[3],false] -==>[v[4],true] -==>[v[5],false] -==>[v[6],true] ----- - -See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asBool-step[asBool()-step], link:https://issues.apache.org/jira/browse/TINKERPOP-3175[TINKERPOP-3175] - -==== none() and discard() +===== none() and discard() There is a complicated relationship with the `none()` and `discard()` steps that begs some discussion. Prior to this version, the `none()` step was used to "throw away" all traversers that passed into it. In 3.8.0, that step has been renamed to `discard()`. The `discard()` step with its verb tone arguably makes for a better name for that feature, but it also helped make room for `none()` to be repurposed as `none(P)` which is a complement to `any(P)` and `all(P) steps. -==== Prevented using cap(), inject() inside repeat() +===== Prevented using cap(), inject() inside repeat() `cap()` inside `repeat()` is now disallowed by the `StandardVerificationStrategy`. Using `cap()` inside `repeat()` would have led to unexpected results since `cap()` isn't "repeat-aware". Because `cap()` is a `SupplyingBarrier` that reduces @@ -254,7 +99,250 @@ gremlin> g.inject('x').repeat(union(constant(['a','b']).limit(1).unfold(),identi ==>x ---- -==== Simplified Comparability Semantics +===== `aggregate()` with `Scope` Removed + +The meaning of `Scope` parameters in `aggregate()` have always been unique compared to all other "scopable" steps. +`aggregate(global)` is a `Barrier`, which blocks the traversal until all traversers have been aggregated into the side +effect, where `aggregate(local)` is non-blocking, and will allow traversers to pass before the side effect has been +fully aggregated. This is inconsistent with the semantics of `Scope` in all other steps. For example `dedup(global)` +filters duplicates across the entire traversal stream, while `dedup(local)` filters duplicates within individual `List` +traversers. + +The `Scope` parameter is being removed from `aggregate()` to fix inconsistency between the two different use cases: flow +control vs. per-element application. This change aligns all side effect steps (none of the others have scope arguments) +and reserves the `Scope` parameter exclusively for "traverser-local" application patterns, eliminating confusion about +its contextual meanings. + +This makes the `AggregateStep` globally scoped by default with eager aggregation. The Lazy evaluation with `aggregate()` is +achieved by wrapping the step in `local()`. + +[source,text] +---- +// 3.7.x - scope is still supported +gremlin> g.V().aggregate(local, "x").by("age").select("x") +==>[29] +==>[29,27] +==>[29,27] +==>[29,27,32] +==>[29,27,32] +==>[29,27,32,35] + +// 3.8.0 - must use aggregate() within local() to achieve lazy aggregation +gremlin> g.V().local(aggregate("x").by("age")).select("x") +==>[29] +==>[29,27] +==>[29,27] +==>[29,27,32] +==>[29,27,32] +==>[29,27,32,35] +---- + +An slight behavioral difference exists between the removed `aggregate(local)` and its replacement `local(aggregate())` +with respect to handling of bulked traversers. In 3.8.0, `local()` changed from traverser-local to object-local processing, +always debulking incoming traversers into individual objects. This causes `local(aggregate())` to show true lazy, 1 object +at a time aggregation, differing from the original `aggregate(local)`, which always consumed bulked traversers atomically. +There is no workaround to preserve the old "traverser-local" semantics. + +[source,text] +---- +// 3.7.x - both local() and local scope will preserve bulked traversers +gremlin> g.V().out().barrier().aggregate(local, "x").select("x") +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3],v[2]] +==>[v[3],v[3],v[3],v[2],v[4]] +==>[v[3],v[3],v[3],v[2],v[4],v[5]] +gremlin> g.V().out().barrier().local(aggregate("x")).select("x") +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3],v[2]] +==>[v[3],v[3],v[3],v[2],v[4]] +==>[v[3],v[3],v[3],v[2],v[4],v[5]] + +// 3.8.0 - bulked traversers are now split to be processed per-object, this affects local aggregation +gremlin> g.V().out().barrier().local(aggregate("x")).select("x") +==>[v[3]] +==>[v[3],v[3]] +==>[v[3],v[3],v[3]] +==>[v[3],v[3],v[3],v[2]] +==>[v[3],v[3],v[3],v[2],v[4]] +==>[v[3],v[3],v[3],v[2],v[4],v[5]] +---- + +See: link:https://github.com/apache/tinkerpop/blob/master/docs/src/dev/future/proposal-scoping-5.asciidoc[Lazy vs. Eager Evaluation] + +===== Removal of `store()` Step + +The `store()` step was a legacy name for `aggregate(local)` that has been deprecated since 3.4.3, and is now removed along +with `aggregate(local)`. To achieve lazy aggregation, use `aggregate()` within `local()`. + +[source,text] +---- +// 3.7.x - store() is still allowed +gremlin> g.V().store("x").by("age").cap("x") +==>[29,27,32,35] + +// 3.8.0 - store() removed, use local(aggregate()) to achieve lazy aggregation +gremlin> g.V().local(aggregate("x").by("age")).cap("x") +==>[29,27,32,35] +---- + +===== Removal of has(key, traversal) + +The has(key, traversal) API has been removed in version 3.8.0 due to its confusing behavior that differed from other +has() variants. As well, most has(key, traversal) usage indicates a misunderstanding of the API. Unlike has(key, value) +which performs equality comparison, has(key, traversal) only checked if the traversal produced any result, creating +inconsistent semantics. + +[source,text] +---- +// 3.7.x - this condition is meaningless but yields result because count() is productive +gremlin> g.V().has("age", __.count()) +==>v[1] +==>v[2] +==>v[3] +==>v[4] +==>v[5] +==>v[6] +// simple example +gremlin> g.V().has("age", __.is(P.gt(30))) +==>v[4] +==>v[6] + +// 3.8.0 - traversals no longer yield results, for proper use cases consider using predicate or where() for filtering +gremlin> g.V().has("age", __.count()) +gremlin> g.V().has("age", __.is(P.gt(30))) +gremlin> g.V().has("age", P.gt(30)) +==>v[4] +==>v[6] +---- + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-1463[TINKERPOP-1463] + +===== Removal of P.getOriginalValue() + +`P.getOriginalValue()` has been removed as it was not offering much value and was often confused with `P.getValue()`. +Usage of `P.getOriginalValue()` often leads to unexpected results if called on a predicate which has had its value reset +after construction. All usages of `P.getOriginalValue()` should be replaced with `P.getValue()`. + +===== SeedStrategy Construction + +The `SeedStrategy` public constructor has been removed for Java and has been replaced by the builder pattern common +to all strategies. This change was made to ensure that the `SeedStrategy` could be constructed consistently. + +===== OptionsStrategy in Python + +The `\\__init__()` syntax has been updated to be both more Pythonic and more aligned to the `gremlin-lang` syntax. +Previously, `OptionsStrategy()` took a single argument `options` which was a `dict` of all options to be set. +Now, all options should be set directly as keyword arguments. + +For example: + +[source,python] +---- +# 3.7 and before: +g.with_strategies(OptionsStrategy(options={'key1': 'value1', 'key2': True})) +# 4.x and newer: +g.with_strategies(OptionsStrategy(key1='value1', key2=True)) + +myOptions = {'key1': 'value1', 'key2': True} +# 3.7 and before: +g.with_strategies(OptionsStrategy(options=myOptions)) +# 4.x and newer: +g.with_strategies(OptionsStrategy(**myOptions)) +---- + +===== Remove Undocumented `with()` modulation + +There has long been a connection between the `with()` modulator, and mutating steps due to the design of +some of the interfaces in the gremlin traversal engine. This has led to several undocumented usages of the +`with()` modulator which have never been officially supported but have previously been functional. + +As of 3.8.0 `with()` modulation of the following steps will no longer work: `addV()`, `addE()`, `property()`, `drop()`, +`mergeV()`, and `mergeE()`. + + +==== Breaking Behavioral Changes + +===== Serialization Changes + +*Properties on Element Serialization in Python & Javascript* + +Element properties handling has been inconsistent across GLVs. Previously,`gremlin-python` deserialized empty properties +as None or array depending on the serializer, while `gremlin-javascript` returned properties as objects or arrays, with +empty properties as empty lists or undefined depending on the serializer. + +This inconsistency is now resolved, aligning to how properties are handled in Gremlin core and in the Java GLV. +Both GLVs will deserialize element properties into lists of property objects, returning empty lists instead of null values +for missing properties. + +For python, the most notable difference is in graphSON when "tokens" is turned on for "materializeProperties". The +properties returned are no longer `None`, but empty lists. Users should update their code accordingly. + +For javascript, the change is slightly more extensive, as user should no longer expect javascript objects to be returned. +All properties are returned as lists of Property or VertexProperty objects. + +[source,javascript] +---- +// 3.7 and before: +g.with_("materializeProperties", "tokens").V(1).next() // skip properties with token +// graphson will return properties as a javascript object, which becomes undefined +Vertex { id: 1, label: 'person', properties: undefined } +// graphbinary will return properties as empty lists +Vertex { id: 1, label: 'person', properties: [] } + +g.V(1).next() // properties returned +// graphson will return properties as a javascript object +Vertex { + id: 1, + label: 'person', + properties: { name: [Array], age: [Array] } +} +// graphbinary will return properties as lists of VertexProperty objects +Vertex { + id: 1, + label: 'person', + properties: [ [VertexProperty], [VertexProperty] ] +} + +// 3.8.0 and newer - properties are always arrays, empty array [] for missing properties: +g.with_("materializeProperties", "tokens").V(1).next() // skip properties with token +// both graphson and graphbinary return +Vertex { id: 1, label: 'person', properties: [] } +g.V(1).next() +// both graphson and graphbinary return +Vertex { + id: 1, + label: 'person', + properties: [ [VertexProperty], [VertexProperty] ] +} + +---- + +This change only affects how GLVs deserialize property data in client applications. The underlying graph serialization +formats and server-side behavior remain unchanged. + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-3186[TINKERPOP-3186] + +*Javascript Set Deserialization* + +Starting from this version, `gremlin-javascript` will deserialize `Set` data into a ECMAScript 2015 Set. Previously, +these were deserialized into arrays. + +*.NET Byte Serialization Change* + +The Gremlin .NET serializers has been updated to correctly handle byte values as signed integers to align with the IO +specification, whereas previously it incorrectly serialized and deserialized bytes as unsigned values. + +This is a breaking change for .NET applications that rely on byte values. Existing applications using byte values +should consider switching to `sbyte` for signed byte operations or `short` for a wider range of values to maintain +compatibility. + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-3161[TINKERPOP-3161] + +===== Simplified Comparability Semantics The previous system of ternary boolean semantics has been replaced with simplified binary semantics. The triggers for "ERROR" states from illegal comparisons are unchanged (typically comparisons with NaN or between incomparable types @@ -270,11 +358,7 @@ See: link:https://tinkerpop.apache.org/docs/3.8.0/dev/provider/#gremlin-semantic See: link:https://issues.apache.org/jira/browse/TINKERPOP-3173[TINKERPOP-3173] -==== Set minimum Java version to 11 - -TinkerPop 3.8 requires a minimum of Java 11 for building and running. Support for Java 1.8 has been dropped. - -==== Auto-promotion of Numbers +===== Auto-promotion of Numbers Previously, operations like `sum` or `sack` that involved mathematical calculations did not automatically promote the result to a larger numeric type (e.g., from int to long) when needed. As a result, values could wrap around within their @@ -316,7 +400,7 @@ gremlin> g.inject([Float.MAX_VALUE, Float.MAX_VALUE], [Double.MAX_VALUE, Double. See link:https://issues.apache.org/jira/browse/TINKERPOP-3115[TINKERPOP-3115] -==== repeat() Step Global Children Semantics Change +===== repeat() Step Global Children Semantics Change The `repeat()` step has been updated to treat the repeat traversal as a global child in all cases. Previously, the repeat traversal behaved as a hybrid between local and global semantics, which could lead to unexpected results in @@ -414,7 +498,7 @@ gremlin> g.V().local(repeat(both().simplePath().order().by("name")).times(2)).pa See: link:https://issues.apache.org/jira/browse/TINKERPOP-3200[TINKERPOP-3200] -==== Prefer OffsetDateTime +===== Prefer OffsetDateTime The default implementation for date type in Gremlin is now changed from the `java.util.Date` to the more encompassing `java.time.OffsetDateTime`. This means the reference implementation for all date manipulation steps, `asDate()`, @@ -439,135 +523,7 @@ For Java GLV, this change would impact users who are expecting the old `Date` ob application, in this case the recommendation is to update code to expect `OffsetDateTime` as part of the version upgrade. -==== Simplify g Construction - -The creation of "g" is the start point to writing Gremlin. There are a number of ways to create it, but TinkerPop has -long recommended the use of the anonymous `traversal()` function for this creation. - -[source,groovy] ----- -// for embedded cases -graph = TinkerGraph.open() -g = traversal().withEmbedded(graph) -// for remote cases -g = traversal().withRemote(DriverRemoteConnection.using(...))) ----- - -As of this release, those two methods have been deprecated in favor of just `with()` which means you could simply write: - -[source,groovy] ----- -// for embedded cases -graph = TinkerGraph.open() -g = traversal().with(graph) -// for remote cases -g = traversal().with(DriverRemoteConnection.using(...))) ----- - -That's a bit less to type, but also removes the need to programmatically decide which function to call, which hopefully -strengthens the abstraction further. To demonstrate this further, consider this next example: - -[source,groovy] ----- -g = traversal().with("config.properties") ----- - -The properties file in the above example can either point to a remote configuration or a embedded configuration allowing -"g" to be switched as needed without code changes. - -See: link:https://issues.apache.org/jira/browse/TINKERPOP-3017[TINKERPOP-3017] - -==== `aggregate()` with `Scope` Removed - -The meaning of `Scope` parameters in `aggregate()` have always been unique compared to all other "scopable" steps. -`aggregate(global)` is a `Barrier`, which blocks the traversal until all traversers have been aggregated into the side -effect, where `aggregate(local)` is non-blocking, and will allow traversers to pass before the side effect has been -fully aggregated. This is inconsistent with the semantics of `Scope` in all other steps. For example `dedup(global)` -filters duplicates across the entire traversal stream, while `dedup(local)` filters duplicates within individual `List` -traversers. - -The `Scope` parameter is being removed from `aggregate()` to fix inconsistency between the two different use cases: flow -control vs. per-element application. This change aligns all side effect steps (none of the others have scope arguments) -and reserves the `Scope` parameter exclusively for "traverser-local" application patterns, eliminating confusion about -its contextual meanings. - -This makes the `AggregateStep` globally scoped by default with eager aggregation. The Lazy evaluation with `aggregate()` is -achieved by wrapping the step in `local()`. - -[source,text] ----- -// 3.7.x - scope is still supported -gremlin> g.V().aggregate(local, "x").by("age").select("x") -==>[29] -==>[29,27] -==>[29,27] -==>[29,27,32] -==>[29,27,32] -==>[29,27,32,35] - -// 3.8.0 - must use aggregate() within local() to achieve lazy aggregation -gremlin> g.V().local(aggregate("x").by("age")).select("x") -==>[29] -==>[29,27] -==>[29,27] -==>[29,27,32] -==>[29,27,32] -==>[29,27,32,35] ----- - -An slight behavioral difference exists between the removed `aggregate(local)` and its replacement `local(aggregate())` -with respect to handling of bulked traversers. In 3.8.0, `local()` changed from traverser-local to object-local processing, -always debulking incoming traversers into individual objects. This causes `local(aggregate())` to show true lazy, 1 object -at a time aggregation, differing from the original `aggregate(local)`, which always consumed bulked traversers atomically. -There is no workaround to preserve the old "traverser-local" semantics. - -[source,text] ----- -// 3.7.x - both local() and local scope will preserve bulked traversers -gremlin> g.V().out().barrier().aggregate(local, "x").select("x") -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3],v[2]] -==>[v[3],v[3],v[3],v[2],v[4]] -==>[v[3],v[3],v[3],v[2],v[4],v[5]] -gremlin> g.V().out().barrier().local(aggregate("x")).select("x") -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3],v[2]] -==>[v[3],v[3],v[3],v[2],v[4]] -==>[v[3],v[3],v[3],v[2],v[4],v[5]] - -// 3.8.0 - bulked traversers are now split to be processed per-object, this affects local aggregation -gremlin> g.V().out().barrier().local(aggregate("x")).select("x") -==>[v[3]] -==>[v[3],v[3]] -==>[v[3],v[3],v[3]] -==>[v[3],v[3],v[3],v[2]] -==>[v[3],v[3],v[3],v[2],v[4]] -==>[v[3],v[3],v[3],v[2],v[4],v[5]] ----- - -See: link:https://github.com/apache/tinkerpop/blob/master/docs/src/dev/future/proposal-scoping-5.asciidoc[Lazy vs. Eager Evaluation] - -==== Removal of `store()` Step - -The `store()` step was a legacy name for `aggregate(local)` that has been deprecated since 3.4.3, and is now removed along -with `aggregate(local)`. To achieve lazy aggregation, use `aggregate()` within `local()`. - -[source,text] ----- -// 3.7.x - store() is still allowed -gremlin> g.V().store("x").by("age").cap("x") -==>[29,27,32,35] - -// 3.8.0 - store() removed, use local(aggregate()) to achieve lazy aggregation -gremlin> g.V().local(aggregate("x").by("age")).cap("x") -==>[29,27,32,35] ----- - -==== split() on Empty String +===== split() on Empty String The `split()` step will now split a string into a list of its characters if the given separator is an empty string. @@ -584,122 +540,14 @@ g.inject("Hello").split("") See: link:https://issues.apache.org/jira/browse/TINKERPOP-3083[TINKERPOP-3083] -==== asString() No Longer Allow Nulls +===== asString() No Longer Allow Nulls The `asString()` step will no longer allow `null` input. An `IllegalArgumentException` will be thrown for consistency with all other parsing steps (i.e. `asDate()`, `asBool()`, `asNumber()`). See: link:https://lists.apache.org/thread/q76pgrvhprosb4lty63bnsnbw2ljyl7m[DISCUSS] thread -==== Removal of has(key, traversal) - -The `has(key, traversal)` API has been removed in version 3.8.0 due to its confusing behavior that differed from other -has() variants. As well, most `has(key, traversal)` usage indicates a misunderstanding of the API. Unlike `has(key, value)` -which performs equality comparison, `has(key, traversal)` only checked if the traversal produced any result, creating -inconsistent semantics. - -[source,text] ----- -// 3.7.x - this condition is meaningless but yields result because count() is productive -gremlin> g.V().has("age", __.count()) -==>v[1] -==>v[2] -==>v[3] -==>v[4] -==>v[5] -==>v[6] -// simple example -gremlin> g.V().has("age", __.is(P.gt(30))) -==>v[4] -==>v[6] - -// 3.8.0 - traversals no longer yield results, for proper use cases consider using predicate or where() for filtering -gremlin> g.V().has("age", __.count()) -gremlin> g.V().has("age", __.is(P.gt(30))) -gremlin> g.V().has("age", P.gt(30)) -==>v[4] -==>v[6] ----- - -See: link:https://issues.apache.org/jira/browse/TINKERPOP-1463[TINKERPOP-1463] - -==== Serialization Changes - -*Properties on Element Serialization in Python & Javascript* - -Element properties handling has been inconsistent across GLVs. Previously,`gremlin-python` deserialized empty properties -as None or array depending on the serializer, while `gremlin-javascript` returned properties as objects or arrays, with -empty properties as empty lists or undefined depending on the serializer. - -This inconsistency is now resolved, aligning to how properties are handled in Gremlin core and in the Java GLV. -Both GLVs will deserialize element properties into lists of property objects, returning empty lists instead of null values -for missing properties. - -For python, the most notable difference is in graphSON when "tokens" is turned on for "materializeProperties". The -properties returned are no longer `None`, but empty lists. Users should update their code accordingly. - -For javascript, the change is slightly more extensive, as user should no longer expect javascript objects to be returned. -All properties are returned as lists of Property or VertexProperty objects. - -[source,javascript] ----- -// 3.7 and before: -g.with_("materializeProperties", "tokens").V(1).next() // skip properties with token -// graphson will return properties as a javascript object, which becomes undefined -Vertex { id: 1, label: 'person', properties: undefined } -// graphbinary will return properties as empty lists -Vertex { id: 1, label: 'person', properties: [] } - -g.V(1).next() // properties returned -// graphson will return properties as a javascript object -Vertex { - id: 1, - label: 'person', - properties: { name: [Array], age: [Array] } -} -// graphbinary will return properties as lists of VertexProperty objects -Vertex { - id: 1, - label: 'person', - properties: [ [VertexProperty], [VertexProperty] ] -} - -// 3.8.0 and newer - properties are always arrays, empty array [] for missing properties: -g.with_("materializeProperties", "tokens").V(1).next() // skip properties with token -// both graphson and graphbinary return -Vertex { id: 1, label: 'person', properties: [] } -g.V(1).next() -// both graphson and graphbinary return -Vertex { - id: 1, - label: 'person', - properties: [ [VertexProperty], [VertexProperty] ] -} - ----- - -This change only affects how GLVs deserialize property data in client applications. The underlying graph serialization -formats and server-side behavior remain unchanged. - -See: link:https://issues.apache.org/jira/browse/TINKERPOP-3186[TINKERPOP-3186] - -*Javascript Set Deserialization* - -Starting from this version, `gremlin-javascript` will deserialize `Set` data into a ECMAScript 2015 Set. Previously, -these were deserialized into arrays. - -*.NET Byte Serialization Change* - -The Gremlin .NET serializers has been updated to correctly handle byte values as signed integers to align with the IO -specification, whereas previously it incorrectly serialized and deserialized bytes as unsigned values. - -This is a breaking change for .NET applications that rely on byte values. Existing applications using byte values -should consider switching to `sbyte` for signed byte operations or `short` for a wider range of values to maintain -compatibility. - -See: link:https://issues.apache.org/jira/browse/TINKERPOP-3161[TINKERPOP-3161] - -==== Split bulked traversers for `local()` +===== Split bulked traversers for `local()` Prior to 3.8.0, local() exhibited "traverser-local" semantics, where the local traversal would apply independently to each individual bulkable `Traverser`. This often led to confusion, especially in the presence of reducing barrier steps, as @@ -727,13 +575,7 @@ gremlin> g.V().out().barrier().local(count()) See: link:https://issues.apache.org/jira/browse/TINKERPOP-3196[TINKERPOP-3196] -==== Removal of P.getOriginalValue() - -`P.getOriginalValue()` has been removed as it was not offering much value and was often confused with `P.getValue()`. -Usage of `P.getOriginalValue()` often leads to unexpected results if called on a predicate which has had its value reset -after construction. All usages of `P.getOriginalValue()` should be replaced with `P.getValue()`. - -==== Gremlin Grammar Changes +===== Gremlin Grammar Changes A number of changes have been introduced to the Gremlin grammar to help make it be more consistent and easier to use. @@ -882,12 +724,7 @@ link:https://issues.apache.org/jira/browse/TINKERPOP-3046[TINKERPOP-3046], link:https://issues.apache.org/jira/browse/TINKERPOP-3047[TINKERPOP-3047], link:https://issues.apache.org/jira/browse/TINKERPOP-3023[TINKERPOP-3023] -==== SeedStrategy Construction - -The `SeedStrategy` public constructor has been removed for Java and has been replaced by the builder pattern common -to all strategies. This change was made to ensure that the `SeedStrategy` could be constructed consistently. - -==== Improved Translators +===== Improved Translators The various Java `Translator` implementations allowing conversion of Gremlin traversals to string forms in various languages have been modified considerably. First, they have been moved from to the @@ -908,41 +745,54 @@ gremlin> GremlinTranslator.translate("g.V().out('knows')", Translator.GO) ==>g.V().Out("knows") ---- -See: link:https://issues.apache.org/jira/browse/TINKERPOP-3028[TINKERPOP-3028] - -==== Deprecated UnifiedChannelizer - -The `UnifiedChannelizer` was added in 3.5.0 in any attempt to streamline Gremlin Server code paths and resource usage. -It was offered as an experimental feature and as releases went on was not further developed, particularly because of the -major changes to Gremlin Server expected in 4.0.0 when websockets are removed. The removal of websockets with a pure -reliance on HTTP will help do what the `UnifiedChannelizer` tried to do with its changes. As a result, there is no need -to continue to refine this `Channelizer` implementation and it can be deprecated. - -See: link:https://issues.apache.org/jira/browse/TINKERPOP-3168[TINKERPOP-3168] +See: link:https://issues.apache.org/jira/browse/TINKERPOP-3028[TINKERPOP-3028] -==== OptionsStrategy in Python +===== Modified limit() skip() range() Semantics in repeat() -The `\\__init__()` syntax has been updated to be both more Pythonic and more aligned to the `gremlin-lang` syntax. -Previously, `OptionsStrategy()` took a single argument `options` which was a `dict` of all options to be set. -Now, all options should be set directly as keyword arguments. +The semantics of `limit()`, `skip()`, and `range()` steps called with default `Scope` or explicit `Scope.global` inside +`repeat()` have been modified to ensure consistent semantics across repeat iterations. Previously, these steps would +track global state across iterations, leading to unexpected filtering behavior between loops. -For example: +Consider the following examples which demonstrate the unexpected behavior. Note that the examples for version 3.7.4 +disable the `RepeatUnrollStrategy` so that strategy optimization does not replace the `repeat()` traversal with a +non-looping equivalent. 3.8.0 examples do not disable the `RepeatUnrollStrategy` as the strategy was modified to be more +restrictive in this version. -[source,python] +[source,groovy] ---- -# 3.7 and before: -g.with_strategies(OptionsStrategy(options={'key1': 'value1', 'key2': True})) -# 4.x and newer: -g.with_strategies(OptionsStrategy(key1='value1', key2=True)) +// 3.7.4 - grateful dead graph examples producing no results due to global counters +gremlin> g.withoutStrategies(RepeatUnrollStrategy).V().has('name','JAM').repeat(out('followedBy').limit(2)).times(2).values('name') +gremlin> +gremlin> g.withoutStrategies(RepeatUnrollStrategy).V().has('name','DRUMS').repeat(__.in('followedBy').range(1,3)).times(2).values('name') +gremlin> +// 3.7.4 - modern graph examples demonstrating too many results with skip in repeat due to global counters +gremlin> g.withoutStrategies(RepeatUnrollStrategy).V(1).repeat(out().skip(1)).times(2).values('name') +==>ripple +==>lop +gremlin> g.withoutStrategies(RepeatUnrollStrategy).V(1).out().skip(1).out().skip(1).values('name') +==>lop -myOptions = {'key1': 'value1', 'key2': True} -# 3.7 and before: -g.with_strategies(OptionsStrategy(options=myOptions)) -# 4.x and newer: -g.with_strategies(OptionsStrategy(**myOptions)) +// 3.8.0 - grateful dead graph examples producing results as limit counters tracked per iteration +gremlin> g.V().has('name','JAM').repeat(out('followedBy').limit(2)).times(2).values('name') +==>HURTS ME TOO +==>BLACK THROATED WIND +gremlin> g.V().has('name','DRUMS').repeat(__.in('followedBy').range(1,3)).times(2).values('name') +==>DEAL +==>WOMEN ARE SMARTER +// 3.8.0 - modern graph examples demonstrating consistent skip semantics +gremlin> g.V(1).repeat(out().skip(1)).times(2).values('name') +==>lop +gremlin> g.V(1).out().skip(1).out().skip(1).values('name') +==>lop ---- -==== Add barrier to most SideEffect steps +This change ensures that `limit()`, `skip()`, and `range()` steps called with default `Scope` or explicit `Scope.global` +inside `repeat()` are more consistent with manually unrolled traversals. Before upgrading, users should determine if any +traversals use `limit()`, skip()`, or `range()` with default `Scope` or explicit `Scope.global` inside `repeat()`. If it +is desired that the limit or range should apply across all loops then the `limit()`, `skip()`, or `range()` step should +be moved out of the `repeat()` step. + +===== Add barrier to most SideEffect steps Prior to 3.8.0, the `group(String)`, `groupCount(String)`, `tree(String)` and `subgraph(String)` steps were non-blocking, in that they allowed traversers to pass through without fully iterating the traversal and fully computing the side @@ -985,7 +835,7 @@ gremlin> g.V().local(groupCount("x")).select("x") ==>[v[1]:1,v[2]:1,v[3]:1,v[4]:1,v[5]:1,v[6]:1] ---- -==== choose() Semantics +===== choose() Semantics Several enhancements and clarifications have been made to the `choose()` step in TinkerPop 3.8.0 to improve its behavior and make it more consistent: @@ -1117,13 +967,13 @@ gremlin> g.V().hasLabel("person").union( See: link:https://issues.apache.org/jira/browse/TINKERPOP-3178[TINKERPOP-3178], link:https://tinkerpop.apache.org/docs/3.8.0/reference/#choose-step[Reference Documentation - choose()] -==== Float Defaults to Double +===== Float Defaults to Double The `GremlinLangScriptEngine` has been modified to treat float literals without explicit type suffixes (like 'm', 'f', or 'd') as Double by default. Users who need `BigDecimal` precision can still use the 'm' suffix (e.g., 1.0m). `GremlinGroovyScriptEngine` will still default to `BigDecimal` for `float` literals. -==== Consistent Output for range(), limit(), tail() +===== Consistent Output for range(), limit(), tail() The `range(local)`, `limit(local)`, and `tail(local)` steps now consistently return collections rather than automatically unfolding single-element results when operating on iterable collections (List, Set, etc.). Previously, when these steps @@ -1177,7 +1027,7 @@ This change affects all three local collection manipulation steps when operating See: link:https://issues.apache.org/jira/browse/TINKERPOP-2491[TINKERPOP-2491] -==== group() Value Traversal Semantics +===== group() Value Traversal Semantics The `group()` step takes two `by()` modulators. The first defines the key for the grouping, and the second acts upon the values grouped to each key. The latter is referred to as the "value traversal". In earlier versions of TinkerPop, @@ -1207,7 +1057,7 @@ gremlin> g.V().has("person","name",P.within("vadas","peter")).group().by().by(__ See: link:https://issues.apache.org/jira/browse/TINKERPOP-2971[TINKERPOP-2971] -==== By Modulation Semantics +===== By Modulation Semantics *valueMap() and propertyMap() Semantics* @@ -1223,16 +1073,7 @@ last one, which was not intuitive. See: link:https://issues.apache.org/jira/browse/TINKERPOP-3121[TINKERPOP-3121], link:https://issues.apache.org/jira/browse/TINKERPOP-2974[TINKERPOP-2974] -==== Remove Undocumented `with()` modulation - -There has long been a connection between the `with()` modulator, and mutating steps due to the design of -some of the interfaces in the gremlin traversal engine. This has led to several undocumented usages of the -`with()` modulator which have never been officially supported but have previously been functional. - -As of 3.8.0 `with()` modulation of the following steps will no longer work: `addV()`, `addE()`, `property()`, `drop()`, -`mergeV()`, and `mergeE()`. - -==== Stricter RepeatUnrollStrategy +===== Stricter RepeatUnrollStrategy The `RepeatUnrollStrategy` has been updated to use a more conservative approach for determining which repeat traversals are safe to unroll. Previously, the strategy would attempt to unroll most usages of `repeat()` used with `times()` @@ -1258,7 +1099,7 @@ g.V().repeat(both().simplePath()).times(4) g.V().repeat(both().sample(1)).times(2) ---- -===== Migration Strategies +====== Migration Strategies Before upgrading, analyze existing traversals which use `repeat()` with any steps other than `out()`, `in()`, `both()`, `inV()`, `outV()`, `otherV()`, `has(key, value)` and determine if the semantics of these traversals are as expected when @@ -1280,50 +1121,183 @@ g.V().repeat(both()).times(2).dedup() See: link:https://issues.apache.org/jira/browse/TINKERPOP-3192[TINKERPOP-3192] -==== Modified limit() skip() range() Semantics in repeat() +==== Other Breaking Changes -The semantics of `limit()`, `skip()`, and `range()` steps called with default `Scope` or explicit `Scope.global` inside -`repeat()` have been modified to ensure consistent semantics across repeat iterations. Previously, these steps would -track global state across iterations, leading to unexpected filtering behavior between loops. +===== Set minimum Java version to 11 -Consider the following examples which demonstrate the unexpected behavior. Note that the examples for version 3.7.4 -disable the `RepeatUnrollStrategy` so that strategy optimization does not replace the `repeat()` traversal with a -non-looping equivalent. 3.8.0 examples do not disable the `RepeatUnrollStrategy` as the strategy was modified to be more -restrictive in this version. +TinkerPop 3.8 requires a minimum of Java 11 for building and running. Support for Java 1.8 has been dropped. -[source,groovy] +==== Additions and New Features + +===== Gremlin MCP Server + +Gremlin MCP Server is an experimental application that implements the link:https://modelcontextprotocol.io/[Model Context Protocol] +(MCP) to expose Gremlin Server-backed graph operations to MCP-capable clients such as Claude Desktop, Cursor, or +Windsurf. Through this integration, graph structure can be discovered, and Gremlin traversals can be executed. Basic +health checks are included to validate connectivity. + +A running Gremlin Server that fronts the target TinkerPop graph is required. An MCP client can be configured to connect +to the Gremlin MCP Server endpoint. + +===== Air Routes Dataset + +The Air Routes sample dataset has long been used to help showcase and teach Gremlin. Popularized by the first edition +of link:https://kelvinlawrence.net/book/PracticalGremlin.html[Practical Gremlin], this dataset offers a real-world graph +structure that allows for practical demonstration of virtually every feature that Gremlin syntax has to offer. While it +was easy to simply get the dataset from the Practical Gremlin link:https://github.com/krlawrence/graph[repository], +including it with the TinkerPop distribution makes it much more convenient to use with Gremlin Server, Gremlin Console, +or directly in code that depends on the `tinkergraph-gremlin` package. + +[source,text] +---- +plugin activated: tinkerpop.tinkergraph +gremlin> graph = TinkerFactory.createAirRoutes() +==>tinkergraph[vertices:3619 edges:50148] +gremlin> g = traversal().with(graph) +==>graphtraversalsource[tinkergraph[vertices:3619 edges:50148], standard] +gremlin> g.V().has('airport','code','IAD').valueMap('code','desc','lon','lat') +==>[code:[IAD],lon:[-77.45580292],lat:[38.94449997],desc:[Washington Dulles International Airport]] ---- -// 3.7.4 - grateful dead graph examples producing no results due to global counters -gremlin> g.withoutStrategies(RepeatUnrollStrategy).V().has('name','JAM').repeat(out('followedBy').limit(2)).times(2).values('name') -gremlin> -gremlin> g.withoutStrategies(RepeatUnrollStrategy).V().has('name','DRUMS').repeat(__.in('followedBy').range(1,3)).times(2).values('name') -gremlin> -// 3.7.4 - modern graph examples demonstrating too many results with skip in repeat due to global counters -gremlin> g.withoutStrategies(RepeatUnrollStrategy).V(1).repeat(out().skip(1)).times(2).values('name') -==>ripple -==>lop -gremlin> g.withoutStrategies(RepeatUnrollStrategy).V(1).out().skip(1).out().skip(1).values('name') -==>lop -// 3.8.0 - grateful dead graph examples producing results as limit counters tracked per iteration -gremlin> g.V().has('name','JAM').repeat(out('followedBy').limit(2)).times(2).values('name') -==>HURTS ME TOO -==>BLACK THROATED WIND -gremlin> g.V().has('name','DRUMS').repeat(__.in('followedBy').range(1,3)).times(2).values('name') -==>DEAL -==>WOMEN ARE SMARTER -// 3.8.0 - modern graph examples demonstrating consistent skip semantics -gremlin> g.V(1).repeat(out().skip(1)).times(2).values('name') -==>lop -gremlin> g.V(1).out().skip(1).out().skip(1).values('name') -==>lop +TinkerPop distributes the 1.0 version of the dataset. + +===== Type Predicate + +The new `P.typeOf()` predicate allows filtering traversers based on their runtime type. It accepts either a `GType` +enum constant or a string representation of a simple class name. This predicate is particularly useful for type-safe +filtering in heterogeneous data scenarios. + +[source,text] ---- +// Filter vertices by property type +gremlin> g.V().values("age","name").is(P.typeOf(GType.INT)) +==>29 +==>27 +==>32 +==>35 -This change ensures that `limit()`, `skip()`, and `range()` steps called with default `Scope` or explicit `Scope.global` -inside `repeat()` are more consistent with manually unrolled traversals. Before upgrading, users should determine if any -traversals use `limit()`, skip()`, or `range()` with default `Scope` or explicit `Scope.global` inside `repeat()`. If it -is desired that the limit or range should apply across all loops then the `limit()`, `skip()`, or `range()` step should -be moved out of the `repeat()` step. +// Type inheritance support - NUMBER matches all numeric types +gremlin> g.union(V(), E()).values().is(P.typeOf(GType.NUMBER)) +==>29 +==>27 +==>32 +==>35 +==>0.5 +==>1.0 +==>0.4 +==>1.0 +==>0.4 +==>0.2 +---- + +The predicate supports type inheritance where `GType.NUMBER` will match any numeric type. Invalid type names will +throw an exception at execution time. + +See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#a-note-on-predicates[Predicates], link:https://issues.apache.org/jira/browse/TINKERPOP-2234[TINKERPOP-2234] + +===== Number Conversion Step + +The new `asNumber()` step provides type casting functionality to Gremlin. It serves as an umbrella step that parses +strings and casts numbers into desired types. For the convenience of remote traversals in GLVs, these available types +are denoted by a set of number tokens (`GType`). + +This new step will allow users to normalize their data by converting string numbers and mixed numeric types to a +consistent format, making it easier to perform downstream mathematical operations. As an example: + +[source,text] +---- +// sum() step can only take numbers +gremlin> g.inject(1.0, 2l, 3, "4", "0x5").sum() +class java.lang.String cannot be cast to class java.lang.Number + +// use asNumber() to avoid casting exceptions +gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum() +==>15.0 + +// given sum() step returned a double, one can use asNumber() to further cast the result into desired type +gremlin> g.inject(1.0, 2l, 3, "4", "0x5").asNumber().sum().asNumber(GType.INT) +==>15 +---- + +Semantically, the `asNumber()` step will convert the incoming traverser to a logical parsable type if no argument is +provided, or to the desired numerical type, based on the number token (`GType`) provided. + +Numerical input will pass through unless a type is specified by the number token. `ArithmeticException` will be thrown +for any overflow as a result of narrowing of types: + +[source,text] +---- +gremlin> g.inject(5.0).asNumber(GType.INT) +==> 5 // casts double to int +gremlin> g.inject(12).asNumber(GType.BYTE) +==> 12 +gremlin> g.inject(128).asNumber(GType.BYTE) +==> ArithmeticException +---- + +String input will be parsed. By default, the smalled unit of number to be parsed into is `int` if no type token is +provided. `NumberFormatException` will be thrown for any unparsable strings: + +[source,text] +---- +gremlin> g.inject("5").asNumber() +==> 5 +gremlin> g.inject("5.7").asNumber(GType.INT) +==> 5 +gremlin> g.inject("1,000").asNumber(GType.INT) +==> NumberFormatException +gremlin> g.inject("128").asNumber(GType.BYTE) +==> ArithmeticException +---- + +All other input types will result in `IllegalArgumentException`: +[source,text] +---- +gremlin> g.inject([1, 2, 3, 4]).asNumber() +==> IllegalArgumentException +---- + +See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asNumber-step[asNumber()-step], link:https://issues.apache.org/jira/browse/TINKERPOP-3166[TINKERPOP-3166] + +===== Boolean Conversion Step + +The `asBool()` step bridges another gap in Gremlin's casting functionalities. Users now have the ability to parse +strings and numbers into boolean values, both for normalization and to perform boolean logic with numerical values. + +[source,text] +---- +gremlin> g.inject(2, "true", 1, 0, false, "FALSE").asBool().fold() +==>[true,true,true,false,false,false] + +// using the modern graph, we can turn count() results into boolean values +gremlin> g.V().local(outE().count()).fold() +==>[3,0,0,2,0,1] +gremlin> g.V().local(outE().count()).asBool().fold() +==>[true,false,false,true,false,true] +// a slightly more complex one using sack for boolean operations for vertices with both 'person' label and has out edges +gremlin> g.V().sack(assign).by(__.hasLabel('person').count().asBool()).sack(and).by(__.outE().count().asBool()).sack().path() +==>[v[1],true] +==>[v[2],false] +==>[v[3],false] +==>[v[4],true] +==>[v[5],false] +==>[v[6],true] +---- + +See: link:https://tinkerpop.apache.org/docs/3.8.0/reference/#asBool-step[asBool()-step], +link:https://issues.apache.org/jira/browse/TINKERPOP-3175[TINKERPOP-3175] + +==== Deprecations + +===== Deprecated UnifiedChannelizer + +The `UnifiedChannelizer` was added in 3.5.0 in any attempt to streamline Gremlin Server code paths and resource usage. +It was offered as an experimental feature and as releases went on was not further developed, particularly because of the +major changes to Gremlin Server expected in 4.0.0 when websockets are removed. The removal of websockets with a pure +reliance on HTTP will help do what the `UnifiedChannelizer` tried to do with its changes. As a result, there is no need +to continue to refine this `Channelizer` implementation and it can be deprecated. + +See: link:https://issues.apache.org/jira/browse/TINKERPOP-3168[TINKERPOP-3168] === Upgrading for Providers
