Ivan, I see your point. I agree that with the automatic updates we step into the schema-last territory.
Actually, if we support automatic evolution, we can as well support creating a cache without schema and inferring it from the first insert. In other words, we can have both "schema-first" and "schema-last" modes. Alexey, what do you think? -Val On Mon, Sep 7, 2020 at 5:59 AM Alexey Goncharuk <alexey.goncha...@gmail.com> wrote: > Ivan, > > Thank you, I got your concern now. As it is mostly regarding the > terminology, I am absolutely fine with changing the name to whatever fits > the approach best. Dynamic or evolving schema sounds great. I will make > corresponding changes to the IEP once we settle on the name. > > пн, 7 сент. 2020 г. в 11:33, Ivan Pavlukhin <vololo...@gmail.com>: > > > Hi Val, > > > > Thank you for your answer! > > > > My understanding is a little bit different. Yes, schema evolution > > definitely should be possible. But I see a main difference in "how > > schema is updated". I treat a common SQL approach schema-first. Schema > > and data manipulation operations are clearly separated and it enables > > interesting capabilities, e.g. preventing untended schema changes by > > mistaken data operations, restricting user permissions to change > > schema. > > > > > Schema-first means that schema exists in advance and all the stored > data > > is compliant with it - that's exactly what is proposed. > > > > A schema-last approach mentioned in [1] also assumes that schema > > exists, but it is inferred from data. Is not it more similar to the > > proposing approach? > > > > And I would like to say, that my main concern so far is mostly about > > terminology. And I suppose if it confuses me then others might be > > confused as well. My feeling is closer to "dynamic or liquid or may be > > evolving schema". > > > > [1] > > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > > > 2020-09-07 0:47 GMT+03:00, Valentin Kulichenko < > > valentin.kuliche...@gmail.com>: > > > Hi Ivan, > > > > > > I don't see an issue with that. Schema-first means that schema exists > in > > > advance and all the stored data is compliant with it - that's exactly > > what > > > is proposed. There are no restrictions prohibiting changes to the > schema. > > > > > > -Val > > > > > > On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin <vololo...@gmail.com> > > wrote: > > > > > >> Alexey, > > >> > > >> I am a little bit confused with terminology. My understanding conforms > > >> to a survey [1] (see part X Semi Structured Data). Can we really treat > > >> a "dynamic schema" approach as a kind of "schema-first"? > > >> > > >> [1] > > >> > https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf > > >> > > >> 2020-09-02 1:53 GMT+03:00, Denis Magda <dma...@apache.org>: > > >> >> > > >> >> However, could you please elaborate on the relation between Ignite > > and > > >> >> ORM? > > >> >> Is there a use case for Hibernate running on top of Ignite (I > haven't > > >> >> seen > > >> >> one so far)? If so, what is missing exactly on the Ignite side to > > >> support > > >> >> this? In my understanding, all you need is SQL API which we already > > >> have. > > >> >> Am I missing something? > > >> > > > >> > > > >> > Good point, yes, if all the ORM integrations use Ignite SQL APIs > > >> > internally, then they can easily translate an Entity object into an > > >> > INSERT/UPDATE statement that lists all the object's fields. Luckily, > > >> > our > > >> > Spring Data integration is already based on the Ignite SQL APIs and > > >> > needs > > >> > to be improved once the schema-first approach is supported. That > would > > >> > solve a ton of usability issues. > > >> > > > >> > I would revise the Hibernate integration as well during the Ignite > 3.0 > > >> dev > > >> > phase. Can't say if it's used a lot but Spring Data is getting > > traction > > >> for > > >> > sure. > > >> > > > >> > @Michael Pollind, I'll loop you in as long as you've started working > > on > > >> the > > >> > Ignite support for Micornaut Data > > >> > <https://micronaut-projects.github.io/micronaut-data/latest/guide/> > > and > > >> > came across some challenges. Just watch this discussion. That's what > > is > > >> > coming in Ignite 3.0. > > >> > > > >> > > > >> > - > > >> > Denis > > >> > > > >> > > > >> > On Mon, Aug 31, 2020 at 5:11 PM Valentin Kulichenko < > > >> > valentin.kuliche...@gmail.com> wrote: > > >> > > > >> >> Hi Denis, > > >> >> > > >> >> Generally speaking, I believe that the schema-first approach > natively > > >> >> addresses the issue if duplicate fields in key and value objects, > > >> because > > >> >> schema will be created for a cache, not for an object, as it > happens > > >> now. > > >> >> Basically, the schema will define whether there is a primary key or > > >> >> not, > > >> >> and which fields are included in case there is one. Any API that we > > >> would > > >> >> have must be compliant with this, so it becomes fairly easy to work > > >> >> with > > >> >> data as with a set of records, rather than key-value pairs. > > >> >> > > >> >> However, could you please elaborate on the relation between Ignite > > and > > >> >> ORM? > > >> >> Is there a use case for Hibernate running on top of Ignite (I > haven't > > >> >> seen > > >> >> one so far)? If so, what is missing exactly on the Ignite side to > > >> support > > >> >> this? In my understanding, all you need is SQL API which we already > > >> have. > > >> >> Am I missing something? > > >> >> > > >> >> -Val > > >> >> > > >> >> On Mon, Aug 31, 2020 at 2:08 PM Denis Magda <dma...@apache.org> > > wrote: > > >> >> > > >> >> > Val, > > >> >> > > > >> >> > I would propose adding another point to the motivations list > which > > >> >> > is > > >> >> > related to the ORM frameworks such as Spring Data, Hibernate, > > >> Micronaut > > >> >> and > > >> >> > many others. > > >> >> > > > >> >> > Presently, the storage engine requires to distinguish key objects > > >> >> > from > > >> >> the > > >> >> > value ones that complicate the usage of Ignite with those ORM > > >> >> > frameworks > > >> >> > (especially if a key object comprises several fields). More on > this > > >> can > > >> >> be > > >> >> > found here: > > >> >> > > > >> >> > > > >> >> > > >> > > > http://apache-ignite-developers.2346864.n4.nabble.com/DISCUSSION-Key-and-Value-fields-with-same-name-and-SQL-DML-td47557.html > > >> >> > > > >> >> > It will be nice if the new schema-first approach allows us to > work > > >> with > > >> >> > a > > >> >> > single entity object when it comes to the ORMs. With no need to > > >> >> > split > > >> >> > the > > >> >> > entity into a key and value. Just want to be sure that the Ignite > > >> >> > 3.0 > > >> >> > has > > >> >> > all the essential public APIs that would support the > single-entity > > >> >> > based > > >> >> > approach. > > >> >> > > > >> >> > What do you think? > > >> >> > > > >> >> > - > > >> >> > Denis > > >> >> > > > >> >> > > > >> >> > On Fri, Aug 28, 2020 at 3:50 PM Valentin Kulichenko < > > >> >> > valentin.kuliche...@gmail.com> wrote: > > >> >> > > > >> >> > > Igniters, > > >> >> > > > > >> >> > > One of the big changes proposed for Ignite 3.0 is the so-called > > >> >> > > "schema-first approach". To add more clarity, I've started > > writing > > >> >> > > the > > >> >> > IEP > > >> >> > > for this change: > > >> >> > > > > >> >> > > > > >> >> > > > >> >> > > >> > > > https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach > > >> >> > > > > >> >> > > Please take a look and let me know if there are any immediate > > >> >> > > thoughts, > > >> >> > > suggestions, or objections. > > >> >> > > > > >> >> > > -Val > > >> >> > > > > >> >> > > > >> >> > > >> > > > >> > > >> > > >> -- > > >> > > >> Best regards, > > >> Ivan Pavlukhin > > >> > > > > > > > > > -- > > > > Best regards, > > Ivan Pavlukhin > > >
