Hi Igniters,
I and Alex Scherbakov had a discussion on how we could write rows in a more
compact way.
Many thanks to Alex for his ideas and critics.
So, in a long-read below I want to share some thoughts.
Motivation.
In Ignite 3.0 we will have versioned schema and most of the meta info will
be
That's the same case for me!. I've just downgraded my typescript version and
everything starts working. How did you notice that the typescript's version
was the problem?
--
Sent from: http://apache-ignite-developers.2346864.n4.nabble.com/
I see, thanks.
Let's discuss the return type - Future is not the one to use.
We should return CompletionStage, CompletableFuture, or introduce our own
interface.
We agreed on the last one (custom interface) for thin clients:
Pavel,
There are 2 PR's for the ticket[1] with two different APIs suggested.
Please, take a look at PR [2].
[1] https://issues.apache.org/jira/browse/IGNITE-14035
[2] https://github.com/apache/ignite-3/pull/69
On Wed, Mar 17, 2021 at 11:11 AM Pavel Tupitsyn
wrote:
> Andrey, I can't find any
Andrey, I can't find any async methods,
can you please check if the changes are pushed?
On Tue, Mar 16, 2021 at 10:06 PM Andrey Mashenkov <
andrey.mashen...@gmail.com> wrote:
> Pavel, good point.
> Thanks. I've added async methods.
>
> On Fri, Mar 12, 2021 at 2:29 PM Pavel Tupitsyn
> wrote:
>
>
Pavel, good point.
Thanks. I've added async methods.
On Fri, Mar 12, 2021 at 2:29 PM Pavel Tupitsyn wrote:
> Andrey,
>
> What about corresponding async APIs, do we add them now or later?
>
> On Thu, Mar 11, 2021 at 8:11 PM Andrey Mashenkov <
> andrey.mashen...@gmail.com>
> wrote:
>
> > Hi
Andrey,
What about corresponding async APIs, do we add them now or later?
On Thu, Mar 11, 2021 at 8:11 PM Andrey Mashenkov
wrote:
> Hi Igniters.
>
> I've created a PR for Table access API [1].
> This is an initial version. So, any suggestions\objections are welcomed.
> Please, do not hesitate
Hi Igniters.
I've created a PR for Table access API [1].
This is an initial version. So, any suggestions\objections are welcomed.
Please, do not hesitate to write your comments and\or examples to the PR.
Ignite-api module contains API classes, e.g. TableView classes as
projections for a table
I've updated a PR regarding your feedback [1].
[1] https://github.com/apache/ignite-3/pull/2
On Mon, Jan 11, 2021 at 10:58 AM Alexey Goncharuk <
alexey.goncha...@gmail.com> wrote:
> Folks,
>
> I updated the IEP to contain the missing pieces; actually, most of the
> questions here were covered
Folks,
I updated the IEP to contain the missing pieces; actually, most of the
questions here were covered by the text. Please let me know if there is
something still missing or unclear.
чт, 31 дек. 2020 г. в 12:48, Alexey Goncharuk :
> Mikhail and Igniters,
>
> Thanks for your comments. The
Mikhail and Igniters,
Thanks for your comments. The questions are reasonable, though I think all
concerns are addressed by the IEP as Val mentioned. I will update the
document according to your questions in the following week or so, so we can
have a constructive discussion further.
ср, 30 дек.
Hi Val, Andrey,
thank you for clarifying.
I still have a few comments.
1. one table == one schema. KV vs SQL:
Looks like all agreed that KV is just a special case of a regular table
with (blob,blob) schema.
I worry about the case when the user starts from KV case and later will try
to expand it
Hello!
I'm not sure why you will need a schema definition API in Java if it's not
available with ignitectl?
After all, you can run SQL with Java as well. If we plan that every cache
is also a SQL table, why do we need the API being discussed? If it is
needed, how to rationalize that not all
Hi Mike,
Thanks for providing your feedback. Please see my comments below.
I would also encourage you to go through the IEP-54 [1] - it has a lot of
detail on the topic.
[1]
https://cwiki.apache.org/confluence/display/IGNITE/IEP-54%3A+Schema-first+Approach
-Val
On Mon, Dec 28, 2020 at 11:22
Michael, thanks for feedback.
1. However, there is no decision approved by the community, but I heard a
lot about it would be nice to have one table\schema per cache.
Actually, we already have CacheGroup, a very useful feature that allows us
to share memory region/persistence files between caches
Ilya,
Thanks for feedback.
1. It is a Nested Builder pattern [1]. With a nested builder user will have
a single entry point to build a schema.
When you have a number of builders and one builder need a result of some
another builder, users can be confused and waste a time while looking for
proper
Hello!
Is there a way to define schema via ignitectl utility and its
hierarchical properties? How would it look like?
Regards,
--
Ilya Kasnacheev
пн, 21 дек. 2020 г. в 17:40, Andrey Mashenkov :
> Hi, Igniters.
>
> We all know that the current QueryEntity API is not convenient and needs to
>
Hello, Andrei!
I have read this thread and PR. The builder idea and API looks good. But I
have some questions.
1.
In SchemaTableBuilder:
When I use a builder in chain style, I expect in every step the same result
type. And as I see this idea is implemented anywhere except pk(), where you
return
Hi all,
I reviewed the mail thread and proposal page and I still don't fully
understand what is going to be changed, I would really appreciate it if you
will answer a few questions:
1. Are you going to leave only one schema per cache? if so, will be there
an option to have a table with arbitrary
Hi, Igniters.
We all know that the current QueryEntity API is not convenient and needs to
be reworked.
So, I'm glad to share PR [1] with schema configuration public API for
Ignite 3.0.
New schema configuration uses Builder pattern, which looks more comfortable
to use.
In the PR you will find a
A little bit my thoughts about unsigned types:
1. Seems we may support unsign types
2. It requires adding new types to the internal representation, protocol,
e.t.c.
3. internal representation should be the same as we keep sign types. So it
will not requires more memory
4. User should be aware of
Andrey,
Understood, thanks. Agree with the idea of creating a task for GraalVM
support. Just want to be sure there won't be any fundamental limitations in
the new serialization protocol that will make it hard or impossible to
generate native images. Probably, we should specify the requirements -
Actually, we can support comparisons in 3.0: once we the actual type
information, we can make proper runtime adjustments and conversions to
treat those values as unsigned - it will be just a bit more expensive.
вт, 24 нояб. 2020 г. в 18:32, Pavel Tupitsyn :
> > SQL range queries it will break
>
> SQL range queries it will break
> WHERE x > y may return wrong results
Yes, range queries, inequality comparisons and so on are broken
for unsigned data types, I think I mentioned this somewhere above.
Again, in my opinion, we can document that SQL is not supported on those
types,
end of
Folks, I think this is a reasonable request. I thought about this when I
was drafting the IEP, but hesitated to add these types right away.
> That is how it works in Ignite since the beginning with .NET and C++ :)
I have some doubts that it actually works as expected, it needs some
checking (will
Pavel,
I believe uLong values beyond 2^63 can't be treated correctly for now
(WHERE x > y may return wrong results)
I think we could make "true" support for unsigned types, but they will have
limitations on the Java side.
Thus, the one will not be able to map uint64 to Java long primitive, but
> That way is impossible.
That is how it works in Ignite since the beginning with .NET and C++ :)
You can use unsigned primitives as cache keys and values, as fields and
properties,
and in SQL queries (even in WHERE x=y clauses) - it works transparently for
the users.
Java side knows nothing and
Pavel,
- Treat uLong as long in Java (bitwise representation is the same)
That way is impossible.
Assume, you have a .NET class with a uByte field and map it to 'uint8'
column.
Then you set the field value to "250" and put the object into a table,
field value perfectly fits to a single byte
Andrey,
I don't think range narrowing is a good idea.
Do you see any problems with the simple approach I described?
On Tue, Nov 24, 2020 at 4:01 PM Andrey Mashenkov
wrote:
> Pavel,
>
> If you are ok with narrowing range for unsigned types then we could add a
> constraint for unsigned types on
Pavel,
If you are ok with narrowing range for unsigned types then we could add a
constraint for unsigned types on schema level (like nullability flag)
and treat them as signed types in storage.
We are going with a separate storage type-system and binary protocol
type-system, however most of type
Actually, I think it is not so hard to implement comparison of unsigned
numbers in
SQL even in Java, so it does not seem to be a big issue from my perspective.
Now to the usage of unsigned types from Java - I think, if a user uses
unsigned type
in a schema and is going to interact with it from
Andrey,
I think it is much simpler:
- Add protocol support for those types (basically, just add more type ids)
- Treat uLong as long in Java (bitwise representation is the same)
ANSI SQL does not have unsigned integers, so we can simply say that
unsigned value relative comparison is not
Thanks, Pavel and Igor.
I like your ideas to have i8 or int8 instead of Integer.
But the naming doesn't address the issue.
I agree internal types should be portable across different systems with and
without unsigned type support.
The only issue here is that unsigned types cover different ranges.
Agree, let's get rid of "long, short, byte" in the protocol definition.
We can use Rust style, which is concise and unambiguous:
i8, u8, i16, u16, etc
On Tue, Nov 24, 2020 at 1:58 PM Igor Sapego wrote:
> Pavel,
>
> I totally support that. Also, if we are aiming for
> stronger
Pavel,
I totally support that. Also, if we are aiming for
stronger platform-independance,
in our schemas we may want to support bit-notation (int32, uint64)? For
example
"long" can mean a different type on different platforms and it's easy to
confuse
them (happens often when using ODBC for
Igniters,
I think we should support unsigned data types:
uByte, uShort, uInt, uLong
Java does not have them, but many other languages do,
and with the growing number of thin clients this is important.
For example, in current Ignite.NET implementation we store unsigned values
as signed
Denis,
Good point. Both serializers use reflection API.
However, we will allow users to configure static schema along with 'strict'
schema mode, we still need to validate user classes on client nodes against
the latest schema in the grid and reflection API is the only way to do it.
One can find
Andrey, thanks for the update,
Does any of the serializers take into consideration the
native-image-generation feature of GraalVM?
https://www.graalvm.org/reference-manual/native-image/
With the current binary marshaller, we can't even generate a native image
for the code using our thin client
Hi Igniters,
I'd like to continue discussion of IEP-54 (Schema-first approach).
Hope everyone who is interested had a chance to get familiar with the
proposal [1].
Please, do not hesitate to ask questions and share your ideas.
I've prepared a prototype of serializer [2] for the data layout
Folks,
Please do not ignore history. We had a thread [1] with many bright
ideas. We can resume it.
[1]
http://apache-ignite-developers.2346864.n4.nabble.com/Applicability-of-term-cache-to-Apache-Ignite-td36541.html
2020-09-10 0:08 GMT+03:00, Denis Magda :
> Val, makes sense, thanks for
Val, makes sense, thanks for explaining.
Agree that we need to have a separate discussion thread for the "table" and
"cache" terms substitution. I'll appreciate it if you start the thread
sharing pointers to any relevant IEPs and reasoning behind the suggested
change.
-
Denis
On Tue, Sep 8,
Hi Denis,
I guess the wording in the IEP is a little bit confusing. All it means is
that you should not create nested POJOs, but rather inline fields into a
single POJO that is mapped to a particular schema. In other words, nested
POJOs are not supported.
Alex, is this correct? Please let me
Ivan,
I see your point. I agree that with the automatic updates we step into the
schema-last territory.
Actually, if we support automatic evolution, we can as well support
creating a cache without schema and inferring it from the first insert. In
other words, we can have both "schema-first" and
Ivan,
Thank you, I got your concern now. As it is mostly regarding the
terminology, I am absolutely fine with changing the name to whatever fits
the approach best. Dynamic or evolving schema sounds great. I will make
corresponding changes to the IEP once we settle on the name.
пн, 7 сент. 2020
Hi Val,
Thank you for your answer!
My understanding is a little bit different. Yes, schema evolution
definitely should be possible. But I see a main difference in "how
schema is updated". I treat a common SQL approach schema-first. Schema
and data manipulation operations are clearly separated
Hi Ivan,
I don't see an issue with that. Schema-first means that schema exists in
advance and all the stored data is compliant with it - that's exactly what
is proposed. There are no restrictions prohibiting changes to the schema.
-Val
On Sat, Sep 5, 2020 at 9:52 PM Ivan Pavlukhin wrote:
>
Alexey,
I am a little bit confused with terminology. My understanding conforms
to a survey [1] (see part X Semi Structured Data). Can we really treat
a "dynamic schema" approach as a kind of "schema-first"?
[1] https://people.cs.umass.edu/~yanlei/courses/CS691LL-f06/papers/SH05.pdf
2020-09-02
>
> However, could you please elaborate on the relation between Ignite and ORM?
> Is there a use case for Hibernate running on top of Ignite (I haven't seen
> one so far)? If so, what is missing exactly on the Ignite side to support
> this? In my understanding, all you need is SQL API which we
Alexey,
Thanks for adding more details to the IEP. I have a question regarding the
following:
*When an object is inserted into a table, we attempt to 'fit' object fields
to the schema columns. If a Java object has some extra fields which are not
present in the current schema, the schema is
Ivan,
Thank you for reminding me about the dynamic schema. I've updated the IEP
draft with more details on the approach, hopefully now it's more clear. I
think we will be able to take the best from both fixed-schema and
schemaless approaches.
вт, 1 сент. 2020 г. в 14:31, Ivan Pavlukhin :
> Hi
Hi Val,
Thank you for raising a discussion about this significant proposal!
The subject looks very significant and can greatly affect product
spirit and user experience.
While I generally think that schema-first is a good idea, I would love
to see a thorough approaches comparison section. As we
Hi Denis,
Generally speaking, I believe that the schema-first approach natively
addresses the issue if duplicate fields in key and value objects, because
schema will be created for a cache, not for an object, as it happens now.
Basically, the schema will define whether there is a primary key or
Val,
I would propose adding another point to the motivations list which is
related to the ORM frameworks such as Spring Data, Hibernate, Micronaut and
many others.
Presently, the storage engine requires to distinguish key objects from the
value ones that complicate the usage of Ignite with those
53 matches
Mail list logo
