Re: [DISCUSS] New data type for vector search

2023-04-27 Thread steve landiss via dev
+1On Thursday, April 27, 2023 at 07:36:19 PM PDT, Caleb Rackliffe wrote: I don’t have a lot to add here, other than to say I’m broadly in agreement w/ David on syntax preference, element selectability, and making this a new type that roughly corresponds to a primitive

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Caleb Rackliffe
I don’t have a lot to add here, other than to say I’m broadly in agreement w/ David on syntax preference, element selectability, and making this a new type that roughly corresponds to a primitive (non-null-allowing) array.On Apr 27, 2023, at 9:18 PM, Anthony Grasso wrote:It would be strange for

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Anthony Grasso
It would be strange for this declaration to look different from other collection types. We may want to reconsider using the collection syntax. I also like the idea of the vector dimensions being declared with the VECTOR keyword. An alternative syntax option to explore is: VECTOR[size] On Fri, 28

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Josh McKenzie
>From a machine learning perspective, vectors are a well-known concept that are >effectively immutable fixed-length n-dimensional values that are then later >used either as part of a model or in conjunction with a model after the fact. While we could have this be non-frozen and not call it a

RE: Re: [DISCUSS] CEP-28: Reading and Writing Cassandra Data with Spark Bulk Analytics

2023-04-27 Thread Francisco Guerrero
Hi folks, We have updated the confluence page with the source code for CEP-28. There are two repositories with contributions. One is the patch [1] for Cassandra Sidecar with the bulk APIs that enable the Cassandra Spark Analytics library. The second is a new repository [2] with contributions

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread David Capwell
> but as you point out it has the problem of allowing nulls. If nulls are not allowed for the elements, then either we need a) a new type, or b) add some way to say elements may not be null…. As much as I do like b, I am leaning towards new type for this use case. So, to flesh out the type

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Benedict
That’s a bounded ring buffer, not a fixed length array.This definitely isn’t a tuple because the types are all the same, which is pretty crucial for matrix operations. Matrix libraries generally work on arrays of known dimensionality, or sparse representations.Whether we draw any semantic link

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Jeff Jirsa
On Thu, Apr 27, 2023 at 7:39 AM Jonathan Ellis wrote: > It's been a while, so I may be missing something, but do we already have > fixed-size lists? If not, I don't see why we'd try to make this fit into a > List-shaped problem. > We do not. The proposal got closed as wont-fix

Re: [DISCUSS] New data type for vector search

2023-04-27 Thread Jonathan Ellis
It's been a while, so I may be missing something, but do we already have fixed-size lists? If not, I don't see why we'd try to make this fit into a List-shaped problem. A tuple would be a better fit from that perspective, but as you point out it has the problem of allowing nulls. The key thing