This worked for me, thank you for the help! Andrew Greer
-----Original Message----- From: Daniel Hinojosa <[email protected]> Sent: Tuesday, August 10, 2021 2:33 PM To: [email protected] Subject: [EXTERNAL]Re: KSQLdb Stream, Getting Topic Key CAUTION:This email originated from outside of the organization. Do not click links or open attachments unless you recognize the sender and know the content is safe. I had to verify myself, I wrote an example that uses a key that is not in the payload and I did where I specify the KEY and now it is a field. Notice the state down below CREATE STREAM my_avro_orders (total BIGINT, shipping VARCHAR, state VARCHAR KEY, discount DOUBLE, gender VARCHAR) WITH (kafka_topic='my_avro_orders', value_format='AVRO'); This came out as the following with a describe: ksql> describe my_avro_orders; Name : MY_AVRO_ORDERS Field | Type ----------------------------------- TOTAL | BIGINT SHIPPING | VARCHAR(STRING) STATE | VARCHAR(STRING) (key) DISCOUNT | DOUBLE GENDER | VARCHAR(STRING) ----------------------------------- Then querying it, I get: ksql> select * from my_avro_orders emit changes; +---------------------+---------------------+---------------------+---------------------+---------------------+ |STATE |TOTAL |SHIPPING |DISCOUNT |GENDER | +---------------------+---------------------+---------------------+---------------------+---------------------+ |ME |48025 |TWO_DAY |0.019999999552965164 |FEMALE | |MD |32813 |NEXT_DAY |0.10999999940395355 |FEMALE | |SC |99302 |TWO_DAY |0.09000000357627869 |MALE | |SD |35621 |TWO_DAY |0.17000000178813934 |FEMALE | |SD |5698 |NEXT_DAY |0.12999999523162842 |MALE | |VT |8567 |NEXT_DAY |0.05000000074505806 |FEMALE | |WA |13737 |NEXT_DAY |0.07999999821186066 |MALE | |NH |60216 |TWO_DAY |0.05000000074505806 |MALE | |UT |47470 |TWO_DAY |0.05000000074505806 |MALE | |IN |69220 |TWO_DAY |0.1899999976158142 |FEMALE | On Tue, Aug 10, 2021 at 12:11 PM Daniel Hinojosa < [email protected]> wrote: > Ah, nevermind, just saw the title that this is KSQLDB. > > On Tue, Aug 10, 2021 at 12:09 PM Daniel Hinojosa < > [email protected]> wrote: > >> The keys are already part of the stream. When you run builder.stream >> or builder.table it returns a Stream<K,V> or a Table<K,V>. From there >> every operation has a lambda that accepts both key and value. You can >> use map for example to accept the key and do something with that. Let >> me know if you have any other questions or if I didn't understand correctly. >> >> On Tue, Aug 10, 2021 at 10:58 AM Greer, Andrew C < >> [email protected]> wrote: >> >>> Hello, I am trying to create a Stream that will accept the data from >>> my topic and be able to use the message keys in the stream as unique >>> identifiers for the sensors the data originated from. The data in >>> the messages does not have anything that would be able to identify >>> which sensor it came from, hence trying to get the keys used in the >>> topic. I have seen various posts online about a ROWKEY column, but >>> that doesn't appear to be an option on my version at least. I have >>> had no success finding a way to incorporate the keys into my Stream >>> and was hoping I would be able to find some help here. >>> >>> Thank you. >>> >>> Kafka version 6.0.1 >>> >>> Andrew >>> >>>
