Hi Rukshan, As per the offline discussion with, @Suho, we have decided to ignore 'field.length' validation for SHARD_ID as it is much cleaner and this column is introduced by Siddhi Aggregation implementation. Please find the pull request here[1].
[1] https://github.com/siddhi-io/siddhi-store-rdbms/pull/214 Best Regards, Niveathika On Mon, Nov 25, 2019 at 12:05 PM Niveathika Rajendran <niveath...@wso2.com> wrote: > Hi Rukshan and Fazlan, > > Regarding aggregation, since this config is based on the store type > (RDBMS), we cannot introduce this support in Siddhi Aggregation > implementation. However, we can introduce a system property config for > the siddhi-store-rdbms extension to ignore validation of 'field.length' > property. This will ensure that 'SHARD_ID' column can be added to the > 'field.length' property by default. > > @Fazlan, using the collation cannot be enabled by default, since we have > to maintain backward compatibility. However, this can be now enabled by a > single property '*use.collation'*. > > Best Regards, > Niveathika > > > On Sat, Nov 23, 2019 at 5:13 PM Fazlan Nazeem <fazl...@wso2.com> wrote: > >> One related and good to have feature is to set a single character >> encoding by default for the tables when siddhi creates them on the >> database. The primary key length issue is mostly due to having collations >> which take more bytes to represent a single character. If we can default to >> a character encoding of a single byte per character for each database type, >> then we could avoid these issues. Just a thought. >> >> On Fri, Nov 22, 2019 at 5:31 PM Rukshan Premathunga <ruks...@wso2.com> >> wrote: >> >>> Hi All, >>> >>> When we configure partitionById in SP, siddhi automatically adds >>> a SHARD_ID column to all the aggregated tables in RDBMS. But we are having >>> a "too long key issue" in the database. As a solution, we need to properly >>> set the column length for each attribute in the aggregated event stream. >>> >>> Limit of columns in the aggregated streams can be defined when we >>> implement the siddhi app. But SHARD_ID is used only when partitionById is >>> configured. So we cannot provide and initial column length for that. >>> >>> So it will be an ideal solution for us to have a configuration for this. >>> Otherwise, users need to alter the database or siddhi applications to >>> define these values. So can you please check the possibility to support >>> this? >>> >>> ex: >>> siddhi: >>> properties: >>> partitionById: true >>> shardId: dc1 >>> shardId_size: 20 # or derive length from 'shardId' >>> >>> Thanks and Regards >>> >>> -- >>> Rukshan C. Premathunga | Associate Technical Lead | WSO2 Inc. >>> (m) +94711822074 | (w) +94112145345 | Email: ruks...@wso2.com >>> GET INTEGRATION AGILE >>> Integration Agility for Digitally Driven Business >>> _______________________________________________ >>> Dev mailing list >>> d...@wso2.org >>> http://wso2.org/cgi-bin/mailman/listinfo/dev >>> >> >> >> -- >> Thanks & Regards, >> >> *Fazlan Nazeem | *Associate Technical Lead | WSO2 Inc >> Mobile : +94772338839 | fazl...@wso2.com >> >> >> _______________________________________________ >> Architecture mailing list >> Architecture@wso2.org >> https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture >> > > > -- > Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. > (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com > <https://wso2.com/signature> > > -- Niveathika Rajendran | Senior Software Engineer | WSO2 Inc. (m) +94779037536 | (w) +9411743 5800 | (e) niveath...@wso2.com <https://wso2.com/signature>
_______________________________________________ Architecture mailing list Architecture@wso2.org https://mail.wso2.org/cgi-bin/mailman/listinfo/architecture
