Hi Javier, VIN is the Vehicle Identity Number, the Vehicle upload the inform from can-bus every 10 second,this table contains about 20 columns,so if we can just VIN as the partition key, every vehicle just has only one partition,the partition will become very large and never stop increasing,this is why we use the create_date in the partition key,this sounds good . But we have requirement that we need to query the history data for a vehicle,for example,we need to query the vehicle data from 2018-01-01 until now.If we use create_month in the partition key,we can only get whole month data but not exact day data. I found an article: https://lostechies.com/ryansvihla/2014/09/22/cassandra-query-patterns-not-using-the-in-query-for-multiple-partitions/ so your suggestion is to get the data by below code: [cid:image001.png@01D3D702.064C0C70] We need to test it. Is there other design pattern to meet this requirement with better performance? Best Regards,
倪项菲/ David Ni 中移德电网络科技有限公司 Virtue Intelligent Network Ltd, co. Add: 2003,20F No.35 Luojia creative city,Luoyu Road,Wuhan,HuBei Mob: +86 13797007811|Tel: + 86 27 5024 2516 发件人: Javier Pareja <pareja.jav...@gmail.com> 发送时间: 2018年4月18日 6:00 收件人: user@cassandra.apache.org 主题: Re: 答复: Time serial column family design Hi David, Could you describe why you chose to include the create date in the partition key? If the vin in enough "partitioning", meaning that the size (number of rows x size of row) of each partition is less than 100MB, then remove the date and just use the create_time, because the date is already included in that column anyways. For example if columns "a" and "b" (from your table) are of max 256 UTF8 characters, then you can have approx 100MB / (2*256*2Bytes) = 100,000 rows per partition. You can actually have many more but you don't want to go much higher for performance reasons. If this is not enough you could use create_month instead of create_date, for example, to reduce the partition size while not being too granular. On Tue, 17 Apr 2018, 22:17 Nate McCall, <n...@thelastpickle.com<mailto:n...@thelastpickle.com>> wrote: Your table design will work fine as you have appropriately bucketed by an integer-based 'create_date' field. Your goal for this refactor should be to remove the "IN" clause from your code. This will move the rollup of multiple partition keys being retrieved into the client instead of relying on the coordinator assembling the results. You have to do more work and add some complexity, but the trade off will be much higher performance as you are removing the single coordinator as the bottleneck. On Tue, Apr 17, 2018 at 10:05 PM, Xiangfei Ni <xiangfei...@cm-dt.com<mailto:xiangfei...@cm-dt.com>> wrote: Hi Nate, Thanks for your reply! Is there other way to design this table to meet this requirement? Best Regards, 倪项菲/ David Ni 中移德电网络科技有限公司 Virtue Intelligent Network Ltd, co. Add: 2003,20F No.35 Luojia creative city,Luoyu Road,Wuhan,HuBei Mob: +86 13797007811|Tel: + 86 27 5024 2516 发件人: Nate McCall <n...@thelastpickle.com<mailto:n...@thelastpickle.com>> 发送时间: 2018年4月17日 7:12 收件人: Cassandra Users <user@cassandra.apache.org<mailto:user@cassandra.apache.org>> 主题: Re: Time serial column family design Select * from test where vin =“ZD41578123DSAFWE12313” and create_date in (20180416, 20180415, 20180414, 20180413, 20180412………………………………….); But this cause the cql query is very long,and I don’t know whether there is limitation for the length of the cql. Please give me some advice,thanks in advance. Using the SELECT ... IN syntax means that: - the driver will not be able to route the queries to the nodes which have the partition - a single coordinator must scatter-gather the query and results Break this up into a series of single statements using the executeAsync method and gather the results via something like Futures in Guava or similar. -- ----------------- Nate McCall Wellington, NZ @zznate CTO Apache Cassandra Consulting http://www.thelastpickle.com
