Hi Josh, It turns out the salt bucket in the create table script is different than what we're using in the generation of row key :D
Anyway thanks for the help :) Marica On Wed, Sep 14, 2016 at 3:33 PM, Marica Tan <[email protected]> wrote: > Hi Josh, > > It works without salting. Tried the getSaltedKey but still the same > problem. > > The generated row key when doing an upsert via phoenix sqlline, is > different and I can query the one created via phoenix. > > via phoenix: \x09O1IC\x00CUSTOMER\x0076s7DeAqt82_1LFY7brsTA > > via hbase api: \x15O1IC\x00CUSTOMER\x0076s7DeAqt82_1LFY7brsTA > > > Btw, I modified the code a little: > > def generateRowKey(nBuckets: Integer, compositeKeys: String*): Array[Byte] = { > val keys = > compositeKeys.tail.foldLeft(ArrayBuffer[Array[Byte]](convertToByteArray(compositeKeys.head)))((a, > b) => { > a += QueryConstants.SEPARATOR_BYTE_ARRAY > a += convertToByteArray(b) > }) > > updateSaltingByte(keys, nBuckets) > } > > def convertToByteArray(key: String): Array[Byte] = key match { > case x if StringUtils.isNotBlank(x) => Bytes.toBytes(x) > case _ => ByteUtil.EMPTY_BYTE_ARRAY > } > > def updateSaltingByte(rowKey: ArrayBuffer[Array[Byte]], nBuckets: Integer): > Array[Byte] = { > if (nBuckets > 0) { > SaltingUtil.getSaltedKey(new > ImmutableBytesWritable(ByteUtil.concat(Array.fill[Byte](1)(0), rowKey.toSeq: > _*)), nBuckets); > > } else { > ByteUtil.concat(rowKey.head, rowKey.tail.toSeq: _*) > > } > } > > > From what I understand, phoenix will just combine those keys separated by > a zero byte. Am I doing it wrong? > > I would greatly appreciate any help you can give me to solve this problem. > > > Thanks! > -- > Marica > > On Wed, Sep 14, 2016 at 8:15 AM, Marica Tan <[email protected]> wrote: > >> Hi Josh, >> >> Haven't tried without salting yet. The problem we encountered is only on >> the firstKey, when we do select with a where condition on the secondKey or >> thirdKey, it returns the correct result. So I'm guessing it has something >> to do with the salted key. >> >> Will try the getSaltedKey first and hoping that it will fix the problem :) >> >> Thanks for the help! >> >> -- >> Marica >> >> On 2016-09-13 21:50 (+0800), Josh Mahonin <[email protected]> wrote: >> > Hi Marica,> >> > >> > Are you able to successfully write your rowkey without salting? If not, >> it> >> > could be that your 'generateRowKey' function is the culprit.> >> > >> > FWIW, we have some code that does something similar, though we use> >> > 'getSaltedKey':> >> > >> > // If salting, we need to prepend an empty byte to 'rowKey', then fill >> it> >> > if (saltBuckets > 0) {> >> > rowKey = ByteUtil.concat(new byte[1], rowKey);> >> > rowKey = SaltingUtil.getSaltedKey(new ImmutableBytesWritable(rowKey),> >> > saltBuckets);> >> > }> >> > >> > Good luck,> >> > >> > Josh> >> >> > >> > On Tue, Sep 13, 2016 at 1:59 AM, Marica Tan <[email protected]> >> wrote:> >> > >> > > Hi,> >> > >> >> > > We have a table created via phoenix with salt bucket, but we're using> >> > > HBase API to insert records since we need to manually set the HBase >> version> >> > > and I believe that it isn't possible via phoenix.> >> > >> >> > > Our table has a composite key (firstKey varchar, secondKey varchar,> >> > > thirdKey varchar) and when we do a select query with a where >> condition on> >> > > the firstKey, not all records are retrieved.> >> > >> >> > > We checked the value of the firstKey and it should return 10 records, >> but> >> > > we're only getting 7.> >> > >> >> > > If we do a where firstKey = 'someValue' we get 7> >> > > If we do a where firstKey like '%someValue' we get 10> >> > >> >> > > So we think the main culprit is the way we generate the row key. >> Here's> >> > > the code:> >> > >> >> > > def generateRowKey(nBuckets: Integer, compositeKeys: String*):> >> > > Array[Byte] = {> >> > >> >> > > val keys = compositeKeys.tail.foldLeft(Ar >> rayBuffer[Array[Byte]](convertToByteArray(compositeKeys.head)))((a, b) >> => {> >> > > a += QueryConstants.SEPARATOR_BYTE_ARRAY> >> > > a += convertToByteArray(b)> >> > > })> >> > >> >> > > val rowKey = ByteUtil.concat(QueryConstants.SEPARATOR_BYTE_ARRAY, >> keys.toSeq: _*)> >> > >> >> > > updateSaltingByte(rowKey, nBuckets)> >> > >> >> > > rowKey> >> > > }> >> > >> >> > > def convertToByteArray(key: String): Array[Byte] = key match {> >> > > case x if StringUtils.isNotBlank(x) => Bytes.toBytes(x)> >> > > case _ => ByteUtil.EMPTY_BYTE_ARRAY> >> > > }> >> > >> >> > > def updateSaltingByte(rowKey: Array[Byte], nBuckets: Integer): Unit = >> {> >> > > if (nBuckets > 0) {> >> > > rowKey(0) = SaltingUtil.getSaltingByte(rowKey,> >> > > SaltingUtil.NUM_SALTING_BYTES, rowKey.length - >> SaltingUtil.NUM_SALTING_BYTES, nBuckets)> >> > > }> >> > > }> >> > >> >> > >> >> > > Btw, we're using phoenix 4.4> >> > >> >> > >> >> > > Thanks,> >> > > --> >> > > Marica Tan> >> > >> >> > >> > >
