zhuzilin commented on pull request #8041: URL: https://github.com/apache/tvm/pull/8041#issuecomment-841694188
@FrozenGene @tkonolige @altanh Thank you for your reviews. I've updated this PR based on them. > I think there are 2 options: 1. Ref Conv2DRel. 2. You could restrict the type here and raise exception. I prefer option 1. @FrozenGene Thank you for the clue! However, I haven't find how to restrict the dtype attributes in `Conv2DRel` or `Conv2DAttrs`... Should I add the type restriction to the `UniformAttrs`, or raise error in the `MakeUniform` and `UniformRel`? > How well does this approach work when we have a large range (high - low)? It seems like we would be loosing a lot of potential randomness. @tkonolige As this approach is only using the fraction bits to represent float, there will be loss of randomness for all floats, at least `(2^nexp-1) / 2^nexp` of the float (nexp stands for the number of exponential digits). However, it's a little tricky to use all 64 digits of the random bit to represent a uniform distributed float number... Do you have any idea on that? > Perhaps this is the operation you're looking for? https://github.com/altanh/tvm/blob/2d9ac7710ab055d4f20e8b5a0a3580836723efac/python/tvm/topi/generic/algorithm.py#L465 @altanh Thank you for your references! The `reinterpret` is exactly what I was looking for. I've tried to turn the algorithm in to the following form: ```python fraction = tvm.topi.right_shift(random_bits, tvm.tir.const(nbits - nfraction, dtype="uint64")) # 127 is the magic number for float32. it will make the exponenial part 1. exponent = tvm.topi.left_shift(tvm.topi.full(out_shape, "uint64", 127), tvm.tir.const(nfraction, dtype="uint64")) mantissa = tvm.topi.bitwise_or(fraction, exponent) standard_uniform_values = tvm.topi.reinterpret(mantissa, out_dtype) - 1 uniform_values = tvm.topi.add(tvm.topi.multiply(standard_uniform_values, high - low), low) ``` But there will be error relevant to llvm bitcast on the second line from bottom (the `-1`): ``` TVMError: LLVM module verification failed with the following errors: Invalid bitcast %39 = bitcast <2 x i64> %37 to <2 x float> Invalid bitcast %40 = bitcast <2 x i64> %38 to <2 x float> Invalid bitcast %52 = bitcast i64 %51 to float ``` Do you have any clue why this kind of error happens? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
