RyanSkraba commented on code in PR #18109:
URL: https://github.com/apache/flink/pull/18109#discussion_r1199094836
##########
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/connector/datagen/table/types/RowDataGenerator.java:
##########
@@ -70,10 +74,13 @@ public boolean hasNext() {
@Override
public RowData next() {
- GenericRowData row = new GenericRowData(fieldNames.size());
- for (int i = 0; i < fieldGenerators.length; i++) {
- row.setField(i, fieldGenerators[i].next());
+ if (nullRate == 0 || ThreadLocalRandom.current().nextFloat() >
nullRate) {
Review Comment:
```suggestion
if (nullRate == 0f || ThreadLocalRandom.current().nextFloat() >
nullRate) {
```
##########
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/connector/datagen/table/RandomGeneratorVisitor.java:
##########
@@ -391,7 +474,11 @@ private static RandomGenerator<StringData>
getRandomStringGenerator(int length)
return new RandomGenerator<StringData>() {
@Override
public StringData next() {
- return StringData.fromString(random.nextHexString(length));
+ if (nullRate == NULL_RATE_DEFAULT
Review Comment:
There's no chance that `NULL_RATE_DEFAULT` would be anything other than
`0f`, right? I kind of think this might be better off as an _explicit_ `0f`,
but I don't feel strongly about it.
##########
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/connector/datagen/table/DataGenConnectorOptions.java:
##########
@@ -109,5 +110,12 @@ public class DataGenConnectorOptions {
.noDefaultValue()
.withDescription("End value of sequence generator.");
+ /** Placeholder {@link ConfigOption}. Not used for retrieving values. */
+ public static final ConfigOption<Float> FIELD_NULL_RATE =
+ ConfigOptions.key(String.format("%s.#.%s", FIELDS, NULL_RATE))
Review Comment:
Just a really minor nitpick, while you're here... this class has half
qualified `keys(....)` calls, and half using the static import. Can you switch
to one or the other while you're here?
##########
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/connector/datagen/table/DataGenConnectorOptions.java:
##########
@@ -109,5 +110,12 @@ public class DataGenConnectorOptions {
.noDefaultValue()
.withDescription("End value of sequence generator.");
+ /** Placeholder {@link ConfigOption}. Not used for retrieving values. */
+ public static final ConfigOption<Float> FIELD_NULL_RATE =
+ ConfigOptions.key(String.format("%s.#.%s", FIELDS, NULL_RATE))
+ .floatType()
+ .defaultValue(0f)
+ .withDescription("End value of sequence generator.");
Review Comment:
```suggestion
.withDescription("Rate at which null will be emitted
instead of generated values.");
```
##########
flink-table/flink-table-api-java-bridge/src/test/java/org/apache/flink/connector/datagen/table/types/DecimalDataRandomGeneratorTest.java:
##########
@@ -96,7 +96,7 @@ void testMinMax() {
DecimalDataRandomGenerator gen =
new DecimalDataRandomGenerator(
- precision, scale, min.doubleValue(),
max.doubleValue());
+ precision, scale, min.doubleValue(),
max.doubleValue(), 0);
Review Comment:
```suggestion
precision, scale, min.doubleValue(),
max.doubleValue(), 0f);
```
##########
flink-table/flink-table-api-java-bridge/src/main/java/org/apache/flink/connector/datagen/table/types/DataGeneratorMapper.java:
##########
@@ -51,6 +57,9 @@ public boolean hasNext() {
@Override
public B next() {
- return mapper.apply(generator.next());
+ if (nullRate == 0 || ThreadLocalRandom.current().nextFloat() >
nullRate) {
Review Comment:
```suggestion
if (nullRate == 0f || ThreadLocalRandom.current().nextFloat() >
nullRate) {
```
##########
flink-table/flink-table-api-java-bridge/src/test/java/org/apache/flink/connector/datagen/table/types/DecimalDataRandomGeneratorTest.java:
##########
@@ -35,7 +35,7 @@ void testGenerateDecimalValues() {
for (int scale = 0; scale <= precision; scale++) {
DecimalDataRandomGenerator gen =
new DecimalDataRandomGenerator(
- precision, scale, Double.MIN_VALUE,
Double.MAX_VALUE);
+ precision, scale, Double.MIN_VALUE,
Double.MAX_VALUE, 0);
Review Comment:
```suggestion
precision, scale, Double.MIN_VALUE,
Double.MAX_VALUE, 0f);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
