[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248939351
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1523,5 +1529,31 @@ private void validateStringCharacterLimit() {
}
}
-
+ /**
+ * This method validates the DETAIL_QUERY_BATCH_SIZE. If some invalid input
is set, we use the
+ * default value for this property
+ */
+ private void validateDetailQueryBatchSize() {
+String batchSizeString =
+
carbonProperties.getProperty(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE);
+if (batchSizeString == null) {
+ carbonProperties.setProperty(DETAIL_QUERY_BATCH_SIZE,
+
Integer.toString(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT));
+ LOGGER.info(
+ "Using default value for carbon.detail.batch.size " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
+} else {
+ int batchSize;
+ try {
+batchSize = Integer.parseInt(batchSizeString);
+if (batchSize < 100 || batchSize > 1000) {
+ throw new NumberFormatException();
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248939286
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1523,5 +1529,31 @@ private void validateStringCharacterLimit() {
}
}
-
+ /**
+ * This method validates the DETAIL_QUERY_BATCH_SIZE. If some invalid input
is set, we use the
+ * default value for this property
+ */
+ private void validateDetailQueryBatchSize() {
+String batchSizeString =
+
carbonProperties.getProperty(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE);
+if (batchSizeString == null) {
+ carbonProperties.setProperty(DETAIL_QUERY_BATCH_SIZE,
+
Integer.toString(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT));
+ LOGGER.info(
+ "Using default value for carbon.detail.batch.size " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
+} else {
+ int batchSize;
+ try {
+batchSize = Integer.parseInt(batchSizeString);
+if (batchSize < 100 || batchSize > 1000) {
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248929210
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1523,5 +1529,31 @@ private void validateStringCharacterLimit() {
}
}
-
+ /**
+ * This method validates the DETAIL_QUERY_BATCH_SIZE. If some invalid input
is set, we use the
+ * default value for this property
+ */
+ private void validateDetailQueryBatchSize() {
+String batchSizeString =
+
carbonProperties.getProperty(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE);
+if (batchSizeString == null) {
+ carbonProperties.setProperty(DETAIL_QUERY_BATCH_SIZE,
+
Integer.toString(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT));
+ LOGGER.info(
+ "Using default value for carbon.detail.batch.size " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
+} else {
+ int batchSize;
+ try {
+batchSize = Integer.parseInt(batchSizeString);
+if (batchSize < 100 || batchSize > 1000) {
+ throw new NumberFormatException();
Review comment:
I was throwing exception because even if batchSize is some invalid character
then it will get caught and I am using the default value and adding a log in
the Catch part.So both the scenarios are covered here min max one and invalid
character also.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248580269
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1523,5 +1529,30 @@ private void validateStringCharacterLimit() {
}
}
-
+ /**
+ * This method validates the DETAIL_QUERY_BATCH_SIZE. If some invalid input
is set, we use the
+ * default value for this property
+ */
+ private void validateDetailQueryBatchSize() {
+String batchSizeString =
+
carbonProperties.getProperty(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE);
+if (batchSizeString == null) {
+ carbonProperties.setProperty(DETAIL_QUERY_BATCH_SIZE,
+
Integer.toString(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT));
+ LOGGER.info("Using default value for Query Batch Size " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
+} else {
+ int batchSize;
+ try {
+batchSize = Integer.parseInt(batchSizeString);
+if (batchSize <= 0) {
+ throw new NumberFormatException();
+}
+ } catch (NumberFormatException ne) {
+LOGGER.info(
+"Invalid Query Batch size. Using default value " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
+int defaultBatchSize =
CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT;
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248580215
##
File path:
core/src/main/java/org/apache/carbondata/core/util/CarbonProperties.java
##
@@ -1523,5 +1529,30 @@ private void validateStringCharacterLimit() {
}
}
-
+ /**
+ * This method validates the DETAIL_QUERY_BATCH_SIZE. If some invalid input
is set, we use the
+ * default value for this property
+ */
+ private void validateDetailQueryBatchSize() {
+String batchSizeString =
+
carbonProperties.getProperty(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE);
+if (batchSizeString == null) {
+ carbonProperties.setProperty(DETAIL_QUERY_BATCH_SIZE,
+
Integer.toString(CarbonCommonConstants.DETAIL_QUERY_BATCH_SIZE_DEFAULT));
+ LOGGER.info("Using default value for Query Batch Size " +
DETAIL_QUERY_BATCH_SIZE_DEFAULT);
Review comment:
Done.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248540901
##
File path:
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/AbstractDetailQueryResultIterator.java
##
@@ -94,6 +94,9 @@
if (null != batchSizeString) {
try {
batchSize = Integer.parseInt(batchSizeString);
+if (0 >= batchSize) {
Review comment:
I have added the validation at CarbonProperties such that now both the flow
of SDK and CarbonSession are covered in this case.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r248205713
##
File path:
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/AbstractDetailQueryResultIterator.java
##
@@ -94,6 +94,9 @@
if (null != batchSizeString) {
try {
batchSize = Integer.parseInt(batchSizeString);
+if (0 >= batchSize) {
Review comment:
Ok. Done!. Added a log also.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r247774451
##
File path:
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/AbstractDetailQueryResultIterator.java
##
@@ -94,6 +94,9 @@
if (null != batchSizeString) {
try {
batchSize = Integer.parseInt(batchSizeString);
+if (0 >= batchSize) {
Review comment:
Done!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r247774431
##
File path:
store/sdk/src/test/java/org/apache/carbondata/sdk/file/ConcurrentSdkReaderTest.java
##
@@ -132,6 +132,45 @@ private void writeDataMultipleFiles(int numFiles, long
numRowsPerFile) {
}
}
+ @Test public void testReadWithZeroBatchSize() throws InterruptedException {
Review comment:
Done!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix sdk reader issue if batch size is given as zero and vectorRead False.
shardul-cr7 commented on a change in pull request #3070: [CARBONDATA-3246]Fix
sdk reader issue if batch size is given as zero and vectorRead False.
URL: https://github.com/apache/carbondata/pull/3070#discussion_r247335015
##
File path:
core/src/main/java/org/apache/carbondata/core/scan/result/iterator/AbstractDetailQueryResultIterator.java
##
@@ -94,6 +94,9 @@
if (null != batchSizeString) {
try {
batchSize = Integer.parseInt(batchSizeString);
+if (0 == batchSize) {
Review comment:
Done!
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
