Shafaq-Siddiqi commented on a change in pull request #1397:
URL: https://github.com/apache/systemds/pull/1397#discussion_r721461423
##########
File path: src/main/java/org/apache/sysds/runtime/matrix/data/FrameBlock.java
##########
@@ -2345,6 +2384,77 @@ public FrameBlock replaceOperations(String pattern,
String replacement){
return ret;
}
+ public FrameBlock removeEmptyOperations(boolean rows, boolean
emptyReturn, MatrixBlock select) {
+ if( rows )
+ return removeEmptyRows(select, emptyReturn);
+ else //cols
+ return removeEmptyColumns(select, emptyReturn);
+ }
+
+ private FrameBlock removeEmptyRows(MatrixBlock select, boolean
emptyReturn) {
+ FrameBlock ret = new FrameBlock(_schema, _colnames);
+
+ for(int i = 0; i < _numRows; i++) {
+ boolean isEmpty = true;
+ Object[] row = new Object[getNumColumns()];
+
+ for(int j = 0; j < getNumColumns(); j++) {
+ Array colData = _coldata[j].clone();
+ row[j] = colData.get(i);
+ ValueType type = _schema[j];
+ isEmpty = isEmpty && (ArrayUtils.contains(new
double[]{0.0, Double.NaN}, UtilFunctions.objectToDoubleSafe(type,
colData.get(i))));
Review comment:
UtilFunctions.objectToDoubleSafe() converts a string representation
"null" to zero and then removes it, but it will not work with the original null
representation that we get while reading a file with missing values. Instead
of creating a new representation for "null" write a frame in your java test
suite and read it in dml by setting the "naString" flag or read an external
dataset.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
