chadselph commented on PR #1111:
URL: https://github.com/apache/parquet-mr/pull/1111#issuecomment-1995928017
@amousavigourabi @wgtmac sorry for the delay, here's an example
```java
import java.io.IOException;
import java.nio.file.Files;
import org.apache.avro.SchemaBuilder;
import org.apache.avro.generic.GenericData;
import org.apache.parquet.avro.AvroParquetWriter;
import org.apache.parquet.hadoop.metadata.CompressionCodecName;
import org.apache.parquet.io.LocalOutputFile;
public class UseMemory {
public static void main(String[] args) throws IOException {
var schema =
SchemaBuilder.builder()
.record("Record")
.fields()
.nullableBoolean("maybe", false)
.endRecord();
var memStart = Runtime.getRuntime().freeMemory();
var temp = Files.createTempFile("parquet-mem", ".parquet");
Files.delete(temp.toAbsolutePath());
var writer =
AvroParquetWriter.<GenericData.Record>builder(new
LocalOutputFile(temp.toAbsolutePath()))
.withCompressionCodec(CompressionCodecName.SNAPPY)
.withSchema(schema)
.build();
System.out.println(
"Used " + (memStart - Runtime.getRuntime().freeMemory()) / (1024 *
1024) + " MBytes.");
writer.close();
Files.delete(temp.toAbsolutePath());
}
}
```
I realize this is a crude way to display memory usage, but in case you're in
doubt, you can run it in intellij in the debugger and calculate the retained
size of the objects, `writer` is 134MB and all of it is from the output stream.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
