Github user harshach commented on a diff in the pull request:
https://github.com/apache/storm/pull/644#discussion_r35240425
--- Diff:
external/storm-hdfs/src/main/java/org/apache/storm/hdfs/trident/HdfsState.java
---
@@ -136,33 +193,40 @@ public void run() {
private transient FSDataOutputStream out;
protected RecordFormat format;
private long offset = 0;
+ private int bufferSize = 131072; // default 128 K
- public HdfsFileOptions withFsUrl(String fsUrl){
+ public HdfsFileOptions withFsUrl(String fsUrl) {
this.fsUrl = fsUrl;
return this;
}
- public HdfsFileOptions withConfigKey(String configKey){
+ public HdfsFileOptions withConfigKey(String configKey) {
this.configKey = configKey;
return this;
}
- public HdfsFileOptions withFileNameFormat(FileNameFormat
fileNameFormat){
+ public HdfsFileOptions withFileNameFormat(FileNameFormat
fileNameFormat) {
this.fileNameFormat = fileNameFormat;
return this;
}
- public HdfsFileOptions withRecordFormat(RecordFormat format){
+ public HdfsFileOptions withRecordFormat(RecordFormat format) {
this.format = format;
return this;
}
- public HdfsFileOptions withRotationPolicy(FileRotationPolicy
rotationPolicy){
+ public HdfsFileOptions withRotationPolicy(FileRotationPolicy
rotationPolicy) {
this.rotationPolicy = rotationPolicy;
return this;
}
- public HdfsFileOptions addRotationAction(RotationAction action){
+ public HdfsFileOptions withBufferSize(int size) {
+ this.bufferSize = Math.max(4096, size); // at least 4K
--- End diff --
4096 should be substituted with buffersize? or do we need to take max it
looks like we are saying the minimum should be default bufferSize. if this is
needed we need to document it.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
