[
https://issues.apache.org/jira/browse/SPARK-48009?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17841211#comment-17841211
]
Rushikesh Kavar commented on SPARK-48009:
-----------------------------------------
import org.apache.spark.sql.SaveMode;
import org.example.avro.Writer;
import java.util.ArrayList;
import java.util.List;
public class AppendAvro {
public static void main(String[] args) {
Writer.writeAvro(getMockData(),
"C:\\Users\\kavarus\\testing\\spark-testing\\data", SaveMode.Append);
}
public static List<Modal> getMockData() {
List<Modal> lst = new ArrayList<>();
lst.add(new Modal("3", "Test3", 27));
lst.add(new Modal("4", "Test4", 27));
return lst;
}
}
> Specifications for Apache Spark hadoop Avro append operation
> ------------------------------------------------------------
>
> Key: SPARK-48009
> URL: https://issues.apache.org/jira/browse/SPARK-48009
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.4.3
> Reporter: Rushikesh Kavar
> Priority: Minor
>
> Consider a path /a/b/c
> Assume, I write the avro to folder using apache spark.
> After it is written, Assume I try to append dataset to this to folder.
> I want to see the specification of what happens in case of append.
> After doing PoC, I found out that when dataet which is appended is having
> same schema as of existing data, data gets just appended. But I want to see
> clear docs of what happens exactly in case of append.
> I am attaching my testing java code.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
