[
https://issues.apache.org/jira/browse/CSV-295?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17506527#comment-17506527
]
Zimo Li commented on CSV-295:
-----------------------------
Follow-up: 1.10.0-SNAPSHOT fixed the issue I'm having.
Scala sample code:
{code:java}
import org.apache.commons.csv.{CSVFormat, CSVPrinter}
import zio.clock.nanoTime
import zio.console.putStrLn
import zio.{Runtime, ZIO}
import java.io.{File, FileWriter}
val header = (0 to 1000).map(_.toString)
val file = new File(s"/Users/zli/Desktop/test.txt")
val fileWriter = new FileWriter(file)
val csvPrinter = new CSVPrinter(
fileWriter,
CSVFormat.RFC4180.builder().setHeader(header: _*).build()
)
Runtime.default.unsafeRun(for {
t0 <- nanoTime
_ <- ZIO
.foreachParN_(8)(0 to 1000) { i =>
val row = (0 to 1000).map(_ => i.toString)
ZIO.effect({
csvPrinter.printRecord(row: _*)
csvPrinter.flush()
})
}
t1 <- nanoTime
_ <- putStrLn("Elapsed time: " + (t1 - t0)/1000000 + "ms")
} yield ())
fileWriter.flush()
fileWriter.close()
file.delete(){code}
Result of running 10 times:
with synchronized: 675ms
without synchronized: 750ms
This is not supposed to be a serious benchmark, but the results are quite
surprising. JVM may be doing some optimization underneath.
> Support for parallelism in CSVPrinter
> -------------------------------------
>
> Key: CSV-295
> URL: https://issues.apache.org/jira/browse/CSV-295
> Project: Commons CSV
> Issue Type: Improvement
> Components: Printer
> Affects Versions: 1.9.0
> Environment:
> https://zio.dev/version-1.x/overview/overview_creating_effects#blocking-synchronous-side-effects
> Reporter: Zimo Li
> Priority: Major
> Fix For: 1.10.0
>
>
> I am trying to write the result of network IO to a CSV file using Scala and
> the ZIO library. The order of the rows does not matter, so I decided to use a
> concurrency of 8.
> Each thread calls {{{}CSVPrinter.printRecord{}}}, and this caused some rows
> to intersect with others. Eventually, I decided to use a
> [Semaphore|https://zio.dev/version-1.x/datatypes/concurrency/semaphore] to
> fix it.
> A locking mechanism for {{printRecord}} can be implemented just like the
> underlying {{FileWriter}} or {{PrintWriter}}.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
