mdedetrich commented on code in PR #2409:
URL: https://github.com/apache/pekko/pull/2409#discussion_r2469518733
##########
project/Dependencies.scala:
##########
@@ -351,7 +351,8 @@ object Dependencies {
// pekko stream
- lazy val stream = l ++= Seq[sbt.ModuleID](reactiveStreams,
TestDependencies.scalatest)
+ lazy val stream =
+ l ++= Seq[sbt.ModuleID](reactiveStreams, "com.github.luben" % "zstd-jni" %
"1.5.7-6", TestDependencies.scalatest)
Review Comment:
> How about having a separate pekko-streams-ztsd or
pekko-streams-compression so that pekko-streams core lib doesn't need this
dependency? ZIO have done something similar where a small number of compression
operators are in zio-streams but others have their own libs, eg
zio-streams-compress-zstd.
You could make the same argument as to why lz4Java isn't its own artifact,
or any of the other compression algorithms. Also, due to the way the current
API is designed, all of the compression algorithms are contained within the
pekko-streams `Compression` object (which we could monkey patch to add more
methods as a separate artifact, but thats starting to look ugly already). Ontop
of that, when adding zstd compression to pekko http request/response headers
(see https://github.com/apache/pekko-http/issues/860), we would likewise have
to create yet another artifact for pekko-http.
`zio-streams-compress` is also not the best to compare to, since if you look
at https://zio.dev/zio-streams-compress/ you will see they also have separate
artifacts for `gzip` and `lz4` something is contained within pekko-streams core
artifact for us. zio appears to have a different design ethos here, where they
err on the side of hyper modularity and put everything in its own artifact
where as pekko is on the other side.
I understand the sentiment, but honestly this is creating a huge amount of
ceremony for no real benefit and the only reason we are having this discussion
is because of how behind Java is in adding what are de facto standard
compression algorithms to the stdlib (of which zstd has been for at least half
a decade now). The only reason I can see to use separate artifacts would be to
support Pekko 1.x series, but zstd-jni only supports JDK 11 or higher and this
would then need to be advertised to end users.
`zio-streams-compress` is also not the best comparison, since if you look at
https://zio.dev/zio-streams-compress/ you will see they also have separate
artifacts for `gzip` and `lz4`, where as for us these compression algorithms
are contained within the core pekko-streams artifact. zio appears to have a
different design ethos here, where they err on the side of hyper modularity and
put everything in its own artifact where as pekko is on the other side.
Put differently, the entire point of `Compression` object for
scaladsl/javadsl is to contain the standard compression/decompression
algorithms, and zstd is one of those. While having to use zstd-jni is not
ideal, to the end user there is no difference as it already supports almost
every architecture that would be used in the wild.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
