Re: [compress] High Level API for Archives

[Matt Sicker]

On 2 May 2018 at 08:41, Stefan Bodewig  wrote:

> It's a bit more complex than that. A hypothetical commons-sevenz would
> stil have an optional dependency on commons-lzma which then has a real
> depedendency on XZ for Java. The 7z code doesn't need XZ for Java if the
> archive you want to read only contains deflated entries. I don't think
> you'd want to split the sevenz module into one that requires lzma and
> one that doesn't.
>

This sounds like a neat use of optional dependencies. However, this also
raises a nice boundary: archives and codecs. The archives depend on codec
implementations, not the other way around.


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-05-01, Matt Sicker wrote:

> On 1 May 2018 at 12:23, Torsten Curdt  wrote:

>> That smell must be something else ;)
>> Just have a look at the dependencies

>>   https://github.com/apache/commons-compress/blob/master/pom.xml#L69

> Right, I see several dependencies marked "optional" which means as an end
> user, I still have to specify those particular dependencies (which also
> means hunting down the correct compatible version if applicable). In a
> modular sense, I'd be able to just use the commons-compress-foo dependency
> and get third party dependencies transitively.

It's a bit more complex than that. A hypothetical commons-sevenz would
stil have an optional dependency on commons-lzma which then has a real
depedendency on XZ for Java. The 7z code doesn't need XZ for Java if the
archive you want to read only contains deflated entries. I don't think
you'd want to split the sevenz module into one that requires lzma and
one that doesn't.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Torsten Curdt]

>
> > That said - optional dependencies are also not really my favorite thing.
> > I am myself a big fan of shading dependencies in libraries (if the
> license
> > allows),
> > which also forces one to re-evaluate the modularity vs inclusiveness of
> > (even transitive) dependencies.
> >
>
> I'm generally not a fan of shading, even when it makes sense to do so. Then
> again, it doesn't seem like many Java developers like the alternatives
> anyways (e.g., OSGi), so it's a necessary evil. In fact, one of the more
> legitimate uses of it that I can think of is to shade in parts of Apache
> Commons libraries so you don't need to depend on the entire library but can
> still receive updates.
>

Sorry, but I fail to follow your argument here - but that's OK.
I guess we disagree anyway so there is no need to follow up on that.


> Having all those separate jars with maybe 5 classes without dependencies
> > does not make that much sense to me.
> > Neither from a dependency nor from bytes-on-disk POV.
> >
>
> This doesn't seem to be a big deal in practice. If it were, I don't think
> Spring Boot would be successful at all (tons of Spring Boot dependencies
> are tiny jars with just some json metadata or pom.xml file).
>

I never used Spring Boot - but you make it sound horrible :)

Popular does not necessarily mean things are a good idea.
Just because Spring decided to maintain a ton of artifacts does not make it
a valid case for it.

Having a simpler dependency tree often outweighs the overhead of a few
extra classes for me.

Re: [compress] High Level API for Archives

[Matt Sicker]

On 1 May 2018 at 12:23, Torsten Curdt  wrote:

> That smell must be something else ;)
> Just have a look at the dependencies
>
>   https://github.com/apache/commons-compress/blob/master/pom.xml#L69


Right, I see several dependencies marked "optional" which means as an end
user, I still have to specify those particular dependencies (which also
means hunting down the correct compatible version if applicable). In a
modular sense, I'd be able to just use the commons-compress-foo dependency
and get third party dependencies transitively.


> commons vfs and compress are very different beasts.
>
> I think the idea is to strike a balance between over-modularization
> (which is actually quite common in the maven universe) and monoliths.
> Both extremes are not great.
>

Agreed. One potential splitting point could be by third party dependencies.
Anything that works without them makes sense in a core type module.


> That said - optional dependencies are also not really my favorite thing.
> I am myself a big fan of shading dependencies in libraries (if the license
> allows),
> which also forces one to re-evaluate the modularity vs inclusiveness of
> (even transitive) dependencies.
>

I'm generally not a fan of shading, even when it makes sense to do so. Then
again, it doesn't seem like many Java developers like the alternatives
anyways (e.g., OSGi), so it's a necessary evil. In fact, one of the more
legitimate uses of it that I can think of is to shade in parts of Apache
Commons libraries so you don't need to depend on the entire library but can
still receive updates.


> Having all those separate jars with maybe 5 classes without dependencies
> does not make that much sense to me.
> Neither from a dependency nor from bytes-on-disk POV.
>

This doesn't seem to be a big deal in practice. If it were, I don't think
Spring Boot would be successful at all (tons of Spring Boot dependencies
are tiny jars with just some json metadata or pom.xml file).

-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Torsten Curdt]

>
> > Just one question: Why?
> >
> > The implementation jars for compress would be tiny and would have close
> to
> > none dependencies.
> >
>
> As an end user, I'd prefer to only have to manage a single maven coordinate
> (e.g., commons-vfs-ssh) instead of multiple ones depending on my use case
> (i.e., optional dependencies). It makes it easier to figure out which
> solutions are actually supported at a glance without having to dig through
> documentation as well.
>
> Also, I feel like one of Commons' goals is to avoid jar hell, and providing
> a monolith with several optional dependencies smells like jar hell to me.
>

That smell must be something else ;)
Just have a look at the dependencies

  https://github.com/apache/commons-compress/blob/master/pom.xml#L69

commons vfs and compress are very different beasts.

I think the idea is to strike a balance between over-modularization
(which is actually quite common in the maven universe) and monoliths.
Both extremes are not great.

That said - optional dependencies are also not really my favorite thing.
I am myself a big fan of shading dependencies in libraries (if the license
allows),
which also forces one to re-evaluate the modularity vs inclusiveness of
(even transitive) dependencies.

Having all those separate jars with maybe 5 classes without dependencies
does not make that much sense to me.
Neither from a dependency nor from bytes-on-disk POV.

Re: [compress] High Level API for Archives

[Matt Sicker]

On 1 May 2018 at 05:12, Torsten Curdt  wrote:

> Just one question: Why?
>
> The implementation jars for compress would be tiny and would have close to
> none dependencies.
>

As an end user, I'd prefer to only have to manage a single maven coordinate
(e.g., commons-vfs-ssh) instead of multiple ones depending on my use case
(i.e., optional dependencies). It makes it easier to figure out which
solutions are actually supported at a glance without having to dig through
documentation as well.

Also, I feel like one of Commons' goals is to avoid jar hell, and providing
a monolith with several optional dependencies smells like jar hell to me.

-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Torsten Curdt]

>
> > As in one module/jar per implementation. One module for a zipfs, one for
> an
> > sshfs, etc., instead of just "commons-vfs" as a monolith with optional
> > dependencies.
>

Just one question: Why?

The implementation jars for compress would be tiny and would have close to
none dependencies.

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-30, Matt Sicker wrote:

> Not sure if it helps much, but based on my experience in writing a totally
> unrelated library from scratch in Java 8, I feel as though the use of
> default interface methods would help tremendously in providing a nice API
> while maintaining BC.

I may have been able to avoid two or three abstract classes with default
methods, that's true. Static methods on interfaces would have helped
avoid some of the factories but all in all I feel Java7 hasn't been an
obstacle.

> I had considered proposing an update to Java 8,

That would be a separate thread, I guess. I'd not oppose such a move but
would probably prefer to get Compress 1.17 out of the door with Java7 as
baseline first (Tika is likely waiting for COMPRESS-445) .

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-29, Matt Sicker wrote:

> On 29 April 2018 at 06:24, Stefan Bodewig  wrote:

>> I'm not sure what you mean by modular in this context.

> As in one module/jar per implementation. One module for a zipfs, one for an
> sshfs, etc., instead of just "commons-vfs" as a monolith with optional
> dependencies.

Oh, you've been talking about VFS here.

If we tried the same with Compress we'd end up with one or two API jars
(compressors could be separate from archivers if we wanted to go to
extremes), an SPI jar with the utils package, eight archiver and
thirteen compressor implementations. Together with some
interdependencies as zip and sevenz would depend on several
compressors. While this might be good from a modularization POV it also
doesn't sound attractive from a maintentance POV to me :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Matt Sicker]

Not sure if it helps much, but based on my experience in writing a totally
unrelated library from scratch in Java 8, I feel as though the use of
default interface methods would help tremendously in providing a nice API
while maintaining BC. I had considered proposing an update to Java 8, but
that wouldn't really help the users who improperly use compress in Java 7
already.

On 29 April 2018 at 23:34, Stefan Bodewig  wrote:

> On 2018-04-29, Stefan Bodewig wrote:
>
> > If this is something that looks acceptable I'll add expansion and remove
> > the other two classes.
>
> Just went ahead and added expansion as well, which doesn't mean it has
> to stay.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-29, Stefan Bodewig wrote:

> If this is something that looks acceptable I'll add expansion and remove
> the other two classes.

Just went ahead and added expansion as well, which doesn't mean it has
to stay.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Oliver Heger]

Hi Matt,

Am 29.04.2018 um 20:30 schrieb Matt Sicker:
> On 26 April 2018 at 14:38, Oliver Heger 
> wrote:
> 
>> Recently I had a closer look at streaming libraries like Akka Streams.
>> So I had the idea to model the problem in a similar way:
>>
> 
> I've used Akka Streams a bit in the past, and while it's pretty awesome, I
> feel as though taking this approach would at least require collaboration
> with an existing reactive streams library or a new Commons one. In that
> case, I wonder if it's worth competing with existing RS APIs like RxJava,
> Reactor, Vert.x, and Akka Streams. I'm not even sure if any RS people are
> active here at Commons as it is.

This is similar to my own line of thoughts. Developing an own streaming
lib is way too ambitious and also not needed as there are already some
really powerful libraries around. So there are the following options:
- Provide an API that resembles the core concepts of such libraries
without going too deep or providing similar flexibility. That would be
my first take and is probably also the route Stefan is following.
- Integrate with one of the existing libraries, e.g. by implementing
special transformers, sources, and/or sinks that handle compression or
extraction. Not sure whether this fits into commons as we currently do
not implement extensions for specific other libraries. It would probably
already be a longish discussion to choose one or more libraries to support.

> 
> 
>> An archive or deflate operation could be modeled by a flow from a source
>> via some filtering or modifying stages to a sink. The source and the
>> sink could both either refer to a directory or an archive file. In order
>> to create a new archive, the source would point to a directory and the
>> sink would represent the archive file to be created. To extract an
>> archive file, it would be the other way around. When both the source and
>> the sink point to an archive file, you have an update operation. So the
>> basic concepts can be combined in a natural way.
>>
> 
> This approach is interesting and could potentially be done without reactive
> streams. It would essentially be a similar looking API, but it just would
> be implemented differently.
Yes, my first option above.

> 
> 
>> There could be stages that filter for files or archive entries to select
>> the content of an archive file to be created or the files to be
>> extracted. Maybe it makes also sense to map on entries to manipulate
>> them somehow (rename?).
>>
> 
> Now you're really starting to describe a high level framework. While this
> sounds really neat, I feel like it might be beyond the scope of Commons and
> would warrant its own project at that point.
> 
> 
Perhaps. Maybe a stream-like API to move or copy files could fit into
[io]? Then [compress] could integrate with this. But I am aware that
this is also an ambitious project and goes beyond what [compress]
currently needs.

Oliver

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Oliver Heger]

Am 29.04.2018 um 19:08 schrieb Stefan Bodewig:
> Hi all
> 
> I've introduced a whole lot of infrastructure code with
> 
> https://github.com/apache/commons-compress/commit/f62c523154dfedcf49a87a865db545bb8c55e795
> 
> but the interface now looks nicer to me, in particual see
> 
> try (Sink sink = FileToArchiveSink.forFile(args[1], new 
> File(args[2]))) {
> Archive.directory(new File(args[0]))
> .to(sink);
> }
> 
> in ArchiveCli - all of this in a new examples package. No unit tests,
> but I've tried the new cli with all formats we support writing for. Only
> archive creation is there, yet, but the opposite direction will reuse a
> lot of the infrastructure.
> 
> If this is something that looks acceptable I'll add expansion and remove
> the other two classes. If not, we'll need to think about an acceptable
> example API a bit longer. :-)

Big +1! Looks very good to me.

Oliver

> 
> Cheers
> 
> Stefan
> 
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
> 

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Matt Sicker]

On 26 April 2018 at 14:38, Oliver Heger 
wrote:

> Recently I had a closer look at streaming libraries like Akka Streams.
> So I had the idea to model the problem in a similar way:
>

I've used Akka Streams a bit in the past, and while it's pretty awesome, I
feel as though taking this approach would at least require collaboration
with an existing reactive streams library or a new Commons one. In that
case, I wonder if it's worth competing with existing RS APIs like RxJava,
Reactor, Vert.x, and Akka Streams. I'm not even sure if any RS people are
active here at Commons as it is.


> An archive or deflate operation could be modeled by a flow from a source
> via some filtering or modifying stages to a sink. The source and the
> sink could both either refer to a directory or an archive file. In order
> to create a new archive, the source would point to a directory and the
> sink would represent the archive file to be created. To extract an
> archive file, it would be the other way around. When both the source and
> the sink point to an archive file, you have an update operation. So the
> basic concepts can be combined in a natural way.
>

This approach is interesting and could potentially be done without reactive
streams. It would essentially be a similar looking API, but it just would
be implemented differently.


> There could be stages that filter for files or archive entries to select
> the content of an archive file to be created or the files to be
> extracted. Maybe it makes also sense to map on entries to manipulate
> them somehow (rename?).
>

Now you're really starting to describe a high level framework. While this
sounds really neat, I feel like it might be beyond the scope of Commons and
would warrant its own project at that point.


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Matt Sicker]

On 29 April 2018 at 06:24, Stefan Bodewig  wrote:

> The current compress-2.0 branch - which is something that I haven't fully
> given up on but is dormant right now - at least uses the file views for
> permissions and stuff.
>

Neat! I've experimented a bit locally in the past to see how big an effort
it would be to migrate vfs to this API, and I got frustrated by having to
implement Path in a completely different way than the existing path
abstraction in vfs (which only has absolute URLs as far as I could tell,
but maybe I'm wrong).


> I must admit I was underwhelmed by the java.nio.file implementation when
> I tried it for real in Ant's setpermissions task (even though Windows
> file system can provide POSIX attributes for everything it won't provide
> a PosixFileAttributes view and you are restricted to set a DOS read-only
> flag). But losing focus here. :-)
>

Yes, maybe that's why I've barely seen anyone implement this API. Perhaps
Commons needs to be that starting point for everyone else? :)


> I'm not sure what you mean by modular in this context.
>

As in one module/jar per implementation. One module for a zipfs, one for an
sshfs, etc., instead of just "commons-vfs" as a monolith with optional
dependencies.

Splitting Compress into an API-only library and separate libs for the
> different formats? This might be doable in a 2.0 version because
> everything else would not be backwards compatible. But we'd end up with
> a double digit number of jars which I'm not sure will be easy to
> maintain. You likely would want to release them independently (why would
> I release the commons-compress-pack200 component if there only is a
> change in commons-compress-tar?) and end up with a strange mix of
> versions inside of a single reactor build. I'm a bit scared by the idea,
> I must admit :-)
>

Depending if the modules are separated by git repo (like how maven handles
its individual plugins they maintain) or all in a single repo, that would
certainly determine how often each FS gets released. Based on the release
process, I'd suggest the all-in-one repo unless each FS has a couple people
to vote on releases individually.


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Stefan Bodewig]

Hi all

I've introduced a whole lot of infrastructure code with

https://github.com/apache/commons-compress/commit/f62c523154dfedcf49a87a865db545bb8c55e795

but the interface now looks nicer to me, in particual see

try (Sink sink = FileToArchiveSink.forFile(args[1], new 
File(args[2]))) {
Archive.directory(new File(args[0]))
.to(sink);
}

in ArchiveCli - all of this in a new examples package. No unit tests,
but I've tried the new cli with all formats we support writing for. Only
archive creation is there, yet, but the opposite direction will reuse a
lot of the infrastructure.

If this is something that looks acceptable I'll add expansion and remove
the other two classes. If not, we'll need to think about an acceptable
example API a bit longer. :-)

Cheers

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-26, Oliver Heger wrote:

> Recently I had a closer look at streaming libraries like Akka Streams.
> So I had the idea to model the problem in a similar way:

> An archive or deflate operation could be modeled by a flow from a source
> via some filtering or modifying stages to a sink. The source and the
> sink could both either refer to a directory or an archive file. In order
> to create a new archive, the source would point to a directory and the
> sink would represent the archive file to be created. To extract an
> archive file, it would be the other way around. When both the source and
> the sink point to an archive file, you have an update operation. So the
> basic concepts can be combined in a natural way.

> There could be stages that filter for files or archive entries to select
> the content of an archive file to be created or the files to be
> extracted. Maybe it makes also sense to map on entries to manipulate
> them somehow (rename?).

> The API could be extensible by allowing clients to create their own
> implementations of sources, sinks, and flow stages.

> Just some high-level thoughts without having thought that through.

This resonates with some ideas I've had, but I'm afraid this is a bit
too much if the target goal is an examples package :-)

In a way the current "lots of overloaded methods" implementation already
does something similar internally - while keeping files and archive
entries strictly separate in two different classes. Trying to find a
common abstraction for Files read from/written to a directory and
ArchiveEntries really adds another layer of complexity.

I'll see how far I can get by turning the internal work of my current
implementation into APIs. From your high level description sources/sinks
for all the different ways archives can be represented and the filter
operations are already there. The current implementation lacks
projection/transformation, I'll think about that separately.

Finally the sink could be something that just discards the contents and
just lists the files/entries and we could re-implement the Lister class
that way.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-26, Matt Sicker wrote:

> If it helps in design, the most promising approach I found was to integrate
> with the java.nio.file API. However, that turns into a sort of hybrid
> library between VFS and Compress.

The current compress-2.0 branch - which is something that I haven't fully
given up on but is dormant right now - at least uses the file views for
permissions and stuff.

I must admit I was underwhelmed by the java.nio.file implementation when
I tried it for real in Ant's setpermissions task (even though Windows
file system can provide POSIX attributes for everything it won't provide
a PosixFileAttributes view and you are restricted to set a DOS read-only
flag). But losing focus here. :-)

> However, they are rather strongly related (if the two libraries were
> modular, then it wouldn't be such a problem to combine them).

I'm not sure what you mean by modular in this context.

Splitting Compress into an API-only library and separate libs for the
different formats? This might be doable in a 2.0 version because
everything else would not be backwards compatible. But we'd end up with
a double digit number of jars which I'm not sure will be easy to
maintain. You likely would want to release them independently (why would
I release the commons-compress-pack200 component if there only is a
change in commons-compress-tar?) and end up with a strange mix of
versions inside of a single reactor build. I'm a bit scared by the idea,
I must admit :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Oliver Heger]

Am 26.04.2018 um 17:37 schrieb Matt Sicker:
> If it helps in design, the most promising approach I found was to integrate
> with the java.nio.file API. However, that turns into a sort of hybrid
> library between VFS and Compress. However, they are rather strongly related
> (if the two libraries were modular, then it wouldn't be such a problem to
> combine them).
> 
> On 26 April 2018 at 10:04, Stefan Bodewig  wrote:
> 
>> On 2018-04-24, sebb wrote:
>>
>>> On 23 April 2018 at 20:48, Torsten Curdt  wrote:
 TBH I am not such super big fan of adding and maintaining a high level
 API at this stage.
 You will never find the right abstraction that everyone is happy with.
 If you would - well, then that should be the real API afterall.
>>
 Honestly - I would just add example code for now. Maybe that can turn
 into a helper class in the long run.
 But for now we would not introduce something we may no longer break.
>>
>>> I like the idea of introducing it as example code.
>>> That makes it obvious that it will change.
>>
>> As the only people who spoke up prefer to not provide the API as an
>> officially supported one, I'm fine with moving the stuff to an examples
>> package, add a warning and add unit tests so we now it keeps on working.
>>
>> Unless anybody yells and says "no we really need this high level API",
>> that is.
>>
>> I'd still like to spend some time and gather feedback on a nicer API
>> than many overloads - is fluent the way to go?

Recently I had a closer look at streaming libraries like Akka Streams.
So I had the idea to model the problem in a similar way:

An archive or deflate operation could be modeled by a flow from a source
via some filtering or modifying stages to a sink. The source and the
sink could both either refer to a directory or an archive file. In order
to create a new archive, the source would point to a directory and the
sink would represent the archive file to be created. To extract an
archive file, it would be the other way around. When both the source and
the sink point to an archive file, you have an update operation. So the
basic concepts can be combined in a natural way.

There could be stages that filter for files or archive entries to select
the content of an archive file to be created or the files to be
extracted. Maybe it makes also sense to map on entries to manipulate
them somehow (rename?).

The API could be extensible by allowing clients to create their own
implementations of sources, sinks, and flow stages.

Just some high-level thoughts without having thought that through.

Oliver

>>
>>> If not, maybe put the code in an experimental package.
>>
>> The changeset package is in this state and has been like this for years
>> now, I doubt we'd ever get anything out of the experimental phase :-)
>>
>> Stefan
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>>
> 
> 

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Matt Sicker]

If it helps in design, the most promising approach I found was to integrate
with the java.nio.file API. However, that turns into a sort of hybrid
library between VFS and Compress. However, they are rather strongly related
(if the two libraries were modular, then it wouldn't be such a problem to
combine them).

On 26 April 2018 at 10:04, Stefan Bodewig  wrote:

> On 2018-04-24, sebb wrote:
>
> > On 23 April 2018 at 20:48, Torsten Curdt  wrote:
> >> TBH I am not such super big fan of adding and maintaining a high level
> >> API at this stage.
> >> You will never find the right abstraction that everyone is happy with.
> >> If you would - well, then that should be the real API afterall.
>
> >> Honestly - I would just add example code for now. Maybe that can turn
> >> into a helper class in the long run.
> >> But for now we would not introduce something we may no longer break.
>
> > I like the idea of introducing it as example code.
> > That makes it obvious that it will change.
>
> As the only people who spoke up prefer to not provide the API as an
> officially supported one, I'm fine with moving the stuff to an examples
> package, add a warning and add unit tests so we now it keeps on working.
>
> Unless anybody yells and says "no we really need this high level API",
> that is.
>
> I'd still like to spend some time and gather feedback on a nicer API
> than many overloads - is fluent the way to go?
>
> > If not, maybe put the code in an experimental package.
>
> The changeset package is in this state and has been like this for years
> now, I doubt we'd ever get anything out of the experimental phase :-)
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-24, sebb wrote:

> On 23 April 2018 at 20:48, Torsten Curdt  wrote:
>> TBH I am not such super big fan of adding and maintaining a high level
>> API at this stage.
>> You will never find the right abstraction that everyone is happy with.
>> If you would - well, then that should be the real API afterall.

>> Honestly - I would just add example code for now. Maybe that can turn
>> into a helper class in the long run.
>> But for now we would not introduce something we may no longer break.

> I like the idea of introducing it as example code.
> That makes it obvious that it will change.

As the only people who spoke up prefer to not provide the API as an
officially supported one, I'm fine with moving the stuff to an examples
package, add a warning and add unit tests so we now it keeps on working.

Unless anybody yells and says "no we really need this high level API",
that is.

I'd still like to spend some time and gather feedback on a nicer API
than many overloads - is fluent the way to go?

> If not, maybe put the code in an experimental package.

The changeset package is in this state and has been like this for years
now, I doubt we'd ever get anything out of the experimental phase :-)

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[sebb]

On 23 April 2018 at 20:48, Torsten Curdt  wrote:
> TBH I am not such super big fan of adding and maintaining a high level
> API at this stage.
> You will never find the right abstraction that everyone is happy with.
> If you would - well, then that should be the real API afterall.
>
> Honestly - I would just add example code for now. Maybe that can turn
> into a helper class in the long run.
> But for now we would not introduce something we may no longer break.

I like the idea of introducing it as example code.
That makes it obvious that it will change.

If not, maybe put the code in an experimental package.

> Maybe we need to look at the various users of the API and take that as
> the basis for the next step.
>
> My 2 cents.
> Torsten
>
> On Mon, Apr 23, 2018 at 8:57 PM, Stefan Bodewig  wrote:
>> Hi all
>>
>> I've started to work on COMPRESS-118 and added Archiver and Expander
>> classes - without any tests for now. As I'm trying to cover a bunch of
>> possible use cases there are lots of inputs or outputs that can
>> represent archives. When expanding archives you may want to use
>> auto-detect the format or specify it explicitly. You may want to filter
>> the files to add/entries to expand. All this leads to an explosion of
>> overloads that I'm not comfortable with.
>>
>> One idea that I came up with is using a fluent interface like
>>
>> Expander.forFile(someFile).filtering(someFilter).expandTo(someDirectory);
>>
>> or similar. But before I delve deeper into it, I'd like to collect
>> feedback.
>>
>> And then I'd like to collect additional features that might be
>> needed. What I can easily imaging is
>>
>> * When expanding, don't overwrite existing files when expanding an
>>   archive (or only if the archive entries are newer.
>>
>> * Add an API for updating archives - of course this boils down to
>>   writing a completely new archive with entries taken from two sources.
>>
>> * we may want to provide a utility class that makes dealing with the
>>   subclasses of ArchiveEntry easier. Many of them provide a userid, but
>>   there is no common interface. I'm thinking about something like
>>   
>> https://github.com/apache/ant-antlibs-compress/blob/master/src/main/org/apache/ant/compress/util/EntryHelper.java
>>
>> Any feedback is very much appreciated.
>>
>> Stefan
>>
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
>> For additional commands, e-mail: dev-h...@commons.apache.org
>>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Stefan Bodewig]

On 2018-04-23, Torsten Curdt wrote:

> TBH I am not such super big fan of adding and maintaining a high level
> API at this stage.  You will never find the right abstraction that
> everyone is happy with.

TIt's not my intention to make everybody happy, there will still be use
cases that won't be covered by the high level API where you stil need to
go to the lower level. The parallel creation of zip archives is
something that is far out of scope for the high level IMHO.

> Maybe we need to look at the various users of the API and take that as
> the basis for the next step.

Yes, this is soemthing I intend to do.

Thanks

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

Re: [compress] High Level API for Archives

[Matt Sicker]

I went through a few iterations of my own over the weekend and wasn't
satisfied with the direction in any of them. This approach looks
interesting, though I need to take a closer look.

On 23 April 2018 at 14:48, Torsten Curdt  wrote:

> TBH I am not such super big fan of adding and maintaining a high level
> API at this stage.
> You will never find the right abstraction that everyone is happy with.
> If you would - well, then that should be the real API afterall.
>
> Honestly - I would just add example code for now. Maybe that can turn
> into a helper class in the long run.
> But for now we would not introduce something we may no longer break.
>
> Maybe we need to look at the various users of the API and take that as
> the basis for the next step.
>
> My 2 cents.
> Torsten
>
> On Mon, Apr 23, 2018 at 8:57 PM, Stefan Bodewig 
> wrote:
> > Hi all
> >
> > I've started to work on COMPRESS-118 and added Archiver and Expander
> > classes - without any tests for now. As I'm trying to cover a bunch of
> > possible use cases there are lots of inputs or outputs that can
> > represent archives. When expanding archives you may want to use
> > auto-detect the format or specify it explicitly. You may want to filter
> > the files to add/entries to expand. All this leads to an explosion of
> > overloads that I'm not comfortable with.
> >
> > One idea that I came up with is using a fluent interface like
> >
> > Expander.forFile(someFile).filtering(someFilter).
> expandTo(someDirectory);
> >
> > or similar. But before I delve deeper into it, I'd like to collect
> > feedback.
> >
> > And then I'd like to collect additional features that might be
> > needed. What I can easily imaging is
> >
> > * When expanding, don't overwrite existing files when expanding an
> >   archive (or only if the archive entries are newer.
> >
> > * Add an API for updating archives - of course this boils down to
> >   writing a completely new archive with entries taken from two sources.
> >
> > * we may want to provide a utility class that makes dealing with the
> >   subclasses of ArchiveEntry easier. Many of them provide a userid, but
> >   there is no common interface. I'm thinking about something like
> >   https://github.com/apache/ant-antlibs-compress/blob/master/
> src/main/org/apache/ant/compress/util/EntryHelper.java
> >
> > Any feedback is very much appreciated.
> >
> > Stefan
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> > For additional commands, e-mail: dev-h...@commons.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>
>


-- 
Matt Sicker 

Re: [compress] High Level API for Archives

[Torsten Curdt]

TBH I am not such super big fan of adding and maintaining a high level
API at this stage.
You will never find the right abstraction that everyone is happy with.
If you would - well, then that should be the real API afterall.

Honestly - I would just add example code for now. Maybe that can turn
into a helper class in the long run.
But for now we would not introduce something we may no longer break.

Maybe we need to look at the various users of the API and take that as
the basis for the next step.

My 2 cents.
Torsten

On Mon, Apr 23, 2018 at 8:57 PM, Stefan Bodewig  wrote:
> Hi all
>
> I've started to work on COMPRESS-118 and added Archiver and Expander
> classes - without any tests for now. As I'm trying to cover a bunch of
> possible use cases there are lots of inputs or outputs that can
> represent archives. When expanding archives you may want to use
> auto-detect the format or specify it explicitly. You may want to filter
> the files to add/entries to expand. All this leads to an explosion of
> overloads that I'm not comfortable with.
>
> One idea that I came up with is using a fluent interface like
>
> Expander.forFile(someFile).filtering(someFilter).expandTo(someDirectory);
>
> or similar. But before I delve deeper into it, I'd like to collect
> feedback.
>
> And then I'd like to collect additional features that might be
> needed. What I can easily imaging is
>
> * When expanding, don't overwrite existing files when expanding an
>   archive (or only if the archive entries are newer.
>
> * Add an API for updating archives - of course this boils down to
>   writing a completely new archive with entries taken from two sources.
>
> * we may want to provide a utility class that makes dealing with the
>   subclasses of ArchiveEntry easier. Many of them provide a userid, but
>   there is no common interface. I'm thinking about something like
>   
> https://github.com/apache/ant-antlibs-compress/blob/master/src/main/org/apache/ant/compress/util/EntryHelper.java
>
> Any feedback is very much appreciated.
>
> Stefan
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
> For additional commands, e-mail: dev-h...@commons.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org

[compress] High Level API for Archives

[Stefan Bodewig]

Hi all

I've started to work on COMPRESS-118 and added Archiver and Expander
classes - without any tests for now. As I'm trying to cover a bunch of
possible use cases there are lots of inputs or outputs that can
represent archives. When expanding archives you may want to use
auto-detect the format or specify it explicitly. You may want to filter
the files to add/entries to expand. All this leads to an explosion of
overloads that I'm not comfortable with.

One idea that I came up with is using a fluent interface like

Expander.forFile(someFile).filtering(someFilter).expandTo(someDirectory);

or similar. But before I delve deeper into it, I'd like to collect
feedback.

And then I'd like to collect additional features that might be
needed. What I can easily imaging is

* When expanding, don't overwrite existing files when expanding an
  archive (or only if the archive entries are newer.

* Add an API for updating archives - of course this boils down to
  writing a completely new archive with entries taken from two sources.

* we may want to provide a utility class that makes dealing with the
  subclasses of ArchiveEntry easier. Many of them provide a userid, but
  there is no common interface. I'm thinking about something like
  
https://github.com/apache/ant-antlibs-compress/blob/master/src/main/org/apache/ant/compress/util/EntryHelper.java

Any feedback is very much appreciated.

Stefan

-
To unsubscribe, e-mail: dev-unsubscr...@commons.apache.org
For additional commands, e-mail: dev-h...@commons.apache.org