Hi Martin,
On Jul 31, 2012, at 4:09 AM, Martin Desruisseaux wrote:
> Hello Chris
>
> Le 31/07/12 03:55, Mattmann, Chris A (388J) a écrit :
>> Before we create a utils package, I like to find classes that don't fit in
>> any of the other
>> categories. Do you have some ideas as to what would fit into here now?
> Example of things in the utilities modules are:
>
> * Specialized collections implementations. Apache has a special
> project for that, but some time it is useful to control the details
> of the implementation like exception handling inside the iterator
> (especially when the iterator is backed by I/O operations), thread
> management when there is some "cleaner" background thread at work, etc.
+1, so a spatial oriented collections package? Sure.
> * Localized resources for internationalisation. Years ago my first
> approach was to let each modules managing its own resources. But I
> realized that the amount of duplicated resources (e.g. "Argument foo
> can not be null") is so high that it was impracticable. We don't
> need to put all resources in a utility module, but at least the most
> frequently used ones.
+1.
> * A few mathematical functions, utility methods dealing with units of
> measurement, an AngleFormat for parsing/formatting angles in a
> DD°MM'SS format, utility methods for formatting tables or trees from
> TreeModel, a TreeTableNode interface in complement to the standard
> TreeNode interface.
> * A small framework which allow to redirect loggings message to Log4J
> or other frameworks. I know there is common-loggings and SLF4J for
> that, but those frameworks define their own API. I don't know why
> they are doing that, since it is possible to do the same on top of
> the standard JDK API, which is what we are doing.
> * For those who are fine with the standard JDK logging framework, a
> log formatter for producing logging message on one line (like Log4J)
> instead of two lines. Additionally this formatter uses colors on
> terminal that support it (warnings messages appear in red, info in
> green, etc.).
+1.
> * Logging level for operations that take a long time ("slow",
> "slower", "slowest") in order to help us detect potential bottleneck
> in production (without profiler). The logging level is determined by
> the time that the operation took to execute. If it is above some
> threshold, a message is logged using one of the above-cited levels.
This would seem useful for commons-logging or some commons project,
but +1 to have it here then try to push it upstream as time permits.
> * Some basic JAXB adapter for XML marshalling/unmashalling, not yet
> specific to an other module (rather the basic elements that we find
> in all ISO standards).
+1.
>
>
>
>> I would imagine metadata here would refer to arbitrary metadata support
>> and/or
>> specific Geo metadata models?
> This is mostly about supporting the ISO 19115 standard
> (http://www.iso.org/iso/catalogue_detail?csnumber=26020) and its ISO 19115-2
> extension. Those standards are mandatory in European governmental agencies as
> per the INSPIRE legislation. I think that NOAA participated to this standard,
> or at least the -2 extension. This module would provide "plain old Java
> objects" (POJO) for the various elements defined by the standard (about 100).
> For example there is a class GeographicBoundingBox with
> getEastBoundLongitude(), getNorthBoundLatitude(), etc. methods. This module
> contains also the JAXB adapters for marshalling/unmarshalling to ISO 19139
> conformant XML (ISO 19115 defines the metadata model, ISO 19139 defines how
> to express it in XML).
Got it. +1 to support ISO 19115.
>
> A general framework for supporting arbitrary metadata would be an other
> topic. Actually we already use Lucene for that, but this is at a higher level
> than providing POJO specifically for the ISO 19115 standard.
Well Lucene is a search framework and yes it can support arbitrary metadata,
but it brings with it
the rest of the things needed for a search framework. I'd like to push as much
metadata support
as possible into Tika since it's a lower level library.
>
> We also provide bridges from the NetCDF CF conventions to ISO 19115 metadata
> and conversely.
+1.
>
>
>>> * referencing
>> Is this like geo-location and geo-referencing for coordinate system support?
> Yes. This is a very big topic on its own. Include also parsing/formatting
> Well Known Text (WKT) format, creating Coordinate Reference System objects
> from the definition provided in the EPSG database, coordinates
> transformations, etc. This topic is one of the strongest point of Geotk.
+1.
>
>
>> If so, I filed an issue already for this:
>>
>> https://issues.apache.org/jira/browse/SIS-9
> Thanks for the tip. If this issue means "create a referencing" module, I
> would see it more as a whole component on the JIRA tracker instead than a
> single issue :).
Yep agreed, just using this email as a guide to reference relevant reported
issues that we can
track progress on (or add more issues to).
>
>
>>> * geometry
>> Seems similar to:
>>
>> https://issues.apache.org/jira/browse/SIS-51
>>
>> (distance being just one of the many possible geometric functions)
> Actually, on our side we put distance calculations in the referencing module
> instead. I see geometry more as an implemention of the ISO 19107 standard.
> Two Ph.D. students worked on this subject up to date and we are not yet
> there, so I think it still a very long-term task...
+1 yep.
>
>
>>> * coverage
>> WCS?
> I would consider web services (WCS, WFS, WMS...) only as protocols for
> transferring data between whatever model the library use, and external
> servers or clients. Here by "coverage" I mean implementation of the ISO 19123
> standard, which define a Coverage class with attributes like "domain",
> "range" and various sub-classes like ContinuousQuadrilateralGridCoverage,
> HexagonalGridCoverage, SegmentedCurveCoverage, ThiessenPolygonCoverage,
> TinCoverage, etc. Once we implemented at least the GridCoverage class, WCS is
> simply a way to export that class on the web.
Agreed. I would like to focus on using JAX-RS and Apache CXF (which provides an
implementation of it).
>
> The coverage module includes also I/O services for reading/formatting from
> various formats like NetCDF, some operations like raster reprojections, etc.
+1.
>
>
>>> * processing
>> What type of processing would this be? Like raster processing? Tiling?
> The idea is to define an abstract framework for launching arbitrary process.
> The goal is to support the "Web Processing Service". But actually I'm not
> sure if "processing" deserve its own module. We could define a basic
> framework in the "utility" module, and let each modules like "coverage"
> define their processes. I think we can defer this group to a later stage.
Yeah I would love to implement WPS using Apache OODT and its workflow manager.
We can maintain the
spatial specific parts of it here in SIS, but leverage OODT for the workflow
part.
>
>
>>> * index
>> I think I called this "storage" -- this would be the persistence layer for
>> the
>> spatial data structures, e.g.,
>>
>> https://issues.apache.org/jira/browse/SIS-8
>> https://issues.apache.org/jira/browse/SIS-33
> Yes. In this proposal "index" was a separated group for RTree, QuadTree and
> the like, but like "processing" I'm not sure it deserve it own module.
> Furthermore some index are closely related to the storage format. So I guess
> we can also omit this group for a while.
Got it. So we could move our existing QuadTree implementation there, and then
add an RTree, etc.
>
>
>>> * display
>> For us, this would be:
>> https://issues.apache.org/jira/browse/SIS-43
> Yes, however the "renderer engine" probably need a separated topic.
Yep agree.
>
>
>>> * client
>> I think for us this would be:
>>
>> https://issues.apache.org/jira/browse/SIS-11
>> https://issues.apache.org/jira/browse/SIS-46
> In this case the proposal is more for clients of web services. For example a
> WCS client would be a source of data connected to a web server. The WCS
> clients is responsible for constructing Coverage object from the data
> received from a server. Same apply for WFS, sensor data, processing...
>
> If we said that SIS support WCS, it can be both ways: as a server or as a
> client. This group of modules would take care of the "client" side of things.
> Maybe "web-client" would be a better name.
Agreed.
>
>
>> Think about the above in the context of the JIRA issues I sent you and I
>> would be happy
>> to discuss smallish, intermediate next steps to get there.
> If we forget "utilities" for now, the first step would be ISO-19115 metadata.
> Everything else depend on it. But if we agree to start with metadata, before
> discussing the details of this module, maybe it would be worth to make some
> plan about the overall architecture? For example if we apply the proposal to
> define some group of modules, it would imply moving around the existing SIS
> classes. Would it be done on the Subversion repository, or would it be easier
> to start an initially empty Git repository and move classes there (possibly
> in different directories) as modules are created?
I think we should start moving around the existing SVN repository. At at the
ASF, small incremental change
is always preferred compared to huge, hard to revert things. I'll also help
build merit and bring more existing
GeoTK folks on board and get them on the path to committership and PPMC
membership here at the ASF.
Are there others in GeoTK that can join up here on the Apache SIS lists and
start to contribute? That would
be great!
Regardless, if you want to take the lead in proposing some issues relevant to
ISO-19115 and then start
coding or creating patches, that would be awesome and I'll be happy to work
with you to shepherd the
work in.
Cheers,
Chris
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: [email protected]
WWW: http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
