Yup agree with that just could not make it write the appropriate zip entries that way.. Open for being enlighted here..
2010/2/8, Bob Jolliffe <[email protected]>: > 2010/2/8 Lars Helge Øverland <[email protected]>: >> >> >> 2010/2/8 Bob Jolliffe <[email protected]> >>> >>> 2010/2/8 Lars Helge Øverland <[email protected]>: >>> > >>> > >>> > On Mon, Feb 8, 2010 at 11:59 AM, <[email protected]> wrote: >>> >> >>> >> ------------------------------------------------------------ >>> >> revno: 1404 >>> >> committer: Lars Helge Oeverland <[email protected]> >>> >> branch nick: trunk >>> >> timestamp: Mon 2010-02-08 11:56:08 +0100 >>> >> message: >>> >> Implemented DHIS 1.4 export of data values >>> >> modified: >>> > >>> > >>> > There is actually still a snag here. DHIS 1.4 only accepts the >>> > 7zip/lzma >>> > format currently. I have talked Greg into making 1.4 accept zip/deflate >>> > too. >>> >>> Isn't it better to have gzip/deflate? We are not talking about >>> compressing an archive of files but rather compressing a single >>> stream. >>> >>> Also I think the original "void write( BufferedWriter writer, >>> ExportParams params )" might be better than "void write( >>> ZipOutputStream out, ExportParams params )". It seems unnecessary >>> that the convertor should have to know anything about zip, gzip, 7zip >>> or what have you. It should know how to write csv to a stream. >>> >>> I suspect that how the stream is later compressed is better deferred to >>> later. >> >> Problem is that DHIS 1.4 uses a CSV file for data and an XML file for >> meta-data and keeps both inside the archive... I was was not able to have >> multiple zip entries while using the Writer.. > > Ah. Ok. I do remember. Then you would need an archive (zip or the > seven thing) rather a gzipped stream. > > But does your CSV writer need to know that? Surely it just has an > interest in churning out csv datavalues. Some other component should > put the csv stream together with the xml stream into the zip. > >> >>> >>> Cheers >>> Bob >>> >>> > >>> >> >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java >>> >> >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml >>> >> >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java >>> >> >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java >>> >> >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties >>> >> >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm >>> >> >>> >> >>> >> -- >>> >> lp:dhis2 >>> >> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk >>> >> >>> >> Your team DHIS 2 developers is subscribed to branch lp:dhis2. >>> >> To unsubscribe from this branch go to >>> >> >>> >> https://code.launchpad.net/~dhis2-devs-core/dhis2/trunk/+edit-subscription. >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java >>> >> 2009-03-03 16:46:36 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/CSVConverter.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -28,7 +28,7 @@ >>> >> */ >>> >> >>> >> import java.io.BufferedReader; >>> >> -import java.io.BufferedWriter; >>> >> +import java.util.zip.ZipOutputStream; >>> >> >>> >> /** >>> >> * @author Lars Helge Overland >>> >> @@ -36,7 +36,7 @@ >>> >> */ >>> >> public interface CSVConverter >>> >> { >>> >> - void write( BufferedWriter writer, ExportParams params ); >>> >> + void write( ZipOutputStream out, ExportParams params ); >>> >> >>> >> void read( BufferedReader reader, ImportParams params ); >>> >> } >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java >>> >> 2009-11-02 15:55:44 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/ExportPipeThread.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -52,7 +52,8 @@ >>> >> >>> >> private List<XMLConverter> xsdConverters = new >>> >> ArrayList<XMLConverter>(); >>> >> private List<XMLConverter> xmlConverters = new >>> >> ArrayList<XMLConverter>(); >>> >> - >>> >> + private List<CSVConverter> csvConverters = new >>> >> ArrayList<CSVConverter>(); >>> >> + >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // Parameters >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> @@ -122,6 +123,11 @@ >>> >> this.xmlConverters.add( converter ); >>> >> } >>> >> >>> >> + public void registerCSVConverter ( CSVConverter converter ) >>> >> + { >>> >> + this.csvConverters.add( converter ); >>> >> + } >>> >> + >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // Thread implementation >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> @@ -159,9 +165,20 @@ >>> >> } >>> >> >>> >> afterXML( writer ); >>> >> - >>> >> + >>> >> closeDocument( writer ); >>> >> - >>> >> + >>> >> + StreamUtils.closeZipEntry( zipOutputStream ); >>> >> + >>> >> + // >>> >> ----------------------------------------------------------------- >>> >> + // CSV >>> >> + // >>> >> ----------------------------------------------------------------- >>> >> + >>> >> + for ( CSVConverter converter : csvConverters ) >>> >> + { >>> >> + converter.write( zipOutputStream, params ); >>> >> + } >>> >> + >>> >> log.info( "Export done" ); >>> >> } >>> >> catch ( Exception ex ) >>> >> @@ -172,12 +189,10 @@ >>> >> } >>> >> finally >>> >> { >>> >> - StreamUtils.finishZipEntry( zipOutputStream ); >>> >> + writer.closeWriter(); >>> >> >>> >> StreamUtils.closeOutputStream( zipOutputStream ); >>> >> >>> >> - writer.closeWriter(); >>> >> - >>> >> NameMappingUtil.clearMapping(); >>> >> } >>> >> } >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java >>> >> 2009-06-10 22:25:07 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/converter/ReportTableDataConverter.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -27,14 +27,17 @@ >>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>> >> */ >>> >> >>> >> -import static org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR; >>> >> +import static >>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.CSV_EXTENSION; >>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE; >>> >> +import static >>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B; >>> >> import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode; >>> >> >>> >> import java.io.BufferedReader; >>> >> -import java.io.BufferedWriter; >>> >> import java.io.IOException; >>> >> import java.util.Iterator; >>> >> import java.util.SortedMap; >>> >> +import java.util.zip.ZipEntry; >>> >> +import java.util.zip.ZipOutputStream; >>> >> >>> >> import org.hisp.dhis.importexport.CSVConverter; >>> >> import org.hisp.dhis.importexport.ExportParams; >>> >> @@ -69,27 +72,29 @@ >>> >> // CSVConverter implementation >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> >>> >> - public void write( BufferedWriter writer, ExportParams params ) >>> >> + public void write( ZipOutputStream out, ExportParams params ) >>> >> { >>> >> try >>> >> { >>> >> - for ( Integer id : params.getReportTables() ) //TODO more >>> >> than one? >>> >> + for ( Integer id : params.getReportTables() ) >>> >> { >>> >> + out.putNextEntry( new ZipEntry( "ReportTable" + id + >>> >> CSV_EXTENSION ) ); >>> >> + >>> >> ReportTableData data = >>> >> reportTableService.getReportTableData( id, params.getFormat() ); >>> >> >>> >> Iterator<String> columns = >>> >> data.getPrettyPrintColumns().iterator(); >>> >> >>> >> while ( columns.hasNext() ) >>> >> { >>> >> - writer.write( csvEncode( columns.next() ) ); >>> >> + out.write( csvEncode( columns.next() ).getBytes() >>> >> ); >>> >> >>> >> if ( columns.hasNext() ) >>> >> { >>> >> - writer.write( SEPARATOR ); >>> >> + out.write( SEPARATOR_B ); >>> >> } >>> >> } >>> >> >>> >> - writer.newLine(); >>> >> + out.write( NEWLINE ); >>> >> >>> >> for ( SortedMap<Integer, String> row : data.getRows() >>> >> ) >>> >> { >>> >> @@ -97,15 +102,15 @@ >>> >> >>> >> while ( values.hasNext() ) >>> >> { >>> >> - writer.write( csvEncode( values.next() ) ); >>> >> + out.write( csvEncode( values.next() >>> >> ).getBytes() >>> >> ); >>> >> >>> >> if ( values.hasNext() ) >>> >> { >>> >> - writer.write( SEPARATOR ); >>> >> + out.write( SEPARATOR_B ); >>> >> } >>> >> } >>> >> >>> >> - writer.newLine(); >>> >> + out.write( NEWLINE ); >>> >> } >>> >> } >>> >> } >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java >>> >> 2009-06-10 22:25:07 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/CSVExportPipeThread.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -27,9 +27,9 @@ >>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>> >> */ >>> >> >>> >> -import java.io.BufferedWriter; >>> >> import java.util.ArrayList; >>> >> import java.util.List; >>> >> +import java.util.zip.ZipOutputStream; >>> >> >>> >> import org.apache.commons.logging.Log; >>> >> import org.apache.commons.logging.LogFactory; >>> >> @@ -48,13 +48,6 @@ >>> >> { >>> >> private static final Log log = LogFactory.getLog( >>> >> CSVExportPipeThread.class ); >>> >> >>> >> - private BufferedWriter writer; >>> >> - >>> >> - public void setWriter( BufferedWriter writer ) >>> >> - { >>> >> - this.writer = writer; >>> >> - } >>> >> - >>> >> private ExportParams params; >>> >> >>> >> public void setParams( ExportParams params ) >>> >> @@ -62,6 +55,13 @@ >>> >> this.params = params; >>> >> } >>> >> >>> >> + private ZipOutputStream outputStream; >>> >> + >>> >> + public void setOutputStream( ZipOutputStream outputStream ) >>> >> + { >>> >> + this.outputStream = outputStream; >>> >> + } >>> >> + >>> >> private List<CSVConverter> converters = new >>> >> ArrayList<CSVConverter>(); >>> >> >>> >> public void registerCSVConverter( CSVConverter converter ) >>> >> @@ -90,14 +90,18 @@ >>> >> >>> >> for ( CSVConverter converter : converters ) >>> >> { >>> >> - converter.write( writer, params ); >>> >> + converter.write( outputStream, params ); >>> >> } >>> >> >>> >> log.info( "Export finished" ); >>> >> } >>> >> + catch ( Exception ex ) >>> >> + { >>> >> + throw new RuntimeException( ex ); >>> >> + } >>> >> finally >>> >> { >>> >> - StreamUtils.closeWriter( writer ); >>> >> + StreamUtils.closeOutputStream( outputStream ); >>> >> } >>> >> } >>> >> } >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java >>> >> 2009-06-10 22:25:07 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/exporter/DefaultCSVExportService.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -28,13 +28,11 @@ >>> >> */ >>> >> >>> >> import java.io.BufferedInputStream; >>> >> -import java.io.BufferedWriter; >>> >> +import java.io.BufferedOutputStream; >>> >> import java.io.IOException; >>> >> import java.io.InputStream; >>> >> -import java.io.OutputStreamWriter; >>> >> import java.io.PipedInputStream; >>> >> import java.io.PipedOutputStream; >>> >> -import java.util.zip.ZipEntry; >>> >> import java.util.zip.ZipOutputStream; >>> >> >>> >> import org.hibernate.SessionFactory; >>> >> @@ -50,8 +48,6 @@ >>> >> public class DefaultCSVExportService >>> >> implements ExportService >>> >> { >>> >> - private static final String ZIP_ENTRY_NAME = "Export.csv"; >>> >> - >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // Dependencies >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> @@ -87,20 +83,18 @@ >>> >> >>> >> PipedInputStream in = new PipedInputStream( out ); >>> >> >>> >> - ZipOutputStream zipOut = new ZipOutputStream( out ); >>> >> + ZipOutputStream zipOut = new ZipOutputStream( new >>> >> BufferedOutputStream( out ) ); >>> >> >>> >> - zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) ); >>> >> + //zipOut.putNextEntry( new ZipEntry( ZIP_ENTRY_NAME ) ); >>> >> >>> >> - BufferedWriter writer = new BufferedWriter( new >>> >> OutputStreamWriter( zipOut ) ); >>> >> - >>> >> // >>> >> ----------------------------------------------------------------- >>> >> // Writes to one end of the pipe >>> >> // >>> >> ----------------------------------------------------------------- >>> >> >>> >> CSVExportPipeThread thread = new CSVExportPipeThread( >>> >> sessionFactory ); >>> >> >>> >> - thread.setWriter( writer ); >>> >> thread.setParams( params ); >>> >> + thread.setOutputStream( zipOut ); >>> >> >>> >> thread.registerCSVConverter( new ReportTableDataConverter( >>> >> reportTableService ) ); >>> >> >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java >>> >> 2009-04-16 09:53:17 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/csv/util/CsvUtil.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -33,15 +33,80 @@ >>> >> */ >>> >> public class CsvUtil >>> >> { >>> >> - public static final char SEPARATOR = ','; >>> >> - >>> >> - private static final String ENCLOSURE = "\""; >>> >> - >>> >> - public static String csvEncode( String string ) >>> >> - { >>> >> - string = string.replaceAll( ENCLOSURE, ENCLOSURE + ENCLOSURE >>> >> ); >>> >> - string = ENCLOSURE + string + ENCLOSURE; >>> >> + public static final String SEPARATOR = ","; >>> >> + public static final byte[] SEPARATOR_B = SEPARATOR.getBytes(); >>> >> + public static final byte[] NEWLINE = "\n".getBytes(); >>> >> + >>> >> + public static final String CSV_EXTENSION = ".csv"; >>> >> + private static final String ENCLOSURE = "\""; >>> >> + private static final String EMPTY = ""; >>> >> + >>> >> + /** >>> >> + * Encodes the given value to a CSV acceptable value. >>> >> + * >>> >> + * @param value the value. >>> >> + * @return the CSV encoded value. >>> >> + */ >>> >> + public static String csvEncode( int value ) >>> >> + { >>> >> + return csvEncode( String.valueOf( value ) ); >>> >> + } >>> >> + >>> >> + /** >>> >> + * Encodes the given value to a CSV acceptable value. >>> >> + * >>> >> + * @param value the value. >>> >> + * @return the CSV encoded value. >>> >> + */ >>> >> + public static String csvEncode( String value ) >>> >> + { >>> >> + if ( value == null ) >>> >> + { >>> >> + value = EMPTY; >>> >> + } >>> >> + else >>> >> + { >>> >> + value = value.replaceAll( ENCLOSURE, ENCLOSURE + >>> >> ENCLOSURE >>> >> ); >>> >> + value = ENCLOSURE + value + ENCLOSURE; >>> >> + } >>> >> + >>> >> + return value; >>> >> + } >>> >> + >>> >> + /** >>> >> + * Appends a separator to the value and returns the value as a >>> >> byte >>> >> array. >>> >> + * >>> >> + * @param value the value. >>> >> + * @return a byte araray. >>> >> + */ >>> >> + public static byte[] getCsvValue( int value ) >>> >> + { >>> >> + return getCsvEndValue( value + SEPARATOR ); >>> >> + } >>> >> + >>> >> + /** >>> >> + * Appends a separator to the value and returns the value as a >>> >> byte >>> >> array. >>> >> + * >>> >> + * @param value the value. >>> >> + * @return a byte araray. >>> >> + */ >>> >> + public static byte[] getCsvValue( String value ) >>> >> + { >>> >> + return getCsvEndValue( value + SEPARATOR ); >>> >> + } >>> >> + >>> >> + public static byte[] getCsvEndValue( int value ) >>> >> + { >>> >> + return getCsvEndValue( String.valueOf( value ) ); >>> >> + } >>> >> + >>> >> + public static byte[] getCsvEndValue( String value ) >>> >> + { >>> >> + if ( value == null ) >>> >> + { >>> >> + return EMPTY.getBytes(); >>> >> + } >>> >> >>> >> - return string; >>> >> + return ( value ).getBytes(); >>> >> } >>> >> } >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java >>> >> 2010-02-08 07:04:26 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/converter/DataValueConverter.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -27,17 +27,28 @@ >>> >> * SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. >>> >> */ >>> >> >>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.NEWLINE; >>> >> +import static >>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.SEPARATOR_B; >>> >> +import static org.hisp.dhis.importexport.csv.util.CsvUtil.csvEncode; >>> >> +import static >>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvValue; >>> >> +import static >>> >> org.hisp.dhis.importexport.csv.util.CsvUtil.getCsvEndValue; >>> >> + >>> >> import java.io.BufferedReader; >>> >> -import java.io.BufferedWriter; >>> >> import java.io.IOException; >>> >> +import java.util.Collection; >>> >> import java.util.Map; >>> >> +import java.util.zip.ZipEntry; >>> >> +import java.util.zip.ZipOutputStream; >>> >> >>> >> import org.amplecode.quick.BatchHandler; >>> >> +import org.amplecode.quick.StatementManager; >>> >> import org.hisp.dhis.dataelement.DataElement; >>> >> import org.hisp.dhis.dataelement.DataElementCategoryOptionCombo; >>> >> import org.hisp.dhis.dataelement.DataElementCategoryService; >>> >> +import org.hisp.dhis.datamart.DataMartService; >>> >> import org.hisp.dhis.datavalue.DataValue; >>> >> import org.hisp.dhis.datavalue.DataValueService; >>> >> +import org.hisp.dhis.datavalue.DeflatedDataValue; >>> >> import org.hisp.dhis.importexport.CSVConverter; >>> >> import org.hisp.dhis.importexport.ExportParams; >>> >> import org.hisp.dhis.importexport.GroupMemberType; >>> >> @@ -47,7 +58,10 @@ >>> >> import >>> >> org.hisp.dhis.importexport.converter.AbstractDataValueConverter; >>> >> import org.hisp.dhis.organisationunit.OrganisationUnit; >>> >> import org.hisp.dhis.period.Period; >>> >> +import org.hisp.dhis.period.PeriodService; >>> >> +import org.hisp.dhis.system.util.DateUtils; >>> >> import org.hisp.dhis.system.util.MimicingHashMap; >>> >> +import org.hisp.dhis.system.util.StreamUtils; >>> >> >>> >> /** >>> >> * @author Lars Helge Overland >>> >> @@ -57,8 +71,11 @@ >>> >> extends AbstractDataValueConverter implements CSVConverter >>> >> { >>> >> private static final String SEPARATOR = ","; >>> >> + private static final String FILENAME = "RoutineData.txt"; >>> >> >>> >> private DataElementCategoryService categoryService; >>> >> + private PeriodService periodService; >>> >> + private StatementManager statementManager; >>> >> >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // Properties >>> >> @@ -72,6 +89,14 @@ >>> >> // Constructor >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> >>> >> + public DataValueConverter( PeriodService periodService, >>> >> DataMartService dataMartService, >>> >> + StatementManager statementManager ) >>> >> + { >>> >> + this.periodService = periodService; >>> >> + this.dataMartService = dataMartService; >>> >> + this.statementManager = statementManager; >>> >> + } >>> >> + >>> >> /** >>> >> * Constructor for read operations. >>> >> */ >>> >> @@ -95,9 +120,81 @@ >>> >> // CSVConverter implementation >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> >>> >> - public void write( BufferedWriter writer, ExportParams params ) >>> >> + public void write( ZipOutputStream out, ExportParams params ) >>> >> { >>> >> - // Not implemented >>> >> + try >>> >> + { >>> >> + out.putNextEntry( new ZipEntry( FILENAME ) ); >>> >> + >>> >> + out.write( getCsvValue( csvEncode( "RoutineDataID" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "OrgUnitID" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "DataElementID" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "DataPeriodID" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryText" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryYesNo" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryNumber" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryDate" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryMemo" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "EntryObject" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "Check" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "Verified" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "Deleted" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "Comment" ) ) ); >>> >> + out.write( getCsvValue( csvEncode( "LastUserID" ) ) ); >>> >> + out.write( getCsvEndValue( csvEncode( "LastUpdated" ) ) >>> >> ); >>> >> + >>> >> + out.write( NEWLINE ); >>> >> + >>> >> + if ( params.isIncludeDataValues() ) >>> >> + { >>> >> + if ( params.getStartDate() != null && >>> >> params.getEndDate() >>> >> != null ) >>> >> + { >>> >> + Collection<DeflatedDataValue> values = null; >>> >> + >>> >> + Collection<Period> periods = >>> >> periodService.getIntersectingPeriods( params.getStartDate(), >>> >> params.getEndDate() ); >>> >> + >>> >> + statementManager.initialise(); >>> >> + >>> >> + for ( final Integer element : >>> >> params.getDataElements() ) >>> >> + { >>> >> + for ( final Period period : periods ) >>> >> + { >>> >> + values = >>> >> dataMartService.getDeflatedDataValues( element, period.getId(), >>> >> params.getOrganisationUnits() ); >>> >> + >>> >> + for ( final DeflatedDataValue value : >>> >> values >>> >> ) >>> >> + { >>> >> + out.write( getCsvValue( 0 ) ); >>> >> + out.write( getCsvValue( >>> >> value.getSourceId() ) ); >>> >> + out.write( getCsvValue( >>> >> value.getDataElementId() ) ); >>> >> + out.write( getCsvValue( >>> >> value.getPeriodId() ) ); >>> >> + out.write( SEPARATOR_B ); >>> >> + out.write( SEPARATOR_B ); >>> >> + out.write( getCsvValue( csvEncode( >>> >> value.getValue() ) ) ); >>> >> + out.write( SEPARATOR_B ); >>> >> + out.write( SEPARATOR_B ); >>> >> + out.write( SEPARATOR_B ); >>> >> + out.write( getCsvValue( 0 ) ); >>> >> + out.write( getCsvValue( 0 ) ); >>> >> + out.write( getCsvValue( 0 ) ); >>> >> + out.write( getCsvValue( csvEncode( >>> >> value.getComment() ) ) ); >>> >> + out.write( getCsvValue( 1 ) ); >>> >> + out.write( getCsvEndValue( >>> >> DateUtils.getAccessDateString( value.getTimestamp() ) ) ); >>> >> + >>> >> + out.write( NEWLINE ); >>> >> + } >>> >> + } >>> >> + } >>> >> + >>> >> + statementManager.destroy(); >>> >> + } >>> >> + } >>> >> + >>> >> + StreamUtils.closeZipEntry( out ); >>> >> + } >>> >> + catch ( IOException ex ) >>> >> + { >>> >> + throw new RuntimeException( "Failed to write data", ex ); >>> >> + } >>> >> } >>> >> >>> >> public void read( BufferedReader reader, ImportParams params ) >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java >>> >> 2009-11-07 14:09:00 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/java/org/hisp/dhis/importexport/dhis14/xml/exporter/DefaultDhis14XMLExportService.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -35,16 +35,19 @@ >>> >> import java.util.zip.ZipEntry; >>> >> import java.util.zip.ZipOutputStream; >>> >> >>> >> +import org.amplecode.quick.StatementManager; >>> >> import org.amplecode.staxwax.factory.XMLFactory; >>> >> import org.amplecode.staxwax.writer.XMLWriter; >>> >> import org.hibernate.SessionFactory; >>> >> import org.hisp.dhis.dataelement.DataElementService; >>> >> +import org.hisp.dhis.datamart.DataMartService; >>> >> import org.hisp.dhis.importexport.ExportParams; >>> >> import org.hisp.dhis.importexport.ExportPipeThread; >>> >> import org.hisp.dhis.importexport.ExportService; >>> >> import >>> >> >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.CalculatedDataElementAssociationConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataElementConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataTypeConverter; >>> >> +import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.DataValueConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.IndicatorTypeConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.PeriodTypeConverter; >>> >> @@ -60,6 +63,7 @@ >>> >> import >>> >> >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserRoleXSDConverter; >>> >> import >>> >> org.hisp.dhis.importexport.dhis14.xml.converter.xsd.UserXSDConverter; >>> >> import org.hisp.dhis.indicator.IndicatorService; >>> >> +import org.hisp.dhis.period.PeriodService; >>> >> >>> >> /** >>> >> * @author Lars Helge Overland >>> >> @@ -100,6 +104,27 @@ >>> >> this.indicatorService = indicatorService; >>> >> } >>> >> >>> >> + private PeriodService periodService; >>> >> + >>> >> + public void setPeriodService( PeriodService periodService ) >>> >> + { >>> >> + this.periodService = periodService; >>> >> + } >>> >> + >>> >> + private DataMartService dataMartService; >>> >> + >>> >> + public void setDataMartService( DataMartService dataMartService ) >>> >> + { >>> >> + this.dataMartService = dataMartService; >>> >> + } >>> >> + >>> >> + private StatementManager statementManager; >>> >> + >>> >> + public void setStatementManager( StatementManager >>> >> statementManager >>> >> ) >>> >> + { >>> >> + this.statementManager = statementManager; >>> >> + } >>> >> + >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // ExportService implementation >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> @@ -122,7 +147,7 @@ >>> >> zipOut.putNextEntry( new ZipEntry( "Export.xml" ) ); >>> >> >>> >> XMLWriter writer = XMLFactory.getPlainXMLWriter( zipOut ); >>> >> - >>> >> + >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> // Writes to one end of the pipe >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> @@ -156,6 +181,8 @@ >>> >> thread.registerXMLConverter( new UserConverter() ); >>> >> thread.registerXMLConverter( new UserRoleConverter() ); >>> >> >>> >> + thread.registerCSVConverter( new DataValueConverter( >>> >> periodService, dataMartService, statementManager ) ); >>> >> + >>> >> thread.start(); >>> >> >>> >> // >>> >> >>> >> ------------------------------------------------------------------------- >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml' >>> >> --- >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml >>> >> 2010-02-02 19:21:58 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-services/dhis-service-importexport/src/main/resources/META-INF/dhis/beans.xml >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -381,6 +381,9 @@ >>> >> <property name="sessionFactory" ref="sessionFactory" /> >>> >> <property name="dataElementService" >>> >> ref="org.hisp.dhis.dataelement.DataElementService" /> >>> >> <property name="indicatorService" >>> >> ref="org.hisp.dhis.indicator.IndicatorService" /> >>> >> + <property name="periodService" >>> >> ref="org.hisp.dhis.period.PeriodService" /> >>> >> + <property name="dataMartService" >>> >> ref="org.hisp.dhis.datamart.DataMartService" /> >>> >> + <property name="statementManager" >>> >> ref="statementManager" >>> >> /> >>> >> </bean> >>> >> >>> >> <!-- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - >>> >> - >>> >> - >>> >> --> >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java >>> >> 2009-11-19 19:16:46 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/DateUtils.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -49,6 +49,20 @@ >>> >> public static final String DEFAULT_DATE_FORMAT = "yyyy-MM-dd"; >>> >> >>> >> /** >>> >> + * Formats a Date to the Access date format. >>> >> + * >>> >> + * @param date the Date to parse. >>> >> + * @return a formatted date string. >>> >> + */ >>> >> + public static String getAccessDateString( Date date ) >>> >> + { >>> >> + final SimpleDateFormat format = new SimpleDateFormat(); >>> >> + format.applyPattern( "yyyy/MM/dd HH:mm:ss" ); >>> >> + >>> >> + return date != null ? format.format( date ) : null; >>> >> + } >>> >> + >>> >> + /** >>> >> * Formats a Date to the IXF date format which is >>> >> YYYY-MM-DD'T'HH:MM:SS. >>> >> * >>> >> * @param date the Date to parse. >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java' >>> >> --- >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java >>> >> 2009-12-19 15:20:41 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-support/dhis-support-system/src/main/java/org/hisp/dhis/system/util/StreamUtils.java >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -422,9 +422,26 @@ >>> >> } >>> >> >>> >> /** >>> >> + * Closes the current ZipEntry and positions the stream for >>> >> writing >>> >> the next entry. >>> >> + * >>> >> + * @param out the ZipOutputStream. >>> >> + */ >>> >> + public static void closeZipEntry( ZipOutputStream out ) >>> >> + { >>> >> + try >>> >> + { >>> >> + out.closeEntry(); >>> >> + } >>> >> + catch ( Exception ex ) >>> >> + { >>> >> + throw new RuntimeException( "Failed to close the current >>> >> ZipEntry", ex ); >>> >> + } >>> >> + } >>> >> + >>> >> + /** >>> >> * Finishes writing the contents of the ZIP output stream without >>> >> closing the underlying stream. >>> >> * >>> >> - * @param out the ZipOutputStream to write to. >>> >> + * @param out the ZipOutputStream. >>> >> */ >>> >> public static void finishZipEntry( ZipOutputStream out ) >>> >> { >>> >> @@ -434,7 +451,7 @@ >>> >> } >>> >> catch ( Exception ex ) >>> >> { >>> >> - throw new RuntimeException( "Failed to finish >>> >> ZipOutputStream", ex ); >>> >> + throw new RuntimeException( "Failed to finish the content >>> >> of >>> >> the ZipOutputStream", ex ); >>> >> } >>> >> } >>> >> >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties' >>> >> --- >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties >>> >> 2010-01-27 21:45:11 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/resources/org/hisp/dhis/importexport/i18n_module.properties >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -146,6 +146,7 @@ >>> >> import_from_other_systems = Import from other systems >>> >> DHIS14_import = DHIS 1.4 Import >>> >> DHIS14_metadata_export = DHIS 1.4 Metadata Export >>> >> +DHIS14_data_export = DHIS 1.4 Data Export >>> >> IXF_import = IXF Import >>> >> accept_incoming_records = Accept incoming records >>> >> include_datavalues = Include data values >>> >> @@ -442,4 +443,5 @@ >>> >> intro_ixf_metadata_export = Do an export of meta-data or dimensional >>> >> data >>> >> describing the facts. Indicator Transmission Format (IXF) is a >>> >> standard >>> >> developed by the WHO. >>> >> intro_DHIS14_metadata_export = Do an export of meta-data or >>> >> dimensional >>> >> data describing the facts. DHIS 1.4 is the predecessor of DHIS 2. >>> >> intro_DHIS14_detailed_metadata_export = Do an export of an detailed >>> >> selection of meta-data. DHIS 1.4 is the predecessor of DHIS 2. >>> >> +intro_DHIS14_data_export = Do an export of data values or facts. DHIS >>> >> 1.4 >>> >> is the predecessor of DHIS 2. >>> >> intro_pdf_metadata_export = Portable Document Format (PDF) is a file >>> >> format for document exchange. >>> >> >>> >> === modified file >>> >> >>> >> 'dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm' >>> >> --- >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm >>> >> 2010-01-28 11:51:39 +0000 >>> >> +++ >>> >> >>> >> dhis-2/dhis-web/dhis-web-importexport/src/main/webapp/dhis-web-importexport/externalExportMenu.vm >>> >> 2010-02-08 10:56:08 +0000 >>> >> @@ -2,8 +2,9 @@ >>> >> <h3>$i18n.getString( "export_to_other_systems" )</h3> >>> >> >>> >> <ul class="introList"> >>> >> + #introListItem( >>> >> "displayMetaDataExportForm.action?exportFormat=IXF" >>> >> "ixf_metadata_export" ) >>> >> #introListItem( >>> >> "displayDataValueExportForm.action?exportFormat=IXF" >>> >> "ixf_data_export" ) >>> >> - #introListItem( >>> >> "displayMetaDataExportForm.action?exportFormat=IXF" >>> >> "ixf_metadata_export" ) >>> >> #introListItem( >>> >> "displayMetaDataExportForm.action?exportFormat=DHIS14XML" >>> >> "DHIS14_metadata_export" ) >>> >> #introListItem( >>> >> "displayDetailedMetaDataExportForm.action?exportFormat=DHIS14XML" >>> >> "DHIS14_detailed_metadata_export" ) >>> >> + #introListItem( >>> >> "displayDataValueExportForm.action?exportFormat=DHIS14XML" >>> >> "DHIS14_data_export" ) >>> >> #introListItem( >>> >> "displayMetaDataExportForm.action?exportFormat=PDF" >>> >> "pdf_metadata_export" ) >>> >> \ No newline at end of file >>> >> >>> >> >>> >> _______________________________________________ >>> >> Mailing list: https://launchpad.net/~dhis2-devs >>> >> Post to : [email protected] >>> >> Unsubscribe : https://launchpad.net/~dhis2-devs >>> >> More help : https://help.launchpad.net/ListHelp >>> >> >>> > >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~dhis2-devs >>> > Post to : [email protected] >>> > Unsubscribe : https://launchpad.net/~dhis2-devs >>> > More help : https://help.launchpad.net/ListHelp >>> > >>> > >> >> > -- Sendt fra min mobile enhet _______________________________________________ Mailing list: https://launchpad.net/~dhis2-devs Post to : [email protected] Unsubscribe : https://launchpad.net/~dhis2-devs More help : https://help.launchpad.net/ListHelp
