Hi Russ (and all), Ken and I very much had in mind when we presented this to Rich and Jeff that we would have a much more doable and sharable co-development task if we stayed away from Big Data transformations. We have many billion rows of data as well, and moving that much data into a new model gets us into highly specialized, database specific tools, as well as dealing with many millions of strange outlier data points.
By transforming only what is necessary for PCOR complex calculations on the cohorts, we can focus more on the data that matters. We are also then free to use the tools that connect us with our patients without worrying how they will all come together with mini-sentinel until the last mile. Thanks, Shawn. From: Russ Waitman [mailto:[email protected]] Sent: Friday, March 21, 2014 6:04 PM To: Dan Connolly; Justin Dale; [email protected] Cc: Murphy, Shawn N.; Mandl, Kenneth ([email protected]); [email protected]; Rothman, Russell ([email protected]); Harris, Paul ([email protected]); Masys, Dan ([email protected]); 'Becich, Michael J' Subject: RE: PCORNet Common Data Model question Hi Justin, Dan (and GPC developers), Been in discussions with Ken and Shawn at Harvard and Trent and Paul at Vandy. Ken and Shawn have a proposal that we'd transform outbound datasets for the specific queries in their format analogous to what we proposed (but not using REDCap). http://scilhs.org/2014/03/11/scilhs-query-workflow/ We think this approach is a manageable task. Since all of us on i2b2 need to pump out the data to PCORNet might co-develop the transform. I still think many investigators might like a REDCap distribution model and since the code already does that, for GPC level requests we can probably have the data export format be a customer preference just like REDCap itself supports CSV, SAS, and Excel output files. PCORNet might say ship it out in the miniSentinel format but another customer might say give me the EAV model that supports more data types not described in miniSentinel (like a medication or lab result). Russ From: [email protected]<mailto:[email protected]> [mailto:[email protected]] On Behalf Of Dan Connolly Sent: Friday, March 21, 2014 2:20 PM To: Justin Dale; [email protected]<mailto:[email protected]> Subject: RE: PCORNet Common Data Model question I was wondering the same thing. More generally: how are we expected to interact with this data model? What are the related protocols/workflows and bytes-on-the-wire formats? I think what you suggest is conceivable. We have code (heron_extract<https://informatics.kumc.edu/changes/heron_extract/>, layered on the R Data Builder in HeronStatsPlugins<https://informatics.kumc.edu/work/wiki/HeronStatsPlugins>) that delivers i2b2 datasets via REDCap, with one form for either patients or encounters, and then one for meds, another for diagnoses, etc. I can imagine adapting that code to this model as a small to medium size departure from our existing data sharing proposal<http://informatics.gpcnetwork.org/trac/Project/wiki/DataSecurity>: The design for the GPC data store consists of a REDCap service and an R Studio service. In addition to the core case-report-form features of REDCap needed for prototyping patient reported outcome instruments, REDCap provides a simple project-based workflow and access control model that is well suited to the access patterns required by the GPC. For example, to submit queries from the GPC DROC to honest brokers at each site, we will add all of the honest brokers as users in a REDCap project that has a query submission survey. On receipt of a query via this survey, each honest broker executes the query against their i2b2 installations and saves the results to an R data file using the R Data Builder. We will use another REDCap project to collect and distribute the results of the query. The honest brokers upload the R data file as file attachments to this project. The REDCap service will use secure HTTP (TLS/SSL) so that the file transfer is encrypted. The GPC honest broker then releases the collection of data files to the investigators by arranging for them to have access via R Studio Server. The R statistical package provides a large toolset for combining de-identified data files for analysis, and the R Studio Server allows researchers to use this toolset while the data remain in the GPC data store. (emphasis mine, in an attempt to extract actionable items from the proposal) -- Dan ________________________________ From: [email protected]<mailto:[email protected]> [[email protected]] on behalf of Justin Dale [[email protected]] Sent: Friday, March 21, 2014 2:05 PM To: [email protected]<mailto:[email protected]> Subject: PCORNet Common Data Model question I'm reviewing the common data model document and trying to understand how it relates to the GPC. Is this the suggested model for the database that will store the aggregated de-identified data that each site sends back to a central location (figure 1.1 in the proposal)? Regards, Justin Dale Manager - Research Development and Support University of Minnesota Academic Health Center - Office of Information Systems Phone: 612.624.9747 Fax: 612.624.7458 [email protected]<mailto:[email protected]> The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
_______________________________________________ Gpc-dev mailing list [email protected] http://listserv.kumc.edu/mailman/listinfo/gpc-dev
