The latest issue of DDI Directions was published October 31st. Headlining this issue is the announcement of the DDI 4 Prototype release. Also highlighted are reports from community meetings and workshops, as well as information about NADDI and EDDI conferences.
In December 2017, we provided an update about development of DDI work products to express continued support for our stable work products -- including DDI-Codebook (DDI 2) and DDI-Lifecycle (DDI 3) -- and to communicate a mid-2018 prototype release of a new work product called DDI 4.
We’re happy to announce the prototype release of DDI 4. This preliminary version is not intended for production but provides an opportunity to test and provide feedback on how the DDI 4 model describes and documents some basic research material, such as a dataset, an instrument, and a codebook.
DDI 4 is based on an information model that can be expressed in different technologies, including standard RDF vocabularies and XML structures. This form of model-based standard is a best practice in the standardization world, and several other domains have used this approach for years to structure their domain-specific standards. The development of DDI 4 will enable the standard to stay abreast of current technological changes and guarantee alignment across different technology implementations. More details about the history and origin of DDI 4 can be found on the “Work Products of the DDI Alliance” web page
We look forward to the DDI community testing this prototype release and providing constructive feedback on its future development. Links to the specification and use cases, as well as instructions for comment, are found at:
We would love to have as much feedback as possible for our next development meeting on November 26th, but the comment period will extend beyond that. We hope to hear from you! Please feel free to contact firstname.lastname@example.org with questions.
Schloss Dagstuhl – Leibniz Center for Informatics, 24 – 28 September 2018, Wadern, Germany
A workshop to increase training capacity on DDI was hosted at the internationally renowned computer science institute at Schloss Dagstuhl in Germany and organized by Joachim Wackerow (GESIS – Leibniz Institute for the Social Sciences). The workshop instructors were Jon Johnson, Dan Smith, and Wendy Thomas, with Arofan Gregory as a volunteer. The DDI Alliance paid the GESIS workshop fees for all participants.
The workshop drew 18 participants from 16 organizations representing 11 countries, including Australia, Canada, Denmark, Estonia, France, Germany, Malawi, Norway, Romania, Sweden, UK, and the United States.
The participants worked through existing training material to build modular training materials. Participants were very motivated and worked often in working groups on specific topics. The resulting slide decks cover core areas of DDI and are, as far as possible, version-agnostic.
All workshop participants promise to conduct a DDI training within the next year.
Wiki site of the workshop:
The output documents are available at:
A subset of the workshop participants have volunteered to continue developing the training materials. The goals are to share the reusable materials with the new trainers and promote these materials on the DDI Alliance website for further reuse and self-guided training.
Workshop Report: “Interoperability of Metadata Standards in Cross-Domain Science, Health, and Social Science Applications”
Schloss Dagstuhl – Leibniz Center for Informatics, 1 – 5 October 2018, Wadern, Germany
A workshop on the practical application of computer science to enable data sharing and data interoperability across disciplinary boundaries was hosted at the internationally renowned computer science institute at Schloss Dagstuhl in Germany. The event was sponsored by CODATA (the Committee on Data of the International Science Council), and the Data Documentation Initiative Alliance (DDI), and subsidized by Dagstuhl; it was organized by Simon Cox (CSIRO Australia and W3C Dataset Exchange Working Group), Simon Hodson (CODATA), Steven McEachern (Australian National University and DDI Alliance), Joachim Wackerow (GESIS - Leibniz Institute for the Social Sciences and DDI Alliance). The workshop brought together 24 participants from many different domains. These included representatives of a number of metadata specifications, as well as researchers involved in pilot projects currently being pursued as part of the ISC and CODATA Data Integration Initiative. A duration of 5 days, and the relative isolation and unique dynamics of Dagstuhl, encourages intense involvement on the part of all participants (as described on the DDI site here).
The workshop examined how modern web-friendly computer science techniques and standards could better enable data-sharing in the context of the Data Integration Initiative pilots. These are major cross-disciplinary data integration projects to advance solutions for three important global challenges: infectious disease outbreaks, resilient cities, and disaster risk reduction. The infectious disease pilot builds on work by the Infectious Diseases Data Observatory (IDDO) to support both research and humanitarian efforts, with Ebola used as the primary example for discussion. The resilient cities pilot focuses on the work in Medellín, Columbia, in partnership with Resilience Brokers. Examples involved air quality measurement, location of hospitals, and geo-spatial data. The disaster risk reduction pilot, led by Public Health England in partnership with the Integrated Research on Disaster Risk is looking at how data could support the Sendai Framework, especially in cases where the SDG indicators would not be sufficient. Different approaches for obtaining data both from within and from outside the realm of official statistics were explored, with an emphasis on research data. In each case, data integration presented significant challenges.
Metadata standards are a part of the computer science landscape which can facilitate the discovery of existing datasets, and their integration and use within a particular scenario. Representatives of many of these standards were present, helping to understand the data integration challenges faced by each of the projects. These standards included many of the W3C Linked Data vocabularies (DCAT, SSN, Data Cube, PROV-O, etc.), DDI, HL7 FHIR, CDISC, DATS, ISO 19115, EML and several others.[i] Some of these standards are focused on the data within a particular discipline or domain. Others are more general in scope. The workshop examined the relationships between these standards in the context of their real-world application (the pilot projects). This required an understanding of the granularity of the metadata being expressed by each standard (at the level of a study or dataset, at the variable and observation level, etc.)
Much of the activity in the workshop was in small working groups composed of both business experts involved in the pilot projects, and experts in the relevant technology and domain standards. Some additional technical topics which arose during the exploration of the pilot projects were also addressed separately by small teams of the appropriate experts.
The workshop was extremely productive, with immediately producing outlines of working papers relating to each of the pilot projects. An article will also be produced describing the overall goals of the effort and the relationship of various standards and technology approaches to the cross-disciplinary data integration projects. The intention is that these will be published in peer-reviewed journals appropriate to their content. In addition, it is anticipated one specific technical output was initiated - for example, a DCAT profile to support granular description of data in online catalogues. The outputs of the workshop will be presented at the upcoming SciDataCon conference (at the 2nd International Data Week organized by CODATA together with the Research Data Alliance and the World Data System) in Gaborone, Botswana in early November of 2018. Further collaborative work between CODATA, the DDI Alliance, and other interested organizations is anticipated in the future, including more intense, focused workshops of this kind.[ii]
Invitation to participate in a joint Technical Committee and DDI 4 Development sprint to be held at DIW Berlin, November 26-30, 2018
Dear DDI Community,
This year we are very pleased to invite you to participate in a joint Technical Committee and DDI 4 Development sprint to be held at DIW Berlin, November 26-30, 2018. Participation is open to the entire DDI community and is a great opportunity to contribute to the DDI standard. We would like to encourage everyone to consider participating, even if you are unable to attend all five days.
The work will fall into the following two, concurrent tracks with exchanges and joint sessions planned where appropriate and as needed:
Finalization of 3.3 based on review
Prepare COGS to accept 3.3 schema content and generate output
XML schema (this would be a future version)
UML model of 3.3 to reflect Moving Forward approach
Evaluate differences between 3.3 and output from COGS
Transition paths between versions across DDI
DDI 4 Moving Forward Project
High-level and conceptual discussion on the direction of DDI 4 regarding content and modeling
Review of post-prototype issues and prototype public release feedback
Role of UML
Use of UML features
Refinement of modeling rules
Binding transformation rules
Role of Canonical XMI as an expression of the model
Plans for new production framework (COGS)
Production process entering content and COGS production line
Clarification of validation checks for entered content
Creation of an iteration cycle regarding modeling, representations, and prototype software implementation to ensure a robust specification on all of these levels.
Creation of validation suites on all of these levels and integration into the production framework
Please let us know if you are interested in joining us, even if not confirmed, and RSVP using this form by October 7, 2018. For questions, please contact Kelly Chatain (email@example.com). Note that there is a limited amount of funding available for the meeting. Funding guidelines and the request form are available online.
Thank you for considering this invitation and we look forward to meeting with you at Berlin!
The DDI Executive Board recently selected the date for the next Annual Meeting of Member Representatives. The meeting will be held on Saturday, 1 June 2019, in Sydney, Australia (directly after the IASSIST Annual Conference, 27-31 May 2019, also to be held in Sydney).
The Annual Meeting of Member Representatives provides a forum for member discussion and feedback. Please save the date. We look forward to seeing many member representatives at the meeting.
The Public Review period of DDI 3.3 has been extended through September 21, 2018. We thank those who have already submitted comments.
For questions or requests, please contact Wendy Thomas, Chair, DDI Technical Committee (firstname.lastname@example.org).
Two articles in the latest issue of the IASSIST Quarterly (Vol 42 No 2 (2018)) discuss DDI. "Flexible DDI Storage," the winner of the IASSIST 2017 conference paper competition, describes "a way to model the binding of DDI to applications in a way that it works independent of most version changes and interpretative differences in a standard like DDI without continuous reimplementation." The other article, "Elaborating a Crosswalk Between Data Documentation Initiative (DDI) and Encoded Archival Description (EAD) for an Emerging Data Archive Service Provider," describes a DDI-to-EAD crosswalk at the State Archives of Belgium.
The DDI Alliance (http://www.ddialliance.org/) is pleased to announce the Public Review of DDI 3.3. This version of the DDI specification includes important updates and new content, including:
- Classification management (based on GSIM/Neuchatel)
- Non-survey data collection (Measurements)
- Questionnaire Design
- Support for DDI as a Property Graph
- Quality Statement improvements (useful for Eurostat reporting)
Additionally, DDI 3.3 has a formal model now, so a detailed changelog can be generated between DDI 3.2 and 3.3.
Links to the specification and instructions for comment are found at: https://ddi-alliance.atlassian.net/wiki/spaces/DDI4/pages/500826115/DDI+Lifecycle+3.3+Public+Review
We are eager to obtain feedback from the DDI and RDF communities on this vocabulary. The comment period is open for six weeks until August 3, 2018, and we hope to hear from you!