Your browser doesn't support the features required by impress.js, so you are presented with a simplified version of this presentation.

For the best experience please use the latest Chrome, Safari or Firefox browser.

Fork me on GitHub
openPMD A meta-data standard
openPMD for mesh based data ...
openPMD and particle data sets.
Basic Concepts Optimizations & Performance Time Series and further Options (upcoming slide) Developer Tools & Processing Domain-Specific Extensions Overview Poster What are you interested in?
Example Standard Clean Common Structure
ED-PIC Extension ... and Domain Specific Extensions
 

ADIOS HDF5

*currently implemented, but not limited to

For hierarchical, self-describing data formats.

How to avoid confusion between actual low-level data sets & attributes (HDF5 speak), variables & attributes (ADIOS speak), files & groups/folders and the actual physical quantities?

openPMD naming convention
Physical quanities are records: Record: each particle property or mesh
Their actual components are stored in (multi-dimensional) arrays (= data sets / variables) inside those records! Position.x: not a record, it's a component
Required attributes for each record: Attributes for all records
unitDimension: Attributes for all records
Required attributes for each mesh record: Required attributes for each mesh record component: Attributes for mesh records
Required records for each particle species:
Optional: Attributes for particle records
Examples for records  
electric field \(\vec E(\vec r)\): temperature \(T(\vec r)\): Legend: Group / (multi-dim) Array
electron position \(\vec r\): electron charge \(Q\): Legend: Group / (1D) Array
Ok ok, I got it! But what about constant components in a record?  
electron charge \(Q\) might be constant for all particles stored in species electrons: Legend: Group / Array / Attribute
Legend: Group / Array / Attribute
possible for any record component, e.g.: Legend: Group / Array / Attribute
All right. But how to handle Petabytes of data written from thousands (-millions) of compute nodes?  
Parallel file formats
(Spatial) Domain Decomposition Particle Patches: Honor Decomposition
(Spatial) Domain Decomposition Particle Patches: Disjoint Particle Sets
(Spatial) Domain Decomposition Particle Patches: [Offset:Offset+Count]
(Spatial) Domain Decomposition Particle Patches: (Spatial) Hyperrectangles

In principle and everywhere*: a human-readable comment (text) attribute is encouraged for everything not covered by the standard.


* reserved for each group and data set

Comment Attribute

openPMD defines a minimal set of attributes.

You can always add more attributes and records!

openPMD is a not exclusive

openPMD defines a minimal set of attributes, e.g.


Required Base Attributes for `/`
Recommended Base Attributes for `/`
Example files and creator scripts:
tools/ $ ./createExamples_h5.py
Developer Tools: Examples
$ ./checkOpenPMD_h5.py -i example.h5 --EDPIC

Warning: Attribute softwareVersion (recommended) does NOT exist in `/`!
Found 1 iteration(s)
Iteration 0 : found 2 meshes
Iteration 0 : found 1 particle species

Result: 0 Errors and 1 Warning.
Developer Tools: Checker Script
Serial processing: high-level API Developer Tools: Serial Processing
openPMD viewer Developer Tools: Parallel Processing
Parallel processing I: integration in suites such as Developer Tools: Parallel Processing
Parallel processing II: integration in parallel python processing via pyDive Developer Tools: Parallel Processing
Common tools Developer Tools: Parallel Processing
Extensions Extensions: Domain-Specific Additions
Implemented by PIConGPU & Warp The ED-PIC Extension
One could add extensions for Other Domain-Specific Extensions
For future versions Future Plans

Use the spacebar or arrow keys to navigate