openPMD-api
Classes | Public Member Functions | Public Attributes | List of all members
openPMD::HDF5IOHandlerImpl Class Reference
Inheritance diagram for openPMD::HDF5IOHandlerImpl:
Inheritance graph
[legend]
Collaboration diagram for openPMD::HDF5IOHandlerImpl:
Collaboration graph
[legend]

Public Member Functions

 HDF5IOHandlerImpl (AbstractIOHandler *, nlohmann::json config)
 
void createFile (Writable *, Parameter< Operation::CREATE_FILE > const &) override
 Create a new file in physical storage, possibly overriding an existing file. More...
 
void createPath (Writable *, Parameter< Operation::CREATE_PATH > const &) override
 Create all necessary groups for a path, possibly recursively. More...
 
void createDataset (Writable *, Parameter< Operation::CREATE_DATASET > const &) override
 Create a new dataset of given type, extent and storage properties. More...
 
void extendDataset (Writable *, Parameter< Operation::EXTEND_DATASET > const &) override
 Increase the extent of an existing dataset. More...
 
void availableChunks (Writable *, Parameter< Operation::AVAILABLE_CHUNKS > &) override
 Report chunks that are available for loading from the dataset represented by this writable. More...
 
void openFile (Writable *, Parameter< Operation::OPEN_FILE > const &) override
 Open an existing file assuming it conforms to openPMD. More...
 
void closeFile (Writable *, Parameter< Operation::CLOSE_FILE > const &) override
 Close the file corresponding with the writable and release file handles. More...
 
void openPath (Writable *, Parameter< Operation::OPEN_PATH > const &) override
 Open all contained groups in a path, possibly recursively. More...
 
void openDataset (Writable *, Parameter< Operation::OPEN_DATASET > &) override
 Open an existing dataset and determine its datatype and extent. More...
 
void deleteFile (Writable *, Parameter< Operation::DELETE_FILE > const &) override
 Delete an existing file from physical storage. More...
 
void deletePath (Writable *, Parameter< Operation::DELETE_PATH > const &) override
 Delete all objects within an existing path. More...
 
void deleteDataset (Writable *, Parameter< Operation::DELETE_DATASET > const &) override
 Delete an existing dataset. More...
 
void deleteAttribute (Writable *, Parameter< Operation::DELETE_ATT > const &) override
 Delete an existing attribute. More...
 
void writeDataset (Writable *, Parameter< Operation::WRITE_DATASET > const &) override
 Write a chunk of data into an existing dataset. More...
 
void writeAttribute (Writable *, Parameter< Operation::WRITE_ATT > const &) override
 Create a single attribute and fill the value, possibly overwriting an existing attribute. More...
 
void readDataset (Writable *, Parameter< Operation::READ_DATASET > &) override
 Read a chunk of data from an existing dataset. More...
 
void readAttribute (Writable *, Parameter< Operation::READ_ATT > &) override
 Read the value of an existing attribute. More...
 
void listPaths (Writable *, Parameter< Operation::LIST_PATHS > &) override
 List all paths/sub-groups inside a group, non-recursively. More...
 
void listDatasets (Writable *, Parameter< Operation::LIST_DATASETS > &) override
 List all datasets inside a group, non-recursively. More...
 
void listAttributes (Writable *, Parameter< Operation::LIST_ATTS > &) override
 List all attributes associated with an object. More...
 
- Public Member Functions inherited from openPMD::AbstractIOHandlerImpl
 AbstractIOHandlerImpl (AbstractIOHandler *handler)
 
virtual std::future< void > flush ()
 
virtual void advance (Writable *, Parameter< Operation::ADVANCE > &)
 Advance the file/stream that this writable belongs to. More...
 
virtual void closePath (Writable *, Parameter< Operation::CLOSE_PATH > const &)
 Close an openPMD group. More...
 
virtual void getBufferView (Writable *, Parameter< Operation::GET_BUFFER_VIEW > &parameters)
 Get a view into a dataset buffer that can be filled by a user. More...
 

Public Attributes

std::unordered_map< Writable *, std::string > m_fileNames
 
std::unordered_map< std::string, hid_t > m_fileNamesWithID
 
std::unordered_set< hid_t > m_openFileIDs
 
hid_t m_datasetTransferProperty
 
hid_t m_fileAccessProperty
 
hid_t m_H5T_BOOL_ENUM
 
hid_t m_H5T_CFLOAT
 
hid_t m_H5T_CDOUBLE
 
hid_t m_H5T_CLONG_DOUBLE
 
- Public Attributes inherited from openPMD::AbstractIOHandlerImpl
AbstractIOHandlerm_handler
 

Member Function Documentation

◆ availableChunks()

void openPMD::HDF5IOHandlerImpl::availableChunks ( Writable ,
Parameter< Operation::AVAILABLE_CHUNKS > &   
)
overridevirtual

Report chunks that are available for loading from the dataset represented by this writable.

The resulting chunks should be stored into parameters.chunks.

Implements openPMD::AbstractIOHandlerImpl.

◆ closeFile()

void openPMD::HDF5IOHandlerImpl::closeFile ( Writable ,
Parameter< Operation::CLOSE_FILE > const &   
)
overridevirtual

Close the file corresponding with the writable and release file handles.

The operation should succeed in any access mode.

Implements openPMD::AbstractIOHandlerImpl.

◆ createDataset()

void openPMD::HDF5IOHandlerImpl::createDataset ( Writable ,
Parameter< Operation::CREATE_DATASET > const &   
)
overridevirtual

Create a new dataset of given type, extent and storage properties.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The path may contain multiple levels (e.g. group/dataset). The new dataset should have the name parameters.name. This name should not start or end with a slash ("/"). The new dataset should be of datatype parameters.dtype. The new dataset should have an extent of parameters.extent. If possible, the new dataset should be extensible. If possible, the new dataset should be divided into chunks with size parameters.chunkSize. If possible, the new dataset should be compressed according to parameters.compression. This may be format-specific. If possible, the new dataset should be transformed accoring to parameters.transform. This may be format-specific. The Writables file position should correspond to the newly created dataset. The Writable should be marked written when the operation completes successfully.

Todo:
add per dataset chunk control from JSON config
Todo:
allow overwrite with user-provided chunk size

Implements openPMD::AbstractIOHandlerImpl.

◆ createFile()

void openPMD::HDF5IOHandlerImpl::createFile ( Writable ,
Parameter< Operation::CREATE_FILE > const &   
)
overridevirtual

Create a new file in physical storage, possibly overriding an existing file.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The new file should be located in m_handler->directory. The new file should have the filename parameters.name. The filename should include the correct corresponding filename extension. Any existing file should be overwritten if m_handler->m_frontendAccess is Access::CREATE. The Writables file position should correspond to the root group "/" of the hierarchy. The Writable should be marked written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ createPath()

void openPMD::HDF5IOHandlerImpl::createPath ( Writable ,
Parameter< Operation::CREATE_PATH > const &   
)
overridevirtual

Create all necessary groups for a path, possibly recursively.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The path parameters.path may contain multiple levels (e.g. first/second/third/). The Writables file position should correspond to the complete newly created path (i.e. first/second/third/ should be assigned to the Writables file position). The Writable should be marked written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ deleteAttribute()

void openPMD::HDF5IOHandlerImpl::deleteAttribute ( Writable ,
Parameter< Operation::DELETE_ATT > const &   
)
overridevirtual

Delete an existing attribute.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The operation should pass if the Writable was not marked written. The attribute should be associated with the Writable and have the name parameters.name before deletion. The attribute should not be accessible in physical storage after the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ deleteDataset()

void openPMD::HDF5IOHandlerImpl::deleteDataset ( Writable ,
Parameter< Operation::DELETE_DATASET > const &   
)
overridevirtual

Delete an existing dataset.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The operation should pass if the Writable was not marked written. The dataset should have the name parameters.name. This name should not start or end with a slash ("/"). It may also contain the current dataset ".". The dataset should not be accessible in physical storage after the operation completes successfully. The Writables file position should be set to an invalid position (i.e. the pointer should be a nullptr). The Writable should be marked not written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ deleteFile()

void openPMD::HDF5IOHandlerImpl::deleteFile ( Writable ,
Parameter< Operation::DELETE_FILE > const &   
)
overridevirtual

Delete an existing file from physical storage.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The operation should pass if the Writable was not marked written. All handles that correspond to the file should be closed before deletion. The file to delete should have the filename parameters.name. The filename should include the correct corresponding filename extension. The Writables file position should be set to an invalid position (i.e. the pointer should be a nullptr). The Writable should be marked not written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ deletePath()

void openPMD::HDF5IOHandlerImpl::deletePath ( Writable ,
Parameter< Operation::DELETE_PATH > const &   
)
overridevirtual

Delete all objects within an existing path.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The operation should pass if the Writable was not marked written. The path parameters.path may contain multiple levels (e.g. first/second/third/). This path should be relative (i.e. it should not start with a slash "/"). It may also contain the current group ".". All groups and datasets starting from the path should not be accessible in physical storage after the operation completes successfully. The Writables file position should be set to an invalid position (i.e. the pointer should be a nullptr). The Writable should be marked not written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ extendDataset()

void openPMD::HDF5IOHandlerImpl::extendDataset ( Writable ,
Parameter< Operation::EXTEND_DATASET > const &   
)
overridevirtual

Increase the extent of an existing dataset.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The operation should fail if the dataset does not yet exist. The dataset should have the name parameters.name. This name should not start or end with a slash ("/"). The operation should fail if the new extent is not strictly large in every dimension. The dataset should have an extent of parameters.extent.

Implements openPMD::AbstractIOHandlerImpl.

◆ listAttributes()

void openPMD::HDF5IOHandlerImpl::listAttributes ( Writable ,
Parameter< Operation::LIST_ATTS > &   
)
overridevirtual

List all attributes associated with an object.

The operation should fail if the Writable was not marked written. The attribute should be associated with the Writable. The list of attribute names should be stored in the location indicated by the pointer parameters.attributes.

Implements openPMD::AbstractIOHandlerImpl.

◆ listDatasets()

void openPMD::HDF5IOHandlerImpl::listDatasets ( Writable ,
Parameter< Operation::LIST_DATASETS > &   
)
overridevirtual

List all datasets inside a group, non-recursively.

The operation should fail if the Writable was not marked written. The operation should fail if the Writable is not a group. The list of dataset names should be stored in the location indicated by the pointer parameters.datasets.

Implements openPMD::AbstractIOHandlerImpl.

◆ listPaths()

void openPMD::HDF5IOHandlerImpl::listPaths ( Writable ,
Parameter< Operation::LIST_PATHS > &   
)
overridevirtual

List all paths/sub-groups inside a group, non-recursively.

The operation should fail if the Writable was not marked written. The operation should fail if the Writable is not a group. The list of group names should be stored in the location indicated by the pointer parameters.paths.

Implements openPMD::AbstractIOHandlerImpl.

◆ openDataset()

void openPMD::HDF5IOHandlerImpl::openDataset ( Writable ,
Parameter< Operation::OPEN_DATASET > &   
)
overridevirtual

Open an existing dataset and determine its datatype and extent.

The opened dataset should be located in a group below the group of the Writables parent writable->parent. The opened datasets name should be parameters.name. This name should not start or end with a slash ("/"). The opened datasets datatype should be stored in *(parameters.dtype). The opened datasets extent should be stored in *(parameters.extent). The Writables file position should correspond to the opened dataset. The Writable should be marked written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ openFile()

void openPMD::HDF5IOHandlerImpl::openFile ( Writable ,
Parameter< Operation::OPEN_FILE > const &   
)
overridevirtual

Open an existing file assuming it conforms to openPMD.

The operation should fail if m_handler->directory is not accessible. The opened file should have filename parameters.name and include the correct corresponding filename extension. The operation should not open files more than once. If possible, the file should be opened with read-only permissions if m_handler->m_frontendAccess is Access::READ_ONLY. The Writables file position should correspond to the root group "/" of the hierarchy in the opened file. The Writable should be marked written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ openPath()

void openPMD::HDF5IOHandlerImpl::openPath ( Writable ,
Parameter< Operation::OPEN_PATH > const &   
)
overridevirtual

Open all contained groups in a path, possibly recursively.

The operation should overwrite existing file positions, even when the Writable was already marked written. The path parameters.path may contain multiple levels (e.g. first/second/third/). This path should be relative (i.e. it should not start with a slash "/"). The number of levels may be zero, i.e. parameters.path may be an empty string. The Writables file position should correspond to the complete opened path (i.e. first/second/third/ should be assigned to the Writables file position). The Writable should be marked written when the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ readAttribute()

void openPMD::HDF5IOHandlerImpl::readAttribute ( Writable ,
Parameter< Operation::READ_ATT > &   
)
overridevirtual

Read the value of an existing attribute.

The operation should fail if the Writable was not marked written. The operation should fail if the attribute does not exist. The attribute should be associated with the Writable and have the name parameters.name. This name should not contain a slash ("/"). The attribute datatype should be stored in the location indicated by the pointer parameters.dtype. The attribute value should be stored as a generic Variant::resource in the location indicated by the pointer parameters.resource. All datatypes of Datatype should be supported in a type-safe way.

Implements openPMD::AbstractIOHandlerImpl.

◆ readDataset()

void openPMD::HDF5IOHandlerImpl::readDataset ( Writable ,
Parameter< Operation::READ_DATASET > &   
)
overridevirtual

Read a chunk of data from an existing dataset.

The dataset should be associated with the Writable. The operation should fail if the dataset does not exist. The operation should fail if the chunk extent parameters.extent is not smaller or equals in every dimension. The operation should fail if chunk positions parameters.offset+parameters.extent do not reside inside the dataset. The dataset should match the dataype parameters.dtype. The data parameters.data should be a cast-to-void pointer to a flattened version of the chunk data. The chunk should be stored row-major. The region of the chunk should be written to the location indicated by the pointer after the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.

◆ writeAttribute()

void openPMD::HDF5IOHandlerImpl::writeAttribute ( Writable ,
Parameter< Operation::WRITE_ATT > const &   
)
overridevirtual

Create a single attribute and fill the value, possibly overwriting an existing attribute.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The attribute should have the name parameters.name. This name should not contain a slash ("/"). The attribute should be of datatype parameters.dtype. Any existing attribute with the same name should be overwritten. If possible, only the value should be changed if the datatype stays the same. The attribute should be written to physical storage after the operation completes successfully. All datatypes of Datatype should be supported in a type-safe way.

Implements openPMD::AbstractIOHandlerImpl.

◆ writeDataset()

void openPMD::HDF5IOHandlerImpl::writeDataset ( Writable ,
Parameter< Operation::WRITE_DATASET > const &   
)
overridevirtual

Write a chunk of data into an existing dataset.

The operation should fail if m_handler->m_frontendAccess is Access::READ_ONLY. The dataset should be associated with the Writable. The operation should fail if the dataset does not exist. The operation should fail if the chunk extent parameters.extent is not smaller or equals in every dimension. The operation should fail if chunk positions parameters.offset+parameters.extent do not reside inside the dataset. The dataset should match the dataype parameters.dtype. The data parameters.data is a cast-to-void pointer to a flattened version of the chunk data. It should be re-cast to the provided datatype. The chunk is stored row-major. The region of the chunk should be written to physical storage after the operation completes successfully.

Implements openPMD::AbstractIOHandlerImpl.


The documentation for this class was generated from the following files: