openPMD::detail::BufferedActions Struct Reference

Collaboration diagram for openPMD::detail::BufferedActions:

Public Types
using	AttributeMap_t = std::map< std::string, adios2::Params >

Public Member Functions
	BufferedActions (BufferedActions const &)=delete
	BufferedActions (ADIOS2IOHandlerImpl &impl, InvalidatableFile file)
void	finalize ()
	Implementation of destructor, will only run once.
adios2::Engine &	getEngine ()
adios2::Engine &	requireActiveStep ()
template<typename BA >
void	enqueue (BA &&ba)
template<typename BA >
void	enqueue (BA &&ba, decltype(m_buffer) &)
template<typename F >
void	flush (FlushLevel level, F &&performPutsGets, bool writeAttributes, bool flushUnconditionally)
	Flush deferred IO actions. More...
void	flush (FlushLevel, bool writeAttributes=false)
	Overload of flush() that uses adios2::Engine::Perform(Puts|Gets) and does not flush unconditionally.
AdvanceStatus	advance (AdvanceMode mode)
	Begin or end an ADIOS step. More...
void	drop ()
const AttributeMap_t &	availableAttributes ()
std::vector< std::string >	availableAttributesPrefixed (std::string const &prefix)
void	invalidateAttributesMap ()
const AttributeMap_t &	availableVariables ()
std::vector< std::string >	availableVariablesPrefixed (std::string const &prefix)
void	invalidateVariablesMap ()

Public Attributes
std::string	m_file
	The full path to the file created on disk, including the containing directory and the file extension, as determined by ADIOS2IOHandlerImpl::fileSuffix(). More...
const std::string	m_IOName
	ADIOS requires giving names to instances of adios2::IO. More...
adios2::ADIOS &	m_ADIOS
adios2::IO	m_IO
std::vector< std::unique_ptr< BufferedAction > >	m_buffer
	The default queue for deferred actions. More...
std::map< std::string, BufferedAttributeWrite >	m_attributeWrites
	Buffer for attributes to be written in the new (variable-based) attribute layout. More...
std::vector< BufferedAttributeRead >	m_attributeReads
std::vector< std::unique_ptr< BufferedAction > >	m_alreadyEnqueued
	This contains deferred actions that have already been enqueued into ADIOS2, but not yet performed in ADIOS2. More...
adios2::Mode	m_mode
std::map< unsigned, std::unique_ptr< I_UpdateSpan > >	m_updateSpans
	The base pointer of an ADIOS2 span might change after reallocations. More...
const detail::WriteDataset	m_writeDataset
const detail::DatasetReader	m_readDataset
const detail::AttributeReader	m_attributeReader
PreloadAdiosAttributes	preloadAttributes
std::set< std::string >	uncommittedAttributes
bool	optimizeAttributesStreaming = false

Member Function Documentation

advance()

AdvanceStatus openPMD::detail::BufferedActions::advance

(

AdvanceMode

mode

)

Begin or end an ADIOS step.

Parameters

mode	Whether to begin or end a step.

Returns

AdvanceStatus

flush()

template<typename F >

void openPMD::detail::BufferedActions::flush	(	FlushLevel	level,
		F &&	performPutsGets,
		bool	writeAttributes,
		bool	flushUnconditionally
	)

Flush deferred IO actions.

Parameters

level	Flush Level. Only execute performPutsGets if UserFlush.
performPutsGets	A functor that takes as parameters (1) *this and (2) the ADIOS2 engine. Its task is to ensure that ADIOS2 performs Put/Get operations. Several options for this: adios2::Engine::EndStep adios2::Engine::Perform(Puts|Gets) adios2::Engine::Close
writeAttributes	If using the new attribute layout, perform deferred attribute writes now.
flushUnconditionally	Whether to run the functor even if no deferred IO tasks had been queued.

Member Data Documentation

m_alreadyEnqueued

std::vector< std::unique_ptr< BufferedAction > > openPMD::detail::BufferedActions::m_alreadyEnqueued

This contains deferred actions that have already been enqueued into ADIOS2, but not yet performed in ADIOS2.

We must store them somewhere until the next PerformPuts/Gets, EndStep or Close in ADIOS2 to avoid use after free conditions.

m_attributeReads

std::vector< BufferedAttributeRead > openPMD::detail::BufferedActions::m_attributeReads

Todo:

This one is unnecessary, in the new schema, attribute reads do not need to be deferred, but can happen instantly without performance penalty, once preloadAttributes has been filled.

m_attributeWrites

std::map< std::string, BufferedAttributeWrite > openPMD::detail::BufferedActions::m_attributeWrites

Buffer for attributes to be written in the new (variable-based) attribute layout.

Reason: If writing one variable twice within the same ADIOS step, it is undefined which value ADIOS2 will store. We want the last write operation to succeed, so this map stores attribute writes by attribute name, allowing us to override older write commands. The queue is drained only when closing a step / the engine.

m_buffer

std::vector< std::unique_ptr< BufferedAction > > openPMD::detail::BufferedActions::m_buffer

The default queue for deferred actions.

Drained upon BufferedActions::flush().

m_file

std::string openPMD::detail::BufferedActions::m_file

The full path to the file created on disk, including the containing directory and the file extension, as determined by ADIOS2IOHandlerImpl::fileSuffix().

(Meaning, in case of the SST engine, no file suffix since the SST engine automatically adds its suffix unconditionally)

m_IOName

const std::string openPMD::detail::BufferedActions::m_IOName

ADIOS requires giving names to instances of adios2::IO.

We make them different from the actual file name, because of the possible following workflow:

1. create file foo.bp -> would create IO object named foo.bp
2. delete that file (let's ignore that we don't support deletion yet and call it preplanning)
3. create file foo.bp a second time -> would create another IO object named foo.bp -> craash

So, we just give out names based on a counter for IO objects. Hence, next to the actual file name, also store the name for the IO.

m_updateSpans

std::map< unsigned, std::unique_ptr< I_UpdateSpan > > openPMD::detail::BufferedActions::m_updateSpans

The base pointer of an ADIOS2 span might change after reallocations.

The frontend will ask the backend for those updated base pointers. Spans given out by the ADIOS2 backend to the frontend are hence identified by an unsigned integer and stored in this member for later retrieval of the updated base pointer. This map is cleared upon flush points.

---

The documentation for this struct was generated from the following files:

• /tmp/tmp.7iNmvGSbic/openPMD-api/include/openPMD/IO/ADIOS/ADIOS2IOHandler.hpp
• /tmp/tmp.7iNmvGSbic/openPMD-api/src/IO/ADIOS/ADIOS2IOHandler.cpp