caoscrawler.converters.hdf5_converter module

caoscrawler.converters.hdf5_converter.convert_attributes(elt: File | Group | Dataset)

Convert hdf5 attributes to a list of either basic scalar structure elements or ndarrays.

Parameters:

elt (Union[h5py.File, h5py.Group, h5py.Dataset]) – The hdf5 element the attributes of which will be converted to structure elements.

Returns:

converted – A list of the attributes converted to StructureElements (either basic scalar elements or ndarray).

Return type:

list[StructureElement]

caoscrawler.converters.hdf5_converter.convert_h5_element(elt: Group | Dataset, name: str)

Convert a given HDF5 element to the corresponding StructureElement.

Parameters:

• elt (Union[h5py.Group, h5py.Dataset]) – The hdf5 element to be converted.

• name (str) – The name of the StructureElement that the hdf5 element is converted to.

Raises:

ValueError – In case of anything that is not convertible to a HDF5 structure element.

Returns:

The converted StructureElement.

Return type:

StructureElement

caoscrawler.converters.hdf5_converter.convert_basic_element_with_nd_array(value, name: str | None = None, internal_path: str | None = None, msg_prefix: str = '')

Convert a given object either to an ndarray structure element or to a basic scalar structure element.

This function extends convert_basic_element() by a special treatment for certain numpy objects, most importantly ndarrays. They are converted to a scalar in case of a size-1 array, to a list in case of a 1-d array, and to a H5NdarrayElement in all other cases. In addition, numpy integers and floats are also converted to IntegerElements and FloatElements, respectively.

Parameters:

• value – The object to be converted.

• name (str, optional) – The name of the structure element value is being converted to. Default is None.

• internal_path (str, optional) – The internal path of value within the HDF5 file. Default is None.

• msg_prefix (str, optional) – The prefix of the error message that will be raised. Default is "".

Returns:

The StructureElement value was converted to.

Return type:

StructureElement

class caoscrawler.converters.hdf5_converter.H5GroupElement(name: str, value: Group)

Bases: DictElement

StructureElement specific for HDF5 groups

class caoscrawler.converters.hdf5_converter.H5DatasetElement(name: str, value: Dataset)

Bases: DictElement

StructureElement specific for HDF5 datasets.

class caoscrawler.converters.hdf5_converter.H5NdarrayElement(name: str, value, internal_path: str)

Bases: DictElement

StructureElement specific for NDArrays within HDF5 files.

Also store the internal path of the array within the HDF5 file in its internal_path attribute.

class caoscrawler.converters.hdf5_converter.H5FileConverter(definition: dict, name: str, converter_registry: dict)

Bases: SimpleFileConverter

Converter for HDF5 files that creates children for the contained attributes, groups, and datasets.

create_children(generalStore: GeneralStore, element: StructureElement)

Create children from root-level file attributes and contained hdf5 elements.

class caoscrawler.converters.hdf5_converter.H5GroupConverter(definition: dict, name: str, converter_registry: dict)

Bases: DictElementConverter

Converter for HDF5 groups that creates children from the group-level attributes and the contained subgroups and datasets.

typecheck(element: StructureElement)

Check whether the current structure element can be converted using this converter.

create_children(generalStore: GeneralStore, element: StructureElement)

Create children from group attributes and hdf5 elements contained in this group.

class caoscrawler.converters.hdf5_converter.H5DatasetConverter(definition: dict, name: str, converter_registry: dict)

Bases: DictElementConverter

Converter for HDF5 datasets that creates children from the dataset attributes and the contained array data.

typecheck(element: StructureElement)

Check whether the current structure element can be converted using this converter.

create_children(generalStore: GeneralStore, element: StructureElement)

Create children from the dataset attributes and append the array data contained in this dataset.

class caoscrawler.converters.hdf5_converter.H5NdarrayConverter(definition: dict, name: str, converter_registry: dict)

Bases: Converter

Converter for ndarrays contained in HDF5 files. Creates the wrapper record for this ndarray.

create_children(values: GeneralStore, element: StructureElement)

The ndarray doesn’t have any further children.

create_records(values: GeneralStore, records: RecordStore, element: StructureElement)

Create a wrapper record with name recordname, type array_recordtype_name (default H5Ndarray) and the internal path stored in a property with name internal_path_property_name (default internal_hdf5_path).

typecheck(element: StructureElement)

Check whether the current structure element can be converted using this converter.

match(element: StructureElement)

This method is used to implement detailed checks for matching compatibility of the current structure element with this converter.

The return value is a dictionary providing possible matched variables from the structure elements information.