caosadvancedtools.models package
Submodules
caosadvancedtools.models.data_model module
- class caosadvancedtools.models.data_model.DataModel(*args)
Bases:
dictProvides tools for managing a data model.
When constructing a data model the CaosDB representation can easily be created using the classes RecordType and Propery, storing them in a Container and inserting it in CaoSDB. However, this has one drawback: You cannot simply change someting and update the container. The container will insist on having valid ids for all contained Entities.
This class allows you to define your model as easily but also provides you with a method (
sync_data_model) that will sync with the data model in an existing CaosDB instance.This is possible because entities, defined in this model, are identified with entities in CaosDB using names. I.e. a RecordType “Experiment” in this model will update an existing RecordType with name “Experiment” in CaosDB. Thus, be carefull not to change existing Entities that were created for a different purpose (e.g. someone else’s experiment).
DataModel inherits from dict. The keys are always the names of the entities. Thus you cannot have unnamed or ambiguously named entities in your model.
Example:
# Create a DataModel with a RecordType and a Property, not assuming any # relation between the two. dm = DataModel([db.RecordType(name=”myRecordType”),
db.Property(name=”myProperty”)])
# Sync the DataModel with the server, so that the server state is consistent # with this DataModel’s content. dm.sync_data_model() # Now the DataModel’s IDs are the same as on the server.
- collect_entities()
Collects all entities: explicitly defined RecordTypes and Properties and those mentioned as Properties
- static entities_without(entities, names)
Return a new list with all entities which do not have certain names.
- get_deep(name: str, visited_props: dict | None = None, visited_parents: set | None = None)
Attempt to resolve references for the given
name.The returned entity has all the properties it inherits from its ancestry and all properties have the correct descriptions and datatypes. This methods only uses data which is available in this DataModel, which acts kind of like a cache pool.
Note that this may change this data model (subsequent “get” like calls may also return deeper content.)
- static get_existing_entities(entities)
Return a list with those entities of the supplied iterable that exist in the CaosDB instance.
- Parameters:
entities (iterable) – The entities to be retrieved. This object will not be moidified.
- Raises:
TransactionError – If the retrieval fails.
- sync_data_model(noquestion: bool = False, verbose: bool = True)
Synchronize this DataModel with a CaosDB instance.
Updates existing entities from the CaosDB instance and inserts non-existing entities into the instance. Note: This allows to easily overwrite changes that were made to an existing data model. Use this function with care and double check its effect.
- Raises:
TransactionError – If one of the involved transactions fails.
- sync_ids_by_name(valid_entities)
Add IDs from valid_entities to the entities in this DataModel.
“By name” means that the valid IDs (from the valid_entities) are assigned to the entities, their properties in this DataModel by their names, also parents are replaced by equally named entities in valid_entities. These changes happen in place to this DataModel!
- Parameters:
valid_entities (list of Entity) – A list (e.g. a Container) of valid entities.
- Return type:
None
caosadvancedtools.models.parser module
This module (and script) provides methods to read a DataModel from a YAML file.
If a file name is passed to parse_model_from_yaml it is parsed and a DataModel is created. The yaml file needs to be structured in a certain way which will be described in the following.
The file should only contain a dictionary. The keys are the names of RecordTypes or Properties. The values are again dictionaries describing the entities. This information can be defined via the keys listed in KEYWORDS. Notably, properties can be given in a dictionary under the xxxx_properties keys and will be added with the respective importance. These properties can be RecordTypes or Properties and can be defined right there. Every Property or RecordType only needs to be defined once anywhere. When it is not defined, simply the name can be supplied with no value. Parents can be provided under the ‘inherit_from_xxxx’ keywords. The value needs to be a list with the names. Here, NO NEW entities can be defined.
- exception caosadvancedtools.models.parser.JsonSchemaDefinitionError(msg)
Bases:
RuntimeError
- class caosadvancedtools.models.parser.JsonSchemaParser(types_for_missing_array_items={}, ignore_unspecified_array_items=False)
Bases:
ParserExtends the yaml parser to read in datamodels defined in a json schema.
EXPERIMENTAL: While this class can already be used to create data models from basic json schemas, there are the following limitations and missing features:
Due to limitations of json-schema itself, we currently do not support inheritance in the imported data models
The same goes for suggested properties of RecordTypes
Already defined RecordTypes and (scalar) Properties can’t be re-used as list properties
Reference properties that are different from the referenced RT. (Although this is possible for list of references)
Values
Roles
The extern keyword from the yaml parser
- class caosadvancedtools.models.parser.Parser(debug: bool = False)
Bases:
object- parse_model_from_string(string, existing_model: dict | None = None)
Create and return a data model from the given YAML string.
- Parameters:
- Returns:
out – The created DataModel
- Return type:
- class caosadvancedtools.models.parser.SafeLineLoader(stream)
Bases:
SafeLoaderLoad a line and keep meta-information.
Note that this will add a
__line__element to all the dicts.- construct_mapping(node, deep=False)
Overwritung the parent method.
- exception caosadvancedtools.models.parser.YamlDefinitionError(line, template=None)
Bases:
RuntimeError
- caosadvancedtools.models.parser.parse_model_from_json_schema(filename: str, top_level_recordtype: bool = True, types_for_missing_array_items: dict = {}, ignore_unspecified_array_items: bool = False, existing_model: dict | None = None)
Return a datamodel parsed from a json schema definition.
- Parameters:
filename (str) – The path of the json schema file that is to be parsed
top_level_recordtype (bool, optional) – Whether there is a record type defined at the top level of the schema. Default is true.
types_for_missing_array_items (dict, optional) – dictionary containing fall-back types for json entries with
type: arraybut withoutitemsspecification. Default is an empty dict.ignore_unspecified_array_items (bool, optional) – Whether to ignore
type: arrayentries the type of which is not specified by theiritemsproperty or given intypes_for_missing_array_items. An error is raised if they are not ignored. Default is False.existing_model (dict, optional) – An existing model to which the created model shall be added. Not implemented yet.
- Returns:
out – The datamodel generated from the input schema which then can be used for synchronizing with CaosDB.
- Return type:
Datamodel
Note
This is an experimental feature, see
JsonSchemaParserfor information about the limitations of the current implementation.
- caosadvancedtools.models.parser.parse_model_from_string(string, existing_model: dict | None = None, debug: bool = False)
Parse a data model from a YAML string
This is a convenience function if the Parser object is not needed, it calls
Parser.parse_model_from_string(...)internally.
Module contents
Submodule for working with data models.