Transform Functions

Often the situation arises, that you cannot use a value as it is found. Maybe a value should be increased by an offset or a string should be split into a list of pieces. In order to allow such simple conversions, transform functions can be named in the converter definition that are then applied to the respective variables when the converter is executed.

<NodeName>:
    type: <ConverterName>
    match: ".*"
    transform:
      <TransformNodeName>:
        in: $<in_var_name>
        out: $<out_var_name>
        functions:
        - <func_name>:                         # name of the function to be applied
            <func_arg1>: <func_arg1_value>     # key value pairs that are passed as parameters
            <func_arg2>: <func_arg2_value>
            # ...

An example that splits the variable a and puts the generated list in b is the following:

Experiment:
    type: Dict
    match: ".*"
    transform:
      param_split:
        in: $a
        out: $b
        functions:
        - split:            # split is a function that is defined by default
            marker: "|"     # its only parameter is the marker that is used to split the string
    records:
      Report:
        tags: $b

This splits the string in ‘$a’ and stores the resulting list in ‘$b’. This is here used to add a list valued property to the Report Record.

There are a number of transform functions that are defined by default (see src/caoscrawler/default_transformers.yml). You can define custom transform functions by adding them to the cfood definition (see CFood Documentation).