# Backup LinkAhead Data Create a backup of LinkAhead via ```sh linkahead -p path/to/profile.yml backup -d path/to/backup/directory ``` in order to save the SQL dump of your data (the `*.dump.sql` file) and a backup of LinkAhead's internal file system (the `*.tar.gz`). ## Regular Backups Of course, it is recommended to do regular backups. One way to do this is to use cronjob. This can be nicely combined with [rsnapshot](https://github.com/rsnapshot/rsnapshot) to prevent the backups from growing too much. With rsnapshot, rotating backups can be created that are kept for predefined intervals. To combine this with LinkAhead backups, do the following. 1. Create a specific LinkAhead backup script that will be executed by rsnapshot and that stores the backups in its current working directory. For example, create a `/usr/local/bin/backup_linkahead_rsnapshot.sh` which contains the following lines (may have to be adapted to your LinkAhead installation, see above). ```sh #!/bin/sh linkahead backup -d ./ ``` 2. Adjust the rsnapshot configuration in `/etc/rsnapshot.conf` (see [its documentation](https://github.com/rsnapshot/rsnapshot#configuration)): * Create reasonable intervals that correspond to the frequency with which backups are rotated, e.g., `daily`, `weekly`, and `monthly` of which 14, 8, and 12 copies are kept, respectively. I.e., you'll be keeping daily backups for two weeks, weekly for the following two months, and monthly for a year. * Choose a meaningful `snapshot_root` in which the backups will be stored. * Add the script created above to the backup scripts in `/etc/rsnapshot.conf`. In the above example, the line looks like ```sh backup_script /usr/local/bin/backup_linkahead_rsnapshot.sh linkahead/ ``` * Possibly remove standard backup directories like `/etc/` and `/home/` from the `/etc/rsnapshot.conf` 3. Test the config with `rsnapshot configtest` for correctness. 4. Make a test run. For example `rsnapshot -t daily` or `rsnapshot daily`. The former performs a dry run, the latter actually creates a backup. 5. Add the rsnapshot commands to execute the backup intervals defined above to your crontab. In the above example, `daily` could be executed each day at 2 am, `weekly` on every Sunday at 1:30, and `monthly` on each first day of the month at 1 am with the following lines. ``` 0 2 * * * /usr/bin/rsnapshot daily 30 1 * * 1 /usr/bin/rsnapshot weekly 0 1 1 * * /usr/bin/rsnapshot monthly ``` 6. Check (after some time) logs in `/var/log/syslog` (default log location) # Manage the SQL database manually ## How to export the SQL database folder manually (e.g. for backup) # If it should ever be necessary to export the SQL database folder manually (for example for low-level backup, development or debugging), use these hints. The idea is: Mount the Docker volume with the database into a temporary container, then export it from there. ``` docker run --rm --mount "src=$(pwd),dst=/sql_export,type=bind" \ --mount "src=compose_caosdb-sqldata,dst=/sql,type=volume,ro" debian bash \ -c "tar -czf /sql_export/sql_data.tar.gz /sql/" ``` The data will be now inside `sql_data.tar.gz`. ## How to migrate the SQL database from a bind mount directory to a Docker volume # **Attention:** Be aware that you can delete Docker volumes with the Docker client (e.g. `docker volume prune` or `docker system prune --volumes`). Thus be careful not to lose data when you use these commands. In order to migrate an SQL database from a directory that was mounted via bind mounts to a Docker volume, you simply need to populate the right volume with the existing data. In the following let us assume that the default project name is used (otherwise replace `compose` with the project name). - Note that if a volume named `compose_caosdb-sqldata` existed before, you should remove it first (`docker volume rm compose_caosdb-sqldata`). - Go to the parent folder of the sql data directory (typically `somepath/profile/paths`) and start a container which mounts the old sql data folder and a new volume with the correct name: ``` docker run --rm --mount "src=$(pwd)/sql_data,dst=/sql_data,type=bind,ro" \ --mount "src=compose_caosdb-sqldata,dst=/new,type=volume" debian bash \ -c "cp -a /sql_data/* /new" ``` You can now start LinkAhead using Docker volumes. Check that the migration from a plain directory to a Docker volume succeeded, then you can remove the old directory. # How to migrate from LinkAhead to a plain CaosDB installation This section describes how to migrate from LinkAhead, the CaosDB distribution based on Docker, to a plain CaosDB installation, compiled from the sources. **Note:** Even without a LinkAhead subscription, running LinkAhead installations will still work, but are not guaranteed to be compatible with future versions of other CaosDB components. In order to use your custom extensions, your LinkAhead configuration, and, most importantly, the contents of your database with a plain CaosDB installation, the following steps should to be taken. Note that you should be familiar with the configuration in your `profile.yml` in your profile directory. Also, some basic knowledge of SQL, MySQL and MariaDB may be helpful. ## Backup Create a backup of the data as described above. Both, the tar ball and the SQL dump, will be restored in the new CaosDB installation later on. Shutdown LinkAhead (`systemctl stop linkahead` or `linkahead -p path/to/profile.yml` depending on your LinkAhead installation) afterwards for the setup of the new CaosDB installation. ## Install CaosDB server and the WebUI Get and install the CaosDB server and the web interface by following the installation instructions for the [server](https://docs.indiscale.com/caosdb-server/README_SETUP.html), the [MySQL backend](https://docs.indiscale.com/caosdb-mysqlbackend/README_SETUP.html), and the [web interface](https://docs.indiscale.com/caosdb-webui/getting_started.html). If you declared specific commits, branches or tags for CaosDB components in your `profile.yml`, you may want to use these for the manual installations as well to ensure the compatibility of your custom extensions to the new CaosDB installation. You may also want to reuse the SSL certificates of your LinkAhead installation. You'll usually find them in the `custom/other/cert` directory. See [below](#adapt-the-configuration) on how to use the configuration files from your LinkAhead installation for the configuration of the CaosDB server, the MySQL backend, and the web interface. ## Restore the backups Restoring the backups from a LinkAhead installation is a two-step process: 1. Restore the SQL dump of the database. 2. Transfer the internal file system from the LinkAhead installation to the new CaosDB installation. ### Restoring the SQL dump Use the script provided in the MySQL backend repository as described in the [documentation](https://docs.indiscale.com/caosdb-mysqlbackend/Maintenance.html#restoring-a-backup) to restore the SQL dump that you created earlier. ### Restoring the CaosDB file system Untar the file-system backup created above into the directory specified as `FILE_SYSTEM_ROOT` in your `server.conf` according to [CaosDB server's maintenance documentation](https://docs.indiscale.com/caosdb-server/administration/maintenance.html#restoring-a-backup). ## Adapt the configuration **Note:** This manual covers only the case of managing user accounts directly in CaosDB. See the [server documentation](https://docs.indiscale.com/caosdb-server/README_SETUP.html#pam) if you use an external authentication service like PAM. ### Configuring the MySQL backend To set up the MySQL backend, configure it according to the [documentation](https://docs.indiscale.com/caosdb-mysqlbackend/README_SETUP.html#create-the-configuration) and make sure that the database name and the user match those of your LinkAhead installation. Usually the defaults, i.e. database `caosdb` and user `caosdb` should do fine. For the tests to run through, you may have to create a `'caosdb'@'%'` user in your MySQL service. ### Configuring the server Copy the server configuration files from your LinkAhead installation (usually in `custom/caosdb-server/`) to your CaosDB server directory instead of creating them during the corresponding steps of the [server installation](https://docs.indiscale.com/caosdb-server/README_SETUP.html#first-setup). Transfer the contents of all customizations from your LinkAhead server configuration (usually the contents of the files in `custom/other/caosdb-server/conf/ext/server.conf.d/`) to the `/conf/ext/server.conf` file that is created during the installation of the server. Mind that some server configurations (e.g. whether there is an anonymous user, the network settings and much more) are contained in your `profile.yml`. Also, don't forget to enter the passwords for your SSL certificates in the `/conf/ext/server.conf` file. If you use server-side scripts in your LinkAhead installation, you should copy them (usually from `custom/caosdb-server/scripting/bin`) to a directory specified in the `SERVER_SIDE_SCRIPTING_BIN_DIRS` variable in your `server.conf`. ### Configuring the web interface Again, copy your configuration files from your LinkAhead installation (usually in `custom/caosdb-server/caosdb-webui/conf/ext`) to the corresponding locations in `conf/ext` of your new WebUI installation. Do the same for the custom build properties (usually in `custom/caosdb-server/caosdb-webui/build.properties.d`). Remember to also copy the settings from the `theme-linkahead` module (usually located in `theme-linkahead`) in the same way. If you have custom extensions for your web interface, copy them (usually from the contents of `custom/caosdb-server/caosdb-webui/src/ext`) to the `src/ext` directory of your new WebUI installation. Remember to do the same for the `theme-linkahead` module if you want to. (Re-)Compile the web interface after these steps. ## Testing the new setup After installation and configuration of the server, backend, and web interface, start the server by executing `make run` from the `caosdb-server` directory. Open the web interface with a browser of your choice at the address and port you specified (e.g. [https://localhost:10443](https://localhost:10443)) and check whether all extensions work as expected. Also try and connect to the server via e.g. the Python client.