Getting Started with the CaosDB Server
Here, you find information on requirements, the installation, configuration and more. Note, that building the CaosDB Server from the sources and maintaining the server requires the knowledge/skill of an experienced Linux administrator.
Note: If you are looking for a quick and easy way to get startet, check out the docker deployment.
Install the requirements on Debian
On Debian, the required packages can be installed with:
apt-get install make mariadb-server maven openjdk-11-jdk-headless \ python3-pip libpam0g-dev unzip
Note that installing MariaDB will uninstall existing MySQL packages and vice versa.
Install the requirements on Fedora
On Fedora, the required packages can be installed with:
sudo dnf install make pam-devel mariadb-server mariadb python3 java-17-openjdk-headless unzip gcc
x86_64, e.g. Ubuntu 18.04
Mounted filesytem(s) with enough space
Working internet connection (for up-to-date python and java libraries)
If the WebUI shall run, check out the respective submodule:
git submodule update --init caosdb-webui
Then configure and compile it according to its documentation.
Authentication via PAM is possible, for this the PAM development library must be installed and the pam user tool must be compiled:
If you want, you can run a test now:
./pam_authentication.sh asdfasks for a password for user
asdf. If no such user exists or the wrong passowrd is entered, it print
[FAILED]and return with a non-zero exit code.
If you want to run the CaosDB server without root privilege, you need to use the setuid bit for the binary. For example, if the user
caosdbruns the server process the permissions of
bin/pam_authenticationshould be the following:
-rwsr-x--- 1 root caosdb pam_authentication
make fails with
pam_authentication.c:4:31: fatal error: security/pam_appl.h: No such file or directory the header files are probably
not installed. You can do so under Debian and Ubuntu with
apt-get install libpam0g-dev. Then try again.
After a fresh clone of the repository, this is what you need to setup the server:
protosubmodule (and submodules for those extensions you want, see above):
git submodule update --init caosdb-proto
Compile the server with
make compile. This may take a while and there needs to be an internet connection as packages are downloaded to be integrated in the java file.
It is recommended to run the unit tests with
make test. It may take a while.
Create an SSL certificate somewhere with a
Java Key Storefile. For self-signed certificates (not recommended for production use) you can do:
mkdir certificates; cd certificates
keytool -genkey -keyalg RSA -alias selfsigned -keystore caosdb.jks -validity 375 -keysize 2048 -ext san=dns:localhostReplace
localhostby your host name, if you want.
keytool -importkeystore -srckeystore caosdb.jks -destkeystore caosdb.p12 -deststoretype PKCS12 -srcalias selfsigned
Export the public part only:
openssl pkcs12 -in caosdb.p12 -nokeys -out cert.pem. The resulting
cert.pemcan safely be given to users to allow ssl verification.
You can check the content of the certificate with
openssl x509 -in cert.pem -text
Alternatively, you can create a keystore from certificate files that you already have:
openssl pkcs12 -export -inkey privkey.pem -in fullchain.pem -out all-certs.pkcs12
keytool -importkeystore -srckeystore all-certs.pkcs12 -srcstoretype PKCS12 -deststoretype pkcs12 -destkeystore caosdb.jks
Install/configure the MySQL back-end: see the
Create an authtoken config (e.g. copy
conf/ext/authtoken.ymland change it)
If any of the settings in
conf/core/server.confdo not fit your needs, create
conf/ext/server.confand add settings as required:
Setup for MySQL back-end: specify the fields
Choose the host and ports under which CaosDB will be accessible. The host is the IP address the server should listen on. Blank (the default) means all interfaces, and
127.0.0.1means only localhost.
Setup the SSL certificate: Assuming that there is an appropriate
Java Key Storefile (see above), change the fields
CERTIFICATES_KEY_STORE_PASSWORD. Make sure that the conf file is not readable by other users because the certificate passwords are stored in plaintext.
Set the path to the authtoken config (see step 4)
Set the file system paths:
FILE_SYSTEM_ROOT: The root for all the files managed by CaosDB.
DROP_OFF_BOX: Files can be put here for insertion into CaosDB.
TMP_FILES: Temporary files go here, for example during script execution or when uploading or moving files.
SHARED_FOLDER: Folder for sharing files via cryptographic tokens, also those created by scripts.
SERVER_SIDE_SCRIPTING_BIN_DIRS: A comma or white space separated list of directories (relative or absolute) where the server will be looking for executables which are then callable as server-side scripts. By default this list only contains
./scripting/bin. If you want to include e.g. scripts which are maintained as part of the caosdb-webui repository (because they are intended for usage by the webui), you should add
INSERT_FILES_IN_DIR_ALLOWED_DIRS: add mounted filesystems here that shall be accessible by CaosDB
Maybe set another
See also CONFIGURATION.rst
You can skip this if you do not want to use an external authentication. Local users (CaosDB realm) are always available.
Define the users/groups who you want to include/exclude.
Assign at least one user the
For example, if the admin user is called
caosdb, there should be the following lines:
include.user = caosdb user.caosdb.roles = administration
It is important that the file complies with the ini file specification. Especially that there are no
keys) without a
value. An emtpy value can be represented by
"". Comments are everything from
;to the end of the line.
Possibly install the PAM caller in
misc/pam_authentication/if you have not do so already. See above.
$ make run
This can take a while. Once you see
Starting org.caosdb.server.CaosDBServer application the server is ready and you can try it out by connecting with a
client, e.g. the web client, if you installed it. Typically, you just need to
https://localhost:10443 in your Browser, assuming you used 10443 as port.
Note, that you will get a security warning if you are using a self-signed
You can use
make run-single to directly start the server without rebuilding
sources (if you called
make run previously).
Run Unit Tests
$ make test
You can run single unit test with
mvn test -X -Dtest=TestCQL#testDecimalNumber
Open Eclipse (tested with 2022-R12)
File > Import > Maven > Existing Maven Projects: Specify location.
You will most likely encounter “Plugin execution not covered by lifecycle configuration: …” errors. Adapt the file
<?xml version="1.0" encoding="UTF-8"?> <lifecycleMappingMetadata> <pluginExecutions> <pluginExecution> <pluginExecutionFilter> <groupId>com.coveo</groupId> <artifactId>fmt-maven-plugin</artifactId> <versionRange>2.5.1</versionRange> <goals> <goal>format</goal> </goals> </pluginExecutionFilter> <action> <ignore /> </action> </pluginExecution> <pluginExecution> <pluginExecutionFilter> <groupId>org.codehaus.mojo</groupId> <artifactId>buildnumber-maven-plugin</artifactId> <versionRange>1.4</versionRange> <goals> <goal>create-metadata</goal> </goals> </pluginExecutionFilter> <action> <ignore /> </action> </pluginExecution> </pluginExecutions> </lifecycleMappingMetadata>
From 0.1 to 0.2
A major change in the code is the renaming of the java packages (from
This makes its necessary to change some of your config files as well. Whenever
you configured a class (e.g. the
EMAIL_HANDLER, or the realms in your
usersources.ini) you would need to reconfigure it there.
sed command could be useful. However, use it with care and backup
before you execute it.
sed -i.bak -e "s/\(\s*\)\([^.]\)caosdb\.server/\1\2org.caosdb.server/g" FILE_TO_BE_CHANGED
Build the documentation
Stand-alone documentation is built using Sphinx:
pip3 install --user javasphinx
Alternative, if javasphinx fails because python3-sphinx is too recent: (
git clone https://github.com/simgrid/javasphinx.git cd javasphinx git checkout 659209069603a pip3 install .
Installation under special conditions
Installation without or with restricted internet connection on the host system
It is of course possible to install CaosDB on servers which do not have an internet connection or only have limited access to the internet. However, maven, which is used to build the CaosDB server, typically accesses the internet very often, and some reconfiguration might be necessary to prevent timeout issues and similar headache.
A recommendation to proceed is:
Clone the repositories (caosdb-server, caosdb-mysqlbackend and subrepository caosdb-webui) on a second machine that has internet connection.
make runin the caosdb-server repository. This will download all required dependencies and setup all maven-related artifacts in a folder called
.m2typically located in the home directory.
In addition to copying over the repositories for the server, mysqlbackend and webui to the target host machine, also copy the complete
.m2folder to the home directory of the user that is running the CaosDB server.
On the host machine, open the file
pom.xmlin the server repository.
Look for the section
<repositories>. There should be a
<id>local-maven-repo</id>. It should be moved on top of the list of repositories, so that the xml block appears directly after
<repositories>and before the repository maven-central.
Troubleshooting / FAQ
I set up and run the server, but I get an error that XSLT parsing failed in the WebUI
You probably forgot to
make the webui.
Make sure that you:
Installed the WebUI according to the section [Web UI]
make installin the webui sub folder
How can I run the CaosDB server using ports 443 and 80?
On standard linux setups, ports 443 and 80 cannot be opened by non-root processes.
There are multiple solutions for this problem:
Use iptables to redirect a low port to a high port.
Here are some resources to read about these options:
Using authbind is a simple solution that is briefly described here:
Install the package authbind
Create (empty) files
.../443), e.g. using
Grant execution permissions for the user who runs the server to the new files.
Run the CaosDB server using authbind with the
authbind --deep make run. The
--deepoption is necessary because the server starts a subprocess which actually opens the ports.