Changelog

All notable changes to this project will be documented in this file.

The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.

[0.13.0] - 2024-10-24

Changed

  • Update LinkAhead MariaDB Backend to version 8.0.0 to make LinkAhead compatible to MariaDB >= 10.6. Old SQL dumps may be incompatible to MariaDB 10.6 and above; see update_dumps/README.md in the LinkAhead MariaDB Backend repo for more information and a script to update incompatible dumps.

[0.12.3] - 2024-09-25

Fixed

  • Workaround in make jar for Debian bug https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1039607

[0.12.2] - 2024-03-18

Fixed

  • Unknown error when trying to access the user list. linkahead-server#250

  • Legacy integer ids are incremented by 1 again, not by 2.

[0.12.1] - 2023-12-13

(Timm Fitschen)

Fixed

  • Insufficient permission checks during subproperty filters of SELECT queries when an entity with retrieve permissions references one without linkahead-server#244

  • Insufficient permission checks in queries when a name of an invisible record is used in a filter where a visible record references the invisible one linkahead-server#242

Security

This is an important security patch release. The bugs linkahead-server#244 and linkahead-server#242 possibly leak sensitive data when an attacker with read access to linkahead (i.e. the attacker needs an active user account or anonymous needs to be enabled) can guess the name of entities or properties of referenced entities and construct a malicious FIND or SELECT statement and when the attacker has read permissions for an entity which references the entities containing the sensitive information. See the bug reports for more information.

[0.12.0] - 2023-10-25

(Timm Fitschen)

Fixed

[0.11.0] 2023-10-13

Added

  • Configuration options REST_RESPONSE_LOG_FORMAT and GRPC_RESPONSE_LOG_FORMAT which control the format and information included in the log message of any response of the respective API. See conf/core/server.conf for more information.

  • REST API: Permanent redirect from “FileSystem” to “FileSystem/”.

Fixed

  • Inheritance of the unit is not working. (GRPC API) linkahead-server#264

  • Curly brackets in query lead to unexpected server error. linkahead-server#138

  • Wrong url returned by FileSystem resource behind proxy.

  • NullPointerException in GRPC API converters when executing SELECT query on NULL values.

  • Fix parsing of decimal numbers. Fixes https://gitlab.com/linkahead/linkahead-server/-/issues/239

[0.10.0] - 2023-06-02

(Florian Spreckelsen)

Changed

  • The default behavior of the query FIND SomeName [...] (as well as COUNT and SELECT) is being made configurable and changes:

    • FIND SomeName will be interpreted as FIND <FIND_QUERY_DEFAULT_ROLE> SomeName from now on where FIND_QUERY_DEFAULT_ROLE is a newly introduced server property.

    • The new FIND_QUERY_DEFAULT_ROLE server property defaults to RECORD which is why the behavior of the server api has a breaking change.

    • The semantics of FIND * are affected as well. FIND * is equivalent to FIND <FIND_QUERY_DEFAULT_ROLE>.

    • Of course, administrators can choose to retain the old behavior by setting FIND_QUERY_DEFAULT_ROLE=ENTITY.

  • CQL now treats WITH and WITH A equivalently. Issue: #192

  • The InsertFilesInDir FlagJob now creates File entities without a name. The previous behavior caused severe performance problems for very large numbers of files. Issue: #197

Fixed

  • Unexpected Server Error when inserting an Entity. #216

  • Bad performance due to the execution of unnecessary jobs during retrieval. #189

  • Query Language: Parentheses change filter to subproperty filter #203

  • Searching for values in scientific notation #143

  • Denying a role permission has no effect #196. See security notes below.

  • Missing RecordType leads to unexpected server error #166

Security

  • Fixed #196. This was an error in the authorization procedure which allowed unprivileged users execute insert, update or delete transactions on entities. However, the unprivileged users would also need the correct entity permissions to do that.

    Without backup, this means possible data loss. Also there was the possibility to spam the database by creating unwanted entities.

Documentation

  • Nested queries.

  • Global entity permissions.

  • DOC: Data model tutorial.

  • Removed old documentation directory /doc/, migrated non-duplicate content to /src/doc/.

[0.9.0] - 2023-01-19

Added

  • SELECT query support for the GRPC API

[0.8.1] - 2022-11-07

(Timm Fitschen)

Added

  • Make time zone configurable via server properties during runtime, but only in debug mode.

Changed

Deprecated

Removed

Fixed

Security

Documentation

  • Extended documentation on role and entity permissions

[0.8.0] - 2022-07-12

(Timm Fitschen)

Added

  • Configurable requirements for user names and passwords. The default is the old hard-coded configuration.

Changed

  • Minimal changes to the error messages for invalid user names and passwords.

Deprecated

Removed

Fixed

  • caosdb-server#130 CQL: Spaces are not recognised in sub-property queries.

  • caosdb-server#148 Cannot delete file entities containing unescaped space characters

  • caosdb-server#145 Searching for large numbers results in wrong results if integer values are used.

Security

[0.7.3] - 2022-05-03

(Daniel Hornung)

Changed

  • misc/pam_authentication/ldap.conf is not used for configuring the ldap_authentication.sh script anymore. Use misc/pam_authentication/ldap.env instead and view the documentation inside the file itself for more information.

Fixed

  • caosdb-server#142 Can’t create users with dots in their user names

  • ldap_authentication.sh <username> failed on every attempt when used in combination with OpenLDAP with default configuration.

  • ldap_authentication.sh allowed empty and even wrong passwords when used in combination with MS Active Directory when AD is configured to allow binding with an empty password.

Security

  • ldap_authentication.sh allowed empty and even wrong passwords when used in combination with MS Active Directory when AD is configured to allow binding with an empty password. This is only relevant for non-default configurations of the PAM.pam_script option in the usersources.ini.

[0.7.2] - 2022-03-25

(Timm Fitschen)

This is an important security update.

Added

  • Implementation for the ACM GRPC-API (caosdb-proto 0.2)

  • Implementation for the EntityACL GRPC-API (caosdb-proto 0.2)

Changed

Deprecated

Removed

Fixed

  • Wrong serialization of date time values in the GRPC-API (resulting in org.caosdb.server.datatime@12347abcd or similar).

  • Missing serialization of file descriptors in the GRPC-API during retrievals.

  • caosdb-server#131 Query: AND does not work with sub-properties

  • Add previously missing Value.equals functions

  • caosdb-server#132 Query: subproperties should not require parentheses

  • caosdb-server#174 GRPC-API: Server should gracefully handle non-file entities with file-like attributes.

  • caosdb-server#217 Server gets list property datatype wrong if description is updated.

  • caosdb-server#220 Entities can be retrieved via GRPC despite insufficient permissions.

  • caosdb-server#221 Unknown error during update of property leaving out datatype.

  • caosdb-server#223 State is being leaked even though RETRIEVE:ENTITY permission is not granted.

Security

  • Update of logging backend log4j to 2.17.2

  • Fix for caosdb-server#220 Entities can be retrieved via GRPC despite insufficient permissions.

  • Fix for caosdb-server#223 State is being leaked even though RETRIEVE:ENTITY permission is not granted.

[v0.7.1] - 2021-12-13

(Timm Fitschen)

This is an important security update.

Added

Changed

Deprecated

Removed

Fixed

Security

  • Update of logging backend log4j after a critical security vulnerability CVE-2021-44228 to v2.15.0.

  • caosdb-deploy#225

    • Denied Edit permission leads to retrieve permission.

[v0.6.1] - 2021-11-13 [YANKED]

(Timm Fitschen)

This version’s release was pulled after some problems during the release process. It is identical to v0.7.1

[v0.6.0] - 2021-11-17

(Timm Fitschen)

Added

  • Endpoint for CaosDB GRPC API 0.1 (see https://gitlab.com/caosdb-proto.git for more). Authentication is supported via a Basic scheme, using the well-known “authentication” header. Notable limitations of the current implementation of the API:

    • It is currently not possible to mix retrievals (caosdb.entity.v1.RetrieveRequest) with any other transaction type - so transaction are either read-only or write-only. The server throws an error if it finds mixed read/write transactions.

    • It is currently not possible to have more that one query (caosdb.entity.v1.Query) in a single transaction. The server throws an error if it finds more than one query.

Changed

Deprecated

  • Legacy XML/HTTP API (also known as the REST API). The API will not be removed until the web interface (caosdb-webui) and the python client libraries have been updated and freed from any dependencies. However, new clients should not implement this API anymore.

Removed

Fixed

Security

[v0.5.0] - 2021-10-19

Added

  • An openAPI specification of the XML api

  • New server configuration option SERVER_BIND_ADDRESS, which is the address to listen to. See server.conf.

Changed

Deprecated

Removed

  • IdOnly flag (see https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/187). The flags was not working anyways. However, SELECT id FROM ... queries are now optimized in the way the IdOnly flag was supposed to do.

Fixed

  • #181 CQL’s UPDATED filter. (https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/181)

  • #183 No reasonable error when using bad datetime format. (https://gitlab.indiscale.com/caosdb/src/caosdb-server/-/issues/183)

  • #127 “nan” as value (list item) in properties with data type “LIST” return with “Cannot parse value to double” error.

  • #170 Updating an abstract list-type property with a default value fails with “unkown error”.

  • #145 Documentation of importances and inheritance

  • Missing sources of the easy-unit dependency.

  • #178 Formatting of tables in documentation

Security

[v0.4.0] - 2021-06-21

Added

  • Related to #146, a new flag for entities and complete transactions: force-missing-obligatory=[ignore|warn|error]. The flag overrides the default behavior of the server (throwing an error when an obligatory property is missing). ignore just discards the consistency check, warn only issues a warning when obligatory properties are missing and error throws an error in that case. The flag can be set for the complete transaction and each single entity, while the entity flag takes precedence.

  • New EntityState plug-in. The plug-in disabled by default and can be enabled by setting the server property EXT_ENTITY_STATE=ENABLED. See !62 for more information.

  • ETag property for the query. The ETag is assigned to the query cache each time the cache is cleared (currently whenever the server state is being updated, i.e. the stored entities change). This can be used to debug the query cache and also allows a client to determine whether the server’s state has changed between queries.

  • Basic caching for queries. The caching is enabled by default and can be controlled by the usual “cache” flag.

  • Documentation for the overall server structure.

  • Add BEFORE, AFTER, UNTIL, SINCE keywords for query transaction

Changed

  • The default session timeout changed from 10 min to 60 min. Please set it to your needs via the server config option SESSION_TIMEOUT_MS.

Deprecated

Removed

Fixed

  • #146 - Default behavior for missing obligatory properties

  • #131 - CQL Parsing error when white space characters before some units.

  • #134 - CQL Parsing error when multiple white space characters after FROM.

  • #130 - Error during FIND ENTITY when QUERY_FILTER_ENTITIES_WITHOUT_RETRIEVE_PERMISSIONS=False.

  • #125 - bend_symlinks script did not allow whitespace in filename.

  • #122 - Dead-lock due to error in the DatabaseAccessManager.

  • #120 - Editing entities that were created with a no longer existing user leads to a server error.

  • #31 - Queries with keywords in the path (e.g. ... STORED AT 0in.txt)

  • #116 - Queries FIND [ANY VERSION OF] * and FIND [ANY VERSION OF] ENTITY.

Security

[0.3.0] - 2021-02-10

Added

  • New version history feature. The “H” container flag retrieves the full version history during a transaction (e.g. during Retrievals) and constructs a tree of successors and predecessors of the requested entity version.

  • New query functionality: ANY VERSION OF modifier. E.g. FIND ANY VERSION OF RECORD WITH pname=val returns all current and old versions of records where pname=val. For further information, examples and limitations see the wiki page on CQL

  • New server property SERVER_SIDE_SCRIPTING_BIN_DIRS which accepts a comma or space separated list as values. The server looks for scripts in all directories in the order or the list and uses the first matching file.

  • Automated documentation builds: make doc

Changed

  • Server can be started without TLS even when not in debug mode.

  • Select queries would originally only select the returned properties by their names and would not check if a property is a subtype of a selected property. This has changed now and select queries will also return subtypes of selected properties.

Deprecated

  • SERVER_SIDE_SCRIPTING_BIN_DIR property is deprecated. SERVER_SIDE_SCRIPTING_BIN_DIRS should be used instead (note the plural form!)

Removed

  • Text user interface (CaosDBTerminal).

Fixed

  • Bug: When the user password is updated the user is deactivated.

  • Semi-fixed a bug which occurs when retrieving old versions of entities which reference entities which have been deleted in the mean time. The current fix adds a warning message to the reference property in question and sets the value to NULL. This might even be desired behavior, however this would have to finally specified during the Delete/Forget phase of the implementation of the versioning.

  • Inheritance job cannot handle inheritance from same container (!54)

  • Bug in the query parser (MR!56) - The parser would throw an error when the query contains a conjunction or disjunction filter with a first element which is another disjunction or conjunction and being wrapped into parenthesis.

Security

[0.2.0] - 2020-09-02

Added

  • Support for deeply nested selectors in SELECT queries.

  • One-time Authentication Tokens for login without credentials and login with particular permissions and roles for the course of the session.

  • Entity/names resource for retrieving all known entity names.

  • Scripting is simplified by adding a home directory, of which a copy is created for each called script and set as the HOME environment variable.

  • bend_symlinks.sh (version 0.1, experimental) fix broken symlinks in the internal file system. See README.md

  • move_files.py (version 0.1, experimental) Script for moving files (change their path) in the internal file system based on a two-column tsv file (with columns “from” and “to”). See README.md.

  • LDAP server may now be given and may be different from LDAP domain. See misc/pam_authentication/ldap.conf

  • #47 - Sub-properties can now be queried, such as in SELECT window.width FROM house.

  • Added support for versioning, if it is enabled on the backend.

Changed

  • All caosdb server java classes moved from caosdb.[...] to org.caosdb.[...] because the new root package is compliant with the java package naming conventions while the old was not. This has some implications for configuring the server. See README_SETUP.md, section “Migration” for additional information.

  • The server by default now only serves TLS 1.2 and 1.3, all previous versions have been disabled in the default settings. Make sure that your clients (especially the Python client) are up to date.

Deprecated

  • CaosDBTerminal

Removed

Fixed

  • Missing handling of list of reference properties in SELECT queries.

  • #51 - name queries (e.g. FIND ENTITY WITH name = ...)

  • #27 - star matches slashes (e.g. for FIND ... STORED AT /*.dat).

  • #30 - file path cannot be in quotes

  • #46 - Server-side scripting failed as an unprivileged user because there was no writable home directory.

  • NaN Double Values (see #41)

  • #14 - Handle files on file system without File entity: Those entries are returned without ID but with a notice now.

  • #11 - pam_authentication leaks the password to unprivileged processes on the same machine.

  • #39 - quotes around datetimes in queries

  • #99 - Checksum updating resulted in infinite loop on server.

Security (in case of vulnerabilities)

  • TLS is by default restricted to v1.2 and v1.3 now.

  • #11 - PAM and LDAP authentication no longer leak the password to unprivileged processes on the same machine.

  • #68 - Shadow sensitive information when logging for debugging purposes.

[0.1.0] - 2018-10-09

Tag v0.1 - Commit 3b17b49

Added

  • everything